2024-01-11T06:54:53.923 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-11T06:54:53.924 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --verbose --name yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --archive /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480 -- /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/orig.config.yaml 2024-01-11T06:54:53.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-11T06:54:54.056 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480 branch: wip-vshankar-testing1-quincy-2024-01-10-2010-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: yweinste@redhat.com first_in_suite: false job_id: '7512480' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing1-quincy-2024-01-10-2010-quincy ceph: cephfs: ec_profile: - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 4bcedb9fc11bed1fd6fd0c4fd59187401f309680 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4bcedb9fc11bed1fd6fd0c4fd59187401f309680 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing1-quincy-2024-01-10-2010-quincy sha1: 4bcedb9fc11bed1fd6fd0c4fd59187401f309680 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 419 sha1: 4bcedb9fc11bed1fd6fd0c4fd59187401f309680 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-vshankar-testing1-quincy-2024-01-10-2010-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_4bcedb9fc11bed1fd6fd0c4fd59187401f309680/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 4bcedb9fc11bed1fd6fd0c4fd59187401f309680 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGz211efhD5EafEB9aDd+x9Gp4uo/MSCiUXSAMaCF7LXcrpy+Q65Jc7jyCCWbGzGhr/661Jg0zmzjivmOuURb2A= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInZZZqUlAa+c4h9rdzlNKl38WHPrfAS2CrZ06y59npUesOmvftW9ucInWPMoRKsDGYZxqPe4Gc529D0iXBIUx8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-10_19:20:36 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-11T06:54:54.056 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_4bcedb9fc11bed1fd6fd0c4fd59187401f309680/qa; will attempt to use it 2024-01-11T06:54:54.057 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_4bcedb9fc11bed1fd6fd0c4fd59187401f309680/qa/tasks 2024-01-11T06:54:54.057 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-11T06:54:54.059 INFO:teuthology.task.internal:Checking packages... 2024-01-11T06:54:54.082 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '4bcedb9fc11bed1fd6fd0c4fd59187401f309680' 2024-01-11T06:54:54.083 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-11T06:54:54.083 INFO:teuthology.packaging:ref: None 2024-01-11T06:54:54.083 INFO:teuthology.packaging:tag: None 2024-01-11T06:54:54.083 INFO:teuthology.packaging:branch: wip-vshankar-testing1-quincy-2024-01-10-2010-quincy 2024-01-11T06:54:54.083 INFO:teuthology.packaging:sha1: 4bcedb9fc11bed1fd6fd0c4fd59187401f309680 2024-01-11T06:54:54.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing1-quincy-2024-01-10-2010-quincy 2024-01-11T06:54:54.223 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-299.g4bcedb9f 2024-01-11T06:54:54.225 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-11T06:54:54.232 INFO:teuthology.task.internal:no buildpackages task found 2024-01-11T06:54:54.232 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-11T06:54:54.240 INFO:teuthology.task.internal:Saving configuration 2024-01-11T06:54:54.257 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-11T06:54:54.268 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-11T06:54:54.290 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 06:49:09.807831', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGz211efhD5EafEB9aDd+x9Gp4uo/MSCiUXSAMaCF7LXcrpy+Q65Jc7jyCCWbGzGhr/661Jg0zmzjivmOuURb2A='} 2024-01-11T06:54:54.312 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 06:49:09.806431', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInZZZqUlAa+c4h9rdzlNKl38WHPrfAS2CrZ06y59npUesOmvftW9ucInWPMoRKsDGYZxqPe4Gc529D0iXBIUx8='} 2024-01-11T06:54:54.313 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-11T06:54:54.324 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-11T06:54:54.324 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-11T06:54:54.324 INFO:teuthology.run_tasks:Running task console_log... 2024-01-11T06:54:54.405 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1496e9e0d0>, signals=[15]) 2024-01-11T06:54:54.406 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-11T06:54:54.415 INFO:teuthology.task.internal:Opening connections... 2024-01-11T06:54:54.415 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T06:54:54.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T06:54:54.492 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T06:54:54.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T06:54:54.563 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-11T06:54:54.576 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-01-11T06:54:54.597 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-01-11T06:54:54.597 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-01-11T06:54:54.658 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="Red Hat Enterprise Linux" 2024-01-11T06:54:54.658 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8.6 (Ootpa)" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:ID="rhel" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="fedora" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8.6" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://www.redhat.com/" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-11T06:54:54.659 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-11T06:54:54.660 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-11T06:54:54.660 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-11T06:54:54.660 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-01-11T06:54:54.684 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-01-11T06:54:54.704 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-01-11T06:54:54.704 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-01-11T06:54:54.765 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="Red Hat Enterprise Linux" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="8.6 (Ootpa)" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:ID="rhel" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="fedora" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="8.6" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el8" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://www.redhat.com/" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-11T06:54:54.766 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-11T06:54:54.767 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T06:54:54.767 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-11T06:54:54.767 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-11T06:54:54.767 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-11T06:54:54.767 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-11T06:54:54.767 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-01-11T06:54:54.792 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-11T06:54:54.802 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-11T06:54:54.814 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-11T06:54:54.814 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-01-11T06:54:54.817 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-01-11T06:54:54.835 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-11T06:54:54.847 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-11T06:54:54.847 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-01-11T06:54:54.877 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-01-11T06:54:54.918 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-11T06:54:54.978 INFO:teuthology.run_tasks:Running task kernel... 2024-01-11T06:54:54.999 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-01-11T06:54:54.999 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-01-11T06:54:54.999 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-01-11T06:54:54.999 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-11T06:54:55.000 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-11T06:54:55.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T06:54:55.018 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-01-11T06:54:55.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T06:54:55.020 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-01-11T06:54:55.077 INFO:teuthology.orchestra.run.smithi113.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-11T06:54:55.077 INFO:teuthology.task.kernel:Running kernel on smithi113: 4.18.0-513.9.1.el8_9.x86_64 2024-01-11T06:54:55.077 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum install -y kernel 2024-01-11T06:54:55.081 INFO:teuthology.orchestra.run.smithi007.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-11T06:54:55.082 INFO:teuthology.task.kernel:Running kernel on smithi007: 4.18.0-513.9.1.el8_9.x86_64 2024-01-11T06:54:55.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-01-11T06:54:56.667 INFO:teuthology.orchestra.run.smithi113.stdout:Updating Subscription Management repositories. 2024-01-11T06:54:56.667 INFO:teuthology.orchestra.run.smithi113.stdout:Unable to read consumer identity 2024-01-11T06:54:56.706 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T06:54:56.706 INFO:teuthology.orchestra.run.smithi007.stdout:Unable to read consumer identity 2024-01-11T06:54:56.917 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T06:54:56.917 INFO:teuthology.orchestra.run.smithi113.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-11T06:54:56.917 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T06:54:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T06:54:57.070 INFO:teuthology.orchestra.run.smithi007.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-11T06:54:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T06:54:58.677 INFO:teuthology.orchestra.run.smithi113.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-01-11T06:54:58.776 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2024-01-11T06:54:59.479 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2024-01-11T06:54:59.762 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-01-11T06:55:03.470 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 344 kB/s | 24 kB 00:00 2024-01-11T06:55:03.501 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 592 kB/s | 24 kB 00:00 2024-01-11T06:55:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-01-11T06:55:04.239 INFO:teuthology.orchestra.run.smithi113.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-01-11T06:55:04.312 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-01-11T06:55:04.313 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-01-11T06:55:04.325 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-01-11T06:55:04.326 INFO:teuthology.orchestra.run.smithi113.stdout:Nothing to do. 2024-01-11T06:55:04.326 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-01-11T06:55:04.376 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-01-11T06:55:04.378 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-01-11T06:55:04.378 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-01-11T06:55:04.460 DEBUG:teuthology.orchestra.run.smithi113:> echo no | sudo yum reinstall kernel || true 2024-01-11T06:55:04.528 DEBUG:teuthology.orchestra.run.smithi007:> echo no | sudo yum reinstall kernel || true 2024-01-11T06:55:04.835 INFO:teuthology.orchestra.run.smithi113.stdout:Updating Subscription Management repositories. 2024-01-11T06:55:04.835 INFO:teuthology.orchestra.run.smithi113.stdout:Unable to read consumer identity 2024-01-11T06:55:04.915 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T06:55:04.915 INFO:teuthology.orchestra.run.smithi007.stdout:Unable to read consumer identity 2024-01-11T06:55:05.065 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T06:55:05.066 INFO:teuthology.orchestra.run.smithi113.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-11T06:55:05.066 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T06:55:05.123 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T06:55:05.123 INFO:teuthology.orchestra.run.smithi007.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-11T06:55:05.123 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T06:55:05.375 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:02 ago on Thu 11 Jan 2024 06:55:03 AM UTC. 2024-01-11T06:55:05.470 INFO:teuthology.orchestra.run.smithi113.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-11T06:55:05.480 INFO:teuthology.orchestra.run.smithi113.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-11T06:55:05.487 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:02 ago on Thu 11 Jan 2024 06:55:03 AM UTC. 2024-01-11T06:55:05.518 INFO:teuthology.orchestra.run.smithi113.stderr:Error: No packages marked for reinstall. 2024-01-11T06:55:05.566 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum reinstall -y kernel || true 2024-01-11T06:55:05.566 INFO:teuthology.orchestra.run.smithi007.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-11T06:55:05.570 INFO:teuthology.orchestra.run.smithi007.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-11T06:55:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:Error: No packages marked for reinstall. 2024-01-11T06:55:05.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum reinstall -y kernel || true 2024-01-11T06:55:05.899 INFO:teuthology.orchestra.run.smithi113.stdout:Updating Subscription Management repositories. 2024-01-11T06:55:05.899 INFO:teuthology.orchestra.run.smithi113.stdout:Unable to read consumer identity 2024-01-11T06:55:06.057 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T06:55:06.057 INFO:teuthology.orchestra.run.smithi007.stdout:Unable to read consumer identity 2024-01-11T06:55:06.127 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T06:55:06.127 INFO:teuthology.orchestra.run.smithi113.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-11T06:55:06.127 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T06:55:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T06:55:06.328 INFO:teuthology.orchestra.run.smithi007.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-11T06:55:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T06:55:06.432 INFO:teuthology.orchestra.run.smithi113.stdout:Last metadata expiration check: 0:00:03 ago on Thu 11 Jan 2024 06:55:03 AM UTC. 2024-01-11T06:55:06.499 INFO:teuthology.orchestra.run.smithi113.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-11T06:55:06.499 INFO:teuthology.orchestra.run.smithi113.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-11T06:55:06.540 INFO:teuthology.orchestra.run.smithi113.stderr:Error: No packages marked for reinstall. 2024-01-11T06:55:06.588 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q kernel | sort -rV | head -n 1 2024-01-11T06:55:06.691 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:03 ago on Thu 11 Jan 2024 06:55:03 AM UTC. 2024-01-11T06:55:06.752 INFO:teuthology.orchestra.run.smithi113.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-01-11T06:55:06.752 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-01-11T06:55:06.752 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-01-11T06:55:06.753 DEBUG:teuthology.parallel:result is None 2024-01-11T06:55:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-11T06:55:06.764 INFO:teuthology.orchestra.run.smithi007.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-11T06:55:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:Error: No packages marked for reinstall. 2024-01-11T06:55:06.849 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-01-11T06:55:07.017 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-01-11T06:55:07.017 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-01-11T06:55:07.017 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-01-11T06:55:07.017 DEBUG:teuthology.parallel:result is None 2024-01-11T06:55:07.018 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-11T06:55:07.025 INFO:teuthology.task.internal:Creating test directory... 2024-01-11T06:55:07.025 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-11T06:55:07.027 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-11T06:55:07.044 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-11T06:55:07.051 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-11T06:55:07.061 INFO:teuthology.task.internal:Creating archive directory... 2024-01-11T06:55:07.061 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-11T06:55:07.086 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-11T06:55:07.115 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-11T06:55:07.122 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-11T06:55:07.123 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-11T06:55:07.152 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-11T06:55:07.185 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T06:55:07.195 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T06:55:07.200 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T06:55:07.246 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-11T06:55:07.248 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-11T06:55:07.256 INFO:teuthology.task.internal:Configuring sudo... 2024-01-11T06:55:07.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-11T06:55:07.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-11T06:55:07.292 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-11T06:55:07.303 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-11T06:55:07.303 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-11T06:55:07.335 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-11T06:55:07.355 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T06:55:07.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T06:55:07.507 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T06:55:07.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T06:55:07.605 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T06:55:07.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-11T06:55:07.681 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T06:55:07.727 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-11T06:55:07.792 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T06:55:07.833 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-11T06:55:07.878 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T06:55:07.878 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-11T06:55:07.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-01-11T06:55:07.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-01-11T06:55:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T06:55:08.031 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T06:55:08.351 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-11T06:55:08.359 INFO:teuthology.task.internal:Starting timer... 2024-01-11T06:55:08.359 INFO:teuthology.run_tasks:Running task pcp... 2024-01-11T06:55:08.382 INFO:teuthology.run_tasks:Running task selinux... 2024-01-11T06:55:08.393 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2024-01-11T06:55:08.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-01-11T06:55:08.480 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-01-11T06:55:08.482 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-01-11T06:55:08.538 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-01-11T06:55:08.539 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-11T06:55:08.540 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-01-11T06:55:08.571 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-01-11T06:55:08.571 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-01-11T06:55:08.593 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-01-11T06:55:08.593 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive'} 2024-01-11T06:55:08.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T06:55:08.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T06:55:08.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T06:55:08.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T06:55:08.669 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-11T06:55:08.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-01-11T06:55:08.713 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-01-11T06:55:08.741 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-11T06:55:08.760 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-11T06:55:08.779 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-11T06:55:08.780 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-11T07:03:31.260 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2024-01-11T07:03:31.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T07:03:31.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T07:03:31.340 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-01-11T07:03:31.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-11T07:03:31.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-01-11T07:03:31.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-11T07:03:31.493 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-01-11T07:03:31.563 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-01-11T07:03:31.563 INFO:teuthology.run_tasks:Running task clock... 2024-01-11T07:03:31.573 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-11T07:03:31.573 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-11T07:03:31.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T07:03:31.576 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-11T07:03:31.576 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T07:03:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-11T07:03:31.633 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-11T07:03:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-11T07:03:31.653 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-11T07:03:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-01-11T07:03:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-01-11T07:03:31.726 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-01-11T07:03:31.730 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-01-11T07:03:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-11T07:03:31.745 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-11T07:03:31.760 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-11T07:03:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-11T07:03:31.823 INFO:teuthology.orchestra.run.smithi113.stderr:bash: ntpq: command not found 2024-01-11T07:03:31.826 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T07:03:31.826 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-01-11T07:03:31.826 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:03:31.826 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:03:31.826 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:03:31.826 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:03:31.841 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2024-01-11T07:03:31.847 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T07:03:31.847 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-01-11T07:03:31.847 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:03:31.847 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:03:31.847 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:03:31.847 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:03:31.847 INFO:teuthology.run_tasks:Running task install... 2024-01-11T07:03:31.856 DEBUG:teuthology.task.install:project ceph 2024-01-11T07:03:31.857 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680'}} 2024-01-11T07:03:31.857 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680'} 2024-01-11T07:03:31.857 INFO:teuthology.task.install:Using flavor: default 2024-01-11T07:03:31.862 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-11T07:03:31.863 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-01-11T07:03:31.863 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680', 'tag': None, 'wait_for_package': False} 2024-01-11T07:03:31.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4bcedb9fc11bed1fd6fd0c4fd59187401f309680 2024-01-11T07:03:31.865 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680', 'tag': None, 'wait_for_package': False} 2024-01-11T07:03:31.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4bcedb9fc11bed1fd6fd0c4fd59187401f309680 2024-01-11T07:03:32.018 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing1-quincy-2024-01-10-2010-quincy/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/centos/8/flavors/default/ 2024-01-11T07:03:32.019 INFO:teuthology.task.install.rpm:Package version is 17.2.7-299.g4bcedb9f 2024-01-11T07:03:32.029 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing1-quincy-2024-01-10-2010-quincy/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/centos/8/flavors/default/ 2024-01-11T07:03:32.029 INFO:teuthology.task.install.rpm:Package version is 17.2.7-299.g4bcedb9f 2024-01-11T07:03:32.216 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing1-quincy-2024-01-10-2010-quincy/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing1-quincy-2024-01-10-2010-quincy/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing1-quincy-2024-01-10-2010-quincy/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-11T07:03:32.216 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:03:32.216 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-11T07:03:32.234 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing1-quincy-2024-01-10-2010-quincy/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing1-quincy-2024-01-10-2010-quincy/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing1-quincy-2024-01-10-2010-quincy/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-11T07:03:32.234 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:03:32.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-11T07:03:32.248 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2024-01-11T07:03:32.248 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-11T07:03:32.276 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2024-01-11T07:03:32.277 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4bcedb9fc11bed1fd6fd0c4fd59187401f309680/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-11T07:03:32.327 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-11T07:03:32.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-11T07:03:32.405 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-11T07:03:32.441 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-11T07:03:32.473 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-01-11T07:03:32.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-01-11T07:03:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-01-11T07:03:32.478 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-01-11T07:03:32.859 INFO:teuthology.orchestra.run.smithi113.stdout:Updating Subscription Management repositories. 2024-01-11T07:03:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T07:03:34.216 INFO:teuthology.orchestra.run.smithi113.stdout:52 files removed 2024-01-11T07:03:34.268 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2024-01-11T07:03:34.289 INFO:teuthology.orchestra.run.smithi007.stdout:52 files removed 2024-01-11T07:03:34.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2024-01-11T07:03:34.604 INFO:teuthology.orchestra.run.smithi113.stdout:Updating Subscription Management repositories. 2024-01-11T07:03:34.729 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2024-01-11T07:03:36.491 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 208 kB/s | 78 kB 00:00 2024-01-11T07:03:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 33 kB/s | 9.2 kB 00:00 2024-01-11T07:03:37.193 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2024-01-11T07:03:37.352 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-01-11T07:03:38.847 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-01-11T07:03:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 23 kB/s | 78 kB 00:03 2024-01-11T07:03:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2024-01-11T07:03:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2024-01-11T07:03:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-01-11T07:03:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2024-01-11T07:03:42.861 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 542 kB/s | 24 kB 00:00 2024-01-11T07:03:44.412 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 64 MB 00:01 2024-01-11T07:03:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 258 kB/s | 24 kB 00:00 2024-01-11T07:03:46.243 INFO:teuthology.orchestra.run.smithi113.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 58 MB 00:01 2024-01-11T07:03:56.288 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 58 MB 00:01 2024-01-11T07:03:56.575 INFO:teuthology.orchestra.run.smithi113.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-01-11T07:03:59.493 INFO:teuthology.orchestra.run.smithi113.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 64 MB 00:01 2024-01-11T07:04:07.830 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-01-11T07:04:13.912 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T07:04:13.913 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T07:04:13.931 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-11T07:04:13.933 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-11T07:04:13.935 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-11T07:04:13.936 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-11T07:04:13.939 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-11T07:04:13.943 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-11T07:04:13.944 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-11T07:04:14.147 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-01-11T07:04:14.160 INFO:teuthology.orchestra.run.smithi113.stdout:=========================================================================================================================== 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout:=========================================================================================================================== 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 6.4 k 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 5.2 M 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 875 k 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 161 k 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 1.6 M 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 121 k 2024-01-11T07:04:14.161 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 1.7 M 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 7.4 M 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 47 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 17 M 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 52 M 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 267 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 80 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 255 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 18 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 25 k 2024-01-11T07:04:14.162 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 796 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 130 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 190 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 370 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 359 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 107 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 91 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 3.5 M 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 174 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-01-11T07:04:14.163 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 3.5 M 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 3.6 M 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-11T07:04:14.164 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 23 M 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 23 k 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 2.3 M 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 239 k 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 3.9 M 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 17 M 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 14 k 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 24 k 2024-01-11T07:04:14.165 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 178 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 507 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 5.6 M 2024-01-11T07:04:14.166 INFO:teuthology.orchestra.run.smithi113.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T07:04:14.167 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 45 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 115 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-11T07:04:14.168 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-11T07:04:14.169 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-11T07:04:14.170 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-11T07:04:14.171 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-11T07:04:14.172 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-11T07:04:14.172 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-11T07:04:14.172 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:04:14.172 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-01-11T07:04:14.172 INFO:teuthology.orchestra.run.smithi113.stdout:=========================================================================================================================== 2024-01-11T07:04:14.172 INFO:teuthology.orchestra.run.smithi113.stdout:Install 124 Packages 2024-01-11T07:04:14.172 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 16 Packages 2024-01-11T07:04:14.172 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:04:14.173 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 212 M 2024-01-11T07:04:14.173 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-01-11T07:04:14.584 INFO:teuthology.orchestra.run.smithi113.stdout:(1/140): ceph-17.2.7-299.g4bcedb9f.el8.x86_64.r 45 kB/s | 6.4 kB 00:00 2024-01-11T07:04:15.441 INFO:teuthology.orchestra.run.smithi113.stdout:(2/140): ceph-fuse-17.2.7-299.g4bcedb9f.el8.x86 1.0 MB/s | 875 kB 00:00 2024-01-11T07:04:15.509 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T07:04:15.511 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-11T07:04:15.531 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-11T07:04:15.533 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-11T07:04:15.535 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-11T07:04:15.536 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-11T07:04:15.539 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-11T07:04:15.544 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-11T07:04:15.546 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-11T07:04:15.610 INFO:teuthology.orchestra.run.smithi113.stdout:(3/140): ceph-immutable-object-cache-17.2.7-299 962 kB/s | 161 kB 00:00 2024-01-11T07:04:15.769 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-01-11T07:04:15.784 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 6.4 k 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 5.2 M 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 875 k 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 161 k 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 1.6 M 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 121 k 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 1.7 M 2024-01-11T07:04:15.785 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 7.4 M 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 47 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 17 M 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 52 M 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 267 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 80 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 255 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 18 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 25 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 796 k 2024-01-11T07:04:15.786 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 130 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 190 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 370 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 359 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 107 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 91 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 3.5 M 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 174 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-11T07:04:15.787 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 3.5 M 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 3.6 M 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-11T07:04:15.788 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 23 M 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 23 k 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 2.3 M 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 239 k 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 3.9 M 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 17 M 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:17.2.7-299.g4bcedb9f.el8 ceph-noarch 14 k 2024-01-11T07:04:15.789 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 24 k 2024-01-11T07:04:15.790 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-11T07:04:15.791 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-11T07:04:15.791 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-11T07:04:15.791 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-11T07:04:15.791 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-11T07:04:15.791 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 178 k 2024-01-11T07:04:15.791 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-11T07:04:15.791 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-11T07:04:15.791 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-11T07:04:15.792 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-11T07:04:15.792 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-11T07:04:15.792 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 507 k 2024-01-11T07:04:15.792 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-11T07:04:15.792 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 5.6 M 2024-01-11T07:04:15.792 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-11T07:04:15.792 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-11T07:04:15.792 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-11T07:04:15.793 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 45 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:17.2.7-299.g4bcedb9f.el8 ceph 115 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-11T07:04:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-11T07:04:15.795 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-11T07:04:15.796 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-01-11T07:04:15.797 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2024-01-11T07:04:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:Install 124 Packages 2024-01-11T07:04:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 16 Packages 2024-01-11T07:04:15.798 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:04:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 212 M 2024-01-11T07:04:15.798 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-01-11T07:04:16.272 INFO:teuthology.orchestra.run.smithi007.stdout:(1/140): ceph-17.2.7-299.g4bcedb9f.el8.x86_64.r 36 kB/s | 6.4 kB 00:00 2024-01-11T07:04:16.343 INFO:teuthology.orchestra.run.smithi113.stdout:(4/140): ceph-mds-17.2.7-299.g4bcedb9f.el8.x86_ 3.1 MB/s | 2.3 MB 00:00 2024-01-11T07:04:16.685 INFO:teuthology.orchestra.run.smithi113.stdout:(5/140): ceph-mgr-17.2.7-299.g4bcedb9f.el8.x86_ 4.6 MB/s | 1.6 MB 00:00 2024-01-11T07:04:16.831 INFO:teuthology.orchestra.run.smithi007.stdout:(2/140): ceph-fuse-17.2.7-299.g4bcedb9f.el8.x86 1.5 MB/s | 875 kB 00:00 2024-01-11T07:04:16.972 INFO:teuthology.orchestra.run.smithi007.stdout:(3/140): ceph-immutable-object-cache-17.2.7-299 1.1 MB/s | 161 kB 00:00 2024-01-11T07:04:17.410 INFO:teuthology.orchestra.run.smithi113.stdout:(6/140): ceph-mon-17.2.7-299.g4bcedb9f.el8.x86_ 5.4 MB/s | 3.9 MB 00:00 2024-01-11T07:04:17.500 INFO:teuthology.orchestra.run.smithi113.stdout:(7/140): ceph-base-17.2.7-299.g4bcedb9f.el8.x86 1.7 MB/s | 5.2 MB 00:03 2024-01-11T07:04:17.998 INFO:teuthology.orchestra.run.smithi007.stdout:(4/140): ceph-mds-17.2.7-299.g4bcedb9f.el8.x86_ 2.2 MB/s | 2.3 MB 00:01 2024-01-11T07:04:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:(5/140): ceph-mgr-17.2.7-299.g4bcedb9f.el8.x86_ 3.2 MB/s | 1.6 MB 00:00 2024-01-11T07:04:18.994 INFO:teuthology.orchestra.run.smithi113.stdout:(8/140): ceph-common-17.2.7-299.g4bcedb9f.el8.x 5.2 MB/s | 23 MB 00:04 2024-01-11T07:04:19.077 INFO:teuthology.orchestra.run.smithi113.stdout:(9/140): ceph-selinux-17.2.7-299.g4bcedb9f.el8. 293 kB/s | 24 kB 00:00 2024-01-11T07:04:19.140 INFO:teuthology.orchestra.run.smithi007.stdout:(6/140): ceph-mon-17.2.7-299.g4bcedb9f.el8.x86_ 6.0 MB/s | 3.9 MB 00:00 2024-01-11T07:04:19.615 INFO:teuthology.orchestra.run.smithi007.stdout:(7/140): ceph-base-17.2.7-299.g4bcedb9f.el8.x86 1.5 MB/s | 5.2 MB 00:03 2024-01-11T07:04:20.907 INFO:teuthology.orchestra.run.smithi007.stdout:(8/140): ceph-osd-17.2.7-299.g4bcedb9f.el8.x86_ 9.5 MB/s | 17 MB 00:01 2024-01-11T07:04:20.974 INFO:teuthology.orchestra.run.smithi007.stdout:(9/140): ceph-selinux-17.2.7-299.g4bcedb9f.el8. 367 kB/s | 24 kB 00:00 2024-01-11T07:04:23.210 INFO:teuthology.orchestra.run.smithi113.stdout:(10/140): ceph-osd-17.2.7-299.g4bcedb9f.el8.x86 2.9 MB/s | 17 MB 00:05 2024-01-11T07:04:23.450 INFO:teuthology.orchestra.run.smithi113.stdout:(11/140): cephfs-mirror-17.2.7-299.g4bcedb9f.el 1.0 MB/s | 255 kB 00:00 2024-01-11T07:04:23.544 INFO:teuthology.orchestra.run.smithi113.stdout:(12/140): libcephfs-devel-17.2.7-299.g4bcedb9f. 266 kB/s | 25 kB 00:00 2024-01-11T07:04:23.986 INFO:teuthology.orchestra.run.smithi113.stdout:(13/140): libcephfs2-17.2.7-299.g4bcedb9f.el8.x 1.8 MB/s | 796 kB 00:00 2024-01-11T07:04:24.144 INFO:teuthology.orchestra.run.smithi113.stdout:(14/140): libcephsqlite-17.2.7-299.g4bcedb9f.el 1.1 MB/s | 178 kB 00:00 2024-01-11T07:04:24.245 INFO:teuthology.orchestra.run.smithi113.stdout:(15/140): librados-devel-17.2.7-299.g4bcedb9f.e 1.3 MB/s | 130 kB 00:00 2024-01-11T07:04:24.412 INFO:teuthology.orchestra.run.smithi113.stdout:(16/140): libradosstriper1-17.2.7-299.g4bcedb9f 3.0 MB/s | 507 kB 00:00 2024-01-11T07:04:24.840 INFO:teuthology.orchestra.run.smithi007.stdout:(10/140): ceph-test-17.2.7-299.g4bcedb9f.el8.x8 13 MB/s | 52 MB 00:03 2024-01-11T07:04:24.907 INFO:teuthology.orchestra.run.smithi007.stdout:(11/140): cephfs-mirror-17.2.7-299.g4bcedb9f.el 3.8 MB/s | 255 kB 00:00 2024-01-11T07:04:24.949 INFO:teuthology.orchestra.run.smithi007.stdout:(12/140): libcephfs-devel-17.2.7-299.g4bcedb9f. 595 kB/s | 25 kB 00:00 2024-01-11T07:04:25.049 INFO:teuthology.orchestra.run.smithi007.stdout:(13/140): libcephfs2-17.2.7-299.g4bcedb9f.el8.x 7.8 MB/s | 796 kB 00:00 2024-01-11T07:04:25.092 INFO:teuthology.orchestra.run.smithi007.stdout:(14/140): libcephsqlite-17.2.7-299.g4bcedb9f.el 4.2 MB/s | 178 kB 00:00 2024-01-11T07:04:25.150 INFO:teuthology.orchestra.run.smithi007.stdout:(15/140): librados-devel-17.2.7-299.g4bcedb9f.e 2.2 MB/s | 130 kB 00:00 2024-01-11T07:04:25.225 INFO:teuthology.orchestra.run.smithi007.stdout:(16/140): libradosstriper1-17.2.7-299.g4bcedb9f 6.6 MB/s | 507 kB 00:00 2024-01-11T07:04:25.545 INFO:teuthology.orchestra.run.smithi113.stdout:(17/140): librgw2-17.2.7-299.g4bcedb9f.el8.x86_ 4.9 MB/s | 5.6 MB 00:01 2024-01-11T07:04:25.637 INFO:teuthology.orchestra.run.smithi113.stdout:(18/140): python3-ceph-argparse-17.2.7-299.g4bc 489 kB/s | 45 kB 00:00 2024-01-11T07:04:25.729 INFO:teuthology.orchestra.run.smithi113.stdout:(19/140): python3-ceph-common-17.2.7-299.g4bced 1.2 MB/s | 115 kB 00:00 2024-01-11T07:04:25.767 INFO:teuthology.orchestra.run.smithi007.stdout:(17/140): librgw2-17.2.7-299.g4bcedb9f.el8.x86_ 10 MB/s | 5.6 MB 00:00 2024-01-11T07:04:25.817 INFO:teuthology.orchestra.run.smithi007.stdout:(18/140): python3-ceph-argparse-17.2.7-299.g4bc 900 kB/s | 45 kB 00:00 2024-01-11T07:04:25.868 INFO:teuthology.orchestra.run.smithi007.stdout:(19/140): python3-ceph-common-17.2.7-299.g4bced 2.3 MB/s | 115 kB 00:00 2024-01-11T07:04:25.918 INFO:teuthology.orchestra.run.smithi007.stdout:(20/140): python3-cephfs-17.2.7-299.g4bcedb9f.e 3.7 MB/s | 190 kB 00:00 2024-01-11T07:04:25.968 INFO:teuthology.orchestra.run.smithi007.stdout:(21/140): python3-rados-17.2.7-299.g4bcedb9f.el 7.3 MB/s | 370 kB 00:00 2024-01-11T07:04:26.019 INFO:teuthology.orchestra.run.smithi007.stdout:(22/140): python3-rbd-17.2.7-299.g4bcedb9f.el8. 7.1 MB/s | 359 kB 00:00 2024-01-11T07:04:26.069 INFO:teuthology.orchestra.run.smithi007.stdout:(23/140): python3-rgw-17.2.7-299.g4bcedb9f.el8. 2.1 MB/s | 107 kB 00:00 2024-01-11T07:04:26.105 INFO:teuthology.orchestra.run.smithi113.stdout:(20/140): python3-cephfs-17.2.7-299.g4bcedb9f.e 507 kB/s | 190 kB 00:00 2024-01-11T07:04:26.119 INFO:teuthology.orchestra.run.smithi007.stdout:(24/140): rbd-fuse-17.2.7-299.g4bcedb9f.el8.x86 1.8 MB/s | 91 kB 00:00 2024-01-11T07:04:26.279 INFO:teuthology.orchestra.run.smithi113.stdout:(21/140): python3-rados-17.2.7-299.g4bcedb9f.el 2.1 MB/s | 370 kB 00:00 2024-01-11T07:04:26.437 INFO:teuthology.orchestra.run.smithi007.stdout:(25/140): rbd-mirror-17.2.7-299.g4bcedb9f.el8.x 11 MB/s | 3.5 MB 00:00 2024-01-11T07:04:26.438 INFO:teuthology.orchestra.run.smithi113.stdout:(22/140): python3-rbd-17.2.7-299.g4bcedb9f.el8. 2.2 MB/s | 359 kB 00:00 2024-01-11T07:04:26.487 INFO:teuthology.orchestra.run.smithi007.stdout:(26/140): rbd-nbd-17.2.7-299.g4bcedb9f.el8.x86_ 3.4 MB/s | 174 kB 00:00 2024-01-11T07:04:26.497 INFO:teuthology.orchestra.run.smithi113.stdout:(23/140): python3-rgw-17.2.7-299.g4bcedb9f.el8. 1.8 MB/s | 107 kB 00:00 2024-01-11T07:04:26.529 INFO:teuthology.orchestra.run.smithi007.stdout:(27/140): ceph-grafana-dashboards-17.2.7-299.g4 558 kB/s | 23 kB 00:00 2024-01-11T07:04:26.547 INFO:teuthology.orchestra.run.smithi113.stdout:(24/140): rbd-fuse-17.2.7-299.g4bcedb9f.el8.x86 1.8 MB/s | 91 kB 00:00 2024-01-11T07:04:26.571 INFO:teuthology.orchestra.run.smithi007.stdout:(28/140): ceph-mgr-cephadm-17.2.7-299.g4bcedb9f 2.8 MB/s | 121 kB 00:00 2024-01-11T07:04:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:(29/140): ceph-mgr-dashboard-17.2.7-299.g4bcedb 12 MB/s | 1.7 MB 00:00 2024-01-11T07:04:26.781 INFO:teuthology.orchestra.run.smithi113.stdout:(25/140): ceph-radosgw-17.2.7-299.g4bcedb9f.el8 1.8 MB/s | 17 MB 00:09 2024-01-11T07:04:27.048 INFO:teuthology.orchestra.run.smithi113.stdout:(26/140): rbd-nbd-17.2.7-299.g4bcedb9f.el8.x86_ 654 kB/s | 174 kB 00:00 2024-01-11T07:04:27.181 INFO:teuthology.orchestra.run.smithi113.stdout:(27/140): ceph-grafana-dashboards-17.2.7-299.g4 174 kB/s | 23 kB 00:00 2024-01-11T07:04:27.282 INFO:teuthology.orchestra.run.smithi113.stdout:(28/140): ceph-mgr-cephadm-17.2.7-299.g4bcedb9f 1.2 MB/s | 121 kB 00:00 2024-01-11T07:04:27.305 INFO:teuthology.orchestra.run.smithi007.stdout:(30/140): ceph-mgr-diskprediction-local-17.2.7- 12 MB/s | 7.4 MB 00:00 2024-01-11T07:04:27.455 INFO:teuthology.orchestra.run.smithi007.stdout:(31/140): ceph-mgr-modules-core-17.2.7-299.g4bc 1.6 MB/s | 239 kB 00:00 2024-01-11T07:04:27.465 INFO:teuthology.orchestra.run.smithi113.stdout:(29/140): rbd-mirror-17.2.7-299.g4bcedb9f.el8.x 3.8 MB/s | 3.5 MB 00:00 2024-01-11T07:04:27.630 INFO:teuthology.orchestra.run.smithi007.stdout:(32/140): ceph-mgr-rook-17.2.7-299.g4bcedb9f.el 271 kB/s | 47 kB 00:00 2024-01-11T07:04:27.672 INFO:teuthology.orchestra.run.smithi007.stdout:(33/140): ceph-prometheus-alerts-17.2.7-299.g4b 347 kB/s | 14 kB 00:00 2024-01-11T07:04:27.731 INFO:teuthology.orchestra.run.smithi007.stdout:(34/140): ceph-volume-17.2.7-299.g4bcedb9f.el8. 4.5 MB/s | 267 kB 00:00 2024-01-11T07:04:27.781 INFO:teuthology.orchestra.run.smithi007.stdout:(35/140): cephadm-17.2.7-299.g4bcedb9f.el8.noar 1.6 MB/s | 80 kB 00:00 2024-01-11T07:04:27.831 INFO:teuthology.orchestra.run.smithi007.stdout:(36/140): cephfs-top-17.2.7-299.g4bcedb9f.el8.n 361 kB/s | 18 kB 00:00 2024-01-11T07:04:28.024 INFO:teuthology.orchestra.run.smithi113.stdout:(30/140): ceph-mgr-dashboard-17.2.7-299.g4bcedb 2.3 MB/s | 1.7 MB 00:00 2024-01-11T07:04:28.124 INFO:teuthology.orchestra.run.smithi007.stdout:(37/140): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2024-01-11T07:04:28.220 INFO:teuthology.orchestra.run.smithi007.stdout:(38/140): liboath-2.6.2-3.el8.x86_64.rpm 612 kB/s | 59 kB 00:00 2024-01-11T07:04:28.291 INFO:teuthology.orchestra.run.smithi007.stdout:(39/140): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2024-01-11T07:04:28.408 INFO:teuthology.orchestra.run.smithi007.stdout:(40/140): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2024-01-11T07:04:28.475 INFO:teuthology.orchestra.run.smithi007.stdout:(41/140): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2024-01-11T07:04:28.559 INFO:teuthology.orchestra.run.smithi007.stdout:(42/140): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2024-01-11T07:04:28.616 INFO:teuthology.orchestra.run.smithi113.stdout:(31/140): ceph-test-17.2.7-299.g4bcedb9f.el8.x8 5.4 MB/s | 52 MB 00:09 2024-01-11T07:04:28.634 INFO:teuthology.orchestra.run.smithi007.stdout:(43/140): python3-cachetools-3.1.1-4.el8.noarch 441 kB/s | 33 kB 00:00 2024-01-11T07:04:28.641 INFO:teuthology.orchestra.run.smithi113.stdout:(32/140): ceph-mgr-modules-core-17.2.7-299.g4bc 387 kB/s | 239 kB 00:00 2024-01-11T07:04:28.666 INFO:teuthology.orchestra.run.smithi113.stdout:(33/140): ceph-mgr-rook-17.2.7-299.g4bcedb9f.el 949 kB/s | 47 kB 00:00 2024-01-11T07:04:28.692 INFO:teuthology.orchestra.run.smithi007.stdout:(44/140): python3-certifi-2018.10.15-7.el8.noar 281 kB/s | 16 kB 00:00 2024-01-11T07:04:28.708 INFO:teuthology.orchestra.run.smithi113.stdout:(34/140): ceph-prometheus-alerts-17.2.7-299.g4b 217 kB/s | 14 kB 00:00 2024-01-11T07:04:28.759 INFO:teuthology.orchestra.run.smithi113.stdout:(35/140): ceph-volume-17.2.7-299.g4bcedb9f.el8. 2.8 MB/s | 267 kB 00:00 2024-01-11T07:04:28.759 INFO:teuthology.orchestra.run.smithi007.stdout:(45/140): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2024-01-11T07:04:28.835 INFO:teuthology.orchestra.run.smithi007.stdout:(46/140): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2024-01-11T07:04:28.876 INFO:teuthology.orchestra.run.smithi113.stdout:(36/140): ceph-mgr-diskprediction-local-17.2.7- 5.2 MB/s | 7.4 MB 00:01 2024-01-11T07:04:28.901 INFO:teuthology.orchestra.run.smithi113.stdout:(37/140): cephfs-top-17.2.7-299.g4bcedb9f.el8.n 127 kB/s | 18 kB 00:00 2024-01-11T07:04:28.902 INFO:teuthology.orchestra.run.smithi007.stdout:(47/140): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2024-01-11T07:04:28.926 INFO:teuthology.orchestra.run.smithi113.stdout:(38/140): cephadm-17.2.7-299.g4bcedb9f.el8.noar 370 kB/s | 80 kB 00:00 2024-01-11T07:04:28.960 INFO:teuthology.orchestra.run.smithi007.stdout:(48/140): python3-jaraco-6.2-6.el8.noarch.rpm 191 kB/s | 11 kB 00:00 2024-01-11T07:04:29.028 INFO:teuthology.orchestra.run.smithi007.stdout:(49/140): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2024-01-11T07:04:29.143 INFO:teuthology.orchestra.run.smithi113.stdout:(39/140): liboath-2.6.2-3.el8.x86_64.rpm 243 kB/s | 59 kB 00:00 2024-01-11T07:04:29.160 INFO:teuthology.orchestra.run.smithi113.stdout:(40/140): libunwind-1.3.1-3.el8.x86_64.rpm 321 kB/s | 75 kB 00:00 2024-01-11T07:04:29.170 INFO:teuthology.orchestra.run.smithi007.stdout:(50/140): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2024-01-11T07:04:29.245 INFO:teuthology.orchestra.run.smithi007.stdout:(51/140): python3-logutils-0.3.5-11.el8.noarch. 652 kB/s | 49 kB 00:00 2024-01-11T07:04:29.252 INFO:teuthology.orchestra.run.smithi113.stdout:(41/140): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 477 kB/s | 44 kB 00:00 2024-01-11T07:04:29.327 INFO:teuthology.orchestra.run.smithi113.stdout:(42/140): gperftools-libs-2.7-9.el8.x86_64.rpm 678 kB/s | 306 kB 00:00 2024-01-11T07:04:29.329 INFO:teuthology.orchestra.run.smithi007.stdout:(52/140): python3-more-itertools-7.2.0-3.el8.no 704 kB/s | 59 kB 00:00 2024-01-11T07:04:29.394 INFO:teuthology.orchestra.run.smithi113.stdout:(43/140): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2024-01-11T07:04:29.396 INFO:teuthology.orchestra.run.smithi007.stdout:(53/140): python3-natsort-7.1.1-2.el8.noarch.rp 905 kB/s | 60 kB 00:00 2024-01-11T07:04:29.419 INFO:teuthology.orchestra.run.smithi113.stdout:(44/140): python3-cachetools-3.1.1-4.el8.noarch 360 kB/s | 33 kB 00:00 2024-01-11T07:04:29.471 INFO:teuthology.orchestra.run.smithi007.stdout:(54/140): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2024-01-11T07:04:29.478 INFO:teuthology.orchestra.run.smithi113.stdout:(45/140): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2024-01-11T07:04:29.512 INFO:teuthology.orchestra.run.smithi113.stdout:(46/140): python3-beautifulsoup4-4.6.3-2.el8.1. 714 kB/s | 185 kB 00:00 2024-01-11T07:04:29.538 INFO:teuthology.orchestra.run.smithi007.stdout:(55/140): python3-portend-2.6-1.el8.noarch.rpm 234 kB/s | 16 kB 00:00 2024-01-11T07:04:29.613 INFO:teuthology.orchestra.run.smithi007.stdout:(56/140): python3-repoze-lru-0.7-6.el8.noarch.r 449 kB/s | 34 kB 00:00 2024-01-11T07:04:29.645 INFO:teuthology.orchestra.run.smithi113.stdout:(47/140): python3-cheroot-8.5.2-1.el8.noarch.rp 765 kB/s | 173 kB 00:00 2024-01-11T07:04:29.670 INFO:teuthology.orchestra.run.smithi113.stdout:(48/140): python3-google-auth-1.1.1-10.el8.noar 512 kB/s | 81 kB 00:00 2024-01-11T07:04:29.689 INFO:teuthology.orchestra.run.smithi007.stdout:(57/140): python3-routes-2.4.1-12.el8.noarch.rp 2.6 MB/s | 196 kB 00:00 2024-01-11T07:04:29.697 INFO:teuthology.orchestra.run.smithi113.stdout:(49/140): python3-cherrypy-18.4.0-1.el8.noarch. 1.7 MB/s | 384 kB 00:00 2024-01-11T07:04:29.729 INFO:teuthology.orchestra.run.smithi113.stdout:(50/140): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2024-01-11T07:04:29.755 INFO:teuthology.orchestra.run.smithi113.stdout:(51/140): python3-jaraco-functools-2.0-4.el8.no 209 kB/s | 18 kB 00:00 2024-01-11T07:04:29.756 INFO:teuthology.orchestra.run.smithi007.stdout:(58/140): python3-rsa-4.9-2.el8.noarch.rpm 966 kB/s | 64 kB 00:00 2024-01-11T07:04:29.814 INFO:teuthology.orchestra.run.smithi113.stdout:(52/140): python3-logutils-0.3.5-11.el8.noarch. 582 kB/s | 49 kB 00:00 2024-01-11T07:04:29.822 INFO:teuthology.orchestra.run.smithi007.stdout:(59/140): python3-simplegeneric-0.8.1-17.el8.no 281 kB/s | 19 kB 00:00 2024-01-11T07:04:29.839 INFO:teuthology.orchestra.run.smithi113.stdout:(53/140): python3-more-itertools-7.2.0-3.el8.no 704 kB/s | 59 kB 00:00 2024-01-11T07:04:29.900 INFO:teuthology.orchestra.run.smithi007.stdout:(60/140): python3-singledispatch-3.4.0.3-18.el8 318 kB/s | 24 kB 00:00 2024-01-11T07:04:29.906 INFO:teuthology.orchestra.run.smithi113.stdout:(54/140): python3-natsort-7.1.1-2.el8.noarch.rp 655 kB/s | 60 kB 00:00 2024-01-11T07:04:29.973 INFO:teuthology.orchestra.run.smithi007.stdout:(61/140): python3-tempora-1.14.1-5.el8.noarch.r 391 kB/s | 29 kB 00:00 2024-01-11T07:04:29.989 INFO:teuthology.orchestra.run.smithi113.stdout:(55/140): python3-portend-2.6-1.el8.noarch.rpm 187 kB/s | 16 kB 00:00 2024-01-11T07:04:30.057 INFO:teuthology.orchestra.run.smithi007.stdout:(62/140): python3-trustme-0.6.0-4.el8.noarch.rp 331 kB/s | 27 kB 00:00 2024-01-11T07:04:30.090 INFO:teuthology.orchestra.run.smithi113.stdout:(56/140): python3-kubernetes-11.0.0-6.el8.noarc 3.0 MB/s | 1.2 MB 00:00 2024-01-11T07:04:30.134 INFO:teuthology.orchestra.run.smithi113.stdout:(57/140): python3-repoze-lru-0.7-6.el8.noarch.r 234 kB/s | 34 kB 00:00 2024-01-11T07:04:30.149 INFO:teuthology.orchestra.run.smithi007.stdout:(63/140): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2024-01-11T07:04:30.165 INFO:teuthology.orchestra.run.smithi113.stdout:(58/140): python3-pecan-1.3.2-9.el8.noarch.rpm 867 kB/s | 283 kB 00:00 2024-01-11T07:04:30.190 INFO:teuthology.orchestra.run.smithi113.stdout:(59/140): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-01-11T07:04:30.216 INFO:teuthology.orchestra.run.smithi113.stdout:(60/140): python3-rsa-4.9-2.el8.noarch.rpm 785 kB/s | 64 kB 00:00 2024-01-11T07:04:30.233 INFO:teuthology.orchestra.run.smithi113.stdout:(61/140): python3-simplegeneric-0.8.1-17.el8.no 278 kB/s | 19 kB 00:00 2024-01-11T07:04:30.275 INFO:teuthology.orchestra.run.smithi113.stdout:(62/140): python3-singledispatch-3.4.0.3-18.el8 285 kB/s | 24 kB 00:00 2024-01-11T07:04:30.292 INFO:teuthology.orchestra.run.smithi113.stdout:(63/140): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2024-01-11T07:04:30.307 INFO:teuthology.orchestra.run.smithi007.stdout:(64/140): ceph-radosgw-17.2.7-299.g4bcedb9f.el8 1.6 MB/s | 17 MB 00:10 2024-01-11T07:04:30.309 INFO:teuthology.orchestra.run.smithi113.stdout:(64/140): python3-trustme-0.6.0-4.el8.noarch.rp 364 kB/s | 27 kB 00:00 2024-01-11T07:04:30.333 INFO:teuthology.orchestra.run.smithi007.stdout:(65/140): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2024-01-11T07:04:30.388 INFO:teuthology.orchestra.run.smithi113.stdout:(65/140): python3-waitress-1.4.3-1.el8.noarch.r 2.2 MB/s | 255 kB 00:00 2024-01-11T07:04:30.408 INFO:teuthology.orchestra.run.smithi007.stdout:(66/140): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2024-01-11T07:04:30.409 INFO:teuthology.orchestra.run.smithi113.stdout:(66/140): python3-websocket-client-0.56.0-5.el8 610 kB/s | 61 kB 00:00 2024-01-11T07:04:30.467 INFO:teuthology.orchestra.run.smithi007.stdout:(67/140): python3-zc-lockfile-2.0-2.el8.noarch. 393 kB/s | 23 kB 00:00 2024-01-11T07:04:30.476 INFO:teuthology.orchestra.run.smithi113.stdout:(67/140): python3-webtest-2.0.33-1.el8.noarch.r 972 kB/s | 85 kB 00:00 2024-01-11T07:04:30.493 INFO:teuthology.orchestra.run.smithi113.stdout:(68/140): python3-zc-lockfile-2.0-2.el8.noarch. 275 kB/s | 23 kB 00:00 2024-01-11T07:04:30.517 INFO:teuthology.orchestra.run.smithi007.stdout:(68/140): python3-websocket-client-0.56.0-5.el8 293 kB/s | 61 kB 00:00 2024-01-11T07:04:30.518 INFO:teuthology.orchestra.run.smithi113.stdout:(69/140): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-01-11T07:04:30.534 INFO:teuthology.orchestra.run.smithi007.stdout:(69/140): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2024-01-11T07:04:30.576 INFO:teuthology.orchestra.run.smithi007.stdout:(70/140): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2024-01-11T07:04:30.577 INFO:teuthology.orchestra.run.smithi113.stdout:(70/140): python3-webob-1.8.5-1.el8.1.noarch.rp 881 kB/s | 251 kB 00:00 2024-01-11T07:04:30.686 INFO:teuthology.orchestra.run.smithi113.stdout:(71/140): python3-jsonpointer-1.10-11.el8.noarc 120 kB/s | 20 kB 00:00 2024-01-11T07:04:30.701 INFO:teuthology.orchestra.run.smithi007.stdout:(71/140): libverto-devel-0.3.0-5.el8.x86_64.rpm 107 kB/s | 18 kB 00:00 2024-01-11T07:04:30.728 INFO:teuthology.orchestra.run.smithi113.stdout:(72/140): python3-prettytable-0.7.2-14.el8.noar 293 kB/s | 44 kB 00:00 2024-01-11T07:04:30.751 INFO:teuthology.orchestra.run.smithi007.stdout:(72/140): python3-jwt-1.6.1-2.el8.noarch.rpm 256 kB/s | 43 kB 00:00 2024-01-11T07:04:30.845 INFO:teuthology.orchestra.run.smithi113.stdout:(73/140): python3-jsonpatch-1.21-2.el8.noarch.r 227 kB/s | 27 kB 00:00 2024-01-11T07:04:30.913 INFO:teuthology.orchestra.run.smithi113.stdout:(74/140): thrift-0.13.0-2.el8.x86_64.rpm 4.0 MB/s | 1.7 MB 00:00 2024-01-11T07:04:30.920 INFO:teuthology.orchestra.run.smithi007.stdout:(73/140): python3-oauthlib-2.1.0-1.el8.noarch.r 714 kB/s | 155 kB 00:00 2024-01-11T07:04:30.935 INFO:teuthology.orchestra.run.smithi007.stdout:(74/140): python3-pycparser-2.14-14.el8.noarch. 594 kB/s | 109 kB 00:00 2024-01-11T07:04:30.945 INFO:teuthology.orchestra.run.smithi113.stdout:(75/140): python3-pyasn1-modules-0.3.7-6.el8.no 425 kB/s | 110 kB 00:00 2024-01-11T07:04:31.044 INFO:teuthology.orchestra.run.smithi007.stdout:(75/140): python3-requests-oauthlib-1.0.0-1.el8 341 kB/s | 43 kB 00:00 2024-01-11T07:04:31.045 INFO:teuthology.orchestra.run.smithi113.stdout:(76/140): python3-pyasn1-0.3.7-6.el8.noarch.rpm 631 kB/s | 126 kB 00:00 2024-01-11T07:04:31.103 INFO:teuthology.orchestra.run.smithi007.stdout:(76/140): libaio-devel-0.3.112-1.el8.x86_64.rpm 112 kB/s | 19 kB 00:00 2024-01-11T07:04:31.104 INFO:teuthology.orchestra.run.smithi113.stdout:(77/140): python3-werkzeug-0.12.2-4.el8.noarch. 2.3 MB/s | 457 kB 00:00 2024-01-11T07:04:31.146 INFO:teuthology.orchestra.run.smithi113.stdout:(78/140): bison-3.0.4-10.el8.x86_64.rpm 3.4 MB/s | 688 kB 00:00 2024-01-11T07:04:31.161 INFO:teuthology.orchestra.run.smithi007.stdout:(77/140): libzstd-devel-1.4.4-1.el8.x86_64.rpm 375 kB/s | 44 kB 00:00 2024-01-11T07:04:31.241 INFO:teuthology.orchestra.run.smithi113.stdout:(79/140): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2024-01-11T07:04:31.262 INFO:teuthology.orchestra.run.smithi007.stdout:(78/140): pcre2-devel-10.32-2.el8.x86_64.rpm 3.7 MB/s | 605 kB 00:00 2024-01-11T07:04:31.295 INFO:teuthology.orchestra.run.smithi007.stdout:(79/140): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2024-01-11T07:04:31.299 INFO:teuthology.orchestra.run.smithi113.stdout:(80/140): python3-markupsafe-0.23-19.el8.x86_64 255 kB/s | 39 kB 00:00 2024-01-11T07:04:31.332 INFO:teuthology.orchestra.run.smithi113.stdout:(81/140): libtool-2.4.6-25.el8.x86_64.rpm 3.0 MB/s | 709 kB 00:00 2024-01-11T07:04:31.366 INFO:teuthology.orchestra.run.smithi113.stdout:(82/140): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-01-11T07:04:31.387 INFO:teuthology.orchestra.run.smithi007.stdout:(80/140): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2024-01-11T07:04:31.413 INFO:teuthology.orchestra.run.smithi007.stdout:(81/140): libselinux-devel-2.9-5.el8.x86_64.rpm 1.7 MB/s | 200 kB 00:00 2024-01-11T07:04:31.508 INFO:teuthology.orchestra.run.smithi113.stdout:(83/140): oniguruma-6.8.2-2.el8.x86_64.rpm 896 kB/s | 187 kB 00:00 2024-01-11T07:04:31.531 INFO:teuthology.orchestra.run.smithi007.stdout:(82/140): keyutils-libs-devel-1.5.10-9.el8.x86_ 339 kB/s | 48 kB 00:00 2024-01-11T07:04:31.533 INFO:teuthology.orchestra.run.smithi113.stdout:(84/140): python3-pyOpenSSL-19.0.0-1.el8.noarch 516 kB/s | 103 kB 00:00 2024-01-11T07:04:31.588 INFO:teuthology.orchestra.run.smithi007.stdout:(83/140): libsepol-devel-2.9-3.el8.x86_64.rpm 496 kB/s | 87 kB 00:00 2024-01-11T07:04:31.639 INFO:teuthology.orchestra.run.smithi007.stdout:(84/140): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 532 kB/s | 58 kB 00:00 2024-01-11T07:04:31.650 INFO:teuthology.orchestra.run.smithi113.stdout:(85/140): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-01-11T07:04:31.684 INFO:teuthology.orchestra.run.smithi113.stdout:(86/140): python3-jinja2-2.10.1-3.el8.noarch.rp 3.0 MB/s | 538 kB 00:00 2024-01-11T07:04:31.714 INFO:teuthology.orchestra.run.smithi007.stdout:(85/140): libcom_err-devel-1.45.6-4.el8.x86_64. 308 kB/s | 39 kB 00:00 2024-01-11T07:04:31.739 INFO:teuthology.orchestra.run.smithi007.stdout:(86/140): fuse-2.9.7-17.el8.x86_64.rpm 829 kB/s | 83 kB 00:00 2024-01-11T07:04:31.776 INFO:teuthology.orchestra.run.smithi113.stdout:(87/140): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2024-01-11T07:04:31.801 INFO:teuthology.orchestra.run.smithi113.stdout:(88/140): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2024-01-11T07:04:31.835 INFO:teuthology.orchestra.run.smithi113.stdout:(89/140): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2024-01-11T07:04:31.873 INFO:teuthology.orchestra.run.smithi007.stdout:(87/140): libgfortran-8.5.0-20.el8.x86_64.rpm 4.0 MB/s | 645 kB 00:00 2024-01-11T07:04:31.877 INFO:teuthology.orchestra.run.smithi113.stdout:(90/140): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2024-01-11T07:04:31.898 INFO:teuthology.orchestra.run.smithi007.stdout:(88/140): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-01-11T07:04:31.965 INFO:teuthology.orchestra.run.smithi007.stdout:(89/140): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.1 MB/s | 100 kB 00:00 2024-01-11T07:04:32.010 INFO:teuthology.orchestra.run.smithi113.stdout:(91/140): python3-mako-1.0.6-14.el8.noarch.rpm 895 kB/s | 157 kB 00:00 2024-01-11T07:04:32.048 INFO:teuthology.orchestra.run.smithi007.stdout:(90/140): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-01-11T07:04:32.094 INFO:teuthology.orchestra.run.smithi113.stdout:(92/140): openblas-threads-0.3.15-6.el8.x86_64. 18 MB/s | 5.2 MB 00:00 2024-01-11T07:04:32.124 INFO:teuthology.orchestra.run.smithi007.stdout:(91/140): python3-cryptography-3.2.1-6.el8.x86_ 3.4 MB/s | 559 kB 00:00 2024-01-11T07:04:32.153 INFO:teuthology.orchestra.run.smithi113.stdout:(93/140): automake-1.16.1-8.el8.noarch.rpm 4.9 MB/s | 714 kB 00:00 2024-01-11T07:04:32.186 INFO:teuthology.orchestra.run.smithi113.stdout:(94/140): jq-1.6-7.el8.x86_64.rpm 2.2 MB/s | 202 kB 00:00 2024-01-11T07:04:32.278 INFO:teuthology.orchestra.run.smithi113.stdout:(95/140): openblas-0.3.15-6.el8.x86_64.rpm 13 MB/s | 5.0 MB 00:00 2024-01-11T07:04:32.303 INFO:teuthology.orchestra.run.smithi113.stdout:(96/140): python3-pytz-2017.2-11.el8.noarch.rpm 360 kB/s | 54 kB 00:00 2024-01-11T07:04:32.349 INFO:teuthology.orchestra.run.smithi007.stdout:(92/140): ceph-common-17.2.7-299.g4bcedb9f.el8. 1.4 MB/s | 23 MB 00:16 2024-01-11T07:04:32.366 INFO:teuthology.orchestra.run.smithi007.stdout:(93/140): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 229 kB/s | 73 kB 00:00 2024-01-11T07:04:32.399 INFO:teuthology.orchestra.run.smithi007.stdout:(94/140): NetworkManager-initscripts-updown-1.4 521 kB/s | 143 kB 00:00 2024-01-11T07:04:32.412 INFO:teuthology.orchestra.run.smithi113.stdout:(97/140): libverto-devel-0.3.0-5.el8.x86_64.rpm 134 kB/s | 18 kB 00:00 2024-01-11T07:04:32.437 INFO:teuthology.orchestra.run.smithi113.stdout:(98/140): python3-jwt-1.6.1-2.el8.noarch.rpm 319 kB/s | 43 kB 00:00 2024-01-11T07:04:32.492 INFO:teuthology.orchestra.run.smithi007.stdout:(95/140): librabbitmq-0.9.0-4.el8.x86_64.rpm 375 kB/s | 47 kB 00:00 2024-01-11T07:04:32.517 INFO:teuthology.orchestra.run.smithi007.stdout:(96/140): elfutils-libelf-devel-0.189-3.el8.x86 369 kB/s | 62 kB 00:00 2024-01-11T07:04:32.559 INFO:teuthology.orchestra.run.smithi007.stdout:(97/140): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.5 MB/s | 562 kB 00:00 2024-01-11T07:04:32.618 INFO:teuthology.orchestra.run.smithi007.stdout:(98/140): python3-jsonpointer-1.10-11.el8.noarc 199 kB/s | 20 kB 00:00 2024-01-11T07:04:32.629 INFO:teuthology.orchestra.run.smithi113.stdout:(99/140): python3-scipy-1.0.0-21.module+el8.9.0 31 MB/s | 14 MB 00:00 2024-01-11T07:04:32.654 INFO:teuthology.orchestra.run.smithi113.stdout:(100/140): python3-pycparser-2.14-14.el8.noarch 502 kB/s | 109 kB 00:00 2024-01-11T07:04:32.672 INFO:teuthology.orchestra.run.smithi113.stdout:(101/140): python3-oauthlib-2.1.0-1.el8.noarch. 598 kB/s | 155 kB 00:00 2024-01-11T07:04:32.701 INFO:teuthology.orchestra.run.smithi007.stdout:(99/140): python3-prettytable-0.7.2-14.el8.noar 310 kB/s | 44 kB 00:00 2024-01-11T07:04:32.726 INFO:teuthology.orchestra.run.smithi007.stdout:(100/140): python3-pyasn1-modules-0.3.7-6.el8.n 1.0 MB/s | 110 kB 00:00 2024-01-11T07:04:32.772 INFO:teuthology.orchestra.run.smithi113.stdout:(102/140): python3-requests-oauthlib-1.0.0-1.el 300 kB/s | 43 kB 00:00 2024-01-11T07:04:32.802 INFO:teuthology.orchestra.run.smithi007.stdout:(101/140): openssl-devel-1.1.1k-12.el8_9.x86_64 7.5 MB/s | 2.3 MB 00:00 2024-01-11T07:04:32.827 INFO:teuthology.orchestra.run.smithi007.stdout:(102/140): python3-jsonpatch-1.21-2.el8.noarch. 212 kB/s | 27 kB 00:00 2024-01-11T07:04:32.838 INFO:teuthology.orchestra.run.smithi113.stdout:(103/140): libaio-devel-0.3.112-1.el8.x86_64.rp 102 kB/s | 19 kB 00:00 2024-01-11T07:04:32.889 INFO:teuthology.orchestra.run.smithi113.stdout:(104/140): libzstd-devel-1.4.4-1.el8.x86_64.rpm 202 kB/s | 44 kB 00:00 2024-01-11T07:04:32.902 INFO:teuthology.orchestra.run.smithi007.stdout:(103/140): python3-pyasn1-0.3.7-6.el8.noarch.rp 721 kB/s | 126 kB 00:00 2024-01-11T07:04:32.936 INFO:teuthology.orchestra.run.smithi007.stdout:(104/140): python3-werkzeug-0.12.2-4.el8.noarch 3.3 MB/s | 457 kB 00:00 2024-01-11T07:04:32.939 INFO:teuthology.orchestra.run.smithi113.stdout:(105/140): pcre2-devel-10.32-2.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2024-01-11T07:04:33.004 INFO:teuthology.orchestra.run.smithi113.stdout:(106/140): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.9 MB/s | 220 kB 00:00 2024-01-11T07:04:33.040 INFO:teuthology.orchestra.run.smithi113.stdout:(107/140): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2024-01-11T07:04:33.107 INFO:teuthology.orchestra.run.smithi113.stdout:(108/140): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2024-01-11T07:04:33.166 INFO:teuthology.orchestra.run.smithi113.stdout:(109/140): keyutils-libs-devel-1.5.10-9.el8.x86 299 kB/s | 48 kB 00:00 2024-01-11T07:04:33.191 INFO:teuthology.orchestra.run.smithi113.stdout:(110/140): libsepol-devel-2.9-3.el8.x86_64.rpm 578 kB/s | 87 kB 00:00 2024-01-11T07:04:33.193 INFO:teuthology.orchestra.run.smithi007.stdout:(105/140): bison-3.0.4-10.el8.x86_64.rpm 1.8 MB/s | 688 kB 00:00 2024-01-11T07:04:33.228 INFO:teuthology.orchestra.run.smithi007.stdout:(106/140): flex-2.6.1-9.el8.x86_64.rpm 981 kB/s | 320 kB 00:00 2024-01-11T07:04:33.233 INFO:teuthology.orchestra.run.smithi113.stdout:(111/140): zlib-devel-1.2.11-18.el8_5.x86_64.rp 459 kB/s | 58 kB 00:00 2024-01-11T07:04:33.262 INFO:teuthology.orchestra.run.smithi007.stdout:(107/140): libtool-2.4.6-25.el8.x86_64.rpm 2.1 MB/s | 709 kB 00:00 2024-01-11T07:04:33.287 INFO:teuthology.orchestra.run.smithi007.stdout:(108/140): python3-markupsafe-0.23-19.el8.x86_6 417 kB/s | 39 kB 00:00 2024-01-11T07:04:33.300 INFO:teuthology.orchestra.run.smithi113.stdout:(112/140): libcom_err-devel-1.45.6-4.el8.x86_64 288 kB/s | 39 kB 00:00 2024-01-11T07:04:33.312 INFO:teuthology.orchestra.run.smithi007.stdout:(109/140): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2024-01-11T07:04:33.325 INFO:teuthology.orchestra.run.smithi113.stdout:(113/140): fuse-2.9.7-17.el8.x86_64.rpm 620 kB/s | 83 kB 00:00 2024-01-11T07:04:33.350 INFO:teuthology.orchestra.run.smithi113.stdout:(114/140): libgfortran-8.5.0-20.el8.x86_64.rpm 5.4 MB/s | 645 kB 00:00 2024-01-11T07:04:33.363 INFO:teuthology.orchestra.run.smithi007.stdout:(110/140): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2024-01-11T07:04:33.388 INFO:teuthology.orchestra.run.smithi007.stdout:(111/140): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.0 MB/s | 103 kB 00:00 2024-01-11T07:04:33.400 INFO:teuthology.orchestra.run.smithi113.stdout:(115/140): libquadmath-8.5.0-20.el8.x86_64.rpm 1.7 MB/s | 172 kB 00:00 2024-01-11T07:04:33.443 INFO:teuthology.orchestra.run.smithi113.stdout:(116/140): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2024-01-11T07:04:33.468 INFO:teuthology.orchestra.run.smithi113.stdout:(117/140): libuuid-devel-2.32.1-43.el8.x86_64.r 700 kB/s | 100 kB 00:00 2024-01-11T07:04:33.551 INFO:teuthology.orchestra.run.smithi113.stdout:(118/140): xfsprogs-devel-5.0.0-12.el8.x86_64.r 670 kB/s | 73 kB 00:00 2024-01-11T07:04:33.555 INFO:teuthology.orchestra.run.smithi007.stdout:(112/140): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2024-01-11T07:04:33.577 INFO:teuthology.orchestra.run.smithi113.stdout:(119/140): python3-cryptography-3.2.1-6.el8.x86 3.1 MB/s | 559 kB 00:00 2024-01-11T07:04:33.605 INFO:teuthology.orchestra.run.smithi007.stdout:(113/140): python3-jinja2-2.10.1-3.el8.noarch.r 2.2 MB/s | 538 kB 00:00 2024-01-11T07:04:33.610 INFO:teuthology.orchestra.run.smithi113.stdout:(120/140): NetworkManager-initscripts-updown-1. 1.0 MB/s | 143 kB 00:00 2024-01-11T07:04:33.656 INFO:teuthology.orchestra.run.smithi007.stdout:(114/140): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2024-01-11T07:04:33.669 INFO:teuthology.orchestra.run.smithi113.stdout:(121/140): elfutils-libelf-devel-0.189-3.el8.x8 526 kB/s | 62 kB 00:00 2024-01-11T07:04:33.722 INFO:teuthology.orchestra.run.smithi007.stdout:(115/140): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2024-01-11T07:04:33.744 INFO:teuthology.orchestra.run.smithi113.stdout:(122/140): librabbitmq-0.9.0-4.el8.x86_64.rpm 281 kB/s | 47 kB 00:00 2024-01-11T07:04:33.764 INFO:teuthology.orchestra.run.smithi007.stdout:(116/140): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2024-01-11T07:04:33.803 INFO:teuthology.orchestra.run.smithi113.stdout:(123/140): krb5-devel-1.18.2-26.el8_9.x86_64.rp 2.9 MB/s | 562 kB 00:00 2024-01-11T07:04:33.823 INFO:teuthology.orchestra.run.smithi007.stdout:(117/140): python3-numpy-f2py-1.14.3-10.el8.x86 1.0 MB/s | 225 kB 00:00 2024-01-11T07:04:33.870 INFO:teuthology.orchestra.run.smithi113.stdout:(124/140): openssl-devel-1.1.1k-12.el8_9.x86_64 12 MB/s | 2.3 MB 00:00 2024-01-11T07:04:33.932 INFO:teuthology.orchestra.run.smithi007.stdout:(118/140): python3-mako-1.0.6-14.el8.noarch.rpm 940 kB/s | 157 kB 00:00 2024-01-11T07:04:34.032 INFO:teuthology.orchestra.run.smithi007.stdout:(119/140): openblas-threads-0.3.15-6.el8.x86_64 17 MB/s | 5.2 MB 00:00 2024-01-11T07:04:34.047 INFO:teuthology.orchestra.run.smithi113.stdout:(125/140): elfutils-libelf-0.189-3.el8.x86_64.r 1.3 MB/s | 232 kB 00:00 2024-01-11T07:04:34.091 INFO:teuthology.orchestra.run.smithi007.stdout:(120/140): automake-1.16.1-8.el8.noarch.rpm 4.4 MB/s | 714 kB 00:00 2024-01-11T07:04:34.116 INFO:teuthology.orchestra.run.smithi007.stdout:(121/140): jq-1.6-7.el8.x86_64.rpm 2.4 MB/s | 202 kB 00:00 2024-01-11T07:04:34.154 INFO:teuthology.orchestra.run.smithi113.stdout:(126/140): elfutils-libs-0.189-3.el8.x86_64.rpm 2.8 MB/s | 303 kB 00:00 2024-01-11T07:04:34.216 INFO:teuthology.orchestra.run.smithi007.stdout:(122/140): openblas-0.3.15-6.el8.x86_64.rpm 13 MB/s | 5.0 MB 00:00 2024-01-11T07:04:34.246 INFO:teuthology.orchestra.run.smithi007.stdout:(123/140): python3-pytz-2017.2-11.el8.noarch.rp 361 kB/s | 54 kB 00:00 2024-01-11T07:04:34.313 INFO:teuthology.orchestra.run.smithi113.stdout:(127/140): librados2-17.2.7-299.g4bcedb9f.el8.x 6.1 MB/s | 3.5 MB 00:00 2024-01-11T07:04:34.346 INFO:teuthology.orchestra.run.smithi113.stdout:(128/140): libmount-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 237 kB 00:00 2024-01-11T07:04:34.455 INFO:teuthology.orchestra.run.smithi113.stdout:(129/140): libsmartcols-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 179 kB 00:00 2024-01-11T07:04:34.513 INFO:teuthology.orchestra.run.smithi113.stdout:(130/140): libuuid-2.32.1-43.el8.x86_64.rpm 593 kB/s | 99 kB 00:00 2024-01-11T07:04:34.584 INFO:teuthology.orchestra.run.smithi007.stdout:(124/140): python3-scipy-1.0.0-21.module+el8.9. 29 MB/s | 14 MB 00:00 2024-01-11T07:04:34.630 INFO:teuthology.orchestra.run.smithi113.stdout:(131/140): util-linux-user-2.32.1-43.el8.x86_64 881 kB/s | 103 kB 00:00 2024-01-11T07:04:34.650 INFO:teuthology.orchestra.run.smithi007.stdout:(125/140): librados2-17.2.7-299.g4bcedb9f.el8.x 8.0 MB/s | 3.5 MB 00:00 2024-01-11T07:04:34.684 INFO:teuthology.orchestra.run.smithi007.stdout:(126/140): elfutils-libelf-0.189-3.el8.x86_64.r 2.3 MB/s | 232 kB 00:00 2024-01-11T07:04:34.689 INFO:teuthology.orchestra.run.smithi113.stdout:(132/140): util-linux-2.32.1-43.el8.x86_64.rpm 11 MB/s | 2.5 MB 00:00 2024-01-11T07:04:34.781 INFO:teuthology.orchestra.run.smithi113.stdout:(133/140): xfsdump-3.1.8-6.el8.x86_64.rpm 2.2 MB/s | 333 kB 00:00 2024-01-11T07:04:34.801 INFO:teuthology.orchestra.run.smithi007.stdout:(127/140): elfutils-libs-0.189-3.el8.x86_64.rpm 2.0 MB/s | 303 kB 00:00 2024-01-11T07:04:34.818 INFO:teuthology.orchestra.run.smithi007.stdout:(128/140): libmount-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 237 kB 00:00 2024-01-11T07:04:34.873 INFO:teuthology.orchestra.run.smithi113.stdout:(134/140): iproute-6.2.0-5.el8_9.x86_64.rpm 4.5 MB/s | 854 kB 00:00 2024-01-11T07:04:34.906 INFO:teuthology.orchestra.run.smithi113.stdout:(135/140): libblkid-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2024-01-11T07:04:34.976 INFO:teuthology.orchestra.run.smithi007.stdout:(129/140): libsmartcols-2.32.1-43.el8.x86_64.rp 1.0 MB/s | 179 kB 00:00 2024-01-11T07:04:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:(130/140): libuuid-2.32.1-43.el8.x86_64.rpm 494 kB/s | 99 kB 00:00 2024-01-11T07:04:35.032 INFO:teuthology.orchestra.run.smithi113.stdout:(136/140): libfdisk-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2024-01-11T07:04:35.107 INFO:teuthology.orchestra.run.smithi113.stdout:(137/140): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.3 MB/s | 1.1 MB 00:00 2024-01-11T07:04:35.144 INFO:teuthology.orchestra.run.smithi007.stdout:(131/140): util-linux-2.32.1-43.el8.x86_64.rpm 15 MB/s | 2.5 MB 00:00 2024-01-11T07:04:35.169 INFO:teuthology.orchestra.run.smithi007.stdout:(132/140): util-linux-user-2.32.1-43.el8.x86_64 682 kB/s | 103 kB 00:00 2024-01-11T07:04:35.199 INFO:teuthology.orchestra.run.smithi113.stdout:(138/140): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.2 MB/s | 711 kB 00:00 2024-01-11T07:04:35.258 INFO:teuthology.orchestra.run.smithi113.stdout:(139/140): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.8 MB/s | 1.5 MB 00:00 2024-01-11T07:04:35.278 INFO:teuthology.orchestra.run.smithi007.stdout:(133/140): xfsdump-3.1.8-6.el8.x86_64.rpm 2.4 MB/s | 333 kB 00:00 2024-01-11T07:04:35.320 INFO:teuthology.orchestra.run.smithi007.stdout:(134/140): iproute-6.2.0-5.el8_9.x86_64.rpm 5.6 MB/s | 854 kB 00:00 2024-01-11T07:04:35.395 INFO:teuthology.orchestra.run.smithi007.stdout:(135/140): libblkid-2.32.1-43.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-01-11T07:04:35.479 INFO:teuthology.orchestra.run.smithi007.stdout:(136/140): libfdisk-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2024-01-11T07:04:35.546 INFO:teuthology.orchestra.run.smithi007.stdout:(137/140): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.1 MB/s | 1.1 MB 00:00 2024-01-11T07:04:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:(140/140): librbd1-17.2.7-299.g4bcedb9f.el8.x86 2.0 MB/s | 3.6 MB 00:01 2024-01-11T07:04:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-01-11T07:04:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:Total 9.9 MB/s | 212 MB 00:21 2024-01-11T07:04:35.588 INFO:teuthology.orchestra.run.smithi007.stdout:(138/140): openssl-1.1.1k-12.el8_9.x86_64.rpm 6.4 MB/s | 711 kB 00:00 2024-01-11T07:04:35.713 INFO:teuthology.orchestra.run.smithi007.stdout:(139/140): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.8 MB/s | 1.5 MB 00:00 2024-01-11T07:04:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:(140/140): librbd1-17.2.7-299.g4bcedb9f.el8.x86 2.0 MB/s | 3.6 MB 00:01 2024-01-11T07:04:36.083 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-01-11T07:04:36.083 INFO:teuthology.orchestra.run.smithi007.stdout:Total 10 MB/s | 212 MB 00:20 2024-01-11T07:04:36.495 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-01-11T07:04:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-01-11T07:04:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-01-11T07:04:37.084 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-01-11T07:04:37.528 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-01-11T07:04:37.528 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-01-11T07:04:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-01-11T07:04:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-01-11T07:04:38.806 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-01-11T07:04:38.819 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-01-11T07:04:44.056 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-01-11T07:04:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-01-11T07:04:44.205 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-11T07:04:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-11T07:04:44.483 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-11T07:04:44.827 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-11T07:04:44.827 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-11T07:04:44.827 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-11T07:04:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/156 2024-01-11T07:04:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-11T07:04:44.827 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-11T07:04:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 2/156 2024-01-11T07:04:45.031 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-11T07:04:45.037 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-11T07:04:45.095 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-11T07:04:45.102 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 3/156 2024-01-11T07:04:45.474 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/156 2024-01-11T07:04:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/156 2024-01-11T07:04:45.678 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/156 2024-01-11T07:04:45.726 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/156 2024-01-11T07:04:45.875 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/156 2024-01-11T07:04:45.962 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/156 2024-01-11T07:04:46.030 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/156 2024-01-11T07:04:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/156 2024-01-11T07:04:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/156 2024-01-11T07:04:46.271 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/156 2024-01-11T07:04:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-11T07:04:46.495 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-11T07:04:46.821 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-11T07:04:46.870 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/156 2024-01-11T07:04:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:17.2.7-299.g4bcedb9f.el8.x86_64 9/156 2024-01-11T07:04:47.111 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:17.2.7-299.g4bcedb9f.el8.x86_64 9/156 2024-01-11T07:04:47.482 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:17.2.7-299.g4bcedb9f.el8.x86_64 9/156 2024-01-11T07:04:47.505 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:17.2.7-299.g4bcedb9f.el8.x86_64 9/156 2024-01-11T07:04:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:17.2.7-299.g4bcedb9f.el8.x86_64 10/156 2024-01-11T07:04:47.720 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:17.2.7-299.g4bcedb9f.el8.x86_64 10/156 2024-01-11T07:04:47.803 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:17.2.7-299.g4bcedb9f.el8.x86_64 10/156 2024-01-11T07:04:47.881 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:17.2.7-299.g4bcedb9f.el8.x86_64 10/156 2024-01-11T07:04:48.005 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:17.2.7-299.g4bcedb9f.el8.x86_64 11/156 2024-01-11T07:04:48.086 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:17.2.7-299.g4bcedb9f.el8.x86_64 11/156 2024-01-11T07:04:48.112 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:17.2.7-299.g4bcedb9f.el8.x86_64 11/156 2024-01-11T07:04:48.183 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:17.2.7-299.g4bcedb9f.el8.x86_64 11/156 2024-01-11T07:04:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:17.2.7-299.g4bcedb9f.el8.x86_64 12/156 2024-01-11T07:04:48.880 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:17.2.7-299.g4bcedb9f.el8.x86_64 12/156 2024-01-11T07:04:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:17.2.7-299.g4bcedb9f.el8.x86_64 13/156 2024-01-11T07:04:49.020 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:17.2.7-299.g4bcedb9f.el8.x86_64 13/156 2024-01-11T07:04:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:17.2.7-299.g4bcedb9f.el8.x86_64 13/156 2024-01-11T07:04:49.245 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:17.2.7-299.g4bcedb9f.el8.x86_64 13/156 2024-01-11T07:04:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-11T07:04:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-11T07:04:49.519 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-11T07:04:49.626 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/156 2024-01-11T07:04:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/156 2024-01-11T07:04:49.847 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/156 2024-01-11T07:04:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/156 2024-01-11T07:04:50.039 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/156 2024-01-11T07:04:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/156 2024-01-11T07:04:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:17.2.7-299.g4bcedb9f.el8.x86_64 18/156 2024-01-11T07:04:50.259 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/156 2024-01-11T07:04:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:17.2.7-299.g4bcedb9f.el8.x86_64 18/156 2024-01-11T07:04:50.444 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:17.2.7-299.g4bcedb9f.el8.x86_64 18/156 2024-01-11T07:04:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:17.2.7-299.g4bcedb9f.el8.x86_ 19/156 2024-01-11T07:04:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:17.2.7-299.g4bcedb9f.el8.x86_ 19/156 2024-01-11T07:04:50.553 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:17.2.7-299.g4bcedb9f.el8.x86_64 18/156 2024-01-11T07:04:50.727 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:17.2.7-299.g4bcedb9f.el8.x86_ 19/156 2024-01-11T07:04:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/156 2024-01-11T07:04:50.804 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:17.2.7-299.g4bcedb9f.el8.x86_ 19/156 2024-01-11T07:04:50.994 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/156 2024-01-11T07:04:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 21/156 2024-01-11T07:04:51.129 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/156 2024-01-11T07:04:51.237 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/156 2024-01-11T07:04:51.302 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/156 2024-01-11T07:04:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 23/156 2024-01-11T07:04:51.564 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 23/156 2024-01-11T07:04:51.721 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 24/156 2024-01-11T07:04:51.937 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/156 2024-01-11T07:04:52.119 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 22/156 2024-01-11T07:04:52.409 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 24/156 2024-01-11T07:04:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 25/156 2024-01-11T07:04:52.753 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 22/156 2024-01-11T07:04:52.919 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.15-6.el8.x86_64 23/156 2024-01-11T07:04:53.273 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 25/156 2024-01-11T07:04:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.15-6.el8.x86_64 26/156 2024-01-11T07:04:53.614 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 23/156 2024-01-11T07:04:53.916 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/156 2024-01-11T07:04:54.108 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/156 2024-01-11T07:04:54.229 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 26/156 2024-01-11T07:04:54.340 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 26/156 2024-01-11T07:04:54.495 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 27/156 2024-01-11T07:04:54.655 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 27/156 2024-01-11T07:04:54.688 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 28/156 2024-01-11T07:04:54.938 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 29/156 2024-01-11T07:04:54.967 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 28/156 2024-01-11T07:04:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:17.2.7-299.g4bcedb9f.el8.x 30/156 2024-01-11T07:04:55.242 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:17.2.7-299.g4bcedb9f.el8 31/156 2024-01-11T07:04:55.253 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 29/156 2024-01-11T07:04:55.466 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:17.2.7-299.g4bcedb9f.el8.x 30/156 2024-01-11T07:04:55.512 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:17.2.7-299.g4bcedb9f.el8.x86_64 32/156 2024-01-11T07:04:55.604 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:17.2.7-299.g4bcedb9f.el8 31/156 2024-01-11T07:04:55.679 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/156 2024-01-11T07:04:55.924 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:17.2.7-299.g4bcedb9f.el8.x86_64 32/156 2024-01-11T07:04:56.951 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/156 2024-01-11T07:04:57.297 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/156 2024-01-11T07:04:57.610 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/156 2024-01-11T07:04:57.646 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/156 2024-01-11T07:04:57.801 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2024-01-11T07:04:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 34/156 2024-01-11T07:04:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/156 2024-01-11T07:04:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/156 2024-01-11T07:04:59.542 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/156 2024-01-11T07:04:59.598 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/156 2024-01-11T07:04:59.848 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/156 2024-01-11T07:04:59.917 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 37/156 2024-01-11T07:05:00.008 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 39/156 2024-01-11T07:05:00.195 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/156 2024-01-11T07:05:00.276 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 40/156 2024-01-11T07:05:00.403 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 39/156 2024-01-11T07:05:00.510 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 41/156 2024-01-11T07:05:00.668 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 40/156 2024-01-11T07:05:00.711 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:17.2.7-299.g4bcedb9f.el8.x86_64 42/156 2024-01-11T07:05:00.913 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 41/156 2024-01-11T07:05:00.955 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/156 2024-01-11T07:05:01.034 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/156 2024-01-11T07:05:01.089 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:17.2.7-299.g4bcedb9f.el8.x86_64 42/156 2024-01-11T07:05:01.236 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-11T07:05:01.310 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-11T07:05:01.373 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/156 2024-01-11T07:05:01.492 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/156 2024-01-11T07:05:01.538 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/156 2024-01-11T07:05:01.691 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-11T07:05:01.723 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/156 2024-01-11T07:05:01.742 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/156 2024-01-11T07:05:01.895 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-11T07:05:01.996 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/156 2024-01-11T07:05:02.219 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/156 2024-01-11T07:05:02.370 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-11T07:05:02.416 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-11T07:05:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-11T07:05:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-11T07:05:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/156 2024-01-11T07:05:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.1-8.el8.noarch 49/156 2024-01-11T07:05:03.144 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 50/156 2024-01-11T07:05:03.259 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 50/156 2024-01-11T07:05:03.267 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-11T07:05:03.284 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/156 2024-01-11T07:05:03.537 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/156 2024-01-11T07:05:03.538 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-7.el8.x86_64 51/156 2024-01-11T07:05:03.678 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 52/156 2024-01-11T07:05:03.728 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/156 2024-01-11T07:05:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/156 2024-01-11T07:05:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/156 2024-01-11T07:05:04.006 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/156 2024-01-11T07:05:04.169 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/156 2024-01-11T07:05:04.403 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/156 2024-01-11T07:05:04.411 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.7-17.el8.x86_64 55/156 2024-01-11T07:05:04.718 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/156 2024-01-11T07:05:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 56/156 2024-01-11T07:05:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/156 2024-01-11T07:05:05.039 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/156 2024-01-11T07:05:05.124 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/156 2024-01-11T07:05:05.279 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/156 2024-01-11T07:05:05.317 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 59/156 2024-01-11T07:05:05.470 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/156 2024-01-11T07:05:05.619 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 60/156 2024-01-11T07:05:05.736 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/156 2024-01-11T07:05:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 61/156 2024-01-11T07:05:06.116 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/156 2024-01-11T07:05:06.125 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 62/156 2024-01-11T07:05:06.288 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 63/156 2024-01-11T07:05:06.435 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/156 2024-01-11T07:05:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 64/156 2024-01-11T07:05:06.701 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/156 2024-01-11T07:05:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 65/156 2024-01-11T07:05:07.022 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/156 2024-01-11T07:05:07.142 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 66/156 2024-01-11T07:05:07.311 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/156 2024-01-11T07:05:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 67/156 2024-01-11T07:05:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/156 2024-01-11T07:05:07.672 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/156 2024-01-11T07:05:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 69/156 2024-01-11T07:05:07.998 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 65/156 2024-01-11T07:05:08.164 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 70/156 2024-01-11T07:05:08.205 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 66/156 2024-01-11T07:05:08.345 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 71/156 2024-01-11T07:05:08.442 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/156 2024-01-11T07:05:08.500 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 72/156 2024-01-11T07:05:08.664 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/156 2024-01-11T07:05:08.721 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 73/156 2024-01-11T07:05:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 74/156 2024-01-11T07:05:08.968 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/156 2024-01-11T07:05:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 75/156 2024-01-11T07:05:09.236 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.1-8.el8.noarch 70/156 2024-01-11T07:05:09.303 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 76/156 2024-01-11T07:05:09.415 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 71/156 2024-01-11T07:05:09.468 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 77/156 2024-01-11T07:05:09.494 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 71/156 2024-01-11T07:05:09.714 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : jq-1.6-7.el8.x86_64 72/156 2024-01-11T07:05:09.748 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 78/156 2024-01-11T07:05:09.938 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 73/156 2024-01-11T07:05:10.002 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 79/156 2024-01-11T07:05:10.113 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 74/156 2024-01-11T07:05:10.182 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 80/156 2024-01-11T07:05:10.324 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 75/156 2024-01-11T07:05:10.364 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 81/156 2024-01-11T07:05:10.515 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 76/156 2024-01-11T07:05:10.616 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 82/156 2024-01-11T07:05:10.761 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 77/156 2024-01-11T07:05:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 83/156 2024-01-11T07:05:11.015 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 84/156 2024-01-11T07:05:11.045 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 78/156 2024-01-11T07:05:11.180 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 85/156 2024-01-11T07:05:11.288 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 79/156 2024-01-11T07:05:11.350 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 86/156 2024-01-11T07:05:11.528 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 80/156 2024-01-11T07:05:11.595 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 87/156 2024-01-11T07:05:11.759 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 81/156 2024-01-11T07:05:11.805 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 88/156 2024-01-11T07:05:11.949 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 89/156 2024-01-11T07:05:12.007 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 82/156 2024-01-11T07:05:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-2.6-1.el8.noarch 90/156 2024-01-11T07:05:12.277 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 83/156 2024-01-11T07:05:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 91/156 2024-01-11T07:05:12.517 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 84/156 2024-01-11T07:05:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 92/156 2024-01-11T07:05:12.692 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 85/156 2024-01-11T07:05:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 93/156 2024-01-11T07:05:13.098 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 86/156 2024-01-11T07:05:13.510 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 87/156 2024-01-11T07:05:13.750 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 88/156 2024-01-11T07:05:13.953 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 89/156 2024-01-11T07:05:14.048 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 94/156 2024-01-11T07:05:14.240 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-2.6-1.el8.noarch 90/156 2024-01-11T07:05:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 95/156 2024-01-11T07:05:14.513 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 91/156 2024-01-11T07:05:14.690 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 96/156 2024-01-11T07:05:14.762 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 92/156 2024-01-11T07:05:15.030 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 93/156 2024-01-11T07:05:15.042 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 97/156 2024-01-11T07:05:15.337 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 98/156 2024-01-11T07:05:15.539 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 99/156 2024-01-11T07:05:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 100/156 2024-01-11T07:05:16.121 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 94/156 2024-01-11T07:05:16.458 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 95/156 2024-01-11T07:05:16.712 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 96/156 2024-01-11T07:05:17.063 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 97/156 2024-01-11T07:05:17.355 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 98/156 2024-01-11T07:05:17.588 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 99/156 2024-01-11T07:05:17.822 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 100/156 2024-01-11T07:05:19.939 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 101/156 2024-01-11T07:05:20.895 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 101/156 2024-01-11T07:05:21.518 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 101/156 2024-01-11T07:05:21.773 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:17.2.7-299.g4bcedb9f.el8.x86_64 102/156 2024-01-11T07:05:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:17.2.7-299.g4bcedb9f.el8.x86_64 102/156 2024-01-11T07:05:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:17.2.7-299.g4bcedb9f.el8.x86_64 103/156 2024-01-11T07:05:22.407 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 101/156 2024-01-11T07:05:22.679 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 101/156 2024-01-11T07:05:23.236 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 101/156 2024-01-11T07:05:23.566 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:17.2.7-299.g4bcedb9f.el8.x86_64 102/156 2024-01-11T07:05:23.650 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:17.2.7-299.g4bcedb9f.el8.x86_64 102/156 2024-01-11T07:05:23.836 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:17.2.7-299.g4bcedb9f.el8.x86_64 103/156 2024-01-11T07:05:43.107 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:17.2.7-299.g4bcedb9f.el8.x86_64 103/156 2024-01-11T07:05:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-01-11T07:05:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-01-11T07:05:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /dev 2024-01-11T07:05:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /run 2024-01-11T07:05:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-01-11T07:05:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-01-11T07:05:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-01-11T07:05:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-01-11T07:05:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /dev 2024-01-11T07:05:43.108 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:05:43.499 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:17.2.7-299.g4bcedb9f.el8.noarch 104/156 2024-01-11T07:05:43.832 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:17.2.7-299.g4bcedb9f.el8.x86_64 103/156 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /run 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2024-01-11T07:05:43.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:05:44.222 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:17.2.7-299.g4bcedb9f.el8.noarch 104/156 2024-01-11T07:05:44.971 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:17.2.7-299.g4bcedb9f.el8.noarch 104/156 2024-01-11T07:05:45.160 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:17.2.7-299.g4bcedb9f.el8.x86_64 105/156 2024-01-11T07:05:45.382 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:17.2.7-299.g4bcedb9f.el8.x86_64 105/156 2024-01-11T07:05:45.557 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:17.2.7-299.g4bcedb9f.el8.x86_64 106/156 2024-01-11T07:05:45.854 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:17.2.7-299.g4bcedb9f.el8.noarch 104/156 2024-01-11T07:05:45.948 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:17.2.7-299.g4bcedb9f.el8.x86_64 106/156 2024-01-11T07:05:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:17.2.7-299.g4bcedb9f.el8.x86_64 105/156 2024-01-11T07:05:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:17.2.7-299.g4bcedb9f.el8.x86_64 107/156 2024-01-11T07:05:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:17.2.7-299.g4bcedb9f.el8.x86_64 107/156 2024-01-11T07:05:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:17.2.7-299.g4bcedb9f.el8.x86_64 105/156 2024-01-11T07:05:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:17.2.7-299.g4bcedb9f.el8.x86_64 106/156 2024-01-11T07:05:46.946 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:17.2.7-299.g4bcedb9f.el8.x86_64 106/156 2024-01-11T07:05:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:17.2.7-299.g4bcedb9f.el8.x86_64 107/156 2024-01-11T07:05:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:17.2.7-299.g4bcedb9f.el8.x86_64 107/156 2024-01-11T07:05:47.965 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 108/156 2024-01-11T07:05:48.179 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 108/156 2024-01-11T07:05:48.305 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-299.g4bcedb9f.el 109/156 2024-01-11T07:05:48.526 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-299.g4bcedb9f.e 110/156 2024-01-11T07:05:48.788 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-299.g4bcedb9f.el8.noar 111/156 2024-01-11T07:05:49.004 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-299.g4bcedb9f.el8.noar 111/156 2024-01-11T07:05:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 108/156 2024-01-11T07:05:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 108/156 2024-01-11T07:05:49.276 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-299.g4bcedb9f.el8.no 112/156 2024-01-11T07:05:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-299.g4bcedb9f.el 109/156 2024-01-11T07:05:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-299.g4bcedb9f.e 110/156 2024-01-11T07:05:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-299.g4bcedb9f.el8.noar 111/156 2024-01-11T07:05:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-299.g4bcedb9f.el8.noar 111/156 2024-01-11T07:05:50.242 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-299.g4bcedb9f.el8.no 112/156 2024-01-11T07:05:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-299.g4bcedb9f.el8.no 112/156 2024-01-11T07:05:50.456 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-299.g4bce 113/156 2024-01-11T07:05:50.548 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-299.g4bce 113/156 2024-01-11T07:05:50.931 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-299.g4bcedb9f.el8 114/156 2024-01-11T07:05:51.126 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:17.2.7-299.g4bcedb9f.el8.x86_64 115/156 2024-01-11T07:05:51.162 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:17.2.7-299.g4bcedb9f.el8.x86_64 115/156 2024-01-11T07:05:51.339 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:17.2.7-299.g4bcedb9f.el8.noarch 116/156 2024-01-11T07:05:51.350 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-299.g4bcedb9f.el8.noarch 116/156 2024-01-11T07:05:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-299.g4bcedb9f.el8.no 112/156 2024-01-11T07:05:51.636 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-299.g4bce 113/156 2024-01-11T07:05:51.746 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-299.g4bce 113/156 2024-01-11T07:05:52.104 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-299.g4bcedb9f.el8 114/156 2024-01-11T07:05:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:17.2.7-299.g4bcedb9f.el8.x86_64 115/156 2024-01-11T07:05:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:17.2.7-299.g4bcedb9f.el8.x86_64 115/156 2024-01-11T07:05:52.461 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:17.2.7-299.g4bcedb9f.el8.noarch 116/156 2024-01-11T07:05:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-299.g4bcedb9f.el8.noarch 116/156 2024-01-11T07:05:52.974 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:17.2.7-299.g4bcedb9f.el8.x86_64 117/156 2024-01-11T07:05:53.192 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 118/156 2024-01-11T07:05:53.282 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 118/156 2024-01-11T07:05:53.456 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-299.g4bcedb 119/156 2024-01-11T07:05:53.498 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-299.g4bcedb 119/156 2024-01-11T07:05:53.664 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 120/156 2024-01-11T07:05:54.010 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 120/156 2024-01-11T07:05:54.180 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 121/156 2024-01-11T07:05:54.209 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:17.2.7-299.g4bcedb9f.el8.x86_64 117/156 2024-01-11T07:05:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 118/156 2024-01-11T07:05:54.463 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 118/156 2024-01-11T07:05:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-299.g4bcedb 119/156 2024-01-11T07:05:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-299.g4bcedb 119/156 2024-01-11T07:05:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 120/156 2024-01-11T07:05:55.231 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 120/156 2024-01-11T07:05:55.377 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 121/156 2024-01-11T07:05:59.074 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 121/156 2024-01-11T07:05:59.406 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:17.2.7-299.g4bcedb9f.el8.x86_64 122/156 2024-01-11T07:05:59.620 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-25.el8.x86_64 123/156 2024-01-11T07:06:00.308 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 123/156 2024-01-11T07:06:00.625 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2024-01-11T07:06:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 121/156 2024-01-11T07:06:00.871 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 125/156 2024-01-11T07:06:01.043 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 126/156 2024-01-11T07:06:01.273 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 127/156 2024-01-11T07:06:01.473 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 128/156 2024-01-11T07:06:01.693 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:17.2.7-299.g4bcedb9f.el8.x86_6 129/156 2024-01-11T07:06:01.857 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:17.2.7-299.g4bcedb9f.el8.x86_64 122/156 2024-01-11T07:06:02.065 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 130/156 2024-01-11T07:06:02.199 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 123/156 2024-01-11T07:06:02.429 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 131/156 2024-01-11T07:06:02.452 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 124/156 2024-01-11T07:06:02.682 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 132/156 2024-01-11T07:06:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 125/156 2024-01-11T07:06:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:17.2.7-299.g4bcedb9f.el8.noarch 133/156 2024-01-11T07:06:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-25.el8.x86_64 126/156 2024-01-11T07:06:03.085 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 126/156 2024-01-11T07:06:03.110 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 134/156 2024-01-11T07:06:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 127/156 2024-01-11T07:06:03.435 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 135/156 2024-01-11T07:06:03.472 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 128/156 2024-01-11T07:06:03.652 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:17.2.7-299.g4bcedb9f.el8.x86_6 129/156 2024-01-11T07:06:03.679 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 136/156 2024-01-11T07:06:03.886 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 137/156 2024-01-11T07:06:03.895 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 137/156 2024-01-11T07:06:04.038 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 130/156 2024-01-11T07:06:04.114 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 138/156 2024-01-11T07:06:04.319 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.1-9.el8.x86_64 139/156 2024-01-11T07:06:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 131/156 2024-01-11T07:06:04.396 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 139/156 2024-01-11T07:06:04.606 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 132/156 2024-01-11T07:06:04.644 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.0.4-10.el8.x86_64 140/156 2024-01-11T07:06:04.657 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 140/156 2024-01-11T07:06:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:17.2.7-299.g4bcedb9f.el8.noarch 133/156 2024-01-11T07:06:04.851 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 141/156 2024-01-11T07:06:05.029 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 142/156 2024-01-11T07:06:05.036 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 134/156 2024-01-11T07:06:05.192 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 143/156 2024-01-11T07:06:05.192 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-11T07:06:05.323 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 135/156 2024-01-11T07:06:05.480 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-11T07:06:05.480 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-11T07:06:05.634 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 136/156 2024-01-11T07:06:05.714 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-11T07:06:05.714 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-11T07:06:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.1-9.el8.x86_64 137/156 2024-01-11T07:06:05.879 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 137/156 2024-01-11T07:06:05.991 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-11T07:06:06.092 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.0.4-10.el8.x86_64 138/156 2024-01-11T07:06:06.108 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 138/156 2024-01-11T07:06:06.221 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 147/156 2024-01-11T07:06:06.259 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 139/156 2024-01-11T07:06:06.270 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 139/156 2024-01-11T07:06:06.387 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 148/156 2024-01-11T07:06:06.395 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/156 2024-01-11T07:06:06.595 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 141/156 2024-01-11T07:06:06.646 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 149/156 2024-01-11T07:06:06.646 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-11T07:06:06.778 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 142/156 2024-01-11T07:06:06.896 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-11T07:06:06.897 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 151/156 2024-01-11T07:06:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 143/156 2024-01-11T07:06:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-11T07:06:07.130 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 151/156 2024-01-11T07:06:07.130 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-11T07:06:07.162 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 144/156 2024-01-11T07:06:07.162 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-11T07:06:07.380 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-11T07:06:07.380 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-11T07:06:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 145/156 2024-01-11T07:06:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-11T07:06:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/156 2024-01-11T07:06:07.621 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-11T07:06:07.621 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-11T07:06:07.811 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 147/156 2024-01-11T07:06:07.855 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-11T07:06:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 148/156 2024-01-11T07:06:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 155/156 2024-01-11T07:06:08.047 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-11T07:06:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 149/156 2024-01-11T07:06:08.202 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-11T07:06:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-11T07:06:08.471 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 151/156 2024-01-11T07:06:08.720 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 151/156 2024-01-11T07:06:08.720 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-11T07:06:08.890 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:17.2.7-299.g4bcedb9f.el8.x86_64 1/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:17.2.7-299.g4bcedb9f.el8.x86_64 2/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 3/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 4/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-299.g4bcedb 5/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:17.2.7-299.g4bcedb9f.el8.x86_64 6/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:17.2.7-299.g4bcedb9f.el8.x86_64 7/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:17.2.7-299.g4bcedb9f.el8.x86_64 8/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:17.2.7-299.g4bcedb9f.el8.x86_64 9/156 2024-01-11T07:06:08.891 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 10/156 2024-01-11T07:06:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:17.2.7-299.g4bcedb9f.el8.x86_64 11/156 2024-01-11T07:06:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:17.2.7-299.g4bcedb9f.el8.x86_64 12/156 2024-01-11T07:06:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 13/156 2024-01-11T07:06:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:17.2.7-299.g4bcedb9f.el8.x86_6 14/156 2024-01-11T07:06:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:17.2.7-299.g4bcedb9f.el8.x86_64 15/156 2024-01-11T07:06:08.892 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:17.2.7-299.g4bcedb9f.el8.x86_64 16/156 2024-01-11T07:06:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:17.2.7-299.g4bcedb9f.el8.x86_64 17/156 2024-01-11T07:06:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:17.2.7-299.g4bcedb9f.el8.x86_ 18/156 2024-01-11T07:06:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:17.2.7-299.g4bcedb9f.el8.x86_64 19/156 2024-01-11T07:06:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:17.2.7-299.g4bcedb9f.el8 20/156 2024-01-11T07:06:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:17.2.7-299.g4bcedb9f.el8.x 21/156 2024-01-11T07:06:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:17.2.7-299.g4bcedb9f.el8.x86_64 22/156 2024-01-11T07:06:08.893 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:17.2.7-299.g4bcedb9f.el8.x86_64 23/156 2024-01-11T07:06:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 24/156 2024-01-11T07:06:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 25/156 2024-01-11T07:06:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 26/156 2024-01-11T07:06:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 27/156 2024-01-11T07:06:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 28/156 2024-01-11T07:06:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-299.g4bcedb9f.e 29/156 2024-01-11T07:06:08.894 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-299.g4bcedb9f.el8.noar 30/156 2024-01-11T07:06:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-299.g4bcedb9f.el8.no 31/156 2024-01-11T07:06:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-299.g4bce 32/156 2024-01-11T07:06:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-299.g4bcedb9f.el8 33/156 2024-01-11T07:06:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:17.2.7-299.g4bcedb9f.el8.noarch 34/156 2024-01-11T07:06:08.914 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-299.g4bcedb9f.el 35/156 2024-01-11T07:06:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:17.2.7-299.g4bcedb9f.el8.noarch 36/156 2024-01-11T07:06:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 37/156 2024-01-11T07:06:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:17.2.7-299.g4bcedb9f.el8.noarch 38/156 2024-01-11T07:06:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/156 2024-01-11T07:06:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/156 2024-01-11T07:06:08.915 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/156 2024-01-11T07:06:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/156 2024-01-11T07:06:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/156 2024-01-11T07:06:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/156 2024-01-11T07:06:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/156 2024-01-11T07:06:08.916 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/156 2024-01-11T07:06:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/156 2024-01-11T07:06:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/156 2024-01-11T07:06:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/156 2024-01-11T07:06:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/156 2024-01-11T07:06:08.917 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/156 2024-01-11T07:06:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/156 2024-01-11T07:06:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/156 2024-01-11T07:06:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/156 2024-01-11T07:06:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/156 2024-01-11T07:06:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/156 2024-01-11T07:06:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/156 2024-01-11T07:06:08.918 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/156 2024-01-11T07:06:08.919 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/156 2024-01-11T07:06:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/156 2024-01-11T07:06:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/156 2024-01-11T07:06:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/156 2024-01-11T07:06:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/156 2024-01-11T07:06:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/156 2024-01-11T07:06:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/156 2024-01-11T07:06:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/156 2024-01-11T07:06:08.920 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/156 2024-01-11T07:06:08.921 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : jq-1.6-7.el8.x86_64 95/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 96/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 97/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 98/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 99/156 2024-01-11T07:06:08.922 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 100/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 101/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 102/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 103/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 104/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 105/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 106/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 108/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 109/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 110/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 111/156 2024-01-11T07:06:08.923 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 112/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 113/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 114/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 115/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 116/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 117/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 118/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 119/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 121/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 122/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 123/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 124/156 2024-01-11T07:06:08.924 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:17.2.7-299.g4bcedb9f.el8.x86_64 125/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 126/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:17.2.7-299.g4bcedb9f.el8.x86_64 127/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 128/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 129/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 130/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 131/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 132/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 133/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 134/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 135/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 136/156 2024-01-11T07:06:08.925 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 137/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 138/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 139/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 140/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 141/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 142/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 143/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 144/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 145/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 146/156 2024-01-11T07:06:08.926 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 147/156 2024-01-11T07:06:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 148/156 2024-01-11T07:06:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 149/156 2024-01-11T07:06:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-11T07:06:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 151/156 2024-01-11T07:06:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 152/156 2024-01-11T07:06:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 153/156 2024-01-11T07:06:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 154/156 2024-01-11T07:06:08.927 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 155/156 2024-01-11T07:06:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 152/156 2024-01-11T07:06:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-11T07:06:09.219 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 153/156 2024-01-11T07:06:09.220 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-11T07:06:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 154/156 2024-01-11T07:06:09.603 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 155/156 2024-01-11T07:06:09.603 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-11T07:06:10.563 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-11T07:06:10.563 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:17.2.7-299.g4bcedb9f.el8.x86_64 1/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:17.2.7-299.g4bcedb9f.el8.x86_64 2/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 3/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 4/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-299.g4bcedb 5/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:17.2.7-299.g4bcedb9f.el8.x86_64 6/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:17.2.7-299.g4bcedb9f.el8.x86_64 7/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:17.2.7-299.g4bcedb9f.el8.x86_64 8/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:17.2.7-299.g4bcedb9f.el8.x86_64 9/156 2024-01-11T07:06:10.564 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 10/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:17.2.7-299.g4bcedb9f.el8.x86_64 11/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:17.2.7-299.g4bcedb9f.el8.x86_64 12/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 13/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:17.2.7-299.g4bcedb9f.el8.x86_6 14/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:17.2.7-299.g4bcedb9f.el8.x86_64 15/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:17.2.7-299.g4bcedb9f.el8.x86_64 16/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:17.2.7-299.g4bcedb9f.el8.x86_64 17/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:17.2.7-299.g4bcedb9f.el8.x86_ 18/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:17.2.7-299.g4bcedb9f.el8.x86_64 19/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:17.2.7-299.g4bcedb9f.el8 20/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:17.2.7-299.g4bcedb9f.el8.x 21/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:17.2.7-299.g4bcedb9f.el8.x86_64 22/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:17.2.7-299.g4bcedb9f.el8.x86_64 23/156 2024-01-11T07:06:10.565 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 24/156 2024-01-11T07:06:10.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 25/156 2024-01-11T07:06:10.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 26/156 2024-01-11T07:06:10.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 27/156 2024-01-11T07:06:10.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 28/156 2024-01-11T07:06:10.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-299.g4bcedb9f.e 29/156 2024-01-11T07:06:10.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-299.g4bcedb9f.el8.noar 30/156 2024-01-11T07:06:10.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-299.g4bcedb9f.el8.no 31/156 2024-01-11T07:06:10.566 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-299.g4bce 32/156 2024-01-11T07:06:10.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-299.g4bcedb9f.el8 33/156 2024-01-11T07:06:10.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:17.2.7-299.g4bcedb9f.el8.noarch 34/156 2024-01-11T07:06:10.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-299.g4bcedb9f.el 35/156 2024-01-11T07:06:10.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:17.2.7-299.g4bcedb9f.el8.noarch 36/156 2024-01-11T07:06:10.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 37/156 2024-01-11T07:06:10.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:17.2.7-299.g4bcedb9f.el8.noarch 38/156 2024-01-11T07:06:10.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/156 2024-01-11T07:06:10.567 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/156 2024-01-11T07:06:10.568 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/156 2024-01-11T07:06:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/156 2024-01-11T07:06:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 92/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 93/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 94/156 2024-01-11T07:06:10.571 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 95/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 96/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 97/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 98/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/156 2024-01-11T07:06:10.572 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 110/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 111/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 113/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 115/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 117/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 118/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 119/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 120/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.1-8.el8.noarch 121/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-7.el8.x86_64 122/156 2024-01-11T07:06:10.573 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 123/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 124/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:17.2.7-299.g4bcedb9f.el8.x86_64 125/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 126/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:17.2.7-299.g4bcedb9f.el8.x86_64 127/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 128/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 129/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 130/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 131/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 132/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 133/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 134/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 135/156 2024-01-11T07:06:10.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 136/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 137/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 138/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 139/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 140/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 141/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 142/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 143/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 144/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 145/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 146/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 147/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 148/156 2024-01-11T07:06:10.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 149/156 2024-01-11T07:06:10.576 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 150/156 2024-01-11T07:06:10.576 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 151/156 2024-01-11T07:06:10.576 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 152/156 2024-01-11T07:06:10.576 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 153/156 2024-01-11T07:06:10.576 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 154/156 2024-01-11T07:06:10.576 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 155/156 2024-01-11T07:06:26.280 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-11T07:06:26.280 INFO:teuthology.orchestra.run.smithi113.stdout:Installed products updated. 2024-01-11T07:06:27.088 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 156/156 2024-01-11T07:06:27.089 INFO:teuthology.orchestra.run.smithi007.stdout:Installed products updated. 2024-01-11T07:06:27.782 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:27.782 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-01-11T07:06:27.782 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.783 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: xfsdump-3.1.8-6.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.1-8.el8.noarch 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.0.4-10.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.784 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.785 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.1-9.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: jq-1.6-7.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-11T07:06:27.786 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-11T07:06:27.787 INFO:teuthology.orchestra.run.smithi113.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-11T07:06:27.788 INFO:teuthology.orchestra.run.smithi113.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-11T07:06:27.789 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-11T07:06:27.790 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-2.6-1.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.791 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-11T07:06:27.792 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-11T07:06:27.793 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-11T07:06:27.794 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-11T07:06:27.794 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:27.794 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-01-11T07:06:27.829 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:27.829 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-01-11T07:06:27.829 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.830 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump-3.1.8-6.el8.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.1-8.el8.noarch 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.0.4-10.el8.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.831 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.832 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:17.2.7-299.g4bcedb9f.el8.noarch 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.1-9.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-7.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-11T07:06:27.833 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.834 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-11T07:06:27.835 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-11T07:06:27.836 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-11T07:06:27.836 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-11T07:06:27.836 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-11T07:06:27.836 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-11T07:06:27.836 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-11T07:06:27.836 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.837 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.837 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.837 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-11T07:06:27.837 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-11T07:06:27.837 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-11T07:06:27.837 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-11T07:06:27.837 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-11T07:06:27.838 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-11T07:06:27.838 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-11T07:06:27.838 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-11T07:06:27.838 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-11T07:06:27.838 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-11T07:06:27.838 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-11T07:06:27.839 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-11T07:06:27.839 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-11T07:06:27.839 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-11T07:06:27.839 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-11T07:06:27.839 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-11T07:06:27.840 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-11T07:06:27.840 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-11T07:06:27.840 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-11T07:06:27.840 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-11T07:06:27.840 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-11T07:06:27.840 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-11T07:06:27.841 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-2.6-1.el8.noarch 2024-01-11T07:06:27.841 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-11T07:06:27.841 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-11T07:06:27.841 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-11T07:06:27.841 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-11T07:06:27.841 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-11T07:06:27.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-11T07:06:27.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-11T07:06:27.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-11T07:06:27.843 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.843 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-11T07:06:27.843 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-11T07:06:27.843 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-11T07:06:27.843 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-11T07:06:27.843 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-11T07:06:27.843 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-11T07:06:27.844 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-11T07:06:27.844 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-11T07:06:27.844 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-11T07:06:27.844 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-11T07:06:27.844 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-11T07:06:27.844 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-11T07:06:27.845 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-11T07:06:27.845 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.845 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.845 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:17.2.7-299.g4bcedb9f.el8.x86_64 2024-01-11T07:06:27.845 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-11T07:06:27.845 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-11T07:06:27.846 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-11T07:06:27.846 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-11T07:06:27.846 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-11T07:06:27.847 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:27.847 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-01-11T07:06:28.038 DEBUG:teuthology.parallel:result is None 2024-01-11T07:06:28.255 DEBUG:teuthology.parallel:result is None 2024-01-11T07:06:28.256 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4bcedb9fc11bed1fd6fd0c4fd59187401f309680 2024-01-11T07:06:28.584 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-11T07:06:28.614 INFO:teuthology.orchestra.run.smithi007.stdout:17.2.7-299.g4bcedb9f.el8 2024-01-11T07:06:28.615 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-299.g4bcedb9f.el8 2024-01-11T07:06:28.615 INFO:teuthology.task.install:The correct ceph version 17.2.7-299.g4bcedb9f is installed. 2024-01-11T07:06:28.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4bcedb9fc11bed1fd6fd0c4fd59187401f309680 2024-01-11T07:06:28.764 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-11T07:06:28.790 INFO:teuthology.orchestra.run.smithi113.stdout:17.2.7-299.g4bcedb9f.el8 2024-01-11T07:06:28.790 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-299.g4bcedb9f.el8 2024-01-11T07:06:28.790 INFO:teuthology.task.install:The correct ceph version 17.2.7-299.g4bcedb9f is installed. 2024-01-11T07:06:28.792 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-11T07:06:28.792 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:28.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-11T07:06:28.845 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:28.845 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-11T07:06:28.887 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-11T07:06:28.887 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:28.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-01-11T07:06:28.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-11T07:06:28.997 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:28.998 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-01-11T07:06:29.025 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-11T07:06:29.092 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-11T07:06:29.092 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:29.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-11T07:06:29.127 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-11T07:06:29.202 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:29.202 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-11T07:06:29.228 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-11T07:06:29.295 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-11T07:06:29.295 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:29.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-01-11T07:06:29.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-11T07:06:29.404 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:29.404 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-01-11T07:06:29.431 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-11T07:06:29.498 INFO:teuthology.run_tasks:Running task ceph... 2024-01-11T07:06:29.560 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-11T07:06:29.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-01-11T07:06:29.563 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2024-01-11T07:06:29.595 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-11T07:06:29.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-11T07:06:29.638 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-11T07:06:29.670 INFO:tasks.ceph:Creating extra log directories... 2024-01-11T07:06:29.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-11T07:06:29.713 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-11T07:06:29.752 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-11T07:06:29.752 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-11T07:06:29.752 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-11T07:06:29.752 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:29.752 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-11T07:06:29.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-11T07:06:29.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-11T07:06:29.969 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:29.969 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-11T07:06:29.986 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-11T07:06:30.101 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-11T07:06:30.133 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-11T07:06:30.133 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-11T07:06:30.134 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480', 'branch': 'wip-vshankar-testing1-quincy-2024-01-10-2010-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7512480', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing1-quincy-2024-01-10-2010-quincy'}, 'ceph': {'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing1-quincy-2024-01-10-2010-quincy', 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 419, 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing1-quincy-2024-01-10-2010-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_4bcedb9fc11bed1fd6fd0c4fd59187401f309680/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGz211efhD5EafEB9aDd+x9Gp4uo/MSCiUXSAMaCF7LXcrpy+Q65Jc7jyCCWbGzGhr/661Jg0zmzjivmOuURb2A=', 'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInZZZqUlAa+c4h9rdzlNKl38WHPrfAS2CrZ06y59npUesOmvftW9ucInWPMoRKsDGYZxqPe4Gc529D0iXBIUx8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '4bcedb9fc11bed1fd6fd0c4fd59187401f309680', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'cd45576300487d997e5a85abed65500b9f5d143b', 'timestamp': '2024-01-10_19:20:36', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-11T07:06:30.134 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-11T07:06:30.137 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-11T07:06:30.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-01-11T07:06:30.199 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2024-01-11T07:06:30.252 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:30.252 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-01-11T07:06:30.293 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-11T07:06:30.293 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-01-11T07:06:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-11T07:06:30.355 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T07:06:30.355 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 169528 Links: 1 2024-01-11T07:06:30.355 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T07:06:30.355 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T07:06:30.356 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-11 07:06:10.104017422 +0000 2024-01-11T07:06:30.356 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-11 07:02:59.792024247 +0000 2024-01-11T07:06:30.356 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-11 07:02:59.792024247 +0000 2024-01-11T07:06:30.356 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-01-11T07:06:30.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-11T07:06:30.428 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-11T07:06:30.428 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-11T07:06:30.428 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000130405 s, 3.9 MB/s 2024-01-11T07:06:30.430 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-11T07:06:30.494 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-01-11T07:06:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-11T07:06:30.557 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T07:06:30.557 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 162687 Links: 1 2024-01-11T07:06:30.558 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T07:06:30.558 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T07:06:30.558 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-11 07:06:10.104017422 +0000 2024-01-11T07:06:30.558 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-11 07:03:00.162018634 +0000 2024-01-11T07:06:30.558 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-11 07:03:00.162018634 +0000 2024-01-11T07:06:30.558 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-01-11T07:06:30.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-11T07:06:30.630 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-11T07:06:30.630 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-11T07:06:30.630 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000135567 s, 3.8 MB/s 2024-01-11T07:06:30.631 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-11T07:06:30.694 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-01-11T07:06:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-11T07:06:30.756 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T07:06:30.757 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 165874 Links: 1 2024-01-11T07:06:30.757 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T07:06:30.757 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T07:06:30.757 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-11 07:06:10.104017422 +0000 2024-01-11T07:06:30.757 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-11 07:03:00.535012975 +0000 2024-01-11T07:06:30.757 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-11 07:03:00.535012975 +0000 2024-01-11T07:06:30.757 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-01-11T07:06:30.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-11T07:06:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-11T07:06:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-11T07:06:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00013866 s, 3.7 MB/s 2024-01-11T07:06:30.830 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-11T07:06:30.893 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-01-11T07:06:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-11T07:06:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T07:06:30.956 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 168696 Links: 1 2024-01-11T07:06:30.956 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T07:06:30.956 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T07:06:30.956 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-01-11 07:06:10.104017422 +0000 2024-01-11T07:06:30.956 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-01-11 07:03:00.904007377 +0000 2024-01-11T07:06:30.956 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-01-11 07:03:00.904007377 +0000 2024-01-11T07:06:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-01-11T07:06:30.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-11T07:06:31.029 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-01-11T07:06:31.029 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-01-11T07:06:31.029 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000128856 s, 4.0 MB/s 2024-01-11T07:06:31.031 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-11T07:06:31.094 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:31.095 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:31.095 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-01-11T07:06:31.114 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-11T07:06:31.114 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-01-11T07:06:31.174 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-11T07:06:31.174 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T07:06:31.174 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 6h/6d Inode: 165815 Links: 1 2024-01-11T07:06:31.174 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T07:06:31.174 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T07:06:31.174 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-01-11 07:06:08.556532406 +0000 2024-01-11T07:06:31.174 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-01-11 07:02:25.171093664 +0000 2024-01-11T07:06:31.174 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-01-11 07:02:25.171093664 +0000 2024-01-11T07:06:31.175 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-01-11T07:06:31.175 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-11T07:06:31.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-01-11T07:06:31.244 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-01-11T07:06:31.244 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.00026417 s, 1.9 MB/s 2024-01-11T07:06:31.245 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-11T07:06:31.304 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-01-11T07:06:31.363 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-11T07:06:31.364 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T07:06:31.364 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 6h/6d Inode: 171182 Links: 1 2024-01-11T07:06:31.364 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T07:06:31.364 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T07:06:31.364 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-01-11 07:06:08.556532406 +0000 2024-01-11T07:06:31.364 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-01-11 07:02:25.513086366 +0000 2024-01-11T07:06:31.364 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-01-11 07:02:25.513086366 +0000 2024-01-11T07:06:31.364 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-01-11T07:06:31.365 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-11T07:06:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-01-11T07:06:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-01-11T07:06:31.431 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000239164 s, 2.1 MB/s 2024-01-11T07:06:31.432 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-11T07:06:31.491 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-01-11T07:06:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-11T07:06:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T07:06:31.551 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 6h/6d Inode: 167612 Links: 1 2024-01-11T07:06:31.551 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T07:06:31.551 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T07:06:31.551 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-01-11 07:06:08.556532406 +0000 2024-01-11T07:06:31.551 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-01-11 07:02:25.849079210 +0000 2024-01-11T07:06:31.551 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-01-11 07:02:25.849079210 +0000 2024-01-11T07:06:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-01-11T07:06:31.552 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-11T07:06:31.615 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-01-11T07:06:31.616 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-01-11T07:06:31.616 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000222231 s, 2.3 MB/s 2024-01-11T07:06:31.617 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-11T07:06:31.675 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 6h/6d Inode: 171407 Links: 1 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-01-11 07:06:08.556532406 +0000 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-01-11 07:02:26.226071196 +0000 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-01-11 07:02:26.226071196 +0000 2024-01-11T07:06:31.733 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-01-11T07:06:31.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-11T07:06:31.799 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-01-11T07:06:31.799 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-01-11T07:06:31.799 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000242619 s, 2.1 MB/s 2024-01-11T07:06:31.800 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-11T07:06:31.858 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:31.858 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-11T07:06:31.858 INFO:tasks.ceph:Generating config... 2024-01-11T07:06:31.859 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client] debug client = 20 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client] debug ms = 1 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client] pid file = 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client.0] debug client = 20 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client.1] debug client = 20 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2024-01-11T07:06:31.860 INFO:tasks.ceph:[client.1] debug objecter = 20 2024-01-11T07:06:31.860 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-01-11T07:06:31.860 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-11T07:06:31.861 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-11T07:06:31.861 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-11T07:06:31.861 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-11T07:06:31.861 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-11T07:06:31.861 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-11T07:06:31.861 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-11T07:06:31.861 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2024-01-11T07:06:31.862 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2024-01-11T07:06:31.862 INFO:tasks.ceph:[osd] bdev async discard = True 2024-01-11T07:06:31.862 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-01-11T07:06:31.862 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] osd mon report interval = 5 2024-01-11T07:06:31.863 INFO:tasks.ceph:[osd] osd objectstore = memstore 2024-01-11T07:06:31.864 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-11T07:06:31.864 INFO:tasks.ceph:Setting up mon.a... 2024-01-11T07:06:31.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-11T07:06:31.922 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-01-11T07:06:31.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-11T07:06:31.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-11T07:06:32.015 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.113'), ('mon.c', '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]')] 2024-01-11T07:06:32.015 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-11T07:06:32.016 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:32.016 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-11T07:06:32.077 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.113 --addv c '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:32.171 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:32.171 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 9c0c5e15-bc38-4016-89e4-891c3586efbb 2024-01-11T07:06:32.171 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = octopus 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 9c0c5e15-bc38-4016-89e4-891c3586efbb 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-01-11T07:06:32.172993+0000 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-01-11T07:06:32.172993+0000 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 15 (octopus) 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] mon.c 2024-01-11T07:06:32.172 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-11T07:06:32.173 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-11T07:06:32.193 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9c0c5e15-bc38-4016-89e4-891c3586efbb... 2024-01-11T07:06:32.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-11T07:06:32.236 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-01-11T07:06:32.283 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.284 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.285 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.7,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = 9c0c5e15-bc38-4016-89e4-891c3586efbb 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.286 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.287 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore allocator = bitmap 2024-01-11T07:06:32.288 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 1/20 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 1/20 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 4/10 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: memstore device bytes = 200000000 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 1.0 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: osd mon report interval = 5 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = memstore 2024-01-11T07:06:32.289 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-01-11T07:06:32.290 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-01-11T07:06:32.291 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull ratio = 0.6 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.7 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.6 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-01-11T07:06:32.292 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-01-11T07:06:32.293 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug objectcacher = 20 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug objecter = 20 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout:[client.1] 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug objectcacher = 20 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug objecter = 20 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-01-11T07:06:32.294 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-01-11T07:06:32.295 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-01-11T07:06:32.295 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2024-01-11T07:06:32.295 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-01-11T07:06:32.295 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2024-01-11T07:06:32.295 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-01-11T07:06:32.295 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-01-11T07:06:32.295 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-01-11T07:06:32.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.297 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-01-11T07:06:32.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 9c0c5e15-bc38-4016-89e4-891c3586efbb 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-01-11T07:06:32.299 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.300 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-01-11T07:06:32.301 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: memstore device bytes = 200000000 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 1.0 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: osd mon report interval = 5 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = memstore 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-01-11T07:06:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-01-11T07:06:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-11T07:06:32.304 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull ratio = 0.6 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.7 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.6 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-11T07:06:32.305 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-01-11T07:06:32.306 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout: debug objectcacher = 20 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout: debug objecter = 20 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout: debug objectcacher = 20 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout: debug objecter = 20 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-01-11T07:06:32.307 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-01-11T07:06:32.308 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-01-11T07:06:32.308 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-01-11T07:06:32.308 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-01-11T07:06:32.308 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-01-11T07:06:32.308 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-01-11T07:06:32.308 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-01-11T07:06:32.308 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-01-11T07:06:32.309 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-11T07:06:32.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-11T07:06:32.499 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-11T07:06:32.499 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:32.499 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-11T07:06:32.521 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:32.521 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-11T07:06:32.582 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:06:32.582 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:32.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-11T07:06:32.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-11T07:06:32.732 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:32.733 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:32.788 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:06:32.788 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:32.788 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-11T07:06:32.788 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-11T07:06:32.828 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:32.828 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:32.886 INFO:tasks.ceph:Setting up mon nodes... 2024-01-11T07:06:32.886 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-11T07:06:32.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-11T07:06:32.950 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-11T07:06:32.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-11T07:06:33.016 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-11T07:06:33.019 INFO:tasks.ceph:Setting up mds nodes... 2024-01-11T07:06:33.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-11T07:06:33.084 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-11T07:06:33.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-11T07:06:33.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-11T07:06:33.223 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-11T07:06:33.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-11T07:06:33.257 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-11T07:06:33.308 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-11T07:06:33.310 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-11T07:06:33.337 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-11T07:06:33.427 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-11T07:06:33.429 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-11T07:06:33.455 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-11T07:06:33.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-11T07:06:33.506 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-11T07:06:33.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-11T07:06:33.569 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-11T07:06:33.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-11T07:06:33.624 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-11T07:06:33.637 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-11T07:06:33.718 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-11T07:06:33.731 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-11T07:06:33.731 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-11T07:06:33.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-11T07:06:33.766 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:33.767 INFO:tasks.ceph:role: osd.0 2024-01-11T07:06:33.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:06:33.767 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-11T07:06:33.841 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-11T07:06:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-11T07:06:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-11T07:06:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-11T07:06:33.841 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-11T07:06:33.841 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-01-11T07:06:33.841 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-11T07:06:33.842 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-11T07:06:33.842 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-11T07:06:33.842 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-11T07:06:33.863 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-01-11T07:06:33.900 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-01-11T07:06:33.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-11T07:06:33.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-11T07:06:34.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-11T07:06:34.104 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:34.104 INFO:tasks.ceph:role: osd.1 2024-01-11T07:06:34.104 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:06:34.104 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-11T07:06:34.178 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-11T07:06:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-11T07:06:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-11T07:06:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-11T07:06:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-11T07:06:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-01-11T07:06:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-11T07:06:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-11T07:06:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-11T07:06:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-11T07:06:34.204 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-01-11T07:06:34.204 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-01-11T07:06:34.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-11T07:06:34.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-11T07:06:34.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-11T07:06:34.405 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:34.407 INFO:tasks.ceph:role: osd.2 2024-01-11T07:06:34.407 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:06:34.407 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-11T07:06:34.479 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-11T07:06:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-11T07:06:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-11T07:06:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-11T07:06:34.479 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-11T07:06:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-01-11T07:06:34.479 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-11T07:06:34.480 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-11T07:06:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-11T07:06:34.480 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-11T07:06:34.501 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-01-11T07:06:34.503 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-01-11T07:06:34.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-11T07:06:34.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-11T07:06:34.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-11T07:06:34.701 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:34.701 INFO:tasks.ceph:role: osd.3 2024-01-11T07:06:34.702 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:06:34.702 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-11T07:06:34.775 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-11T07:06:34.798 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-01-11T07:06:34.800 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-01-11T07:06:34.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-11T07:06:34.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-11T07:06:34.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.020+0000 7fb520bf1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-11T07:06:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.020+0000 7fb520bf1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-11T07:06:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.020+0000 7fb520bf1540 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2024-01-11T07:06:35.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-11T07:06:35.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:35.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.155+0000 7f496b302540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-11T07:06:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.155+0000 7f496b302540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-11T07:06:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.155+0000 7f496b302540 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2024-01-11T07:06:35.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-11T07:06:35.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:35.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.289+0000 7f5cb4bd2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-11T07:06:35.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.289+0000 7f5cb4bd2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-11T07:06:35.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.289+0000 7f5cb4bd2540 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2024-01-11T07:06:35.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-11T07:06:35.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:35.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.423+0000 7f6a05ee6540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-11T07:06:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.423+0000 7f6a05ee6540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-11T07:06:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:35.423+0000 7f6a05ee6540 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2024-01-11T07:06:35.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-11T07:06:35.463 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-11T07:06:35.495 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:35.495 INFO:tasks.ceph:role: osd.4 2024-01-11T07:06:35.495 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:06:35.495 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-11T07:06:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-11T07:06:35.573 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-11T07:06:35.573 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-11T07:06:35.573 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-11T07:06:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-11T07:06:35.574 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-01-11T07:06:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-11T07:06:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-11T07:06:35.574 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-11T07:06:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-11T07:06:36.055 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-01-11T07:06:36.057 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-01-11T07:06:36.057 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-11T07:06:36.173 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-11T07:06:36.209 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-11T07:06:36.278 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:36.278 INFO:tasks.ceph:role: osd.5 2024-01-11T07:06:36.278 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:06:36.278 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-11T07:06:36.354 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-11T07:06:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-11T07:06:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-01-11T07:06:36.802 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-01-11T07:06:36.802 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-11T07:06:36.950 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-11T07:06:36.985 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-11T07:06:37.059 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:37.059 INFO:tasks.ceph:role: osd.6 2024-01-11T07:06:37.060 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:06:37.060 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-11T07:06:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-11T07:06:37.149 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-11T07:06:37.149 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-11T07:06:37.149 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-11T07:06:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-11T07:06:37.150 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-01-11T07:06:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-11T07:06:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-11T07:06:37.150 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-11T07:06:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-11T07:06:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-01-11T07:06:37.638 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-01-11T07:06:37.638 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-11T07:06:37.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-11T07:06:37.804 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-11T07:06:37.873 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-11T07:06:37.873 INFO:tasks.ceph:role: osd.7 2024-01-11T07:06:37.873 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:06:37.873 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-11T07:06:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-11T07:06:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-11T07:06:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-01-11T07:06:38.429 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-01-11T07:06:38.429 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-11T07:06:38.554 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-11T07:06:38.589 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.674+0000 7efdf1801540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-11T07:06:38.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.676+0000 7efdf1801540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-11T07:06:38.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.677+0000 7efdf1801540 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2024-01-11T07:06:38.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-11T07:06:38.713 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.795+0000 7f0f4de7c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-11T07:06:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.795+0000 7f0f4de7c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-11T07:06:38.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.796+0000 7f0f4de7c540 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2024-01-11T07:06:38.804 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-11T07:06:38.871 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:38.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.954+0000 7f0044f4e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-11T07:06:38.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.957+0000 7f0044f4e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-11T07:06:38.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:38.962+0000 7f0044f4e540 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2024-01-11T07:06:38.971 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-11T07:06:39.037 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:39.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:39.120+0000 7f1bed6a1540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-11T07:06:39.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:39.122+0000 7f1bed6a1540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-11T07:06:39.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:39.123+0000 7f1bed6a1540 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2024-01-11T07:06:39.132 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-11T07:06:39.199 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-11T07:06:39.199 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:39.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-11T07:06:39.235 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:39.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-11T07:06:39.309 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:39.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-11T07:06:39.384 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:39.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-11T07:06:39.459 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:39.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-11T07:06:39.534 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:39.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-11T07:06:39.609 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:39.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-11T07:06:39.684 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:39.684 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-11T07:06:39.714 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:39.714 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-11T07:06:39.783 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:39.783 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-11T07:06:39.851 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:39.851 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-11T07:06:39.918 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:39.919 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-11T07:06:39.985 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:39.985 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-11T07:06:40.052 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:40.052 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-11T07:06:40.118 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:40.118 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-11T07:06:40.139 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:40.139 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-11T07:06:40.201 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:40.201 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-11T07:06:40.217 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:40.218 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-11T07:06:40.274 INFO:tasks.ceph:Adding keys to all mons... 2024-01-11T07:06:40.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-11T07:06:40.277 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-11T07:06:40.317 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-01-11T07:06:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng== 2024-01-11T07:06:40.317 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-01-11T07:06:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A== 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ== 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg== 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw== 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA== 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A== 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw== 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-01-11T07:06:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA== 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ== 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A== 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA== 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w== 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA== 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw== 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout:[client.2] 2024-01-11T07:06:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q== 2024-01-11T07:06:40.320 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-01-11T07:06:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw== 2024-01-11T07:06:40.320 INFO:teuthology.orchestra.run.smithi007.stdout:[client.3] 2024-01-11T07:06:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA== 2024-01-11T07:06:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2024-01-11T07:06:40.339 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng== 2024-01-11T07:06:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-01-11T07:06:40.339 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A== 2024-01-11T07:06:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-01-11T07:06:40.339 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ== 2024-01-11T07:06:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg== 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw== 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA== 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A== 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw== 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA== 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ== 2024-01-11T07:06:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A== 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA== 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w== 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA== 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw== 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:[client.2] 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q== 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:[client.1] 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw== 2024-01-11T07:06:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:[client.3] 2024-01-11T07:06:40.343 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA== 2024-01-11T07:06:40.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-11T07:06:40.364 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-11T07:06:40.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-11T07:06:40.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-11T07:06:40.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-11T07:06:40.498 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-11T07:06:40.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.558 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.615 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.727 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-11T07:06:40.782 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-11T07:06:40.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-11T07:06:40.837 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-11T07:06:40.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-11T07:06:40.896 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-11T07:06:40.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:40.954 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:41.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:41.008 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:41.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:41.064 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:41.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:41.119 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-11T07:06:41.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-11T07:06:41.174 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-11T07:06:41.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-11T07:06:41.232 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-11T07:06:41.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-11T07:06:41.290 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-11T07:06:41.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-11T07:06:41.349 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-11T07:06:41.404 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-11T07:06:41.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-11T07:06:41.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-11T07:06:41.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-11T07:06:41.569 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-11T07:06:41.596 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-11T07:06:41.690 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-11T07:06:41.716 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-11T07:06:41.783 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-11T07:06:41.876 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-11T07:06:41.903 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:41.905 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-11T07:06:41.959 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-11T07:06:41.959 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-11T07:06:41.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-11T07:06:41.967 INFO:tasks.ceph.mon.a:Started 2024-01-11T07:06:41.967 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-11T07:06:41.967 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-11T07:06:42.001 INFO:tasks.ceph.mon.b:Started 2024-01-11T07:06:42.002 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-11T07:06:42.002 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-11T07:06:42.004 INFO:tasks.ceph.mon.c:Started 2024-01-11T07:06:42.004 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-11T07:06:42.004 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-11T07:06:42.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-11T07:06:42.006 INFO:tasks.ceph.mgr.y:Started 2024-01-11T07:06:42.007 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-11T07:06:42.007 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-11T07:06:42.008 INFO:tasks.ceph.mgr.x:Started 2024-01-11T07:06:42.009 INFO:tasks.ceph:Setting crush tunables to default 2024-01-11T07:06:42.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-01-11T07:06:42.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.143+0000 7ff1e9340700 1 Processor -- start 2024-01-11T07:06:42.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.144+0000 7ff1e9340700 1 -- start start 2024-01-11T07:06:42.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.144+0000 7ff1e9340700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e4164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:42.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.144+0000 7ff1e9340700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e40cee70 con 0x7ff1e4159ea0 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.144+0000 7ff1e9340700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e40ce860 con 0x7ff1e40cdf10 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.144+0000 7ff1e9340700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40afee0 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.145+0000 7ff1e2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff1e4159ea0 0x7ff1e40af7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40516/0 (socket says 172.21.15.7:40516) 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.145+0000 7ff1e2ffd700 1 -- 172.21.15.7:0/391711277 learned_addr learned my addr 172.21.15.7:0/391711277 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.145+0000 7ff1e37fe700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e4164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.145+0000 7ff1e37fe700 1 -- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 msgr2=0x7ff1e4164ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.145+0000 7ff1e37fe700 1 -- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 msgr2=0x7ff1e4164ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.145+0000 7ff1e37fe700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e4164ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-01-11T07:06:42.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.145+0000 7ff1e37fe700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e4164ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.328+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225183969 0 0) 0x7ff1e40cee70 con 0x7ff1e4159ea0 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.328+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d0003570 con 0x7ff1e4159ea0 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911119115 0 0) 0x7ff1e40ce860 con 0x7ff1e40cdf10 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e40cee70 con 0x7ff1e40cdf10 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1113095046 0 0) 0x7ff1e40cee70 con 0x7ff1e40cdf10 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 msgr2=0x7ff1e4164ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e4164ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 >> v1:172.21.15.7:6789/0 conn(0x7ff1e4159ea0 legacy=0x7ff1e40af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40b0100 con 0x7ff1e40cdf10 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7ff1d40014b0 con 0x7ff1e40cdf10 2024-01-11T07:06:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 >> v1:172.21.15.113:6789/0 conn(0x7ff1e40cdf10 legacy=0x7ff1e40cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d0003bf0 0x7ff1d0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d0004810 0x7ff1d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1d0005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40afee0 con 0x7ff1d0004810 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d000bc20 con 0x7ff1d0003bf0 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d000bde0 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.329+0000 7ff1e37fe700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1d0005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e2ffd700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d0003bf0 0x7ff1d0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e27fc700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d0004810 0x7ff1d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e37fe700 1 -- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d0003bf0 msgr2=0x7ff1d0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e37fe700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1d0003bf0 0x7ff1d0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e37fe700 1 -- 172.21.15.7:0/391711277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d0004810 msgr2=0x7ff1d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e37fe700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d0004810 0x7ff1d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e37fe700 1 -- 172.21.15.7:0/391711277 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40cee70 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e27fc700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1d0004810 0x7ff1d0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.330+0000 7ff1e37fe700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1d0005290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4089ae0 tx=0x7ff1dc009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7143c5646b9b38fa server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.331+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc009a50 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.331+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc007c90 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.331+0000 7ff1e1ffb700 1 -- 172.21.15.7:0/391711277 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc014630 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.332+0000 7ff1e9340700 1 -- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 msgr2=0x7ff1d0005290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.332+0000 7ff1e9340700 1 --2- 172.21.15.7:0/391711277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1d0005290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4089ae0 tx=0x7ff1dc009740 comp rx=0 tx=0).stop 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.332+0000 7ff1e37fe700 1 -- 172.21.15.7:0/391711277 reap_dead start 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.332+0000 7ff1e9340700 1 -- 172.21.15.7:0/391711277 shutdown_connections 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.332+0000 7ff1e9340700 1 -- 172.21.15.7:0/391711277 >> 172.21.15.7:0/391711277 conn(0x7ff1e4071120 msgr2=0x7ff1e406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.333+0000 7ff1e9340700 1 -- 172.21.15.7:0/391711277 shutdown_connections 2024-01-11T07:06:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.333+0000 7ff1e9340700 1 -- 172.21.15.7:0/391711277 wait complete. 2024-01-11T07:06:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.335+0000 7ff1e9340700 1 Processor -- start 2024-01-11T07:06:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.335+0000 7ff1e9340700 1 -- start start 2024-01-11T07:06:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e9340700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e40c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e9340700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e40cb160 0x7ff1e40d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e9340700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e40d1130 0x7ff1e416a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e9340700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40d15b0 con 0x7ff1e40d1130 2024-01-11T07:06:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e9340700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40ca0c0 con 0x7ff1e40cb160 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e9340700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40ca3a0 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e2ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e40c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e40d1130 0x7ff1e416a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e27fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e40cb160 0x7ff1e40d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e2ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e40c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:36424/0 (socket says 172.21.15.7:36424) 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.336+0000 7ff1e2ffd700 1 -- 172.21.15.7:0/1251576249 learned_addr learned my addr 172.21.15.7:0/1251576249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e2ffd700 1 -- 172.21.15.7:0/1251576249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e40cb160 msgr2=0x7ff1e40d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e2ffd700 1 --2- 172.21.15.7:0/1251576249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e40cb160 0x7ff1e40d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e2ffd700 1 -- 172.21.15.7:0/1251576249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e40d1130 msgr2=0x7ff1e416a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e2ffd700 1 --2- 172.21.15.7:0/1251576249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e40d1130 0x7ff1e416a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e2ffd700 1 -- 172.21.15.7:0/1251576249 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40ca680 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e27fc700 1 --2- 172.21.15.7:0/1251576249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e40cb160 0x7ff1e40d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e2ffd700 1 --2- 172.21.15.7:0/1251576249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e40c99c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0016b0 tx=0x7ff1cc007de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1daffd700 1 -- 172.21.15.7:0/1251576249 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011590 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1daffd700 1 -- 172.21.15.7:0/1251576249 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc011710 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1daffd700 1 -- 172.21.15.7:0/1251576249 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011ad0 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e40ca960 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e416a7c0 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.337+0000 7ff1daffd700 1 -- 172.21.15.7:0/1251576249 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff1cc0146d0 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.338+0000 7ff1daffd700 1 -- 172.21.15.7:0/1251576249 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff1cc01e790 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.339+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e40cafb0 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.342+0000 7ff1daffd700 1 -- 172.21.15.7:0/1251576249 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff1cc019070 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.423+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff1e40695e0 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.426+0000 7ff1daffd700 1 -- 172.21.15.7:0/1251576249 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff1cc011e40 con 0x7ff1e40cc0d0 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 msgr2=0x7ff1e40c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 --2- 172.21.15.7:0/1251576249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e40c99c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0016b0 tx=0x7ff1cc007de0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 shutdown_connections 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 --2- 172.21.15.7:0/1251576249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e40d1130 0x7ff1e416a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 --2- 172.21.15.7:0/1251576249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff1e40cc0d0 0x7ff1e40c99c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 --2- 172.21.15.7:0/1251576249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1e40cb160 0x7ff1e40d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 >> 172.21.15.7:0/1251576249 conn(0x7ff1e4071120 msgr2=0x7ff1e41566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 shutdown_connections 2024-01-11T07:06:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:42.427+0000 7ff1e9340700 1 -- 172.21.15.7:0/1251576249 wait complete. 2024-01-11T07:06:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-01-11T07:06:42.440 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-11T07:06:42.440 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:42.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-11T07:06:42.470 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:42.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-11T07:06:42.540 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:42.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-11T07:06:42.610 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-01-11T07:06:42.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-11T07:06:42.670 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:42.671+0000 7f28f79c21c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-11T07:06:42.680 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:42.680 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-11T07:06:42.710 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:42.710 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-11T07:06:42.733 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:42.734+0000 7ff30e1e61c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-11T07:06:42.755 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:42.755+0000 7f28f79c21c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-11T07:06:42.777 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:42.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-11T07:06:42.821 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:42.822+0000 7ff30e1e61c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-11T07:06:42.830 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:42.830+0000 7f28f79c21c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-11T07:06:42.844 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-01-11T07:06:42.844 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-11T07:06:42.900 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:42.901+0000 7f28f79c21c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-11T07:06:42.902 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:42.903+0000 7ff30e1e61c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-11T07:06:42.911 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2024-01-11T07:06:42.979 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:42.980+0000 7ff30e1e61c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-11T07:06:43.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.071+0000 7f7375ef6700 1 Processor -- start 2024-01-11T07:06:43.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.071+0000 7f7375ef6700 1 -- start start 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7375ef6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7370157b50 0x7f73700b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7375ef6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73700b0740 0x7f737006b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7375ef6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73700afd70 con 0x7f737006b8d0 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7375ef6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737015bab0 con 0x7f7370157b50 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7375ef6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737006bd50 con 0x7f73700b0740 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7367fff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73700b0740 0x7f737006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7367fff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73700b0740 0x7f737006b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52362/0 (socket says 172.21.15.113:52362) 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f736ffff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f737006b8d0 0x7f73700c8f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33190/0 (socket says 172.21.15.113:33190) 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7367fff700 1 -- 172.21.15.113:0/1896728992 learned_addr learned my addr 172.21.15.113:0/1896728992 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f736f7fe700 1 --2- 172.21.15.113:0/1896728992 >> v2:172.21.15.7:3300/0 conn(0x7f7370157b50 0x7f73700b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7367fff700 1 -- 172.21.15.113:0/1896728992 >> v2:172.21.15.7:3300/0 conn(0x7f7370157b50 msgr2=0x7f73700b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7367fff700 1 --2- 172.21.15.113:0/1896728992 >> v2:172.21.15.7:3300/0 conn(0x7f7370157b50 0x7f73700b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7367fff700 1 -- 172.21.15.113:0/1896728992 >> v1:172.21.15.113:6789/0 conn(0x7f737006b8d0 legacy=0x7f73700c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f7367fff700 1 -- 172.21.15.113:0/1896728992 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700c9b90 con 0x7f73700b0740 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.072+0000 7f736effd700 1 -- 172.21.15.113:0/1896728992 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334682403 0 0) 0x7f73700afd70 con 0x7f737006b8d0 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f736f7fe700 1 --2- 172.21.15.113:0/1896728992 >> v2:172.21.15.7:3300/0 conn(0x7f7370157b50 0x7f73700b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f7367fff700 1 --2- 172.21.15.113:0/1896728992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73700b0740 0x7f737006b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f737006eb90 tx=0x7f7358009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9587a207a46cd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f736effd700 1 -- 172.21.15.113:0/1896728992 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735800b860 con 0x7f73700b0740 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f736effd700 1 -- 172.21.15.113:0/1896728992 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735800b9c0 con 0x7f73700b0740 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f736effd700 1 -- 172.21.15.113:0/1896728992 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735800bd60 con 0x7f73700b0740 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f7375ef6700 1 -- 172.21.15.113:0/1896728992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73700b0740 msgr2=0x7f737006b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f7375ef6700 1 --2- 172.21.15.113:0/1896728992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73700b0740 0x7f737006b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f737006eb90 tx=0x7f7358009250 comp rx=0 tx=0).stop 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f7375ef6700 1 -- 172.21.15.113:0/1896728992 shutdown_connections 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f7375ef6700 1 --2- 172.21.15.113:0/1896728992 >> v2:172.21.15.7:3300/0 conn(0x7f7370157b50 0x7f73700b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f7375ef6700 1 --2- 172.21.15.113:0/1896728992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73700b0740 0x7f737006b390 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.073+0000 7f7375ef6700 1 -- 172.21.15.113:0/1896728992 >> 172.21.15.113:0/1896728992 conn(0x7f7370070690 msgr2=0x7f7370070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:43.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.074+0000 7f7375ef6700 1 -- 172.21.15.113:0/1896728992 shutdown_connections 2024-01-11T07:06:43.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.074+0000 7f7375ef6700 1 -- 172.21.15.113:0/1896728992 wait complete. 2024-01-11T07:06:43.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.075+0000 7f7375ef6700 1 Processor -- start 2024-01-11T07:06:43.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.075+0000 7f7375ef6700 1 -- start start 2024-01-11T07:06:43.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7375ef6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f737006b8d0 0x7f73700c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7375ef6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73700b0740 0x7f7370067610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7375ef6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7370157b50 0x7f7370067b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7375ef6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73700acc20 con 0x7f737006b8d0 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7375ef6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370068180 con 0x7f7370157b50 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7375ef6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7370068460 con 0x7f73700b0740 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7367fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73700b0740 0x7f7370067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f736ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7370157b50 0x7f7370067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7367fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73700b0740 0x7f7370067610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46782/0 (socket says 172.21.15.113:46782) 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f736ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7370157b50 0x7f7370067b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52364/0 (socket says 172.21.15.113:52364) 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7367fff700 1 -- 172.21.15.113:0/1846269090 learned_addr learned my addr 172.21.15.113:0/1846269090 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f736f7fe700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f737006b8d0 0x7f73700c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7367fff700 1 -- 172.21.15.113:0/1846269090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7370157b50 msgr2=0x7f7370067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7367fff700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7370157b50 0x7f7370067b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7367fff700 1 -- 172.21.15.113:0/1846269090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f737006b8d0 msgr2=0x7f73700c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7367fff700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f737006b8d0 0x7f73700c5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f7367fff700 1 -- 172.21.15.113:0/1846269090 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73580094f0 con 0x7f73700b0740 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.076+0000 7f736ffff700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7370157b50 0x7f7370067b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.077+0000 7f7367fff700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73700b0740 0x7f7370067610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7358006f50 tx=0x7f7358017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.077+0000 7f73677fe700 1 -- 172.21.15.113:0/1846269090 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358011b20 con 0x7f73700b0740 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.077+0000 7f73677fe700 1 -- 172.21.15.113:0/1846269090 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358011c80 con 0x7f73700b0740 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.077+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7370068740 con 0x7f73700b0740 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.077+0000 7f73677fe700 1 -- 172.21.15.113:0/1846269090 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358025420 con 0x7f73700b0740 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.077+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7370068d00 con 0x7f73700b0740 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.077+0000 7f73677fe700 1 -- 172.21.15.113:0/1846269090 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f7358023070 con 0x7f73700b0740 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.077+0000 7f73677fe700 1 -- 172.21.15.113:0/1846269090 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f735802f4d0 con 0x7f73700b0740 2024-01-11T07:06:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.078+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737004c3c0 con 0x7f73700b0740 2024-01-11T07:06:43.081 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:43.080+0000 7f28f79c21c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-11T07:06:43.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.080+0000 7f73677fe700 1 -- 172.21.15.113:0/1846269090 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f735802a070 con 0x7f73700b0740 2024-01-11T07:06:43.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.161+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f7370068ef0 con 0x7f73700b0740 2024-01-11T07:06:43.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.164+0000 7f73677fe700 1 -- 172.21.15.113:0/1846269090 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7358015020 con 0x7f73700b0740 2024-01-11T07:06:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:0 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73700b0740 msgr2=0x7f7370067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73700b0740 0x7f7370067610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7358006f50 tx=0x7f7358017000 comp rx=0 tx=0).stop 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 shutdown_connections 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7370157b50 0x7f7370067b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f737006b8d0 0x7f73700c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 --2- 172.21.15.113:0/1846269090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73700b0740 0x7f7370067610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 >> 172.21.15.113:0/1846269090 conn(0x7f7370070690 msgr2=0x7f737006ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 shutdown_connections 2024-01-11T07:06:43.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.165+0000 7f7375ef6700 1 -- 172.21.15.113:0/1846269090 wait complete. 2024-01-11T07:06:43.169 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:43.170+0000 7ff30e1e61c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-11T07:06:43.176 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2024-01-11T07:06:43.230 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:43.230+0000 7f28f79c21c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-11T07:06:43.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.293+0000 7fa796b17700 1 Processor -- start 2024-01-11T07:06:43.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.293+0000 7fa796b17700 1 -- start start 2024-01-11T07:06:43.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa796b17700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa790155720 0x7fa7901580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa796b17700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7900cb680 con 0x7fa79006b5e0 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa796b17700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa790155af0 con 0x7fa79006baa0 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa796b17700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7900caa70 con 0x7fa790155720 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa78ffff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa79006baa0 0x7fa79006aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33206/0 (socket says 172.21.15.113:33206) 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa78ffff700 1 -- 172.21.15.113:0/1976869500 learned_addr learned my addr 172.21.15.113:0/1976869500 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa7948b3700 1 --2- 172.21.15.113:0/1976869500 >> v2:172.21.15.7:3300/0 conn(0x7fa790155720 0x7fa7901580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa78f7fe700 1 -- 172.21.15.113:0/1976869500 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042452542 0 0) 0x7fa790155af0 con 0x7fa79006baa0 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa78f7fe700 1 -- 172.21.15.113:0/1976869500 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa780003570 con 0x7fa79006baa0 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa78f7fe700 1 -- 172.21.15.113:0/1976869500 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437155930 0 0) 0x7fa7900cb680 con 0x7fa79006b5e0 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa78f7fe700 1 -- 172.21.15.113:0/1976869500 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa790155af0 con 0x7fa79006b5e0 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa78f7fe700 1 -- 172.21.15.113:0/1976869500 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2618066017 0 0) 0x7fa780003570 con 0x7fa79006baa0 2024-01-11T07:06:43.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa7948b3700 1 -- 172.21.15.113:0/1976869500 >> v1:172.21.15.113:6789/0 conn(0x7fa79006baa0 legacy=0x7fa79006aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa7948b3700 1 -- 172.21.15.113:0/1976869500 >> v1:172.21.15.7:6789/0 conn(0x7fa79006b5e0 legacy=0x7fa7901592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.294+0000 7fa7948b3700 1 -- 172.21.15.113:0/1976869500 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7900cad50 con 0x7fa790155720 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa7948b3700 1 --2- 172.21.15.113:0/1976869500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa790155720 0x7fa7901580e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa784009860 tx=0x7fa7840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb608b31f1269607 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa78f7fe700 1 -- 172.21.15.113:0/1976869500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c001470 con 0x7fa790155720 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa78f7fe700 1 -- 172.21.15.113:0/1976869500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784011420 con 0x7fa790155720 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa78f7fe700 1 -- 172.21.15.113:0/1976869500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840117c0 con 0x7fa790155720 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa796b17700 1 -- 172.21.15.113:0/1976869500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa790155720 msgr2=0x7fa7901580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa796b17700 1 --2- 172.21.15.113:0/1976869500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa790155720 0x7fa7901580e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa784009860 tx=0x7fa7840091a0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa796b17700 1 -- 172.21.15.113:0/1976869500 shutdown_connections 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa796b17700 1 --2- 172.21.15.113:0/1976869500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa790155720 0x7fa7901580e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa796b17700 1 -- 172.21.15.113:0/1976869500 >> 172.21.15.113:0/1976869500 conn(0x7fa79006ed90 msgr2=0x7fa79006f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:43.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa796b17700 1 -- 172.21.15.113:0/1976869500 shutdown_connections 2024-01-11T07:06:43.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.295+0000 7fa796b17700 1 -- 172.21.15.113:0/1976869500 wait complete. 2024-01-11T07:06:43.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.297+0000 7fa796b17700 1 Processor -- start 2024-01-11T07:06:43.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.297+0000 7fa796b17700 1 -- start start 2024-01-11T07:06:43.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa796b17700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa79006b5e0 0x7fa790152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa796b17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79006baa0 0x7fa79015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa796b17700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa790155720 0x7fa79015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa796b17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7900683c0 con 0x7fa79006baa0 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa796b17700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7900677b0 con 0x7fa79006b5e0 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa796b17700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa790067a90 con 0x7fa790155720 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7948b3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa79006b5e0 0x7fa790152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7950b4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa790155720 0x7fa79015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7948b3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa79006b5e0 0x7fa790152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52368/0 (socket says 172.21.15.113:52368) 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7950b4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa790155720 0x7fa79015a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46786/0 (socket says 172.21.15.113:46786) 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7948b3700 1 -- 172.21.15.113:0/2043120903 learned_addr learned my addr 172.21.15.113:0/2043120903 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa78ffff700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79006baa0 0x7fa79015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7948b3700 1 -- 172.21.15.113:0/2043120903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa790155720 msgr2=0x7fa79015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7948b3700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa790155720 0x7fa79015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7948b3700 1 -- 172.21.15.113:0/2043120903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79006baa0 msgr2=0x7fa79015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7948b3700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79006baa0 0x7fa79015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7948b3700 1 -- 172.21.15.113:0/2043120903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa784009420 con 0x7fa79006b5e0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa7950b4700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa790155720 0x7fa79015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.298+0000 7fa78ffff700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79006baa0 0x7fa79015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.299+0000 7fa7948b3700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa79006b5e0 0x7fa790152a40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7900c5980 tx=0x7fa784016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.299+0000 7fa78cff9700 1 -- 172.21.15.113:0/2043120903 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78401cb30 con 0x7fa79006b5e0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.299+0000 7fa78cff9700 1 -- 172.21.15.113:0/2043120903 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78401cc90 con 0x7fa79006b5e0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.299+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa790067d70 con 0x7fa79006b5e0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.299+0000 7fa78cff9700 1 -- 172.21.15.113:0/2043120903 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78401a6a0 con 0x7fa79006b5e0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.299+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7900c7560 con 0x7fa79006b5e0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.299+0000 7fa78cff9700 1 -- 172.21.15.113:0/2043120903 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa78401acd0 con 0x7fa79006b5e0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.299+0000 7fa78cff9700 1 -- 172.21.15.113:0/2043120903 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fa78402ebf0 con 0x7fa79006b5e0 2024-01-11T07:06:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.300+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79004c3c0 con 0x7fa79006b5e0 2024-01-11T07:06:43.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.302+0000 7fa78cff9700 1 -- 172.21.15.113:0/2043120903 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa784029070 con 0x7fa79006b5e0 2024-01-11T07:06:43.303 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:43.302+0000 7f28f79c21c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-11T07:06:43.332 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:43.333+0000 7ff30e1e61c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-11T07:06:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.387+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7fa79006b180 con 0x7fa79006b5e0 2024-01-11T07:06:43.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.391+0000 7fa78cff9700 1 -- 172.21.15.113:0/2043120903 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa78401ce00 con 0x7fa79006b5e0 2024-01-11T07:06:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.391+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa79006b5e0 msgr2=0x7fa790152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.391+0000 7fa796b17700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa79006b5e0 0x7fa790152a40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7900c5980 tx=0x7fa784016000 comp rx=0 tx=0).stop 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.392+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 shutdown_connections 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.392+0000 7fa796b17700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa790155720 0x7fa79015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.392+0000 7fa796b17700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa79006b5e0 0x7fa790152a40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.392+0000 7fa796b17700 1 --2- 172.21.15.113:0/2043120903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79006baa0 0x7fa79015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.392+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 >> 172.21.15.113:0/2043120903 conn(0x7fa79006ed90 msgr2=0x7fa7901547f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.392+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 shutdown_connections 2024-01-11T07:06:43.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.392+0000 7fa796b17700 1 -- 172.21.15.113:0/2043120903 wait complete. 2024-01-11T07:06:43.403 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2024-01-11T07:06:43.411 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:43.412+0000 7ff30e1e61c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-11T07:06:43.441 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:43.442+0000 7f28f79c21c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-11T07:06:43.518 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:43.519+0000 7f28f79c21c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-11T07:06:43.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.521+0000 7f0dd70e1700 1 Processor -- start 2024-01-11T07:06:43.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.521+0000 7f0dd70e1700 1 -- start start 2024-01-11T07:06:43.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dd70e1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ada70 0x7f0dd015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dd70e1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0dd00af7a0 0x7f0dd006a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dd70e1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd015b860 con 0x7f0dd00ade40 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dd70e1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd015b530 con 0x7f0dd00af7a0 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dd70e1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd015ac30 con 0x7f0dd00ada70 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dcffff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0dd00af7a0 0x7f0dd006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dd4e7d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ada70 0x7f0dd015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dcffff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0dd00af7a0 0x7f0dd006a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52378/0 (socket says 172.21.15.113:52378) 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dcffff700 1 -- 172.21.15.113:0/1251334138 learned_addr learned my addr 172.21.15.113:0/1251334138 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dcffff700 1 -- 172.21.15.113:0/1251334138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ada70 msgr2=0x7f0dd015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dcffff700 1 --2- 172.21.15.113:0/1251334138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ada70 0x7f0dd015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dcffff700 1 -- 172.21.15.113:0/1251334138 >> v1:172.21.15.7:6789/0 conn(0x7f0dd00ade40 legacy=0x7f0dd015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dcffff700 1 -- 172.21.15.113:0/1251334138 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd015af10 con 0x7f0dd00af7a0 2024-01-11T07:06:43.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.522+0000 7f0dd4e7d700 1 --2- 172.21.15.113:0/1251334138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ada70 0x7f0dd015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dcffff700 1 --2- 172.21.15.113:0/1251334138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00af7a0 0x7f0dd006a600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0099b0 tx=0x7f0dbc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=216627f56dfa9160 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dcf7fe700 1 -- 172.21.15.113:0/1251334138 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc009ea0 con 0x7f0dd00af7a0 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dcf7fe700 1 -- 172.21.15.113:0/1251334138 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc00b860 con 0x7f0dd00af7a0 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dcf7fe700 1 -- 172.21.15.113:0/1251334138 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00bc00 con 0x7f0dd00af7a0 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dd70e1700 1 -- 172.21.15.113:0/1251334138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00af7a0 msgr2=0x7f0dd006a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dd70e1700 1 --2- 172.21.15.113:0/1251334138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00af7a0 0x7f0dd006a600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0099b0 tx=0x7f0dbc009250 comp rx=0 tx=0).stop 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dd70e1700 1 -- 172.21.15.113:0/1251334138 shutdown_connections 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dd70e1700 1 --2- 172.21.15.113:0/1251334138 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ada70 0x7f0dd015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dd70e1700 1 --2- 172.21.15.113:0/1251334138 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00af7a0 0x7f0dd006a600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dd70e1700 1 -- 172.21.15.113:0/1251334138 >> 172.21.15.113:0/1251334138 conn(0x7f0dd00cd070 msgr2=0x7f0dd00cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dd70e1700 1 -- 172.21.15.113:0/1251334138 shutdown_connections 2024-01-11T07:06:43.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.523+0000 7f0dd70e1700 1 -- 172.21.15.113:0/1251334138 wait complete. 2024-01-11T07:06:43.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.524+0000 7f0dd70e1700 1 Processor -- start 2024-01-11T07:06:43.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd70e1700 1 -- start start 2024-01-11T07:06:43.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd70e1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00ada70 0x7f0dd00afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd70e1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ade40 0x7f0dd0152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd70e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dd00af7a0 0x7f0dd00cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd70e1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00b0350 con 0x7f0dd00af7a0 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd70e1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0155910 con 0x7f0dd00ada70 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd70e1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0153460 con 0x7f0dd00ade40 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dcffff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ade40 0x7f0dd0152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd4e7d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00ada70 0x7f0dd00afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dcffff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ade40 0x7f0dd0152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46798/0 (socket says 172.21.15.113:46798) 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dcffff700 1 -- 172.21.15.113:0/4259538635 learned_addr learned my addr 172.21.15.113:0/4259538635 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.525+0000 7f0dd4e7d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00ada70 0x7f0dd00afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52392/0 (socket says 172.21.15.113:52392) 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dd567e700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dd00af7a0 0x7f0dd00cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dcffff700 1 -- 172.21.15.113:0/4259538635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00ada70 msgr2=0x7f0dd00afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dcffff700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00ada70 0x7f0dd00afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dcffff700 1 -- 172.21.15.113:0/4259538635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dd00af7a0 msgr2=0x7f0dd00cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dcffff700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dd00af7a0 0x7f0dd00cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dcffff700 1 -- 172.21.15.113:0/4259538635 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc4009420 con 0x7f0dd00ade40 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dd4e7d700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00ada70 0x7f0dd00afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dcffff700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ade40 0x7f0dd0152f20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0063d40 tx=0x7f0dbc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dccff9700 1 -- 172.21.15.113:0/4259538635 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00b9d0 con 0x7f0dd00ade40 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dccff9700 1 -- 172.21.15.113:0/4259538635 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc00bb50 con 0x7f0dd00ade40 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc0094f0 con 0x7f0dd00ade40 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dccff9700 1 -- 172.21.15.113:0/4259538635 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc025670 con 0x7f0dd00ade40 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd00b2490 con 0x7f0dd00ade40 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dccff9700 1 -- 172.21.15.113:0/4259538635 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0dbc023070 con 0x7f0dd00ade40 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.526+0000 7f0dccff9700 1 -- 172.21.15.113:0/4259538635 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f0dbc011850 con 0x7f0dd00ade40 2024-01-11T07:06:43.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.527+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd004c3c0 con 0x7f0dd00ade40 2024-01-11T07:06:43.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.529+0000 7f0dccff9700 1 -- 172.21.15.113:0/4259538635 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0dbc02a070 con 0x7f0dd00ade40 2024-01-11T07:06:43.562 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:43.562+0000 7ff30e1e61c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-11T07:06:43.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.616+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7f0dd015b1f0 con 0x7f0dd00ade40 2024-01-11T07:06:43.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.621+0000 7f0dccff9700 1 -- 172.21.15.113:0/4259538635 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0dbc015020 con 0x7f0dd00ade40 2024-01-11T07:06:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:2 2024-01-11T07:06:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ade40 msgr2=0x7f0dd0152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ade40 0x7f0dd0152f20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0063d40 tx=0x7f0dbc017000 comp rx=0 tx=0).stop 2024-01-11T07:06:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 shutdown_connections 2024-01-11T07:06:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dd00af7a0 0x7f0dd00cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd00ada70 0x7f0dd00afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 --2- 172.21.15.113:0/4259538635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0dd00ade40 0x7f0dd0152f20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 >> 172.21.15.113:0/4259538635 conn(0x7f0dd00cd070 msgr2=0x7f0dd00cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:43.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 shutdown_connections 2024-01-11T07:06:43.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.622+0000 7f0dd70e1700 1 -- 172.21.15.113:0/4259538635 wait complete. 2024-01-11T07:06:43.637 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2024-01-11T07:06:43.646 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:43.646+0000 7ff30e1e61c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-11T07:06:43.721 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:43.721+0000 7f28f79c21c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-11T07:06:43.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.799+0000 7f1a4c268700 1 Processor -- start 2024-01-11T07:06:43.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.799+0000 7f1a4c268700 1 -- start start 2024-01-11T07:06:43.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4c268700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a4413c410 0x7f1a440c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4c268700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a440c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4c268700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a441575e0 0x7f1a440c7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4c268700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440abab0 con 0x7f1a441575e0 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4c268700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440c1630 con 0x7f1a4413c410 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4c268700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440c2490 con 0x7f1a44157170 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4a004700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a4413c410 0x7f1a440c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a49803700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a440c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4a004700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a4413c410 0x7f1a440c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52400/0 (socket says 172.21.15.113:52400) 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a49803700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a440c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46802/0 (socket says 172.21.15.113:46802) 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4a004700 1 -- 172.21.15.113:0/3133383285 learned_addr learned my addr 172.21.15.113:0/3133383285 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4a805700 1 --2- 172.21.15.113:0/3133383285 >> v2:172.21.15.7:3300/0 conn(0x7f1a441575e0 0x7f1a440c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a49803700 1 -- 172.21.15.113:0/3133383285 >> v2:172.21.15.113:3300/0 conn(0x7f1a4413c410 msgr2=0x7f1a440c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a49803700 1 --2- 172.21.15.113:0/3133383285 >> v2:172.21.15.113:3300/0 conn(0x7f1a4413c410 0x7f1a440c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a49803700 1 -- 172.21.15.113:0/3133383285 >> v2:172.21.15.7:3300/0 conn(0x7f1a441575e0 msgr2=0x7f1a440c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a49803700 1 --2- 172.21.15.113:0/3133383285 >> v2:172.21.15.7:3300/0 conn(0x7f1a441575e0 0x7f1a440c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a49803700 1 -- 172.21.15.113:0/3133383285 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4413c7e0 con 0x7f1a44157170 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.800+0000 7f1a4a004700 1 --2- 172.21.15.113:0/3133383285 >> v2:172.21.15.113:3300/0 conn(0x7f1a4413c410 0x7f1a440c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a49803700 1 --2- 172.21.15.113:0/3133383285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a440c6ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a440716f0 tx=0x7f1a340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38315da501313da2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a49002700 1 -- 172.21.15.113:0/3133383285 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34011630 con 0x7f1a44157170 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a49002700 1 -- 172.21.15.113:0/3133383285 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a340117b0 con 0x7f1a44157170 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a49002700 1 -- 172.21.15.113:0/3133383285 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34011b70 con 0x7f1a44157170 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a4c268700 1 -- 172.21.15.113:0/3133383285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 msgr2=0x7f1a440c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a4c268700 1 --2- 172.21.15.113:0/3133383285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a440c6ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a440716f0 tx=0x7f1a340091a0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a4c268700 1 -- 172.21.15.113:0/3133383285 shutdown_connections 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a4c268700 1 --2- 172.21.15.113:0/3133383285 >> v2:172.21.15.7:3300/0 conn(0x7f1a441575e0 0x7f1a440c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a4c268700 1 --2- 172.21.15.113:0/3133383285 >> v2:172.21.15.113:3300/0 conn(0x7f1a4413c410 0x7f1a440c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a4c268700 1 --2- 172.21.15.113:0/3133383285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a440c6ee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.801+0000 7f1a4c268700 1 -- 172.21.15.113:0/3133383285 >> 172.21.15.113:0/3133383285 conn(0x7f1a44063230 msgr2=0x7f1a44063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:43.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.802+0000 7f1a4c268700 1 -- 172.21.15.113:0/3133383285 shutdown_connections 2024-01-11T07:06:43.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.802+0000 7f1a4c268700 1 -- 172.21.15.113:0/3133383285 wait complete. 2024-01-11T07:06:43.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.803+0000 7f1a4c268700 1 Processor -- start 2024-01-11T07:06:43.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.803+0000 7f1a4c268700 1 -- start start 2024-01-11T07:06:43.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4c268700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a4413c410 0x7f1a4406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4c268700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a441586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4c268700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a441575e0 0x7f1a441361f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4c268700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440c3870 con 0x7f1a4413c410 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4c268700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4406d870 con 0x7f1a441575e0 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4c268700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440c40a0 con 0x7f1a44157170 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a441586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4a805700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a441575e0 0x7f1a441361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a441586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46814/0 (socket says 172.21.15.113:46814) 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4a805700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a441575e0 0x7f1a441361f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52410/0 (socket says 172.21.15.113:52410) 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 -- 172.21.15.113:0/2912289900 learned_addr learned my addr 172.21.15.113:0/2912289900 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4a004700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a4413c410 0x7f1a4406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 -- 172.21.15.113:0/2912289900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a441575e0 msgr2=0x7f1a441361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a441575e0 0x7f1a441361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 -- 172.21.15.113:0/2912289900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a4413c410 msgr2=0x7f1a4406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a4413c410 0x7f1a4406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 -- 172.21.15.113:0/2912289900 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400094f0 con 0x7f1a44157170 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a4a004700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a4413c410 0x7f1a4406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a49803700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a441586a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1a34013000 tx=0x7f1a34017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.804+0000 7f1a3a7fc700 1 -- 172.21.15.113:0/2912289900 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34021070 con 0x7f1a44157170 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.805+0000 7f1a3a7fc700 1 -- 172.21.15.113:0/2912289900 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34011920 con 0x7f1a44157170 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.805+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a34009420 con 0x7f1a44157170 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.805+0000 7f1a3a7fc700 1 -- 172.21.15.113:0/2912289900 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34027420 con 0x7f1a44157170 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.805+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a440c3a50 con 0x7f1a44157170 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.805+0000 7f1a3a7fc700 1 -- 172.21.15.113:0/2912289900 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1a34025070 con 0x7f1a44157170 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.805+0000 7f1a3a7fc700 1 -- 172.21.15.113:0/2912289900 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f1a34030850 con 0x7f1a44157170 2024-01-11T07:06:43.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.806+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a440c4380 con 0x7f1a44157170 2024-01-11T07:06:43.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.808+0000 7f1a3a7fc700 1 -- 172.21.15.113:0/2912289900 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1a3402c070 con 0x7f1a44157170 2024-01-11T07:06:43.839 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:43.839+0000 7ff30e1e61c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-11T07:06:43.850 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:43.851+0000 7f28f79c21c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-11T07:06:43.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.891+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f1a4404c3c0 con 0x7f1a44157170 2024-01-11T07:06:43.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.895+0000 7f1a3a7fc700 1 -- 172.21.15.113:0/2912289900 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a34015020 con 0x7f1a44157170 2024-01-11T07:06:43.894 INFO:teuthology.orchestra.run.smithi113.stdout:3 2024-01-11T07:06:43.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.895+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 msgr2=0x7f1a441586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:43.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.895+0000 7f1a4c268700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a441586a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1a34013000 tx=0x7f1a34017000 comp rx=0 tx=0).stop 2024-01-11T07:06:43.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.895+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 shutdown_connections 2024-01-11T07:06:43.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.895+0000 7f1a4c268700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a441575e0 0x7f1a441361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.895+0000 7f1a4c268700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a4413c410 0x7f1a4406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.895+0000 7f1a4c268700 1 --2- 172.21.15.113:0/2912289900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a44157170 0x7f1a441586a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:43.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.895+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 >> 172.21.15.113:0/2912289900 conn(0x7f1a44063230 msgr2=0x7f1a440cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:43.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.896+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 shutdown_connections 2024-01-11T07:06:43.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:43.896+0000 7f1a4c268700 1 -- 172.21.15.113:0/2912289900 wait complete. 2024-01-11T07:06:43.907 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2024-01-11T07:06:43.980 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:43.981+0000 7ff30e1e61c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-11T07:06:44.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.066+0000 7f51f912e700 1 Processor -- start 2024-01-11T07:06:44.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.066+0000 7f51f912e700 1 -- start start 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.067+0000 7f51f912e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f51f40ce2f0 0x7f51f406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.067+0000 7f51f912e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cc8b0 0x7f51f40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.067+0000 7f51f912e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f415c6a0 con 0x7f51f40cdf20 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.067+0000 7f51f912e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f40c9fb0 con 0x7f51f40ce2f0 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.067+0000 7f51f912e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f415ba90 con 0x7f51f40cc8b0 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.067+0000 7f51f359e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cc8b0 0x7f51f40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f359e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cc8b0 0x7f51f40ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46822/0 (socket says 172.21.15.113:46822) 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f359e700 1 -- 172.21.15.113:0/3020945688 learned_addr learned my addr 172.21.15.113:0/3020945688 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f259c700 1 --2- 172.21.15.113:0/3020945688 >> v2:172.21.15.7:3300/0 conn(0x7f51f40ce2f0 0x7f51f406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f359e700 1 -- 172.21.15.113:0/3020945688 >> v2:172.21.15.7:3300/0 conn(0x7f51f40ce2f0 msgr2=0x7f51f406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f359e700 1 --2- 172.21.15.113:0/3020945688 >> v2:172.21.15.7:3300/0 conn(0x7f51f40ce2f0 0x7f51f406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f359e700 1 -- 172.21.15.113:0/3020945688 >> v1:172.21.15.7:6789/0 conn(0x7f51f40cdf20 legacy=0x7f51f406aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f359e700 1 -- 172.21.15.113:0/3020945688 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f415bd70 con 0x7f51f40cc8b0 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f1d9b700 1 -- 172.21.15.113:0/3020945688 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708539892 0 0) 0x7f51f415c6a0 con 0x7f51f40cdf20 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f359e700 1 --2- 172.21.15.113:0/3020945688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cc8b0 0x7f51f40ccd30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009f20 tx=0x7f51ec009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c858593f205dc6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f1d9b700 1 -- 172.21.15.113:0/3020945688 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00e070 con 0x7f51f40cc8b0 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f1d9b700 1 -- 172.21.15.113:0/3020945688 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00b9d0 con 0x7f51f40cc8b0 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.068+0000 7f51f1d9b700 1 -- 172.21.15.113:0/3020945688 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0092e0 con 0x7f51f40cc8b0 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.069+0000 7f51f912e700 1 -- 172.21.15.113:0/3020945688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cc8b0 msgr2=0x7f51f40ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.069+0000 7f51f912e700 1 --2- 172.21.15.113:0/3020945688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cc8b0 0x7f51f40ccd30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009f20 tx=0x7f51ec009840 comp rx=0 tx=0).stop 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.069+0000 7f51f912e700 1 -- 172.21.15.113:0/3020945688 shutdown_connections 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.069+0000 7f51f912e700 1 --2- 172.21.15.113:0/3020945688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cc8b0 0x7f51f40ccd30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.069+0000 7f51f912e700 1 --2- 172.21.15.113:0/3020945688 >> v2:172.21.15.7:3300/0 conn(0x7f51f40ce2f0 0x7f51f406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.069+0000 7f51f912e700 1 -- 172.21.15.113:0/3020945688 >> 172.21.15.113:0/3020945688 conn(0x7f51f406ed70 msgr2=0x7f51f406f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.069+0000 7f51f912e700 1 -- 172.21.15.113:0/3020945688 shutdown_connections 2024-01-11T07:06:44.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.069+0000 7f51f912e700 1 -- 172.21.15.113:0/3020945688 wait complete. 2024-01-11T07:06:44.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.070+0000 7f51f912e700 1 Processor -- start 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.070+0000 7f51f912e700 1 -- start start 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f912e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f40cc8b0 0x7f51f40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f912e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cdf20 0x7f51f4151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f912e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f40ce2f0 0x7f51f40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f912e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40b2180 con 0x7f51f40ce2f0 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f912e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40b2460 con 0x7f51f40cc8b0 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f912e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40b2740 con 0x7f51f40cdf20 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f2d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f40cc8b0 0x7f51f40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f259c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cdf20 0x7f51f4151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f2d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f40cc8b0 0x7f51f40c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52422/0 (socket says 172.21.15.113:52422) 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f2d9d700 1 -- 172.21.15.113:0/4015995743 learned_addr learned my addr 172.21.15.113:0/4015995743 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f359e700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f40ce2f0 0x7f51f40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f2d9d700 1 -- 172.21.15.113:0/4015995743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cdf20 msgr2=0x7f51f4151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f2d9d700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cdf20 0x7f51f4151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f2d9d700 1 -- 172.21.15.113:0/4015995743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f40ce2f0 msgr2=0x7f51f40b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f2d9d700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f40ce2f0 0x7f51f40b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f2d9d700 1 -- 172.21.15.113:0/4015995743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec009ae0 con 0x7f51f40cc8b0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.071+0000 7f51f359e700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f40ce2f0 0x7f51f40b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.072+0000 7f51f2d9d700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f40cc8b0 0x7f51f40c6dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c7e0 tx=0x7f51e4007df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.072+0000 7f51daffd700 1 -- 172.21.15.113:0/4015995743 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4011590 con 0x7f51f40cc8b0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.072+0000 7f51daffd700 1 -- 172.21.15.113:0/4015995743 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e4011710 con 0x7f51f40cc8b0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.072+0000 7f51daffd700 1 -- 172.21.15.113:0/4015995743 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4011ad0 con 0x7f51f40cc8b0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.072+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f40b2a20 con 0x7f51f40cc8b0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.072+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f41522b0 con 0x7f51f40cc8b0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.072+0000 7f51daffd700 1 -- 172.21.15.113:0/4015995743 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f51e401c6c0 con 0x7f51f40cc8b0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.072+0000 7f51daffd700 1 -- 172.21.15.113:0/4015995743 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f51e4012930 con 0x7f51f40cc8b0 2024-01-11T07:06:44.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.073+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f404c3c0 con 0x7f51f40cc8b0 2024-01-11T07:06:44.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.075+0000 7f51daffd700 1 -- 172.21.15.113:0/4015995743 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f51e4017070 con 0x7f51f40cc8b0 2024-01-11T07:06:44.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.155+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f51f40b3610 con 0x7f51f40cc8b0 2024-01-11T07:06:44.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.159+0000 7f51daffd700 1 -- 172.21.15.113:0/4015995743 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f51f40b3610 con 0x7f51f40cc8b0 2024-01-11T07:06:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:4 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f40cc8b0 msgr2=0x7f51f40c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f40cc8b0 0x7f51f40c6dd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c7e0 tx=0x7f51e4007df0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 shutdown_connections 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51f40ce2f0 0x7f51f40b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51f40cc8b0 0x7f51f40c6dd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 --2- 172.21.15.113:0/4015995743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51f40cdf20 0x7f51f4151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 >> 172.21.15.113:0/4015995743 conn(0x7f51f406ed70 msgr2=0x7f51f40ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 shutdown_connections 2024-01-11T07:06:44.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.160+0000 7f51f912e700 1 -- 172.21.15.113:0/4015995743 wait complete. 2024-01-11T07:06:44.171 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2024-01-11T07:06:44.239 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:44.240+0000 7f28f79c21c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-11T07:06:44.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.289+0000 7f274217b700 1 Processor -- start 2024-01-11T07:06:44.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.289+0000 7f274217b700 1 -- start start 2024-01-11T07:06:44.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.289+0000 7f274217b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f273c0cb790 0x7f273c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.289+0000 7f274217b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cbb60 0x7f273c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.289+0000 7f274217b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f273c15e5a0 0x7f273c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.289+0000 7f274217b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c15b330 con 0x7f273c0cb790 2024-01-11T07:06:44.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.289+0000 7f274217b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c0ce700 con 0x7f273c15e5a0 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.289+0000 7f274217b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c15e110 con 0x7f273c0cbb60 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273bfff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f273c15e5a0 0x7f273c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cbb60 0x7f273c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cbb60 0x7f273c0ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46842/0 (socket says 172.21.15.113:46842) 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273bfff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f273c15e5a0 0x7f273c15a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52430/0 (socket says 172.21.15.113:52430) 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 -- 172.21.15.113:0/1606169237 learned_addr learned my addr 172.21.15.113:0/1606169237 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273b7fe700 1 --2- 172.21.15.113:0/1606169237 >> v2:172.21.15.7:3300/0 conn(0x7f273c0cb790 0x7f273c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 -- 172.21.15.113:0/1606169237 >> v2:172.21.15.113:3300/0 conn(0x7f273c15e5a0 msgr2=0x7f273c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 --2- 172.21.15.113:0/1606169237 >> v2:172.21.15.113:3300/0 conn(0x7f273c15e5a0 0x7f273c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 -- 172.21.15.113:0/1606169237 >> v2:172.21.15.7:3300/0 conn(0x7f273c0cb790 msgr2=0x7f273c0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:44.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 --2- 172.21.15.113:0/1606169237 >> v2:172.21.15.7:3300/0 conn(0x7f273c0cb790 0x7f273c0cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 -- 172.21.15.113:0/1606169237 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c15b050 con 0x7f273c0cbb60 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273b7fe700 1 --2- 172.21.15.113:0/1606169237 >> v2:172.21.15.7:3300/0 conn(0x7f273c0cb790 0x7f273c0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273affd700 1 --2- 172.21.15.113:0/1606169237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cbb60 0x7f273c0ce1c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f273c157e70 tx=0x7f2730009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7205ce27f178b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273a7fc700 1 -- 172.21.15.113:0/1606169237 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730009e70 con 0x7f273c0cbb60 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273a7fc700 1 -- 172.21.15.113:0/1606169237 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273000b860 con 0x7f273c0cbb60 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.290+0000 7f273a7fc700 1 -- 172.21.15.113:0/1606169237 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000bc00 con 0x7f273c0cbb60 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 -- 172.21.15.113:0/1606169237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cbb60 msgr2=0x7f273c0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 --2- 172.21.15.113:0/1606169237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cbb60 0x7f273c0ce1c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f273c157e70 tx=0x7f2730009250 comp rx=0 tx=0).stop 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 -- 172.21.15.113:0/1606169237 shutdown_connections 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 --2- 172.21.15.113:0/1606169237 >> v2:172.21.15.7:3300/0 conn(0x7f273c0cb790 0x7f273c0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 --2- 172.21.15.113:0/1606169237 >> v2:172.21.15.113:3300/0 conn(0x7f273c15e5a0 0x7f273c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 --2- 172.21.15.113:0/1606169237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cbb60 0x7f273c0ce1c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 -- 172.21.15.113:0/1606169237 >> 172.21.15.113:0/1606169237 conn(0x7f273c06ae00 msgr2=0x7f273c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 -- 172.21.15.113:0/1606169237 shutdown_connections 2024-01-11T07:06:44.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.291+0000 7f274217b700 1 -- 172.21.15.113:0/1606169237 wait complete. 2024-01-11T07:06:44.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.292+0000 7f274217b700 1 Processor -- start 2024-01-11T07:06:44.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.292+0000 7f274217b700 1 -- start start 2024-01-11T07:06:44.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f274217b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cb790 0x7f273c14f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f274217b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273c0cbb60 0x7f273c0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f274217b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f273c15e5a0 0x7f273c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f274217b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c14fb80 con 0x7f273c0cbb60 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f274217b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0b3240 con 0x7f273c15e5a0 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f274217b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0b2630 con 0x7f273c0cb790 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273b7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cb790 0x7f273c14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273b7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cb790 0x7f273c14f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46848/0 (socket says 172.21.15.113:46848) 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273b7fe700 1 -- 172.21.15.113:0/573555224 learned_addr learned my addr 172.21.15.113:0/573555224 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273bfff700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f273c15e5a0 0x7f273c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273affd700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273c0cbb60 0x7f273c0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273b7fe700 1 -- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f273c15e5a0 msgr2=0x7f273c0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273b7fe700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f273c15e5a0 0x7f273c0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273b7fe700 1 -- 172.21.15.113:0/573555224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273c0cbb60 msgr2=0x7f273c0b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273b7fe700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273c0cbb60 0x7f273c0b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273b7fe700 1 -- 172.21.15.113:0/573555224 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27300094f0 con 0x7f273c0cb790 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.293+0000 7f273bfff700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f273c15e5a0 0x7f273c0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.294+0000 7f273b7fe700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cb790 0x7f273c14f640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f272c00e8d0 tx=0x7f272c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.294+0000 7f27237fe700 1 -- 172.21.15.113:0/573555224 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c013410 con 0x7f273c0cb790 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.294+0000 7f27237fe700 1 -- 172.21.15.113:0/573555224 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c013570 con 0x7f273c0cb790 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.294+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c0b2910 con 0x7f273c0cb790 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.294+0000 7f27237fe700 1 -- 172.21.15.113:0/573555224 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c013910 con 0x7f273c0cb790 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.294+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c0b2ed0 con 0x7f273c0cb790 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.294+0000 7f27237fe700 1 -- 172.21.15.113:0/573555224 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f272c013d00 con 0x7f273c0cb790 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.294+0000 7f27237fe700 1 -- 172.21.15.113:0/573555224 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f272c00c3a0 con 0x7f273c0cb790 2024-01-11T07:06:44.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.295+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c04c3c0 con 0x7f273c0cb790 2024-01-11T07:06:44.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.298+0000 7f27237fe700 1 -- 172.21.15.113:0/573555224 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f272c020060 con 0x7f273c0cb790 2024-01-11T07:06:44.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.404+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f273c15dac0 con 0x7f273c0cb790 2024-01-11T07:06:44.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.410+0000 7f27237fe700 1 -- 172.21.15.113:0/573555224 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f272c00f040 con 0x7f273c0cb790 2024-01-11T07:06:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:5 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cb790 msgr2=0x7f273c14f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cb790 0x7f273c14f640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f272c00e8d0 tx=0x7f272c008040 comp rx=0 tx=0).stop 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 shutdown_connections 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f273c15e5a0 0x7f273c0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f273c0cb790 0x7f273c14f640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 --2- 172.21.15.113:0/573555224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273c0cbb60 0x7f273c0b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 >> 172.21.15.113:0/573555224 conn(0x7f273c06ae00 msgr2=0x7f273c067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 shutdown_connections 2024-01-11T07:06:44.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.411+0000 7f274217b700 1 -- 172.21.15.113:0/573555224 wait complete. 2024-01-11T07:06:44.422 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2024-01-11T07:06:44.480 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:44.480+0000 7f28f79c21c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-11T07:06:44.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.540+0000 7f63ce597700 1 Processor -- start 2024-01-11T07:06:44.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.540+0000 7f63ce597700 1 -- start start 2024-01-11T07:06:44.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ce597700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63c806ad10 0x7f63c8164c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ce597700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c815b860 con 0x7f63c80af7a0 2024-01-11T07:06:44.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ce597700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c815a9d0 con 0x7f63c80ada70 2024-01-11T07:06:44.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ce597700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c815b530 con 0x7f63c806ad10 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ccb34700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63c806ad10 0x7f63c8164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63c7fff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f63c80ada70 0x7f63c815a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33216/0 (socket says 172.21.15.113:33216) 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63c7fff700 1 -- 172.21.15.113:0/446582106 learned_addr learned my addr 172.21.15.113:0/446582106 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ccb34700 1 -- 172.21.15.113:0/446582106 >> v1:172.21.15.113:6789/0 conn(0x7f63c80ada70 legacy=0x7f63c815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ccb34700 1 -- 172.21.15.113:0/446582106 >> v1:172.21.15.7:6789/0 conn(0x7f63c80af7a0 legacy=0x7f63c806a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63c6ffd700 1 -- 172.21.15.113:0/446582106 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517909913 0 0) 0x7f63c815a9d0 con 0x7f63c80ada70 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ccb34700 1 -- 172.21.15.113:0/446582106 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c815ac30 con 0x7f63c806ad10 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.541+0000 7f63ccb34700 1 --2- 172.21.15.113:0/446582106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c806ad10 0x7f63c8164c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009e60 tx=0x7f63c0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d7c73793512fab5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63c6ffd700 1 -- 172.21.15.113:0/446582106 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c000e070 con 0x7f63c806ad10 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63c6ffd700 1 -- 172.21.15.113:0/446582106 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c000bc30 con 0x7f63c806ad10 2024-01-11T07:06:44.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63c6ffd700 1 -- 172.21.15.113:0/446582106 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0012430 con 0x7f63c806ad10 2024-01-11T07:06:44.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63ce597700 1 -- 172.21.15.113:0/446582106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c806ad10 msgr2=0x7f63c8164c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63ce597700 1 --2- 172.21.15.113:0/446582106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c806ad10 0x7f63c8164c60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009e60 tx=0x7f63c0009780 comp rx=0 tx=0).stop 2024-01-11T07:06:44.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63ce597700 1 -- 172.21.15.113:0/446582106 shutdown_connections 2024-01-11T07:06:44.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63ce597700 1 --2- 172.21.15.113:0/446582106 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c806ad10 0x7f63c8164c60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63ce597700 1 -- 172.21.15.113:0/446582106 >> 172.21.15.113:0/446582106 conn(0x7f63c80cd070 msgr2=0x7f63c80cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:44.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63ce597700 1 -- 172.21.15.113:0/446582106 shutdown_connections 2024-01-11T07:06:44.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.542+0000 7f63ce597700 1 -- 172.21.15.113:0/446582106 wait complete. 2024-01-11T07:06:44.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.543+0000 7f63ce597700 1 Processor -- start 2024-01-11T07:06:44.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ce597700 1 -- start start 2024-01-11T07:06:44.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ce597700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63c806ad10 0x7f63c80ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ce597700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63c80ada70 0x7f63c8154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ce597700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c80af7a0 0x7f63c8156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ce597700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c8155eb0 con 0x7f63c80ada70 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ce597700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c8155330 con 0x7f63c80af7a0 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ce597700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c8155610 con 0x7f63c806ad10 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ccb34700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c80af7a0 0x7f63c8156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ccb34700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c80af7a0 0x7f63c8156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52462/0 (socket says 172.21.15.113:52462) 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63c7fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63c806ad10 0x7f63c80ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63ccb34700 1 -- 172.21.15.113:0/3227150949 learned_addr learned my addr 172.21.15.113:0/3227150949 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.544+0000 7f63c77fe700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63c80ada70 0x7f63c8154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63ccb34700 1 -- 172.21.15.113:0/3227150949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63c806ad10 msgr2=0x7f63c80ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63ccb34700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63c806ad10 0x7f63c80ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63ccb34700 1 -- 172.21.15.113:0/3227150949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63c80ada70 msgr2=0x7f63c8154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63ccb34700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63c80ada70 0x7f63c8154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63ccb34700 1 -- 172.21.15.113:0/3227150949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c0009a20 con 0x7f63c80af7a0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63c7fff700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63c806ad10 0x7f63c80ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63ccb34700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c80af7a0 0x7f63c8156ab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63c815ab60 tx=0x7f63c0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63affff700 1 -- 172.21.15.113:0/3227150949 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c001b420 con 0x7f63c80af7a0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63affff700 1 -- 172.21.15.113:0/3227150949 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c001b580 con 0x7f63c80af7a0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c81558f0 con 0x7f63c80af7a0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63affff700 1 -- 172.21.15.113:0/3227150949 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c001b920 con 0x7f63c80af7a0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.545+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c8154eb0 con 0x7f63c80af7a0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.546+0000 7f63affff700 1 -- 172.21.15.113:0/3227150949 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f63c001bc90 con 0x7f63c80af7a0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.546+0000 7f63affff700 1 -- 172.21.15.113:0/3227150949 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f63c001ca00 con 0x7f63c80af7a0 2024-01-11T07:06:44.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.546+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c81550a0 con 0x7f63c80af7a0 2024-01-11T07:06:44.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.548+0000 7f63affff700 1 -- 172.21.15.113:0/3227150949 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f63c002a070 con 0x7f63c80af7a0 2024-01-11T07:06:44.588 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:44.589+0000 7ff30e1e61c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-11T07:06:44.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.633+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7f63c804c3c0 con 0x7f63c80af7a0 2024-01-11T07:06:44.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.640+0000 7f63affff700 1 -- 172.21.15.113:0/3227150949 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f63c002f440 con 0x7f63c80af7a0 2024-01-11T07:06:44.640 INFO:teuthology.orchestra.run.smithi113.stdout:6 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c80af7a0 msgr2=0x7f63c8156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c80af7a0 0x7f63c8156ab0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63c815ab60 tx=0x7f63c0019000 comp rx=0 tx=0).stop 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 shutdown_connections 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63c80af7a0 0x7f63c8156ab0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63c806ad10 0x7f63c80ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 --2- 172.21.15.113:0/3227150949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63c80ada70 0x7f63c8154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 >> 172.21.15.113:0/3227150949 conn(0x7f63c80cd070 msgr2=0x7f63c8153c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 shutdown_connections 2024-01-11T07:06:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.641+0000 7f63ce597700 1 -- 172.21.15.113:0/3227150949 wait complete. 2024-01-11T07:06:44.652 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2024-01-11T07:06:44.719 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:44.719+0000 7f28f79c21c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-11T07:06:44.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.771+0000 7f78f9afa700 1 Processor -- start 2024-01-11T07:06:44.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.771+0000 7f78f9afa700 1 -- start start 2024-01-11T07:06:44.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f9afa700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78f40ada70 0x7f78f415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f9afa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f406ad10 0x7f78f415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f9afa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f415b860 con 0x7f78f40af7a0 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f9afa700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f415b530 con 0x7f78f40ada70 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f9afa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f415ac30 con 0x7f78f406ad10 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f37fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78f40ada70 0x7f78f415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f3fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f406ad10 0x7f78f415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f37fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78f40ada70 0x7f78f415a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52478/0 (socket says 172.21.15.113:52478) 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f3fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f406ad10 0x7f78f415bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46858/0 (socket says 172.21.15.113:46858) 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f37fe700 1 -- 172.21.15.113:0/967568659 learned_addr learned my addr 172.21.15.113:0/967568659 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f37fe700 1 -- 172.21.15.113:0/967568659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f406ad10 msgr2=0x7f78f415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f37fe700 1 --2- 172.21.15.113:0/967568659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f406ad10 0x7f78f415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f37fe700 1 -- 172.21.15.113:0/967568659 >> v1:172.21.15.7:6789/0 conn(0x7f78f40af7a0 legacy=0x7f78f406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f37fe700 1 -- 172.21.15.113:0/967568659 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f415af10 con 0x7f78f40ada70 2024-01-11T07:06:44.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.772+0000 7f78f3fff700 1 --2- 172.21.15.113:0/967568659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f406ad10 0x7f78f415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f37fe700 1 --2- 172.21.15.113:0/967568659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40ada70 0x7f78f415a2c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f78e40099b0 tx=0x7f78e4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a559c06639688648 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f2ffd700 1 -- 172.21.15.113:0/967568659 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4009ea0 con 0x7f78f40ada70 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f2ffd700 1 -- 172.21.15.113:0/967568659 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e400b860 con 0x7f78f40ada70 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f2ffd700 1 -- 172.21.15.113:0/967568659 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e400bc00 con 0x7f78f40ada70 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f9afa700 1 -- 172.21.15.113:0/967568659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40ada70 msgr2=0x7f78f415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f9afa700 1 --2- 172.21.15.113:0/967568659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40ada70 0x7f78f415a2c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f78e40099b0 tx=0x7f78e4009250 comp rx=0 tx=0).stop 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f9afa700 1 -- 172.21.15.113:0/967568659 shutdown_connections 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f9afa700 1 --2- 172.21.15.113:0/967568659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f406ad10 0x7f78f415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f9afa700 1 --2- 172.21.15.113:0/967568659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40ada70 0x7f78f415a2c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.773+0000 7f78f9afa700 1 -- 172.21.15.113:0/967568659 >> 172.21.15.113:0/967568659 conn(0x7f78f40cd070 msgr2=0x7f78f40cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.774+0000 7f78f9afa700 1 -- 172.21.15.113:0/967568659 shutdown_connections 2024-01-11T07:06:44.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.774+0000 7f78f9afa700 1 -- 172.21.15.113:0/967568659 wait complete. 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.775+0000 7f78f9afa700 1 Processor -- start 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.775+0000 7f78f9afa700 1 -- start start 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f9afa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f406ad10 0x7f78f4140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f9afa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f40ada70 0x7f78f40c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f9afa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40af7a0 0x7f78f40c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f9afa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4154ca0 con 0x7f78f406ad10 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f9afa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40caf50 con 0x7f78f40af7a0 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f9afa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40ca340 con 0x7f78f40ada70 2024-01-11T07:06:44.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f40ada70 0x7f78f40c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f40ada70 0x7f78f40c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:46866/0 (socket says 172.21.15.113:46866) 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 -- 172.21.15.113:0/3808472091 learned_addr learned my addr 172.21.15.113:0/3808472091 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f37fe700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f406ad10 0x7f78f4140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f3fff700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40af7a0 0x7f78f40c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 -- 172.21.15.113:0/3808472091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40af7a0 msgr2=0x7f78f40c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40af7a0 0x7f78f40c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 -- 172.21.15.113:0/3808472091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f406ad10 msgr2=0x7f78f4140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f406ad10 0x7f78f4140670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 -- 172.21.15.113:0/3808472091 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009a20 con 0x7f78f40ada70 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78eaffd700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f40ada70 0x7f78f40c9760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c590 tx=0x7f78dc007f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78ebfff700 1 -- 172.21.15.113:0/3808472091 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc011590 con 0x7f78f40ada70 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78ebfff700 1 -- 172.21.15.113:0/3808472091 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc011710 con 0x7f78f40ada70 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e40094f0 con 0x7f78f40ada70 2024-01-11T07:06:44.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78ebfff700 1 -- 172.21.15.113:0/3808472091 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc011ad0 con 0x7f78f40ada70 2024-01-11T07:06:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.776+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f40ca900 con 0x7f78f40ada70 2024-01-11T07:06:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.777+0000 7f78ebfff700 1 -- 172.21.15.113:0/3808472091 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f78dc006aa0 con 0x7f78f40ada70 2024-01-11T07:06:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.777+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f404c3c0 con 0x7f78f40ada70 2024-01-11T07:06:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.777+0000 7f78ebfff700 1 -- 172.21.15.113:0/3808472091 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f78dc006d50 con 0x7f78f40ada70 2024-01-11T07:06:44.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.781+0000 7f78ebfff700 1 -- 172.21.15.113:0/3808472091 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f78dc017070 con 0x7f78f40ada70 2024-01-11T07:06:44.802 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:44.802+0000 7f28f79c21c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-11T07:06:44.827 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:44.827+0000 7ff30e1e61c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-11T07:06:44.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.865+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7f78f40cb100 con 0x7f78f40ada70 2024-01-11T07:06:44.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.870+0000 7f78ebfff700 1 -- 172.21.15.113:0/3808472091 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f78f40cb100 con 0x7f78f40ada70 2024-01-11T07:06:44.870 INFO:teuthology.orchestra.run.smithi113.stdout:7 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f40ada70 msgr2=0x7f78f40c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f40ada70 0x7f78f40c9760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c590 tx=0x7f78dc007f50 comp rx=0 tx=0).stop 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 shutdown_connections 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78f40af7a0 0x7f78f40c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78f406ad10 0x7f78f4140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 --2- 172.21.15.113:0/3808472091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78f40ada70 0x7f78f40c9760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 >> 172.21.15.113:0/3808472091 conn(0x7f78f40cd070 msgr2=0x7f78f4152de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 shutdown_connections 2024-01-11T07:06:44.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-01-11T07:06:44.871+0000 7f78f9afa700 1 -- 172.21.15.113:0/3808472091 wait complete. 2024-01-11T07:06:44.882 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-11T07:06:44.883 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-11T07:06:44.885 INFO:tasks.ceph.osd.0:Started 2024-01-11T07:06:44.885 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-11T07:06:44.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-11T07:06:44.887 INFO:tasks.ceph.osd.1:Started 2024-01-11T07:06:44.887 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-11T07:06:44.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-11T07:06:44.889 INFO:tasks.ceph.osd.2:Started 2024-01-11T07:06:44.889 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-11T07:06:44.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-11T07:06:44.892 INFO:tasks.ceph.osd.3:Started 2024-01-11T07:06:44.892 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-11T07:06:44.892 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-11T07:06:44.894 INFO:tasks.ceph.osd.4:Started 2024-01-11T07:06:44.894 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-11T07:06:44.894 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-11T07:06:44.896 INFO:tasks.ceph.osd.5:Started 2024-01-11T07:06:44.896 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-11T07:06:44.896 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-11T07:06:44.898 INFO:tasks.ceph.osd.6:Started 2024-01-11T07:06:44.898 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-11T07:06:44.898 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-11T07:06:44.900 INFO:tasks.ceph.osd.7:Started 2024-01-11T07:06:44.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:06:44.974 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-01-11T07:06:44.975+0000 7f762658b540 -1 Falling back to public interface 2024-01-11T07:06:44.975 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-01-11T07:06:44.976+0000 7f9fbe683540 -1 Falling back to public interface 2024-01-11T07:06:44.980 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-01-11T07:06:44.980+0000 7f96a60af540 -1 Falling back to public interface 2024-01-11T07:06:44.989 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-01-11T07:06:44.989+0000 7f9fbe683540 -1 osd.6 0 log_to_monitors true 2024-01-11T07:06:44.989 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-01-11T07:06:44.990+0000 7f762658b540 -1 osd.4 0 log_to_monitors true 2024-01-11T07:06:44.992 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-01-11T07:06:44.993+0000 7ff09bff1540 -1 Falling back to public interface 2024-01-11T07:06:44.998 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-01-11T07:06:44.999+0000 7f96a60af540 -1 osd.5 0 log_to_monitors true 2024-01-11T07:06:45.000 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-01-11T07:06:45.001+0000 7fef1737c540 -1 Falling back to public interface 2024-01-11T07:06:45.003 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-01-11T07:06:45.003+0000 7f73307e3540 -1 Falling back to public interface 2024-01-11T07:06:45.010 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-01-11T07:06:45.010+0000 7f88728fb540 -1 Falling back to public interface 2024-01-11T07:06:45.010 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-01-11T07:06:45.011+0000 7ff09bff1540 -1 osd.3 0 log_to_monitors true 2024-01-11T07:06:45.013 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-01-11T07:06:45.014+0000 7f81dd2ce540 -1 Falling back to public interface 2024-01-11T07:06:45.017 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-01-11T07:06:45.018+0000 7f73307e3540 -1 osd.7 0 log_to_monitors true 2024-01-11T07:06:45.019 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-01-11T07:06:45.019+0000 7fef1737c540 -1 osd.1 0 log_to_monitors true 2024-01-11T07:06:45.032 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-01-11T07:06:45.032+0000 7f81dd2ce540 -1 osd.2 0 log_to_monitors true 2024-01-11T07:06:45.032 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-01-11T07:06:45.033+0000 7f88728fb540 -1 osd.0 0 log_to_monitors true 2024-01-11T07:06:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.085+0000 7fc6e065f700 1 Processor -- start 2024-01-11T07:06:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.085+0000 7fc6e065f700 1 -- start start 2024-01-11T07:06:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.086+0000 7fc6e065f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d8159ca0 0x7fc6d806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.086+0000 7fc6e065f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d80d18c0 con 0x7fc6d815a070 2024-01-11T07:06:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.086+0000 7fc6e065f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d80d0010 con 0x7fc6d80cef00 2024-01-11T07:06:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.086+0000 7fc6e065f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80cf400 con 0x7fc6d8159ca0 2024-01-11T07:06:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.086+0000 7fc6ddbfa700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc6d815a070 0x7fc6d80ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40532/0 (socket says 172.21.15.7:40532) 2024-01-11T07:06:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6ddbfa700 1 -- 172.21.15.7:0/1634667486 learned_addr learned my addr 172.21.15.7:0/1634667486 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6de3fb700 1 --2- 172.21.15.7:0/1634667486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d8159ca0 0x7fc6d806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6dd3f9700 1 -- 172.21.15.7:0/1634667486 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121703121 0 0) 0x7fc6d80d18c0 con 0x7fc6d815a070 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6dd3f9700 1 -- 172.21.15.7:0/1634667486 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c0003570 con 0x7fc6d815a070 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6dd3f9700 1 -- 172.21.15.7:0/1634667486 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826774214 0 0) 0x7fc6d80d0010 con 0x7fc6d80cef00 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6dd3f9700 1 -- 172.21.15.7:0/1634667486 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d80d18c0 con 0x7fc6d80cef00 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6de3fb700 1 -- 172.21.15.7:0/1634667486 >> v1:172.21.15.113:6789/0 conn(0x7fc6d80cef00 legacy=0x7fc6d80b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6de3fb700 1 -- 172.21.15.7:0/1634667486 >> v1:172.21.15.7:6789/0 conn(0x7fc6d815a070 legacy=0x7fc6d80ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6dd3f9700 1 -- 172.21.15.7:0/1634667486 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3510286349 0 0) 0x7fc6c0003570 con 0x7fc6d815a070 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6de3fb700 1 -- 172.21.15.7:0/1634667486 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80cf6e0 con 0x7fc6d8159ca0 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6de3fb700 1 --2- 172.21.15.7:0/1634667486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d8159ca0 0x7fc6d806dc30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009e90 tx=0x7fc6c8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=122157bbcd88e498 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.087+0000 7fc6dd3f9700 1 -- 172.21.15.7:0/1634667486 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c800e070 con 0x7fc6d8159ca0 2024-01-11T07:06:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6dd3f9700 1 -- 172.21.15.7:0/1634667486 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c800b870 con 0x7fc6d8159ca0 2024-01-11T07:06:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6dd3f9700 1 -- 172.21.15.7:0/1634667486 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc001470 con 0x7fc6d8159ca0 2024-01-11T07:06:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6e065f700 1 -- 172.21.15.7:0/1634667486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d8159ca0 msgr2=0x7fc6d806dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6e065f700 1 --2- 172.21.15.7:0/1634667486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d8159ca0 0x7fc6d806dc30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009e90 tx=0x7fc6c8009780 comp rx=0 tx=0).stop 2024-01-11T07:06:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6e065f700 1 -- 172.21.15.7:0/1634667486 shutdown_connections 2024-01-11T07:06:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6e065f700 1 --2- 172.21.15.7:0/1634667486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d8159ca0 0x7fc6d806dc30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6e065f700 1 -- 172.21.15.7:0/1634667486 >> 172.21.15.7:0/1634667486 conn(0x7fc6d8063cc0 msgr2=0x7fc6d80d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6e065f700 1 -- 172.21.15.7:0/1634667486 shutdown_connections 2024-01-11T07:06:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.088+0000 7fc6e065f700 1 -- 172.21.15.7:0/1634667486 wait complete. 2024-01-11T07:06:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.090+0000 7fc6e065f700 1 Processor -- start 2024-01-11T07:06:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.090+0000 7fc6e065f700 1 -- start start 2024-01-11T07:06:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.090+0000 7fc6e065f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d80cef00 0x7fc6d80b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.090+0000 7fc6e065f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6d8159ca0 0x7fc6d80c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.090+0000 7fc6e065f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d815a070 0x7fc6d80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.090+0000 7fc6e065f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d816a750 con 0x7fc6d815a070 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.090+0000 7fc6e065f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d816aa30 con 0x7fc6d8159ca0 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.090+0000 7fc6e065f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d816ad10 con 0x7fc6d80cef00 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d80cef00 0x7fc6d80b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6debfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d815a070 0x7fc6d80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d80cef00 0x7fc6d80b64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:36478/0 (socket says 172.21.15.7:36478) 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6ddbfa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6d8159ca0 0x7fc6d80c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 -- 172.21.15.7:0/3618874266 learned_addr learned my addr 172.21.15.7:0/3618874266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6debfc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d815a070 0x7fc6d80b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45764/0 (socket says 172.21.15.7:45764) 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 -- 172.21.15.7:0/3618874266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6d8159ca0 msgr2=0x7fc6d80c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6d8159ca0 0x7fc6d80c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 -- 172.21.15.7:0/3618874266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d815a070 msgr2=0x7fc6d80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d815a070 0x7fc6d80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 -- 172.21.15.7:0/3618874266 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c8009a20 con 0x7fc6d80cef00 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6debfc700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d815a070 0x7fc6d80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6ddbfa700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6d8159ca0 0x7fc6d80c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6de3fb700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d80cef00 0x7fc6d80b64e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8068ad0 tx=0x7fc6c8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6d67fc700 1 -- 172.21.15.7:0/3618874266 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8012940 con 0x7fc6d80cef00 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6d67fc700 1 -- 172.21.15.7:0/3618874266 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c8012ac0 con 0x7fc6d80cef00 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d816aff0 con 0x7fc6d80cef00 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6d67fc700 1 -- 172.21.15.7:0/3618874266 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8012e80 con 0x7fc6d80cef00 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.091+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d8154940 con 0x7fc6d80cef00 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.092+0000 7fc6d67fc700 1 -- 172.21.15.7:0/3618874266 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc6c801e660 con 0x7fc6d80cef00 2024-01-11T07:06:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.092+0000 7fc6d67fc700 1 -- 172.21.15.7:0/3618874266 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fc6c8026ae0 con 0x7fc6d80cef00 2024-01-11T07:06:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.092+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d804c3c0 con 0x7fc6d80cef00 2024-01-11T07:06:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.095+0000 7fc6d67fc700 1 -- 172.21.15.7:0/3618874266 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc6c8016020 con 0x7fc6d80cef00 2024-01-11T07:06:45.130 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:45.130+0000 7ff30e1e61c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-11T07:06:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.169+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6d80cf9c0 con 0x7fc6d80cef00 2024-01-11T07:06:45.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.170+0000 7fc6d67fc700 1 -- 172.21.15.7:0/3618874266 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fc6c801ebb0 con 0x7fc6d80cef00 2024-01-11T07:06:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:45.169 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:44.867018+0000","last_up_change":"0.000000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:06:45.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d80cef00 msgr2=0x7fc6d80b64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d80cef00 0x7fc6d80b64e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8068ad0 tx=0x7fc6c8017000 comp rx=0 tx=0).stop 2024-01-11T07:06:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 shutdown_connections 2024-01-11T07:06:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d815a070 0x7fc6d80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc6d80cef00 0x7fc6d80b64e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 --2- 172.21.15.7:0/3618874266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6d8159ca0 0x7fc6d80c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 >> 172.21.15.7:0/3618874266 conn(0x7fc6d8063cc0 msgr2=0x7fc6d8155540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 shutdown_connections 2024-01-11T07:06:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:45.171+0000 7fc6e065f700 1 -- 172.21.15.7:0/3618874266 wait complete. 2024-01-11T07:06:45.183 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-11T07:06:45.184 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-11T07:06:45.217 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:45.218+0000 7ff30e1e61c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-11T07:06:45.524 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:45.525+0000 7f28f79c21c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-11T07:06:45.603 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:45.603+0000 7f28f79c21c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-11T07:06:45.732 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:45.732+0000 7f28f79c21c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-11T07:06:45.979 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:45.979+0000 7ff30e1e61c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-11T07:06:45.989 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-11T07:06:46.069 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:46.069+0000 7ff30e1e61c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-11T07:06:46.110 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.111+0000 7f4a001bb700 1 Processor -- start 2024-01-11T07:06:46.110 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.111+0000 7f4a001bb700 1 -- start start 2024-01-11T07:06:46.112 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f4a001bb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80c9020 0x7f49f80cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:46.112 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f4a001bb700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49f80aafd0 0x7f49f80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f4a001bb700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f806a7e0 con 0x7f49f80c93f0 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f4a001bb700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f80c86c0 con 0x7f49f80aafd0 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f4a001bb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8069ed0 con 0x7f49f80c9020 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f49fe758700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49f80aafd0 0x7f49f80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f49fdf57700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80c9020 0x7f49f80cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f49fd756700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f49f80c93f0 0x7f49f80aa8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:51434/0 (socket says 172.21.15.7:51434) 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f49fd756700 1 -- 172.21.15.7:0/1291424904 learned_addr learned my addr 172.21.15.7:0/1291424904 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f49fcf55700 1 -- 172.21.15.7:0/1291424904 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490443849 0 0) 0x7f49f806a7e0 con 0x7f49f80c93f0 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f49fcf55700 1 -- 172.21.15.7:0/1291424904 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e4003570 con 0x7f49f80c93f0 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f49fdf57700 1 -- 172.21.15.7:0/1291424904 >> v2:172.21.15.113:3300/0 conn(0x7f49f80aafd0 msgr2=0x7f49f80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.112+0000 7f49fdf57700 1 --2- 172.21.15.7:0/1291424904 >> v2:172.21.15.113:3300/0 conn(0x7f49f80aafd0 0x7f49f80cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f49fdf57700 1 -- 172.21.15.7:0/1291424904 >> v1:172.21.15.113:6789/0 conn(0x7f49f80c93f0 legacy=0x7f49f80aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f49fdf57700 1 -- 172.21.15.7:0/1291424904 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f80cb220 con 0x7f49f80c9020 2024-01-11T07:06:46.113 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f49fe758700 1 --2- 172.21.15.7:0/1291424904 >> v2:172.21.15.113:3300/0 conn(0x7f49f80aafd0 0x7f49f80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:46.114 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f49fdf57700 1 --2- 172.21.15.7:0/1291424904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80c9020 0x7f49f80cac40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f49f40098e0 tx=0x7f49f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6257f958f2080a33 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:46.114 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f49fcf55700 1 -- 172.21.15.7:0/1291424904 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4011630 con 0x7f49f80c9020 2024-01-11T07:06:46.114 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f49fcf55700 1 -- 172.21.15.7:0/1291424904 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f4011790 con 0x7f49f80c9020 2024-01-11T07:06:46.114 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f49fcf55700 1 -- 172.21.15.7:0/1291424904 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4011b30 con 0x7f49f80c9020 2024-01-11T07:06:46.116 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f4a001bb700 1 -- 172.21.15.7:0/1291424904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80c9020 msgr2=0x7f49f80cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:46.116 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.113+0000 7f4a001bb700 1 --2- 172.21.15.7:0/1291424904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80c9020 0x7f49f80cac40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f49f40098e0 tx=0x7f49f40091a0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.114+0000 7f4a001bb700 1 -- 172.21.15.7:0/1291424904 shutdown_connections 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.114+0000 7f4a001bb700 1 --2- 172.21.15.7:0/1291424904 >> v2:172.21.15.113:3300/0 conn(0x7f49f80aafd0 0x7f49f80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.114+0000 7f4a001bb700 1 --2- 172.21.15.7:0/1291424904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80c9020 0x7f49f80cac40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.114+0000 7f4a001bb700 1 -- 172.21.15.7:0/1291424904 >> 172.21.15.7:0/1291424904 conn(0x7f49f80cd070 msgr2=0x7f49f80cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.114+0000 7f4a001bb700 1 -- 172.21.15.7:0/1291424904 shutdown_connections 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.114+0000 7f4a001bb700 1 -- 172.21.15.7:0/1291424904 wait complete. 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.115+0000 7f4a001bb700 1 Processor -- start 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.115+0000 7f4a001bb700 1 -- start start 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f4a001bb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80aafd0 0x7f49f80c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f4a001bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f80c9020 0x7f49f814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f4a001bb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49f80c93f0 0x7f49f80c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f4a001bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80b1330 con 0x7f49f80c9020 2024-01-11T07:06:46.117 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f4a001bb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f816db90 con 0x7f49f80c93f0 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f4a001bb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f816de70 con 0x7f49f80aafd0 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49f80c93f0 0x7f49f80c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fd756700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f80c9020 0x7f49f814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fdf57700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80aafd0 0x7f49f80c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49f80c93f0 0x7f49f80c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:48462/0 (socket says 172.21.15.7:48462) 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 -- 172.21.15.7:0/212891533 learned_addr learned my addr 172.21.15.7:0/212891533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fd756700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f80c9020 0x7f49f814da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45766/0 (socket says 172.21.15.7:45766) 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 -- 172.21.15.7:0/212891533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80aafd0 msgr2=0x7f49f80c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80aafd0 0x7f49f80c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 -- 172.21.15.7:0/212891533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f80c9020 msgr2=0x7f49f814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f80c9020 0x7f49f814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 -- 172.21.15.7:0/212891533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0009a20 con 0x7f49f80c93f0 2024-01-11T07:06:46.118 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fdf57700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80aafd0 0x7f49f80c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49fe758700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49f80c93f0 0x7f49f80c2de0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009f60 tx=0x7f49f000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.116+0000 7f49edffb700 1 -- 172.21.15.7:0/212891533 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f000cb70 con 0x7f49f80c93f0 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.117+0000 7f49edffb700 1 -- 172.21.15.7:0/212891533 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f000f040 con 0x7f49f80c93f0 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.117+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f4009420 con 0x7f49f80c93f0 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.117+0000 7f49edffb700 1 -- 172.21.15.7:0/212891533 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0014480 con 0x7f49f80c93f0 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.117+0000 7f49edffb700 1 -- 172.21.15.7:0/212891533 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f49f0003020 con 0x7f49f80c93f0 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.117+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f816e430 con 0x7f49f80c93f0 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.117+0000 7f49edffb700 1 -- 172.21.15.7:0/212891533 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1879+0+0 (secure 0 0 0) 0x7f49f0014c50 con 0x7f49f80c93f0 2024-01-11T07:06:46.119 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.118+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f804c3c0 con 0x7f49f80c93f0 2024-01-11T07:06:46.120 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.120+0000 7f49edffb700 1 -- 172.21.15.7:0/212891533 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f49f0018030 con 0x7f49f80c93f0 2024-01-11T07:06:46.194 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.195+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49f80b1470 con 0x7f49f80c93f0 2024-01-11T07:06:46.195 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.196+0000 7f49edffb700 1 -- 172.21.15.7:0/212891533 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f49f0014e10 con 0x7f49f80c93f0 2024-01-11T07:06:46.195 INFO:teuthology.misc.health.smithi007.stdout: 2024-01-11T07:06:46.195 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":11,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:45.868078+0000","last_up_change":"0.000000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:06:46.196 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49f80c93f0 msgr2=0x7f49f80c2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:46.197 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49f80c93f0 0x7f49f80c2de0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009f60 tx=0x7f49f000b040 comp rx=0 tx=0).stop 2024-01-11T07:06:46.197 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 shutdown_connections 2024-01-11T07:06:46.197 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49f80c93f0 0x7f49f80c2de0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.197 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49f80aafd0 0x7f49f80c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.197 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 --2- 172.21.15.7:0/212891533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49f80c9020 0x7f49f814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:46.197 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 >> 172.21.15.7:0/212891533 conn(0x7f49f80cd070 msgr2=0x7f49f815bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:46.197 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 shutdown_connections 2024-01-11T07:06:46.197 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:46.197+0000 7f4a001bb700 1 -- 172.21.15.7:0/212891533 wait complete. 2024-01-11T07:06:46.208 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-11T07:06:46.221 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:46.222+0000 7ff30e1e61c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-11T07:06:46.274 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:46.274+0000 7f28f79c21c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-11T07:06:46.446 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:46.446+0000 7f28f79c21c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-11T07:06:46.605 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:46.605+0000 7f28f79c21c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-11T07:06:46.853 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:46.854+0000 7ff30e1e61c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-11T07:06:46.888 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-01-11T07:06:46.889+0000 7f761e1a6700 -1 osd.4 0 waiting for initial osdmap 2024-01-11T07:06:46.889 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-01-11T07:06:46.889+0000 7f9fb5a9d700 -1 osd.6 0 waiting for initial osdmap 2024-01-11T07:06:46.889 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-01-11T07:06:46.890+0000 7f7327bfd700 -1 osd.7 0 waiting for initial osdmap 2024-01-11T07:06:46.890 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-01-11T07:06:46.891+0000 7f761779e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T07:06:46.890 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-01-11T07:06:46.891+0000 7f9faf896700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T07:06:46.891 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-01-11T07:06:46.891+0000 7f73219f6700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T07:06:46.894 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-01-11T07:06:46.895+0000 7f969d4c9700 -1 osd.5 0 waiting for initial osdmap 2024-01-11T07:06:46.896 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-01-11T07:06:46.896+0000 7f96972c2700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T07:06:46.897 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-01-11T07:06:46.897+0000 7f8869d15700 -1 osd.0 0 waiting for initial osdmap 2024-01-11T07:06:46.899 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-01-11T07:06:46.899+0000 7f8863b0e700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T07:06:46.899 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-01-11T07:06:46.900+0000 7f81d46e8700 -1 osd.2 0 waiting for initial osdmap 2024-01-11T07:06:46.899 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-01-11T07:06:46.900+0000 7ff09340b700 -1 osd.3 0 waiting for initial osdmap 2024-01-11T07:06:46.901 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-01-11T07:06:46.901+0000 7f81ce4e1700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T07:06:46.901 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-01-11T07:06:46.901+0000 7ff08d204700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T07:06:46.901 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-01-11T07:06:46.902+0000 7fef0e796700 -1 osd.1 0 waiting for initial osdmap 2024-01-11T07:06:46.902 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-01-11T07:06:46.903+0000 7fef0858f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-11T07:06:47.037 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:47.037+0000 7ff30e1e61c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-11T07:06:47.219 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:47.220+0000 7ff30e1e61c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-11T07:06:47.389 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:47.390+0000 7f28f79c21c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-11T07:06:47.601 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:47.601+0000 7f28f79c21c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-11T07:06:47.670 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:47.671+0000 7f28f79c21c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-11T07:06:47.826 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:47.827+0000 7f28f79c21c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-11T07:06:48.121 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:48.122+0000 7ff30e1e61c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-11T07:06:48.362 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:48.363+0000 7ff30e1e61c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-11T07:06:48.442 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:48.443+0000 7ff30e1e61c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-11T07:06:48.620 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-01-11T07:06:48.620+0000 7ff30e1e61c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-11T07:06:48.844 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:48.845+0000 7f28b647f700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-11T07:06:48.847 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-01-11T07:06:48.848+0000 7f28b647f700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-11T07:06:53.015 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-11T07:06:53.145 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.146+0000 7fb018d56700 1 Processor -- start 2024-01-11T07:06:53.146 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.146+0000 7fb018d56700 1 -- start start 2024-01-11T07:06:53.149 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb018d56700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0140cd160 0x7fb0141571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.149 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb018d56700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0140c9530 0x7fb0140c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.149 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb018d56700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb01406dbc0 0x7fb01406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.149 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb018d56700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01415a830 con 0x7fb01406dbc0 2024-01-11T07:06:53.149 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb018d56700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01406e540 con 0x7fb0140c9530 2024-01-11T07:06:53.149 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb018d56700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0141577d0 con 0x7fb0140cd160 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb012d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb01406dbc0 0x7fb01406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb012d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb01406dbc0 0x7fb01406e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54714/0 (socket says 172.21.15.7:54714) 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb012d9d700 1 -- 172.21.15.7:0/2302247290 learned_addr learned my addr 172.21.15.7:0/2302247290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb011d9b700 1 --2- 172.21.15.7:0/2302247290 >> v2:172.21.15.113:3300/0 conn(0x7fb0140c9530 0x7fb0140c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb01259c700 1 --2- 172.21.15.7:0/2302247290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0140cd160 0x7fb0141571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb012d9d700 1 -- 172.21.15.7:0/2302247290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0140cd160 msgr2=0x7fb0141571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb012d9d700 1 --2- 172.21.15.7:0/2302247290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0140cd160 0x7fb0141571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb012d9d700 1 -- 172.21.15.7:0/2302247290 >> v2:172.21.15.113:3300/0 conn(0x7fb0140c9530 msgr2=0x7fb0140c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb012d9d700 1 --2- 172.21.15.7:0/2302247290 >> v2:172.21.15.113:3300/0 conn(0x7fb0140c9530 0x7fb0140c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.147+0000 7fb012d9d700 1 -- 172.21.15.7:0/2302247290 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb014157ab0 con 0x7fb01406dbc0 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb012d9d700 1 --2- 172.21.15.7:0/2302247290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01406dbc0 0x7fb01406e000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb014069a00 tx=0x7fb00c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb68f1453fcfd08d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb01159a700 1 -- 172.21.15.7:0/2302247290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00e070 con 0x7fb01406dbc0 2024-01-11T07:06:53.150 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb01159a700 1 -- 172.21.15.7:0/2302247290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c00ba50 con 0x7fb01406dbc0 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb01159a700 1 -- 172.21.15.7:0/2302247290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0092e0 con 0x7fb01406dbc0 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 -- 172.21.15.7:0/2302247290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01406dbc0 msgr2=0x7fb01406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 --2- 172.21.15.7:0/2302247290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01406dbc0 0x7fb01406e000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb014069a00 tx=0x7fb00c009840 comp rx=0 tx=0).stop 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 -- 172.21.15.7:0/2302247290 shutdown_connections 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 --2- 172.21.15.7:0/2302247290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb01406dbc0 0x7fb01406e000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 --2- 172.21.15.7:0/2302247290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0140cd160 0x7fb0141571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 --2- 172.21.15.7:0/2302247290 >> v2:172.21.15.113:3300/0 conn(0x7fb0140c9530 0x7fb0140c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 -- 172.21.15.7:0/2302247290 >> 172.21.15.7:0/2302247290 conn(0x7fb0140cd8e0 msgr2=0x7fb0140cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 -- 172.21.15.7:0/2302247290 shutdown_connections 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.148+0000 7fb018d56700 1 -- 172.21.15.7:0/2302247290 wait complete. 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb018d56700 1 Processor -- start 2024-01-11T07:06:53.151 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb018d56700 1 -- start start 2024-01-11T07:06:53.154 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb018d56700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb01406dbc0 0x7fb0140b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.154 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb018d56700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0140c9530 0x7fb014154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.154 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb018d56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0140cd160 0x7fb014156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb018d56700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb014157510 con 0x7fb0140cd160 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb018d56700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140b16b0 con 0x7fb0140c9530 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb018d56700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140b1990 con 0x7fb01406dbc0 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb012d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0140cd160 0x7fb014156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb012d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0140cd160 0x7fb014156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54718/0 (socket says 172.21.15.7:54718) 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.150+0000 7fb012d9d700 1 -- 172.21.15.7:0/3310227165 learned_addr learned my addr 172.21.15.7:0/3310227165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb01259c700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb01406dbc0 0x7fb0140b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb011d9b700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0140c9530 0x7fb014154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb012d9d700 1 -- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb01406dbc0 msgr2=0x7fb0140b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb012d9d700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb01406dbc0 0x7fb0140b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb012d9d700 1 -- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0140c9530 msgr2=0x7fb014154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.155 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb012d9d700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0140c9530 0x7fb014154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb012d9d700 1 -- 172.21.15.7:0/3310227165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c009ae0 con 0x7fb0140cd160 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb012d9d700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0140cd160 0x7fb014156fd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb014069990 tx=0x7fb00c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb0027fc700 1 -- 172.21.15.7:0/3310227165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0032f0 con 0x7fb0140cd160 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb0027fc700 1 -- 172.21.15.7:0/3310227165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c008d20 con 0x7fb0140cd160 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb0027fc700 1 -- 172.21.15.7:0/3310227165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0256b0 con 0x7fb0140cd160 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0140b1c70 con 0x7fb0140cd160 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.151+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0140b2230 con 0x7fb0140cd160 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.152+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb014064a40 con 0x7fb0140cd160 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.153+0000 7fb0027fc700 1 -- 172.21.15.7:0/3310227165 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7fb00c003040 con 0x7fb0140cd160 2024-01-11T07:06:53.156 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.153+0000 7fb0027fc700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faff4066ea0 0x7faff4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.157 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.153+0000 7fb0027fc700 1 -- 172.21.15.7:0/3310227165 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb00c085690 con 0x7fb0140cd160 2024-01-11T07:06:53.157 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.153+0000 7fb01259c700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faff4066ea0 0x7faff4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.157 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.154+0000 7fb01259c700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faff4066ea0 0x7faff4069350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7fb004008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.157 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.158+0000 7fb0027fc700 1 -- 172.21.15.7:0/3310227165 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb00c052d40 con 0x7fb0140cd160 2024-01-11T07:06:53.362 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.362+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb01404c3c0 con 0x7fb0140cd160 2024-01-11T07:06:53.362 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.363+0000 7fb0027fc700 1 -- 172.21.15.7:0/3310227165 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12079 (secure 0 0 0) 0x7fb00c009940 con 0x7fb0140cd160 2024-01-11T07:06:53.363 INFO:teuthology.misc.health.smithi007.stdout: 2024-01-11T07:06:53.363 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":18,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:52.929534+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:06:53.365 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faff4066ea0 msgr2=0x7faff4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.365 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faff4066ea0 0x7faff4069350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7fb004008040 comp rx=0 tx=0).stop 2024-01-11T07:06:53.365 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0140cd160 msgr2=0x7fb014156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.365 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0140cd160 0x7fb014156fd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb014069990 tx=0x7fb00c018000 comp rx=0 tx=0).stop 2024-01-11T07:06:53.365 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 shutdown_connections 2024-01-11T07:06:53.365 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faff4066ea0 0x7faff4069350 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.365 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0140cd160 0x7fb014156fd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.366 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb01406dbc0 0x7fb0140b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.366 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 --2- 172.21.15.7:0/3310227165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0140c9530 0x7fb014154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.366 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 >> 172.21.15.7:0/3310227165 conn(0x7fb0140cd8e0 msgr2=0x7fb01406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:53.366 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 shutdown_connections 2024-01-11T07:06:53.366 INFO:teuthology.misc.health.smithi007.stderr:2024-01-11T07:06:53.365+0000 7fb018d56700 1 -- 172.21.15.7:0/3310227165 wait complete. 2024-01-11T07:06:53.377 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-11T07:06:53.377 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-11T07:06:53.377 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:06:53.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:06:53.380 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:06:53.380 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:06:53.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:06:53.382 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:06:53.383 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:06:53.383 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:06:53.385 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:06:53.385 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:06:53.385 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:06:53.387 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:06:53.387 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-11T07:06:53.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:06:53.467 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:06:53.467 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:06:53.470 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:06:53.470 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:06:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.528+0000 7f4291844700 1 Processor -- start 2024-01-11T07:06:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.528+0000 7f4291844700 1 -- start start 2024-01-11T07:06:53.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f4291844700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f428c0aed00 0x7f428c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f4291844700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f428c156e90 0x7f428c069d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f4291844700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f428c158010 con 0x7f428c06a240 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f4291844700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f428c06a6e0 con 0x7f428c0aed00 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f4291844700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f428c15b530 con 0x7f428c156e90 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f428affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f428c0aed00 0x7f428c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f428b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f428c06a240 0x7f428c0c91b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34532/0 (socket says 172.21.15.7:34532) 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f428affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f428c0aed00 0x7f428c15aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54748/0 (socket says 172.21.15.7:54748) 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f428affd700 1 -- 172.21.15.7:0/3547361053 learned_addr learned my addr 172.21.15.7:0/3547361053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.529+0000 7f428a7fc700 1 --2- 172.21.15.7:0/3547361053 >> v2:172.21.15.113:3300/0 conn(0x7f428c156e90 0x7f428c069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f428affd700 1 -- 172.21.15.7:0/3547361053 >> v2:172.21.15.113:3300/0 conn(0x7f428c156e90 msgr2=0x7f428c069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f428affd700 1 --2- 172.21.15.7:0/3547361053 >> v2:172.21.15.113:3300/0 conn(0x7f428c156e90 0x7f428c069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f428affd700 1 -- 172.21.15.7:0/3547361053 >> v1:172.21.15.7:6789/0 conn(0x7f428c06a240 legacy=0x7f428c0c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f428affd700 1 -- 172.21.15.7:0/3547361053 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c157400 con 0x7f428c0aed00 2024-01-11T07:06:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f4289ffb700 1 -- 172.21.15.7:0/3547361053 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321223925 0 0) 0x7f428c158010 con 0x7f428c06a240 2024-01-11T07:06:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f428affd700 1 --2- 172.21.15.7:0/3547361053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c15aff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f428c06abe0 tx=0x7f427c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce988dd1c8c6f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f4289ffb700 1 -- 172.21.15.7:0/3547361053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c012770 con 0x7f428c0aed00 2024-01-11T07:06:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f4289ffb700 1 -- 172.21.15.7:0/3547361053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c0128d0 con 0x7f428c0aed00 2024-01-11T07:06:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.530+0000 7f4289ffb700 1 -- 172.21.15.7:0/3547361053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c012c70 con 0x7f428c0aed00 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.531+0000 7f4291844700 1 -- 172.21.15.7:0/3547361053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 msgr2=0x7f428c15aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.531+0000 7f4291844700 1 --2- 172.21.15.7:0/3547361053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c15aff0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f428c06abe0 tx=0x7f427c009250 comp rx=0 tx=0).stop 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.531+0000 7f4291844700 1 -- 172.21.15.7:0/3547361053 shutdown_connections 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.531+0000 7f4291844700 1 --2- 172.21.15.7:0/3547361053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c15aff0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.531+0000 7f4291844700 1 --2- 172.21.15.7:0/3547361053 >> v2:172.21.15.113:3300/0 conn(0x7f428c156e90 0x7f428c069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.531+0000 7f4291844700 1 -- 172.21.15.7:0/3547361053 >> 172.21.15.7:0/3547361053 conn(0x7f428c063230 msgr2=0x7f428c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.531+0000 7f4291844700 1 -- 172.21.15.7:0/3547361053 shutdown_connections 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.531+0000 7f4291844700 1 -- 172.21.15.7:0/3547361053 wait complete. 2024-01-11T07:06:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.532+0000 7f4291844700 1 Processor -- start 2024-01-11T07:06:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.532+0000 7f4291844700 1 -- start start 2024-01-11T07:06:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f4291844700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c06a240 0x7f428c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f4291844700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f4291844700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c156e90 0x7f428c13ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f4291844700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c151e00 con 0x7f428c0aed00 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f4291844700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c1590f0 con 0x7f428c06a240 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f4291844700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c166160 con 0x7f428c156e90 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c165c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54754/0 (socket says 172.21.15.7:54754) 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428a7fc700 1 -- 172.21.15.7:0/2849524994 learned_addr learned my addr 172.21.15.7:0/2849524994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428affd700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c06a240 0x7f428c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428b7fe700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c156e90 0x7f428c13ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428a7fc700 1 -- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c156e90 msgr2=0x7f428c13ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428a7fc700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c156e90 0x7f428c13ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428a7fc700 1 -- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c06a240 msgr2=0x7f428c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.533+0000 7f428a7fc700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c06a240 0x7f428c158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f428a7fc700 1 -- 172.21.15.7:0/2849524994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0094f0 con 0x7f428c0aed00 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f428affd700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c06a240 0x7f428c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f428b7fe700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c156e90 0x7f428c13ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f428a7fc700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c165c20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f427400ecb0 tx=0x7f42740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f4282ffd700 1 -- 172.21.15.7:0/2849524994 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427400c9d0 con 0x7f428c0aed00 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f4282ffd700 1 -- 172.21.15.7:0/2849524994 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427400cb30 con 0x7f428c0aed00 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f4282ffd700 1 -- 172.21.15.7:0/2849524994 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427400ced0 con 0x7f428c0aed00 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c0c1710 con 0x7f428c0aed00 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.534+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c166400 con 0x7f428c0aed00 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.535+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c04c3c0 con 0x7f428c0aed00 2024-01-11T07:06:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.535+0000 7f4282ffd700 1 -- 172.21.15.7:0/2849524994 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7f4274013680 con 0x7f428c0aed00 2024-01-11T07:06:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.536+0000 7f4282ffd700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f426c066ea0 0x7f426c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.536+0000 7f4282ffd700 1 -- 172.21.15.7:0/2849524994 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4274084480 con 0x7f428c0aed00 2024-01-11T07:06:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.536+0000 7f428affd700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f426c066ea0 0x7f426c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.536+0000 7f428affd700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f426c066ea0 0x7f426c069350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f428c06abe0 tx=0x7f427c01b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.540+0000 7f4282ffd700 1 -- 172.21.15.7:0/2849524994 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4274051b20 con 0x7f428c0aed00 2024-01-11T07:06:53.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.747+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f428c0ca8f0 con 0x7f428c0aed00 2024-01-11T07:06:53.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.747+0000 7f4282ffd700 1 -- 172.21.15.7:0/2849524994 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12079 (secure 0 0 0) 0x7f4274055140 con 0x7f428c0aed00 2024-01-11T07:06:53.747 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:53.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:52.929534+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:06:53.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f426c066ea0 msgr2=0x7f426c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f426c066ea0 0x7f426c069350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f428c06abe0 tx=0x7f427c01b000 comp rx=0 tx=0).stop 2024-01-11T07:06:53.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 msgr2=0x7f428c165c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c165c20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f427400ecb0 tx=0x7f42740091a0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 shutdown_connections 2024-01-11T07:06:53.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f426c066ea0 0x7f426c069350 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c156e90 0x7f428c13ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c06a240 0x7f428c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.749+0000 7f4291844700 1 --2- 172.21.15.7:0/2849524994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c0aed00 0x7f428c165c20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.750+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 >> 172.21.15.7:0/2849524994 conn(0x7f428c063230 msgr2=0x7f428c0c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:53.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.750+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 shutdown_connections 2024-01-11T07:06:53.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.750+0000 7f4291844700 1 -- 172.21.15.7:0/2849524994 wait complete. 2024-01-11T07:06:53.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:06:53.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:06:53.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.901+0000 7ff29db77700 1 Processor -- start 2024-01-11T07:06:53.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.901+0000 7ff29db77700 1 -- start start 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.901+0000 7ff29db77700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2980b30d0 0x7ff2980c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.901+0000 7ff29db77700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2980ca080 con 0x7ff298157bc0 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.901+0000 7ff29db77700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2980ca360 con 0x7ff298157f90 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff29db77700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2980ca640 con 0x7ff2980b30d0 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff2977fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff298157bc0 0x7ff2980cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34534/0 (socket says 172.21.15.7:34534) 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff2977fe700 1 -- 172.21.15.7:0/2859621182 learned_addr learned my addr 172.21.15.7:0/2859621182 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff297fff700 1 --2- 172.21.15.7:0/2859621182 >> v2:172.21.15.113:3300/0 conn(0x7ff2980b30d0 0x7ff2980c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff2967fc700 1 -- 172.21.15.7:0/2859621182 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792955833 0 0) 0x7ff2980ca080 con 0x7ff298157bc0 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff2967fc700 1 -- 172.21.15.7:0/2859621182 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff280003570 con 0x7ff298157bc0 2024-01-11T07:06:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff2967fc700 1 -- 172.21.15.7:0/2859621182 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665554690 0 0) 0x7ff2980ca360 con 0x7ff298157f90 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff2967fc700 1 -- 172.21.15.7:0/2859621182 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2980ca080 con 0x7ff298157f90 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff297fff700 1 -- 172.21.15.7:0/2859621182 >> v1:172.21.15.113:6789/0 conn(0x7ff298157f90 legacy=0x7ff2980c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff297fff700 1 -- 172.21.15.7:0/2859621182 >> v1:172.21.15.7:6789/0 conn(0x7ff298157bc0 legacy=0x7ff2980cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff2967fc700 1 -- 172.21.15.7:0/2859621182 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2167167269 0 0) 0x7ff280003570 con 0x7ff298157bc0 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff297fff700 1 -- 172.21.15.7:0/2859621182 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980b18c0 con 0x7ff2980b30d0 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.902+0000 7ff297fff700 1 --2- 172.21.15.7:0/2859621182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2980b30d0 0x7ff2980c9670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff290009e60 tx=0x7ff290009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9488403da8446e2d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff2967fc700 1 -- 172.21.15.7:0/2859621182 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29000e070 con 0x7ff2980b30d0 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff2967fc700 1 -- 172.21.15.7:0/2859621182 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29000b7a0 con 0x7ff2980b30d0 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff2967fc700 1 -- 172.21.15.7:0/2859621182 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880015f0 con 0x7ff2980b30d0 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff29db77700 1 -- 172.21.15.7:0/2859621182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2980b30d0 msgr2=0x7ff2980c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff29db77700 1 --2- 172.21.15.7:0/2859621182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2980b30d0 0x7ff2980c9670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff290009e60 tx=0x7ff290009780 comp rx=0 tx=0).stop 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff29db77700 1 -- 172.21.15.7:0/2859621182 shutdown_connections 2024-01-11T07:06:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff29db77700 1 --2- 172.21.15.7:0/2859621182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2980b30d0 0x7ff2980c9670 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff29db77700 1 -- 172.21.15.7:0/2859621182 >> 172.21.15.7:0/2859621182 conn(0x7ff298097a50 msgr2=0x7ff2980ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:53.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff29db77700 1 -- 172.21.15.7:0/2859621182 shutdown_connections 2024-01-11T07:06:53.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.903+0000 7ff29db77700 1 -- 172.21.15.7:0/2859621182 wait complete. 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff29db77700 1 Processor -- start 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff29db77700 1 -- start start 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff29db77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2980b30d0 0x7ff2980c2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff29db77700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff298157bc0 0x7ff2980c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff29db77700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff298157f90 0x7ff2980c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff29db77700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980b1ca0 con 0x7ff2980b30d0 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff29db77700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980c3610 con 0x7ff298157bc0 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff29db77700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980c2a00 con 0x7ff298157f90 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.905+0000 7ff2977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2980b30d0 0x7ff2980c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff2977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2980b30d0 0x7ff2980c2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54762/0 (socket says 172.21.15.7:54762) 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff2977fe700 1 -- 172.21.15.7:0/674386726 learned_addr learned my addr 172.21.15.7:0/674386726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff297fff700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff298157f90 0x7ff2980c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff296ffd700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff298157bc0 0x7ff2980c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff2977fe700 1 -- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff298157f90 msgr2=0x7ff2980c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff2977fe700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff298157f90 0x7ff2980c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff2977fe700 1 -- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff298157bc0 msgr2=0x7ff2980c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff2977fe700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff298157bc0 0x7ff2980c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff2977fe700 1 -- 172.21.15.7:0/674386726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff290009a20 con 0x7ff2980b30d0 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff296ffd700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff298157bc0 0x7ff2980c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff297fff700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff298157f90 0x7ff2980c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff2977fe700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2980b30d0 0x7ff2980c2420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff28800cb70 tx=0x7ff288002bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff27f7fe700 1 -- 172.21.15.7:0/674386726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288007300 con 0x7ff2980b30d0 2024-01-11T07:06:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff27f7fe700 1 -- 172.21.15.7:0/674386726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288007480 con 0x7ff2980b30d0 2024-01-11T07:06:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2980c2ce0 con 0x7ff2980b30d0 2024-01-11T07:06:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff27f7fe700 1 -- 172.21.15.7:0/674386726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880057f0 con 0x7ff2980b30d0 2024-01-11T07:06:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.906+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2980c5300 con 0x7ff2980b30d0 2024-01-11T07:06:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.907+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29804c3c0 con 0x7ff2980b30d0 2024-01-11T07:06:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.908+0000 7ff27f7fe700 1 -- 172.21.15.7:0/674386726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7ff288005ba0 con 0x7ff2980b30d0 2024-01-11T07:06:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.908+0000 7ff27f7fe700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff284066ea0 0x7ff284069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.908+0000 7ff27f7fe700 1 -- 172.21.15.7:0/674386726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff288083f00 con 0x7ff2980b30d0 2024-01-11T07:06:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.908+0000 7ff296ffd700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff284066ea0 0x7ff284069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.909+0000 7ff296ffd700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff284066ea0 0x7ff284069350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff2980c9b80 tx=0x7ff28c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:53.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:53.913+0000 7ff27f7fe700 1 -- 172.21.15.7:0/674386726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff288051620 con 0x7ff2980b30d0 2024-01-11T07:06:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.117+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2980c9bb0 con 0x7ff2980b30d0 2024-01-11T07:06:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.118+0000 7ff27f7fe700 1 -- 172.21.15.7:0/674386726 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff288054c40 con 0x7ff2980b30d0 2024-01-11T07:06:54.117 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff284066ea0 msgr2=0x7ff284069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff284066ea0 0x7ff284069350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff2980c9b80 tx=0x7ff28c009040 comp rx=0 tx=0).stop 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2980b30d0 msgr2=0x7ff2980c2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2980b30d0 0x7ff2980c2420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff28800cb70 tx=0x7ff288002bb0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 shutdown_connections 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff284066ea0 0x7ff284069350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff298157f90 0x7ff2980c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2980b30d0 0x7ff2980c2420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 --2- 172.21.15.7:0/674386726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff298157bc0 0x7ff2980c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 >> 172.21.15.7:0/674386726 conn(0x7ff298097a50 msgr2=0x7ff29806a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 shutdown_connections 2024-01-11T07:06:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.120+0000 7ff29db77700 1 -- 172.21.15.7:0/674386726 wait complete. 2024-01-11T07:06:54.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:06:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.264+0000 7f7f59be2700 1 Processor -- start 2024-01-11T07:06:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.264+0000 7f7f59be2700 1 -- start start 2024-01-11T07:06:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f59be2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f54159c90 0x7f7f540b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f59be2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f5415a060 0x7f7f540b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f59be2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f540ce9b0 con 0x7f7f54069320 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f59be2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f540cf530 con 0x7f7f5415a060 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f59be2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f540cf9c0 con 0x7f7f54159c90 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f52ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f5415a060 0x7f7f540b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f53fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f54069320 0x7f7f54067610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34546/0 (socket says 172.21.15.7:34546) 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f52ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f5415a060 0x7f7f540b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54770/0 (socket says 172.21.15.7:54770) 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f53fff700 1 -- 172.21.15.7:0/1482324152 learned_addr learned my addr 172.21.15.7:0/1482324152 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.265+0000 7f7f537fe700 1 --2- 172.21.15.7:0/1482324152 >> v2:172.21.15.113:3300/0 conn(0x7f7f54159c90 0x7f7f540b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f52ffd700 1 -- 172.21.15.7:0/1482324152 >> v2:172.21.15.113:3300/0 conn(0x7f7f54159c90 msgr2=0x7f7f540b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f52ffd700 1 --2- 172.21.15.7:0/1482324152 >> v2:172.21.15.113:3300/0 conn(0x7f7f54159c90 0x7f7f540b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f52ffd700 1 -- 172.21.15.7:0/1482324152 >> v1:172.21.15.7:6789/0 conn(0x7f7f54069320 legacy=0x7f7f54067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f52ffd700 1 -- 172.21.15.7:0/1482324152 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540cec00 con 0x7f7f5415a060 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f527fc700 1 -- 172.21.15.7:0/1482324152 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277718504 0 0) 0x7f7f540ce9b0 con 0x7f7f54069320 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f52ffd700 1 --2- 172.21.15.7:0/1482324152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f540b0430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f5405f2e0 tx=0x7f7f3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c442b7a45f455b35 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f527fc700 1 -- 172.21.15.7:0/1482324152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c011630 con 0x7f7f5415a060 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f527fc700 1 -- 172.21.15.7:0/1482324152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c011790 con 0x7f7f5415a060 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.266+0000 7f7f527fc700 1 -- 172.21.15.7:0/1482324152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c011b30 con 0x7f7f5415a060 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.267+0000 7f7f59be2700 1 -- 172.21.15.7:0/1482324152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 msgr2=0x7f7f540b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.267+0000 7f7f59be2700 1 --2- 172.21.15.7:0/1482324152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f540b0430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f5405f2e0 tx=0x7f7f3c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.267+0000 7f7f59be2700 1 -- 172.21.15.7:0/1482324152 shutdown_connections 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.267+0000 7f7f59be2700 1 --2- 172.21.15.7:0/1482324152 >> v2:172.21.15.113:3300/0 conn(0x7f7f54159c90 0x7f7f540b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.267+0000 7f7f59be2700 1 --2- 172.21.15.7:0/1482324152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f540b0430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.267+0000 7f7f59be2700 1 -- 172.21.15.7:0/1482324152 >> 172.21.15.7:0/1482324152 conn(0x7f7f54071100 msgr2=0x7f7f5406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.267+0000 7f7f59be2700 1 -- 172.21.15.7:0/1482324152 shutdown_connections 2024-01-11T07:06:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.267+0000 7f7f59be2700 1 -- 172.21.15.7:0/1482324152 wait complete. 2024-01-11T07:06:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.268+0000 7f7f59be2700 1 Processor -- start 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.268+0000 7f7f59be2700 1 -- start start 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f59be2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f54069320 0x7f7f54146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f59be2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f54159c90 0x7f7f54169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f59be2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f54157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f59be2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5416ad40 con 0x7f7f5415a060 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f59be2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540c8120 con 0x7f7f54159c90 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f59be2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5416a780 con 0x7f7f54069320 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f54157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f54157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54778/0 (socket says 172.21.15.7:54778) 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 -- 172.21.15.7:0/386981180 learned_addr learned my addr 172.21.15.7:0/386981180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f537fe700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f54069320 0x7f7f54146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f52ffd700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f54159c90 0x7f7f54169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 -- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f54069320 msgr2=0x7f7f54146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f54069320 0x7f7f54146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 -- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f54159c90 msgr2=0x7f7f54169cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f54159c90 0x7f7f54169cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 -- 172.21.15.7:0/386981180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c009420 con 0x7f7f5415a060 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.269+0000 7f7f53fff700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f54157990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00cc90 tx=0x7f7f4c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.270+0000 7f7f4b7fe700 1 -- 172.21.15.7:0/386981180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00a8b0 con 0x7f7f5415a060 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.270+0000 7f7f4b7fe700 1 -- 172.21.15.7:0/386981180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c00aa10 con 0x7f7f5415a060 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.270+0000 7f7f4b7fe700 1 -- 172.21.15.7:0/386981180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00add0 con 0x7f7f5415a060 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.270+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5416aa60 con 0x7f7f5415a060 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.270+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f541469b0 con 0x7f7f5415a060 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.271+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f540c7c10 con 0x7f7f5415a060 2024-01-11T07:06:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.271+0000 7f7f4b7fe700 1 -- 172.21.15.7:0/386981180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 84686+0+0 (secure 0 0 0) 0x7f7f4c007300 con 0x7f7f5415a060 2024-01-11T07:06:54.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.272+0000 7f7f4b7fe700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f34066ea0 0x7f7f34069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.272+0000 7f7f4b7fe700 1 -- 172.21.15.7:0/386981180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7f4c0843e0 con 0x7f7f5415a060 2024-01-11T07:06:54.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.272+0000 7f7f537fe700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f34066ea0 0x7f7f34069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.272+0000 7f7f537fe700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f34066ea0 0x7f7f34069350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009db0 tx=0x7f7f440091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.276+0000 7f7f4b7fe700 1 -- 172.21.15.7:0/386981180 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f540c7c10 con 0x7f7f5415a060 2024-01-11T07:06:54.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.482+0000 7f7f4b7fe700 1 -- 172.21.15.7:0/386981180 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7f4c051ed0 con 0x7f7f5415a060 2024-01-11T07:06:54.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.535+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f540c7c10 con 0x7f7f5415a060 2024-01-11T07:06:54.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.535+0000 7f7f4b7fe700 1 -- 172.21.15.7:0/386981180 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f7f540c7c10 con 0x7f7f5415a060 2024-01-11T07:06:54.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:54.535 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-01-11T07:06:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f34066ea0 msgr2=0x7f7f34069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f34066ea0 0x7f7f34069350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009db0 tx=0x7f7f440091a0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 msgr2=0x7f7f54157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f54157990 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00cc90 tx=0x7f7f4c001a80 comp rx=0 tx=0).stop 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 shutdown_connections 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f34066ea0 0x7f7f34069350 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f5415a060 0x7f7f54157990 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f54069320 0x7f7f54146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 --2- 172.21.15.7:0/386981180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f54159c90 0x7f7f54169cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.537+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 >> 172.21.15.7:0/386981180 conn(0x7f7f54071100 msgr2=0x7f7f540b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.538+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 shutdown_connections 2024-01-11T07:06:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.538+0000 7f7f59be2700 1 -- 172.21.15.7:0/386981180 wait complete. 2024-01-11T07:06:54.539 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-01-11T07:06:54.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:06:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.682+0000 7f0f633f6700 1 Processor -- start 2024-01-11T07:06:54.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.682+0000 7f0f633f6700 1 -- start start 2024-01-11T07:06:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f633f6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c06e0d0 0x7f0f5c159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f633f6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f5c159ff0 0x7f0f5c069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f633f6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c15b770 con 0x7f0f5c06dd00 2024-01-11T07:06:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f633f6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c15abf0 con 0x7f0f5c159ff0 2024-01-11T07:06:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f633f6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15aed0 con 0x7f0f5c06e0d0 2024-01-11T07:06:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f60991700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c06e0d0 0x7f0f5c159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f60991700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c06e0d0 0x7f0f5c159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:43932/0 (socket says 172.21.15.7:43932) 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f61993700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f5c159ff0 0x7f0f5c069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f60991700 1 -- 172.21.15.7:0/3043566367 learned_addr learned my addr 172.21.15.7:0/3043566367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f60991700 1 -- 172.21.15.7:0/3043566367 >> v2:172.21.15.113:3300/0 conn(0x7f0f5c159ff0 msgr2=0x7f0f5c069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f60991700 1 --2- 172.21.15.7:0/3043566367 >> v2:172.21.15.113:3300/0 conn(0x7f0f5c159ff0 0x7f0f5c069fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f60991700 1 -- 172.21.15.7:0/3043566367 >> v1:172.21.15.113:6789/0 conn(0x7f0f5c06dd00 legacy=0x7f0f5c0c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f60991700 1 -- 172.21.15.7:0/3043566367 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c15b1b0 con 0x7f0f5c06e0d0 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f53fff700 1 -- 172.21.15.7:0/3043566367 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945533300 0 0) 0x7f0f5c15b770 con 0x7f0f5c06dd00 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.683+0000 7f0f61993700 1 --2- 172.21.15.7:0/3043566367 >> v2:172.21.15.113:3300/0 conn(0x7f0f5c159ff0 0x7f0f5c069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f60991700 1 --2- 172.21.15.7:0/3043566367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c06e0d0 0x7f0f5c159ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0099b0 tx=0x7f0f4c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21d80ee2d812dafc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f53fff700 1 -- 172.21.15.7:0/3043566367 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c009ea0 con 0x7f0f5c06e0d0 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f53fff700 1 -- 172.21.15.7:0/3043566367 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c00b860 con 0x7f0f5c06e0d0 2024-01-11T07:06:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f53fff700 1 -- 172.21.15.7:0/3043566367 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00bc00 con 0x7f0f5c06e0d0 2024-01-11T07:06:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f633f6700 1 -- 172.21.15.7:0/3043566367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c06e0d0 msgr2=0x7f0f5c159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f633f6700 1 --2- 172.21.15.7:0/3043566367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c06e0d0 0x7f0f5c159ab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0099b0 tx=0x7f0f4c009250 comp rx=0 tx=0).stop 2024-01-11T07:06:54.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f633f6700 1 -- 172.21.15.7:0/3043566367 shutdown_connections 2024-01-11T07:06:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f633f6700 1 --2- 172.21.15.7:0/3043566367 >> v2:172.21.15.113:3300/0 conn(0x7f0f5c159ff0 0x7f0f5c069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f633f6700 1 --2- 172.21.15.7:0/3043566367 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c06e0d0 0x7f0f5c159ab0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f633f6700 1 -- 172.21.15.7:0/3043566367 >> 172.21.15.7:0/3043566367 conn(0x7f0f5c0680e0 msgr2=0x7f0f5c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f633f6700 1 -- 172.21.15.7:0/3043566367 shutdown_connections 2024-01-11T07:06:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.684+0000 7f0f633f6700 1 -- 172.21.15.7:0/3043566367 wait complete. 2024-01-11T07:06:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.686+0000 7f0f633f6700 1 Processor -- start 2024-01-11T07:06:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.686+0000 7f0f633f6700 1 -- start start 2024-01-11T07:06:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f633f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f5c06dd00 0x7f0f5c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f633f6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f5c06e0d0 0x7f0f5c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f633f6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c159ff0 0x7f0f5c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f633f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c167d60 con 0x7f0f5c06dd00 2024-01-11T07:06:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f633f6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c167150 con 0x7f0f5c06e0d0 2024-01-11T07:06:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f633f6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c167430 con 0x7f0f5c159ff0 2024-01-11T07:06:54.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f61192700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f5c06dd00 0x7f0f5c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f61192700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f5c06dd00 0x7f0f5c0c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54784/0 (socket says 172.21.15.7:54784) 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.687+0000 7f0f61192700 1 -- 172.21.15.7:0/101668764 learned_addr learned my addr 172.21.15.7:0/101668764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f60991700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f5c06e0d0 0x7f0f5c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f61993700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c159ff0 0x7f0f5c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f61192700 1 -- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c159ff0 msgr2=0x7f0f5c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f61192700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c159ff0 0x7f0f5c0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f61192700 1 -- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f5c06e0d0 msgr2=0x7f0f5c157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f61192700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f5c06e0d0 0x7f0f5c157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f61192700 1 -- 172.21.15.7:0/101668764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54009a20 con 0x7f0f5c06dd00 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f61192700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f5c06dd00 0x7f0f5c0c6820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800e9f0 tx=0x7f0f58007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f517fa700 1 -- 172.21.15.7:0/101668764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800ee80 con 0x7f0f5c06dd00 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f517fa700 1 -- 172.21.15.7:0/101668764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5800c9f0 con 0x7f0f5c06dd00 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c0094f0 con 0x7f0f5c06dd00 2024-01-11T07:06:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f517fa700 1 -- 172.21.15.7:0/101668764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5800cdb0 con 0x7f0f5c06dd00 2024-01-11T07:06:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.688+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c1679f0 con 0x7f0f5c06dd00 2024-01-11T07:06:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.689+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c04c3c0 con 0x7f0f5c06dd00 2024-01-11T07:06:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.690+0000 7f0f517fa700 1 -- 172.21.15.7:0/101668764 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0f58011470 con 0x7f0f5c06dd00 2024-01-11T07:06:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.690+0000 7f0f517fa700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f40066f30 0x7f0f400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.690+0000 7f0f517fa700 1 -- 172.21.15.7:0/101668764 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0f58015070 con 0x7f0f5c06dd00 2024-01-11T07:06:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.690+0000 7f0f60991700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f40066f30 0x7f0f400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:54.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.691+0000 7f0f60991700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f40066f30 0x7f0f400693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0099b0 tx=0x7f0f4c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:54.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.695+0000 7f0f517fa700 1 -- 172.21.15.7:0/101668764 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f58053e70 con 0x7f0f5c06dd00 2024-01-11T07:06:54.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.899+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f5c15b490 con 0x7f0f5c06dd00 2024-01-11T07:06:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.899+0000 7f0f517fa700 1 -- 172.21.15.7:0/101668764 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12079 (secure 0 0 0) 0x7f0f58057490 con 0x7f0f5c06dd00 2024-01-11T07:06:54.899 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:54.899 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:52.929534+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f40066f30 msgr2=0x7f0f400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f40066f30 0x7f0f400693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0099b0 tx=0x7f0f4c017000 comp rx=0 tx=0).stop 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f5c06dd00 msgr2=0x7f0f5c0c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f5c06dd00 0x7f0f5c0c6820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800e9f0 tx=0x7f0f58007e80 comp rx=0 tx=0).stop 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 shutdown_connections 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f40066f30 0x7f0f400693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f5c159ff0 0x7f0f5c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f5c06dd00 0x7f0f5c0c6820 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 --2- 172.21.15.7:0/101668764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f5c06e0d0 0x7f0f5c157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 >> 172.21.15.7:0/101668764 conn(0x7f0f5c0680e0 msgr2=0x7f0f5c0ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 shutdown_connections 2024-01-11T07:06:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:54.902+0000 7f0f633f6700 1 -- 172.21.15.7:0/101668764 wait complete. 2024-01-11T07:06:54.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:06:54.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:06:55.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.050+0000 7f8aff1f0700 1 Processor -- start 2024-01-11T07:06:55.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.050+0000 7f8aff1f0700 1 -- start start 2024-01-11T07:06:55.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.051+0000 7f8aff1f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af8154920 0x7f8af806ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.051+0000 7f8aff1f0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8af806e200 0x7f8af806b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.051+0000 7f8aff1f0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af815a790 con 0x7f8af806de30 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.051+0000 7f8aff1f0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af815b7a0 con 0x7f8af806e200 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.051+0000 7f8aff1f0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af806b7c0 con 0x7f8af8154920 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8afd78d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8af806e200 0x7f8af806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8afcf8c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8af806de30 0x7f8af8154210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34552/0 (socket says 172.21.15.7:34552) 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8afcf8c700 1 -- 172.21.15.7:0/2747214343 learned_addr learned my addr 172.21.15.7:0/2747214343 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8af7fff700 1 --2- 172.21.15.7:0/2747214343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af8154920 0x7f8af806ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8afd78d700 1 -- 172.21.15.7:0/2747214343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af8154920 msgr2=0x7f8af806ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8afd78d700 1 --2- 172.21.15.7:0/2747214343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af8154920 0x7f8af806ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8afd78d700 1 -- 172.21.15.7:0/2747214343 >> v1:172.21.15.7:6789/0 conn(0x7f8af806de30 legacy=0x7f8af8154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8afd78d700 1 -- 172.21.15.7:0/2747214343 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af815a9e0 con 0x7f8af806e200 2024-01-11T07:06:55.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8af77fe700 1 -- 172.21.15.7:0/2747214343 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938125528 0 0) 0x7f8af815a790 con 0x7f8af806de30 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8afd78d700 1 --2- 172.21.15.7:0/2747214343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af806e200 0x7f8af806b280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009e60 tx=0x7f8af0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6512fc99eec18270 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8af77fe700 1 -- 172.21.15.7:0/2747214343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af000e070 con 0x7f8af806e200 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8af77fe700 1 -- 172.21.15.7:0/2747214343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af000bc30 con 0x7f8af806e200 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.052+0000 7f8af77fe700 1 -- 172.21.15.7:0/2747214343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0012430 con 0x7f8af806e200 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.053+0000 7f8aff1f0700 1 -- 172.21.15.7:0/2747214343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af806e200 msgr2=0x7f8af806b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.053+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/2747214343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af806e200 0x7f8af806b280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009e60 tx=0x7f8af0009780 comp rx=0 tx=0).stop 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.053+0000 7f8aff1f0700 1 -- 172.21.15.7:0/2747214343 shutdown_connections 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.053+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/2747214343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af806e200 0x7f8af806b280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.053+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/2747214343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af8154920 0x7f8af806ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.053+0000 7f8aff1f0700 1 -- 172.21.15.7:0/2747214343 >> 172.21.15.7:0/2747214343 conn(0x7f8af80cd8f0 msgr2=0x7f8af80cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.053+0000 7f8aff1f0700 1 -- 172.21.15.7:0/2747214343 shutdown_connections 2024-01-11T07:06:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.053+0000 7f8aff1f0700 1 -- 172.21.15.7:0/2747214343 wait complete. 2024-01-11T07:06:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.054+0000 7f8aff1f0700 1 Processor -- start 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.054+0000 7f8aff1f0700 1 -- start start 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8aff1f0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8af806de30 0x7f8af80b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8aff1f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af806e200 0x7f8af814af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8aff1f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af8154920 0x7f8af8156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8aff1f0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af81558e0 con 0x7f8af8154920 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8aff1f0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af8156740 con 0x7f8af806de30 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8aff1f0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80b1790 con 0x7f8af806e200 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8afd78d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af8154920 0x7f8af8156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8afd78d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af8154920 0x7f8af8156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54810/0 (socket says 172.21.15.7:54810) 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8afd78d700 1 -- 172.21.15.7:0/1104913754 learned_addr learned my addr 172.21.15.7:0/1104913754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8af7fff700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af806e200 0x7f8af814af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.055+0000 7f8afcf8c700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8af806de30 0x7f8af80b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8afd78d700 1 -- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af806e200 msgr2=0x7f8af814af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8afd78d700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af806e200 0x7f8af814af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8afd78d700 1 -- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8af806de30 msgr2=0x7f8af80b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8afd78d700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8af806de30 0x7f8af80b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8afd78d700 1 -- 172.21.15.7:0/1104913754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af0009a20 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8afd78d700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af8154920 0x7f8af8156bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8af0013000 tx=0x7f8af0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8af4ff9700 1 -- 172.21.15.7:0/1104913754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0012e00 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8af4ff9700 1 -- 172.21.15.7:0/1104913754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af001d630 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8af4ff9700 1 -- 172.21.15.7:0/1104913754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af001d9f0 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af8155b30 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.056+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af81560f0 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.057+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af8064a40 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.058+0000 7f8af4ff9700 1 -- 172.21.15.7:0/1104913754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8af0024070 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.058+0000 7f8af4ff9700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8adc066f30 0x7f8adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.058+0000 7f8af4ff9700 1 -- 172.21.15.7:0/1104913754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8af0086a20 con 0x7f8af8154920 2024-01-11T07:06:55.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.058+0000 7f8afcf8c700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8adc066f30 0x7f8adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.059+0000 7f8afcf8c700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8adc066f30 0x7f8adc0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8aec007fd0 tx=0x7f8aec007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.063+0000 7f8af4ff9700 1 -- 172.21.15.7:0/1104913754 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8af0053fe0 con 0x7f8af8154920 2024-01-11T07:06:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.265+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8af804c3c0 con 0x7f8af8154920 2024-01-11T07:06:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.265+0000 7f8af4ff9700 1 -- 172.21.15.7:0/1104913754 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8af0057600 con 0x7f8af8154920 2024-01-11T07:06:55.265 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:06:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8adc066f30 msgr2=0x7f8adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8adc066f30 0x7f8adc0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8aec007fd0 tx=0x7f8aec007d30 comp rx=0 tx=0).stop 2024-01-11T07:06:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af8154920 msgr2=0x7f8af8156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af8154920 0x7f8af8156bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8af0013000 tx=0x7f8af0019000 comp rx=0 tx=0).stop 2024-01-11T07:06:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 shutdown_connections 2024-01-11T07:06:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8adc066f30 0x7f8adc0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8af8154920 0x7f8af8156bd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8af806de30 0x7f8af80b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 --2- 172.21.15.7:0/1104913754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8af806e200 0x7f8af814af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 >> 172.21.15.7:0/1104913754 conn(0x7f8af80cd8f0 msgr2=0x7f8af8159990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 shutdown_connections 2024-01-11T07:06:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.267+0000 7f8aff1f0700 1 -- 172.21.15.7:0/1104913754 wait complete. 2024-01-11T07:06:55.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:06:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.412+0000 7f329900f700 1 Processor -- start 2024-01-11T07:06:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.412+0000 7f329900f700 1 -- start start 2024-01-11T07:06:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.413+0000 7f329900f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f32940b0b40 0x7f32940b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.413+0000 7f329900f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3294067de0 con 0x7f3294159c20 2024-01-11T07:06:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.413+0000 7f329900f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32940b20e0 con 0x7f3294159ff0 2024-01-11T07:06:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.413+0000 7f329900f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3294069570 con 0x7f32940b0b40 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.413+0000 7f329359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f32940b0b40 0x7f32940b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.413+0000 7f329359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f32940b0b40 0x7f32940b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54818/0 (socket says 172.21.15.7:54818) 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.413+0000 7f329359e700 1 -- 172.21.15.7:0/3336284075 learned_addr learned my addr 172.21.15.7:0/3336284075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.413+0000 7f3292d9d700 1 --1- 172.21.15.7:0/3336284075 >> v1:172.21.15.7:6789/0 conn(0x7f3294159c20 0x7f3294068270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34558/0 (socket says 172.21.15.7:34558) 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.414+0000 7f329359e700 1 -- 172.21.15.7:0/3336284075 >> v1:172.21.15.113:6789/0 conn(0x7f3294159ff0 legacy=0x7f32940b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.414+0000 7f329359e700 1 -- 172.21.15.7:0/3336284075 >> v1:172.21.15.7:6789/0 conn(0x7f3294159c20 legacy=0x7f3294068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.414+0000 7f329359e700 1 -- 172.21.15.7:0/3336284075 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940b1e00 con 0x7f32940b0b40 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.414+0000 7f3291d9b700 1 -- 172.21.15.7:0/3336284075 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035733023 0 0) 0x7f3294067de0 con 0x7f3294159c20 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.414+0000 7f329359e700 1 --2- 172.21.15.7:0/3336284075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32940b0b40 0x7f32940b1820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f328c009f20 tx=0x7f328c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd513c476b576643 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.414+0000 7f3291d9b700 1 -- 172.21.15.7:0/3336284075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c00e070 con 0x7f32940b0b40 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.414+0000 7f3291d9b700 1 -- 172.21.15.7:0/3336284075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c00ba80 con 0x7f32940b0b40 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.414+0000 7f3291d9b700 1 -- 172.21.15.7:0/3336284075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c0092e0 con 0x7f32940b0b40 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.415+0000 7f329900f700 1 -- 172.21.15.7:0/3336284075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32940b0b40 msgr2=0x7f32940b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.415+0000 7f329900f700 1 --2- 172.21.15.7:0/3336284075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32940b0b40 0x7f32940b1820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f328c009f20 tx=0x7f328c009840 comp rx=0 tx=0).stop 2024-01-11T07:06:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.415+0000 7f329900f700 1 -- 172.21.15.7:0/3336284075 shutdown_connections 2024-01-11T07:06:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.415+0000 7f329900f700 1 --2- 172.21.15.7:0/3336284075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32940b0b40 0x7f32940b1820 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.415+0000 7f329900f700 1 -- 172.21.15.7:0/3336284075 >> 172.21.15.7:0/3336284075 conn(0x7f3294071160 msgr2=0x7f329406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.415+0000 7f329900f700 1 -- 172.21.15.7:0/3336284075 shutdown_connections 2024-01-11T07:06:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.415+0000 7f329900f700 1 -- 172.21.15.7:0/3336284075 wait complete. 2024-01-11T07:06:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.416+0000 7f329900f700 1 Processor -- start 2024-01-11T07:06:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.417+0000 7f329900f700 1 -- start start 2024-01-11T07:06:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329900f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32940b0b40 0x7f32940a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329900f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3294159c20 0x7f3294144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329900f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3294159ff0 0x7f32940cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329900f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32940cbc80 con 0x7f3294159c20 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329900f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32940cbed0 con 0x7f3294159ff0 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329900f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32940cc1b0 con 0x7f32940b0b40 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3294159c20 0x7f3294144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3294159c20 0x7f3294144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54828/0 (socket says 172.21.15.7:54828) 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 -- 172.21.15.7:0/2866564791 learned_addr learned my addr 172.21.15.7:0/2866564791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f3292d9d700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32940b0b40 0x7f32940a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329359e700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3294159ff0 0x7f32940cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 -- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32940b0b40 msgr2=0x7f32940a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32940b0b40 0x7f32940a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 -- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3294159ff0 msgr2=0x7f32940cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3294159ff0 0x7f32940cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 -- 172.21.15.7:0/2866564791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c009ae0 con 0x7f3294159c20 2024-01-11T07:06:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329259c700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3294159c20 0x7f3294144300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c4e0 tx=0x7f327c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f328affd700 1 -- 172.21.15.7:0/2866564791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c011590 con 0x7f3294159c20 2024-01-11T07:06:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f328affd700 1 -- 172.21.15.7:0/2866564791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c011710 con 0x7f3294159c20 2024-01-11T07:06:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32940cc490 con 0x7f3294159c20 2024-01-11T07:06:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f328affd700 1 -- 172.21.15.7:0/2866564791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c011ad0 con 0x7f3294159c20 2024-01-11T07:06:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.418+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32940b0250 con 0x7f3294159c20 2024-01-11T07:06:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.419+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32940cc930 con 0x7f3294159c20 2024-01-11T07:06:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.420+0000 7f328affd700 1 -- 172.21.15.7:0/2866564791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f327c01b3b0 con 0x7f3294159c20 2024-01-11T07:06:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.420+0000 7f328affd700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3274066fb0 0x7f3274069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.421+0000 7f328affd700 1 -- 172.21.15.7:0/2866564791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f327c083740 con 0x7f3294159c20 2024-01-11T07:06:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.421+0000 7f3292d9d700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3274066fb0 0x7f3274069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.421+0000 7f3292d9d700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3274066fb0 0x7f3274069460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f32940b0b10 tx=0x7f3284007bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.425+0000 7f328affd700 1 -- 172.21.15.7:0/2866564791 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f327c050c60 con 0x7f3294159c20 2024-01-11T07:06:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.633+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f329404c3c0 con 0x7f3294159c20 2024-01-11T07:06:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.634+0000 7f328affd700 1 -- 172.21.15.7:0/2866564791 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f327c017020 con 0x7f3294159c20 2024-01-11T07:06:55.633 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:55.634 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:06:55.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:06:55.634 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:06:55.634 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:06:55.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:06:55.634 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3274066fb0 msgr2=0x7f3274069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3274066fb0 0x7f3274069460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f32940b0b10 tx=0x7f3284007bd0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3294159c20 msgr2=0x7f3294144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3294159c20 0x7f3294144300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c4e0 tx=0x7f327c00c8a0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 shutdown_connections 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3274066fb0 0x7f3274069460 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3294159ff0 0x7f32940cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32940b0b40 0x7f32940a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 --2- 172.21.15.7:0/2866564791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3294159c20 0x7f3294144300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 >> 172.21.15.7:0/2866564791 conn(0x7f3294071160 msgr2=0x7f3294158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 shutdown_connections 2024-01-11T07:06:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.636+0000 7f329900f700 1 -- 172.21.15.7:0/2866564791 wait complete. 2024-01-11T07:06:55.649 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:06:55.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:06:55.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.785+0000 7fee36bc3700 1 Processor -- start 2024-01-11T07:06:55.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.785+0000 7fee36bc3700 1 -- start start 2024-01-11T07:06:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.785+0000 7fee36bc3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee30069ea0 0x7fee3006db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.785+0000 7fee36bc3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee3006e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.785+0000 7fee36bc3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3015b0a0 con 0x7fee3006a270 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.785+0000 7fee36bc3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee300c9860 con 0x7fee30069ea0 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.785+0000 7fee36bc3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee300c8e80 con 0x7fee3015a1c0 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee2ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fee3006a270 0x7fee30159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34562/0 (socket says 172.21.15.7:34562) 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee2ffff700 1 -- 172.21.15.7:0/1355660604 learned_addr learned my addr 172.21.15.7:0/1355660604 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee35160700 1 --2- 172.21.15.7:0/1355660604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee3006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee3495f700 1 --2- 172.21.15.7:0/1355660604 >> v2:172.21.15.113:3300/0 conn(0x7fee30069ea0 0x7fee3006db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee2f7fe700 1 -- 172.21.15.7:0/1355660604 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024244102 0 0) 0x7fee3015b0a0 con 0x7fee3006a270 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee2f7fe700 1 -- 172.21.15.7:0/1355660604 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee1c003570 con 0x7fee3006a270 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee2f7fe700 1 -- 172.21.15.7:0/1355660604 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2039626536 0 0) 0x7fee1c003570 con 0x7fee3006a270 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee35160700 1 -- 172.21.15.7:0/1355660604 >> v2:172.21.15.113:3300/0 conn(0x7fee30069ea0 msgr2=0x7fee3006db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:55.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee35160700 1 --2- 172.21.15.7:0/1355660604 >> v2:172.21.15.113:3300/0 conn(0x7fee30069ea0 0x7fee3006db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee35160700 1 -- 172.21.15.7:0/1355660604 >> v1:172.21.15.7:6789/0 conn(0x7fee3006a270 legacy=0x7fee30159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee35160700 1 -- 172.21.15.7:0/1355660604 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3006e6f0 con 0x7fee3015a1c0 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee3495f700 1 --2- 172.21.15.7:0/1355660604 >> v2:172.21.15.113:3300/0 conn(0x7fee30069ea0 0x7fee3006db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.786+0000 7fee35160700 1 --2- 172.21.15.7:0/1355660604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee3006e090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fee28009e60 tx=0x7fee28009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed4bfd8a7ab8ef14 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee2f7fe700 1 -- 172.21.15.7:0/1355660604 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2800e070 con 0x7fee3015a1c0 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee2f7fe700 1 -- 172.21.15.7:0/1355660604 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2800b7a0 con 0x7fee3015a1c0 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee2f7fe700 1 -- 172.21.15.7:0/1355660604 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18001470 con 0x7fee3015a1c0 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee36bc3700 1 -- 172.21.15.7:0/1355660604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 msgr2=0x7fee3006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee36bc3700 1 --2- 172.21.15.7:0/1355660604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee3006e090 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fee28009e60 tx=0x7fee28009780 comp rx=0 tx=0).stop 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee36bc3700 1 -- 172.21.15.7:0/1355660604 shutdown_connections 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee36bc3700 1 --2- 172.21.15.7:0/1355660604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee3006e090 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee36bc3700 1 --2- 172.21.15.7:0/1355660604 >> v2:172.21.15.113:3300/0 conn(0x7fee30069ea0 0x7fee3006db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee36bc3700 1 -- 172.21.15.7:0/1355660604 >> 172.21.15.7:0/1355660604 conn(0x7fee300680a0 msgr2=0x7fee300cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee36bc3700 1 -- 172.21.15.7:0/1355660604 shutdown_connections 2024-01-11T07:06:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.787+0000 7fee36bc3700 1 -- 172.21.15.7:0/1355660604 wait complete. 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.788+0000 7fee36bc3700 1 Processor -- start 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.789+0000 7fee36bc3700 1 -- start start 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.789+0000 7fee36bc3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee30069ea0 0x7fee300c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.789+0000 7fee36bc3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3006a270 0x7fee30153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.789+0000 7fee36bc3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee30153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.789+0000 7fee36bc3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee30154840 con 0x7fee3006a270 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.789+0000 7fee36bc3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee300c5690 con 0x7fee30069ea0 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.789+0000 7fee36bc3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee300c4a80 con 0x7fee3015a1c0 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3006a270 0x7fee30153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3006a270 0x7fee30153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54830/0 (socket says 172.21.15.7:54830) 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 -- 172.21.15.7:0/2589258371 learned_addr learned my addr 172.21.15.7:0/2589258371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee3495f700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee30069ea0 0x7fee300c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee35160700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee30153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 -- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 msgr2=0x7fee30153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee30153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 -- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee30069ea0 msgr2=0x7fee300c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee30069ea0 0x7fee300c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 -- 172.21.15.7:0/2589258371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20009420 con 0x7fee3006a270 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2ffff700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3006a270 0x7fee30153300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee1800cbc0 tx=0x7fee18001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2cff9700 1 -- 172.21.15.7:0/2589258371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18007360 con 0x7fee3006a270 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2cff9700 1 -- 172.21.15.7:0/2589258371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee180074e0 con 0x7fee3006a270 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee28009a20 con 0x7fee3006a270 2024-01-11T07:06:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee2cff9700 1 -- 172.21.15.7:0/2589258371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18004830 con 0x7fee3006a270 2024-01-11T07:06:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.790+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee300c5040 con 0x7fee3006a270 2024-01-11T07:06:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.791+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3004c3c0 con 0x7fee3006a270 2024-01-11T07:06:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.792+0000 7fee2cff9700 1 -- 172.21.15.7:0/2589258371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fee18004ba0 con 0x7fee3006a270 2024-01-11T07:06:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.792+0000 7fee2cff9700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee10066f30 0x7fee100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.793+0000 7fee2cff9700 1 -- 172.21.15.7:0/2589258371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fee180836b0 con 0x7fee3006a270 2024-01-11T07:06:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.793+0000 7fee3495f700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee10066f30 0x7fee100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.793+0000 7fee3495f700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee10066f30 0x7fee100693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee200093f0 tx=0x7fee20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:55.797+0000 7fee2cff9700 1 -- 172.21.15.7:0/2589258371 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee18050c50 con 0x7fee3006a270 2024-01-11T07:06:56.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.007+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fee3015a770 con 0x7fee3006a270 2024-01-11T07:06:56.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.949+0000 7fee2cff9700 1 -- 172.21.15.7:0/2589258371 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fee180180b0 con 0x7fee3006a270 2024-01-11T07:06:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee10066f30 msgr2=0x7fee100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee10066f30 0x7fee100693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee200093f0 tx=0x7fee20008040 comp rx=0 tx=0).stop 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3006a270 msgr2=0x7fee30153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3006a270 0x7fee30153300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee1800cbc0 tx=0x7fee18001ad0 comp rx=0 tx=0).stop 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 shutdown_connections 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee10066f30 0x7fee100693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee3015a1c0 0x7fee30153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee30069ea0 0x7fee300c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 --2- 172.21.15.7:0/2589258371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3006a270 0x7fee30153300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 >> 172.21.15.7:0/2589258371 conn(0x7fee300680a0 msgr2=0x7fee3006a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 shutdown_connections 2024-01-11T07:06:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:56.954+0000 7fee36bc3700 1 -- 172.21.15.7:0/2589258371 wait complete. 2024-01-11T07:06:56.955 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:06:56.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:06:57.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.112+0000 7fa6892f9700 1 Processor -- start 2024-01-11T07:06:57.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.112+0000 7fa6892f9700 1 -- start start 2024-01-11T07:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa6892f9700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa68415dd90 0x7fa6840aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa6892f9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68406ac30 con 0x7fa68415d920 2024-01-11T07:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa6892f9700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6840cb370 con 0x7fa6840ac3a0 2024-01-11T07:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa6892f9700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6840ac770 con 0x7fa68415dd90 2024-01-11T07:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa67a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa68415d920 0x7fa6840aa8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34568/0 (socket says 172.21.15.7:34568) 2024-01-11T07:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa67a7fc700 1 -- 172.21.15.7:0/221261892 learned_addr learned my addr 172.21.15.7:0/221261892 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa6837fe700 1 --2- 172.21.15.7:0/221261892 >> v2:172.21.15.113:3300/0 conn(0x7fa68415dd90 0x7fa6840aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796826100 0 0) 0x7fa68406ac30 con 0x7fa68415d920 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.113+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa670003570 con 0x7fa68415d920 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468391135 0 0) 0x7fa670003570 con 0x7fa68415d920 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 >> v2:172.21.15.113:3300/0 conn(0x7fa68415dd90 msgr2=0x7fa6840aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 --2- 172.21.15.7:0/221261892 >> v2:172.21.15.113:3300/0 conn(0x7fa68415dd90 0x7fa6840aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 >> v1:172.21.15.113:6789/0 conn(0x7fa6840ac3a0 legacy=0x7fa6840cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68406a300 con 0x7fa68415d920 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa66c001470 con 0x7fa68415d920 2024-01-11T07:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 >> v1:172.21.15.7:6789/0 conn(0x7fa68415d920 legacy=0x7fa6840aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa670003a60 0x7fa670003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa670004810 0x7fa670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa67000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840ac770 con 0x7fa6700051c0 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67000c050 con 0x7fa670003a60 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67000c210 con 0x7fa670004810 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6837fe700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa67000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa682ffd700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa670003a60 0x7fa670003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa67a7fc700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa670004810 0x7fa670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6837fe700 1 -- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa670004810 msgr2=0x7fa670004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6837fe700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa670004810 0x7fa670004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6837fe700 1 -- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa670003a60 msgr2=0x7fa670003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6837fe700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa670003a60 0x7fa670003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6837fe700 1 -- 172.21.15.7:0/221261892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa670003570 con 0x7fa6700051c0 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa682ffd700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa670003a60 0x7fa670003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6837fe700 1 -- 172.21.15.7:0/221261892 reap_dead start 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.114+0000 7fa6837fe700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa67000ba60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00e210 tx=0x7fa67c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cd325fa617c204 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00eac0 con 0x7fa6700051c0 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c00f040 con 0x7fa6700051c0 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6827fc700 1 -- 172.21.15.7:0/221261892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00ee70 con 0x7fa6700051c0 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6892f9700 1 -- 172.21.15.7:0/221261892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 msgr2=0x7fa67000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6892f9700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa67000ba60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00e210 tx=0x7fa67c00b040 comp rx=0 tx=0).stop 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6892f9700 1 -- 172.21.15.7:0/221261892 shutdown_connections 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6892f9700 1 --2- 172.21.15.7:0/221261892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa67000ba60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6892f9700 1 -- 172.21.15.7:0/221261892 >> 172.21.15.7:0/221261892 conn(0x7fa6840cd090 msgr2=0x7fa6840cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6892f9700 1 -- 172.21.15.7:0/221261892 shutdown_connections 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.115+0000 7fa6892f9700 1 -- 172.21.15.7:0/221261892 wait complete. 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa6892f9700 1 Processor -- start 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa6892f9700 1 -- start start 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa6892f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa68416d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa6892f9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6840aa8c0 0x7fa6840aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa6892f9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6840b1470 0x7fa6840b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa6892f9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68416de30 con 0x7fa6700051c0 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa6892f9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840ab270 con 0x7fa6840aa8c0 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa6892f9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68413c370 con 0x7fa6840b1470 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.117+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa68416d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa682ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa68416d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54852/0 (socket says 172.21.15.7:54852) 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa682ffd700 1 -- 172.21.15.7:0/941751740 learned_addr learned my addr 172.21.15.7:0/941751740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa6837fe700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6840b1470 0x7fa6840b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa67a7fc700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6840aa8c0 0x7fa6840aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa682ffd700 1 -- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6840b1470 msgr2=0x7fa6840b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa682ffd700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6840b1470 0x7fa6840b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa682ffd700 1 -- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6840aa8c0 msgr2=0x7fa6840aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa682ffd700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6840aa8c0 0x7fa6840aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa682ffd700 1 -- 172.21.15.7:0/941751740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6840afe90 con 0x7fa6700051c0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa682ffd700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa68416d8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa67400fcb0 tx=0x7fa674009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa67b7fe700 1 -- 172.21.15.7:0/941751740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67400d920 con 0x7fa6700051c0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa67b7fe700 1 -- 172.21.15.7:0/941751740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67400da80 con 0x7fa6700051c0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6840b0ea0 con 0x7fa6700051c0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa67b7fe700 1 -- 172.21.15.7:0/941751740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67400de20 con 0x7fa6700051c0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.118+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6840b03c0 con 0x7fa6700051c0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.119+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68404c3c0 con 0x7fa6700051c0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.120+0000 7fa67b7fe700 1 -- 172.21.15.7:0/941751740 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa674013600 con 0x7fa6700051c0 2024-01-11T07:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.120+0000 7fa67b7fe700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa664066f30 0x7fa6640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.120+0000 7fa67b7fe700 1 -- 172.21.15.7:0/941751740 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fa6740846a0 con 0x7fa6700051c0 2024-01-11T07:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.120+0000 7fa67a7fc700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa664066f30 0x7fa6640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.121+0000 7fa67a7fc700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa664066f30 0x7fa6640693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000e50 tx=0x7fa66c001df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.125+0000 7fa67b7fe700 1 -- 172.21.15.7:0/941751740 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa674051a50 con 0x7fa6700051c0 2024-01-11T07:06:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.334+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa68406a5e0 con 0x7fa6700051c0 2024-01-11T07:06:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.953+0000 7fa67b7fe700 1 -- 172.21.15.7:0/941751740 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa674055070 con 0x7fa6700051c0 2024-01-11T07:06:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.961+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa664066f30 msgr2=0x7fa6640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.961+0000 7fa6892f9700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa664066f30 0x7fa6640693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000e50 tx=0x7fa66c001df0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.961+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 msgr2=0x7fa68416d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.961+0000 7fa6892f9700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa68416d8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa67400fcb0 tx=0x7fa674009040 comp rx=0 tx=0).stop 2024-01-11T07:06:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.965+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 shutdown_connections 2024-01-11T07:06:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.965+0000 7fa6892f9700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa664066f30 0x7fa6640693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.965+0000 7fa6892f9700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6840b1470 0x7fa6840b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.965+0000 7fa6892f9700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6700051c0 0x7fa68416d8f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.965+0000 7fa6892f9700 1 --2- 172.21.15.7:0/941751740 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6840aa8c0 0x7fa6840aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.965+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 >> 172.21.15.7:0/941751740 conn(0x7fa6840cd090 msgr2=0x7fa6840c1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.966+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 shutdown_connections 2024-01-11T07:06:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:57.966+0000 7fa6892f9700 1 -- 172.21.15.7:0/941751740 wait complete. 2024-01-11T07:06:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:06:57.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:06:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.120+0000 7f5d5ffa8700 1 Processor -- start 2024-01-11T07:06:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.121+0000 7f5d5ffa8700 1 -- start start 2024-01-11T07:06:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5ffa8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d5806de30 0x7f5d58154210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5ffa8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 0x7f5d58154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5ffa8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d5806ad40 0x7f5d5806b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5ffa8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5815b7a0 con 0x7f5d5806ad40 2024-01-11T07:06:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5ffa8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5815a790 con 0x7f5d5806de30 2024-01-11T07:06:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5ffa8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5815a9e0 con 0x7f5d5806e200 2024-01-11T07:06:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d5806ad40 0x7f5d5806b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d5806ad40 0x7f5d5806b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59730/0 (socket says 172.21.15.7:59730) 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 -- 172.21.15.7:0/2255402661 learned_addr learned my addr 172.21.15.7:0/2255402661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5d543700 1 --2- 172.21.15.7:0/2255402661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 0x7f5d58154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5dd44700 1 --2- 172.21.15.7:0/2255402661 >> v2:172.21.15.113:3300/0 conn(0x7f5d5806de30 0x7f5d58154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 -- 172.21.15.7:0/2255402661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 msgr2=0x7f5d58154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 --2- 172.21.15.7:0/2255402661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 0x7f5d58154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 -- 172.21.15.7:0/2255402661 >> v2:172.21.15.113:3300/0 conn(0x7f5d5806de30 msgr2=0x7f5d58154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 --2- 172.21.15.7:0/2255402661 >> v2:172.21.15.113:3300/0 conn(0x7f5d5806de30 0x7f5d58154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 -- 172.21.15.7:0/2255402661 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5815acc0 con 0x7f5d5806ad40 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.122+0000 7f5d5e545700 1 --2- 172.21.15.7:0/2255402661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806ad40 0x7f5d5806b1c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d5806d180 tx=0x7f5d5000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c6de27592ddab7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5cd42700 1 -- 172.21.15.7:0/2255402661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50010070 con 0x7f5d5806ad40 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5cd42700 1 -- 172.21.15.7:0/2255402661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5000de90 con 0x7f5d5806ad40 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5cd42700 1 -- 172.21.15.7:0/2255402661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5000b320 con 0x7f5d5806ad40 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2255402661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806ad40 msgr2=0x7f5d5806b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2255402661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806ad40 0x7f5d5806b1c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5d5806d180 tx=0x7f5d5000ba50 comp rx=0 tx=0).stop 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2255402661 shutdown_connections 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2255402661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806ad40 0x7f5d5806b1c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2255402661 >> v2:172.21.15.113:3300/0 conn(0x7f5d5806de30 0x7f5d58154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2255402661 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 0x7f5d58154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2255402661 >> 172.21.15.7:0/2255402661 conn(0x7f5d580cd8f0 msgr2=0x7f5d580cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2255402661 shutdown_connections 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.123+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2255402661 wait complete. 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.124+0000 7f5d5ffa8700 1 Processor -- start 2024-01-11T07:06:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.124+0000 7f5d5ffa8700 1 -- start start 2024-01-11T07:06:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5ffa8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5806ad40 0x7f5d5816add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5ffa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806de30 0x7f5d58155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5ffa8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 0x7f5d580aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5ffa8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d581554a0 con 0x7f5d5806de30 2024-01-11T07:06:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5ffa8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5816b310 con 0x7f5d5806ad40 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5ffa8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d580b27b0 con 0x7f5d5806e200 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5d543700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806de30 0x7f5d58155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5d543700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806de30 0x7f5d58155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59732/0 (socket says 172.21.15.7:59732) 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5d543700 1 -- 172.21.15.7:0/2266031297 learned_addr learned my addr 172.21.15.7:0/2266031297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5e545700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 0x7f5d580aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.125+0000 7f5d5dd44700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5806ad40 0x7f5d5816add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5d543700 1 -- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 msgr2=0x7f5d580aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5d543700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 0x7f5d580aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5d543700 1 -- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5806ad40 msgr2=0x7f5d5816add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5d543700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5806ad40 0x7f5d5816add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5d543700 1 -- 172.21.15.7:0/2266031297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5000bcf0 con 0x7f5d5806de30 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5dd44700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5806ad40 0x7f5d5816add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5d543700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806de30 0x7f5d58155930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d860 tx=0x7f5d54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d4dffb700 1 -- 172.21.15.7:0/2266031297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540125f0 con 0x7f5d5806de30 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d4dffb700 1 -- 172.21.15.7:0/2266031297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d54012750 con 0x7f5d5806de30 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d580b2a90 con 0x7f5d5806de30 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d4dffb700 1 -- 172.21.15.7:0/2266031297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54012b10 con 0x7f5d5806de30 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.126+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d58154e50 con 0x7f5d5806de30 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.127+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d58064a40 con 0x7f5d5806de30 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.128+0000 7f5d4dffb700 1 -- 172.21.15.7:0/2266031297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5d5401e460 con 0x7f5d5806de30 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.128+0000 7f5d4dffb700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d3c066f30 0x7f5d3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.128+0000 7f5d4dffb700 1 -- 172.21.15.7:0/2266031297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f5d54084a70 con 0x7f5d5806de30 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.128+0000 7f5d5dd44700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d3c066f30 0x7f5d3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.129+0000 7f5d5dd44700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d3c066f30 0x7f5d3c0693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5d48006fd0 tx=0x7f5d48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.133+0000 7f5d4dffb700 1 -- 172.21.15.7:0/2266031297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d54051d40 con 0x7f5d5806de30 2024-01-11T07:06:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:58.383+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5d5804c3c0 con 0x7f5d5806de30 2024-01-11T07:06:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.151+0000 7f5d4dffb700 1 -- 172.21.15.7:0/2266031297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5d54055360 con 0x7f5d5806de30 2024-01-11T07:06:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d3c066f30 msgr2=0x7f5d3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d3c066f30 0x7f5d3c0693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5d48006fd0 tx=0x7f5d48008040 comp rx=0 tx=0).stop 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806de30 msgr2=0x7f5d58155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806de30 0x7f5d58155930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d860 tx=0x7f5d54008040 comp rx=0 tx=0).stop 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 shutdown_connections 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d3c066f30 0x7f5d3c0693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d5806e200 0x7f5d580aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d5806ad40 0x7f5d5816add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 --2- 172.21.15.7:0/2266031297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5806de30 0x7f5d58155930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 >> 172.21.15.7:0/2266031297 conn(0x7f5d580cd8f0 msgr2=0x7f5d58158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 shutdown_connections 2024-01-11T07:06:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.156+0000 7f5d5ffa8700 1 -- 172.21.15.7:0/2266031297 wait complete. 2024-01-11T07:06:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-01-11T07:06:59.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:06:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.307+0000 7f15676c8700 1 Processor -- start 2024-01-11T07:06:59.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.307+0000 7f15676c8700 1 -- start start 2024-01-11T07:06:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f15676c8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f156006e0d0 0x7f1560159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f15676c8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1560159ff0 0x7f1560069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f15676c8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156015b770 con 0x7f156006dd00 2024-01-11T07:06:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f15676c8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156015abf0 con 0x7f156006e0d0 2024-01-11T07:06:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f15676c8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156015aed0 con 0x7f1560159ff0 2024-01-11T07:06:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1564c63700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f156006e0d0 0x7f1560159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1564c63700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f156006e0d0 0x7f1560159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59744/0 (socket says 172.21.15.7:59744) 2024-01-11T07:06:59.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1564c63700 1 -- 172.21.15.7:0/3038652675 learned_addr learned my addr 172.21.15.7:0/3038652675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1565c65700 1 --2- 172.21.15.7:0/3038652675 >> v2:172.21.15.113:3300/0 conn(0x7f1560159ff0 0x7f1560069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1564c63700 1 -- 172.21.15.7:0/3038652675 >> v2:172.21.15.113:3300/0 conn(0x7f1560159ff0 msgr2=0x7f1560069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1564c63700 1 --2- 172.21.15.7:0/3038652675 >> v2:172.21.15.113:3300/0 conn(0x7f1560159ff0 0x7f1560069fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1564c63700 1 -- 172.21.15.7:0/3038652675 >> v1:172.21.15.113:6789/0 conn(0x7f156006dd00 legacy=0x7f15600c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1564c63700 1 -- 172.21.15.7:0/3038652675 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156015b1b0 con 0x7f156006e0d0 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.308+0000 7f1564c63700 1 --2- 172.21.15.7:0/3038652675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006e0d0 0x7f1560159ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f15600c7e00 tx=0x7f155c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892943d2c3a4aa95 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f1557fff700 1 -- 172.21.15.7:0/3038652675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c009e70 con 0x7f156006e0d0 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f1557fff700 1 -- 172.21.15.7:0/3038652675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c00b860 con 0x7f156006e0d0 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f1557fff700 1 -- 172.21.15.7:0/3038652675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00bc00 con 0x7f156006e0d0 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f15676c8700 1 -- 172.21.15.7:0/3038652675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006e0d0 msgr2=0x7f1560159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f15676c8700 1 --2- 172.21.15.7:0/3038652675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006e0d0 0x7f1560159ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f15600c7e00 tx=0x7f155c009250 comp rx=0 tx=0).stop 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f15676c8700 1 -- 172.21.15.7:0/3038652675 shutdown_connections 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f15676c8700 1 --2- 172.21.15.7:0/3038652675 >> v2:172.21.15.113:3300/0 conn(0x7f1560159ff0 0x7f1560069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f15676c8700 1 --2- 172.21.15.7:0/3038652675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006e0d0 0x7f1560159ab0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f15676c8700 1 -- 172.21.15.7:0/3038652675 >> 172.21.15.7:0/3038652675 conn(0x7f15600680e0 msgr2=0x7f15600cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f15676c8700 1 -- 172.21.15.7:0/3038652675 shutdown_connections 2024-01-11T07:06:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.309+0000 7f15676c8700 1 -- 172.21.15.7:0/3038652675 wait complete. 2024-01-11T07:06:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.311+0000 7f15676c8700 1 Processor -- start 2024-01-11T07:06:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.311+0000 7f15676c8700 1 -- start start 2024-01-11T07:06:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.311+0000 7f15676c8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006dd00 0x7f1560158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.311+0000 7f15676c8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156006e0d0 0x7f15600c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15676c8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1560159ff0 0x7f156015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15676c8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600ad6c0 con 0x7f156006dd00 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15676c8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600ad910 con 0x7f156006e0d0 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15676c8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15600adbf0 con 0x7f1560159ff0 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006dd00 0x7f1560158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006dd00 0x7f1560158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59754/0 (socket says 172.21.15.7:59754) 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 -- 172.21.15.7:0/2144354027 learned_addr learned my addr 172.21.15.7:0/2144354027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565c65700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1560159ff0 0x7f156015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1564c63700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156006e0d0 0x7f15600c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 -- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1560159ff0 msgr2=0x7f156015c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1560159ff0 0x7f156015c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 -- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156006e0d0 msgr2=0x7f15600c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156006e0d0 0x7f15600c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 -- 172.21.15.7:0/2144354027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c0094f0 con 0x7f156006dd00 2024-01-11T07:06:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f1565464700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006dd00 0x7f1560158860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f155000c590 tx=0x7f155000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15557fa700 1 -- 172.21.15.7:0/2144354027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550011590 con 0x7f156006dd00 2024-01-11T07:06:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15557fa700 1 -- 172.21.15.7:0/2144354027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1550011710 con 0x7f156006dd00 2024-01-11T07:06:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15600aded0 con 0x7f156006dd00 2024-01-11T07:06:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15557fa700 1 -- 172.21.15.7:0/2144354027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550011ad0 con 0x7f156006dd00 2024-01-11T07:06:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.312+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15600ab810 con 0x7f156006dd00 2024-01-11T07:06:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.313+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156004c3c0 con 0x7f156006dd00 2024-01-11T07:06:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.314+0000 7f15557fa700 1 -- 172.21.15.7:0/2144354027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f155001b3b0 con 0x7f156006dd00 2024-01-11T07:06:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.314+0000 7f15557fa700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.314+0000 7f15557fa700 1 -- 172.21.15.7:0/2144354027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15500839a0 con 0x7f156006dd00 2024-01-11T07:06:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.314+0000 7f1564c63700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.315+0000 7f1564c63700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1544066f30 0x7f15440693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15600c7e00 tx=0x7f155c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:59.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.319+0000 7f15557fa700 1 -- 172.21.15.7:0/2144354027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1550050c10 con 0x7f156006dd00 2024-01-11T07:06:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.522+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15600ae3f0 con 0x7f156006dd00 2024-01-11T07:06:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.523+0000 7f15557fa700 1 -- 172.21.15.7:0/2144354027 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f15600ae3f0 con 0x7f156006dd00 2024-01-11T07:06:59.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:59.522 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:58.384920+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:06:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1544066f30 msgr2=0x7f15440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1544066f30 0x7f15440693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15600c7e00 tx=0x7f155c017000 comp rx=0 tx=0).stop 2024-01-11T07:06:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006dd00 msgr2=0x7f1560158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006dd00 0x7f1560158860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f155000c590 tx=0x7f155000c8a0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 shutdown_connections 2024-01-11T07:06:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1544066f30 0x7f15440693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1560159ff0 0x7f156015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f156006dd00 0x7f1560158860 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 --2- 172.21.15.7:0/2144354027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156006e0d0 0x7f15600c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 >> 172.21.15.7:0/2144354027 conn(0x7f15600680e0 msgr2=0x7f15600cd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.525+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 shutdown_connections 2024-01-11T07:06:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.526+0000 7f15676c8700 1 -- 172.21.15.7:0/2144354027 wait complete. 2024-01-11T07:06:59.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:06:59.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.670+0000 7f1aa21e0700 1 Processor -- start 2024-01-11T07:06:59.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.670+0000 7f1aa21e0700 1 -- start start 2024-01-11T07:06:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.671+0000 7f1aa21e0700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a9c069320 0x7f1a9c1649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.671+0000 7f1aa21e0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9c0ce9b0 con 0x7f1a9c159c90 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.671+0000 7f1aa21e0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9c0cf530 con 0x7f1a9c15a060 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.671+0000 7f1aa21e0700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9c0b1f30 con 0x7f1a9c069320 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.671+0000 7f1a9b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1a9c159c90 0x7f1a9c0b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38266/0 (socket says 172.21.15.7:38266) 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.671+0000 7f1a9b7fe700 1 -- 172.21.15.7:0/2586731753 learned_addr learned my addr 172.21.15.7:0/2586731753 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9bfff700 1 --2- 172.21.15.7:0/2586731753 >> v2:172.21.15.113:3300/0 conn(0x7f1a9c069320 0x7f1a9c1649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648499938 0 0) 0x7f1a9c0ce9b0 con 0x7f1a9c159c90 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a88003570 con 0x7f1a9c159c90 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3582015687 0 0) 0x7f1a88003570 con 0x7f1a9c159c90 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 >> v2:172.21.15.113:3300/0 conn(0x7f1a9c069320 msgr2=0x7f1a9c1649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 --2- 172.21.15.7:0/2586731753 >> v2:172.21.15.113:3300/0 conn(0x7f1a9c069320 0x7f1a9c1649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 >> v1:172.21.15.113:6789/0 conn(0x7f1a9c15a060 legacy=0x7f1a9c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c0cf9c0 con 0x7f1a9c159c90 2024-01-11T07:06:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f1a8c001470 con 0x7f1a9c159c90 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9bfff700 1 --2- 172.21.15.7:0/2586731753 >> v2:172.21.15.113:3300/0 conn(0x7f1a9c069320 0x7f1a9c1649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 >> v1:172.21.15.7:6789/0 conn(0x7f1a9c159c90 legacy=0x7f1a9c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a88003a60 0x7f1a88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a88004810 0x7f1a88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a880051c0 0x7f1a8800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8800c130 con 0x7f1a880051c0 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8800c2a0 con 0x7f1a88004810 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8800c460 con 0x7f1a88003a60 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9bfff700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a880051c0 0x7f1a8800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9b7fe700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a88003a60 0x7f1a88003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.672+0000 7f1a9affd700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a88004810 0x7f1a88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9bfff700 1 -- 172.21.15.7:0/2586731753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a88003a60 msgr2=0x7f1a88003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9bfff700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a88003a60 0x7f1a88003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9bfff700 1 -- 172.21.15.7:0/2586731753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a88004810 msgr2=0x7f1a88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:06:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9bfff700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a88004810 0x7f1a88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9bfff700 1 -- 172.21.15.7:0/2586731753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a88003570 con 0x7f1a880051c0 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9bfff700 1 -- 172.21.15.7:0/2586731753 reap_dead start 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9bfff700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a880051c0 0x7f1a8800ba60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e3c0 tx=0x7f1a9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a44cc246c5a5eb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94009c50 con 0x7f1a880051c0 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400f040 con 0x7f1a880051c0 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.673+0000 7f1a9a7fc700 1 -- 172.21.15.7:0/2586731753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400eec0 con 0x7f1a880051c0 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.674+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2586731753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a880051c0 msgr2=0x7f1a8800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.674+0000 7f1aa21e0700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a880051c0 0x7f1a8800ba60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e3c0 tx=0x7f1a9400b040 comp rx=0 tx=0).stop 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.674+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2586731753 shutdown_connections 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.674+0000 7f1aa21e0700 1 --2- 172.21.15.7:0/2586731753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a880051c0 0x7f1a8800ba60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.674+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2586731753 >> 172.21.15.7:0/2586731753 conn(0x7f1a9c071100 msgr2=0x7f1a9c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.674+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2586731753 shutdown_connections 2024-01-11T07:06:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.674+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2586731753 wait complete. 2024-01-11T07:06:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.675+0000 7f1aa21e0700 1 Processor -- start 2024-01-11T07:06:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.675+0000 7f1aa21e0700 1 -- start start 2024-01-11T07:06:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1aa21e0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880051c0 0x7f1a9c0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1aa21e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9c0b02b0 0x7f1a9c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1aa21e0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a9c14f730 0x7f1a9c15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1aa21e0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c16e350 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1aa21e0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c16e7e0 con 0x7f1a880051c0 2024-01-11T07:06:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1aa21e0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c16da20 con 0x7f1a9c14f730 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9c0b02b0 0x7f1a9c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9c0b02b0 0x7f1a9c15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59772/0 (socket says 172.21.15.7:59772) 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9b7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880051c0 0x7f1a9c0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9affd700 1 -- 172.21.15.7:0/2890576488 learned_addr learned my addr 172.21.15.7:0/2890576488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9bfff700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a9c14f730 0x7f1a9c15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9affd700 1 -- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a9c14f730 msgr2=0x7f1a9c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9affd700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a9c14f730 0x7f1a9c15ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9affd700 1 -- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880051c0 msgr2=0x7f1a9c0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9affd700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880051c0 0x7f1a9c0cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.676+0000 7f1a9affd700 1 -- 172.21.15.7:0/2890576488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c16dd00 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.677+0000 7f1a9affd700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9c0b02b0 0x7f1a9c15c550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1a84002980 tx=0x7f1a84007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.677+0000 7f1a937fe700 1 -- 172.21.15.7:0/2890576488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84013600 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.677+0000 7f1a937fe700 1 -- 172.21.15.7:0/2890576488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a84013760 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.677+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c16dfe0 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.677+0000 7f1a937fe700 1 -- 172.21.15.7:0/2890576488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84013b20 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.677+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c0adcd0 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.678+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c16e630 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.678+0000 7f1a937fe700 1 -- 172.21.15.7:0/2890576488 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1a8400f400 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.679+0000 7f1a937fe700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a7c066f30 0x7f1a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.679+0000 7f1a937fe700 1 -- 172.21.15.7:0/2890576488 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a840841e0 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.679+0000 7f1a9b7fe700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a7c066f30 0x7f1a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:06:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.679+0000 7f1a9b7fe700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a7c066f30 0x7f1a7c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c001050 tx=0x7f1a8c001e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:06:59.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.683+0000 7f1a937fe700 1 -- 172.21.15.7:0/2890576488 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a84051450 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.886+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a9c16e170 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.887+0000 7f1a937fe700 1 -- 172.21.15.7:0/2890576488 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f1a84054a70 con 0x7f1a9c0b02b0 2024-01-11T07:06:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:06:59.888 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:58.384920+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:06:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.889+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a7c066f30 msgr2=0x7f1a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.889+0000 7f1aa21e0700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a7c066f30 0x7f1a7c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c001050 tx=0x7f1a8c001e50 comp rx=0 tx=0).stop 2024-01-11T07:06:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.889+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9c0b02b0 msgr2=0x7f1a9c15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:06:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.889+0000 7f1aa21e0700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9c0b02b0 0x7f1a9c15c550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1a84002980 tx=0x7f1a84007d30 comp rx=0 tx=0).stop 2024-01-11T07:06:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.890+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 shutdown_connections 2024-01-11T07:06:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.890+0000 7f1aa21e0700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a7c066f30 0x7f1a7c0693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.890+0000 7f1aa21e0700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a9c14f730 0x7f1a9c15ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.890+0000 7f1aa21e0700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880051c0 0x7f1a9c0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.890+0000 7f1aa21e0700 1 --2- 172.21.15.7:0/2890576488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a9c0b02b0 0x7f1a9c15c550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:06:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.890+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 >> 172.21.15.7:0/2890576488 conn(0x7f1a9c071100 msgr2=0x7f1a9c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:06:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.890+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 shutdown_connections 2024-01-11T07:06:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:06:59.890+0000 7f1aa21e0700 1 -- 172.21.15.7:0/2890576488 wait complete. 2024-01-11T07:06:59.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:07:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.041+0000 7fc39d937700 1 Processor -- start 2024-01-11T07:07:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.041+0000 7fc39d937700 1 -- start start 2024-01-11T07:07:00.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc39d937700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc39806e340 0x7fc398164a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:00.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc39d937700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3980afeb0 con 0x7fc3980d16e0 2024-01-11T07:07:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc39d937700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc39815c080 con 0x7fc398159e30 2024-01-11T07:07:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc39d937700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980b00f0 con 0x7fc39806e340 2024-01-11T07:07:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc3967fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc3980d16e0 0x7fc39806dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38268/0 (socket says 172.21.15.7:38268) 2024-01-11T07:07:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc3967fc700 1 -- 172.21.15.7:0/1492708522 learned_addr learned my addr 172.21.15.7:0/1492708522 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc3977fe700 1 --2- 172.21.15.7:0/1492708522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc39806e340 0x7fc398164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc395ffb700 1 -- 172.21.15.7:0/1492708522 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762967346 0 0) 0x7fc380001110 con 0x7fc3980d16e0 2024-01-11T07:07:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.042+0000 7fc395ffb700 1 -- 172.21.15.7:0/1492708522 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc384003570 con 0x7fc3980d16e0 2024-01-11T07:07:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc395ffb700 1 -- 172.21.15.7:0/1492708522 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986933473 0 0) 0x7fc39815c080 con 0x7fc398159e30 2024-01-11T07:07:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc395ffb700 1 -- 172.21.15.7:0/1492708522 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc380001110 con 0x7fc398159e30 2024-01-11T07:07:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc3977fe700 1 -- 172.21.15.7:0/1492708522 >> v1:172.21.15.113:6789/0 conn(0x7fc398159e30 legacy=0x7fc3980af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc3977fe700 1 -- 172.21.15.7:0/1492708522 >> v1:172.21.15.7:6789/0 conn(0x7fc3980d16e0 legacy=0x7fc39806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc395ffb700 1 -- 172.21.15.7:0/1492708522 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1492274802 0 0) 0x7fc384003570 con 0x7fc3980d16e0 2024-01-11T07:07:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc3977fe700 1 -- 172.21.15.7:0/1492708522 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39815ba30 con 0x7fc39806e340 2024-01-11T07:07:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc3977fe700 1 --2- 172.21.15.7:0/1492708522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc39806e340 0x7fc398164a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc390009e90 tx=0x7fc390009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d68840ff3ca8f4e7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc395ffb700 1 -- 172.21.15.7:0/1492708522 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39000e070 con 0x7fc39806e340 2024-01-11T07:07:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc395ffb700 1 -- 172.21.15.7:0/1492708522 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39000b870 con 0x7fc39806e340 2024-01-11T07:07:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.043+0000 7fc395ffb700 1 -- 172.21.15.7:0/1492708522 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380001600 con 0x7fc39806e340 2024-01-11T07:07:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.044+0000 7fc39d937700 1 -- 172.21.15.7:0/1492708522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc39806e340 msgr2=0x7fc398164a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.044+0000 7fc39d937700 1 --2- 172.21.15.7:0/1492708522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc39806e340 0x7fc398164a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc390009e90 tx=0x7fc390009780 comp rx=0 tx=0).stop 2024-01-11T07:07:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.044+0000 7fc39d937700 1 -- 172.21.15.7:0/1492708522 shutdown_connections 2024-01-11T07:07:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.044+0000 7fc39d937700 1 --2- 172.21.15.7:0/1492708522 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc39806e340 0x7fc398164a40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.044+0000 7fc39d937700 1 -- 172.21.15.7:0/1492708522 >> 172.21.15.7:0/1492708522 conn(0x7fc398063d00 msgr2=0x7fc3980d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.044+0000 7fc39d937700 1 -- 172.21.15.7:0/1492708522 shutdown_connections 2024-01-11T07:07:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.044+0000 7fc39d937700 1 -- 172.21.15.7:0/1492708522 wait complete. 2024-01-11T07:07:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.045+0000 7fc39d937700 1 Processor -- start 2024-01-11T07:07:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.045+0000 7fc39d937700 1 -- start start 2024-01-11T07:07:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc39d937700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc39806e340 0x7fc3980c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc39d937700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3980d16e0 0x7fc398158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc39d937700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc398159e30 0x7fc398158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc39d937700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980b39c0 con 0x7fc39806e340 2024-01-11T07:07:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc39d937700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980b3ca0 con 0x7fc398159e30 2024-01-11T07:07:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc39d937700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980b3f80 con 0x7fc3980d16e0 2024-01-11T07:07:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc396ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc39806e340 0x7fc3980c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc396ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc39806e340 0x7fc3980c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59788/0 (socket says 172.21.15.7:59788) 2024-01-11T07:07:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc396ffd700 1 -- 172.21.15.7:0/2572408827 learned_addr learned my addr 172.21.15.7:0/2572408827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc3967fc700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3980d16e0 0x7fc398158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc3977fe700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc398159e30 0x7fc398158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc396ffd700 1 -- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3980d16e0 msgr2=0x7fc398158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc396ffd700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3980d16e0 0x7fc398158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc396ffd700 1 -- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc398159e30 msgr2=0x7fc398158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc396ffd700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc398159e30 0x7fc398158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.046+0000 7fc396ffd700 1 -- 172.21.15.7:0/2572408827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390009a20 con 0x7fc39806e340 2024-01-11T07:07:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.047+0000 7fc396ffd700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc39806e340 0x7fc3980c9dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c580 tx=0x7fc388007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:00.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.047+0000 7fc38effd700 1 -- 172.21.15.7:0/2572408827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388010510 con 0x7fc39806e340 2024-01-11T07:07:00.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.047+0000 7fc38effd700 1 -- 172.21.15.7:0/2572408827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc388010670 con 0x7fc39806e340 2024-01-11T07:07:00.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.047+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3980b4260 con 0x7fc39806e340 2024-01-11T07:07:00.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.047+0000 7fc38effd700 1 -- 172.21.15.7:0/2572408827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388010a30 con 0x7fc39806e340 2024-01-11T07:07:00.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.047+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3980b4820 con 0x7fc39806e340 2024-01-11T07:07:00.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.048+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39804c3c0 con 0x7fc39806e340 2024-01-11T07:07:00.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.048+0000 7fc38effd700 1 -- 172.21.15.7:0/2572408827 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc38801b400 con 0x7fc39806e340 2024-01-11T07:07:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.049+0000 7fc38effd700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc378066f30 0x7fc3780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.049+0000 7fc38effd700 1 -- 172.21.15.7:0/2572408827 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc388084110 con 0x7fc39806e340 2024-01-11T07:07:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.049+0000 7fc3967fc700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc378066f30 0x7fc3780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.050+0000 7fc3967fc700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc378066f30 0x7fc3780693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc380001ed0 tx=0x7fc380000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.053+0000 7fc38effd700 1 -- 172.21.15.7:0/2572408827 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc388051380 con 0x7fc39806e340 2024-01-11T07:07:00.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:07:00.139 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:07:00.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:00.310+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc3980b0450 con 0x7fc39806e340 2024-01-11T07:07:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.162+0000 7fc38effd700 1 -- 172.21.15.7:0/2572408827 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc3880549a0 con 0x7fc39806e340 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc378066f30 msgr2=0x7fc3780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc378066f30 0x7fc3780693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc380001ed0 tx=0x7fc380000bc0 comp rx=0 tx=0).stop 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc39806e340 msgr2=0x7fc3980c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc39806e340 0x7fc3980c9dd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c580 tx=0x7fc388007e10 comp rx=0 tx=0).stop 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 shutdown_connections 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc378066f30 0x7fc3780693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc398159e30 0x7fc398158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc39806e340 0x7fc3980c9dd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 --2- 172.21.15.7:0/2572408827 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3980d16e0 0x7fc398158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 >> 172.21.15.7:0/2572408827 conn(0x7fc398063d00 msgr2=0x7fc39815a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 shutdown_connections 2024-01-11T07:07:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.165+0000 7fc39d937700 1 -- 172.21.15.7:0/2572408827 wait complete. 2024-01-11T07:07:01.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:07:01.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.318+0000 7f9ce3025700 1 Processor -- start 2024-01-11T07:07:01.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.318+0000 7f9ce3025700 1 -- start start 2024-01-11T07:07:01.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9ce3025700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9cdc070690 0x7f9cdc0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9ce3025700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cdc070ac0 con 0x7f9cdc0b28b0 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9ce3025700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cdc0c99c0 con 0x7f9cdc157d50 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9ce3025700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cdc070d20 con 0x7f9cdc070690 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9cdbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9cdc0b28b0 0x7f9cdc0aca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38280/0 (socket says 172.21.15.7:38280) 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9cdbfff700 1 -- 172.21.15.7:0/2735388095 learned_addr learned my addr 172.21.15.7:0/2735388095 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9ce15c2700 1 --2- 172.21.15.7:0/2735388095 >> v2:172.21.15.113:3300/0 conn(0x7f9cdc070690 0x7f9cdc0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9cdb7fe700 1 -- 172.21.15.7:0/2735388095 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697252281 0 0) 0x7f9cdc070ac0 con 0x7f9cdc0b28b0 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9cdb7fe700 1 -- 172.21.15.7:0/2735388095 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ccc003570 con 0x7f9cdc0b28b0 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9ce15c2700 1 -- 172.21.15.7:0/2735388095 >> v1:172.21.15.113:6789/0 conn(0x7f9cdc157d50 legacy=0x7f9cdc0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.319+0000 7f9ce15c2700 1 -- 172.21.15.7:0/2735388095 >> v1:172.21.15.7:6789/0 conn(0x7f9cdc0b28b0 legacy=0x7f9cdc0aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.320+0000 7f9cdb7fe700 1 -- 172.21.15.7:0/2735388095 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 754513346 0 0) 0x7f9ccc003570 con 0x7f9cdc0b28b0 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.320+0000 7f9ce15c2700 1 -- 172.21.15.7:0/2735388095 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc0cc490 con 0x7f9cdc070690 2024-01-11T07:07:01.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.320+0000 7f9ce15c2700 1 --2- 172.21.15.7:0/2735388095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc070690 0x7f9cdc0c9480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009e60 tx=0x7f9cd4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2088ae935e3b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:01.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.320+0000 7f9cdb7fe700 1 -- 172.21.15.7:0/2735388095 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd400e070 con 0x7f9cdc070690 2024-01-11T07:07:01.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.320+0000 7f9cdb7fe700 1 -- 172.21.15.7:0/2735388095 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd400b850 con 0x7f9cdc070690 2024-01-11T07:07:01.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.320+0000 7f9cdb7fe700 1 -- 172.21.15.7:0/2735388095 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8001470 con 0x7f9cdc070690 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.320+0000 7f9ce3025700 1 -- 172.21.15.7:0/2735388095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc070690 msgr2=0x7f9cdc0c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.320+0000 7f9ce3025700 1 --2- 172.21.15.7:0/2735388095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc070690 0x7f9cdc0c9480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009e60 tx=0x7f9cd4009780 comp rx=0 tx=0).stop 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.321+0000 7f9ce3025700 1 -- 172.21.15.7:0/2735388095 shutdown_connections 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.321+0000 7f9ce3025700 1 --2- 172.21.15.7:0/2735388095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc070690 0x7f9cdc0c9480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.321+0000 7f9ce3025700 1 -- 172.21.15.7:0/2735388095 >> 172.21.15.7:0/2735388095 conn(0x7f9cdc068080 msgr2=0x7f9cdc0cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.321+0000 7f9ce3025700 1 -- 172.21.15.7:0/2735388095 shutdown_connections 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.321+0000 7f9ce3025700 1 -- 172.21.15.7:0/2735388095 wait complete. 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.322+0000 7f9ce3025700 1 Processor -- start 2024-01-11T07:07:01.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.322+0000 7f9ce3025700 1 -- start start 2024-01-11T07:07:01.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce3025700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cdc070690 0x7f9cdc14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:01.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce3025700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc0b28b0 0x7f9cdc16aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:01.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce3025700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cdc157d50 0x7f9cdc159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:01.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce3025700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc159500 con 0x7f9cdc157d50 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce3025700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc16b820 con 0x7f9cdc0b28b0 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce3025700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc16bb00 con 0x7f9cdc070690 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce15c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cdc157d50 0x7f9cdc159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce15c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cdc157d50 0x7f9cdc159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59790/0 (socket says 172.21.15.7:59790) 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce15c2700 1 -- 172.21.15.7:0/1109934069 learned_addr learned my addr 172.21.15.7:0/1109934069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9cdbfff700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc0b28b0 0x7f9cdc16aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce0dc1700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cdc070690 0x7f9cdc14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce15c2700 1 -- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cdc070690 msgr2=0x7f9cdc14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce15c2700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cdc070690 0x7f9cdc14ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce15c2700 1 -- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc0b28b0 msgr2=0x7f9cdc16aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce15c2700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc0b28b0 0x7f9cdc16aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce15c2700 1 -- 172.21.15.7:0/1109934069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd4009a20 con 0x7f9cdc157d50 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.323+0000 7f9ce0dc1700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cdc070690 0x7f9cdc14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.324+0000 7f9ce15c2700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cdc157d50 0x7f9cdc159990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc070c50 tx=0x7f9cd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.324+0000 7f9cd8ff9700 1 -- 172.21.15.7:0/1109934069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4012810 con 0x7f9cdc157d50 2024-01-11T07:07:01.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.324+0000 7f9cd8ff9700 1 -- 172.21.15.7:0/1109934069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd4012970 con 0x7f9cdc157d50 2024-01-11T07:07:01.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.324+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc159190 con 0x7f9cdc157d50 2024-01-11T07:07:01.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.324+0000 7f9cd8ff9700 1 -- 172.21.15.7:0/1109934069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4012d30 con 0x7f9cdc157d50 2024-01-11T07:07:01.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.324+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc159ed0 con 0x7f9cdc157d50 2024-01-11T07:07:01.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.325+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cdc15a0f0 con 0x7f9cdc157d50 2024-01-11T07:07:01.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.325+0000 7f9cd8ff9700 1 -- 172.21.15.7:0/1109934069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9cd4036440 con 0x7f9cdc157d50 2024-01-11T07:07:01.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.326+0000 7f9cd8ff9700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:01.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.326+0000 7f9cd8ff9700 1 -- 172.21.15.7:0/1109934069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9cd408ea90 con 0x7f9cdc157d50 2024-01-11T07:07:01.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.326+0000 7f9ce0dc1700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:01.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.327+0000 7f9ce0dc1700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9cc0066f30 0x7f9cc00693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0c9b50 tx=0x7f9cd0010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:01.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.331+0000 7f9cd8ff9700 1 -- 172.21.15.7:0/1109934069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cdc15a0f0 con 0x7f9cdc157d50 2024-01-11T07:07:01.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:01.580+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9cdc15a0f0 con 0x7f9cdc157d50 2024-01-11T07:07:02.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.168+0000 7f9cd8ff9700 1 -- 172.21.15.7:0/1109934069 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9cdc15a0f0 con 0x7f9cdc157d50 2024-01-11T07:07:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9cc0066f30 msgr2=0x7f9cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9cc0066f30 0x7f9cc00693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0c9b50 tx=0x7f9cd0010040 comp rx=0 tx=0).stop 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cdc157d50 msgr2=0x7f9cdc159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cdc157d50 0x7f9cdc159990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc070c50 tx=0x7f9cd4017000 comp rx=0 tx=0).stop 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 shutdown_connections 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9cc0066f30 0x7f9cc00693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cdc157d50 0x7f9cdc159990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cdc070690 0x7f9cdc14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 --2- 172.21.15.7:0/1109934069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cdc0b28b0 0x7f9cdc16aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 >> 172.21.15.7:0/1109934069 conn(0x7f9cdc068080 msgr2=0x7f9cdc152fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 shutdown_connections 2024-01-11T07:07:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.171+0000 7f9ce3025700 1 -- 172.21.15.7:0/1109934069 wait complete. 2024-01-11T07:07:02.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:02.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.321+0000 7f7c352e7700 1 Processor -- start 2024-01-11T07:07:02.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.321+0000 7f7c352e7700 1 -- start start 2024-01-11T07:07:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c352e7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c30159d80 0x7f7c3006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c352e7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c3006e170 0x7f7c3015da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c352e7700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c300d0220 con 0x7f7c3015dfa0 2024-01-11T07:07:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c352e7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3006e610 con 0x7f7c3006e170 2024-01-11T07:07:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c352e7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c300d06b0 con 0x7f7c30159d80 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c2e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c3006e170 0x7f7c3015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c2e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c3006e170 0x7f7c3015da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59804/0 (socket says 172.21.15.7:59804) 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/3577645171 learned_addr learned my addr 172.21.15.7:0/3577645171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.322+0000 7f7c2effd700 1 --2- 172.21.15.7:0/3577645171 >> v2:172.21.15.113:3300/0 conn(0x7f7c30159d80 0x7f7c3006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.323+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/3577645171 >> v2:172.21.15.113:3300/0 conn(0x7f7c30159d80 msgr2=0x7f7c3006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.323+0000 7f7c2e7fc700 1 --2- 172.21.15.7:0/3577645171 >> v2:172.21.15.113:3300/0 conn(0x7f7c30159d80 0x7f7c3006dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.323+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/3577645171 >> v1:172.21.15.113:6789/0 conn(0x7f7c3015dfa0 legacy=0x7f7c3015aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.323+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/3577645171 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c300cf610 con 0x7f7c3006e170 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.323+0000 7f7c2e7fc700 1 --2- 172.21.15.7:0/3577645171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3006e170 0x7f7c3015da60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7c30063ce0 tx=0x7f7c18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a080c15849fc4f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.323+0000 7f7c2dffb700 1 -- 172.21.15.7:0/3577645171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18009e70 con 0x7f7c3006e170 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.323+0000 7f7c2dffb700 1 -- 172.21.15.7:0/3577645171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1800b860 con 0x7f7c3006e170 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.323+0000 7f7c2dffb700 1 -- 172.21.15.7:0/3577645171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800bc00 con 0x7f7c3006e170 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.324+0000 7f7c352e7700 1 -- 172.21.15.7:0/3577645171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3006e170 msgr2=0x7f7c3015da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.324+0000 7f7c352e7700 1 --2- 172.21.15.7:0/3577645171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3006e170 0x7f7c3015da60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7c30063ce0 tx=0x7f7c18009250 comp rx=0 tx=0).stop 2024-01-11T07:07:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.324+0000 7f7c352e7700 1 -- 172.21.15.7:0/3577645171 shutdown_connections 2024-01-11T07:07:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.324+0000 7f7c352e7700 1 --2- 172.21.15.7:0/3577645171 >> v2:172.21.15.113:3300/0 conn(0x7f7c30159d80 0x7f7c3006dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.324+0000 7f7c352e7700 1 --2- 172.21.15.7:0/3577645171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3006e170 0x7f7c3015da60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.324+0000 7f7c352e7700 1 -- 172.21.15.7:0/3577645171 >> 172.21.15.7:0/3577645171 conn(0x7f7c30071130 msgr2=0x7f7c300b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.324+0000 7f7c352e7700 1 -- 172.21.15.7:0/3577645171 shutdown_connections 2024-01-11T07:07:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.324+0000 7f7c352e7700 1 -- 172.21.15.7:0/3577645171 wait complete. 2024-01-11T07:07:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.325+0000 7f7c352e7700 1 Processor -- start 2024-01-11T07:07:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.325+0000 7f7c352e7700 1 -- start start 2024-01-11T07:07:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c352e7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c3006e170 0x7f7c300afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c352e7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c30159d80 0x7f7c3015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c352e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3015dfa0 0x7f7c301583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c352e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300b0020 con 0x7f7c3015dfa0 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c352e7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c30145f40 con 0x7f7c30159d80 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c352e7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3016bf30 con 0x7f7c3006e170 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c2effd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c3006e170 0x7f7c300afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c2e7fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c30159d80 0x7f7c3015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c2f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3015dfa0 0x7f7c301583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c2effd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c3006e170 0x7f7c300afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:51038/0 (socket says 172.21.15.7:51038) 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c2e7fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c30159d80 0x7f7c3015cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:39474/0 (socket says 172.21.15.7:39474) 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.326+0000 7f7c2effd700 1 -- 172.21.15.7:0/2288564475 learned_addr learned my addr 172.21.15.7:0/2288564475 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c2f7fe700 1 -- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c3006e170 msgr2=0x7f7c300afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c2f7fe700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c3006e170 0x7f7c300afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c2f7fe700 1 -- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c30159d80 msgr2=0x7f7c3015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c2f7fe700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c30159d80 0x7f7c3015cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c2f7fe700 1 -- 172.21.15.7:0/2288564475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c180094f0 con 0x7f7c3015dfa0 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c2effd700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c3006e170 0x7f7c300afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c2e7fc700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c30159d80 0x7f7c3015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c2f7fe700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3015dfa0 0x7f7c301583a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c7c0 tx=0x7f7c280015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c26ffd700 1 -- 172.21.15.7:0/2288564475 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28011590 con 0x7f7c3015dfa0 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c26ffd700 1 -- 172.21.15.7:0/2288564475 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28011710 con 0x7f7c3015dfa0 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3016b320 con 0x7f7c3015dfa0 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c26ffd700 1 -- 172.21.15.7:0/2288564475 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28011ad0 con 0x7f7c3015dfa0 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.327+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3016b8e0 con 0x7f7c3015dfa0 2024-01-11T07:07:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.328+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c30089080 con 0x7f7c3015dfa0 2024-01-11T07:07:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.328+0000 7f7c26ffd700 1 -- 172.21.15.7:0/2288564475 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7c28003030 con 0x7f7c3015dfa0 2024-01-11T07:07:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.329+0000 7f7c26ffd700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c10066f30 0x7f7c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.329+0000 7f7c26ffd700 1 -- 172.21.15.7:0/2288564475 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7c28084020 con 0x7f7c3015dfa0 2024-01-11T07:07:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.329+0000 7f7c2effd700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c10066f30 0x7f7c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.330+0000 7f7c2effd700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c10066f30 0x7f7c100693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009fd0 tx=0x7f7c200091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.333+0000 7f7c26ffd700 1 -- 172.21.15.7:0/2288564475 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c28051290 con 0x7f7c3015dfa0 2024-01-11T07:07:02.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.588+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c3004c3c0 con 0x7f7c3015dfa0 2024-01-11T07:07:02.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.589+0000 7f7c26ffd700 1 -- 172.21.15.7:0/2288564475 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f7c280548b0 con 0x7f7c3015dfa0 2024-01-11T07:07:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:02.589 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:06:58.385324+0000","modified":"2024-01-11T07:07:02.162269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-11T07:07:02.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c10066f30 msgr2=0x7f7c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c10066f30 0x7f7c100693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009fd0 tx=0x7f7c200091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3015dfa0 msgr2=0x7f7c301583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3015dfa0 0x7f7c301583a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c7c0 tx=0x7f7c280015f0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 shutdown_connections 2024-01-11T07:07:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c10066f30 0x7f7c100693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3015dfa0 0x7f7c301583a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c3006e170 0x7f7c300afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.591+0000 7f7c352e7700 1 --2- 172.21.15.7:0/2288564475 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c30159d80 0x7f7c3015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.592+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 >> 172.21.15.7:0/2288564475 conn(0x7f7c30071130 msgr2=0x7f7c301576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.592+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 shutdown_connections 2024-01-11T07:07:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.592+0000 7f7c352e7700 1 -- 172.21.15.7:0/2288564475 wait complete. 2024-01-11T07:07:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T07:07:02.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:02.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.736+0000 7fd861b1e700 1 Processor -- start 2024-01-11T07:07:02.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.736+0000 7fd861b1e700 1 -- start start 2024-01-11T07:07:02.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd861b1e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c158280 0x7fd85c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd861b1e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c159300 con 0x7fd85c15aa80 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd861b1e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c0cf000 con 0x7fd85c0cafb0 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd861b1e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0cf260 con 0x7fd85c158280 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd85bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd85c15aa80 0x7fd85c161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38294/0 (socket says 172.21.15.7:38294) 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd85bfff700 1 -- 172.21.15.7:0/1090157218 learned_addr learned my addr 172.21.15.7:0/1090157218 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd85affd700 1 --2- 172.21.15.7:0/1090157218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c158280 0x7fd85c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd85a7fc700 1 -- 172.21.15.7:0/1090157218 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312829215 0 0) 0x7fd85c159300 con 0x7fd85c15aa80 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.737+0000 7fd85a7fc700 1 -- 172.21.15.7:0/1090157218 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd844003570 con 0x7fd85c15aa80 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.738+0000 7fd85a7fc700 1 -- 172.21.15.7:0/1090157218 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374671189 0 0) 0x7fd85c0cf000 con 0x7fd85c0cafb0 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.738+0000 7fd85affd700 1 -- 172.21.15.7:0/1090157218 >> v1:172.21.15.113:6789/0 conn(0x7fd85c0cafb0 legacy=0x7fd85c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.738+0000 7fd85affd700 1 -- 172.21.15.7:0/1090157218 >> v1:172.21.15.7:6789/0 conn(0x7fd85c15aa80 legacy=0x7fd85c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.738+0000 7fd85affd700 1 -- 172.21.15.7:0/1090157218 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0cf540 con 0x7fd85c158280 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.738+0000 7fd85affd700 1 --2- 172.21.15.7:0/1090157218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c158280 0x7fd85c0b0430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd850009910 tx=0x7fd8500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfb4c34aaa334aa3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.738+0000 7fd85a7fc700 1 -- 172.21.15.7:0/1090157218 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000b680 con 0x7fd85c158280 2024-01-11T07:07:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.738+0000 7fd85a7fc700 1 -- 172.21.15.7:0/1090157218 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85000b7e0 con 0x7fd85c158280 2024-01-11T07:07:02.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.738+0000 7fd85a7fc700 1 -- 172.21.15.7:0/1090157218 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000bba0 con 0x7fd85c158280 2024-01-11T07:07:02.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.739+0000 7fd861b1e700 1 -- 172.21.15.7:0/1090157218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c158280 msgr2=0x7fd85c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.739+0000 7fd861b1e700 1 --2- 172.21.15.7:0/1090157218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c158280 0x7fd85c0b0430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd850009910 tx=0x7fd8500091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.739+0000 7fd861b1e700 1 -- 172.21.15.7:0/1090157218 shutdown_connections 2024-01-11T07:07:02.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.739+0000 7fd861b1e700 1 --2- 172.21.15.7:0/1090157218 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c158280 0x7fd85c0b0430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.739+0000 7fd861b1e700 1 -- 172.21.15.7:0/1090157218 >> 172.21.15.7:0/1090157218 conn(0x7fd85c071100 msgr2=0x7fd85c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:02.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.739+0000 7fd861b1e700 1 -- 172.21.15.7:0/1090157218 shutdown_connections 2024-01-11T07:07:02.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.739+0000 7fd861b1e700 1 -- 172.21.15.7:0/1090157218 wait complete. 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.740+0000 7fd861b1e700 1 Processor -- start 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.741+0000 7fd861b1e700 1 -- start start 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.741+0000 7fd861b1e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c0cafb0 0x7fd85c067610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.741+0000 7fd861b1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c158280 0x7fd85c0691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd861b1e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c15aa80 0x7fd85c14a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd861b1e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c14abb0 con 0x7fd85c158280 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd861b1e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c067cf0 con 0x7fd85c15aa80 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd861b1e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c067fd0 con 0x7fd85c0cafb0 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c158280 0x7fd85c0691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c158280 0x7fd85c0691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59820/0 (socket says 172.21.15.7:59820) 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 -- 172.21.15.7:0/3470443633 learned_addr learned my addr 172.21.15.7:0/3470443633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85b7fe700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c0cafb0 0x7fd85c067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85bfff700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c15aa80 0x7fd85c14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 -- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c0cafb0 msgr2=0x7fd85c067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c0cafb0 0x7fd85c067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 -- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c15aa80 msgr2=0x7fd85c14a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c15aa80 0x7fd85c14a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 -- 172.21.15.7:0/3470443633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850009420 con 0x7fd85c158280 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85bfff700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c15aa80 0x7fd85c14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd85affd700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c158280 0x7fd85c0691d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8500093f0 tx=0x7fd850017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd8437fe700 1 -- 172.21.15.7:0/3470443633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85001f070 con 0x7fd85c158280 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd8437fe700 1 -- 172.21.15.7:0/3470443633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850000bc0 con 0x7fd85c158280 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c0682b0 con 0x7fd85c158280 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd8437fe700 1 -- 172.21.15.7:0/3470443633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000bba0 con 0x7fd85c158280 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.742+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c0b1800 con 0x7fd85c158280 2024-01-11T07:07:02.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.743+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c0b44a0 con 0x7fd85c158280 2024-01-11T07:07:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.744+0000 7fd8437fe700 1 -- 172.21.15.7:0/3470443633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd850023070 con 0x7fd85c158280 2024-01-11T07:07:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.744+0000 7fd8437fe700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd848066f30 0x7fd8480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.744+0000 7fd8437fe700 1 -- 172.21.15.7:0/3470443633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8500858c0 con 0x7fd85c158280 2024-01-11T07:07:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.744+0000 7fd85b7fe700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd848066f30 0x7fd8480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:02.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.745+0000 7fd85b7fe700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd848066f30 0x7fd8480693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0011e0 tx=0x7fd84c007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:02.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.749+0000 7fd8437fe700 1 -- 172.21.15.7:0/3470443633 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd850052b30 con 0x7fd85c158280 2024-01-11T07:07:02.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.956+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd85c0cf820 con 0x7fd85c158280 2024-01-11T07:07:02.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.957+0000 7fd8437fe700 1 -- 172.21.15.7:0/3470443633 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fd850056150 con 0x7fd85c158280 2024-01-11T07:07:02.959 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:02.959 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:02.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd848066f30 msgr2=0x7fd8480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd848066f30 0x7fd8480693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0011e0 tx=0x7fd84c007ce0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c158280 msgr2=0x7fd85c0691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:02.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c158280 0x7fd85c0691d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8500093f0 tx=0x7fd850017000 comp rx=0 tx=0).stop 2024-01-11T07:07:02.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 shutdown_connections 2024-01-11T07:07:02.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd848066f30 0x7fd8480693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85c15aa80 0x7fd85c14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd85c0cafb0 0x7fd85c067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 --2- 172.21.15.7:0/3470443633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c158280 0x7fd85c0691d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:02.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 >> 172.21.15.7:0/3470443633 conn(0x7fd85c071100 msgr2=0x7fd85c0ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:02.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 shutdown_connections 2024-01-11T07:07:02.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:02.959+0000 7fd861b1e700 1 -- 172.21.15.7:0/3470443633 wait complete. 2024-01-11T07:07:02.972 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:07:02.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.105+0000 7f68170cd700 1 Processor -- start 2024-01-11T07:07:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.105+0000 7f68170cd700 1 -- start start 2024-01-11T07:07:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f68170cd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68100cafb0 0x7f68100ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f68170cd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6810158280 0x7f68100b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f68170cd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68100b0970 0x7f6810159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f68170cd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100b0df0 con 0x7f68100cafb0 2024-01-11T07:07:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f68170cd700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6810159300 con 0x7f68100b0970 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f68170cd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68100cf260 con 0x7f6810158280 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68100cafb0 0x7f68100ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68100cafb0 0x7f68100ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59832/0 (socket says 172.21.15.7:59832) 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 -- 172.21.15.7:0/3819101889 learned_addr learned my addr 172.21.15.7:0/3819101889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f681566a700 1 --2- 172.21.15.7:0/3819101889 >> v2:172.21.15.113:3300/0 conn(0x7f68100b0970 0x7f6810159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f680ffff700 1 --2- 172.21.15.7:0/3819101889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6810158280 0x7f68100b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 -- 172.21.15.7:0/3819101889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6810158280 msgr2=0x7f68100b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 --2- 172.21.15.7:0/3819101889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6810158280 0x7f68100b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 -- 172.21.15.7:0/3819101889 >> v2:172.21.15.113:3300/0 conn(0x7f68100b0970 msgr2=0x7f6810159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 --2- 172.21.15.7:0/3819101889 >> v2:172.21.15.113:3300/0 conn(0x7f68100b0970 0x7f6810159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 -- 172.21.15.7:0/3819101889 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100cf540 con 0x7f68100cafb0 2024-01-11T07:07:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f681566a700 1 --2- 172.21.15.7:0/3819101889 >> v2:172.21.15.113:3300/0 conn(0x7f68100b0970 0x7f6810159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.106+0000 7f6814e69700 1 --2- 172.21.15.7:0/3819101889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100ce8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6810063d90 tx=0x7f6804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ed700c42fc32fd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f680f7fe700 1 -- 172.21.15.7:0/3819101889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400b860 con 0x7f68100cafb0 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f680f7fe700 1 -- 172.21.15.7:0/3819101889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680400b9c0 con 0x7f68100cafb0 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f680f7fe700 1 -- 172.21.15.7:0/3819101889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400bd60 con 0x7f68100cafb0 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 -- 172.21.15.7:0/3819101889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 msgr2=0x7f68100ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 --2- 172.21.15.7:0/3819101889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100ce8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6810063d90 tx=0x7f6804009250 comp rx=0 tx=0).stop 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 -- 172.21.15.7:0/3819101889 shutdown_connections 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 --2- 172.21.15.7:0/3819101889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100ce8f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 --2- 172.21.15.7:0/3819101889 >> v2:172.21.15.113:3300/0 conn(0x7f68100b0970 0x7f6810159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 --2- 172.21.15.7:0/3819101889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6810158280 0x7f68100b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 -- 172.21.15.7:0/3819101889 >> 172.21.15.7:0/3819101889 conn(0x7f6810071100 msgr2=0x7f681006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 -- 172.21.15.7:0/3819101889 shutdown_connections 2024-01-11T07:07:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.107+0000 7f68170cd700 1 -- 172.21.15.7:0/3819101889 wait complete. 2024-01-11T07:07:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.109+0000 7f68170cd700 1 Processor -- start 2024-01-11T07:07:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.109+0000 7f68170cd700 1 -- start start 2024-01-11T07:07:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.109+0000 7f68170cd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68100b0970 0x7f68100c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.109+0000 7f68170cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f68170cd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6810158280 0x7f68101491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f68170cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100b2be0 con 0x7f68100cafb0 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f68170cd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100c8690 con 0x7f6810158280 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f68170cd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68100c8970 con 0x7f68100b0970 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59840/0 (socket says 172.21.15.7:59840) 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 -- 172.21.15.7:0/2242547972 learned_addr learned my addr 172.21.15.7:0/2242547972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f6814e69700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68100b0970 0x7f68100c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f681566a700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6810158280 0x7f68101491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 -- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68100b0970 msgr2=0x7f68100c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68100b0970 0x7f68100c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 -- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6810158280 msgr2=0x7f68101491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6810158280 0x7f68101491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 -- 172.21.15.7:0/2242547972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68040094f0 con 0x7f68100cafb0 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f681566a700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6810158280 0x7f68101491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680ffff700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100c7c10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006b00 tx=0x7f67fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680cff9700 1 -- 172.21.15.7:0/2242547972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0125f0 con 0x7f68100cafb0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680cff9700 1 -- 172.21.15.7:0/2242547972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc012770 con 0x7f68100cafb0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68100c8c50 con 0x7f68100cafb0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f680cff9700 1 -- 172.21.15.7:0/2242547972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc012b30 con 0x7f68100cafb0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.110+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6810068920 con 0x7f68100cafb0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.112+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68100b2d90 con 0x7f68100cafb0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.112+0000 7f680cff9700 1 -- 172.21.15.7:0/2242547972 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f67fc01e460 con 0x7f68100cafb0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.112+0000 7f680cff9700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67f4066f30 0x7f67f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.112+0000 7f680cff9700 1 -- 172.21.15.7:0/2242547972 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f67fc084b20 con 0x7f68100cafb0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.112+0000 7f6814e69700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67f4066f30 0x7f67f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.113+0000 7f6814e69700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67f4066f30 0x7f67f40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6810063d90 tx=0x7f6804017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.117+0000 7f680cff9700 1 -- 172.21.15.7:0/2242547972 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68100b2d90 con 0x7f68100cafb0 2024-01-11T07:07:03.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.316+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f681004c3c0 con 0x7f67f4066f30 2024-01-11T07:07:03.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.324+0000 7f680cff9700 1 -- 172.21.15.7:0/2242547972 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257334 (secure 0 0 0) 0x7f681004c3c0 con 0x7f67f4066f30 2024-01-11T07:07:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:03.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67f4066f30 msgr2=0x7f67f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67f4066f30 0x7f67f40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6810063d90 tx=0x7f6804017000 comp rx=0 tx=0).stop 2024-01-11T07:07:03.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 msgr2=0x7f68100c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100c7c10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006b00 tx=0x7f67fc008040 comp rx=0 tx=0).stop 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 shutdown_connections 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67f4066f30 0x7f67f40693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6810158280 0x7f68101491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68100b0970 0x7f68100c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 --2- 172.21.15.7:0/2242547972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68100cafb0 0x7f68100c7c10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 >> 172.21.15.7:0/2242547972 conn(0x7f6810071100 msgr2=0x7f68100ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.327+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 shutdown_connections 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.328+0000 7f68170cd700 1 -- 172.21.15.7:0/2242547972 wait complete. 2024-01-11T07:07:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:03.340 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-01-11T07:07:01.839201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1659,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560840,"statfs":{"total":1600000000,"available":1598300877,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.890491"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958913+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954450+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.963329+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958900+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.962044+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.960864+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958424+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957752+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957902+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955355+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955857+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955774+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:36.474065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.961036+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959719+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955406+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959984+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959384+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.953618+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957477+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964441+0000","last_change":"2024-01-11T07:06:57.964441+0000","last_active":"2024-01-11T07:06:57.964441+0000","last_peered":"2024-01-11T07:06:57.964441+0000","last_clean":"2024-01-11T07:06:57.964441+0000","last_became_active":"2024-01-11T07:06:57.963971+0000","last_became_peered":"2024-01-11T07:06:57.963971+0000","last_unstale":"2024-01-11T07:06:57.964441+0000","last_undegraded":"2024-01-11T07:06:57.964441+0000","last_fullsized":"2024-01-11T07:06:57.964441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957295+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.958809+0000","last_change":"2024-01-11T07:06:57.958809+0000","last_active":"2024-01-11T07:06:57.958809+0000","last_peered":"2024-01-11T07:06:57.958809+0000","last_clean":"2024-01-11T07:06:57.958809+0000","last_became_active":"2024-01-11T07:06:57.958478+0000","last_became_peered":"2024-01-11T07:06:57.958478+0000","last_unstale":"2024-01-11T07:06:57.958809+0000","last_undegraded":"2024-01-11T07:06:57.958809+0000","last_fullsized":"2024-01-11T07:06:57.958809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.962782+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967660+0000","last_change":"2024-01-11T07:06:57.967660+0000","last_active":"2024-01-11T07:06:57.967660+0000","last_peered":"2024-01-11T07:06:57.967660+0000","last_clean":"2024-01-11T07:06:57.967660+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:57.967660+0000","last_undegraded":"2024-01-11T07:06:57.967660+0000","last_fullsized":"2024-01-11T07:06:57.967660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954178+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967196+0000","last_change":"2024-01-11T07:06:57.967196+0000","last_active":"2024-01-11T07:06:57.967196+0000","last_peered":"2024-01-11T07:06:57.967196+0000","last_clean":"2024-01-11T07:06:57.967196+0000","last_became_active":"2024-01-11T07:06:57.966964+0000","last_became_peered":"2024-01-11T07:06:57.966964+0000","last_unstale":"2024-01-11T07:06:57.967196+0000","last_undegraded":"2024-01-11T07:06:57.967196+0000","last_fullsized":"2024-01-11T07:06:57.967196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:27.038629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955140+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964730+0000","last_change":"2024-01-11T07:06:57.964730+0000","last_active":"2024-01-11T07:06:57.964730+0000","last_peered":"2024-01-11T07:06:57.964730+0000","last_clean":"2024-01-11T07:06:57.964730+0000","last_became_active":"2024-01-11T07:06:57.964464+0000","last_became_peered":"2024-01-11T07:06:57.964464+0000","last_unstale":"2024-01-11T07:06:57.964730+0000","last_undegraded":"2024-01-11T07:06:57.964730+0000","last_fullsized":"2024-01-11T07:06:57.964730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956253+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:20.911572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.961158+0000","last_change":"2024-01-11T07:06:57.961158+0000","last_active":"2024-01-11T07:06:57.961158+0000","last_peered":"2024-01-11T07:06:57.961158+0000","last_clean":"2024-01-11T07:06:57.961158+0000","last_became_active":"2024-01-11T07:06:57.960836+0000","last_became_peered":"2024-01-11T07:06:57.960836+0000","last_unstale":"2024-01-11T07:06:57.961158+0000","last_undegraded":"2024-01-11T07:06:57.961158+0000","last_fullsized":"2024-01-11T07:06:57.961158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958785+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963583+0000","last_change":"2024-01-11T07:06:57.963583+0000","last_active":"2024-01-11T07:06:57.963583+0000","last_peered":"2024-01-11T07:06:57.963583+0000","last_clean":"2024-01-11T07:06:57.963583+0000","last_became_active":"2024-01-11T07:06:57.962966+0000","last_became_peered":"2024-01-11T07:06:57.962966+0000","last_unstale":"2024-01-11T07:06:57.963583+0000","last_undegraded":"2024-01-11T07:06:57.963583+0000","last_fullsized":"2024-01-11T07:06:57.963583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956072+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965145+0000","last_change":"2024-01-11T07:06:57.965145+0000","last_active":"2024-01-11T07:06:57.965145+0000","last_peered":"2024-01-11T07:06:57.965145+0000","last_clean":"2024-01-11T07:06:57.965145+0000","last_became_active":"2024-01-11T07:06:57.964890+0000","last_became_peered":"2024-01-11T07:06:57.964890+0000","last_unstale":"2024-01-11T07:06:57.965145+0000","last_undegraded":"2024-01-11T07:06:57.965145+0000","last_fullsized":"2024-01-11T07:06:57.965145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955622+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967613+0000","last_change":"2024-01-11T07:06:57.967613+0000","last_active":"2024-01-11T07:06:57.967613+0000","last_peered":"2024-01-11T07:06:57.967613+0000","last_clean":"2024-01-11T07:06:57.967613+0000","last_became_active":"2024-01-11T07:06:57.967356+0000","last_became_peered":"2024-01-11T07:06:57.967356+0000","last_unstale":"2024-01-11T07:06:57.967613+0000","last_undegraded":"2024-01-11T07:06:57.967613+0000","last_fullsized":"2024-01-11T07:06:57.967613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956598+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966453+0000","last_change":"2024-01-11T07:06:57.966453+0000","last_active":"2024-01-11T07:06:57.966453+0000","last_peered":"2024-01-11T07:06:57.966453+0000","last_clean":"2024-01-11T07:06:57.966453+0000","last_became_active":"2024-01-11T07:06:57.966083+0000","last_became_peered":"2024-01-11T07:06:57.966083+0000","last_unstale":"2024-01-11T07:06:57.966453+0000","last_undegraded":"2024-01-11T07:06:57.966453+0000","last_fullsized":"2024-01-11T07:06:57.966453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958698+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966021+0000","last_change":"2024-01-11T07:06:57.966021+0000","last_active":"2024-01-11T07:06:57.966021+0000","last_peered":"2024-01-11T07:06:57.966021+0000","last_clean":"2024-01-11T07:06:57.966021+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:57.966021+0000","last_undegraded":"2024-01-11T07:06:57.966021+0000","last_fullsized":"2024-01-11T07:06:57.966021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958190+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.962108+0000","last_change":"2024-01-11T07:06:57.962108+0000","last_active":"2024-01-11T07:06:57.962108+0000","last_peered":"2024-01-11T07:06:57.962108+0000","last_clean":"2024-01-11T07:06:57.962108+0000","last_became_active":"2024-01-11T07:06:57.961848+0000","last_became_peered":"2024-01-11T07:06:57.961848+0000","last_unstale":"2024-01-11T07:06:57.962108+0000","last_undegraded":"2024-01-11T07:06:57.962108+0000","last_fullsized":"2024-01-11T07:06:57.962108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957563+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960489+0000","last_change":"2024-01-11T07:06:57.960489+0000","last_active":"2024-01-11T07:06:57.960489+0000","last_peered":"2024-01-11T07:06:57.960489+0000","last_clean":"2024-01-11T07:06:57.960489+0000","last_became_active":"2024-01-11T07:06:57.960225+0000","last_became_peered":"2024-01-11T07:06:57.960225+0000","last_unstale":"2024-01-11T07:06:57.960489+0000","last_undegraded":"2024-01-11T07:06:57.960489+0000","last_fullsized":"2024-01-11T07:06:57.960489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956384+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965935+0000","last_change":"2024-01-11T07:06:57.965935+0000","last_active":"2024-01-11T07:06:57.965935+0000","last_peered":"2024-01-11T07:06:57.965935+0000","last_clean":"2024-01-11T07:06:57.965935+0000","last_became_active":"2024-01-11T07:06:57.965303+0000","last_became_peered":"2024-01-11T07:06:57.965303+0000","last_unstale":"2024-01-11T07:06:57.965935+0000","last_undegraded":"2024-01-11T07:06:57.965935+0000","last_fullsized":"2024-01-11T07:06:57.965935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958869+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960488+0000","last_change":"2024-01-11T07:06:57.960488+0000","last_active":"2024-01-11T07:06:57.960488+0000","last_peered":"2024-01-11T07:06:57.960488+0000","last_clean":"2024-01-11T07:06:57.960488+0000","last_became_active":"2024-01-11T07:06:57.959963+0000","last_became_peered":"2024-01-11T07:06:57.959963+0000","last_unstale":"2024-01-11T07:06:57.960488+0000","last_undegraded":"2024-01-11T07:06:57.960488+0000","last_fullsized":"2024-01-11T07:06:57.960488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955978+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.961526+0000","last_change":"2024-01-11T07:06:57.961526+0000","last_active":"2024-01-11T07:06:57.961526+0000","last_peered":"2024-01-11T07:06:57.961526+0000","last_clean":"2024-01-11T07:06:57.961526+0000","last_became_active":"2024-01-11T07:06:57.961256+0000","last_became_peered":"2024-01-11T07:06:57.961256+0000","last_unstale":"2024-01-11T07:06:57.961526+0000","last_undegraded":"2024-01-11T07:06:57.961526+0000","last_fullsized":"2024-01-11T07:06:57.961526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956737+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967501+0000","last_change":"2024-01-11T07:06:57.967501+0000","last_active":"2024-01-11T07:06:57.967501+0000","last_peered":"2024-01-11T07:06:57.967501+0000","last_clean":"2024-01-11T07:06:57.967501+0000","last_became_active":"2024-01-11T07:06:57.966742+0000","last_became_peered":"2024-01-11T07:06:57.966742+0000","last_unstale":"2024-01-11T07:06:57.967501+0000","last_undegraded":"2024-01-11T07:06:57.967501+0000","last_fullsized":"2024-01-11T07:06:57.967501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:43:51.416880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958479+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960497+0000","last_change":"2024-01-11T07:06:57.960497+0000","last_active":"2024-01-11T07:06:57.960497+0000","last_peered":"2024-01-11T07:06:57.960497+0000","last_clean":"2024-01-11T07:06:57.960497+0000","last_became_active":"2024-01-11T07:06:57.959939+0000","last_became_peered":"2024-01-11T07:06:57.959939+0000","last_unstale":"2024-01-11T07:06:57.960497+0000","last_undegraded":"2024-01-11T07:06:57.960497+0000","last_fullsized":"2024-01-11T07:06:57.960497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958461+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966310+0000","last_change":"2024-01-11T07:06:57.966310+0000","last_active":"2024-01-11T07:06:57.966310+0000","last_peered":"2024-01-11T07:06:57.966310+0000","last_clean":"2024-01-11T07:06:57.966310+0000","last_became_active":"2024-01-11T07:06:57.965787+0000","last_became_peered":"2024-01-11T07:06:57.965787+0000","last_unstale":"2024-01-11T07:06:57.966310+0000","last_undegraded":"2024-01-11T07:06:57.966310+0000","last_fullsized":"2024-01-11T07:06:57.966310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955926+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:44.410296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954184+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963281+0000","last_change":"2024-01-11T07:06:57.963281+0000","last_active":"2024-01-11T07:06:57.963281+0000","last_peered":"2024-01-11T07:06:57.963281+0000","last_clean":"2024-01-11T07:06:57.963281+0000","last_became_active":"2024-01-11T07:06:57.962993+0000","last_became_peered":"2024-01-11T07:06:57.962993+0000","last_unstale":"2024-01-11T07:06:57.963281+0000","last_undegraded":"2024-01-11T07:06:57.963281+0000","last_fullsized":"2024-01-11T07:06:57.963281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957364+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965590+0000","last_change":"2024-01-11T07:06:57.965590+0000","last_active":"2024-01-11T07:06:57.965590+0000","last_peered":"2024-01-11T07:06:57.965590+0000","last_clean":"2024-01-11T07:06:57.965590+0000","last_became_active":"2024-01-11T07:06:57.964910+0000","last_became_peered":"2024-01-11T07:06:57.964910+0000","last_unstale":"2024-01-11T07:06:57.965590+0000","last_undegraded":"2024-01-11T07:06:57.965590+0000","last_fullsized":"2024-01-11T07:06:57.965590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-11T07:06:56.953593+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:06:56.953593+0000","last_peered":"2024-01-11T07:06:56.953593+0000","last_clean":"2024-01-11T07:06:56.953593+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:06:56.953593+0000","last_undegraded":"2024-01-11T07:06:56.953593+0000","last_fullsized":"2024-01-11T07:06:56.953593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956951+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964928+0000","last_change":"2024-01-11T07:06:57.964928+0000","last_active":"2024-01-11T07:06:57.964928+0000","last_peered":"2024-01-11T07:06:57.964928+0000","last_clean":"2024-01-11T07:06:57.964928+0000","last_became_active":"2024-01-11T07:06:57.964692+0000","last_became_peered":"2024-01-11T07:06:57.964692+0000","last_unstale":"2024-01-11T07:06:57.964928+0000","last_undegraded":"2024-01-11T07:06:57.964928+0000","last_fullsized":"2024-01-11T07:06:57.964928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967422+0000","last_change":"2024-01-11T07:06:57.967422+0000","last_active":"2024-01-11T07:06:57.967422+0000","last_peered":"2024-01-11T07:06:57.967422+0000","last_clean":"2024-01-11T07:06:57.967422+0000","last_became_active":"2024-01-11T07:06:57.967213+0000","last_became_peered":"2024-01-11T07:06:57.967213+0000","last_unstale":"2024-01-11T07:06:57.967422+0000","last_undegraded":"2024-01-11T07:06:57.967422+0000","last_fullsized":"2024-01-11T07:06:57.967422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:16.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954989+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.958705+0000","last_change":"2024-01-11T07:06:57.958705+0000","last_active":"2024-01-11T07:06:57.958705+0000","last_peered":"2024-01-11T07:06:57.958705+0000","last_clean":"2024-01-11T07:06:57.958705+0000","last_became_active":"2024-01-11T07:06:57.958421+0000","last_became_peered":"2024-01-11T07:06:57.958421+0000","last_unstale":"2024-01-11T07:06:57.958705+0000","last_undegraded":"2024-01-11T07:06:57.958705+0000","last_fullsized":"2024-01-11T07:06:57.958705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959243+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964975+0000","last_change":"2024-01-11T07:06:57.964975+0000","last_active":"2024-01-11T07:06:57.964975+0000","last_peered":"2024-01-11T07:06:57.964975+0000","last_clean":"2024-01-11T07:06:57.964975+0000","last_became_active":"2024-01-11T07:06:57.964709+0000","last_became_peered":"2024-01-11T07:06:57.964709+0000","last_unstale":"2024-01-11T07:06:57.964975+0000","last_undegraded":"2024-01-11T07:06:57.964975+0000","last_fullsized":"2024-01-11T07:06:57.964975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957392+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963735+0000","last_change":"2024-01-11T07:06:57.963735+0000","last_active":"2024-01-11T07:06:57.963735+0000","last_peered":"2024-01-11T07:06:57.963735+0000","last_clean":"2024-01-11T07:06:57.963735+0000","last_became_active":"2024-01-11T07:06:57.963471+0000","last_became_peered":"2024-01-11T07:06:57.963471+0000","last_unstale":"2024-01-11T07:06:57.963735+0000","last_undegraded":"2024-01-11T07:06:57.963735+0000","last_fullsized":"2024-01-11T07:06:57.963735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957366+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965092+0000","last_change":"2024-01-11T07:06:57.965092+0000","last_active":"2024-01-11T07:06:57.965092+0000","last_peered":"2024-01-11T07:06:57.965092+0000","last_clean":"2024-01-11T07:06:57.965092+0000","last_became_active":"2024-01-11T07:06:57.964872+0000","last_became_peered":"2024-01-11T07:06:57.964872+0000","last_unstale":"2024-01-11T07:06:57.965092+0000","last_undegraded":"2024-01-11T07:06:57.965092+0000","last_fullsized":"2024-01-11T07:06:57.965092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955602+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.962256+0000","last_change":"2024-01-11T07:06:57.962256+0000","last_active":"2024-01-11T07:06:57.962256+0000","last_peered":"2024-01-11T07:06:57.962256+0000","last_clean":"2024-01-11T07:06:57.962256+0000","last_became_active":"2024-01-11T07:06:57.961595+0000","last_became_peered":"2024-01-11T07:06:57.961595+0000","last_unstale":"2024-01-11T07:06:57.962256+0000","last_undegraded":"2024-01-11T07:06:57.962256+0000","last_fullsized":"2024-01-11T07:06:57.962256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955709+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963683+0000","last_change":"2024-01-11T07:06:57.963683+0000","last_active":"2024-01-11T07:06:57.963683+0000","last_peered":"2024-01-11T07:06:57.963683+0000","last_clean":"2024-01-11T07:06:57.963683+0000","last_became_active":"2024-01-11T07:06:57.963452+0000","last_became_peered":"2024-01-11T07:06:57.963452+0000","last_unstale":"2024-01-11T07:06:57.963683+0000","last_undegraded":"2024-01-11T07:06:57.963683+0000","last_fullsized":"2024-01-11T07:06:57.963683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955034+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966946+0000","last_change":"2024-01-11T07:06:57.966946+0000","last_active":"2024-01-11T07:06:57.966946+0000","last_peered":"2024-01-11T07:06:57.966946+0000","last_clean":"2024-01-11T07:06:57.966946+0000","last_became_active":"2024-01-11T07:06:57.966738+0000","last_became_peered":"2024-01-11T07:06:57.966738+0000","last_unstale":"2024-01-11T07:06:57.966946+0000","last_undegraded":"2024-01-11T07:06:57.966946+0000","last_fullsized":"2024-01-11T07:06:57.966946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957201+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960096+0000","last_change":"2024-01-11T07:06:57.960096+0000","last_active":"2024-01-11T07:06:57.960096+0000","last_peered":"2024-01-11T07:06:57.960096+0000","last_clean":"2024-01-11T07:06:57.960096+0000","last_became_active":"2024-01-11T07:06:57.959880+0000","last_became_peered":"2024-01-11T07:06:57.959880+0000","last_unstale":"2024-01-11T07:06:57.960096+0000","last_undegraded":"2024-01-11T07:06:57.960096+0000","last_fullsized":"2024-01-11T07:06:57.960096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954142+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963359+0000","last_change":"2024-01-11T07:06:57.963359+0000","last_active":"2024-01-11T07:06:57.963359+0000","last_peered":"2024-01-11T07:06:57.963359+0000","last_clean":"2024-01-11T07:06:57.963359+0000","last_became_active":"2024-01-11T07:06:57.963116+0000","last_became_peered":"2024-01-11T07:06:57.963116+0000","last_unstale":"2024-01-11T07:06:57.963359+0000","last_undegraded":"2024-01-11T07:06:57.963359+0000","last_fullsized":"2024-01-11T07:06:57.963359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957041+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956322+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.952744+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955821+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959313+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958284+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.953941+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954890+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958024+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956187+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.960581+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957687+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959186+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954871+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:03.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:03.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.515+0000 7f2950c18700 1 Processor -- start 2024-01-11T07:07:03.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.515+0000 7f2950c18700 1 -- start start 2024-01-11T07:07:03.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f2950c18700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c156760 0x7f294c06e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f2950c18700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c06e3f0 con 0x7f294c069c80 2024-01-11T07:07:03.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f2950c18700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c0b1f10 con 0x7f294c15b260 2024-01-11T07:07:03.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f2950c18700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0b1d20 con 0x7f294c156760 2024-01-11T07:07:03.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f294ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f294c069c80 0x7f294c156b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38300/0 (socket says 172.21.15.7:38300) 2024-01-11T07:07:03.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f294ad9d700 1 -- 172.21.15.7:0/3912571403 learned_addr learned my addr 172.21.15.7:0/3912571403 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:03.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f294a59c700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c156760 0x7f294c06e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668792532 0 0) 0x7f294c06e3f0 con 0x7f294c069c80 2024-01-11T07:07:03.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.516+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2934003570 con 0x7f294c069c80 2024-01-11T07:07:03.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762172303 0 0) 0x7f294c0b1f10 con 0x7f294c15b260 2024-01-11T07:07:03.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c06e3f0 con 0x7f294c15b260 2024-01-11T07:07:03.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2567696401 0 0) 0x7f2934003570 con 0x7f294c069c80 2024-01-11T07:07:03.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c156760 msgr2=0x7f294c06e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c156760 0x7f294c06e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 >> v1:172.21.15.113:6789/0 conn(0x7f294c15b260 legacy=0x7f294c0b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c06dda0 con 0x7f294c069c80 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f29440017f0 con 0x7f294c069c80 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 >> v1:172.21.15.7:6789/0 conn(0x7f294c069c80 legacy=0x7f294c156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c156760 0x7f2934003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f2934004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2934004d50 0x7f29340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293400bbf0 con 0x7f29340041d0 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293400bd30 con 0x7f2934004d50 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293400bef0 con 0x7f294c156760 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f2949d9b700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f2934004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294a59c700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c156760 0x7f2934003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f294ad9d700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2934004d50 0x7f29340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f2949d9b700 1 -- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c156760 msgr2=0x7f2934003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f2949d9b700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c156760 0x7f2934003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f2949d9b700 1 -- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2934004d50 msgr2=0x7f29340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f2949d9b700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2934004d50 0x7f29340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f2949d9b700 1 -- 172.21.15.7:0/3912571403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934003570 con 0x7f29340041d0 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.517+0000 7f2949d9b700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f2934004810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2940001a10 tx=0x7f2940007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7839b17b468143c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000c5d0 con 0x7f29340041d0 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294000c730 con 0x7f29340041d0 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f294959a700 1 -- 172.21.15.7:0/3912571403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000cad0 con 0x7f29340041d0 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f2950c18700 1 -- 172.21.15.7:0/3912571403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 msgr2=0x7f2934004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f2950c18700 1 --2- 172.21.15.7:0/3912571403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f2934004810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2940001a10 tx=0x7f2940007d30 comp rx=0 tx=0).stop 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f294ad9d700 1 -- 172.21.15.7:0/3912571403 reap_dead start 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f2950c18700 1 -- 172.21.15.7:0/3912571403 shutdown_connections 2024-01-11T07:07:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f2950c18700 1 -- 172.21.15.7:0/3912571403 >> 172.21.15.7:0/3912571403 conn(0x7f294c0cd910 msgr2=0x7f294c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:03.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f2950c18700 1 -- 172.21.15.7:0/3912571403 shutdown_connections 2024-01-11T07:07:03.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.518+0000 7f2950c18700 1 -- 172.21.15.7:0/3912571403 wait complete. 2024-01-11T07:07:03.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.519+0000 7f2950c18700 1 Processor -- start 2024-01-11T07:07:03.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.519+0000 7f2950c18700 1 -- start start 2024-01-11T07:07:03.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f2950c18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f294c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f2950c18700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c14d530 0x7f294c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f2950c18700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c157580 0x7f294c063230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f2950c18700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c15df50 con 0x7f29340041d0 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f2950c18700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c156b30 con 0x7f294c14d530 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f2950c18700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c1570f0 con 0x7f294c157580 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f294a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f294c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f294a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f294c0cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59858/0 (socket says 172.21.15.7:59858) 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f294a59c700 1 -- 172.21.15.7:0/1041965597 learned_addr learned my addr 172.21.15.7:0/1041965597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f294ad9d700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c157580 0x7f294c063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f2949d9b700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c14d530 0x7f294c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f294a59c700 1 -- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c157580 msgr2=0x7f294c063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f294a59c700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c157580 0x7f294c063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f294a59c700 1 -- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c14d530 msgr2=0x7f294c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.520+0000 7f294a59c700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c14d530 0x7f294c15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f294a59c700 1 -- 172.21.15.7:0/1041965597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c15e180 con 0x7f29340041d0 2024-01-11T07:07:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f294ad9d700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c157580 0x7f294c063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f2949d9b700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c14d530 0x7f294c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f294a59c700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f294c0cc7f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f293c011d70 tx=0x7f293c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f293a7fc700 1 -- 172.21.15.7:0/1041965597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c009e20 con 0x7f29340041d0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f293a7fc700 1 -- 172.21.15.7:0/1041965597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c00fc70 con 0x7f29340041d0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c15e460 con 0x7f29340041d0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f293a7fc700 1 -- 172.21.15.7:0/1041965597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c014440 con 0x7f29340041d0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.521+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c14d9a0 con 0x7f29340041d0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.522+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c04c3c0 con 0x7f29340041d0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.522+0000 7f293a7fc700 1 -- 172.21.15.7:0/1041965597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f293c014870 con 0x7f29340041d0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.523+0000 7f293a7fc700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f292c066f30 0x7f292c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.523+0000 7f293a7fc700 1 -- 172.21.15.7:0/1041965597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f293c018030 con 0x7f29340041d0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.523+0000 7f2949d9b700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f292c066f30 0x7f292c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.523+0000 7f2949d9b700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f292c066f30 0x7f292c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f294c063e60 tx=0x7f2940015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.527+0000 7f293a7fc700 1 -- 172.21.15.7:0/1041965597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f293c0521c0 con 0x7f29340041d0 2024-01-11T07:07:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.724+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f294c157230 con 0x7f292c066f30 2024-01-11T07:07:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.731+0000 7f293a7fc700 1 -- 172.21.15.7:0/1041965597 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257334 (secure 0 0 0) 0x7f294000ce40 con 0x7f292c066f30 2024-01-11T07:07:03.734 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f292c066f30 msgr2=0x7f292c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f292c066f30 0x7f292c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f294c063e60 tx=0x7f2940015000 comp rx=0 tx=0).stop 2024-01-11T07:07:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 msgr2=0x7f294c0cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f294c0cc7f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f293c011d70 tx=0x7f293c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 shutdown_connections 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f292c066f30 0x7f292c0693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c157580 0x7f294c063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29340041d0 0x7f294c0cc7f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 --2- 172.21.15.7:0/1041965597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c14d530 0x7f294c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.734+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 >> 172.21.15.7:0/1041965597 conn(0x7f294c0cd910 msgr2=0x7f294c1553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.735+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 shutdown_connections 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.735+0000 7f2950c18700 1 -- 172.21.15.7:0/1041965597 wait complete. 2024-01-11T07:07:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:03.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-01-11T07:07:01.839201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1659,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560840,"statfs":{"total":1600000000,"available":1598300877,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.890491"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958913+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954450+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.963329+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958900+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.962044+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.960864+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958424+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957752+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957902+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955355+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955857+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955774+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:36.474065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.961036+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959719+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955406+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959984+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959384+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.953618+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957477+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964441+0000","last_change":"2024-01-11T07:06:57.964441+0000","last_active":"2024-01-11T07:06:57.964441+0000","last_peered":"2024-01-11T07:06:57.964441+0000","last_clean":"2024-01-11T07:06:57.964441+0000","last_became_active":"2024-01-11T07:06:57.963971+0000","last_became_peered":"2024-01-11T07:06:57.963971+0000","last_unstale":"2024-01-11T07:06:57.964441+0000","last_undegraded":"2024-01-11T07:06:57.964441+0000","last_fullsized":"2024-01-11T07:06:57.964441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957295+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.958809+0000","last_change":"2024-01-11T07:06:57.958809+0000","last_active":"2024-01-11T07:06:57.958809+0000","last_peered":"2024-01-11T07:06:57.958809+0000","last_clean":"2024-01-11T07:06:57.958809+0000","last_became_active":"2024-01-11T07:06:57.958478+0000","last_became_peered":"2024-01-11T07:06:57.958478+0000","last_unstale":"2024-01-11T07:06:57.958809+0000","last_undegraded":"2024-01-11T07:06:57.958809+0000","last_fullsized":"2024-01-11T07:06:57.958809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.962782+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967660+0000","last_change":"2024-01-11T07:06:57.967660+0000","last_active":"2024-01-11T07:06:57.967660+0000","last_peered":"2024-01-11T07:06:57.967660+0000","last_clean":"2024-01-11T07:06:57.967660+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:57.967660+0000","last_undegraded":"2024-01-11T07:06:57.967660+0000","last_fullsized":"2024-01-11T07:06:57.967660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954178+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967196+0000","last_change":"2024-01-11T07:06:57.967196+0000","last_active":"2024-01-11T07:06:57.967196+0000","last_peered":"2024-01-11T07:06:57.967196+0000","last_clean":"2024-01-11T07:06:57.967196+0000","last_became_active":"2024-01-11T07:06:57.966964+0000","last_became_peered":"2024-01-11T07:06:57.966964+0000","last_unstale":"2024-01-11T07:06:57.967196+0000","last_undegraded":"2024-01-11T07:06:57.967196+0000","last_fullsized":"2024-01-11T07:06:57.967196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:27.038629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955140+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964730+0000","last_change":"2024-01-11T07:06:57.964730+0000","last_active":"2024-01-11T07:06:57.964730+0000","last_peered":"2024-01-11T07:06:57.964730+0000","last_clean":"2024-01-11T07:06:57.964730+0000","last_became_active":"2024-01-11T07:06:57.964464+0000","last_became_peered":"2024-01-11T07:06:57.964464+0000","last_unstale":"2024-01-11T07:06:57.964730+0000","last_undegraded":"2024-01-11T07:06:57.964730+0000","last_fullsized":"2024-01-11T07:06:57.964730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956253+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:20.911572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.961158+0000","last_change":"2024-01-11T07:06:57.961158+0000","last_active":"2024-01-11T07:06:57.961158+0000","last_peered":"2024-01-11T07:06:57.961158+0000","last_clean":"2024-01-11T07:06:57.961158+0000","last_became_active":"2024-01-11T07:06:57.960836+0000","last_became_peered":"2024-01-11T07:06:57.960836+0000","last_unstale":"2024-01-11T07:06:57.961158+0000","last_undegraded":"2024-01-11T07:06:57.961158+0000","last_fullsized":"2024-01-11T07:06:57.961158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958785+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963583+0000","last_change":"2024-01-11T07:06:57.963583+0000","last_active":"2024-01-11T07:06:57.963583+0000","last_peered":"2024-01-11T07:06:57.963583+0000","last_clean":"2024-01-11T07:06:57.963583+0000","last_became_active":"2024-01-11T07:06:57.962966+0000","last_became_peered":"2024-01-11T07:06:57.962966+0000","last_unstale":"2024-01-11T07:06:57.963583+0000","last_undegraded":"2024-01-11T07:06:57.963583+0000","last_fullsized":"2024-01-11T07:06:57.963583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956072+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965145+0000","last_change":"2024-01-11T07:06:57.965145+0000","last_active":"2024-01-11T07:06:57.965145+0000","last_peered":"2024-01-11T07:06:57.965145+0000","last_clean":"2024-01-11T07:06:57.965145+0000","last_became_active":"2024-01-11T07:06:57.964890+0000","last_became_peered":"2024-01-11T07:06:57.964890+0000","last_unstale":"2024-01-11T07:06:57.965145+0000","last_undegraded":"2024-01-11T07:06:57.965145+0000","last_fullsized":"2024-01-11T07:06:57.965145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955622+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967613+0000","last_change":"2024-01-11T07:06:57.967613+0000","last_active":"2024-01-11T07:06:57.967613+0000","last_peered":"2024-01-11T07:06:57.967613+0000","last_clean":"2024-01-11T07:06:57.967613+0000","last_became_active":"2024-01-11T07:06:57.967356+0000","last_became_peered":"2024-01-11T07:06:57.967356+0000","last_unstale":"2024-01-11T07:06:57.967613+0000","last_undegraded":"2024-01-11T07:06:57.967613+0000","last_fullsized":"2024-01-11T07:06:57.967613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956598+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966453+0000","last_change":"2024-01-11T07:06:57.966453+0000","last_active":"2024-01-11T07:06:57.966453+0000","last_peered":"2024-01-11T07:06:57.966453+0000","last_clean":"2024-01-11T07:06:57.966453+0000","last_became_active":"2024-01-11T07:06:57.966083+0000","last_became_peered":"2024-01-11T07:06:57.966083+0000","last_unstale":"2024-01-11T07:06:57.966453+0000","last_undegraded":"2024-01-11T07:06:57.966453+0000","last_fullsized":"2024-01-11T07:06:57.966453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958698+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966021+0000","last_change":"2024-01-11T07:06:57.966021+0000","last_active":"2024-01-11T07:06:57.966021+0000","last_peered":"2024-01-11T07:06:57.966021+0000","last_clean":"2024-01-11T07:06:57.966021+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:57.966021+0000","last_undegraded":"2024-01-11T07:06:57.966021+0000","last_fullsized":"2024-01-11T07:06:57.966021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958190+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.962108+0000","last_change":"2024-01-11T07:06:57.962108+0000","last_active":"2024-01-11T07:06:57.962108+0000","last_peered":"2024-01-11T07:06:57.962108+0000","last_clean":"2024-01-11T07:06:57.962108+0000","last_became_active":"2024-01-11T07:06:57.961848+0000","last_became_peered":"2024-01-11T07:06:57.961848+0000","last_unstale":"2024-01-11T07:06:57.962108+0000","last_undegraded":"2024-01-11T07:06:57.962108+0000","last_fullsized":"2024-01-11T07:06:57.962108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957563+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960489+0000","last_change":"2024-01-11T07:06:57.960489+0000","last_active":"2024-01-11T07:06:57.960489+0000","last_peered":"2024-01-11T07:06:57.960489+0000","last_clean":"2024-01-11T07:06:57.960489+0000","last_became_active":"2024-01-11T07:06:57.960225+0000","last_became_peered":"2024-01-11T07:06:57.960225+0000","last_unstale":"2024-01-11T07:06:57.960489+0000","last_undegraded":"2024-01-11T07:06:57.960489+0000","last_fullsized":"2024-01-11T07:06:57.960489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956384+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965935+0000","last_change":"2024-01-11T07:06:57.965935+0000","last_active":"2024-01-11T07:06:57.965935+0000","last_peered":"2024-01-11T07:06:57.965935+0000","last_clean":"2024-01-11T07:06:57.965935+0000","last_became_active":"2024-01-11T07:06:57.965303+0000","last_became_peered":"2024-01-11T07:06:57.965303+0000","last_unstale":"2024-01-11T07:06:57.965935+0000","last_undegraded":"2024-01-11T07:06:57.965935+0000","last_fullsized":"2024-01-11T07:06:57.965935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958869+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960488+0000","last_change":"2024-01-11T07:06:57.960488+0000","last_active":"2024-01-11T07:06:57.960488+0000","last_peered":"2024-01-11T07:06:57.960488+0000","last_clean":"2024-01-11T07:06:57.960488+0000","last_became_active":"2024-01-11T07:06:57.959963+0000","last_became_peered":"2024-01-11T07:06:57.959963+0000","last_unstale":"2024-01-11T07:06:57.960488+0000","last_undegraded":"2024-01-11T07:06:57.960488+0000","last_fullsized":"2024-01-11T07:06:57.960488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955978+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.961526+0000","last_change":"2024-01-11T07:06:57.961526+0000","last_active":"2024-01-11T07:06:57.961526+0000","last_peered":"2024-01-11T07:06:57.961526+0000","last_clean":"2024-01-11T07:06:57.961526+0000","last_became_active":"2024-01-11T07:06:57.961256+0000","last_became_peered":"2024-01-11T07:06:57.961256+0000","last_unstale":"2024-01-11T07:06:57.961526+0000","last_undegraded":"2024-01-11T07:06:57.961526+0000","last_fullsized":"2024-01-11T07:06:57.961526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956737+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967501+0000","last_change":"2024-01-11T07:06:57.967501+0000","last_active":"2024-01-11T07:06:57.967501+0000","last_peered":"2024-01-11T07:06:57.967501+0000","last_clean":"2024-01-11T07:06:57.967501+0000","last_became_active":"2024-01-11T07:06:57.966742+0000","last_became_peered":"2024-01-11T07:06:57.966742+0000","last_unstale":"2024-01-11T07:06:57.967501+0000","last_undegraded":"2024-01-11T07:06:57.967501+0000","last_fullsized":"2024-01-11T07:06:57.967501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:43:51.416880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958479+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960497+0000","last_change":"2024-01-11T07:06:57.960497+0000","last_active":"2024-01-11T07:06:57.960497+0000","last_peered":"2024-01-11T07:06:57.960497+0000","last_clean":"2024-01-11T07:06:57.960497+0000","last_became_active":"2024-01-11T07:06:57.959939+0000","last_became_peered":"2024-01-11T07:06:57.959939+0000","last_unstale":"2024-01-11T07:06:57.960497+0000","last_undegraded":"2024-01-11T07:06:57.960497+0000","last_fullsized":"2024-01-11T07:06:57.960497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958461+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966310+0000","last_change":"2024-01-11T07:06:57.966310+0000","last_active":"2024-01-11T07:06:57.966310+0000","last_peered":"2024-01-11T07:06:57.966310+0000","last_clean":"2024-01-11T07:06:57.966310+0000","last_became_active":"2024-01-11T07:06:57.965787+0000","last_became_peered":"2024-01-11T07:06:57.965787+0000","last_unstale":"2024-01-11T07:06:57.966310+0000","last_undegraded":"2024-01-11T07:06:57.966310+0000","last_fullsized":"2024-01-11T07:06:57.966310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955926+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:44.410296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954184+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963281+0000","last_change":"2024-01-11T07:06:57.963281+0000","last_active":"2024-01-11T07:06:57.963281+0000","last_peered":"2024-01-11T07:06:57.963281+0000","last_clean":"2024-01-11T07:06:57.963281+0000","last_became_active":"2024-01-11T07:06:57.962993+0000","last_became_peered":"2024-01-11T07:06:57.962993+0000","last_unstale":"2024-01-11T07:06:57.963281+0000","last_undegraded":"2024-01-11T07:06:57.963281+0000","last_fullsized":"2024-01-11T07:06:57.963281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957364+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965590+0000","last_change":"2024-01-11T07:06:57.965590+0000","last_active":"2024-01-11T07:06:57.965590+0000","last_peered":"2024-01-11T07:06:57.965590+0000","last_clean":"2024-01-11T07:06:57.965590+0000","last_became_active":"2024-01-11T07:06:57.964910+0000","last_became_peered":"2024-01-11T07:06:57.964910+0000","last_unstale":"2024-01-11T07:06:57.965590+0000","last_undegraded":"2024-01-11T07:06:57.965590+0000","last_fullsized":"2024-01-11T07:06:57.965590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-11T07:06:56.953593+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:06:56.953593+0000","last_peered":"2024-01-11T07:06:56.953593+0000","last_clean":"2024-01-11T07:06:56.953593+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:06:56.953593+0000","last_undegraded":"2024-01-11T07:06:56.953593+0000","last_fullsized":"2024-01-11T07:06:56.953593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956951+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964928+0000","last_change":"2024-01-11T07:06:57.964928+0000","last_active":"2024-01-11T07:06:57.964928+0000","last_peered":"2024-01-11T07:06:57.964928+0000","last_clean":"2024-01-11T07:06:57.964928+0000","last_became_active":"2024-01-11T07:06:57.964692+0000","last_became_peered":"2024-01-11T07:06:57.964692+0000","last_unstale":"2024-01-11T07:06:57.964928+0000","last_undegraded":"2024-01-11T07:06:57.964928+0000","last_fullsized":"2024-01-11T07:06:57.964928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967422+0000","last_change":"2024-01-11T07:06:57.967422+0000","last_active":"2024-01-11T07:06:57.967422+0000","last_peered":"2024-01-11T07:06:57.967422+0000","last_clean":"2024-01-11T07:06:57.967422+0000","last_became_active":"2024-01-11T07:06:57.967213+0000","last_became_peered":"2024-01-11T07:06:57.967213+0000","last_unstale":"2024-01-11T07:06:57.967422+0000","last_undegraded":"2024-01-11T07:06:57.967422+0000","last_fullsized":"2024-01-11T07:06:57.967422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:16.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954989+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.958705+0000","last_change":"2024-01-11T07:06:57.958705+0000","last_active":"2024-01-11T07:06:57.958705+0000","last_peered":"2024-01-11T07:06:57.958705+0000","last_clean":"2024-01-11T07:06:57.958705+0000","last_became_active":"2024-01-11T07:06:57.958421+0000","last_became_peered":"2024-01-11T07:06:57.958421+0000","last_unstale":"2024-01-11T07:06:57.958705+0000","last_undegraded":"2024-01-11T07:06:57.958705+0000","last_fullsized":"2024-01-11T07:06:57.958705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959243+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964975+0000","last_change":"2024-01-11T07:06:57.964975+0000","last_active":"2024-01-11T07:06:57.964975+0000","last_peered":"2024-01-11T07:06:57.964975+0000","last_clean":"2024-01-11T07:06:57.964975+0000","last_became_active":"2024-01-11T07:06:57.964709+0000","last_became_peered":"2024-01-11T07:06:57.964709+0000","last_unstale":"2024-01-11T07:06:57.964975+0000","last_undegraded":"2024-01-11T07:06:57.964975+0000","last_fullsized":"2024-01-11T07:06:57.964975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957392+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963735+0000","last_change":"2024-01-11T07:06:57.963735+0000","last_active":"2024-01-11T07:06:57.963735+0000","last_peered":"2024-01-11T07:06:57.963735+0000","last_clean":"2024-01-11T07:06:57.963735+0000","last_became_active":"2024-01-11T07:06:57.963471+0000","last_became_peered":"2024-01-11T07:06:57.963471+0000","last_unstale":"2024-01-11T07:06:57.963735+0000","last_undegraded":"2024-01-11T07:06:57.963735+0000","last_fullsized":"2024-01-11T07:06:57.963735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957366+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965092+0000","last_change":"2024-01-11T07:06:57.965092+0000","last_active":"2024-01-11T07:06:57.965092+0000","last_peered":"2024-01-11T07:06:57.965092+0000","last_clean":"2024-01-11T07:06:57.965092+0000","last_became_active":"2024-01-11T07:06:57.964872+0000","last_became_peered":"2024-01-11T07:06:57.964872+0000","last_unstale":"2024-01-11T07:06:57.965092+0000","last_undegraded":"2024-01-11T07:06:57.965092+0000","last_fullsized":"2024-01-11T07:06:57.965092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955602+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.962256+0000","last_change":"2024-01-11T07:06:57.962256+0000","last_active":"2024-01-11T07:06:57.962256+0000","last_peered":"2024-01-11T07:06:57.962256+0000","last_clean":"2024-01-11T07:06:57.962256+0000","last_became_active":"2024-01-11T07:06:57.961595+0000","last_became_peered":"2024-01-11T07:06:57.961595+0000","last_unstale":"2024-01-11T07:06:57.962256+0000","last_undegraded":"2024-01-11T07:06:57.962256+0000","last_fullsized":"2024-01-11T07:06:57.962256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955709+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963683+0000","last_change":"2024-01-11T07:06:57.963683+0000","last_active":"2024-01-11T07:06:57.963683+0000","last_peered":"2024-01-11T07:06:57.963683+0000","last_clean":"2024-01-11T07:06:57.963683+0000","last_became_active":"2024-01-11T07:06:57.963452+0000","last_became_peered":"2024-01-11T07:06:57.963452+0000","last_unstale":"2024-01-11T07:06:57.963683+0000","last_undegraded":"2024-01-11T07:06:57.963683+0000","last_fullsized":"2024-01-11T07:06:57.963683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955034+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966946+0000","last_change":"2024-01-11T07:06:57.966946+0000","last_active":"2024-01-11T07:06:57.966946+0000","last_peered":"2024-01-11T07:06:57.966946+0000","last_clean":"2024-01-11T07:06:57.966946+0000","last_became_active":"2024-01-11T07:06:57.966738+0000","last_became_peered":"2024-01-11T07:06:57.966738+0000","last_unstale":"2024-01-11T07:06:57.966946+0000","last_undegraded":"2024-01-11T07:06:57.966946+0000","last_fullsized":"2024-01-11T07:06:57.966946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957201+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960096+0000","last_change":"2024-01-11T07:06:57.960096+0000","last_active":"2024-01-11T07:06:57.960096+0000","last_peered":"2024-01-11T07:06:57.960096+0000","last_clean":"2024-01-11T07:06:57.960096+0000","last_became_active":"2024-01-11T07:06:57.959880+0000","last_became_peered":"2024-01-11T07:06:57.959880+0000","last_unstale":"2024-01-11T07:06:57.960096+0000","last_undegraded":"2024-01-11T07:06:57.960096+0000","last_fullsized":"2024-01-11T07:06:57.960096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954142+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963359+0000","last_change":"2024-01-11T07:06:57.963359+0000","last_active":"2024-01-11T07:06:57.963359+0000","last_peered":"2024-01-11T07:06:57.963359+0000","last_clean":"2024-01-11T07:06:57.963359+0000","last_became_active":"2024-01-11T07:06:57.963116+0000","last_became_peered":"2024-01-11T07:06:57.963116+0000","last_unstale":"2024-01-11T07:06:57.963359+0000","last_undegraded":"2024-01-11T07:06:57.963359+0000","last_fullsized":"2024-01-11T07:06:57.963359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957041+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956322+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.952744+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955821+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959313+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958284+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.953941+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954890+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958024+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956187+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.960581+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957687+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959186+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954871+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:03.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.921+0000 7efd70f7b700 1 Processor -- start 2024-01-11T07:07:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.921+0000 7efd70f7b700 1 -- start start 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.922+0000 7efd70f7b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd6c0cafb0 0x7efd6c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.922+0000 7efd70f7b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0b0b40 0x7efd6c161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.922+0000 7efd70f7b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6c159300 con 0x7efd6c158280 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd70f7b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd6c0cf260 con 0x7efd6c0cafb0 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd70f7b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c0cf540 con 0x7efd6c0b0b40 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd6c0cafb0 0x7efd6c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd69d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efd6c158280 0x7efd6c0b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38302/0 (socket says 172.21.15.7:38302) 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd69d9b700 1 -- 172.21.15.7:0/1255798310 learned_addr learned my addr 172.21.15.7:0/1255798310 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6ad9d700 1 --2- 172.21.15.7:0/1255798310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0b0b40 0x7efd6c161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6a59c700 1 -- 172.21.15.7:0/1255798310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0b0b40 msgr2=0x7efd6c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6a59c700 1 --2- 172.21.15.7:0/1255798310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0b0b40 0x7efd6c161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6a59c700 1 -- 172.21.15.7:0/1255798310 >> v1:172.21.15.7:6789/0 conn(0x7efd6c158280 legacy=0x7efd6c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6959a700 1 -- 172.21.15.7:0/1255798310 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017779120 0 0) 0x7efd6c159300 con 0x7efd6c158280 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6a59c700 1 -- 172.21.15.7:0/1255798310 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c0cf820 con 0x7efd6c0cafb0 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6a59c700 1 --2- 172.21.15.7:0/1255798310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0cafb0 0x7efd6c0ce8f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd540098e0 tx=0x7efd540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90296065584c2d42 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6959a700 1 -- 172.21.15.7:0/1255798310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400b640 con 0x7efd6c0cafb0 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6959a700 1 -- 172.21.15.7:0/1255798310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5400b7a0 con 0x7efd6c0cafb0 2024-01-11T07:07:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.923+0000 7efd6959a700 1 -- 172.21.15.7:0/1255798310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400bb40 con 0x7efd6c0cafb0 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.924+0000 7efd70f7b700 1 -- 172.21.15.7:0/1255798310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0cafb0 msgr2=0x7efd6c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.924+0000 7efd70f7b700 1 --2- 172.21.15.7:0/1255798310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0cafb0 0x7efd6c0ce8f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd540098e0 tx=0x7efd540091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.924+0000 7efd70f7b700 1 -- 172.21.15.7:0/1255798310 shutdown_connections 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.924+0000 7efd70f7b700 1 --2- 172.21.15.7:0/1255798310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0b0b40 0x7efd6c161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.924+0000 7efd70f7b700 1 --2- 172.21.15.7:0/1255798310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0cafb0 0x7efd6c0ce8f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.924+0000 7efd70f7b700 1 -- 172.21.15.7:0/1255798310 >> 172.21.15.7:0/1255798310 conn(0x7efd6c071100 msgr2=0x7efd6c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.924+0000 7efd70f7b700 1 -- 172.21.15.7:0/1255798310 shutdown_connections 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.924+0000 7efd70f7b700 1 -- 172.21.15.7:0/1255798310 wait complete. 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.925+0000 7efd70f7b700 1 Processor -- start 2024-01-11T07:07:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.925+0000 7efd70f7b700 1 -- start start 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd70f7b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0b0b40 0x7efd6c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd70f7b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0cafb0 0x7efd6c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd70f7b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c158280 0x7efd6c0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd70f7b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c15c8b0 con 0x7efd6c0b0b40 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd70f7b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c068f70 con 0x7efd6c158280 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd70f7b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c15bca0 con 0x7efd6c0cafb0 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd6a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0b0b40 0x7efd6c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd6a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0b0b40 0x7efd6c15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59866/0 (socket says 172.21.15.7:59866) 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.926+0000 7efd6a59c700 1 -- 172.21.15.7:0/227410552 learned_addr learned my addr 172.21.15.7:0/227410552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd69d9b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0cafb0 0x7efd6c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd6ad9d700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c158280 0x7efd6c0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd6a59c700 1 -- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0cafb0 msgr2=0x7efd6c068a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd6a59c700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0cafb0 0x7efd6c068a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd6a59c700 1 -- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c158280 msgr2=0x7efd6c0b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd6a59c700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c158280 0x7efd6c0b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd6a59c700 1 -- 172.21.15.7:0/227410552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54009420 con 0x7efd6c0b0b40 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd69d9b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0cafb0 0x7efd6c068a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd6a59c700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0b0b40 0x7efd6c15b6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd6c06a1d0 tx=0x7efd54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd627fc700 1 -- 172.21.15.7:0/227410552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400bb20 con 0x7efd6c0b0b40 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd627fc700 1 -- 172.21.15.7:0/227410552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd54011420 con 0x7efd6c0b0b40 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c15bf80 con 0x7efd6c0b0b40 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd627fc700 1 -- 172.21.15.7:0/227410552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd540117a0 con 0x7efd6c0b0b40 2024-01-11T07:07:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.927+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c15c540 con 0x7efd6c0b0b40 2024-01-11T07:07:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.928+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6c04c3c0 con 0x7efd6c0b0b40 2024-01-11T07:07:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.928+0000 7efd627fc700 1 -- 172.21.15.7:0/227410552 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efd54011b50 con 0x7efd6c0b0b40 2024-01-11T07:07:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.929+0000 7efd627fc700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efd4c066f30 0x7efd4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.929+0000 7efd627fc700 1 -- 172.21.15.7:0/227410552 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd54085c60 con 0x7efd6c0b0b40 2024-01-11T07:07:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.929+0000 7efd69d9b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efd4c066f30 0x7efd4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.930+0000 7efd69d9b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efd4c066f30 0x7efd4c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0cd560 tx=0x7efd5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:03.933+0000 7efd627fc700 1 -- 172.21.15.7:0/227410552 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd54052ed0 con 0x7efd6c0b0b40 2024-01-11T07:07:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.133+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd6c15c9f0 con 0x7efd4c066f30 2024-01-11T07:07:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.140+0000 7efd627fc700 1 -- 172.21.15.7:0/227410552 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257334 (secure 0 0 0) 0x7efd5c001c50 con 0x7efd4c066f30 2024-01-11T07:07:04.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efd4c066f30 msgr2=0x7efd4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efd4c066f30 0x7efd4c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0cd560 tx=0x7efd5c009040 comp rx=0 tx=0).stop 2024-01-11T07:07:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0b0b40 msgr2=0x7efd6c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0b0b40 0x7efd6c15b6c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd6c06a1d0 tx=0x7efd54017000 comp rx=0 tx=0).stop 2024-01-11T07:07:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 shutdown_connections 2024-01-11T07:07:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efd4c066f30 0x7efd4c0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd6c158280 0x7efd6c0b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd6c0b0b40 0x7efd6c15b6c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 --2- 172.21.15.7:0/227410552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efd6c0cafb0 0x7efd6c068a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 >> 172.21.15.7:0/227410552 conn(0x7efd6c071100 msgr2=0x7efd6c0653b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 shutdown_connections 2024-01-11T07:07:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.144+0000 7efd70f7b700 1 -- 172.21.15.7:0/227410552 wait complete. 2024-01-11T07:07:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:04.157 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-11T07:07:03.839766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1659,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560840,"statfs":{"total":1600000000,"available":1598300877,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001343"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958913+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954450+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.963329+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958900+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.962044+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.960864+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958424+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957752+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957902+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955355+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955857+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955774+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:36.474065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.961036+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959719+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955406+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959984+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959384+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.953618+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957477+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964441+0000","last_change":"2024-01-11T07:06:57.964441+0000","last_active":"2024-01-11T07:06:57.964441+0000","last_peered":"2024-01-11T07:06:57.964441+0000","last_clean":"2024-01-11T07:06:57.964441+0000","last_became_active":"2024-01-11T07:06:57.963971+0000","last_became_peered":"2024-01-11T07:06:57.963971+0000","last_unstale":"2024-01-11T07:06:57.964441+0000","last_undegraded":"2024-01-11T07:06:57.964441+0000","last_fullsized":"2024-01-11T07:06:57.964441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957295+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.958809+0000","last_change":"2024-01-11T07:06:57.958809+0000","last_active":"2024-01-11T07:06:57.958809+0000","last_peered":"2024-01-11T07:06:57.958809+0000","last_clean":"2024-01-11T07:06:57.958809+0000","last_became_active":"2024-01-11T07:06:57.958478+0000","last_became_peered":"2024-01-11T07:06:57.958478+0000","last_unstale":"2024-01-11T07:06:57.958809+0000","last_undegraded":"2024-01-11T07:06:57.958809+0000","last_fullsized":"2024-01-11T07:06:57.958809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.962782+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967660+0000","last_change":"2024-01-11T07:06:57.967660+0000","last_active":"2024-01-11T07:06:57.967660+0000","last_peered":"2024-01-11T07:06:57.967660+0000","last_clean":"2024-01-11T07:06:57.967660+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:57.967660+0000","last_undegraded":"2024-01-11T07:06:57.967660+0000","last_fullsized":"2024-01-11T07:06:57.967660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954178+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967196+0000","last_change":"2024-01-11T07:06:57.967196+0000","last_active":"2024-01-11T07:06:57.967196+0000","last_peered":"2024-01-11T07:06:57.967196+0000","last_clean":"2024-01-11T07:06:57.967196+0000","last_became_active":"2024-01-11T07:06:57.966964+0000","last_became_peered":"2024-01-11T07:06:57.966964+0000","last_unstale":"2024-01-11T07:06:57.967196+0000","last_undegraded":"2024-01-11T07:06:57.967196+0000","last_fullsized":"2024-01-11T07:06:57.967196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:27.038629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955140+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964730+0000","last_change":"2024-01-11T07:06:57.964730+0000","last_active":"2024-01-11T07:06:57.964730+0000","last_peered":"2024-01-11T07:06:57.964730+0000","last_clean":"2024-01-11T07:06:57.964730+0000","last_became_active":"2024-01-11T07:06:57.964464+0000","last_became_peered":"2024-01-11T07:06:57.964464+0000","last_unstale":"2024-01-11T07:06:57.964730+0000","last_undegraded":"2024-01-11T07:06:57.964730+0000","last_fullsized":"2024-01-11T07:06:57.964730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956253+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:20.911572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.961158+0000","last_change":"2024-01-11T07:06:57.961158+0000","last_active":"2024-01-11T07:06:57.961158+0000","last_peered":"2024-01-11T07:06:57.961158+0000","last_clean":"2024-01-11T07:06:57.961158+0000","last_became_active":"2024-01-11T07:06:57.960836+0000","last_became_peered":"2024-01-11T07:06:57.960836+0000","last_unstale":"2024-01-11T07:06:57.961158+0000","last_undegraded":"2024-01-11T07:06:57.961158+0000","last_fullsized":"2024-01-11T07:06:57.961158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958785+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963583+0000","last_change":"2024-01-11T07:06:57.963583+0000","last_active":"2024-01-11T07:06:57.963583+0000","last_peered":"2024-01-11T07:06:57.963583+0000","last_clean":"2024-01-11T07:06:57.963583+0000","last_became_active":"2024-01-11T07:06:57.962966+0000","last_became_peered":"2024-01-11T07:06:57.962966+0000","last_unstale":"2024-01-11T07:06:57.963583+0000","last_undegraded":"2024-01-11T07:06:57.963583+0000","last_fullsized":"2024-01-11T07:06:57.963583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956072+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965145+0000","last_change":"2024-01-11T07:06:57.965145+0000","last_active":"2024-01-11T07:06:57.965145+0000","last_peered":"2024-01-11T07:06:57.965145+0000","last_clean":"2024-01-11T07:06:57.965145+0000","last_became_active":"2024-01-11T07:06:57.964890+0000","last_became_peered":"2024-01-11T07:06:57.964890+0000","last_unstale":"2024-01-11T07:06:57.965145+0000","last_undegraded":"2024-01-11T07:06:57.965145+0000","last_fullsized":"2024-01-11T07:06:57.965145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955622+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967613+0000","last_change":"2024-01-11T07:06:57.967613+0000","last_active":"2024-01-11T07:06:57.967613+0000","last_peered":"2024-01-11T07:06:57.967613+0000","last_clean":"2024-01-11T07:06:57.967613+0000","last_became_active":"2024-01-11T07:06:57.967356+0000","last_became_peered":"2024-01-11T07:06:57.967356+0000","last_unstale":"2024-01-11T07:06:57.967613+0000","last_undegraded":"2024-01-11T07:06:57.967613+0000","last_fullsized":"2024-01-11T07:06:57.967613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956598+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966453+0000","last_change":"2024-01-11T07:06:57.966453+0000","last_active":"2024-01-11T07:06:57.966453+0000","last_peered":"2024-01-11T07:06:57.966453+0000","last_clean":"2024-01-11T07:06:57.966453+0000","last_became_active":"2024-01-11T07:06:57.966083+0000","last_became_peered":"2024-01-11T07:06:57.966083+0000","last_unstale":"2024-01-11T07:06:57.966453+0000","last_undegraded":"2024-01-11T07:06:57.966453+0000","last_fullsized":"2024-01-11T07:06:57.966453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958698+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966021+0000","last_change":"2024-01-11T07:06:57.966021+0000","last_active":"2024-01-11T07:06:57.966021+0000","last_peered":"2024-01-11T07:06:57.966021+0000","last_clean":"2024-01-11T07:06:57.966021+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:57.966021+0000","last_undegraded":"2024-01-11T07:06:57.966021+0000","last_fullsized":"2024-01-11T07:06:57.966021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958190+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.962108+0000","last_change":"2024-01-11T07:06:57.962108+0000","last_active":"2024-01-11T07:06:57.962108+0000","last_peered":"2024-01-11T07:06:57.962108+0000","last_clean":"2024-01-11T07:06:57.962108+0000","last_became_active":"2024-01-11T07:06:57.961848+0000","last_became_peered":"2024-01-11T07:06:57.961848+0000","last_unstale":"2024-01-11T07:06:57.962108+0000","last_undegraded":"2024-01-11T07:06:57.962108+0000","last_fullsized":"2024-01-11T07:06:57.962108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957563+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960489+0000","last_change":"2024-01-11T07:06:57.960489+0000","last_active":"2024-01-11T07:06:57.960489+0000","last_peered":"2024-01-11T07:06:57.960489+0000","last_clean":"2024-01-11T07:06:57.960489+0000","last_became_active":"2024-01-11T07:06:57.960225+0000","last_became_peered":"2024-01-11T07:06:57.960225+0000","last_unstale":"2024-01-11T07:06:57.960489+0000","last_undegraded":"2024-01-11T07:06:57.960489+0000","last_fullsized":"2024-01-11T07:06:57.960489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956384+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965935+0000","last_change":"2024-01-11T07:06:57.965935+0000","last_active":"2024-01-11T07:06:57.965935+0000","last_peered":"2024-01-11T07:06:57.965935+0000","last_clean":"2024-01-11T07:06:57.965935+0000","last_became_active":"2024-01-11T07:06:57.965303+0000","last_became_peered":"2024-01-11T07:06:57.965303+0000","last_unstale":"2024-01-11T07:06:57.965935+0000","last_undegraded":"2024-01-11T07:06:57.965935+0000","last_fullsized":"2024-01-11T07:06:57.965935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958869+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960488+0000","last_change":"2024-01-11T07:06:57.960488+0000","last_active":"2024-01-11T07:06:57.960488+0000","last_peered":"2024-01-11T07:06:57.960488+0000","last_clean":"2024-01-11T07:06:57.960488+0000","last_became_active":"2024-01-11T07:06:57.959963+0000","last_became_peered":"2024-01-11T07:06:57.959963+0000","last_unstale":"2024-01-11T07:06:57.960488+0000","last_undegraded":"2024-01-11T07:06:57.960488+0000","last_fullsized":"2024-01-11T07:06:57.960488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955978+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.961526+0000","last_change":"2024-01-11T07:06:57.961526+0000","last_active":"2024-01-11T07:06:57.961526+0000","last_peered":"2024-01-11T07:06:57.961526+0000","last_clean":"2024-01-11T07:06:57.961526+0000","last_became_active":"2024-01-11T07:06:57.961256+0000","last_became_peered":"2024-01-11T07:06:57.961256+0000","last_unstale":"2024-01-11T07:06:57.961526+0000","last_undegraded":"2024-01-11T07:06:57.961526+0000","last_fullsized":"2024-01-11T07:06:57.961526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956737+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967501+0000","last_change":"2024-01-11T07:06:57.967501+0000","last_active":"2024-01-11T07:06:57.967501+0000","last_peered":"2024-01-11T07:06:57.967501+0000","last_clean":"2024-01-11T07:06:57.967501+0000","last_became_active":"2024-01-11T07:06:57.966742+0000","last_became_peered":"2024-01-11T07:06:57.966742+0000","last_unstale":"2024-01-11T07:06:57.967501+0000","last_undegraded":"2024-01-11T07:06:57.967501+0000","last_fullsized":"2024-01-11T07:06:57.967501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:43:51.416880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958479+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960497+0000","last_change":"2024-01-11T07:06:57.960497+0000","last_active":"2024-01-11T07:06:57.960497+0000","last_peered":"2024-01-11T07:06:57.960497+0000","last_clean":"2024-01-11T07:06:57.960497+0000","last_became_active":"2024-01-11T07:06:57.959939+0000","last_became_peered":"2024-01-11T07:06:57.959939+0000","last_unstale":"2024-01-11T07:06:57.960497+0000","last_undegraded":"2024-01-11T07:06:57.960497+0000","last_fullsized":"2024-01-11T07:06:57.960497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958461+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966310+0000","last_change":"2024-01-11T07:06:57.966310+0000","last_active":"2024-01-11T07:06:57.966310+0000","last_peered":"2024-01-11T07:06:57.966310+0000","last_clean":"2024-01-11T07:06:57.966310+0000","last_became_active":"2024-01-11T07:06:57.965787+0000","last_became_peered":"2024-01-11T07:06:57.965787+0000","last_unstale":"2024-01-11T07:06:57.966310+0000","last_undegraded":"2024-01-11T07:06:57.966310+0000","last_fullsized":"2024-01-11T07:06:57.966310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955926+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:44.410296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954184+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963281+0000","last_change":"2024-01-11T07:06:57.963281+0000","last_active":"2024-01-11T07:06:57.963281+0000","last_peered":"2024-01-11T07:06:57.963281+0000","last_clean":"2024-01-11T07:06:57.963281+0000","last_became_active":"2024-01-11T07:06:57.962993+0000","last_became_peered":"2024-01-11T07:06:57.962993+0000","last_unstale":"2024-01-11T07:06:57.963281+0000","last_undegraded":"2024-01-11T07:06:57.963281+0000","last_fullsized":"2024-01-11T07:06:57.963281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957364+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965590+0000","last_change":"2024-01-11T07:06:57.965590+0000","last_active":"2024-01-11T07:06:57.965590+0000","last_peered":"2024-01-11T07:06:57.965590+0000","last_clean":"2024-01-11T07:06:57.965590+0000","last_became_active":"2024-01-11T07:06:57.964910+0000","last_became_peered":"2024-01-11T07:06:57.964910+0000","last_unstale":"2024-01-11T07:06:57.965590+0000","last_undegraded":"2024-01-11T07:06:57.965590+0000","last_fullsized":"2024-01-11T07:06:57.965590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-11T07:06:56.953593+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:06:56.953593+0000","last_peered":"2024-01-11T07:06:56.953593+0000","last_clean":"2024-01-11T07:06:56.953593+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:06:56.953593+0000","last_undegraded":"2024-01-11T07:06:56.953593+0000","last_fullsized":"2024-01-11T07:06:56.953593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956951+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964928+0000","last_change":"2024-01-11T07:06:57.964928+0000","last_active":"2024-01-11T07:06:57.964928+0000","last_peered":"2024-01-11T07:06:57.964928+0000","last_clean":"2024-01-11T07:06:57.964928+0000","last_became_active":"2024-01-11T07:06:57.964692+0000","last_became_peered":"2024-01-11T07:06:57.964692+0000","last_unstale":"2024-01-11T07:06:57.964928+0000","last_undegraded":"2024-01-11T07:06:57.964928+0000","last_fullsized":"2024-01-11T07:06:57.964928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967422+0000","last_change":"2024-01-11T07:06:57.967422+0000","last_active":"2024-01-11T07:06:57.967422+0000","last_peered":"2024-01-11T07:06:57.967422+0000","last_clean":"2024-01-11T07:06:57.967422+0000","last_became_active":"2024-01-11T07:06:57.967213+0000","last_became_peered":"2024-01-11T07:06:57.967213+0000","last_unstale":"2024-01-11T07:06:57.967422+0000","last_undegraded":"2024-01-11T07:06:57.967422+0000","last_fullsized":"2024-01-11T07:06:57.967422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:16.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954989+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.958705+0000","last_change":"2024-01-11T07:06:57.958705+0000","last_active":"2024-01-11T07:06:57.958705+0000","last_peered":"2024-01-11T07:06:57.958705+0000","last_clean":"2024-01-11T07:06:57.958705+0000","last_became_active":"2024-01-11T07:06:57.958421+0000","last_became_peered":"2024-01-11T07:06:57.958421+0000","last_unstale":"2024-01-11T07:06:57.958705+0000","last_undegraded":"2024-01-11T07:06:57.958705+0000","last_fullsized":"2024-01-11T07:06:57.958705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959243+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964975+0000","last_change":"2024-01-11T07:06:57.964975+0000","last_active":"2024-01-11T07:06:57.964975+0000","last_peered":"2024-01-11T07:06:57.964975+0000","last_clean":"2024-01-11T07:06:57.964975+0000","last_became_active":"2024-01-11T07:06:57.964709+0000","last_became_peered":"2024-01-11T07:06:57.964709+0000","last_unstale":"2024-01-11T07:06:57.964975+0000","last_undegraded":"2024-01-11T07:06:57.964975+0000","last_fullsized":"2024-01-11T07:06:57.964975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957392+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963735+0000","last_change":"2024-01-11T07:06:57.963735+0000","last_active":"2024-01-11T07:06:57.963735+0000","last_peered":"2024-01-11T07:06:57.963735+0000","last_clean":"2024-01-11T07:06:57.963735+0000","last_became_active":"2024-01-11T07:06:57.963471+0000","last_became_peered":"2024-01-11T07:06:57.963471+0000","last_unstale":"2024-01-11T07:06:57.963735+0000","last_undegraded":"2024-01-11T07:06:57.963735+0000","last_fullsized":"2024-01-11T07:06:57.963735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957366+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965092+0000","last_change":"2024-01-11T07:06:57.965092+0000","last_active":"2024-01-11T07:06:57.965092+0000","last_peered":"2024-01-11T07:06:57.965092+0000","last_clean":"2024-01-11T07:06:57.965092+0000","last_became_active":"2024-01-11T07:06:57.964872+0000","last_became_peered":"2024-01-11T07:06:57.964872+0000","last_unstale":"2024-01-11T07:06:57.965092+0000","last_undegraded":"2024-01-11T07:06:57.965092+0000","last_fullsized":"2024-01-11T07:06:57.965092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955602+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.962256+0000","last_change":"2024-01-11T07:06:57.962256+0000","last_active":"2024-01-11T07:06:57.962256+0000","last_peered":"2024-01-11T07:06:57.962256+0000","last_clean":"2024-01-11T07:06:57.962256+0000","last_became_active":"2024-01-11T07:06:57.961595+0000","last_became_peered":"2024-01-11T07:06:57.961595+0000","last_unstale":"2024-01-11T07:06:57.962256+0000","last_undegraded":"2024-01-11T07:06:57.962256+0000","last_fullsized":"2024-01-11T07:06:57.962256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955709+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963683+0000","last_change":"2024-01-11T07:06:57.963683+0000","last_active":"2024-01-11T07:06:57.963683+0000","last_peered":"2024-01-11T07:06:57.963683+0000","last_clean":"2024-01-11T07:06:57.963683+0000","last_became_active":"2024-01-11T07:06:57.963452+0000","last_became_peered":"2024-01-11T07:06:57.963452+0000","last_unstale":"2024-01-11T07:06:57.963683+0000","last_undegraded":"2024-01-11T07:06:57.963683+0000","last_fullsized":"2024-01-11T07:06:57.963683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955034+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966946+0000","last_change":"2024-01-11T07:06:57.966946+0000","last_active":"2024-01-11T07:06:57.966946+0000","last_peered":"2024-01-11T07:06:57.966946+0000","last_clean":"2024-01-11T07:06:57.966946+0000","last_became_active":"2024-01-11T07:06:57.966738+0000","last_became_peered":"2024-01-11T07:06:57.966738+0000","last_unstale":"2024-01-11T07:06:57.966946+0000","last_undegraded":"2024-01-11T07:06:57.966946+0000","last_fullsized":"2024-01-11T07:06:57.966946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957201+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960096+0000","last_change":"2024-01-11T07:06:57.960096+0000","last_active":"2024-01-11T07:06:57.960096+0000","last_peered":"2024-01-11T07:06:57.960096+0000","last_clean":"2024-01-11T07:06:57.960096+0000","last_became_active":"2024-01-11T07:06:57.959880+0000","last_became_peered":"2024-01-11T07:06:57.959880+0000","last_unstale":"2024-01-11T07:06:57.960096+0000","last_undegraded":"2024-01-11T07:06:57.960096+0000","last_fullsized":"2024-01-11T07:06:57.960096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954142+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963359+0000","last_change":"2024-01-11T07:06:57.963359+0000","last_active":"2024-01-11T07:06:57.963359+0000","last_peered":"2024-01-11T07:06:57.963359+0000","last_clean":"2024-01-11T07:06:57.963359+0000","last_became_active":"2024-01-11T07:06:57.963116+0000","last_became_peered":"2024-01-11T07:06:57.963116+0000","last_unstale":"2024-01-11T07:06:57.963359+0000","last_undegraded":"2024-01-11T07:06:57.963359+0000","last_fullsized":"2024-01-11T07:06:57.963359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957041+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956322+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.952744+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955821+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959313+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958284+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.953941+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954890+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958024+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956187+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.960581+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957687+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959186+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954871+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958913+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.954450+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.963329+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958900+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-01-11T07:07:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.962044+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.960864+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958424+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957752+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957902+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955355+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-01-11T07:07:04.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955857+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955774+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:00:36.474065+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.961036+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.959719+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955406+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.959984+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.959384+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.953618+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957477+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957295+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.962782+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.954178+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955140+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.956253+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T14:14:20.911572+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958785+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.956072+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955622+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-11T07:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.956598+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958698+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958190+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957563+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.956384+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-01-11T07:07:04.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958869+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955978+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.956737+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958479+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958461+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-01-11T07:07:04.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955926+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T13:01:44.410296+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-11T07:07:04.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-01-11T07:07:04.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.954184+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-11T07:07:04.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-01-11T07:07:04.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957364+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.956951+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.954989+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.959243+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957392+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957366+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-01-11T07:07:04.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955602+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955709+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955034+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957201+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.954142+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957041+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-01-11T07:07:04.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.956322+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.952744+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.955821+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.959313+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958284+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-01-11T07:07:04.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.953941+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-11T07:07:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-01-11T07:07:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.954890+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-11T07:07:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-01-11T07:07:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.958024+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-01-11T07:07:04.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.956187+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-01-11T07:07:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.960581+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-11T07:07:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-01-11T07:07:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.957687+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-11T07:07:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-01-11T07:07:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.959186+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-11T07:07:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-01-11T07:07:04.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-01-11T07:06:57.948014+0000', 'last_change': '2024-01-11T07:06:57.954871+0000', 'last_active': '2024-01-11T07:06:57.948014+0000', 'last_peered': '2024-01-11T07:06:57.948014+0000', 'last_clean': '2024-01-11T07:06:57.948014+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-11T07:06:57.948014+0000', 'last_undegraded': '2024-01-11T07:06:57.948014+0000', 'last_fullsized': '2024-01-11T07:06:57.948014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'last_clean_scrub_stamp': '2024-01-11T07:06:57.948014+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-11T07:07:04.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-11T07:07:04.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.330+0000 7fa558270700 1 Processor -- start 2024-01-11T07:07:04.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.330+0000 7fa558270700 1 -- start start 2024-01-11T07:07:04.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa558270700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5500b0b40 0x7fa550162770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa558270700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa550159300 con 0x7fa550158280 2024-01-11T07:07:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa558270700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5500cf000 con 0x7fa5500cafb0 2024-01-11T07:07:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa558270700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5500cf260 con 0x7fa5500b0b40 2024-01-11T07:07:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa55680d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5500b0b40 0x7fa550162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa55580b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa550158280 0x7fa5500b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38318/0 (socket says 172.21.15.7:38318) 2024-01-11T07:07:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa55580b700 1 -- 172.21.15.7:0/2147411781 learned_addr learned my addr 172.21.15.7:0/2147411781 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa55680d700 1 -- 172.21.15.7:0/2147411781 >> v1:172.21.15.113:6789/0 conn(0x7fa5500cafb0 legacy=0x7fa5500ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa55680d700 1 -- 172.21.15.7:0/2147411781 >> v1:172.21.15.7:6789/0 conn(0x7fa550158280 legacy=0x7fa5500b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.331+0000 7fa55680d700 1 -- 172.21.15.7:0/2147411781 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500cf540 con 0x7fa5500b0b40 2024-01-11T07:07:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa55680d700 1 --2- 172.21.15.7:0/2147411781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500b0b40 0x7fa550162770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa548009e60 tx=0x7fa548009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8287dcdbc14c7b62 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa55500a700 1 -- 172.21.15.7:0/2147411781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54800e070 con 0x7fa5500b0b40 2024-01-11T07:07:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa55500a700 1 -- 172.21.15.7:0/2147411781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54800bb30 con 0x7fa5500b0b40 2024-01-11T07:07:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa55500a700 1 -- 172.21.15.7:0/2147411781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54800bed0 con 0x7fa5500b0b40 2024-01-11T07:07:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa558270700 1 -- 172.21.15.7:0/2147411781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500b0b40 msgr2=0x7fa550162770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa558270700 1 --2- 172.21.15.7:0/2147411781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500b0b40 0x7fa550162770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa548009e60 tx=0x7fa548009780 comp rx=0 tx=0).stop 2024-01-11T07:07:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa558270700 1 -- 172.21.15.7:0/2147411781 shutdown_connections 2024-01-11T07:07:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa558270700 1 --2- 172.21.15.7:0/2147411781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500b0b40 0x7fa550162770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa558270700 1 -- 172.21.15.7:0/2147411781 >> 172.21.15.7:0/2147411781 conn(0x7fa550071100 msgr2=0x7fa55006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa558270700 1 -- 172.21.15.7:0/2147411781 shutdown_connections 2024-01-11T07:07:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.332+0000 7fa558270700 1 -- 172.21.15.7:0/2147411781 wait complete. 2024-01-11T07:07:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.334+0000 7fa558270700 1 Processor -- start 2024-01-11T07:07:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.334+0000 7fa558270700 1 -- start start 2024-01-11T07:07:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa558270700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5500b0b40 0x7fa550168420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa558270700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500cafb0 0x7fa5500af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa558270700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa550158280 0x7fa5500a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa558270700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa550143da0 con 0x7fa5500cafb0 2024-01-11T07:07:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa558270700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa550168960 con 0x7fa550158280 2024-01-11T07:07:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa558270700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500ae2f0 con 0x7fa5500b0b40 2024-01-11T07:07:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500cafb0 0x7fa5500af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500cafb0 0x7fa5500af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59876/0 (socket says 172.21.15.7:59876) 2024-01-11T07:07:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 -- 172.21.15.7:0/621216618 learned_addr learned my addr 172.21.15.7:0/621216618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55680d700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa550158280 0x7fa5500a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55600c700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5500b0b40 0x7fa550168420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 -- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5500b0b40 msgr2=0x7fa550168420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5500b0b40 0x7fa550168420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 -- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa550158280 msgr2=0x7fa5500a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa550158280 0x7fa5500a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 -- 172.21.15.7:0/621216618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa548009a20 con 0x7fa5500cafb0 2024-01-11T07:07:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa55580b700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500cafb0 0x7fa5500af670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c810 tx=0x7fa540007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.335+0000 7fa5467fc700 1 -- 172.21.15.7:0/621216618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540005b30 con 0x7fa5500cafb0 2024-01-11T07:07:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.336+0000 7fa5467fc700 1 -- 172.21.15.7:0/621216618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa540005c90 con 0x7fa5500cafb0 2024-01-11T07:07:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.336+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5500ae5d0 con 0x7fa5500cafb0 2024-01-11T07:07:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.336+0000 7fa5467fc700 1 -- 172.21.15.7:0/621216618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540007320 con 0x7fa5500cafb0 2024-01-11T07:07:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.336+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5500aeb90 con 0x7fa5500cafb0 2024-01-11T07:07:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.337+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55004c3c0 con 0x7fa5500cafb0 2024-01-11T07:07:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.337+0000 7fa5467fc700 1 -- 172.21.15.7:0/621216618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa54001b400 con 0x7fa5500cafb0 2024-01-11T07:07:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.337+0000 7fa5467fc700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa534066f30 0x7fa5340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.337+0000 7fa5467fc700 1 -- 172.21.15.7:0/621216618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa540084000 con 0x7fa5500cafb0 2024-01-11T07:07:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.338+0000 7fa55600c700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa534066f30 0x7fa5340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.338+0000 7fa55600c700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa534066f30 0x7fa5340693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa550157c40 tx=0x7fa54c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:04.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.342+0000 7fa5467fc700 1 -- 172.21.15.7:0/621216618 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5400512f0 con 0x7fa5500cafb0 2024-01-11T07:07:04.488 INFO:tasks.ceph.mds.b.smithi113.stderr:2024-01-11T07:07:04.489+0000 7f5638400700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:07:04.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.629+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa5500cf820 con 0x7fa5500cafb0 2024-01-11T07:07:04.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.630+0000 7fa5467fc700 1 -- 172.21.15.7:0/621216618 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1027 (secure 0 0 0) 0x7fa540054910 con 0x7fa5500cafb0 2024-01-11T07:07:04.630 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:04.630 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":22,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1704956807,"num_in_osds":8,"osd_in_since":1704956804,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":1699123,"bytes_avail":1598300877,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4257}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-01-11T07:06:42.318890+0000","services":{}},"progress_events":{}} 2024-01-11T07:07:04.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa534066f30 msgr2=0x7fa5340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa534066f30 0x7fa5340693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa550157c40 tx=0x7fa54c009040 comp rx=0 tx=0).stop 2024-01-11T07:07:04.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500cafb0 msgr2=0x7fa5500af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500cafb0 0x7fa5500af670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c810 tx=0x7fa540007ec0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 shutdown_connections 2024-01-11T07:07:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa534066f30 0x7fa5340693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa550158280 0x7fa5500a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5500b0b40 0x7fa550168420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 --2- 172.21.15.7:0/621216618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5500cafb0 0x7fa5500af670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 >> 172.21.15.7:0/621216618 conn(0x7fa550071100 msgr2=0x7fa5501552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 shutdown_connections 2024-01-11T07:07:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.632+0000 7fa558270700 1 -- 172.21.15.7:0/621216618 wait complete. 2024-01-11T07:07:04.645 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-11T07:07:04.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:04.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.776+0000 7f7dc9ab0700 1 Processor -- start 2024-01-11T07:07:04.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.776+0000 7f7dc9ab0700 1 -- start start 2024-01-11T07:07:04.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc9ab0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7dc41579b0 0x7f7dc415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc9ab0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 0x7f7dc4069c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc9ab0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc40aeab0 con 0x7f7dc4157d80 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc9ab0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc40aed90 con 0x7f7dc41579b0 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc9ab0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40af070 con 0x7f7dc4063940 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7dc41579b0 0x7f7dc415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7dc4157d80 0x7f7dc4063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38322/0 (socket says 172.21.15.7:38322) 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7dc41579b0 0x7f7dc415bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59884/0 (socket says 172.21.15.7:59884) 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc2ffd700 1 -- 172.21.15.7:0/2295093601 learned_addr learned my addr 172.21.15.7:0/2295093601 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.777+0000 7f7dc3fff700 1 --2- 172.21.15.7:0/2295093601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 0x7f7dc4069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc37fe700 1 -- 172.21.15.7:0/2295093601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 msgr2=0x7f7dc4069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc37fe700 1 --2- 172.21.15.7:0/2295093601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 0x7f7dc4069c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc37fe700 1 -- 172.21.15.7:0/2295093601 >> v1:172.21.15.7:6789/0 conn(0x7f7dc4157d80 legacy=0x7f7dc4063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc37fe700 1 -- 172.21.15.7:0/2295093601 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc40af350 con 0x7f7dc41579b0 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc27fc700 1 -- 172.21.15.7:0/2295093601 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901330284 0 0) 0x7f7dc40aeab0 con 0x7f7dc4157d80 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc37fe700 1 --2- 172.21.15.7:0/2295093601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc415bf50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7db40098e0 tx=0x7f7db40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23abff9f5386e8d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc27fc700 1 -- 172.21.15.7:0/2295093601 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db400b640 con 0x7f7dc41579b0 2024-01-11T07:07:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc27fc700 1 -- 172.21.15.7:0/2295093601 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db400b7a0 con 0x7f7dc41579b0 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.778+0000 7f7dc27fc700 1 -- 172.21.15.7:0/2295093601 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db400bb40 con 0x7f7dc41579b0 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.779+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2295093601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 msgr2=0x7f7dc415bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.779+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2295093601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc415bf50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7db40098e0 tx=0x7f7db40091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.779+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2295093601 shutdown_connections 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.779+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2295093601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 0x7f7dc4069c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.779+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2295093601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc415bf50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.779+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2295093601 >> 172.21.15.7:0/2295093601 conn(0x7f7dc40680e0 msgr2=0x7f7dc40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.779+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2295093601 shutdown_connections 2024-01-11T07:07:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.779+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2295093601 wait complete. 2024-01-11T07:07:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.780+0000 7f7dc9ab0700 1 Processor -- start 2024-01-11T07:07:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.780+0000 7f7dc9ab0700 1 -- start start 2024-01-11T07:07:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc9ab0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 0x7f7dc40abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc9ab0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc40c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc9ab0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7dc4157d80 0x7f7dc416cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc9ab0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40c78f0 con 0x7f7dc41579b0 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc9ab0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc41590d0 con 0x7f7dc4157d80 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc9ab0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc4158c40 con 0x7f7dc4063940 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc40c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc40c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59888/0 (socket says 172.21.15.7:59888) 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc2ffd700 1 -- 172.21.15.7:0/2733509442 learned_addr learned my addr 172.21.15.7:0/2733509442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc3fff700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7dc4157d80 0x7f7dc416cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc37fe700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 0x7f7dc40abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc2ffd700 1 -- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 msgr2=0x7f7dc40abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc2ffd700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 0x7f7dc40abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc2ffd700 1 -- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7dc4157d80 msgr2=0x7f7dc416cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc2ffd700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7dc4157d80 0x7f7dc416cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.781+0000 7f7dc2ffd700 1 -- 172.21.15.7:0/2733509442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db4009420 con 0x7f7dc41579b0 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.782+0000 7f7dc2ffd700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc40c6ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c5e0 tx=0x7f7db8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.782+0000 7f7dab7fe700 1 -- 172.21.15.7:0/2733509442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8005b30 con 0x7f7dc41579b0 2024-01-11T07:07:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.782+0000 7f7dab7fe700 1 -- 172.21.15.7:0/2733509442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db8005c90 con 0x7f7dc41579b0 2024-01-11T07:07:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.782+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc4158310 con 0x7f7dc41579b0 2024-01-11T07:07:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.782+0000 7f7dab7fe700 1 -- 172.21.15.7:0/2733509442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8007300 con 0x7f7dc41579b0 2024-01-11T07:07:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.782+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc4158950 con 0x7f7dc41579b0 2024-01-11T07:07:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.783+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc4158df0 con 0x7f7dc41579b0 2024-01-11T07:07:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.783+0000 7f7dab7fe700 1 -- 172.21.15.7:0/2733509442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7db801b3b0 con 0x7f7dc41579b0 2024-01-11T07:07:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.784+0000 7f7dab7fe700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7db0066f30 0x7f7db00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.784+0000 7f7dab7fe700 1 -- 172.21.15.7:0/2733509442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7db8083930 con 0x7f7dc41579b0 2024-01-11T07:07:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.784+0000 7f7dc37fe700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7db0066f30 0x7f7db00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.784+0000 7f7dc37fe700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7db0066f30 0x7f7db00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7db40091e0 tx=0x7f7db4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.788+0000 7f7dab7fe700 1 -- 172.21.15.7:0/2733509442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7dc4158df0 con 0x7f7dc41579b0 2024-01-11T07:07:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.988+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7dc4158db0 con 0x7f7db0066f30 2024-01-11T07:07:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.996+0000 7f7dab7fe700 1 -- 172.21.15.7:0/2733509442 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257334 (secure 0 0 0) 0x7f7dc4158db0 con 0x7f7db0066f30 2024-01-11T07:07:04.999 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7db0066f30 msgr2=0x7f7db00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7db0066f30 0x7f7db00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7db40091e0 tx=0x7f7db4017000 comp rx=0 tx=0).stop 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 msgr2=0x7f7dc40c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc40c6ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c5e0 tx=0x7f7db8001070 comp rx=0 tx=0).stop 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 shutdown_connections 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7db0066f30 0x7f7db00693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7dc4157d80 0x7f7dc416cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7dc4063940 0x7f7dc40abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 --2- 172.21.15.7:0/2733509442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dc41579b0 0x7f7dc40c6ef0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 >> 172.21.15.7:0/2733509442 conn(0x7f7dc40680e0 msgr2=0x7f7dc406a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 shutdown_connections 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:04.999+0000 7f7dc9ab0700 1 -- 172.21.15.7:0/2733509442 wait complete. 2024-01-11T07:07:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:05.012 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-11T07:07:03.839766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1659,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560840,"statfs":{"total":1600000000,"available":1598300877,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001343"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958913+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954450+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.963329+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958900+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.962044+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.960864+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958424+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957752+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957902+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955355+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955857+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955774+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:36.474065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.961036+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959719+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955406+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959984+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959384+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.953618+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957477+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964441+0000","last_change":"2024-01-11T07:06:57.964441+0000","last_active":"2024-01-11T07:06:57.964441+0000","last_peered":"2024-01-11T07:06:57.964441+0000","last_clean":"2024-01-11T07:06:57.964441+0000","last_became_active":"2024-01-11T07:06:57.963971+0000","last_became_peered":"2024-01-11T07:06:57.963971+0000","last_unstale":"2024-01-11T07:06:57.964441+0000","last_undegraded":"2024-01-11T07:06:57.964441+0000","last_fullsized":"2024-01-11T07:06:57.964441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957295+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.958809+0000","last_change":"2024-01-11T07:06:57.958809+0000","last_active":"2024-01-11T07:06:57.958809+0000","last_peered":"2024-01-11T07:06:57.958809+0000","last_clean":"2024-01-11T07:06:57.958809+0000","last_became_active":"2024-01-11T07:06:57.958478+0000","last_became_peered":"2024-01-11T07:06:57.958478+0000","last_unstale":"2024-01-11T07:06:57.958809+0000","last_undegraded":"2024-01-11T07:06:57.958809+0000","last_fullsized":"2024-01-11T07:06:57.958809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.962782+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967660+0000","last_change":"2024-01-11T07:06:57.967660+0000","last_active":"2024-01-11T07:06:57.967660+0000","last_peered":"2024-01-11T07:06:57.967660+0000","last_clean":"2024-01-11T07:06:57.967660+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:57.967660+0000","last_undegraded":"2024-01-11T07:06:57.967660+0000","last_fullsized":"2024-01-11T07:06:57.967660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954178+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967196+0000","last_change":"2024-01-11T07:06:57.967196+0000","last_active":"2024-01-11T07:06:57.967196+0000","last_peered":"2024-01-11T07:06:57.967196+0000","last_clean":"2024-01-11T07:06:57.967196+0000","last_became_active":"2024-01-11T07:06:57.966964+0000","last_became_peered":"2024-01-11T07:06:57.966964+0000","last_unstale":"2024-01-11T07:06:57.967196+0000","last_undegraded":"2024-01-11T07:06:57.967196+0000","last_fullsized":"2024-01-11T07:06:57.967196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:27.038629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955140+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964730+0000","last_change":"2024-01-11T07:06:57.964730+0000","last_active":"2024-01-11T07:06:57.964730+0000","last_peered":"2024-01-11T07:06:57.964730+0000","last_clean":"2024-01-11T07:06:57.964730+0000","last_became_active":"2024-01-11T07:06:57.964464+0000","last_became_peered":"2024-01-11T07:06:57.964464+0000","last_unstale":"2024-01-11T07:06:57.964730+0000","last_undegraded":"2024-01-11T07:06:57.964730+0000","last_fullsized":"2024-01-11T07:06:57.964730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956253+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:20.911572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.961158+0000","last_change":"2024-01-11T07:06:57.961158+0000","last_active":"2024-01-11T07:06:57.961158+0000","last_peered":"2024-01-11T07:06:57.961158+0000","last_clean":"2024-01-11T07:06:57.961158+0000","last_became_active":"2024-01-11T07:06:57.960836+0000","last_became_peered":"2024-01-11T07:06:57.960836+0000","last_unstale":"2024-01-11T07:06:57.961158+0000","last_undegraded":"2024-01-11T07:06:57.961158+0000","last_fullsized":"2024-01-11T07:06:57.961158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958785+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963583+0000","last_change":"2024-01-11T07:06:57.963583+0000","last_active":"2024-01-11T07:06:57.963583+0000","last_peered":"2024-01-11T07:06:57.963583+0000","last_clean":"2024-01-11T07:06:57.963583+0000","last_became_active":"2024-01-11T07:06:57.962966+0000","last_became_peered":"2024-01-11T07:06:57.962966+0000","last_unstale":"2024-01-11T07:06:57.963583+0000","last_undegraded":"2024-01-11T07:06:57.963583+0000","last_fullsized":"2024-01-11T07:06:57.963583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956072+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965145+0000","last_change":"2024-01-11T07:06:57.965145+0000","last_active":"2024-01-11T07:06:57.965145+0000","last_peered":"2024-01-11T07:06:57.965145+0000","last_clean":"2024-01-11T07:06:57.965145+0000","last_became_active":"2024-01-11T07:06:57.964890+0000","last_became_peered":"2024-01-11T07:06:57.964890+0000","last_unstale":"2024-01-11T07:06:57.965145+0000","last_undegraded":"2024-01-11T07:06:57.965145+0000","last_fullsized":"2024-01-11T07:06:57.965145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955622+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967613+0000","last_change":"2024-01-11T07:06:57.967613+0000","last_active":"2024-01-11T07:06:57.967613+0000","last_peered":"2024-01-11T07:06:57.967613+0000","last_clean":"2024-01-11T07:06:57.967613+0000","last_became_active":"2024-01-11T07:06:57.967356+0000","last_became_peered":"2024-01-11T07:06:57.967356+0000","last_unstale":"2024-01-11T07:06:57.967613+0000","last_undegraded":"2024-01-11T07:06:57.967613+0000","last_fullsized":"2024-01-11T07:06:57.967613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956598+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966453+0000","last_change":"2024-01-11T07:06:57.966453+0000","last_active":"2024-01-11T07:06:57.966453+0000","last_peered":"2024-01-11T07:06:57.966453+0000","last_clean":"2024-01-11T07:06:57.966453+0000","last_became_active":"2024-01-11T07:06:57.966083+0000","last_became_peered":"2024-01-11T07:06:57.966083+0000","last_unstale":"2024-01-11T07:06:57.966453+0000","last_undegraded":"2024-01-11T07:06:57.966453+0000","last_fullsized":"2024-01-11T07:06:57.966453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958698+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966021+0000","last_change":"2024-01-11T07:06:57.966021+0000","last_active":"2024-01-11T07:06:57.966021+0000","last_peered":"2024-01-11T07:06:57.966021+0000","last_clean":"2024-01-11T07:06:57.966021+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:57.966021+0000","last_undegraded":"2024-01-11T07:06:57.966021+0000","last_fullsized":"2024-01-11T07:06:57.966021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958190+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.962108+0000","last_change":"2024-01-11T07:06:57.962108+0000","last_active":"2024-01-11T07:06:57.962108+0000","last_peered":"2024-01-11T07:06:57.962108+0000","last_clean":"2024-01-11T07:06:57.962108+0000","last_became_active":"2024-01-11T07:06:57.961848+0000","last_became_peered":"2024-01-11T07:06:57.961848+0000","last_unstale":"2024-01-11T07:06:57.962108+0000","last_undegraded":"2024-01-11T07:06:57.962108+0000","last_fullsized":"2024-01-11T07:06:57.962108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957563+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960489+0000","last_change":"2024-01-11T07:06:57.960489+0000","last_active":"2024-01-11T07:06:57.960489+0000","last_peered":"2024-01-11T07:06:57.960489+0000","last_clean":"2024-01-11T07:06:57.960489+0000","last_became_active":"2024-01-11T07:06:57.960225+0000","last_became_peered":"2024-01-11T07:06:57.960225+0000","last_unstale":"2024-01-11T07:06:57.960489+0000","last_undegraded":"2024-01-11T07:06:57.960489+0000","last_fullsized":"2024-01-11T07:06:57.960489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956384+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965935+0000","last_change":"2024-01-11T07:06:57.965935+0000","last_active":"2024-01-11T07:06:57.965935+0000","last_peered":"2024-01-11T07:06:57.965935+0000","last_clean":"2024-01-11T07:06:57.965935+0000","last_became_active":"2024-01-11T07:06:57.965303+0000","last_became_peered":"2024-01-11T07:06:57.965303+0000","last_unstale":"2024-01-11T07:06:57.965935+0000","last_undegraded":"2024-01-11T07:06:57.965935+0000","last_fullsized":"2024-01-11T07:06:57.965935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958869+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960488+0000","last_change":"2024-01-11T07:06:57.960488+0000","last_active":"2024-01-11T07:06:57.960488+0000","last_peered":"2024-01-11T07:06:57.960488+0000","last_clean":"2024-01-11T07:06:57.960488+0000","last_became_active":"2024-01-11T07:06:57.959963+0000","last_became_peered":"2024-01-11T07:06:57.959963+0000","last_unstale":"2024-01-11T07:06:57.960488+0000","last_undegraded":"2024-01-11T07:06:57.960488+0000","last_fullsized":"2024-01-11T07:06:57.960488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955978+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.961526+0000","last_change":"2024-01-11T07:06:57.961526+0000","last_active":"2024-01-11T07:06:57.961526+0000","last_peered":"2024-01-11T07:06:57.961526+0000","last_clean":"2024-01-11T07:06:57.961526+0000","last_became_active":"2024-01-11T07:06:57.961256+0000","last_became_peered":"2024-01-11T07:06:57.961256+0000","last_unstale":"2024-01-11T07:06:57.961526+0000","last_undegraded":"2024-01-11T07:06:57.961526+0000","last_fullsized":"2024-01-11T07:06:57.961526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956737+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967501+0000","last_change":"2024-01-11T07:06:57.967501+0000","last_active":"2024-01-11T07:06:57.967501+0000","last_peered":"2024-01-11T07:06:57.967501+0000","last_clean":"2024-01-11T07:06:57.967501+0000","last_became_active":"2024-01-11T07:06:57.966742+0000","last_became_peered":"2024-01-11T07:06:57.966742+0000","last_unstale":"2024-01-11T07:06:57.967501+0000","last_undegraded":"2024-01-11T07:06:57.967501+0000","last_fullsized":"2024-01-11T07:06:57.967501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:43:51.416880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958479+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960497+0000","last_change":"2024-01-11T07:06:57.960497+0000","last_active":"2024-01-11T07:06:57.960497+0000","last_peered":"2024-01-11T07:06:57.960497+0000","last_clean":"2024-01-11T07:06:57.960497+0000","last_became_active":"2024-01-11T07:06:57.959939+0000","last_became_peered":"2024-01-11T07:06:57.959939+0000","last_unstale":"2024-01-11T07:06:57.960497+0000","last_undegraded":"2024-01-11T07:06:57.960497+0000","last_fullsized":"2024-01-11T07:06:57.960497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958461+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966310+0000","last_change":"2024-01-11T07:06:57.966310+0000","last_active":"2024-01-11T07:06:57.966310+0000","last_peered":"2024-01-11T07:06:57.966310+0000","last_clean":"2024-01-11T07:06:57.966310+0000","last_became_active":"2024-01-11T07:06:57.965787+0000","last_became_peered":"2024-01-11T07:06:57.965787+0000","last_unstale":"2024-01-11T07:06:57.966310+0000","last_undegraded":"2024-01-11T07:06:57.966310+0000","last_fullsized":"2024-01-11T07:06:57.966310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955926+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:44.410296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954184+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963281+0000","last_change":"2024-01-11T07:06:57.963281+0000","last_active":"2024-01-11T07:06:57.963281+0000","last_peered":"2024-01-11T07:06:57.963281+0000","last_clean":"2024-01-11T07:06:57.963281+0000","last_became_active":"2024-01-11T07:06:57.962993+0000","last_became_peered":"2024-01-11T07:06:57.962993+0000","last_unstale":"2024-01-11T07:06:57.963281+0000","last_undegraded":"2024-01-11T07:06:57.963281+0000","last_fullsized":"2024-01-11T07:06:57.963281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957364+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965590+0000","last_change":"2024-01-11T07:06:57.965590+0000","last_active":"2024-01-11T07:06:57.965590+0000","last_peered":"2024-01-11T07:06:57.965590+0000","last_clean":"2024-01-11T07:06:57.965590+0000","last_became_active":"2024-01-11T07:06:57.964910+0000","last_became_peered":"2024-01-11T07:06:57.964910+0000","last_unstale":"2024-01-11T07:06:57.965590+0000","last_undegraded":"2024-01-11T07:06:57.965590+0000","last_fullsized":"2024-01-11T07:06:57.965590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-11T07:06:56.953593+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:06:56.953593+0000","last_peered":"2024-01-11T07:06:56.953593+0000","last_clean":"2024-01-11T07:06:56.953593+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:06:56.953593+0000","last_undegraded":"2024-01-11T07:06:56.953593+0000","last_fullsized":"2024-01-11T07:06:56.953593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956951+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964928+0000","last_change":"2024-01-11T07:06:57.964928+0000","last_active":"2024-01-11T07:06:57.964928+0000","last_peered":"2024-01-11T07:06:57.964928+0000","last_clean":"2024-01-11T07:06:57.964928+0000","last_became_active":"2024-01-11T07:06:57.964692+0000","last_became_peered":"2024-01-11T07:06:57.964692+0000","last_unstale":"2024-01-11T07:06:57.964928+0000","last_undegraded":"2024-01-11T07:06:57.964928+0000","last_fullsized":"2024-01-11T07:06:57.964928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.967422+0000","last_change":"2024-01-11T07:06:57.967422+0000","last_active":"2024-01-11T07:06:57.967422+0000","last_peered":"2024-01-11T07:06:57.967422+0000","last_clean":"2024-01-11T07:06:57.967422+0000","last_became_active":"2024-01-11T07:06:57.967213+0000","last_became_peered":"2024-01-11T07:06:57.967213+0000","last_unstale":"2024-01-11T07:06:57.967422+0000","last_undegraded":"2024-01-11T07:06:57.967422+0000","last_fullsized":"2024-01-11T07:06:57.967422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:16.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954989+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.958705+0000","last_change":"2024-01-11T07:06:57.958705+0000","last_active":"2024-01-11T07:06:57.958705+0000","last_peered":"2024-01-11T07:06:57.958705+0000","last_clean":"2024-01-11T07:06:57.958705+0000","last_became_active":"2024-01-11T07:06:57.958421+0000","last_became_peered":"2024-01-11T07:06:57.958421+0000","last_unstale":"2024-01-11T07:06:57.958705+0000","last_undegraded":"2024-01-11T07:06:57.958705+0000","last_fullsized":"2024-01-11T07:06:57.958705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959243+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.964975+0000","last_change":"2024-01-11T07:06:57.964975+0000","last_active":"2024-01-11T07:06:57.964975+0000","last_peered":"2024-01-11T07:06:57.964975+0000","last_clean":"2024-01-11T07:06:57.964975+0000","last_became_active":"2024-01-11T07:06:57.964709+0000","last_became_peered":"2024-01-11T07:06:57.964709+0000","last_unstale":"2024-01-11T07:06:57.964975+0000","last_undegraded":"2024-01-11T07:06:57.964975+0000","last_fullsized":"2024-01-11T07:06:57.964975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957392+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963735+0000","last_change":"2024-01-11T07:06:57.963735+0000","last_active":"2024-01-11T07:06:57.963735+0000","last_peered":"2024-01-11T07:06:57.963735+0000","last_clean":"2024-01-11T07:06:57.963735+0000","last_became_active":"2024-01-11T07:06:57.963471+0000","last_became_peered":"2024-01-11T07:06:57.963471+0000","last_unstale":"2024-01-11T07:06:57.963735+0000","last_undegraded":"2024-01-11T07:06:57.963735+0000","last_fullsized":"2024-01-11T07:06:57.963735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957366+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:14.319327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.965092+0000","last_change":"2024-01-11T07:06:57.965092+0000","last_active":"2024-01-11T07:06:57.965092+0000","last_peered":"2024-01-11T07:06:57.965092+0000","last_clean":"2024-01-11T07:06:57.965092+0000","last_became_active":"2024-01-11T07:06:57.964872+0000","last_became_peered":"2024-01-11T07:06:57.964872+0000","last_unstale":"2024-01-11T07:06:57.965092+0000","last_undegraded":"2024-01-11T07:06:57.965092+0000","last_fullsized":"2024-01-11T07:06:57.965092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955602+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:37.085991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.962256+0000","last_change":"2024-01-11T07:06:57.962256+0000","last_active":"2024-01-11T07:06:57.962256+0000","last_peered":"2024-01-11T07:06:57.962256+0000","last_clean":"2024-01-11T07:06:57.962256+0000","last_became_active":"2024-01-11T07:06:57.961595+0000","last_became_peered":"2024-01-11T07:06:57.961595+0000","last_unstale":"2024-01-11T07:06:57.962256+0000","last_undegraded":"2024-01-11T07:06:57.962256+0000","last_fullsized":"2024-01-11T07:06:57.962256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955709+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963683+0000","last_change":"2024-01-11T07:06:57.963683+0000","last_active":"2024-01-11T07:06:57.963683+0000","last_peered":"2024-01-11T07:06:57.963683+0000","last_clean":"2024-01-11T07:06:57.963683+0000","last_became_active":"2024-01-11T07:06:57.963452+0000","last_became_peered":"2024-01-11T07:06:57.963452+0000","last_unstale":"2024-01-11T07:06:57.963683+0000","last_undegraded":"2024-01-11T07:06:57.963683+0000","last_fullsized":"2024-01-11T07:06:57.963683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955034+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.966946+0000","last_change":"2024-01-11T07:06:57.966946+0000","last_active":"2024-01-11T07:06:57.966946+0000","last_peered":"2024-01-11T07:06:57.966946+0000","last_clean":"2024-01-11T07:06:57.966946+0000","last_became_active":"2024-01-11T07:06:57.966738+0000","last_became_peered":"2024-01-11T07:06:57.966738+0000","last_unstale":"2024-01-11T07:06:57.966946+0000","last_undegraded":"2024-01-11T07:06:57.966946+0000","last_fullsized":"2024-01-11T07:06:57.966946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957201+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.960096+0000","last_change":"2024-01-11T07:06:57.960096+0000","last_active":"2024-01-11T07:06:57.960096+0000","last_peered":"2024-01-11T07:06:57.960096+0000","last_clean":"2024-01-11T07:06:57.960096+0000","last_became_active":"2024-01-11T07:06:57.959880+0000","last_became_peered":"2024-01-11T07:06:57.959880+0000","last_unstale":"2024-01-11T07:06:57.960096+0000","last_undegraded":"2024-01-11T07:06:57.960096+0000","last_fullsized":"2024-01-11T07:06:57.960096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954142+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":8,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-11T07:06:57.963359+0000","last_change":"2024-01-11T07:06:57.963359+0000","last_active":"2024-01-11T07:06:57.963359+0000","last_peered":"2024-01-11T07:06:57.963359+0000","last_clean":"2024-01-11T07:06:57.963359+0000","last_became_active":"2024-01-11T07:06:57.963116+0000","last_became_peered":"2024-01-11T07:06:57.963116+0000","last_unstale":"2024-01-11T07:06:57.963359+0000","last_undegraded":"2024-01-11T07:06:57.963359+0000","last_fullsized":"2024-01-11T07:06:57.963359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957041+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956322+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.952744+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.955821+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959313+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958284+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.953941+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954890+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.958024+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.956187+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:08.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.960581+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.957687+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.959186+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:06:57.948014+0000","last_change":"2024-01-11T07:06:57.954871+0000","last_active":"2024-01-11T07:06:57.948014+0000","last_peered":"2024-01-11T07:06:57.948014+0000","last_clean":"2024-01-11T07:06:57.948014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:06:57.948014+0000","last_undegraded":"2024-01-11T07:06:57.948014+0000","last_fullsized":"2024-01-11T07:06:57.948014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":59,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195253,"statfs":{"total":200000000,"available":199939116,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":65,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195246,"statfs":{"total":200000000,"available":199932853,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:08.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.160+0000 7f226c85b700 1 Processor -- start 2024-01-11T07:07:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.160+0000 7f226c85b700 1 -- start start 2024-01-11T07:07:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226c85b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2268159c90 0x7f22680b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226c85b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226815a060 0x7f22680b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226c85b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2268069320 0x7f22680b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226c85b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22680ce9b0 con 0x7f2268159c90 2024-01-11T07:07:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226c85b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22680cf530 con 0x7f2268069320 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226c85b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22680cf9c0 con 0x7f226815a060 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2268159c90 0x7f22680b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2268159c90 0x7f22680b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41192/0 (socket says 172.21.15.7:41192) 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 -- 172.21.15.7:0/3153225699 learned_addr learned my addr 172.21.15.7:0/3153225699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f2265d9b700 1 --2- 172.21.15.7:0/3153225699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226815a060 0x7f22680b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f2266d9d700 1 --2- 172.21.15.7:0/3153225699 >> v2:172.21.15.113:3300/0 conn(0x7f2268069320 0x7f22680b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 -- 172.21.15.7:0/3153225699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226815a060 msgr2=0x7f22680b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 --2- 172.21.15.7:0/3153225699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226815a060 0x7f22680b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 -- 172.21.15.7:0/3153225699 >> v2:172.21.15.113:3300/0 conn(0x7f2268069320 msgr2=0x7f22680b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 --2- 172.21.15.7:0/3153225699 >> v2:172.21.15.113:3300/0 conn(0x7f2268069320 0x7f22680b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 -- 172.21.15.7:0/3153225699 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22680cec00 con 0x7f2268159c90 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f2265d9b700 1 --2- 172.21.15.7:0/3153225699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226815a060 0x7f22680b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.161+0000 7f226659c700 1 --2- 172.21.15.7:0/3153225699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f22680b1820 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f22680ada00 tx=0x7f2258009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2990e362a83dcfe8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226559a700 1 -- 172.21.15.7:0/3153225699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800b860 con 0x7f2268159c90 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226559a700 1 -- 172.21.15.7:0/3153225699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225800b9c0 con 0x7f2268159c90 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226559a700 1 -- 172.21.15.7:0/3153225699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800bd60 con 0x7f2268159c90 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 -- 172.21.15.7:0/3153225699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 msgr2=0x7f22680b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 --2- 172.21.15.7:0/3153225699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f22680b1820 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f22680ada00 tx=0x7f2258009250 comp rx=0 tx=0).stop 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 -- 172.21.15.7:0/3153225699 shutdown_connections 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 --2- 172.21.15.7:0/3153225699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f22680b1820 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 --2- 172.21.15.7:0/3153225699 >> v2:172.21.15.113:3300/0 conn(0x7f2268069320 0x7f22680b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 --2- 172.21.15.7:0/3153225699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226815a060 0x7f22680b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 -- 172.21.15.7:0/3153225699 >> 172.21.15.7:0/3153225699 conn(0x7f2268071100 msgr2=0x7f226806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:08.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 -- 172.21.15.7:0/3153225699 shutdown_connections 2024-01-11T07:07:08.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.162+0000 7f226c85b700 1 -- 172.21.15.7:0/3153225699 wait complete. 2024-01-11T07:07:08.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.164+0000 7f226c85b700 1 Processor -- start 2024-01-11T07:07:08.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.164+0000 7f226c85b700 1 -- start start 2024-01-11T07:07:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.164+0000 7f226c85b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2268069320 0x7f2268151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.164+0000 7f226c85b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f2268152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.164+0000 7f226c85b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f226815a060 0x7f2268157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.164+0000 7f226c85b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226816b090 con 0x7f2268159c90 2024-01-11T07:07:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.164+0000 7f226c85b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226816aa80 con 0x7f226815a060 2024-01-11T07:07:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.164+0000 7f226c85b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2268151fe0 con 0x7f2268069320 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f2268152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f2268152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41204/0 (socket says 172.21.15.7:41204) 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 -- 172.21.15.7:0/433171691 learned_addr learned my addr 172.21.15.7:0/433171691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2266d9d700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f226815a060 0x7f2268157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f226659c700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2268069320 0x7f2268151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 -- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2268069320 msgr2=0x7f2268151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2268069320 0x7f2268151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 -- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f226815a060 msgr2=0x7f2268157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f226815a060 0x7f2268157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 -- 172.21.15.7:0/433171691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22580094f0 con 0x7f2268159c90 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f2265d9b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f2268152230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e8d0 tx=0x7f225c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f22567fc700 1 -- 172.21.15.7:0/433171691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c013410 con 0x7f2268159c90 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f22567fc700 1 -- 172.21.15.7:0/433171691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c013570 con 0x7f2268159c90 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2268159070 con 0x7f2268159c90 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f22567fc700 1 -- 172.21.15.7:0/433171691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c013910 con 0x7f2268159c90 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.165+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226816ad20 con 0x7f2268159c90 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.166+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226804c3c0 con 0x7f2268159c90 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.167+0000 7f22567fc700 1 -- 172.21.15.7:0/433171691 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f225c013d00 con 0x7f2268159c90 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.167+0000 7f22567fc700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2248066f30 0x7f22480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.167+0000 7f22567fc700 1 -- 172.21.15.7:0/433171691 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f225c0842c0 con 0x7f2268159c90 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.167+0000 7f226659c700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2248066f30 0x7f22480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.168+0000 7f226659c700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2248066f30 0x7f22480693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f22680ada00 tx=0x7f2258017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:08.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.172+0000 7f22567fc700 1 -- 172.21.15.7:0/433171691 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f225c051530 con 0x7f2268159c90 2024-01-11T07:07:08.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.372+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f226816b1d0 con 0x7f2248066f30 2024-01-11T07:07:08.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.380+0000 7f22567fc700 1 -- 172.21.15.7:0/433171691 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260119 (secure 0 0 0) 0x7f226816b1d0 con 0x7f2248066f30 2024-01-11T07:07:08.383 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:08.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2248066f30 msgr2=0x7f22480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2248066f30 0x7f22480693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f22680ada00 tx=0x7f2258017000 comp rx=0 tx=0).stop 2024-01-11T07:07:08.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 msgr2=0x7f2268152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f2268152230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e8d0 tx=0x7f225c008040 comp rx=0 tx=0).stop 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 shutdown_connections 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2248066f30 0x7f22480693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f226815a060 0x7f2268157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2268069320 0x7f2268151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 --2- 172.21.15.7:0/433171691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2268159c90 0x7f2268152230 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 >> 172.21.15.7:0/433171691 conn(0x7f2268071100 msgr2=0x7f226815d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 shutdown_connections 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.383+0000 7f226c85b700 1 -- 172.21.15.7:0/433171691 wait complete. 2024-01-11T07:07:08.386 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:08.396 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-01-11T07:07:07.840860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560715,"statfs":{"total":1600000000,"available":1598173068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.875325"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978080+0000","last_change":"2024-01-11T07:06:58.977426+0000","last_active":"2024-01-11T07:06:58.978080+0000","last_peered":"2024-01-11T07:06:58.978080+0000","last_clean":"2024-01-11T07:06:58.978080+0000","last_became_active":"2024-01-11T07:06:58.977173+0000","last_became_peered":"2024-01-11T07:06:58.977173+0000","last_unstale":"2024-01-11T07:06:58.978080+0000","last_undegraded":"2024-01-11T07:06:58.978080+0000","last_fullsized":"2024-01-11T07:06:58.978080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:27:13.892155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982635+0000","last_change":"2024-01-11T07:06:58.982509+0000","last_active":"2024-01-11T07:06:58.982635+0000","last_peered":"2024-01-11T07:06:58.982635+0000","last_clean":"2024-01-11T07:06:58.982635+0000","last_became_active":"2024-01-11T07:06:58.982234+0000","last_became_peered":"2024-01-11T07:06:58.982234+0000","last_unstale":"2024-01-11T07:06:58.982635+0000","last_undegraded":"2024-01-11T07:06:58.982635+0000","last_fullsized":"2024-01-11T07:06:58.982635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981566+0000","last_change":"2024-01-11T07:06:58.981034+0000","last_active":"2024-01-11T07:06:58.981566+0000","last_peered":"2024-01-11T07:06:58.981566+0000","last_clean":"2024-01-11T07:06:58.981566+0000","last_became_active":"2024-01-11T07:06:58.979996+0000","last_became_peered":"2024-01-11T07:06:58.979996+0000","last_unstale":"2024-01-11T07:06:58.981566+0000","last_undegraded":"2024-01-11T07:06:58.981566+0000","last_fullsized":"2024-01-11T07:06:58.981566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979116+0000","last_change":"2024-01-11T07:06:58.979053+0000","last_active":"2024-01-11T07:06:58.979116+0000","last_peered":"2024-01-11T07:06:58.979116+0000","last_clean":"2024-01-11T07:06:58.979116+0000","last_became_active":"2024-01-11T07:06:58.978648+0000","last_became_peered":"2024-01-11T07:06:58.978648+0000","last_unstale":"2024-01-11T07:06:58.979116+0000","last_undegraded":"2024-01-11T07:06:58.979116+0000","last_fullsized":"2024-01-11T07:06:58.979116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:11:14.542059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978059+0000","last_change":"2024-01-11T07:06:58.977393+0000","last_active":"2024-01-11T07:06:58.978059+0000","last_peered":"2024-01-11T07:06:58.978059+0000","last_clean":"2024-01-11T07:06:58.978059+0000","last_became_active":"2024-01-11T07:06:58.977145+0000","last_became_peered":"2024-01-11T07:06:58.977145+0000","last_unstale":"2024-01-11T07:06:58.978059+0000","last_undegraded":"2024-01-11T07:06:58.978059+0000","last_fullsized":"2024-01-11T07:06:58.978059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:07:39.492394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972053+0000","last_change":"2024-01-11T07:06:58.971961+0000","last_active":"2024-01-11T07:06:58.972053+0000","last_peered":"2024-01-11T07:06:58.972053+0000","last_clean":"2024-01-11T07:06:58.972053+0000","last_became_active":"2024-01-11T07:06:58.971577+0000","last_became_peered":"2024-01-11T07:06:58.971577+0000","last_unstale":"2024-01-11T07:06:58.972053+0000","last_undegraded":"2024-01-11T07:06:58.972053+0000","last_fullsized":"2024-01-11T07:06:58.972053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:27:13.892155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.984886+0000","last_change":"2024-01-11T07:06:58.984793+0000","last_active":"2024-01-11T07:06:58.984886+0000","last_peered":"2024-01-11T07:06:58.984886+0000","last_clean":"2024-01-11T07:06:58.984886+0000","last_became_active":"2024-01-11T07:06:58.984465+0000","last_became_peered":"2024-01-11T07:06:58.984465+0000","last_unstale":"2024-01-11T07:06:58.984886+0000","last_undegraded":"2024-01-11T07:06:58.984886+0000","last_fullsized":"2024-01-11T07:06:58.984886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:11:14.542059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977071+0000","last_change":"2024-01-11T07:06:58.976965+0000","last_active":"2024-01-11T07:06:58.977071+0000","last_peered":"2024-01-11T07:06:58.977071+0000","last_clean":"2024-01-11T07:06:58.977071+0000","last_became_active":"2024-01-11T07:06:58.976113+0000","last_became_peered":"2024-01-11T07:06:58.976113+0000","last_unstale":"2024-01-11T07:06:58.977071+0000","last_undegraded":"2024-01-11T07:06:58.977071+0000","last_fullsized":"2024-01-11T07:06:58.977071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978104+0000","last_change":"2024-01-11T07:06:58.977843+0000","last_active":"2024-01-11T07:06:58.978104+0000","last_peered":"2024-01-11T07:06:58.978104+0000","last_clean":"2024-01-11T07:06:58.978104+0000","last_became_active":"2024-01-11T07:06:58.976923+0000","last_became_peered":"2024-01-11T07:06:58.976923+0000","last_unstale":"2024-01-11T07:06:58.978104+0000","last_undegraded":"2024-01-11T07:06:58.978104+0000","last_fullsized":"2024-01-11T07:06:58.978104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:04:53.068090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975533+0000","last_change":"2024-01-11T07:06:58.975456+0000","last_active":"2024-01-11T07:06:58.975533+0000","last_peered":"2024-01-11T07:06:58.975533+0000","last_clean":"2024-01-11T07:06:58.975533+0000","last_became_active":"2024-01-11T07:06:58.974898+0000","last_became_peered":"2024-01-11T07:06:58.974898+0000","last_unstale":"2024-01-11T07:06:58.975533+0000","last_undegraded":"2024-01-11T07:06:58.975533+0000","last_fullsized":"2024-01-11T07:06:58.975533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975115+0000","last_change":"2024-01-11T07:06:58.975019+0000","last_active":"2024-01-11T07:06:58.975115+0000","last_peered":"2024-01-11T07:06:58.975115+0000","last_clean":"2024-01-11T07:06:58.975115+0000","last_became_active":"2024-01-11T07:06:58.974797+0000","last_became_peered":"2024-01-11T07:06:58.974797+0000","last_unstale":"2024-01-11T07:06:58.975115+0000","last_undegraded":"2024-01-11T07:06:58.975115+0000","last_fullsized":"2024-01-11T07:06:58.975115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975231+0000","last_change":"2024-01-11T07:06:58.974746+0000","last_active":"2024-01-11T07:06:58.975231+0000","last_peered":"2024-01-11T07:06:58.975231+0000","last_clean":"2024-01-11T07:06:58.975231+0000","last_became_active":"2024-01-11T07:06:58.974520+0000","last_became_peered":"2024-01-11T07:06:58.974520+0000","last_unstale":"2024-01-11T07:06:58.975231+0000","last_undegraded":"2024-01-11T07:06:58.975231+0000","last_fullsized":"2024-01-11T07:06:58.975231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982637+0000","last_change":"2024-01-11T07:06:58.982409+0000","last_active":"2024-01-11T07:06:58.982637+0000","last_peered":"2024-01-11T07:06:58.982637+0000","last_clean":"2024-01-11T07:06:58.982637+0000","last_became_active":"2024-01-11T07:06:58.981807+0000","last_became_peered":"2024-01-11T07:06:58.981807+0000","last_unstale":"2024-01-11T07:06:58.982637+0000","last_undegraded":"2024-01-11T07:06:58.982637+0000","last_fullsized":"2024-01-11T07:06:58.982637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978084+0000","last_change":"2024-01-11T07:06:58.977853+0000","last_active":"2024-01-11T07:06:58.978084+0000","last_peered":"2024-01-11T07:06:58.978084+0000","last_clean":"2024-01-11T07:06:58.978084+0000","last_became_active":"2024-01-11T07:06:58.977594+0000","last_became_peered":"2024-01-11T07:06:58.977594+0000","last_unstale":"2024-01-11T07:06:58.978084+0000","last_undegraded":"2024-01-11T07:06:58.978084+0000","last_fullsized":"2024-01-11T07:06:58.978084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982542+0000","last_change":"2024-01-11T07:06:58.982470+0000","last_active":"2024-01-11T07:06:58.982542+0000","last_peered":"2024-01-11T07:06:58.982542+0000","last_clean":"2024-01-11T07:06:58.982542+0000","last_became_active":"2024-01-11T07:06:58.982209+0000","last_became_peered":"2024-01-11T07:06:58.982209+0000","last_unstale":"2024-01-11T07:06:58.982542+0000","last_undegraded":"2024-01-11T07:06:58.982542+0000","last_fullsized":"2024-01-11T07:06:58.982542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971907+0000","last_change":"2024-01-11T07:06:58.971802+0000","last_active":"2024-01-11T07:06:58.971907+0000","last_peered":"2024-01-11T07:06:58.971907+0000","last_clean":"2024-01-11T07:06:58.971907+0000","last_became_active":"2024-01-11T07:06:58.971251+0000","last_became_peered":"2024-01-11T07:06:58.971251+0000","last_unstale":"2024-01-11T07:06:58.971907+0000","last_undegraded":"2024-01-11T07:06:58.971907+0000","last_fullsized":"2024-01-11T07:06:58.971907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:19.418614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983779+0000","last_change":"2024-01-11T07:06:58.983363+0000","last_active":"2024-01-11T07:06:58.983779+0000","last_peered":"2024-01-11T07:06:58.983779+0000","last_clean":"2024-01-11T07:06:58.983779+0000","last_became_active":"2024-01-11T07:06:58.982003+0000","last_became_peered":"2024-01-11T07:06:58.982003+0000","last_unstale":"2024-01-11T07:06:58.983779+0000","last_undegraded":"2024-01-11T07:06:58.983779+0000","last_fullsized":"2024-01-11T07:06:58.983779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:22:00.354437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975549+0000","last_change":"2024-01-11T07:06:58.975439+0000","last_active":"2024-01-11T07:06:58.975549+0000","last_peered":"2024-01-11T07:06:58.975549+0000","last_clean":"2024-01-11T07:06:58.975549+0000","last_became_active":"2024-01-11T07:06:58.975201+0000","last_became_peered":"2024-01-11T07:06:58.975201+0000","last_unstale":"2024-01-11T07:06:58.975549+0000","last_undegraded":"2024-01-11T07:06:58.975549+0000","last_fullsized":"2024-01-11T07:06:58.975549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978122+0000","last_change":"2024-01-11T07:06:58.977996+0000","last_active":"2024-01-11T07:06:58.978122+0000","last_peered":"2024-01-11T07:06:58.978122+0000","last_clean":"2024-01-11T07:06:58.978122+0000","last_became_active":"2024-01-11T07:06:58.977268+0000","last_became_peered":"2024-01-11T07:06:58.977268+0000","last_unstale":"2024-01-11T07:06:58.978122+0000","last_undegraded":"2024-01-11T07:06:58.978122+0000","last_fullsized":"2024-01-11T07:06:58.978122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:19.418614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.158069+0000","last_change":"2024-01-11T07:06:57.964441+0000","last_active":"2024-01-11T07:06:59.158069+0000","last_peered":"2024-01-11T07:06:59.158069+0000","last_clean":"2024-01-11T07:06:59.158069+0000","last_became_active":"2024-01-11T07:06:57.963971+0000","last_became_peered":"2024-01-11T07:06:57.963971+0000","last_unstale":"2024-01-11T07:06:59.158069+0000","last_undegraded":"2024-01-11T07:06:59.158069+0000","last_fullsized":"2024-01-11T07:06:59.158069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980035+0000","last_change":"2024-01-11T07:06:58.979891+0000","last_active":"2024-01-11T07:06:58.980035+0000","last_peered":"2024-01-11T07:06:58.980035+0000","last_clean":"2024-01-11T07:06:58.980035+0000","last_became_active":"2024-01-11T07:06:58.979638+0000","last_became_peered":"2024-01-11T07:06:58.979638+0000","last_unstale":"2024-01-11T07:06:58.980035+0000","last_undegraded":"2024-01-11T07:06:58.980035+0000","last_fullsized":"2024-01-11T07:06:58.980035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157701+0000","last_change":"2024-01-11T07:06:57.958809+0000","last_active":"2024-01-11T07:06:59.157701+0000","last_peered":"2024-01-11T07:06:59.157701+0000","last_clean":"2024-01-11T07:06:59.157701+0000","last_became_active":"2024-01-11T07:06:57.958478+0000","last_became_peered":"2024-01-11T07:06:57.958478+0000","last_unstale":"2024-01-11T07:06:59.157701+0000","last_undegraded":"2024-01-11T07:06:59.157701+0000","last_fullsized":"2024-01-11T07:06:59.157701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.969856+0000","last_change":"2024-01-11T07:06:58.969781+0000","last_active":"2024-01-11T07:06:58.969856+0000","last_peered":"2024-01-11T07:06:58.969856+0000","last_clean":"2024-01-11T07:06:58.969856+0000","last_became_active":"2024-01-11T07:06:58.969505+0000","last_became_peered":"2024-01-11T07:06:58.969505+0000","last_unstale":"2024-01-11T07:06:58.969856+0000","last_undegraded":"2024-01-11T07:06:58.969856+0000","last_fullsized":"2024-01-11T07:06:58.969856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156412+0000","last_change":"2024-01-11T07:06:57.967660+0000","last_active":"2024-01-11T07:06:59.156412+0000","last_peered":"2024-01-11T07:06:59.156412+0000","last_clean":"2024-01-11T07:06:59.156412+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:59.156412+0000","last_undegraded":"2024-01-11T07:06:59.156412+0000","last_fullsized":"2024-01-11T07:06:59.156412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970064+0000","last_change":"2024-01-11T07:06:58.969973+0000","last_active":"2024-01-11T07:06:58.970064+0000","last_peered":"2024-01-11T07:06:58.970064+0000","last_clean":"2024-01-11T07:06:58.970064+0000","last_became_active":"2024-01-11T07:06:58.969722+0000","last_became_peered":"2024-01-11T07:06:58.969722+0000","last_unstale":"2024-01-11T07:06:58.970064+0000","last_undegraded":"2024-01-11T07:06:58.970064+0000","last_fullsized":"2024-01-11T07:06:58.970064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.154688+0000","last_change":"2024-01-11T07:06:57.967196+0000","last_active":"2024-01-11T07:06:59.154688+0000","last_peered":"2024-01-11T07:06:59.154688+0000","last_clean":"2024-01-11T07:06:59.154688+0000","last_became_active":"2024-01-11T07:06:57.966964+0000","last_became_peered":"2024-01-11T07:06:57.966964+0000","last_unstale":"2024-01-11T07:06:59.154688+0000","last_undegraded":"2024-01-11T07:06:59.154688+0000","last_fullsized":"2024-01-11T07:06:59.154688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:27.038629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970044+0000","last_change":"2024-01-11T07:06:58.969967+0000","last_active":"2024-01-11T07:06:58.970044+0000","last_peered":"2024-01-11T07:06:58.970044+0000","last_clean":"2024-01-11T07:06:58.970044+0000","last_became_active":"2024-01-11T07:06:58.969708+0000","last_became_peered":"2024-01-11T07:06:58.969708+0000","last_unstale":"2024-01-11T07:06:58.970044+0000","last_undegraded":"2024-01-11T07:06:58.970044+0000","last_fullsized":"2024-01-11T07:06:58.970044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157494+0000","last_change":"2024-01-11T07:06:57.964730+0000","last_active":"2024-01-11T07:06:59.157494+0000","last_peered":"2024-01-11T07:06:59.157494+0000","last_clean":"2024-01-11T07:06:59.157494+0000","last_became_active":"2024-01-11T07:06:57.964464+0000","last_became_peered":"2024-01-11T07:06:57.964464+0000","last_unstale":"2024-01-11T07:06:59.157494+0000","last_undegraded":"2024-01-11T07:06:59.157494+0000","last_fullsized":"2024-01-11T07:06:59.157494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979054+0000","last_change":"2024-01-11T07:06:58.978898+0000","last_active":"2024-01-11T07:06:58.979054+0000","last_peered":"2024-01-11T07:06:58.979054+0000","last_clean":"2024-01-11T07:06:58.979054+0000","last_became_active":"2024-01-11T07:06:58.978593+0000","last_became_peered":"2024-01-11T07:06:58.978593+0000","last_unstale":"2024-01-11T07:06:58.979054+0000","last_undegraded":"2024-01-11T07:06:58.979054+0000","last_fullsized":"2024-01-11T07:06:58.979054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156518+0000","last_change":"2024-01-11T07:06:57.961158+0000","last_active":"2024-01-11T07:06:59.156518+0000","last_peered":"2024-01-11T07:06:59.156518+0000","last_clean":"2024-01-11T07:06:59.156518+0000","last_became_active":"2024-01-11T07:06:57.960836+0000","last_became_peered":"2024-01-11T07:06:57.960836+0000","last_unstale":"2024-01-11T07:06:59.156518+0000","last_undegraded":"2024-01-11T07:06:59.156518+0000","last_fullsized":"2024-01-11T07:06:59.156518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968566+0000","last_change":"2024-01-11T07:06:58.968497+0000","last_active":"2024-01-11T07:06:58.968566+0000","last_peered":"2024-01-11T07:06:58.968566+0000","last_clean":"2024-01-11T07:06:58.968566+0000","last_became_active":"2024-01-11T07:06:58.968280+0000","last_became_peered":"2024-01-11T07:06:58.968280+0000","last_unstale":"2024-01-11T07:06:58.968566+0000","last_undegraded":"2024-01-11T07:06:58.968566+0000","last_fullsized":"2024-01-11T07:06:58.968566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975862+0000","last_change":"2024-01-11T07:06:57.963583+0000","last_active":"2024-01-11T07:06:58.975862+0000","last_peered":"2024-01-11T07:06:58.975862+0000","last_clean":"2024-01-11T07:06:58.975862+0000","last_became_active":"2024-01-11T07:06:57.962966+0000","last_became_peered":"2024-01-11T07:06:57.962966+0000","last_unstale":"2024-01-11T07:06:58.975862+0000","last_undegraded":"2024-01-11T07:06:58.975862+0000","last_fullsized":"2024-01-11T07:06:58.975862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980008+0000","last_change":"2024-01-11T07:06:58.979636+0000","last_active":"2024-01-11T07:06:58.980008+0000","last_peered":"2024-01-11T07:06:58.980008+0000","last_clean":"2024-01-11T07:06:58.980008+0000","last_became_active":"2024-01-11T07:06:58.979362+0000","last_became_peered":"2024-01-11T07:06:58.979362+0000","last_unstale":"2024-01-11T07:06:58.980008+0000","last_undegraded":"2024-01-11T07:06:58.980008+0000","last_fullsized":"2024-01-11T07:06:58.980008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968179+0000","last_change":"2024-01-11T07:06:57.965145+0000","last_active":"2024-01-11T07:06:58.968179+0000","last_peered":"2024-01-11T07:06:58.968179+0000","last_clean":"2024-01-11T07:06:58.968179+0000","last_became_active":"2024-01-11T07:06:57.964890+0000","last_became_peered":"2024-01-11T07:06:57.964890+0000","last_unstale":"2024-01-11T07:06:58.968179+0000","last_undegraded":"2024-01-11T07:06:58.968179+0000","last_fullsized":"2024-01-11T07:06:58.968179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982521+0000","last_change":"2024-01-11T07:06:58.982437+0000","last_active":"2024-01-11T07:06:58.982521+0000","last_peered":"2024-01-11T07:06:58.982521+0000","last_clean":"2024-01-11T07:06:58.982521+0000","last_became_active":"2024-01-11T07:06:58.982170+0000","last_became_peered":"2024-01-11T07:06:58.982170+0000","last_unstale":"2024-01-11T07:06:58.982521+0000","last_undegraded":"2024-01-11T07:06:58.982521+0000","last_fullsized":"2024-01-11T07:06:58.982521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972373+0000","last_change":"2024-01-11T07:06:57.967613+0000","last_active":"2024-01-11T07:06:58.972373+0000","last_peered":"2024-01-11T07:06:58.972373+0000","last_clean":"2024-01-11T07:06:58.972373+0000","last_became_active":"2024-01-11T07:06:57.967356+0000","last_became_peered":"2024-01-11T07:06:57.967356+0000","last_unstale":"2024-01-11T07:06:58.972373+0000","last_undegraded":"2024-01-11T07:06:58.972373+0000","last_fullsized":"2024-01-11T07:06:58.972373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983217+0000","last_change":"2024-01-11T07:06:58.983040+0000","last_active":"2024-01-11T07:06:58.983217+0000","last_peered":"2024-01-11T07:06:58.983217+0000","last_clean":"2024-01-11T07:06:58.983217+0000","last_became_active":"2024-01-11T07:06:58.981241+0000","last_became_peered":"2024-01-11T07:06:58.981241+0000","last_unstale":"2024-01-11T07:06:58.983217+0000","last_undegraded":"2024-01-11T07:06:58.983217+0000","last_fullsized":"2024-01-11T07:06:58.983217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:57:17.525152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.967926+0000","last_change":"2024-01-11T07:06:57.966453+0000","last_active":"2024-01-11T07:06:58.967926+0000","last_peered":"2024-01-11T07:06:58.967926+0000","last_clean":"2024-01-11T07:06:58.967926+0000","last_became_active":"2024-01-11T07:06:57.966083+0000","last_became_peered":"2024-01-11T07:06:57.966083+0000","last_unstale":"2024-01-11T07:06:58.967926+0000","last_undegraded":"2024-01-11T07:06:58.967926+0000","last_fullsized":"2024-01-11T07:06:58.967926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970715+0000","last_change":"2024-01-11T07:06:58.970636+0000","last_active":"2024-01-11T07:06:58.970715+0000","last_peered":"2024-01-11T07:06:58.970715+0000","last_clean":"2024-01-11T07:06:58.970715+0000","last_became_active":"2024-01-11T07:06:58.970180+0000","last_became_peered":"2024-01-11T07:06:58.970180+0000","last_unstale":"2024-01-11T07:06:58.970715+0000","last_undegraded":"2024-01-11T07:06:58.970715+0000","last_fullsized":"2024-01-11T07:06:58.970715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156721+0000","last_change":"2024-01-11T07:06:57.966021+0000","last_active":"2024-01-11T07:06:59.156721+0000","last_peered":"2024-01-11T07:06:59.156721+0000","last_clean":"2024-01-11T07:06:59.156721+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:59.156721+0000","last_undegraded":"2024-01-11T07:06:59.156721+0000","last_fullsized":"2024-01-11T07:06:59.156721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980149+0000","last_change":"2024-01-11T07:06:58.980082+0000","last_active":"2024-01-11T07:06:58.980149+0000","last_peered":"2024-01-11T07:06:58.980149+0000","last_clean":"2024-01-11T07:06:58.980149+0000","last_became_active":"2024-01-11T07:06:58.979407+0000","last_became_peered":"2024-01-11T07:06:58.979407+0000","last_unstale":"2024-01-11T07:06:58.980149+0000","last_undegraded":"2024-01-11T07:06:58.980149+0000","last_fullsized":"2024-01-11T07:06:58.980149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.158497+0000","last_change":"2024-01-11T07:06:57.962108+0000","last_active":"2024-01-11T07:06:59.158497+0000","last_peered":"2024-01-11T07:06:59.158497+0000","last_clean":"2024-01-11T07:06:59.158497+0000","last_became_active":"2024-01-11T07:06:57.961848+0000","last_became_peered":"2024-01-11T07:06:57.961848+0000","last_unstale":"2024-01-11T07:06:59.158497+0000","last_undegraded":"2024-01-11T07:06:59.158497+0000","last_fullsized":"2024-01-11T07:06:59.158497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980308+0000","last_change":"2024-01-11T07:06:58.980239+0000","last_active":"2024-01-11T07:06:58.980308+0000","last_peered":"2024-01-11T07:06:58.980308+0000","last_clean":"2024-01-11T07:06:58.980308+0000","last_became_active":"2024-01-11T07:06:58.979974+0000","last_became_peered":"2024-01-11T07:06:58.979974+0000","last_unstale":"2024-01-11T07:06:58.980308+0000","last_undegraded":"2024-01-11T07:06:58.980308+0000","last_fullsized":"2024-01-11T07:06:58.980308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:57:17.525152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155524+0000","last_change":"2024-01-11T07:06:57.960489+0000","last_active":"2024-01-11T07:06:59.155524+0000","last_peered":"2024-01-11T07:06:59.155524+0000","last_clean":"2024-01-11T07:06:59.155524+0000","last_became_active":"2024-01-11T07:06:57.960225+0000","last_became_peered":"2024-01-11T07:06:57.960225+0000","last_unstale":"2024-01-11T07:06:59.155524+0000","last_undegraded":"2024-01-11T07:06:59.155524+0000","last_fullsized":"2024-01-11T07:06:59.155524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978545+0000","last_change":"2024-01-11T07:06:58.978471+0000","last_active":"2024-01-11T07:06:58.978545+0000","last_peered":"2024-01-11T07:06:58.978545+0000","last_clean":"2024-01-11T07:06:58.978545+0000","last_became_active":"2024-01-11T07:06:58.977884+0000","last_became_peered":"2024-01-11T07:06:58.977884+0000","last_unstale":"2024-01-11T07:06:58.978545+0000","last_undegraded":"2024-01-11T07:06:58.978545+0000","last_fullsized":"2024-01-11T07:06:58.978545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968025+0000","last_change":"2024-01-11T07:06:57.965935+0000","last_active":"2024-01-11T07:06:58.968025+0000","last_peered":"2024-01-11T07:06:58.968025+0000","last_clean":"2024-01-11T07:06:58.968025+0000","last_became_active":"2024-01-11T07:06:57.965303+0000","last_became_peered":"2024-01-11T07:06:57.965303+0000","last_unstale":"2024-01-11T07:06:58.968025+0000","last_undegraded":"2024-01-11T07:06:58.968025+0000","last_fullsized":"2024-01-11T07:06:58.968025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981838+0000","last_change":"2024-01-11T07:06:58.981581+0000","last_active":"2024-01-11T07:06:58.981838+0000","last_peered":"2024-01-11T07:06:58.981838+0000","last_clean":"2024-01-11T07:06:58.981838+0000","last_became_active":"2024-01-11T07:06:58.981032+0000","last_became_peered":"2024-01-11T07:06:58.981032+0000","last_unstale":"2024-01-11T07:06:58.981838+0000","last_undegraded":"2024-01-11T07:06:58.981838+0000","last_fullsized":"2024-01-11T07:06:58.981838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:05:09.439404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973296+0000","last_change":"2024-01-11T07:06:57.960488+0000","last_active":"2024-01-11T07:06:58.973296+0000","last_peered":"2024-01-11T07:06:58.973296+0000","last_clean":"2024-01-11T07:06:58.973296+0000","last_became_active":"2024-01-11T07:06:57.959963+0000","last_became_peered":"2024-01-11T07:06:57.959963+0000","last_unstale":"2024-01-11T07:06:58.973296+0000","last_undegraded":"2024-01-11T07:06:58.973296+0000","last_fullsized":"2024-01-11T07:06:58.973296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982184+0000","last_change":"2024-01-11T07:06:58.982110+0000","last_active":"2024-01-11T07:06:58.982184+0000","last_peered":"2024-01-11T07:06:58.982184+0000","last_clean":"2024-01-11T07:06:58.982184+0000","last_became_active":"2024-01-11T07:06:58.981828+0000","last_became_peered":"2024-01-11T07:06:58.981828+0000","last_unstale":"2024-01-11T07:06:58.982184+0000","last_undegraded":"2024-01-11T07:06:58.982184+0000","last_fullsized":"2024-01-11T07:06:58.982184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.159786+0000","last_change":"2024-01-11T07:06:57.961526+0000","last_active":"2024-01-11T07:06:59.159786+0000","last_peered":"2024-01-11T07:06:59.159786+0000","last_clean":"2024-01-11T07:06:59.159786+0000","last_became_active":"2024-01-11T07:06:57.961256+0000","last_became_peered":"2024-01-11T07:06:57.961256+0000","last_unstale":"2024-01-11T07:06:59.159786+0000","last_undegraded":"2024-01-11T07:06:59.159786+0000","last_fullsized":"2024-01-11T07:06:59.159786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980041+0000","last_change":"2024-01-11T07:06:58.979973+0000","last_active":"2024-01-11T07:06:58.980041+0000","last_peered":"2024-01-11T07:06:58.980041+0000","last_clean":"2024-01-11T07:06:58.980041+0000","last_became_active":"2024-01-11T07:06:58.979738+0000","last_became_peered":"2024-01-11T07:06:58.979738+0000","last_unstale":"2024-01-11T07:06:58.980041+0000","last_undegraded":"2024-01-11T07:06:58.980041+0000","last_fullsized":"2024-01-11T07:06:58.980041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:05.350780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155041+0000","last_change":"2024-01-11T07:06:57.967501+0000","last_active":"2024-01-11T07:06:59.155041+0000","last_peered":"2024-01-11T07:06:59.155041+0000","last_clean":"2024-01-11T07:06:59.155041+0000","last_became_active":"2024-01-11T07:06:57.966742+0000","last_became_peered":"2024-01-11T07:06:57.966742+0000","last_unstale":"2024-01-11T07:06:59.155041+0000","last_undegraded":"2024-01-11T07:06:59.155041+0000","last_fullsized":"2024-01-11T07:06:59.155041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:43:51.416880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981871+0000","last_change":"2024-01-11T07:06:58.981779+0000","last_active":"2024-01-11T07:06:58.981871+0000","last_peered":"2024-01-11T07:06:58.981871+0000","last_clean":"2024-01-11T07:06:58.981871+0000","last_became_active":"2024-01-11T07:06:58.981332+0000","last_became_peered":"2024-01-11T07:06:58.981332+0000","last_unstale":"2024-01-11T07:06:58.981871+0000","last_undegraded":"2024-01-11T07:06:58.981871+0000","last_fullsized":"2024-01-11T07:06:58.981871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:04:40.488711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.976544+0000","last_change":"2024-01-11T07:06:57.960497+0000","last_active":"2024-01-11T07:06:58.976544+0000","last_peered":"2024-01-11T07:06:58.976544+0000","last_clean":"2024-01-11T07:06:58.976544+0000","last_became_active":"2024-01-11T07:06:57.959939+0000","last_became_peered":"2024-01-11T07:06:57.959939+0000","last_unstale":"2024-01-11T07:06:58.976544+0000","last_undegraded":"2024-01-11T07:06:58.976544+0000","last_fullsized":"2024-01-11T07:06:58.976544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982201+0000","last_change":"2024-01-11T07:06:58.982135+0000","last_active":"2024-01-11T07:06:58.982201+0000","last_peered":"2024-01-11T07:06:58.982201+0000","last_clean":"2024-01-11T07:06:58.982201+0000","last_became_active":"2024-01-11T07:06:58.981852+0000","last_became_peered":"2024-01-11T07:06:58.981852+0000","last_unstale":"2024-01-11T07:06:58.982201+0000","last_undegraded":"2024-01-11T07:06:58.982201+0000","last_fullsized":"2024-01-11T07:06:58.982201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:05.350780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968680+0000","last_change":"2024-01-11T07:06:57.966310+0000","last_active":"2024-01-11T07:06:58.968680+0000","last_peered":"2024-01-11T07:06:58.968680+0000","last_clean":"2024-01-11T07:06:58.968680+0000","last_became_active":"2024-01-11T07:06:57.965787+0000","last_became_peered":"2024-01-11T07:06:57.965787+0000","last_unstale":"2024-01-11T07:06:58.968680+0000","last_undegraded":"2024-01-11T07:06:58.968680+0000","last_fullsized":"2024-01-11T07:06:58.968680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970968+0000","last_change":"2024-01-11T07:06:58.970832+0000","last_active":"2024-01-11T07:06:58.970968+0000","last_peered":"2024-01-11T07:06:58.970968+0000","last_clean":"2024-01-11T07:06:58.970968+0000","last_became_active":"2024-01-11T07:06:58.970599+0000","last_became_peered":"2024-01-11T07:06:58.970599+0000","last_unstale":"2024-01-11T07:06:58.970968+0000","last_undegraded":"2024-01-11T07:06:58.970968+0000","last_fullsized":"2024-01-11T07:06:58.970968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977892+0000","last_change":"2024-01-11T07:06:58.977802+0000","last_active":"2024-01-11T07:06:58.977892+0000","last_peered":"2024-01-11T07:06:58.977892+0000","last_clean":"2024-01-11T07:06:58.977892+0000","last_became_active":"2024-01-11T07:06:58.977558+0000","last_became_peered":"2024-01-11T07:06:58.977558+0000","last_unstale":"2024-01-11T07:06:58.977892+0000","last_undegraded":"2024-01-11T07:06:58.977892+0000","last_fullsized":"2024-01-11T07:06:58.977892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973773+0000","last_change":"2024-01-11T07:06:57.963281+0000","last_active":"2024-01-11T07:06:58.973773+0000","last_peered":"2024-01-11T07:06:58.973773+0000","last_clean":"2024-01-11T07:06:58.973773+0000","last_became_active":"2024-01-11T07:06:57.962993+0000","last_became_peered":"2024-01-11T07:06:57.962993+0000","last_unstale":"2024-01-11T07:06:58.973773+0000","last_undegraded":"2024-01-11T07:06:58.973773+0000","last_fullsized":"2024-01-11T07:06:58.973773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982425+0000","last_change":"2024-01-11T07:06:58.982334+0000","last_active":"2024-01-11T07:06:58.982425+0000","last_peered":"2024-01-11T07:06:58.982425+0000","last_clean":"2024-01-11T07:06:58.982425+0000","last_became_active":"2024-01-11T07:06:58.981895+0000","last_became_peered":"2024-01-11T07:06:58.981895+0000","last_unstale":"2024-01-11T07:06:58.982425+0000","last_undegraded":"2024-01-11T07:06:58.982425+0000","last_fullsized":"2024-01-11T07:06:58.982425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968343+0000","last_change":"2024-01-11T07:06:57.965590+0000","last_active":"2024-01-11T07:06:58.968343+0000","last_peered":"2024-01-11T07:06:58.968343+0000","last_clean":"2024-01-11T07:06:58.968343+0000","last_became_active":"2024-01-11T07:06:57.964910+0000","last_became_peered":"2024-01-11T07:06:57.964910+0000","last_unstale":"2024-01-11T07:06:58.968343+0000","last_undegraded":"2024-01-11T07:06:58.968343+0000","last_fullsized":"2024-01-11T07:06:58.968343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972745+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:06:58.972745+0000","last_peered":"2024-01-11T07:06:58.972745+0000","last_clean":"2024-01-11T07:06:58.972745+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:06:58.972745+0000","last_undegraded":"2024-01-11T07:06:58.972745+0000","last_fullsized":"2024-01-11T07:06:58.972745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.969817+0000","last_change":"2024-01-11T07:06:58.969690+0000","last_active":"2024-01-11T07:06:58.969817+0000","last_peered":"2024-01-11T07:06:58.969817+0000","last_clean":"2024-01-11T07:06:58.969817+0000","last_became_active":"2024-01-11T07:06:58.969269+0000","last_became_peered":"2024-01-11T07:06:58.969269+0000","last_unstale":"2024-01-11T07:06:58.969817+0000","last_undegraded":"2024-01-11T07:06:58.969817+0000","last_fullsized":"2024-01-11T07:06:58.969817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971432+0000","last_change":"2024-01-11T07:06:57.964928+0000","last_active":"2024-01-11T07:06:58.971432+0000","last_peered":"2024-01-11T07:06:58.971432+0000","last_clean":"2024-01-11T07:06:58.971432+0000","last_became_active":"2024-01-11T07:06:57.964692+0000","last_became_peered":"2024-01-11T07:06:57.964692+0000","last_unstale":"2024-01-11T07:06:58.971432+0000","last_undegraded":"2024-01-11T07:06:58.971432+0000","last_fullsized":"2024-01-11T07:06:58.971432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971975+0000","last_change":"2024-01-11T07:06:57.967422+0000","last_active":"2024-01-11T07:06:58.971975+0000","last_peered":"2024-01-11T07:06:58.971975+0000","last_clean":"2024-01-11T07:06:58.971975+0000","last_became_active":"2024-01-11T07:06:57.967213+0000","last_became_peered":"2024-01-11T07:06:57.967213+0000","last_unstale":"2024-01-11T07:06:58.971975+0000","last_undegraded":"2024-01-11T07:06:58.971975+0000","last_fullsized":"2024-01-11T07:06:58.971975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:16.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977579+0000","last_change":"2024-01-11T07:06:58.977325+0000","last_active":"2024-01-11T07:06:58.977579+0000","last_peered":"2024-01-11T07:06:58.977579+0000","last_clean":"2024-01-11T07:06:58.977579+0000","last_became_active":"2024-01-11T07:06:58.977041+0000","last_became_peered":"2024-01-11T07:06:58.977041+0000","last_unstale":"2024-01-11T07:06:58.977579+0000","last_undegraded":"2024-01-11T07:06:58.977579+0000","last_fullsized":"2024-01-11T07:06:58.977579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-11T07:07:00.222933+0000","last_change":"2024-01-11T07:06:57.958705+0000","last_active":"2024-01-11T07:07:00.222933+0000","last_peered":"2024-01-11T07:07:00.222933+0000","last_clean":"2024-01-11T07:07:00.222933+0000","last_became_active":"2024-01-11T07:06:57.958421+0000","last_became_peered":"2024-01-11T07:06:57.958421+0000","last_unstale":"2024-01-11T07:07:00.222933+0000","last_undegraded":"2024-01-11T07:07:00.222933+0000","last_fullsized":"2024-01-11T07:07:00.222933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980832+0000","last_change":"2024-01-11T07:06:58.980504+0000","last_active":"2024-01-11T07:06:58.980832+0000","last_peered":"2024-01-11T07:06:58.980832+0000","last_clean":"2024-01-11T07:06:58.980832+0000","last_became_active":"2024-01-11T07:06:58.980255+0000","last_became_peered":"2024-01-11T07:06:58.980255+0000","last_unstale":"2024-01-11T07:06:58.980832+0000","last_undegraded":"2024-01-11T07:06:58.980832+0000","last_fullsized":"2024-01-11T07:06:58.980832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157334+0000","last_change":"2024-01-11T07:06:57.964975+0000","last_active":"2024-01-11T07:06:59.157334+0000","last_peered":"2024-01-11T07:06:59.157334+0000","last_clean":"2024-01-11T07:06:59.157334+0000","last_became_active":"2024-01-11T07:06:57.964709+0000","last_became_peered":"2024-01-11T07:06:57.964709+0000","last_unstale":"2024-01-11T07:06:59.157334+0000","last_undegraded":"2024-01-11T07:06:59.157334+0000","last_fullsized":"2024-01-11T07:06:59.157334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981582+0000","last_change":"2024-01-11T07:06:58.981191+0000","last_active":"2024-01-11T07:06:58.981582+0000","last_peered":"2024-01-11T07:06:58.981582+0000","last_clean":"2024-01-11T07:06:58.981582+0000","last_became_active":"2024-01-11T07:06:58.980786+0000","last_became_peered":"2024-01-11T07:06:58.980786+0000","last_unstale":"2024-01-11T07:06:58.981582+0000","last_undegraded":"2024-01-11T07:06:58.981582+0000","last_fullsized":"2024-01-11T07:06:58.981582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975354+0000","last_change":"2024-01-11T07:06:57.963735+0000","last_active":"2024-01-11T07:06:58.975354+0000","last_peered":"2024-01-11T07:06:58.975354+0000","last_clean":"2024-01-11T07:06:58.975354+0000","last_became_active":"2024-01-11T07:06:57.963471+0000","last_became_peered":"2024-01-11T07:06:57.963471+0000","last_unstale":"2024-01-11T07:06:58.975354+0000","last_undegraded":"2024-01-11T07:06:58.975354+0000","last_fullsized":"2024-01-11T07:06:58.975354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978396+0000","last_change":"2024-01-11T07:06:58.978278+0000","last_active":"2024-01-11T07:06:58.978396+0000","last_peered":"2024-01-11T07:06:58.978396+0000","last_clean":"2024-01-11T07:06:58.978396+0000","last_became_active":"2024-01-11T07:06:58.976004+0000","last_became_peered":"2024-01-11T07:06:58.976004+0000","last_unstale":"2024-01-11T07:06:58.978396+0000","last_undegraded":"2024-01-11T07:06:58.978396+0000","last_fullsized":"2024-01-11T07:06:58.978396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.976655+0000","last_change":"2024-01-11T07:06:57.965092+0000","last_active":"2024-01-11T07:06:58.976655+0000","last_peered":"2024-01-11T07:06:58.976655+0000","last_clean":"2024-01-11T07:06:58.976655+0000","last_became_active":"2024-01-11T07:06:57.964872+0000","last_became_peered":"2024-01-11T07:06:57.964872+0000","last_unstale":"2024-01-11T07:06:58.976655+0000","last_undegraded":"2024-01-11T07:06:58.976655+0000","last_fullsized":"2024-01-11T07:06:58.976655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981559+0000","last_change":"2024-01-11T07:06:58.981370+0000","last_active":"2024-01-11T07:06:58.981559+0000","last_peered":"2024-01-11T07:06:58.981559+0000","last_clean":"2024-01-11T07:06:58.981559+0000","last_became_active":"2024-01-11T07:06:58.981071+0000","last_became_peered":"2024-01-11T07:06:58.981071+0000","last_unstale":"2024-01-11T07:06:58.981559+0000","last_undegraded":"2024-01-11T07:06:58.981559+0000","last_fullsized":"2024-01-11T07:06:58.981559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973250+0000","last_change":"2024-01-11T07:06:57.962256+0000","last_active":"2024-01-11T07:06:58.973250+0000","last_peered":"2024-01-11T07:06:58.973250+0000","last_clean":"2024-01-11T07:06:58.973250+0000","last_became_active":"2024-01-11T07:06:57.961595+0000","last_became_peered":"2024-01-11T07:06:57.961595+0000","last_unstale":"2024-01-11T07:06:58.973250+0000","last_undegraded":"2024-01-11T07:06:58.973250+0000","last_fullsized":"2024-01-11T07:06:58.973250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.984461+0000","last_change":"2024-01-11T07:06:58.984384+0000","last_active":"2024-01-11T07:06:58.984461+0000","last_peered":"2024-01-11T07:06:58.984461+0000","last_clean":"2024-01-11T07:06:58.984461+0000","last_became_active":"2024-01-11T07:06:58.984107+0000","last_became_peered":"2024-01-11T07:06:58.984107+0000","last_unstale":"2024-01-11T07:06:58.984461+0000","last_undegraded":"2024-01-11T07:06:58.984461+0000","last_fullsized":"2024-01-11T07:06:58.984461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.154946+0000","last_change":"2024-01-11T07:06:57.963683+0000","last_active":"2024-01-11T07:06:59.154946+0000","last_peered":"2024-01-11T07:06:59.154946+0000","last_clean":"2024-01-11T07:06:59.154946+0000","last_became_active":"2024-01-11T07:06:57.963452+0000","last_became_peered":"2024-01-11T07:06:57.963452+0000","last_unstale":"2024-01-11T07:06:59.154946+0000","last_undegraded":"2024-01-11T07:06:59.154946+0000","last_fullsized":"2024-01-11T07:06:59.154946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978185+0000","last_change":"2024-01-11T07:06:58.978001+0000","last_active":"2024-01-11T07:06:58.978185+0000","last_peered":"2024-01-11T07:06:58.978185+0000","last_clean":"2024-01-11T07:06:58.978185+0000","last_became_active":"2024-01-11T07:06:58.977325+0000","last_became_peered":"2024-01-11T07:06:58.977325+0000","last_unstale":"2024-01-11T07:06:58.978185+0000","last_undegraded":"2024-01-11T07:06:58.978185+0000","last_fullsized":"2024-01-11T07:06:58.978185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973136+0000","last_change":"2024-01-11T07:06:57.966946+0000","last_active":"2024-01-11T07:06:58.973136+0000","last_peered":"2024-01-11T07:06:58.973136+0000","last_clean":"2024-01-11T07:06:58.973136+0000","last_became_active":"2024-01-11T07:06:57.966738+0000","last_became_peered":"2024-01-11T07:06:57.966738+0000","last_unstale":"2024-01-11T07:06:58.973136+0000","last_undegraded":"2024-01-11T07:06:58.973136+0000","last_fullsized":"2024-01-11T07:06:58.973136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981855+0000","last_change":"2024-01-11T07:06:58.981768+0000","last_active":"2024-01-11T07:06:58.981855+0000","last_peered":"2024-01-11T07:06:58.981855+0000","last_clean":"2024-01-11T07:06:58.981855+0000","last_became_active":"2024-01-11T07:06:58.981467+0000","last_became_peered":"2024-01-11T07:06:58.981467+0000","last_unstale":"2024-01-11T07:06:58.981855+0000","last_undegraded":"2024-01-11T07:06:58.981855+0000","last_fullsized":"2024-01-11T07:06:58.981855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:22:00.354437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156189+0000","last_change":"2024-01-11T07:06:57.960096+0000","last_active":"2024-01-11T07:06:59.156189+0000","last_peered":"2024-01-11T07:06:59.156189+0000","last_clean":"2024-01-11T07:06:59.156189+0000","last_became_active":"2024-01-11T07:06:57.959880+0000","last_became_peered":"2024-01-11T07:06:57.959880+0000","last_unstale":"2024-01-11T07:06:59.156189+0000","last_undegraded":"2024-01-11T07:06:59.156189+0000","last_fullsized":"2024-01-11T07:06:59.156189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983031+0000","last_change":"2024-01-11T07:06:58.982845+0000","last_active":"2024-01-11T07:06:58.983031+0000","last_peered":"2024-01-11T07:06:58.983031+0000","last_clean":"2024-01-11T07:06:58.983031+0000","last_became_active":"2024-01-11T07:06:58.980928+0000","last_became_peered":"2024-01-11T07:06:58.980928+0000","last_unstale":"2024-01-11T07:06:58.983031+0000","last_undegraded":"2024-01-11T07:06:58.983031+0000","last_fullsized":"2024-01-11T07:06:58.983031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155763+0000","last_change":"2024-01-11T07:06:57.963359+0000","last_active":"2024-01-11T07:06:59.155763+0000","last_peered":"2024-01-11T07:06:59.155763+0000","last_clean":"2024-01-11T07:06:59.155763+0000","last_became_active":"2024-01-11T07:06:57.963116+0000","last_became_peered":"2024-01-11T07:06:57.963116+0000","last_unstale":"2024-01-11T07:06:59.155763+0000","last_undegraded":"2024-01-11T07:06:59.155763+0000","last_fullsized":"2024-01-11T07:06:59.155763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978190+0000","last_change":"2024-01-11T07:06:58.978133+0000","last_active":"2024-01-11T07:06:58.978190+0000","last_peered":"2024-01-11T07:06:58.978190+0000","last_clean":"2024-01-11T07:06:58.978190+0000","last_became_active":"2024-01-11T07:06:58.977605+0000","last_became_peered":"2024-01-11T07:06:58.977605+0000","last_unstale":"2024-01-11T07:06:58.978190+0000","last_undegraded":"2024-01-11T07:06:58.978190+0000","last_fullsized":"2024-01-11T07:06:58.978190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982813+0000","last_change":"2024-01-11T07:06:58.982744+0000","last_active":"2024-01-11T07:06:58.982813+0000","last_peered":"2024-01-11T07:06:58.982813+0000","last_clean":"2024-01-11T07:06:58.982813+0000","last_became_active":"2024-01-11T07:06:58.982442+0000","last_became_peered":"2024-01-11T07:06:58.982442+0000","last_unstale":"2024-01-11T07:06:58.982813+0000","last_undegraded":"2024-01-11T07:06:58.982813+0000","last_fullsized":"2024-01-11T07:06:58.982813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975116+0000","last_change":"2024-01-11T07:06:58.974838+0000","last_active":"2024-01-11T07:06:58.975116+0000","last_peered":"2024-01-11T07:06:58.975116+0000","last_clean":"2024-01-11T07:06:58.975116+0000","last_became_active":"2024-01-11T07:06:58.974628+0000","last_became_peered":"2024-01-11T07:06:58.974628+0000","last_unstale":"2024-01-11T07:06:58.975116+0000","last_undegraded":"2024-01-11T07:06:58.975116+0000","last_fullsized":"2024-01-11T07:06:58.975116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980086+0000","last_change":"2024-01-11T07:06:58.979993+0000","last_active":"2024-01-11T07:06:58.980086+0000","last_peered":"2024-01-11T07:06:58.980086+0000","last_clean":"2024-01-11T07:06:58.980086+0000","last_became_active":"2024-01-11T07:06:58.979782+0000","last_became_peered":"2024-01-11T07:06:58.979782+0000","last_unstale":"2024-01-11T07:06:58.980086+0000","last_undegraded":"2024-01-11T07:06:58.980086+0000","last_fullsized":"2024-01-11T07:06:58.980086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979408+0000","last_change":"2024-01-11T07:06:58.979256+0000","last_active":"2024-01-11T07:06:58.979408+0000","last_peered":"2024-01-11T07:06:58.979408+0000","last_clean":"2024-01-11T07:06:58.979408+0000","last_became_active":"2024-01-11T07:06:58.979025+0000","last_became_peered":"2024-01-11T07:06:58.979025+0000","last_unstale":"2024-01-11T07:06:58.979408+0000","last_undegraded":"2024-01-11T07:06:58.979408+0000","last_fullsized":"2024-01-11T07:06:58.979408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978671+0000","last_change":"2024-01-11T07:06:58.978602+0000","last_active":"2024-01-11T07:06:58.978671+0000","last_peered":"2024-01-11T07:06:58.978671+0000","last_clean":"2024-01-11T07:06:58.978671+0000","last_became_active":"2024-01-11T07:06:58.978265+0000","last_became_peered":"2024-01-11T07:06:58.978265+0000","last_unstale":"2024-01-11T07:06:58.978671+0000","last_undegraded":"2024-01-11T07:06:58.978671+0000","last_fullsized":"2024-01-11T07:06:58.978671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978598+0000","last_change":"2024-01-11T07:06:58.978524+0000","last_active":"2024-01-11T07:06:58.978598+0000","last_peered":"2024-01-11T07:06:58.978598+0000","last_clean":"2024-01-11T07:06:58.978598+0000","last_became_active":"2024-01-11T07:06:58.978266+0000","last_became_peered":"2024-01-11T07:06:58.978266+0000","last_unstale":"2024-01-11T07:06:58.978598+0000","last_undegraded":"2024-01-11T07:06:58.978598+0000","last_fullsized":"2024-01-11T07:06:58.978598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977723+0000","last_change":"2024-01-11T07:06:58.977673+0000","last_active":"2024-01-11T07:06:58.977723+0000","last_peered":"2024-01-11T07:06:58.977723+0000","last_clean":"2024-01-11T07:06:58.977723+0000","last_became_active":"2024-01-11T07:06:58.977490+0000","last_became_peered":"2024-01-11T07:06:58.977490+0000","last_unstale":"2024-01-11T07:06:58.977723+0000","last_undegraded":"2024-01-11T07:06:58.977723+0000","last_fullsized":"2024-01-11T07:06:58.977723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982405+0000","last_change":"2024-01-11T07:06:58.982014+0000","last_active":"2024-01-11T07:06:58.982405+0000","last_peered":"2024-01-11T07:06:58.982405+0000","last_clean":"2024-01-11T07:06:58.982405+0000","last_became_active":"2024-01-11T07:06:58.981670+0000","last_became_peered":"2024-01-11T07:06:58.981670+0000","last_unstale":"2024-01-11T07:06:58.982405+0000","last_undegraded":"2024-01-11T07:06:58.982405+0000","last_fullsized":"2024-01-11T07:06:58.982405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979096+0000","last_change":"2024-01-11T07:06:58.978874+0000","last_active":"2024-01-11T07:06:58.979096+0000","last_peered":"2024-01-11T07:06:58.979096+0000","last_clean":"2024-01-11T07:06:58.979096+0000","last_became_active":"2024-01-11T07:06:58.978319+0000","last_became_peered":"2024-01-11T07:06:58.978319+0000","last_unstale":"2024-01-11T07:06:58.979096+0000","last_undegraded":"2024-01-11T07:06:58.979096+0000","last_fullsized":"2024-01-11T07:06:58.979096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977051+0000","last_change":"2024-01-11T07:06:58.976808+0000","last_active":"2024-01-11T07:06:58.977051+0000","last_peered":"2024-01-11T07:06:58.977051+0000","last_clean":"2024-01-11T07:06:58.977051+0000","last_became_active":"2024-01-11T07:06:58.975696+0000","last_became_peered":"2024-01-11T07:06:58.975696+0000","last_unstale":"2024-01-11T07:06:58.977051+0000","last_undegraded":"2024-01-11T07:06:58.977051+0000","last_fullsized":"2024-01-11T07:06:58.977051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983760+0000","last_change":"2024-01-11T07:06:58.983202+0000","last_active":"2024-01-11T07:06:58.983760+0000","last_peered":"2024-01-11T07:06:58.983760+0000","last_clean":"2024-01-11T07:06:58.983760+0000","last_became_active":"2024-01-11T07:06:58.981685+0000","last_became_peered":"2024-01-11T07:06:58.981685+0000","last_unstale":"2024-01-11T07:06:58.983760+0000","last_undegraded":"2024-01-11T07:06:58.983760+0000","last_fullsized":"2024-01-11T07:06:58.983760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:04:53.068090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978684+0000","last_change":"2024-01-11T07:06:58.978620+0000","last_active":"2024-01-11T07:06:58.978684+0000","last_peered":"2024-01-11T07:06:58.978684+0000","last_clean":"2024-01-11T07:06:58.978684+0000","last_became_active":"2024-01-11T07:06:58.978229+0000","last_became_peered":"2024-01-11T07:06:58.978229+0000","last_unstale":"2024-01-11T07:06:58.978684+0000","last_undegraded":"2024-01-11T07:06:58.978684+0000","last_fullsized":"2024-01-11T07:06:58.978684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981445+0000","last_change":"2024-01-11T07:06:58.981078+0000","last_active":"2024-01-11T07:06:58.981445+0000","last_peered":"2024-01-11T07:06:58.981445+0000","last_clean":"2024-01-11T07:06:58.981445+0000","last_became_active":"2024-01-11T07:06:58.980821+0000","last_became_peered":"2024-01-11T07:06:58.980821+0000","last_unstale":"2024-01-11T07:06:58.981445+0000","last_undegraded":"2024-01-11T07:06:58.981445+0000","last_fullsized":"2024-01-11T07:06:58.981445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":655,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199328880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":79,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195232,"statfs":{"total":200000000,"available":199918266,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":655,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199328824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":79,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195232,"statfs":{"total":200000000,"available":199918266,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919676,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919712,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:08.400 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:07:08.400 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-11T07:07:08.401 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-11T07:07:08.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-11T07:07:08.401 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-11T07:07:08.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.571+0000 7f36a6dc6700 1 Processor -- start 2024-01-11T07:07:08.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.571+0000 7f36a6dc6700 1 -- start start 2024-01-11T07:07:08.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.572+0000 7f36a6dc6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a0159e20 0x7f36a015e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.572+0000 7f36a6dc6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a015e010 con 0x7f36a006bc50 2024-01-11T07:07:08.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.572+0000 7f36a6dc6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a00acf80 con 0x7f36a0159a50 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.572+0000 7f36a6dc6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a015d9c0 con 0x7f36a0159e20 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.572+0000 7f36a5363700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f36a006bc50 0x7f36a006dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40132/0 (socket says 172.21.15.7:40132) 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.572+0000 7f36a5363700 1 -- 172.21.15.7:0/249895306 learned_addr learned my addr 172.21.15.7:0/249895306 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.572+0000 7f369ffff700 1 --2- 172.21.15.7:0/249895306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a0159e20 0x7f36a015e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.572+0000 7f369f7fe700 1 -- 172.21.15.7:0/249895306 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091859982 0 0) 0x7f36a015e010 con 0x7f36a006bc50 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369f7fe700 1 -- 172.21.15.7:0/249895306 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3688003570 con 0x7f36a006bc50 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369f7fe700 1 -- 172.21.15.7:0/249895306 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227392359 0 0) 0x7f36a00acf80 con 0x7f36a0159a50 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369f7fe700 1 -- 172.21.15.7:0/249895306 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36a015e010 con 0x7f36a0159a50 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369ffff700 1 -- 172.21.15.7:0/249895306 >> v1:172.21.15.113:6789/0 conn(0x7f36a0159a50 legacy=0x7f36a00ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369ffff700 1 -- 172.21.15.7:0/249895306 >> v1:172.21.15.7:6789/0 conn(0x7f36a006bc50 legacy=0x7f36a006dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369f7fe700 1 -- 172.21.15.7:0/249895306 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 940837085 0 0) 0x7f3688003570 con 0x7f36a006bc50 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369ffff700 1 -- 172.21.15.7:0/249895306 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a015dca0 con 0x7f36a0159e20 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369ffff700 1 --2- 172.21.15.7:0/249895306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a0159e20 0x7f36a015e4a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3694009910 tx=0x7f36940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=881a4f1cce18cae3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369f7fe700 1 -- 172.21.15.7:0/249895306 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980017f0 con 0x7f36a0159e20 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369f7fe700 1 -- 172.21.15.7:0/249895306 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3694011420 con 0x7f36a0159e20 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.573+0000 7f369f7fe700 1 -- 172.21.15.7:0/249895306 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36940117c0 con 0x7f36a0159e20 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.574+0000 7f36a6dc6700 1 -- 172.21.15.7:0/249895306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a0159e20 msgr2=0x7f36a015e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.574+0000 7f36a6dc6700 1 --2- 172.21.15.7:0/249895306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a0159e20 0x7f36a015e4a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3694009910 tx=0x7f36940091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.574+0000 7f36a6dc6700 1 -- 172.21.15.7:0/249895306 shutdown_connections 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.574+0000 7f36a6dc6700 1 --2- 172.21.15.7:0/249895306 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a0159e20 0x7f36a015e4a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.574+0000 7f36a6dc6700 1 -- 172.21.15.7:0/249895306 >> 172.21.15.7:0/249895306 conn(0x7f36a00cd910 msgr2=0x7f36a00cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.574+0000 7f36a6dc6700 1 -- 172.21.15.7:0/249895306 shutdown_connections 2024-01-11T07:07:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.574+0000 7f36a6dc6700 1 -- 172.21.15.7:0/249895306 wait complete. 2024-01-11T07:07:08.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.575+0000 7f36a6dc6700 1 Processor -- start 2024-01-11T07:07:08.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.575+0000 7f36a6dc6700 1 -- start start 2024-01-11T07:07:08.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a6dc6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a006bc50 0x7f36a014a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a6dc6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36a0159a50 0x7f36a015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a6dc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36a0159e20 0x7f36a014b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a6dc6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a014dcf0 con 0x7f36a0159e20 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a6dc6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a014a940 con 0x7f36a0159a50 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a6dc6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a015bc10 con 0x7f36a006bc50 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a5363700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36a0159e20 0x7f36a014b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a5363700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36a0159e20 0x7f36a014b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41206/0 (socket says 172.21.15.7:41206) 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a5363700 1 -- 172.21.15.7:0/3064737935 learned_addr learned my addr 172.21.15.7:0/3064737935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f36a4b62700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a006bc50 0x7f36a014a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.576+0000 7f369ffff700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36a0159a50 0x7f36a015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a5363700 1 -- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a006bc50 msgr2=0x7f36a014a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a5363700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a006bc50 0x7f36a014a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a5363700 1 -- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36a0159a50 msgr2=0x7f36a015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a5363700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36a0159a50 0x7f36a015c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a5363700 1 -- 172.21.15.7:0/3064737935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694009420 con 0x7f36a0159e20 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f369ffff700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36a0159a50 0x7f36a015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a4b62700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a006bc50 0x7f36a014a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a5363700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36a0159e20 0x7f36a014b850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f369800cc80 tx=0x7f3698007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f369cff9700 1 -- 172.21.15.7:0/3064737935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800ce30 con 0x7f36a0159e20 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f369cff9700 1 -- 172.21.15.7:0/3064737935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3698004dc0 con 0x7f36a0159e20 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a016ad10 con 0x7f36a0159e20 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f369cff9700 1 -- 172.21.15.7:0/3064737935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800ab10 con 0x7f36a0159e20 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.577+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a015b640 con 0x7f36a0159e20 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.578+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a015bd80 con 0x7f36a0159e20 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.578+0000 7f369cff9700 1 -- 172.21.15.7:0/3064737935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3698007150 con 0x7f36a0159e20 2024-01-11T07:07:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.579+0000 7f369cff9700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.579+0000 7f369cff9700 1 -- 172.21.15.7:0/3064737935 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3698083fc0 con 0x7f36a0159e20 2024-01-11T07:07:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.579+0000 7f36a4b62700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.580+0000 7f36a4b62700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f368c066f30 0x7f368c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f36a006d670 tx=0x7f3690009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:08.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.583+0000 7f369cff9700 1 -- 172.21.15.7:0/3064737935 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36980512b0 con 0x7f36a0159e20 2024-01-11T07:07:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.844+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f36a004c3c0 con 0x7f36a0159e20 2024-01-11T07:07:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.847+0000 7f369cff9700 1 -- 172.21.15.7:0/3064737935 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163706 (secure 0 0 0) 0x7f3698018090 con 0x7f36a0159e20 2024-01-11T07:07:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f368c066f30 msgr2=0x7f368c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f368c066f30 0x7f368c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f36a006d670 tx=0x7f3690009040 comp rx=0 tx=0).stop 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36a0159e20 msgr2=0x7f36a014b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36a0159e20 0x7f36a014b850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f369800cc80 tx=0x7f3698007d20 comp rx=0 tx=0).stop 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 shutdown_connections 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36a0159e20 0x7f36a014b850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36a006bc50 0x7f36a014a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 --2- 172.21.15.7:0/3064737935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36a0159a50 0x7f36a015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 >> 172.21.15.7:0/3064737935 conn(0x7f36a00cd910 msgr2=0x7f36a00b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 shutdown_connections 2024-01-11T07:07:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.850+0000 7f36a6dc6700 1 -- 172.21.15.7:0/3064737935 wait complete. 2024-01-11T07:07:08.863 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":36527},{"type":"v1","addr":"172.21.15.113:6833","nonce":36527}]},"active_addr":"172.21.15.113:6833/36527","active_change":"2024-01-11T07:06:47.830102+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1974885379}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":553680188}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":420421956}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":3743808411}]}]} 2024-01-11T07:07:08.866 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-11T07:07:08.866 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-11T07:07:08.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:08.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.999+0000 7f5271363700 1 Processor -- start 2024-01-11T07:07:08.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:08.999+0000 7f5271363700 1 -- start start 2024-01-11T07:07:09.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f5271363700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f526c0cd6d0 0x7f526c0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f5271363700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15e450 0x7f526c159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f5271363700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526c15acf0 con 0x7f526c15d8a0 2024-01-11T07:07:09.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f5271363700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526c15afd0 con 0x7f526c0cd6d0 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f5271363700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526c0ce980 con 0x7f526c15e450 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f526affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f526c0cd6d0 0x7f526c0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f526affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f526c0cd6d0 0x7f526c0cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41220/0 (socket says 172.21.15.7:41220) 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f526affd700 1 -- 172.21.15.7:0/1159134335 learned_addr learned my addr 172.21.15.7:0/1159134335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.000+0000 7f526b7fe700 1 --2- 172.21.15.7:0/1159134335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15e450 0x7f526c159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f526affd700 1 -- 172.21.15.7:0/1159134335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15e450 msgr2=0x7f526c159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f526affd700 1 --2- 172.21.15.7:0/1159134335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15e450 0x7f526c159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f526affd700 1 -- 172.21.15.7:0/1159134335 >> v1:172.21.15.113:6789/0 conn(0x7f526c15d8a0 legacy=0x7f526c15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f526affd700 1 -- 172.21.15.7:0/1159134335 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c0cec60 con 0x7f526c0cd6d0 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f5269ffb700 1 -- 172.21.15.7:0/1159134335 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229408713 0 0) 0x7f526c15acf0 con 0x7f526c15d8a0 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f526affd700 1 --2- 172.21.15.7:0/1159134335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0cdaa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f525c0099b0 tx=0x7f525c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632ee330a9d12b80 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f5269ffb700 1 -- 172.21.15.7:0/1159134335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c009ea0 con 0x7f526c0cd6d0 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f5269ffb700 1 -- 172.21.15.7:0/1159134335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c00b860 con 0x7f526c0cd6d0 2024-01-11T07:07:09.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f5269ffb700 1 -- 172.21.15.7:0/1159134335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c00bc00 con 0x7f526c0cd6d0 2024-01-11T07:07:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f5271363700 1 -- 172.21.15.7:0/1159134335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 msgr2=0x7f526c0cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f5271363700 1 --2- 172.21.15.7:0/1159134335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0cdaa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f525c0099b0 tx=0x7f525c009250 comp rx=0 tx=0).stop 2024-01-11T07:07:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.001+0000 7f5271363700 1 -- 172.21.15.7:0/1159134335 shutdown_connections 2024-01-11T07:07:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.002+0000 7f5271363700 1 --2- 172.21.15.7:0/1159134335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15e450 0x7f526c159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.002+0000 7f5271363700 1 --2- 172.21.15.7:0/1159134335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0cdaa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.002+0000 7f5271363700 1 -- 172.21.15.7:0/1159134335 >> 172.21.15.7:0/1159134335 conn(0x7f526c071120 msgr2=0x7f526c06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.002+0000 7f5271363700 1 -- 172.21.15.7:0/1159134335 shutdown_connections 2024-01-11T07:07:09.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.002+0000 7f5271363700 1 -- 172.21.15.7:0/1159134335 wait complete. 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.003+0000 7f5271363700 1 Processor -- start 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.003+0000 7f5271363700 1 -- start start 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f5271363700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f5271363700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15d8a0 0x7f526c0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f5271363700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526c15e450 0x7f526c14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f5271363700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c0c84b0 con 0x7f526c0cd6d0 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f5271363700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c0b5970 con 0x7f526c15e450 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f5271363700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526c0b5c50 con 0x7f526c15d8a0 2024-01-11T07:07:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41232/0 (socket says 172.21.15.7:41232) 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 -- 172.21.15.7:0/2954429245 learned_addr learned my addr 172.21.15.7:0/2954429245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526a7fc700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15d8a0 0x7f526c0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526b7fe700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526c15e450 0x7f526c14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 -- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15d8a0 msgr2=0x7f526c0c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15d8a0 0x7f526c0c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 -- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526c15e450 msgr2=0x7f526c14c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526c15e450 0x7f526c14c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 -- 172.21.15.7:0/2954429245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0094f0 con 0x7f526c0cd6d0 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f526affd700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0b5390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f526c063db0 tx=0x7f525c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.004+0000 7f5262ffd700 1 -- 172.21.15.7:0/2954429245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c000d10 con 0x7f526c0cd6d0 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.005+0000 7f5262ffd700 1 -- 172.21.15.7:0/2954429245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c00bbe0 con 0x7f526c0cd6d0 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.005+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526c0b5f30 con 0x7f526c0cd6d0 2024-01-11T07:07:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.005+0000 7f5262ffd700 1 -- 172.21.15.7:0/2954429245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c011690 con 0x7f526c0cd6d0 2024-01-11T07:07:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.005+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c169c70 con 0x7f526c0cd6d0 2024-01-11T07:07:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.006+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526c169e60 con 0x7f526c0cd6d0 2024-01-11T07:07:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.006+0000 7f5262ffd700 1 -- 172.21.15.7:0/2954429245 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f525c011ac0 con 0x7f526c0cd6d0 2024-01-11T07:07:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.006+0000 7f5262ffd700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.007+0000 7f5262ffd700 1 -- 172.21.15.7:0/2954429245 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f525c085e20 con 0x7f526c0cd6d0 2024-01-11T07:07:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.007+0000 7f526a7fc700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.007+0000 7f526a7fc700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f526c06eb10 tx=0x7f5254007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.011+0000 7f5262ffd700 1 -- 172.21.15.7:0/2954429245 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f525c053090 con 0x7f526c0cd6d0 2024-01-11T07:07:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.217+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f526c169ff0 con 0x7f526c0cd6d0 2024-01-11T07:07:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.218+0000 7f5262ffd700 1 -- 172.21.15.7:0/2954429245 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f525c0566b0 con 0x7f526c0cd6d0 2024-01-11T07:07:09.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:09.218 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 msgr2=0x7f524c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f526c06eb10 tx=0x7f5254007d30 comp rx=0 tx=0).stop 2024-01-11T07:07:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 msgr2=0x7f526c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0b5390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f526c063db0 tx=0x7f525c017000 comp rx=0 tx=0).stop 2024-01-11T07:07:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 shutdown_connections 2024-01-11T07:07:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f526c15e450 0x7f526c14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f526c0cd6d0 0x7f526c0b5390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 --2- 172.21.15.7:0/2954429245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f526c15d8a0 0x7f526c0c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 >> 172.21.15.7:0/2954429245 conn(0x7f526c071120 msgr2=0x7f526c15b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 shutdown_connections 2024-01-11T07:07:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.220+0000 7f5271363700 1 -- 172.21.15.7:0/2954429245 wait complete. 2024-01-11T07:07:09.233 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-11T07:07:09.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.366+0000 7f3e1dc47700 1 Processor -- start 2024-01-11T07:07:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.366+0000 7f3e1dc47700 1 -- start start 2024-01-11T07:07:09.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e1dc47700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e181579b0 0x7f3e1815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e1dc47700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e180aeab0 con 0x7f3e18063940 2024-01-11T07:07:09.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e1dc47700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e180aed90 con 0x7f3e18157d80 2024-01-11T07:07:09.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e1dc47700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e180af070 con 0x7f3e181579b0 2024-01-11T07:07:09.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e177fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e181579b0 0x7f3e1815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e17fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3e18063940 0x7f3e18069c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40146/0 (socket says 172.21.15.7:40146) 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e17fff700 1 -- 172.21.15.7:0/1513503686 learned_addr learned my addr 172.21.15.7:0/1513503686 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e167fc700 1 -- 172.21.15.7:0/1513503686 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372033689 0 0) 0x7f3e180aeab0 con 0x7f3e18063940 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e167fc700 1 -- 172.21.15.7:0/1513503686 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e00003570 con 0x7f3e18063940 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e177fe700 1 -- 172.21.15.7:0/1513503686 >> v1:172.21.15.113:6789/0 conn(0x7f3e18157d80 legacy=0x7f3e18063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e177fe700 1 -- 172.21.15.7:0/1513503686 >> v1:172.21.15.7:6789/0 conn(0x7f3e18063940 legacy=0x7f3e18069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.367+0000 7f3e177fe700 1 -- 172.21.15.7:0/1513503686 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e180af350 con 0x7f3e181579b0 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e177fe700 1 --2- 172.21.15.7:0/1513503686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e181579b0 0x7f3e1815bf50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e080098e0 tx=0x7f3e080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409e3ed1a7e4a5f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e167fc700 1 -- 172.21.15.7:0/1513503686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800b640 con 0x7f3e181579b0 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e167fc700 1 -- 172.21.15.7:0/1513503686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0800b7a0 con 0x7f3e181579b0 2024-01-11T07:07:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e167fc700 1 -- 172.21.15.7:0/1513503686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800bb40 con 0x7f3e181579b0 2024-01-11T07:07:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1513503686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e181579b0 msgr2=0x7f3e1815bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e1dc47700 1 --2- 172.21.15.7:0/1513503686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e181579b0 0x7f3e1815bf50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e080098e0 tx=0x7f3e080091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1513503686 shutdown_connections 2024-01-11T07:07:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e1dc47700 1 --2- 172.21.15.7:0/1513503686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e181579b0 0x7f3e1815bf50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1513503686 >> 172.21.15.7:0/1513503686 conn(0x7f3e180680e0 msgr2=0x7f3e180cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.368+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1513503686 shutdown_connections 2024-01-11T07:07:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.369+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1513503686 wait complete. 2024-01-11T07:07:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.370+0000 7f3e1dc47700 1 Processor -- start 2024-01-11T07:07:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.370+0000 7f3e1dc47700 1 -- start start 2024-01-11T07:07:09.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e1dc47700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e18063940 0x7f3e1815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e1dc47700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e181579b0 0x7f3e1815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e1dc47700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e18157d80 0x7f3e18168390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e1dc47700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180c3160 con 0x7f3e18063940 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e1dc47700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180c3440 con 0x7f3e181579b0 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e1dc47700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180c3720 con 0x7f3e18157d80 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e18063940 0x7f3e1815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e18063940 0x7f3e1815bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41244/0 (socket says 172.21.15.7:41244) 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 -- 172.21.15.7:0/1642422169 learned_addr learned my addr 172.21.15.7:0/1642422169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e16ffd700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e181579b0 0x7f3e1815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e17fff700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e18157d80 0x7f3e18168390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 -- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e18157d80 msgr2=0x7f3e18168390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e18157d80 0x7f3e18168390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 -- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e181579b0 msgr2=0x7f3e1815c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e181579b0 0x7f3e1815c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 -- 172.21.15.7:0/1642422169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08009420 con 0x7f3e18063940 2024-01-11T07:07:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e17fff700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e18157d80 0x7f3e18168390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e177fe700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e18063940 0x7f3e1815bf50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e080097b0 tx=0x7f3e08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3dff7fe700 1 -- 172.21.15.7:0/1642422169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08000d10 con 0x7f3e18063940 2024-01-11T07:07:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3dff7fe700 1 -- 172.21.15.7:0/1642422169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0800bb20 con 0x7f3e18063940 2024-01-11T07:07:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e180c3a00 con 0x7f3e18063940 2024-01-11T07:07:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.371+0000 7f3dff7fe700 1 -- 172.21.15.7:0/1642422169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08011690 con 0x7f3e18063940 2024-01-11T07:07:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.372+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e181688d0 con 0x7f3e18063940 2024-01-11T07:07:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.373+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1804c3c0 con 0x7f3e18063940 2024-01-11T07:07:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.373+0000 7f3dff7fe700 1 -- 172.21.15.7:0/1642422169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3e08011a40 con 0x7f3e18063940 2024-01-11T07:07:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.373+0000 7f3dff7fe700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e04066f30 0x7f3e040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.373+0000 7f3dff7fe700 1 -- 172.21.15.7:0/1642422169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e08085fd0 con 0x7f3e18063940 2024-01-11T07:07:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.374+0000 7f3e16ffd700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e04066f30 0x7f3e040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.374+0000 7f3e16ffd700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e04066f30 0x7f3e040693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e180c4080 tx=0x7f3e0c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.378+0000 7f3dff7fe700 1 -- 172.21.15.7:0/1642422169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e08053240 con 0x7f3e18063940 2024-01-11T07:07:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.584+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e180af630 con 0x7f3e18063940 2024-01-11T07:07:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.585+0000 7f3dff7fe700 1 -- 172.21.15.7:0/1642422169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f3e08056860 con 0x7f3e18063940 2024-01-11T07:07:09.585 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:09.585 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e04066f30 msgr2=0x7f3e040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e04066f30 0x7f3e040693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e180c4080 tx=0x7f3e0c007ec0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e18063940 msgr2=0x7f3e1815bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e18063940 0x7f3e1815bf50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e080097b0 tx=0x7f3e08017000 comp rx=0 tx=0).stop 2024-01-11T07:07:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 shutdown_connections 2024-01-11T07:07:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e04066f30 0x7f3e040693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e18157d80 0x7f3e18168390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e18063940 0x7f3e1815bf50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 --2- 172.21.15.7:0/1642422169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e181579b0 0x7f3e1815c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 >> 172.21.15.7:0/1642422169 conn(0x7f3e180680e0 msgr2=0x7f3e1806de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.587+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 shutdown_connections 2024-01-11T07:07:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.588+0000 7f3e1dc47700 1 -- 172.21.15.7:0/1642422169 wait complete. 2024-01-11T07:07:09.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-11T07:07:09.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-11T07:07:09.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-11T07:07:09.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-11T07:07:09.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-11T07:07:09.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-11T07:07:09.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-11T07:07:09.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-11T07:07:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.832+0000 7fd41af53700 1 Processor -- start 2024-01-11T07:07:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.832+0000 7fd41af53700 1 -- start start 2024-01-11T07:07:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd41af53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4140c6f30 0x7fd414155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd41af53700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414155d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd41af53700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4140ca3a0 con 0x7fd4140cba20 2024-01-11T07:07:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd41af53700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4141562b0 con 0x7fd4140c6f30 2024-01-11T07:07:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd41af53700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140cb200 con 0x7fd4140cbdf0 2024-01-11T07:07:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd4194f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd4194f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:38920/0 (socket says 172.21.15.7:38920) 2024-01-11T07:07:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd4194f0700 1 -- 172.21.15.7:0/1386705609 learned_addr learned my addr 172.21.15.7:0/1386705609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd413fff700 1 --2- 172.21.15.7:0/1386705609 >> v2:172.21.15.7:3300/0 conn(0x7fd4140c6f30 0x7fd414155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd4194f0700 1 -- 172.21.15.7:0/1386705609 >> v2:172.21.15.7:3300/0 conn(0x7fd4140c6f30 msgr2=0x7fd414155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.834+0000 7fd4194f0700 1 --2- 172.21.15.7:0/1386705609 >> v2:172.21.15.7:3300/0 conn(0x7fd4140c6f30 0x7fd414155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.835+0000 7fd4194f0700 1 -- 172.21.15.7:0/1386705609 >> v1:172.21.15.113:6789/0 conn(0x7fd4140cba20 legacy=0x7fd41415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.835+0000 7fd4194f0700 1 -- 172.21.15.7:0/1386705609 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140ca5f0 con 0x7fd4140cbdf0 2024-01-11T07:07:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.835+0000 7fd4194f0700 1 --2- 172.21.15.7:0/1386705609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414155d70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd40c007a70 tx=0x7fd40c00ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7f4948e94c4c7b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.835+0000 7fd4137fe700 1 -- 172.21.15.7:0/1386705609 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00a2e0 con 0x7fd4140cbdf0 2024-01-11T07:07:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.835+0000 7fd4137fe700 1 -- 172.21.15.7:0/1386705609 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c00a440 con 0x7fd4140cbdf0 2024-01-11T07:07:09.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.835+0000 7fd4137fe700 1 -- 172.21.15.7:0/1386705609 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00a800 con 0x7fd4140cbdf0 2024-01-11T07:07:09.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.836+0000 7fd41af53700 1 -- 172.21.15.7:0/1386705609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 msgr2=0x7fd414155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.836+0000 7fd41af53700 1 --2- 172.21.15.7:0/1386705609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414155d70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd40c007a70 tx=0x7fd40c00ad00 comp rx=0 tx=0).stop 2024-01-11T07:07:09.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.837+0000 7fd41af53700 1 -- 172.21.15.7:0/1386705609 shutdown_connections 2024-01-11T07:07:09.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.837+0000 7fd41af53700 1 --2- 172.21.15.7:0/1386705609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414155d70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.837+0000 7fd41af53700 1 --2- 172.21.15.7:0/1386705609 >> v2:172.21.15.7:3300/0 conn(0x7fd4140c6f30 0x7fd414155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.837+0000 7fd41af53700 1 -- 172.21.15.7:0/1386705609 >> 172.21.15.7:0/1386705609 conn(0x7fd414063230 msgr2=0x7fd414063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.837+0000 7fd41af53700 1 -- 172.21.15.7:0/1386705609 shutdown_connections 2024-01-11T07:07:09.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.837+0000 7fd41af53700 1 -- 172.21.15.7:0/1386705609 wait complete. 2024-01-11T07:07:09.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.839+0000 7fa7015c4700 1 Processor -- start 2024-01-11T07:07:09.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.839+0000 7fa7015c4700 1 -- start start 2024-01-11T07:07:09.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.839+0000 7fd41af53700 1 Processor -- start 2024-01-11T07:07:09.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.840+0000 7fd41af53700 1 -- start start 2024-01-11T07:07:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.840+0000 7fa7015c4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc0693a0 0x7fa6fc06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.840+0000 7fa7015c4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6fc06f1d0 0x7fa6fc0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.840+0000 7fa7015c4700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc15dbb0 con 0x7fa6fc159d10 2024-01-11T07:07:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.840+0000 7fa7015c4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc15d9a0 con 0x7fa6fc06f1d0 2024-01-11T07:07:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.840+0000 7fa7015c4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc06f650 con 0x7fa6fc0693a0 2024-01-11T07:07:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.840+0000 7fa6faffd700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa6fc159d10 0x7fa6fc068c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40262/0 (socket says 172.21.15.7:40262) 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.840+0000 7fa6faffd700 1 -- 172.21.15.7:0/2692788358 learned_addr learned my addr 172.21.15.7:0/2692788358 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6fa7fc700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc0693a0 0x7fa6fc06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/2692788358 >> v2:172.21.15.7:3300/0 conn(0x7fa6fc06f1d0 0x7fa6fc0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541768832 0 0) 0x7fa6fc15dbb0 con 0x7fa6fc159d10 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e4003570 con 0x7fa6fc159d10 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd41af53700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4140c6f30 0x7fd4140c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd41af53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4140cba20 0x7fd4140c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd41af53700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd41af53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd414159920 con 0x7fd4140cba20 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd41af53700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd414159c00 con 0x7fd4140c6f30 2024-01-11T07:07:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd41af53700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140b1890 con 0x7fd4140cbdf0 2024-01-11T07:07:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 777900562 0 0) 0x7fa6e4003570 con 0x7fa6fc159d10 2024-01-11T07:07:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc0693a0 msgr2=0x7fa6fc06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc0693a0 0x7fa6fc06ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 >> v2:172.21.15.7:3300/0 conn(0x7fa6fc06f1d0 msgr2=0x7fa6fc0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 --2- 172.21.15.7:0/2692788358 >> v2:172.21.15.7:3300/0 conn(0x7fa6fc06f1d0 0x7fa6fc0b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd4194f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd4194f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:38938/0 (socket says 172.21.15.7:38938) 2024-01-11T07:07:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd4194f0700 1 -- 172.21.15.7:0/78912327 learned_addr learned my addr 172.21.15.7:0/78912327 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd418cef700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4140c6f30 0x7fd4140c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fd413fff700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4140cba20 0x7fd4140c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc15de90 con 0x7fa6fc159d10 2024-01-11T07:07:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa6ec001470 con 0x7fa6fc159d10 2024-01-11T07:07:09.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 >> v1:172.21.15.113:6789/0 conn(0x7fa6fc159d10 legacy=0x7fa6fc068c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e4003ae0 0x7fa6e4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e4004810 0x7fa6e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc15d9a0 con 0x7fa6e4003ae0 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc06f650 con 0x7fa6e40051c0 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.841+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400c020 con 0x7fa6e4004810 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fa7fc700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc0693a0 0x7fa6fc06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fa7fc700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e4004810 0x7fa6e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd413fff700 1 -- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 msgr2=0x7fd414150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd413fff700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4140cbdf0 0x7fd414150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd413fff700 1 -- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4140c6f30 msgr2=0x7fd4140c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd413fff700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4140c6f30 0x7fd4140c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd413fff700 1 -- 172.21.15.7:0/78912327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c00be70 con 0x7fd4140cba20 2024-01-11T07:07:09.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6faffd700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e4003ae0 0x7fa6e4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fb7fe700 1 -- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e4004810 msgr2=0x7fa6e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6e4004810 0x7fa6e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fb7fe700 1 -- 172.21.15.7:0/2692788358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e4003ae0 msgr2=0x7fa6e4003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6e4003ae0 0x7fa6e4003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fb7fe700 1 -- 172.21.15.7:0/2692788358 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc15de90 con 0x7fa6e40051c0 2024-01-11T07:07:09.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd413fff700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4140cba20 0x7fd4140c3ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d860 tx=0x7fd408008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e240 tx=0x7fa6f400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c3567b5db8efb9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.843+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400eaf0 con 0x7fa6e40051c0 2024-01-11T07:07:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.843+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f400f040 con 0x7fa6e40051c0 2024-01-11T07:07:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.843+0000 7fa6f9ffb700 1 -- 172.21.15.7:0/2692788358 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400eea0 con 0x7fa6e40051c0 2024-01-11T07:07:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080125f0 con 0x7fd4140cba20 2024-01-11T07:07:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd408012750 con 0x7fd4140cba20 2024-01-11T07:07:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.842+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408012af0 con 0x7fd4140cba20 2024-01-11T07:07:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.843+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4140b1b70 con 0x7fd4140cba20 2024-01-11T07:07:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.843+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4140c49e0 con 0x7fd4140cba20 2024-01-11T07:07:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.844+0000 7fa7015c4700 1 -- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 msgr2=0x7fa6e400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.844+0000 7fa7015c4700 1 --2- 172.21.15.7:0/2692788358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 0x7fa6e400ba60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e240 tx=0x7fa6f400b040 comp rx=0 tx=0).stop 2024-01-11T07:07:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.844+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd41404c3c0 con 0x7fd4140cba20 2024-01-11T07:07:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.844+0000 7fa6fb7fe700 1 -- 172.21.15.7:0/2692788358 reap_dead start 2024-01-11T07:07:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.844+0000 7fa7015c4700 1 -- 172.21.15.7:0/2692788358 shutdown_connections 2024-01-11T07:07:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.844+0000 7fa7015c4700 1 -- 172.21.15.7:0/2692788358 >> 172.21.15.7:0/2692788358 conn(0x7fa6fc0ce9f0 msgr2=0x7fa6fc0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.844+0000 7fb57bae2700 1 Processor -- start 2024-01-11T07:07:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.845+0000 7fb57bae2700 1 -- start start 2024-01-11T07:07:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.845+0000 7f1eb5388700 1 Processor -- start 2024-01-11T07:07:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.845+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd408013480 con 0x7fd4140cba20 2024-01-11T07:07:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.845+0000 7fd410ff9700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3f8066fd0 0x7fd3f8069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.845+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd408084760 con 0x7fd4140cba20 2024-01-11T07:07:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fd410ff9700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fd3f806f1e0 0x7fd3f80715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 --> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3f8071ca0 con 0x7fd3f806f1e0 2024-01-11T07:07:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd408086c60 con 0x7fd4140cba20 2024-01-11T07:07:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fd4194f0700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fd3f806f1e0 0x7fd3f80715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fb57bae2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5740cd6d0 0x7fb5740cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fb57bae2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb57415e450 0x7fb574159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7f70fc756700 1 Processor -- start 2024-01-11T07:07:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fb57bae2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57415acf0 con 0x7fb57415d8a0 2024-01-11T07:07:09.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fb57bae2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57415afd0 con 0x7fb57415e450 2024-01-11T07:07:09.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fb57bae2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740ce980 con 0x7fb5740cd6d0 2024-01-11T07:07:09.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7f70fc756700 1 -- start start 2024-01-11T07:07:09.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.845+0000 7f1eb5388700 1 -- start start 2024-01-11T07:07:09.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eb5388700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1eb00cba20 0x7f1eb015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eb5388700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb00ca3a0 con 0x7f1eb0155f40 2024-01-11T07:07:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eb5388700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb00cb200 con 0x7f1eb00c6f30 2024-01-11T07:07:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eb5388700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb015a8e0 con 0x7f1eb00cba20 2024-01-11T07:07:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.846+0000 7fa7015c4700 1 -- 172.21.15.7:0/2692788358 shutdown_connections 2024-01-11T07:07:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eae7fc700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1eb00c6f30 0x7f1eb0155830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40272/0 (socket says 172.21.15.7:40272) 2024-01-11T07:07:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eae7fc700 1 -- 172.21.15.7:0/2991789257 learned_addr learned my addr 172.21.15.7:0/2991789257 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7fd4194f0700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fd3f806f1e0 0x7fd3f80715f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eaeffd700 1 --2- 172.21.15.7:0/2991789257 >> v2:172.21.15.113:3300/0 conn(0x7f1eb00cba20 0x7f1eb015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039523401 0 0) 0x7f1eb00cb200 con 0x7f1eb00c6f30 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e98004680 con 0x7f1eb00c6f30 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1151280566 0 0) 0x7f1e98004680 con 0x7f1eb00c6f30 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 >> v2:172.21.15.113:3300/0 conn(0x7f1eb00cba20 msgr2=0x7f1eb015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.847+0000 7f1eadffb700 1 --2- 172.21.15.7:0/2991789257 >> v2:172.21.15.113:3300/0 conn(0x7f1eb00cba20 0x7f1eb015a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 >> v1:172.21.15.7:6789/0 conn(0x7f1eb0155f40 legacy=0x7f1eb00c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00ca5f0 con 0x7f1eb00c6f30 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f1ea8004ea0 con 0x7f1eb00c6f30 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 >> v1:172.21.15.113:6789/0 conn(0x7f1eb00c6f30 legacy=0x7f1eb0155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98004b70 0x7f1e98004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e98005920 0x7f1e98005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e980062d0 0x7f1e98003490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb015a8e0 con 0x7f1e98004b70 2024-01-11T07:07:09.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980039d0 con 0x7f1e980062d0 2024-01-11T07:07:09.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e98003b90 con 0x7f1e98005920 2024-01-11T07:07:09.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eae7fc700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e98005920 0x7f1e98005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70fc756700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 0x7f70f4069e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70fc756700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70f40c7520 0x7f70f40cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70fc756700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f40699a0 con 0x7f70f40cadf0 2024-01-11T07:07:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70fc756700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f40cc2f0 con 0x7f70f40c7520 2024-01-11T07:07:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70fc756700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f4068d90 con 0x7f70f41560d0 2024-01-11T07:07:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70f9cf1700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f70f40cadf0 0x7f70f40c6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40280/0 (socket says 172.21.15.7:40280) 2024-01-11T07:07:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70f9cf1700 1 -- 172.21.15.7:0/1811578090 learned_addr learned my addr 172.21.15.7:0/1811578090 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eaeffd700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98004b70 0x7f1e98004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70fa4f2700 1 --2- 172.21.15.7:0/1811578090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 0x7f70f4069e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eae7fc700 1 -- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e980062d0 msgr2=0x7f1e98003490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T07:07:09.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eae7fc700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e980062d0 0x7f1e98003490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eae7fc700 1 -- 172.21.15.7:0/2991789257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98004b70 msgr2=0x7f1e98004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eae7fc700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98004b70 0x7f1e98004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f1eae7fc700 1 -- 172.21.15.7:0/2991789257 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00ca5f0 con 0x7f1e98005920 2024-01-11T07:07:09.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7f70facf3700 1 --2- 172.21.15.7:0/1811578090 >> v2:172.21.15.113:3300/0 conn(0x7f70f40c7520 0x7f70f40cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7fb57a07f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb57415e450 0x7fb574159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7fb57a07f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb57415e450 0x7fb574159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41286/0 (socket says 172.21.15.7:41286) 2024-01-11T07:07:09.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7fb57a07f700 1 -- 172.21.15.7:0/1428436437 learned_addr learned my addr 172.21.15.7:0/1428436437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7fb57907d700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb57415d8a0 0x7fb57415dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40286/0 (socket says 172.21.15.7:40286) 2024-01-11T07:07:09.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.848+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 <== osd.1 v2:172.21.15.7:6808/36943 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fd3f8071ca0 con 0x7fd3f806f1e0 2024-01-11T07:07:09.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f1eae7fc700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e98005920 0x7f1e98005d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8001910 tx=0x7f1ea80089b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c16df4992e02a54 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f1eaf7fe700 1 -- 172.21.15.7:0/2991789257 reap_dead start 2024-01-11T07:07:09.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8008ce0 con 0x7f1e98005920 2024-01-11T07:07:09.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea8008110 con 0x7f1e98005920 2024-01-11T07:07:09.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f1eadffb700 1 -- 172.21.15.7:0/2991789257 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8006530 con 0x7f1e98005920 2024-01-11T07:07:09.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fb57987e700 1 --2- 172.21.15.7:0/1428436437 >> v2:172.21.15.113:3300/0 conn(0x7fb5740cd6d0 0x7fb5740cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fd418cef700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3f8066fd0 0x7fd3f8069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fa7015c4700 1 -- 172.21.15.7:0/2692788358 wait complete. 2024-01-11T07:07:09.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7f1eb5388700 1 -- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e98005920 msgr2=0x7f1e98005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7f1eb5388700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e98005920 0x7f1e98005d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8001910 tx=0x7f1ea80089b0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f70facf3700 1 -- 172.21.15.7:0/1811578090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 msgr2=0x7f70f4069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f70facf3700 1 --2- 172.21.15.7:0/1811578090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 0x7f70f4069e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f70facf3700 1 -- 172.21.15.7:0/1811578090 >> v1:172.21.15.113:6789/0 conn(0x7f70f40cadf0 legacy=0x7f70f40c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f70facf3700 1 -- 172.21.15.7:0/1811578090 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f4069070 con 0x7f70f40c7520 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.849+0000 7f70facf3700 1 --2- 172.21.15.7:0/1811578090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40c7520 0x7f70f40cb260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009e60 tx=0x7f70ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=613f2243eaca1e01 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7f70f94f0700 1 -- 172.21.15.7:0/1811578090 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00ba60 con 0x7f70f40c7520 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7f70f94f0700 1 -- 172.21.15.7:0/1811578090 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec00bbc0 con 0x7f70f40c7520 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7f70f94f0700 1 -- 172.21.15.7:0/1811578090 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0185c0 con 0x7f70f40c7520 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7f1e63f4a700 1 Processor -- start 2024-01-11T07:07:09.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7f1e63f4a700 1 -- start start 2024-01-11T07:07:09.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.851+0000 7f70fc756700 1 -- 172.21.15.7:0/1811578090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40c7520 msgr2=0x7f70f40cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.851+0000 7f70fc756700 1 --2- 172.21.15.7:0/1811578090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40c7520 0x7f70f40cb260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009e60 tx=0x7f70ec009780 comp rx=0 tx=0).stop 2024-01-11T07:07:09.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fb57887c700 1 -- 172.21.15.7:0/1428436437 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687357362 0 0) 0x7fb57415acf0 con 0x7fb57415d8a0 2024-01-11T07:07:09.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fb57887c700 1 -- 172.21.15.7:0/1428436437 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb560003570 con 0x7fb57415d8a0 2024-01-11T07:07:09.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fb57a07f700 1 -- 172.21.15.7:0/1428436437 >> v2:172.21.15.113:3300/0 conn(0x7fb5740cd6d0 msgr2=0x7fb5740cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fb57a07f700 1 --2- 172.21.15.7:0/1428436437 >> v2:172.21.15.113:3300/0 conn(0x7fb5740cd6d0 0x7fb5740cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fb57a07f700 1 -- 172.21.15.7:0/1428436437 >> v1:172.21.15.113:6789/0 conn(0x7fb57415d8a0 legacy=0x7fb57415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.850+0000 7fb57a07f700 1 -- 172.21.15.7:0/1428436437 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740cec60 con 0x7fb57415e450 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.851+0000 7fb57a07f700 1 --2- 172.21.15.7:0/1428436437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415e450 0x7fb574159600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb5740677c0 tx=0x7fb56c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49639d45307df7a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.851+0000 7fb57887c700 1 -- 172.21.15.7:0/1428436437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00e070 con 0x7fb57415e450 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.851+0000 7fb57887c700 1 -- 172.21.15.7:0/1428436437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c00b930 con 0x7fb57415e450 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.851+0000 7fb57887c700 1 -- 172.21.15.7:0/1428436437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0092e0 con 0x7fb57415e450 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1eb5388700 1 -- 172.21.15.7:0/2991789257 shutdown_connections 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1eb5388700 1 --2- 172.21.15.7:0/2991789257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e98005920 0x7f1e98005d90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1eb5388700 1 -- 172.21.15.7:0/2991789257 >> 172.21.15.7:0/2991789257 conn(0x7f1eb0063230 msgr2=0x7f1eb0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1e63f4a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e5c0c6f30 0x7f1e5c15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1e63f4a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0ca3a0 con 0x7f1e5c0cbee0 2024-01-11T07:07:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1e63f4a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e5c0cb200 con 0x7f1e5c063d40 2024-01-11T07:07:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1e63f4a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5c15a8e0 con 0x7f1e5c0c6f30 2024-01-11T07:07:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7fa7015c4700 1 Processor -- start 2024-01-11T07:07:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7fb57bae2700 1 -- 172.21.15.7:0/1428436437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415e450 msgr2=0x7fb574159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7fb57bae2700 1 --2- 172.21.15.7:0/1428436437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415e450 0x7fb574159600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb5740677c0 tx=0x7fb56c009840 comp rx=0 tx=0).stop 2024-01-11T07:07:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1e61ce6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e5c0c6f30 0x7f1e5c15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1e61ce6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e5c0c6f30 0x7f1e5c15a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36222/0 (socket says 172.21.15.7:36222) 2024-01-11T07:07:09.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f1e61ce6700 1 -- 172.21.15.7:0/1047967502 learned_addr learned my addr 172.21.15.7:0/1047967502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fb57bae2700 1 -- 172.21.15.7:0/1428436437 shutdown_connections 2024-01-11T07:07:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fb57bae2700 1 --2- 172.21.15.7:0/1428436437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415e450 0x7fb574159600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fb57bae2700 1 --2- 172.21.15.7:0/1428436437 >> v2:172.21.15.113:3300/0 conn(0x7fb5740cd6d0 0x7fb5740cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fb57bae2700 1 -- 172.21.15.7:0/1428436437 >> 172.21.15.7:0/1428436437 conn(0x7fb574071120 msgr2=0x7fb57406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fb57bae2700 1 -- 172.21.15.7:0/1428436437 shutdown_connections 2024-01-11T07:07:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fb57bae2700 1 -- 172.21.15.7:0/1428436437 wait complete. 2024-01-11T07:07:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f665e722700 1 Processor -- start 2024-01-11T07:07:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7f665e722700 1 -- start start 2024-01-11T07:07:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7f1e61ce6700 1 -- 172.21.15.7:0/1047967502 >> v1:172.21.15.113:6789/0 conn(0x7f1e5c063d40 legacy=0x7f1e5c0cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7f1e61ce6700 1 -- 172.21.15.7:0/1047967502 >> v1:172.21.15.7:6789/0 conn(0x7f1e5c0cbee0 legacy=0x7f1e5c155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7f1e61ce6700 1 -- 172.21.15.7:0/1047967502 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c0ca8d0 con 0x7f1e5c0c6f30 2024-01-11T07:07:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7f1e61ce6700 1 --2- 172.21.15.7:0/1047967502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0c6f30 0x7f1e5c15a3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e580099b0 tx=0x7f1e58009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c641c968fb7689 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7f665e722700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f665806cb60 0x7f6658069300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f665e722700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66580ca540 con 0x7f665806e800 2024-01-11T07:07:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f665e722700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66580ca820 con 0x7f66580c85f0 2024-01-11T07:07:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f665e722700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66580cb1c0 con 0x7f665806cb60 2024-01-11T07:07:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f6657fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f665806e800 0x7f6658068bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40180/0 (socket says 172.21.15.7:40180) 2024-01-11T07:07:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f6657fff700 1 -- 172.21.15.7:0/2512125499 learned_addr learned my addr 172.21.15.7:0/2512125499 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f66577fe700 1 --2- 172.21.15.7:0/2512125499 >> v2:172.21.15.113:3300/0 conn(0x7f665806cb60 0x7f6658069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f66577fe700 1 -- 172.21.15.7:0/2512125499 >> v1:172.21.15.113:6789/0 conn(0x7f66580c85f0 legacy=0x7f66580c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f66577fe700 1 -- 172.21.15.7:0/2512125499 >> v1:172.21.15.7:6789/0 conn(0x7f665806e800 legacy=0x7f6658068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f66577fe700 1 -- 172.21.15.7:0/2512125499 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66580cab00 con 0x7f665806cb60 2024-01-11T07:07:09.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.852+0000 7fa7015c4700 1 -- start start 2024-01-11T07:07:09.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa7015c4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 0x7fa6fc0c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa7015c4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc159010 0x7fa6fc151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa7015c4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6fc0cedf0 0x7fa6fc0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa7015c4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0c3470 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa7015c4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cf7b0 con 0x7fa6e40051c0 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa7015c4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc16ee90 con 0x7fa6fc159010 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa6fa7fc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc159010 0x7fa6fc151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa6faffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 0x7fa6fc0c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa6faffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 0x7fa6fc0c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36224/0 (socket says 172.21.15.7:36224) 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa6faffd700 1 -- 172.21.15.7:0/726738192 learned_addr learned my addr 172.21.15.7:0/726738192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.853+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6fc0cedf0 0x7fa6fc0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7fb57bae2700 1 Processor -- start 2024-01-11T07:07:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1eb5388700 1 -- 172.21.15.7:0/2991789257 shutdown_connections 2024-01-11T07:07:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f1e60ce4700 1 -- 172.21.15.7:0/1047967502 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800b8b0 con 0x7f1e5c0c6f30 2024-01-11T07:07:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f1e60ce4700 1 -- 172.21.15.7:0/1047967502 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5800ba10 con 0x7f1e5c0c6f30 2024-01-11T07:07:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7f1e60ce4700 1 -- 172.21.15.7:0/1047967502 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800bd90 con 0x7f1e5c0c6f30 2024-01-11T07:07:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1e63f4a700 1 -- 172.21.15.7:0/1047967502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0c6f30 msgr2=0x7f1e5c15a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1e63f4a700 1 --2- 172.21.15.7:0/1047967502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0c6f30 0x7f1e5c15a3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e580099b0 tx=0x7f1e58009250 comp rx=0 tx=0).stop 2024-01-11T07:07:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7fa6fb7fe700 1 -- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc159010 msgr2=0x7fa6fc151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6fc159010 0x7fa6fc151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7fa6fb7fe700 1 -- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 msgr2=0x7fa6fc0c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.854+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6e40051c0 0x7fa6fc0c2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1e63f4a700 1 -- 172.21.15.7:0/1047967502 shutdown_connections 2024-01-11T07:07:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1e63f4a700 1 --2- 172.21.15.7:0/1047967502 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0c6f30 0x7f1e5c15a3a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1e63f4a700 1 -- 172.21.15.7:0/1047967502 >> 172.21.15.7:0/1047967502 conn(0x7f1e5c063230 msgr2=0x7f1e5c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7fa6fb7fe700 1 -- 172.21.15.7:0/726738192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc16e2f0 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1e63f4a700 1 -- 172.21.15.7:0/1047967502 shutdown_connections 2024-01-11T07:07:09.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f66577fe700 1 --2- 172.21.15.7:0/2512125499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665806cb60 0x7f6658069300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f664c009860 tx=0x7f664c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3059c841edfc4a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1e63f4a700 1 -- 172.21.15.7:0/1047967502 wait complete. 2024-01-11T07:07:09.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f6656ffd700 1 -- 172.21.15.7:0/2512125499 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c011630 con 0x7f665806cb60 2024-01-11T07:07:09.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f6656ffd700 1 -- 172.21.15.7:0/2512125499 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c011790 con 0x7f665806cb60 2024-01-11T07:07:09.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f6656ffd700 1 -- 172.21.15.7:0/2512125499 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c011b30 con 0x7f665806cb60 2024-01-11T07:07:09.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fa6fb7fe700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6fc0cedf0 0x7fa6fc0cf270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e210 tx=0x7fa6f401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7fb57bae2700 1 -- start start 2024-01-11T07:07:09.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57bae2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5740cd6d0 0x7fb574157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57bae2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415d8a0 0x7fb5740b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57bae2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb57415e450 0x7fb5740b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57bae2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740b1690 con 0x7fb57415d8a0 2024-01-11T07:07:09.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57bae2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740b0a80 con 0x7fb5740cd6d0 2024-01-11T07:07:09.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57bae2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740b0d60 con 0x7fb57415e450 2024-01-11T07:07:09.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57907d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415d8a0 0x7fb5740b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57907d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415d8a0 0x7fb5740b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41316/0 (socket says 172.21.15.7:41316) 2024-01-11T07:07:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57907d700 1 -- 172.21.15.7:0/1462175126 learned_addr learned my addr 172.21.15.7:0/1462175126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57987e700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5740cd6d0 0x7fb574157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.856+0000 7fb57a07f700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb57415e450 0x7fb5740b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7f70fc756700 1 -- 172.21.15.7:0/1811578090 shutdown_connections 2024-01-11T07:07:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7f70fc756700 1 --2- 172.21.15.7:0/1811578090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40c7520 0x7f70f40cb260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7f70fc756700 1 --2- 172.21.15.7:0/1811578090 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 0x7f70f4069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7f70fc756700 1 -- 172.21.15.7:0/1811578090 >> 172.21.15.7:0/1811578090 conn(0x7f70f4063230 msgr2=0x7f70f4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fb57907d700 1 -- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb57415e450 msgr2=0x7fb5740b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fb57907d700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb57415e450 0x7fb5740b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fb57907d700 1 -- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5740cd6d0 msgr2=0x7fb574157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fb57907d700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5740cd6d0 0x7fb574157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fb57907d700 1 -- 172.21.15.7:0/1462175126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c009ae0 con 0x7fb57415d8a0 2024-01-11T07:07:09.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.855+0000 7f1eb5388700 1 -- 172.21.15.7:0/2991789257 wait complete. 2024-01-11T07:07:09.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7f1eb5388700 1 Processor -- start 2024-01-11T07:07:09.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4014b70 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f4014cd0 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4048680 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7f1e63f4a700 1 Processor -- start 2024-01-11T07:07:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fb57907d700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415d8a0 0x7fb5740b48d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c810 tx=0x7fb564007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7f1e63f4a700 1 -- start start 2024-01-11T07:07:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1e63f4a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5c063d40 0x7f1e5c0c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1e63f4a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5c0c6f30 0x7f1e5c1765f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1e63f4a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0cbee0 0x7f1e5c144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1e63f4a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c06d750 con 0x7f1e5c063d40 2024-01-11T07:07:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1e63f4a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c06cb60 con 0x7f1e5c0cbee0 2024-01-11T07:07:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1e63f4a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c06ce20 con 0x7f1e5c0c6f30 2024-01-11T07:07:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7f1eb5388700 1 -- start start 2024-01-11T07:07:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1eb5388700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98005920 0x7f1eb00af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1eb5388700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb00c48b0 0x7f1eb00b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1eb5388700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eb00b1570 0x7f1eb00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1eb5388700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00afb10 con 0x7f1e98005920 2024-01-11T07:07:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1eb5388700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00b1f50 con 0x7f1eb00c48b0 2024-01-11T07:07:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7f1eb5388700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00c4e20 con 0x7f1eb00b1570 2024-01-11T07:07:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f665e722700 1 -- 172.21.15.7:0/2512125499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665806cb60 msgr2=0x7f6658069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f665e722700 1 --2- 172.21.15.7:0/2512125499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665806cb60 0x7f6658069300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f664c009860 tx=0x7f664c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eaf7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eb00b1570 0x7f1eb00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eaf7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eb00b1570 0x7f1eb00b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:38974/0 (socket says 172.21.15.7:38974) 2024-01-11T07:07:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eaf7fe700 1 -- 172.21.15.7:0/2804102289 learned_addr learned my addr 172.21.15.7:0/2804102289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eae7fc700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb00c48b0 0x7f1eb00b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f70fc756700 1 -- 172.21.15.7:0/1811578090 shutdown_connections 2024-01-11T07:07:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.857+0000 7fb569ffb700 1 -- 172.21.15.7:0/1462175126 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564005b40 con 0x7fb57415d8a0 2024-01-11T07:07:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7fb569ffb700 1 -- 172.21.15.7:0/1462175126 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb564005cc0 con 0x7fb57415d8a0 2024-01-11T07:07:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7fb569ffb700 1 -- 172.21.15.7:0/1462175126 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564007320 con 0x7fb57415d8a0 2024-01-11T07:07:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5740b1040 con 0x7fb57415d8a0 2024-01-11T07:07:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5741579b0 con 0x7fb57415d8a0 2024-01-11T07:07:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eaf7fe700 1 -- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb00c48b0 msgr2=0x7f1eb00b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eaf7fe700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb00c48b0 0x7f1eb00b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eaf7fe700 1 -- 172.21.15.7:0/2804102289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98005920 msgr2=0x7f1eb00af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eaf7fe700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98005920 0x7f1eb00af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f1eaf7fe700 1 -- 172.21.15.7:0/2804102289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00c5100 con 0x7f1eb00b1570 2024-01-11T07:07:09.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.858+0000 7fd418cef700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3f8066fd0 0x7fd3f8069480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd404003a80 tx=0x7fd404007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e624e7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0cbee0 0x7f1e5c144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e624e7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0cbee0 0x7f1e5c144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36270/0 (socket says 172.21.15.7:36270) 2024-01-11T07:07:09.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e624e7700 1 -- 172.21.15.7:0/902285917 learned_addr learned my addr 172.21.15.7:0/902285917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1eaf7fe700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eb00b1570 0x7f1eb00b1a10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1eb006c290 tx=0x7f1ea0007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc16e560 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc16eb20 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e61ce6700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5c063d40 0x7f1e5c0c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e61ce6700 1 -- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5c0c6f30 msgr2=0x7f1e5c1765f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T07:07:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e61ce6700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e5c0c6f30 0x7f1e5c1765f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e61ce6700 1 -- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0cbee0 msgr2=0x7f1e5c144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e61ce6700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e5c0cbee0 0x7f1e5c144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.860+0000 7f1e61ce6700 1 -- 172.21.15.7:0/902285917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e580094f0 con 0x7f1e5c063d40 2024-01-11T07:07:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7f1e61ce6700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5c063d40 0x7f1e5c0c2f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0cb170 tx=0x7f1e58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.859+0000 7f70fc756700 1 -- 172.21.15.7:0/1811578090 wait complete. 2024-01-11T07:07:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7f70fc756700 1 Processor -- start 2024-01-11T07:07:09.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7f1e51ffb700 1 -- 172.21.15.7:0/902285917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e58011e30 con 0x7f1e5c063d40 2024-01-11T07:07:09.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7f1e51ffb700 1 -- 172.21.15.7:0/902285917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e58020070 con 0x7f1e5c063d40 2024-01-11T07:07:09.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7f1e51ffb700 1 -- 172.21.15.7:0/902285917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5801b6c0 con 0x7f1e5c063d40 2024-01-11T07:07:09.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7f70fc756700 1 -- start start 2024-01-11T07:07:09.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c06d100 con 0x7f1e5c063d40 2024-01-11T07:07:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c0af5d0 con 0x7f1e5c063d40 2024-01-11T07:07:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6fc069770 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7fb569ffb700 1 -- 172.21.15.7:0/1462175126 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb56401b3b0 con 0x7fb57415d8a0 2024-01-11T07:07:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7fb569ffb700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb560067430 0x7fb5600698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7fb569ffb700 1 -- 172.21.15.7:0/1462175126 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb564083ad0 con 0x7fb57415d8a0 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7f70fc756700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70f40c7520 0x7f70f40ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7f70fc756700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40cadf0 0x7f70f40ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7f70fc756700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 0x7f70f40b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7f70fc756700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f416a050 con 0x7f70f40c7520 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7f70fc756700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40adce0 con 0x7f70f40cadf0 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.862+0000 7f70fc756700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40ac720 con 0x7f70f41560d0 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70f9cf1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40cadf0 0x7f70f40ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70f9cf1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40cadf0 0x7f70f40ad7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36280/0 (socket says 172.21.15.7:36280) 2024-01-11T07:07:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70f9cf1700 1 -- 172.21.15.7:0/1808784610 learned_addr learned my addr 172.21.15.7:0/1808784610 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f665e722700 1 -- 172.21.15.7:0/2512125499 shutdown_connections 2024-01-11T07:07:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f665e722700 1 --2- 172.21.15.7:0/2512125499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665806cb60 0x7f6658069300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f665e722700 1 -- 172.21.15.7:0/2512125499 >> 172.21.15.7:0/2512125499 conn(0x7f6658063230 msgr2=0x7f6658063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7fb57bae2700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7fb5540015e0 0x7fb554003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 --> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb554006b80 con 0x7fb5540015e0 2024-01-11T07:07:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70facf3700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 0x7f70f40b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7fb57987e700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb560067430 0x7fb5600698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.861+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000cee0 con 0x7f1eb00b1570 2024-01-11T07:07:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f665e722700 1 -- 172.21.15.7:0/2512125499 shutdown_connections 2024-01-11T07:07:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa6f4045070 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70fa4f2700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70f40c7520 0x7f70f40ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea00070d0 con 0x7f1eb00b1570 2024-01-11T07:07:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00074b0 con 0x7f1eb00b1570 2024-01-11T07:07:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70facf3700 1 -- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40cadf0 msgr2=0x7f70f40ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70facf3700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70f40cadf0 0x7f70f40ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70facf3700 1 -- 172.21.15.7:0/1808784610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70f40c7520 msgr2=0x7f70f40ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f70facf3700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70f40c7520 0x7f70f40ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f70facf3700 1 -- 172.21.15.7:0/1808784610 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec009a20 con 0x7f70f41560d0 2024-01-11T07:07:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7fa6e2ffd700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6e4068660 0x7fa6e406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa6f40a7e90 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7fa6e2ffd700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fa6e40707c0 0x7fa6e4072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 --> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6e4073280 con 0x7fa6e40707c0 2024-01-11T07:07:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f70facf3700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 0x7f70f40b1830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f70ec019000 tx=0x7f70ec01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7fb57987e700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb560067430 0x7fb5600698e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb570009db0 tx=0x7fb5700091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f730edc5700 1 Processor -- start 2024-01-11T07:07:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec018e90 con 0x7f70f41560d0 2024-01-11T07:07:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec011670 con 0x7f70f41560d0 2024-01-11T07:07:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec011a50 con 0x7f70f41560d0 2024-01-11T07:07:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb00c53e0 con 0x7f1eb00b1570 2024-01-11T07:07:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb0143690 con 0x7f1eb00b1570 2024-01-11T07:07:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6f4075160 con 0x7fa6fc0cedf0 2024-01-11T07:07:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1eb006ca60 con 0x7f1eb00b1570 2024-01-11T07:07:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.864+0000 7f1e51ffb700 1 -- 172.21.15.7:0/902285917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1e58022030 con 0x7f1e5c063d40 2024-01-11T07:07:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7f1e51ffb700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e40066fb0 0x7f1e40069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7f1e51ffb700 1 -- 172.21.15.7:0/902285917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e58086430 con 0x7f1e5c063d40 2024-01-11T07:07:09.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7fa6faffd700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6e4068660 0x7fa6e406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.863+0000 7f665e722700 1 -- 172.21.15.7:0/2512125499 wait complete. 2024-01-11T07:07:09.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7f665e722700 1 Processor -- start 2024-01-11T07:07:09.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f40aca00 con 0x7f70f41560d0 2024-01-11T07:07:09.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f40acfc0 con 0x7f70f41560d0 2024-01-11T07:07:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7f1e614e5700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e40066fb0 0x7f1e40069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.866+0000 7f1e614e5700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e40066fb0 0x7f1e40069460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c001050 tx=0x7f1e4c007ae0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.866+0000 7fb57a07f700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7fb5540015e0 0x7fb554003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.866+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1ea0005a20 con 0x7f1eb00b1570 2024-01-11T07:07:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7fa6fa7fc700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fa6e40707c0 0x7fa6e4072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70f404c3c0 con 0x7f70f41560d0 2024-01-11T07:07:09.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7fb57a07f700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7fb5540015e0 0x7fb554003a90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f1e96ffd700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e9c066f30 0x7f1e9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ea0083970 con 0x7f1eb00b1570 2024-01-11T07:07:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.865+0000 7f730edc5700 1 -- start start 2024-01-11T07:07:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f730edc5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7308155f40 0x7f7308160d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f730edc5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080ca3a0 con 0x7f73080c6f30 2024-01-11T07:07:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f730edc5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080cb200 con 0x7f73080cba20 2024-01-11T07:07:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f730edc5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73080ca5f0 con 0x7f7308155f40 2024-01-11T07:07:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f7307fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73080c6f30 0x7f7308155830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40186/0 (socket says 172.21.15.7:40186) 2024-01-11T07:07:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f7307fff700 1 -- 172.21.15.7:0/2733910617 learned_addr learned my addr 172.21.15.7:0/2733910617 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f1e96ffd700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f1e9c06f0c0 0x7f1e9c0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f1eaeffd700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e9c066f30 0x7f1e9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 --> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e9c071b80 con 0x7f1e9c06f0c0 2024-01-11T07:07:09.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea0083d40 con 0x7f1eb00b1570 2024-01-11T07:07:09.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7fa6fa7fc700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fa6e40707c0 0x7fa6e4072bd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f1e63f4a700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f1e5c0c6270 0x7f1e5c149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 --> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e5c0cabb0 con 0x7f1e5c0c6270 2024-01-11T07:07:09.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7fb569ffb700 1 -- 172.21.15.7:0/1462175126 <== osd.0 v2:172.21.15.7:6816/36945 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fb554006b80 con 0x7fb5540015e0 2024-01-11T07:07:09.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7fa6faffd700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6e4068660 0x7fa6e406ab10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec001010 tx=0x7fa6ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.870+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 <== osd.2 v2:172.21.15.7:6823/36946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fa6e4073280 con 0x7fa6e40707c0 2024-01-11T07:07:09.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f1eae7fc700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f1e9c06f0c0 0x7f1e9c0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f1eae7fc700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f1e9c06f0c0 0x7f1e9c0714d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 <== osd.3 v2:172.21.15.7:6800/36944 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f1e9c071b80 con 0x7f1e9c06f0c0 2024-01-11T07:07:09.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.870+0000 7f1eaeffd700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e9c066f30 0x7f1e9c0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006f00 tx=0x7f1ea40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f70ec017070 con 0x7f70f41560d0 2024-01-11T07:07:09.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f70ea7fc700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d8067080 0x7f70d8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70ec086920 con 0x7f70f41560d0 2024-01-11T07:07:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f70ea7fc700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f70d806f0a0 0x7f70d80714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 --> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70d8071b60 con 0x7f70d806f0a0 2024-01-11T07:07:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f70fa4f2700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d8067080 0x7f70d8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f70f9cf1700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f70d806f0a0 0x7f70d80714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.870+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70ec053b70 con 0x7f70f41560d0 2024-01-11T07:07:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f1e624e7700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f1e5c0c6270 0x7f1e5c149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f1e624e7700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f1e5c0c6270 0x7f1e5c149c00 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f665e722700 1 -- start start 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665e722700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f665806cb60 0x7f66580c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665e722700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665806e800 0x7f66580cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665e722700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66580c85f0 0x7f665815fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665e722700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665817b760 con 0x7f665806cb60 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665e722700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66580b0e90 con 0x7f665806e800 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665e722700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66580b05a0 con 0x7f66580c85f0 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665ccbf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66580c85f0 0x7f665815fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665ccbf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66580c85f0 0x7f665815fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:39002/0 (socket says 172.21.15.7:39002) 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f665ccbf700 1 -- 172.21.15.7:0/3291280024 learned_addr learned my addr 172.21.15.7:0/3291280024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f66577fe700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665806e800 0x7f66580cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.870+0000 7f665ccbf700 1 -- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665806e800 msgr2=0x7f66580cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.870+0000 7f665ccbf700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f665806e800 0x7f66580cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.870+0000 7f665ccbf700 1 -- 172.21.15.7:0/3291280024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f665806cb60 msgr2=0x7f66580c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.870+0000 7f665ccbf700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f665806cb60 0x7f66580c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.870+0000 7f665ccbf700 1 -- 172.21.15.7:0/3291280024 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c009420 con 0x7f66580c85f0 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.867+0000 7f730d362700 1 --2- 172.21.15.7:0/2733910617 >> v2:172.21.15.113:3300/0 conn(0x7f7308155f40 0x7f7308160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f73077fe700 1 -- 172.21.15.7:0/2733910617 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277637750 0 0) 0x7f73080ca3a0 con 0x7f73080c6f30 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.868+0000 7f73077fe700 1 -- 172.21.15.7:0/2733910617 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72f0003570 con 0x7f73080c6f30 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f730d362700 1 -- 172.21.15.7:0/2733910617 >> v1:172.21.15.113:6789/0 conn(0x7f73080cba20 legacy=0x7f730815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f730d362700 1 -- 172.21.15.7:0/2733910617 >> v1:172.21.15.7:6789/0 conn(0x7f73080c6f30 legacy=0x7f7308155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.869+0000 7f730d362700 1 -- 172.21.15.7:0/2733910617 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080ca8d0 con 0x7f7308155f40 2024-01-11T07:07:09.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f70fa4f2700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d8067080 0x7f70d8069530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f70f000ad20 tx=0x7f70f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f70f9cf1700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f70d806f0a0 0x7f70d80714b0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f665ccbf700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66580c85f0 0x7f665815fdb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f665000cbc0 tx=0x7f6650001780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665000a6c0 con 0x7f66580c85f0 2024-01-11T07:07:09.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 <== osd.4 v2:172.21.15.113:6800/37188 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f70d8071b60 con 0x7f70d806f0a0 2024-01-11T07:07:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f1e51ffb700 1 -- 172.21.15.7:0/902285917 <== osd.5 v2:172.21.15.113:6816/37190 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f1e5c0cabb0 con 0x7f1e5c0c6270 2024-01-11T07:07:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665000a840 con 0x7f66580c85f0 2024-01-11T07:07:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66580c45f0 con 0x7f66580c85f0 2024-01-11T07:07:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665000ac00 con 0x7f66580c85f0 2024-01-11T07:07:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66580b0840 con 0x7f66580c85f0 2024-01-11T07:07:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.872+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6638000fc0 con 0x7f66580c85f0 2024-01-11T07:07:09.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.871+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 --> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd4140ca8d0 con 0x7fd3f806f1e0 2024-01-11T07:07:09.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.873+0000 7f730d362700 1 --2- 172.21.15.7:0/2733910617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7308155f40 0x7f7308160d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f72f800aa90 tx=0x7f72f800a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f091e4ac3233e69e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.873+0000 7fd410ff9700 1 -- 172.21.15.7:0/78912327 <== osd.1 v2:172.21.15.7:6808/36943 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd4140ca8d0 con 0x7fd3f806f1e0 2024-01-11T07:07:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.874+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6650012420 con 0x7f66580c85f0 2024-01-11T07:07:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.874+0000 7f663ffff700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6644066f80 0x7f6644069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.875+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f66500182b0 con 0x7f66580c85f0 2024-01-11T07:07:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.875+0000 7f663ffff700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7f664406f190 0x7f66440715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.875+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 --> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6644071c50 con 0x7f664406f190 2024-01-11T07:07:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.875+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6650050fd0 con 0x7f66580c85f0 2024-01-11T07:07:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.879+0000 7f6657fff700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6644066f80 0x7f6644069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.879+0000 7f66577fe700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7f664406f190 0x7f66440715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.883+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fd3f806f1e0 msgr2=0x7fd3f80715f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.883+0000 7fd41af53700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fd3f806f1e0 0x7fd3f80715f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.883+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3f8066fd0 msgr2=0x7fd3f8069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.883+0000 7fd41af53700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3f8066fd0 0x7fd3f8069480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd404003a80 tx=0x7fd404007ec0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.883+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4140cba20 msgr2=0x7fd4140c3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.883+0000 7fd41af53700 1 --2- 172.21.15.7:0/78912327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4140cba20 0x7fd4140c3ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d860 tx=0x7fd408008040 comp rx=0 tx=0).stop 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.883+0000 7fd4194f0700 1 -- 172.21.15.7:0/78912327 reap_dead start 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.883+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 shutdown_connections 2024-01-11T07:07:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 >> 172.21.15.7:0/78912327 conn(0x7fd414063230 msgr2=0x7fd4140ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7f6657fff700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6644066f80 0x7f6644069430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f66480050b0 tx=0x7f6648007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7f73077fe700 1 -- 172.21.15.7:0/2733910617 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f800c930 con 0x7f7308155f40 2024-01-11T07:07:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7f73077fe700 1 -- 172.21.15.7:0/2733910617 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f800ca90 con 0x7f7308155f40 2024-01-11T07:07:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7f73077fe700 1 -- 172.21.15.7:0/2733910617 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f800ce30 con 0x7f7308155f40 2024-01-11T07:07:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 shutdown_connections 2024-01-11T07:07:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7fd41af53700 1 -- 172.21.15.7:0/78912327 wait complete. 2024-01-11T07:07:09.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7f730edc5700 1 -- 172.21.15.7:0/2733910617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7308155f40 msgr2=0x7f7308160d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.884+0000 7f730edc5700 1 --2- 172.21.15.7:0/2733910617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7308155f40 0x7f7308160d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f72f800aa90 tx=0x7f72f800a330 comp rx=0 tx=0).stop 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.885+0000 7f730edc5700 1 -- 172.21.15.7:0/2733910617 shutdown_connections 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.885+0000 7f730edc5700 1 --2- 172.21.15.7:0/2733910617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7308155f40 0x7f7308160d30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.885+0000 7f730edc5700 1 -- 172.21.15.7:0/2733910617 >> 172.21.15.7:0/2733910617 conn(0x7f7308063230 msgr2=0x7f7308063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.885+0000 7f730edc5700 1 -- 172.21.15.7:0/2733910617 shutdown_connections 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.885+0000 7f730edc5700 1 -- 172.21.15.7:0/2733910617 wait complete. 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.886+0000 7f730edc5700 1 Processor -- start 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.887+0000 7f730edc5700 1 -- start start 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730edc5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73080c6f30 0x7f730815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730edc5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73080cba20 0x7f73080aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730edc5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7308155f40 0x7f7308150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730edc5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730815c760 con 0x7f73080cba20 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730edc5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080c5410 con 0x7f73080c6f30 2024-01-11T07:07:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730edc5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73080b04c0 con 0x7f7308155f40 2024-01-11T07:07:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730cb61700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73080c6f30 0x7f730815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730cb61700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73080c6f30 0x7f730815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36310/0 (socket says 172.21.15.7:36310) 2024-01-11T07:07:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730cb61700 1 -- 172.21.15.7:0/69218927 learned_addr learned my addr 172.21.15.7:0/69218927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.888+0000 7f730d362700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7308155f40 0x7f7308150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.889+0000 7f66577fe700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7f664406f190 0x7f66440715a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.889+0000 7f730cb61700 1 -- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7308155f40 msgr2=0x7f7308150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.889+0000 7f730cb61700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7308155f40 0x7f7308150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.889+0000 7f730cb61700 1 -- 172.21.15.7:0/69218927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73080cba20 msgr2=0x7f73080aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.889+0000 7f730cb61700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73080cba20 0x7f73080aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.889+0000 7f730cb61700 1 -- 172.21.15.7:0/69218927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f800a5d0 con 0x7f73080c6f30 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.889+0000 7f730cb61700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73080c6f30 0x7f730815c220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00c5d0 tx=0x7f72fc007f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.890+0000 7f7304ff9700 1 -- 172.21.15.7:0/69218927 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc011590 con 0x7f73080c6f30 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.890+0000 7f7304ff9700 1 -- 172.21.15.7:0/69218927 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc011710 con 0x7f73080c6f30 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.890+0000 7f7304ff9700 1 -- 172.21.15.7:0/69218927 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc011ad0 con 0x7f73080c6f30 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.890+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73080b0750 con 0x7f73080c6f30 2024-01-11T07:07:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.890+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73080c50a0 con 0x7f73080c6f30 2024-01-11T07:07:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.892+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 <== osd.6 v2:172.21.15.113:6808/37189 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f6644071c50 con 0x7f664406f190 2024-01-11T07:07:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.893+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 --> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1eb015a770 con 0x7f1e9c06f0c0 2024-01-11T07:07:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.893+0000 7f7304ff9700 1 -- 172.21.15.7:0/69218927 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f72fc01c460 con 0x7f73080c6f30 2024-01-11T07:07:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.893+0000 7f7304ff9700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72f4067080 0x7f72f4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.893+0000 7f7304ff9700 1 -- 172.21.15.7:0/69218927 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72fc084240 con 0x7f73080c6f30 2024-01-11T07:07:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.893+0000 7f1e96ffd700 1 -- 172.21.15.7:0/2804102289 <== osd.3 v2:172.21.15.7:6800/36944 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1eb015a770 con 0x7f1e9c06f0c0 2024-01-11T07:07:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.893+0000 7f7307fff700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72f4067080 0x7f72f4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.895+0000 7f7307fff700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72f4067080 0x7f72f4069530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f73080b0db0 tx=0x7f7300005e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f1e9c06f0c0 msgr2=0x7f1e9c0714d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eb5388700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f1e9c06f0c0 0x7f1e9c0714d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f730edc5700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f7308159920 0x7f7308169390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 --> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73080cabb0 con 0x7f7308159920 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e9c066f30 msgr2=0x7f1e9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eb5388700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e9c066f30 0x7f1e9c0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006f00 tx=0x7f1ea40091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eb00b1570 msgr2=0x7f1eb00b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eb5388700 1 --2- 172.21.15.7:0/2804102289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1eb00b1570 0x7f1eb00b1a10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1eb006c290 tx=0x7f1ea0007c80 comp rx=0 tx=0).stop 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eaf7fe700 1 -- 172.21.15.7:0/2804102289 reap_dead start 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 shutdown_connections 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 >> 172.21.15.7:0/2804102289 conn(0x7f1eb0063230 msgr2=0x7f1eb0063600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.900+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 shutdown_connections 2024-01-11T07:07:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.900+0000 7f1eb5388700 1 -- 172.21.15.7:0/2804102289 wait complete. 2024-01-11T07:07:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.900+0000 7f730d362700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f7308159920 0x7f7308169390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.900+0000 7f730d362700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f7308159920 0x7f7308169390 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.899+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 --> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb554005c10 con 0x7fb5540015e0 2024-01-11T07:07:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.901+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 --> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa6fc04c3c0 con 0x7fa6e40707c0 2024-01-11T07:07:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.901+0000 7f7304ff9700 1 -- 172.21.15.7:0/69218927 <== osd.7 v2:172.21.15.113:6824/37240 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f73080cabb0 con 0x7f7308159920 2024-01-11T07:07:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.901+0000 7fb569ffb700 1 -- 172.21.15.7:0/1462175126 <== osd.0 v2:172.21.15.7:6816/36945 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb554005c10 con 0x7fb5540015e0 2024-01-11T07:07:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.901+0000 7fa6e2ffd700 1 -- 172.21.15.7:0/726738192 <== osd.2 v2:172.21.15.7:6823/36946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa6fc04c3c0 con 0x7fa6e40707c0 2024-01-11T07:07:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.902+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 --> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70f4069350 con 0x7f70d806f0a0 2024-01-11T07:07:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.902+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fa6e40707c0 msgr2=0x7fa6e4072bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.902+0000 7fa7015c4700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fa6e40707c0 0x7fa6e4072bd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.902+0000 7f70ea7fc700 1 -- 172.21.15.7:0/1808784610 <== osd.4 v2:172.21.15.113:6800/37188 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f70f4069350 con 0x7f70d806f0a0 2024-01-11T07:07:09.929 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:09.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-11T07:07:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.903+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 --> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e5c0c6d10 con 0x7f1e5c0c6270 2024-01-11T07:07:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.903+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6e4068660 msgr2=0x7fa6e406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.903+0000 7fa7015c4700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6e4068660 0x7fa6e406ab10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec001010 tx=0x7fa6ec009040 comp rx=0 tx=0).stop 2024-01-11T07:07:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.903+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6fc0cedf0 msgr2=0x7fa6fc0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.903+0000 7fa7015c4700 1 --2- 172.21.15.7:0/726738192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6fc0cedf0 0x7fa6fc0cf270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400e210 tx=0x7fa6f401a000 comp rx=0 tx=0).stop 2024-01-11T07:07:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fa6fb7fe700 1 -- 172.21.15.7:0/726738192 reap_dead start 2024-01-11T07:07:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f70d806f0a0 msgr2=0x7f70d80714b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7f70fc756700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f70d806f0a0 0x7f70d80714b0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 shutdown_connections 2024-01-11T07:07:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 >> 172.21.15.7:0/726738192 conn(0x7fa6fc0ce9f0 msgr2=0x7fa6fc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7fb5540015e0 msgr2=0x7fb554003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fb57bae2700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7fb5540015e0 0x7fb554003a90 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 shutdown_connections 2024-01-11T07:07:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fa7015c4700 1 -- 172.21.15.7:0/726738192 wait complete. 2024-01-11T07:07:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d8067080 msgr2=0x7f70d8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7f70fc756700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d8067080 0x7f70d8069530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f70f000ad20 tx=0x7f70f0008040 comp rx=0 tx=0).stop 2024-01-11T07:07:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 msgr2=0x7f70f40b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7f70fc756700 1 --2- 172.21.15.7:0/1808784610 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70f41560d0 0x7f70f40b1830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f70ec019000 tx=0x7f70ec01c000 comp rx=0 tx=0).stop 2024-01-11T07:07:09.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7f1e51ffb700 1 -- 172.21.15.7:0/902285917 <== osd.5 v2:172.21.15.113:6816/37190 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e5c0c6d10 con 0x7f1e5c0c6270 2024-01-11T07:07:09.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7f70facf3700 1 -- 172.21.15.7:0/1808784610 reap_dead start 2024-01-11T07:07:09.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb560067430 msgr2=0x7fb5600698e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.904+0000 7fb57bae2700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb560067430 0x7fb5600698e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb570009db0 tx=0x7fb5700091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415d8a0 msgr2=0x7fb5740b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7fb57bae2700 1 --2- 172.21.15.7:0/1462175126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb57415d8a0 0x7fb5740b48d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c810 tx=0x7fb564007d90 comp rx=0 tx=0).stop 2024-01-11T07:07:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7fb57a07f700 1 -- 172.21.15.7:0/1462175126 reap_dead start 2024-01-11T07:07:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 shutdown_connections 2024-01-11T07:07:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 >> 172.21.15.7:0/1462175126 conn(0x7fb574071120 msgr2=0x7fb574159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 shutdown_connections 2024-01-11T07:07:09.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7fb57bae2700 1 -- 172.21.15.7:0/1462175126 wait complete. 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f1e5c0c6270 msgr2=0x7f1e5c149c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e63f4a700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f1e5c0c6270 0x7f1e5c149c00 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e40066fb0 msgr2=0x7f1e40069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e63f4a700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e40066fb0 0x7f1e40069460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c001050 tx=0x7f1e4c007ae0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5c063d40 msgr2=0x7f1e5c0c2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e63f4a700 1 --2- 172.21.15.7:0/902285917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e5c063d40 0x7f1e5c0c2f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0cb170 tx=0x7f1e58016000 comp rx=0 tx=0).stop 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e624e7700 1 -- 172.21.15.7:0/902285917 reap_dead start 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 shutdown_connections 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.905+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 >> 172.21.15.7:0/902285917 conn(0x7f1e5c063230 msgr2=0x7f1e5c0c1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.906+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 shutdown_connections 2024-01-11T07:07:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.906+0000 7f1e63f4a700 1 -- 172.21.15.7:0/902285917 wait complete. 2024-01-11T07:07:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.907+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 shutdown_connections 2024-01-11T07:07:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.907+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 >> 172.21.15.7:0/1808784610 conn(0x7f70f4063230 msgr2=0x7f70f40ab710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.908+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 shutdown_connections 2024-01-11T07:07:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.908+0000 7f70fc756700 1 -- 172.21.15.7:0/1808784610 wait complete. 2024-01-11T07:07:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.912+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 --> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6638002d20 con 0x7f664406f190 2024-01-11T07:07:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.913+0000 7f663ffff700 1 -- 172.21.15.7:0/3291280024 <== osd.6 v2:172.21.15.113:6808/37189 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6638002d20 con 0x7f664406f190 2024-01-11T07:07:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.919+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 --> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f730806a380 con 0x7f7308159920 2024-01-11T07:07:09.938 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:09.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-11T07:07:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f7304ff9700 1 -- 172.21.15.7:0/69218927 <== osd.7 v2:172.21.15.113:6824/37240 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f730806a380 con 0x7f7308159920 2024-01-11T07:07:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f7308159920 msgr2=0x7f7308169390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730edc5700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f7308159920 0x7f7308169390 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72f4067080 msgr2=0x7f72f4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730edc5700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72f4067080 0x7f72f4069530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f73080b0db0 tx=0x7f7300005e90 comp rx=0 tx=0).stop 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73080c6f30 msgr2=0x7f730815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730edc5700 1 --2- 172.21.15.7:0/69218927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73080c6f30 0x7f730815c220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00c5d0 tx=0x7f72fc007f60 comp rx=0 tx=0).stop 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730d362700 1 -- 172.21.15.7:0/69218927 reap_dead start 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 shutdown_connections 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.920+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 >> 172.21.15.7:0/69218927 conn(0x7f7308063230 msgr2=0x7f73080c1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.921+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 shutdown_connections 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.921+0000 7f730edc5700 1 -- 172.21.15.7:0/69218927 wait complete. 2024-01-11T07:07:09.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7f664406f190 msgr2=0x7f66440715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7f664406f190 0x7f66440715a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6644066f80 msgr2=0x7f6644069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6644066f80 0x7f6644069430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f66480050b0 tx=0x7f6648007c10 comp rx=0 tx=0).stop 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66580c85f0 msgr2=0x7f665815fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 --2- 172.21.15.7:0/3291280024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66580c85f0 0x7f665815fdb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f665000cbc0 tx=0x7f6650001780 comp rx=0 tx=0).stop 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665ccbf700 1 -- 172.21.15.7:0/3291280024 reap_dead start 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 shutdown_connections 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 >> 172.21.15.7:0/3291280024 conn(0x7f6658063230 msgr2=0x7f66580c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 shutdown_connections 2024-01-11T07:07:09.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:09.923+0000 7f665e722700 1 -- 172.21.15.7:0/3291280024 wait complete. 2024-01-11T07:07:09.942 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:09.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-11T07:07:09.943 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:09.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-11T07:07:09.944 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:09.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-11T07:07:09.945 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:09.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-11T07:07:09.946 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:09.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-11T07:07:09.947 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:09.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-11T07:07:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.135+0000 7f7c66a90700 1 Processor -- start 2024-01-11T07:07:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.135+0000 7f7c66a90700 1 -- start start 2024-01-11T07:07:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.136+0000 7f7c66a90700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c6006e630 0x7f7c6006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.136+0000 7f7c66a90700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6016a620 con 0x7f7c600c98b0 2024-01-11T07:07:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.136+0000 7f7c66a90700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6016b480 con 0x7f7c6006ea00 2024-01-11T07:07:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.136+0000 7f7c66a90700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6016a870 con 0x7f7c6006e630 2024-01-11T07:07:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.137+0000 7f7c5ffff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7c6006ea00 0x7f7c600c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40326/0 (socket says 172.21.15.7:40326) 2024-01-11T07:07:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.137+0000 7f7c5ffff700 1 -- 172.21.15.7:0/2804267748 learned_addr learned my addr 172.21.15.7:0/2804267748 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.138+0000 7f7c6482c700 1 --2- 172.21.15.7:0/2804267748 >> v2:172.21.15.7:3300/0 conn(0x7f7c6006e630 0x7f7c6006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.138+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534033276 0 0) 0x7f7c6016b480 con 0x7f7c6006ea00 2024-01-11T07:07:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c480039c0 con 0x7f7c6006ea00 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571441141 0 0) 0x7f7c6016a620 con 0x7f7c600c98b0 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6016b480 con 0x7f7c600c98b0 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 793532985 0 0) 0x7f7c6016b480 con 0x7f7c600c98b0 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 >> v2:172.21.15.7:3300/0 conn(0x7f7c6006e630 msgr2=0x7f7c6006cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 --2- 172.21.15.7:0/2804267748 >> v2:172.21.15.7:3300/0 conn(0x7f7c6006e630 0x7f7c6006cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 >> v1:172.21.15.113:6789/0 conn(0x7f7c6006ea00 legacy=0x7f7c600c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6016ab50 con 0x7f7c600c98b0 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.139+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f7c54001470 con 0x7f7c600c98b0 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.140+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 >> v1:172.21.15.7:6789/0 conn(0x7f7c600c98b0 legacy=0x7f7c600c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.140+0000 7f7c5f7fe700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480050a0 0x7f7c48005490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.140+0000 7f7c5f7fe700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c48003c00 0x7f7c48004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.140+0000 7f7c5f7fe700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c480045b0 0x7f7c4800beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.140+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6016a870 con 0x7f7c48003c00 2024-01-11T07:07:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.140+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4800c500 con 0x7f7c480050a0 2024-01-11T07:07:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.140+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4800c6c0 con 0x7f7c480045b0 2024-01-11T07:07:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.140+0000 7f7c6502d700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c480045b0 0x7f7c4800beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.141+0000 7f7c5ffff700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c48003c00 0x7f7c48004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.141+0000 7f7c5ffff700 1 -- 172.21.15.7:0/2804267748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c480045b0 msgr2=0x7f7c4800beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.141+0000 7f7c5ffff700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c480045b0 0x7f7c4800beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.141+0000 7f7c5ffff700 1 -- 172.21.15.7:0/2804267748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480050a0 msgr2=0x7f7c48005490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.141+0000 7f7c5ffff700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c480050a0 0x7f7c48005490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.141+0000 7f7c5ffff700 1 -- 172.21.15.7:0/2804267748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6016ab50 con 0x7f7c48003c00 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.141+0000 7f7c5ffff700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c48003c00 0x7f7c48004070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c50001b30 tx=0x7f7c50007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9ff4536046294f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.142+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500048f0 con 0x7f7c48003c00 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.142+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50004a70 con 0x7f7c48003c00 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.142+0000 7f7c5f7fe700 1 -- 172.21.15.7:0/2804267748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004e30 con 0x7f7c48003c00 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.143+0000 7f7c66a90700 1 -- 172.21.15.7:0/2804267748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c48003c00 msgr2=0x7f7c48004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.143+0000 7f7c66a90700 1 --2- 172.21.15.7:0/2804267748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c48003c00 0x7f7c48004070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c50001b30 tx=0x7f7c50007de0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.143+0000 7f7c6502d700 1 -- 172.21.15.7:0/2804267748 reap_dead start 2024-01-11T07:07:10.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.143+0000 7f7c66a90700 1 -- 172.21.15.7:0/2804267748 shutdown_connections 2024-01-11T07:07:10.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.143+0000 7f7c66a90700 1 -- 172.21.15.7:0/2804267748 >> 172.21.15.7:0/2804267748 conn(0x7f7c60063230 msgr2=0x7f7c60063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.144+0000 7f7c66a90700 1 -- 172.21.15.7:0/2804267748 shutdown_connections 2024-01-11T07:07:10.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.144+0000 7f7c66a90700 1 -- 172.21.15.7:0/2804267748 wait complete. 2024-01-11T07:07:10.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.146+0000 7f7c66a90700 1 Processor -- start 2024-01-11T07:07:10.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.147+0000 7f7c66a90700 1 -- start start 2024-01-11T07:07:10.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.148+0000 7f7c66a90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c601592b0 0x7f7c601679f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.148+0000 7f7c66a90700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c60166600 0x7f7c60169500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.148+0000 7f7c66a90700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c60169a40 0x7f7c600c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.148+0000 7f7c66a90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600b0080 con 0x7f7c601592b0 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.148+0000 7f7c66a90700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600c4850 con 0x7f7c60169a40 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.148+0000 7f7c66a90700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600c4e10 con 0x7f7c60166600 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.149+0000 7f7c6482c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c601592b0 0x7f7c601679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.149+0000 7f7c6482c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c601592b0 0x7f7c601679f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41396/0 (socket says 172.21.15.7:41396) 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.149+0000 7f7c6482c700 1 -- 172.21.15.7:0/4159722440 learned_addr learned my addr 172.21.15.7:0/4159722440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.149+0000 7f7c6482c700 1 -- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c60166600 msgr2=0x7f7c60169500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.149+0000 7f7c6482c700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c60166600 0x7f7c60169500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.149+0000 7f7c6482c700 1 -- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c60169a40 msgr2=0x7f7c600c52a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.149+0000 7f7c6482c700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c60169a40 0x7f7c600c52a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.149+0000 7f7c6482c700 1 -- 172.21.15.7:0/4159722440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60159680 con 0x7f7c601592b0 2024-01-11T07:07:10.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.150+0000 7f7c6482c700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c601592b0 0x7f7c601679f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009ec0 tx=0x7f7c5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.150+0000 7f7c5cff9700 1 -- 172.21.15.7:0/4159722440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800cc20 con 0x7f7c601592b0 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.150+0000 7f7c5cff9700 1 -- 172.21.15.7:0/4159722440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5800f040 con 0x7f7c601592b0 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.150+0000 7f7c5cff9700 1 -- 172.21.15.7:0/4159722440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c58014480 con 0x7f7c601592b0 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.150+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c600b02d0 con 0x7f7c601592b0 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.150+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c600c4aa0 con 0x7f7c601592b0 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.154+0000 7f7c5cff9700 1 -- 172.21.15.7:0/4159722440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7c58014870 con 0x7f7c601592b0 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.154+0000 7f7c5cff9700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c48068d50 0x7f7c4806b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.154+0000 7f7c5cff9700 1 -- 172.21.15.7:0/4159722440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7c58084e60 con 0x7f7c601592b0 2024-01-11T07:07:10.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.155+0000 7f7c5ffff700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c48068d50 0x7f7c4806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.159+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c400052a0 con 0x7f7c601592b0 2024-01-11T07:07:10.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.168+0000 7f7c5cff9700 1 -- 172.21.15.7:0/4159722440 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c580520b0 con 0x7f7c601592b0 2024-01-11T07:07:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.173+0000 7f7c5ffff700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c48068d50 0x7f7c4806b200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c601675f0 tx=0x7f7c50015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.216+0000 7fbdc958a700 1 Processor -- start 2024-01-11T07:07:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.216+0000 7fbdc958a700 1 -- start start 2024-01-11T07:07:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc958a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdc406f2e0 0x7fbdc406f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc958a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40af8e0 0x7fbdc40afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc958a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc40c9e80 con 0x7fbdc40cbeb0 2024-01-11T07:07:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc958a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc406fca0 con 0x7fbdc406f2e0 2024-01-11T07:07:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc958a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c8f90 con 0x7fbdc40af8e0 2024-01-11T07:07:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc2ffd700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbdc40cbeb0 0x7fbdc40cf150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40330/0 (socket says 172.21.15.7:40330) 2024-01-11T07:07:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc2ffd700 1 -- 172.21.15.7:0/4147741589 learned_addr learned my addr 172.21.15.7:0/4147741589 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc37fe700 1 --2- 172.21.15.7:0/4147741589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40af8e0 0x7fbdc40afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc37fe700 1 -- 172.21.15.7:0/4147741589 >> v2:172.21.15.7:3300/0 conn(0x7fbdc406f2e0 msgr2=0x7fbdc406f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc37fe700 1 --2- 172.21.15.7:0/4147741589 >> v2:172.21.15.7:3300/0 conn(0x7fbdc406f2e0 0x7fbdc406f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.218+0000 7fbdc37fe700 1 -- 172.21.15.7:0/4147741589 >> v1:172.21.15.113:6789/0 conn(0x7fbdc40cbeb0 legacy=0x7fbdc40cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.219+0000 7fbdc37fe700 1 -- 172.21.15.7:0/4147741589 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40c9270 con 0x7fbdc40af8e0 2024-01-11T07:07:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.219+0000 7fbdc37fe700 1 --2- 172.21.15.7:0/4147741589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40af8e0 0x7fbdc40afd40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009e60 tx=0x7fbdbc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05abfeb4a736c9a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7fbdc1ffb700 1 -- 172.21.15.7:0/4147741589 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc00e070 con 0x7fbdc40af8e0 2024-01-11T07:07:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7fbdc1ffb700 1 -- 172.21.15.7:0/4147741589 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc00bc30 con 0x7fbdc40af8e0 2024-01-11T07:07:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7fbdc1ffb700 1 -- 172.21.15.7:0/4147741589 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc012430 con 0x7fbdc40af8e0 2024-01-11T07:07:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7fbdc958a700 1 -- 172.21.15.7:0/4147741589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40af8e0 msgr2=0x7fbdc40afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7fbdc958a700 1 --2- 172.21.15.7:0/4147741589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40af8e0 0x7fbdc40afd40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009e60 tx=0x7fbdbc009780 comp rx=0 tx=0).stop 2024-01-11T07:07:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7fbdc958a700 1 -- 172.21.15.7:0/4147741589 shutdown_connections 2024-01-11T07:07:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7fbdc958a700 1 --2- 172.21.15.7:0/4147741589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40af8e0 0x7fbdc40afd40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7fbdc958a700 1 --2- 172.21.15.7:0/4147741589 >> v2:172.21.15.7:3300/0 conn(0x7fbdc406f2e0 0x7fbdc406f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7fbdc958a700 1 -- 172.21.15.7:0/4147741589 >> 172.21.15.7:0/4147741589 conn(0x7fbdc4063230 msgr2=0x7fbdc4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.220+0000 7f2fd00b3700 1 Processor -- start 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.220+0000 7f2fd00b3700 1 -- start start 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.221+0000 7f2fd00b3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.221+0000 7f2fd00b3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2fc806e0d0 0x7f2fc8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.221+0000 7f2fd00b3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc815b770 con 0x7f2fc8159ff0 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.221+0000 7f2fd00b3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc815abf0 con 0x7f2fc806e0d0 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.221+0000 7f2fd00b3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc815aed0 con 0x7f2fc806dd00 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.221+0000 7f2fcde4f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.221+0000 7f2fcde4f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:39054/0 (socket says 172.21.15.7:39054) 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.221+0000 7f2fcde4f700 1 -- 172.21.15.7:0/130569120 learned_addr learned my addr 172.21.15.7:0/130569120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcd64e700 1 --2- 172.21.15.7:0/130569120 >> v2:172.21.15.7:3300/0 conn(0x7f2fc806e0d0 0x7f2fc8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcde4f700 1 -- 172.21.15.7:0/130569120 >> v2:172.21.15.7:3300/0 conn(0x7f2fc806e0d0 msgr2=0x7f2fc8159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcde4f700 1 --2- 172.21.15.7:0/130569120 >> v2:172.21.15.7:3300/0 conn(0x7f2fc806e0d0 0x7f2fc8159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcde4f700 1 -- 172.21.15.7:0/130569120 >> v1:172.21.15.113:6789/0 conn(0x7f2fc8159ff0 legacy=0x7f2fc80c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcde4f700 1 -- 172.21.15.7:0/130569120 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc815b1b0 con 0x7f2fc806dd00 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcde4f700 1 --2- 172.21.15.7:0/130569120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80c98d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009f20 tx=0x7f2fc0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ec54812f922f4b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcce4d700 1 -- 172.21.15.7:0/130569120 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000e070 con 0x7f2fc806dd00 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcce4d700 1 -- 172.21.15.7:0/130569120 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc000ba40 con 0x7f2fc806dd00 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.222+0000 7f2fcce4d700 1 -- 172.21.15.7:0/130569120 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00092e0 con 0x7f2fc806dd00 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7f2fd00b3700 1 -- 172.21.15.7:0/130569120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 msgr2=0x7f2fc80c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/130569120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80c98d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009f20 tx=0x7f2fc0009840 comp rx=0 tx=0).stop 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7f2fd00b3700 1 -- 172.21.15.7:0/130569120 shutdown_connections 2024-01-11T07:07:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/130569120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80c98d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/130569120 >> v2:172.21.15.7:3300/0 conn(0x7f2fc806e0d0 0x7f2fc8159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7f2fd00b3700 1 -- 172.21.15.7:0/130569120 >> 172.21.15.7:0/130569120 conn(0x7f2fc80680e0 msgr2=0x7f2fc80cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7f2fd00b3700 1 -- 172.21.15.7:0/130569120 shutdown_connections 2024-01-11T07:07:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.223+0000 7f2fd00b3700 1 -- 172.21.15.7:0/130569120 wait complete. 2024-01-11T07:07:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.224+0000 7f9dbce98700 1 Processor -- start 2024-01-11T07:07:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.224+0000 7fbdc958a700 1 -- 172.21.15.7:0/4147741589 shutdown_connections 2024-01-11T07:07:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.224+0000 7f9dbce98700 1 -- start start 2024-01-11T07:07:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.226+0000 7f9dbce98700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9db8063d40 0x7f9db80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.226+0000 7f9dbce98700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db80ca3a0 con 0x7f9db80cbd10 2024-01-11T07:07:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.226+0000 7f9dbce98700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db80cb200 con 0x7f9db80c6f30 2024-01-11T07:07:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.226+0000 7f9dbce98700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db80cc150 con 0x7f9db8063d40 2024-01-11T07:07:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.226+0000 7f9db659c700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9db80c6f30 0x7f9db815a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40338/0 (socket says 172.21.15.7:40338) 2024-01-11T07:07:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.226+0000 7f9db659c700 1 -- 172.21.15.7:0/84387262 learned_addr learned my addr 172.21.15.7:0/84387262 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.225+0000 7f2fd00b3700 1 Processor -- start 2024-01-11T07:07:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.225+0000 7f2fd00b3700 1 -- start start 2024-01-11T07:07:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fd00b3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fd00b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc806e0d0 0x7f2fc80c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fd00b3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc8159ff0 0x7f2fc80c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fd00b3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80c7420 con 0x7f2fc806e0d0 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fd00b3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80c6810 con 0x7f2fc8159ff0 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fd00b3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc80c6af0 con 0x7f2fc806dd00 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcde4f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcde4f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:39070/0 (socket says 172.21.15.7:39070) 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcde4f700 1 -- 172.21.15.7:0/3167777428 learned_addr learned my addr 172.21.15.7:0/3167777428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fce650700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc8159ff0 0x7f2fc80c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcd64e700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc806e0d0 0x7f2fc80c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcde4f700 1 -- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc8159ff0 msgr2=0x7f2fc80c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcde4f700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc8159ff0 0x7f2fc80c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcde4f700 1 -- 172.21.15.7:0/3167777428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc806e0d0 msgr2=0x7f2fc80c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcde4f700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc806e0d0 0x7f2fc80c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f2fcde4f700 1 -- 172.21.15.7:0/3167777428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc0009ae0 con 0x7f2fc806dd00 2024-01-11T07:07:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7fbdc958a700 1 -- 172.21.15.7:0/4147741589 wait complete. 2024-01-11T07:07:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.228+0000 7f2fcde4f700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80ca8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2fc815b010 tx=0x7f2fc0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f7294dc5700 1 Processor -- start 2024-01-11T07:07:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.230+0000 7f2fbdffb700 1 -- 172.21.15.7:0/3167777428 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00129d0 con 0x7f2fc806dd00 2024-01-11T07:07:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.230+0000 7f2fbdffb700 1 -- 172.21.15.7:0/3167777428 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc0012b70 con 0x7f2fc806dd00 2024-01-11T07:07:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.230+0000 7f2fbdffb700 1 -- 172.21.15.7:0/3167777428 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc001e430 con 0x7f2fc806dd00 2024-01-11T07:07:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.230+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc80c6dd0 con 0x7f2fc806dd00 2024-01-11T07:07:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.230+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc80cae30 con 0x7f2fc806dd00 2024-01-11T07:07:10.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.226+0000 7f9db5d9b700 1 --2- 172.21.15.7:0/84387262 >> v2:172.21.15.7:3300/0 conn(0x7f9db8063d40 0x7f9db80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f9db559a700 1 -- 172.21.15.7:0/84387262 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382243821 0 0) 0x7f9db80ca3a0 con 0x7f9db80cbd10 2024-01-11T07:07:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f9db559a700 1 -- 172.21.15.7:0/84387262 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da0003570 con 0x7f9db80cbd10 2024-01-11T07:07:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f9db559a700 1 -- 172.21.15.7:0/84387262 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 183553086 0 0) 0x7f9da0003570 con 0x7f9db80cbd10 2024-01-11T07:07:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f9db5d9b700 1 -- 172.21.15.7:0/84387262 >> v1:172.21.15.113:6789/0 conn(0x7f9db80c6f30 legacy=0x7f9db815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f9db5d9b700 1 -- 172.21.15.7:0/84387262 >> v1:172.21.15.7:6789/0 conn(0x7f9db80cbd10 legacy=0x7f9db8155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.227+0000 7f9db5d9b700 1 -- 172.21.15.7:0/84387262 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db80ca8d0 con 0x7f9db8063d40 2024-01-11T07:07:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.228+0000 7f9db5d9b700 1 --2- 172.21.15.7:0/84387262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 0x7f9db80cb7d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0098e0 tx=0x7f9dac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed0796316636fb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.228+0000 7f9db559a700 1 -- 172.21.15.7:0/84387262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00017f0 con 0x7f9db8063d40 2024-01-11T07:07:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f7294dc5700 1 -- start start 2024-01-11T07:07:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.231+0000 7f7294dc5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f729006a2f0 0x7f7290069360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.231+0000 7f7294dc5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900698a0 0x7f72900ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.231+0000 7f7294dc5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7290068bf0 con 0x7f72900ca5f0 2024-01-11T07:07:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.231+0000 7f7294dc5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729015eda0 con 0x7f729006a2f0 2024-01-11T07:07:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.231+0000 7f7294dc5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7290068ed0 con 0x7f72900698a0 2024-01-11T07:07:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7fbdc958a700 1 Processor -- start 2024-01-11T07:07:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728dd9b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f729006a2f0 0x7f7290069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728dd9b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f729006a2f0 0x7f7290069360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36358/0 (socket says 172.21.15.7:36358) 2024-01-11T07:07:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728dd9b700 1 -- 172.21.15.7:0/3384199103 learned_addr learned my addr 172.21.15.7:0/3384199103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728ed9d700 1 --2- 172.21.15.7:0/3384199103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900698a0 0x7f72900ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7fbdc958a700 1 -- start start 2024-01-11T07:07:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa80052a0 con 0x7f2fc806dd00 2024-01-11T07:07:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728ed9d700 1 -- 172.21.15.7:0/3384199103 >> v2:172.21.15.113:3300/0 conn(0x7f729006a2f0 msgr2=0x7f7290069360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728ed9d700 1 --2- 172.21.15.7:0/3384199103 >> v2:172.21.15.113:3300/0 conn(0x7f729006a2f0 0x7f7290069360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728ed9d700 1 -- 172.21.15.7:0/3384199103 >> v1:172.21.15.113:6789/0 conn(0x7f72900ca5f0 legacy=0x7f729015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728ed9d700 1 -- 172.21.15.7:0/3384199103 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729015efb0 con 0x7f72900698a0 2024-01-11T07:07:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728dd9b700 1 --2- 172.21.15.7:0/3384199103 >> v2:172.21.15.113:3300/0 conn(0x7f729006a2f0 0x7f7290069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f728ed9d700 1 --2- 172.21.15.7:0/3384199103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900698a0 0x7f72900ae7f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7288009e60 tx=0x7f7288009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcba13e605fbe562 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.228+0000 7f9db559a700 1 -- 172.21.15.7:0/84387262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac011420 con 0x7f9db8063d40 2024-01-11T07:07:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.228+0000 7f9db559a700 1 -- 172.21.15.7:0/84387262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0117a0 con 0x7f9db8063d40 2024-01-11T07:07:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f9dbce98700 1 -- 172.21.15.7:0/84387262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 msgr2=0x7f9db80cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f9dbce98700 1 --2- 172.21.15.7:0/84387262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 0x7f9db80cb7d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0098e0 tx=0x7f9dac0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f9dbce98700 1 -- 172.21.15.7:0/84387262 shutdown_connections 2024-01-11T07:07:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f9dbce98700 1 --2- 172.21.15.7:0/84387262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 0x7f9db80cb7d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f9dbce98700 1 -- 172.21.15.7:0/84387262 >> 172.21.15.7:0/84387262 conn(0x7f9db8063230 msgr2=0x7f9db8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f9dbce98700 1 -- 172.21.15.7:0/84387262 shutdown_connections 2024-01-11T07:07:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.229+0000 7f9dbce98700 1 -- 172.21.15.7:0/84387262 wait complete. 2024-01-11T07:07:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7f9dbce98700 1 Processor -- start 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7fbdc958a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc406f2e0 0x7fbdc40c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7fbdc958a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc40af8e0 0x7fbdc416f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7fbdc958a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40cbeb0 0x7fbdc416e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7fbdc958a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc416e580 con 0x7fbdc40af8e0 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7fbdc958a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c2440 con 0x7fbdc406f2e0 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7fbdc958a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40c26e0 con 0x7fbdc40cbeb0 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7f9dbce98700 1 -- start start 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f9dbce98700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 0x7f9db80c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f9dbce98700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9db80c6f30 0x7f9db806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f9dbce98700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db80cbd10 0x7f9db806e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f9dbce98700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db80c5a30 con 0x7f9db8063d40 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f9dbce98700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db817ab40 con 0x7f9db80cbd10 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f9dbce98700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db817ae20 con 0x7f9db80c6f30 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f9db659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 0x7f9db80c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 0x7f9db80c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41454/0 (socket says 172.21.15.7:41454) 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db659c700 1 -- 172.21.15.7:0/1105181143 learned_addr learned my addr 172.21.15.7:0/1105181143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc40af8e0 0x7fbdc416f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc2ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc406f2e0 0x7fbdc40c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc40af8e0 0x7fbdc416f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41450/0 (socket says 172.21.15.7:41450) 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc27fc700 1 -- 172.21.15.7:0/867592528 learned_addr learned my addr 172.21.15.7:0/867592528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc27fc700 1 -- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40cbeb0 msgr2=0x7fbdc416e040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc27fc700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40cbeb0 0x7fbdc416e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc27fc700 1 -- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc406f2e0 msgr2=0x7fbdc40c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc27fc700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc406f2e0 0x7fbdc40c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7fbdc27fc700 1 -- 172.21.15.7:0/867592528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc009a20 con 0x7fbdc40af8e0 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7f2fbdffb700 1 -- 172.21.15.7:0/3167777428 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2fc001e820 con 0x7f2fc806dd00 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f2fbdffb700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2fac0670b0 0x7f2fac069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f2fbdffb700 1 -- 172.21.15.7:0/3167777428 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fc0086950 con 0x7f2fc806dd00 2024-01-11T07:07:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db5d9b700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9db80c6f30 0x7f9db806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7fbdc27fc700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc40af8e0 0x7fbdc416f160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800b810 tx=0x7fbdb80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.232+0000 7f0d23a92700 1 Processor -- start 2024-01-11T07:07:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.233+0000 7f0d23a92700 1 -- start start 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f0d23a92700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d1c06e800 0x7f0d1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f0d23a92700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d1c06cb60 0x7f0d1c069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f0d23a92700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c0ca540 con 0x7f0d1c069670 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f0d23a92700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cb1c0 con 0x7f0d1c06e800 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f0d23a92700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c0ca820 con 0x7f0d1c06cb60 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f0d2202f700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0d1c069670 0x7f0d1c161d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40350/0 (socket says 172.21.15.7:40350) 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.234+0000 7f0d2202f700 1 -- 172.21.15.7:0/2878698874 learned_addr learned my addr 172.21.15.7:0/2878698874 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206479136 0 0) 0x7f0d1c0ca540 con 0x7f0d1c069670 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d08003570 con 0x7f0d1c069670 2024-01-11T07:07:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2182e700 1 --2- 172.21.15.7:0/2878698874 >> v2:172.21.15.7:3300/0 conn(0x7f0d1c06e800 0x7f0d1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7fbdaaffd700 1 -- 172.21.15.7:0/867592528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800f570 con 0x7fbdc40af8e0 2024-01-11T07:07:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7fbdc2ffd700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc406f2e0 0x7fbdc40c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7fbdaaffd700 1 -- 172.21.15.7:0/867592528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb800f6d0 con 0x7fbdc40af8e0 2024-01-11T07:07:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f2fcd64e700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2fac0670b0 0x7f2fac069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7fbdaaffd700 1 -- 172.21.15.7:0/867592528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800fa90 con 0x7fbdc40af8e0 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1150617572 0 0) 0x7f0d08003570 con 0x7f0d1c069670 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 >> v2:172.21.15.113:3300/0 conn(0x7f0d1c06cb60 msgr2=0x7f0d1c069130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 --2- 172.21.15.7:0/2878698874 >> v2:172.21.15.113:3300/0 conn(0x7f0d1c06cb60 0x7f0d1c069130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 >> v2:172.21.15.7:3300/0 conn(0x7f0d1c06e800 msgr2=0x7f0d1c068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 --2- 172.21.15.7:0/2878698874 >> v2:172.21.15.7:3300/0 conn(0x7f0d1c06e800 0x7f0d1c068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0cab00 con 0x7f0d1c069670 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f0d140017f0 con 0x7f0d1c069670 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 >> v1:172.21.15.113:6789/0 conn(0x7f0d1c069670 legacy=0x7f0d1c161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d08003ae0 0x7f0d08003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d08004810 0x7f0d08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d080052e0 0x7f0d0800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cb1c0 con 0x7f0d08003ae0 2024-01-11T07:07:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0ca820 con 0x7f0d080052e0 2024-01-11T07:07:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800c280 con 0x7f0d08004810 2024-01-11T07:07:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2202f700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d080052e0 0x7f0d0800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2182e700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d08003ae0 0x7f0d08003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db6d9d700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db80cbd10 0x7f9db806e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db5d9b700 1 -- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db80cbd10 msgr2=0x7f9db806e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db5d9b700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db80cbd10 0x7f9db806e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db5d9b700 1 -- 172.21.15.7:0/1105181143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 msgr2=0x7f9db80c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db5d9b700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 0x7f9db80c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7f9db5d9b700 1 -- 172.21.15.7:0/1105181143 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac009420 con 0x7f9db80c6f30 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f9db5d9b700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9db80c6f30 0x7f9db806e160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9dac015000 tx=0x7f9dac016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2202f700 1 -- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d08004810 msgr2=0x7f0d08004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2202f700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d08004810 0x7f0d08004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2202f700 1 -- 172.21.15.7:0/2878698874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d08003ae0 msgr2=0x7f0d08003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2202f700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d08003ae0 0x7f0d08003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2202f700 1 -- 172.21.15.7:0/2878698874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0cab00 con 0x7f0d080052e0 2024-01-11T07:07:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2202f700 1 -- 172.21.15.7:0/2878698874 reap_dead start 2024-01-11T07:07:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7f0d2202f700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d080052e0 0x7f0d0800ba60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400aa30 tx=0x7f0d1400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3ccba30fc7e768 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.235+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40c29c0 con 0x7fbdc40af8e0 2024-01-11T07:07:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.236+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc416f860 con 0x7fbdc40af8e0 2024-01-11T07:07:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.237+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400ae70 con 0x7f0d080052e0 2024-01-11T07:07:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.237+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1400c550 con 0x7f0d080052e0 2024-01-11T07:07:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.237+0000 7f0d2082c700 1 -- 172.21.15.7:0/2878698874 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400c910 con 0x7f0d080052e0 2024-01-11T07:07:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.237+0000 7f4daed12700 1 Processor -- start 2024-01-11T07:07:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.237+0000 7f4daed12700 1 -- start start 2024-01-11T07:07:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.237+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc404c3c0 con 0x7fbdc40af8e0 2024-01-11T07:07:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f2fcd64e700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2fac0670b0 0x7f2fac069560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8088010 tx=0x7f2fb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f0d23a92700 1 -- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d080052e0 msgr2=0x7f0d0800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f0d23a92700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d080052e0 0x7f0d0800ba60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400aa30 tx=0x7f0d1400a060 comp rx=0 tx=0).stop 2024-01-11T07:07:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f5e43bdd700 1 Processor -- start 2024-01-11T07:07:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f4daed12700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4da806e800 0x7f4da8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f4daed12700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4da806cb60 0x7f4da8069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f4daed12700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da80ca540 con 0x7f4da8069670 2024-01-11T07:07:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f4daed12700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da80cb1c0 con 0x7f4da806cb60 2024-01-11T07:07:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f4daed12700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da80ca820 con 0x7f4da806e800 2024-01-11T07:07:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f728d59a700 1 -- 172.21.15.7:0/3384199103 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800e070 con 0x7f72900698a0 2024-01-11T07:07:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f728d59a700 1 -- 172.21.15.7:0/3384199103 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728800bb20 con 0x7f72900698a0 2024-01-11T07:07:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f728d59a700 1 -- 172.21.15.7:0/3384199103 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288012430 con 0x7f72900698a0 2024-01-11T07:07:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f9da67fc700 1 -- 172.21.15.7:0/1105181143 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac01cb80 con 0x7f9db80c6f30 2024-01-11T07:07:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db817b100 con 0x7f9db80c6f30 2024-01-11T07:07:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db817b900 con 0x7f9db80c6f30 2024-01-11T07:07:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f5e43bdd700 1 -- start start 2024-01-11T07:07:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f4dad2af700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4da8069670 0x7f4da8161d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40366/0 (socket says 172.21.15.7:40366) 2024-01-11T07:07:10.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f4dad2af700 1 -- 172.21.15.7:0/2613810702 learned_addr learned my addr 172.21.15.7:0/2613810702 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020631005 0 0) 0x7f4da80ca540 con 0x7f4da8069670 2024-01-11T07:07:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d90003570 con 0x7f4da8069670 2024-01-11T07:07:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4dacaae700 1 --2- 172.21.15.7:0/2613810702 >> v2:172.21.15.113:3300/0 conn(0x7f4da806e800 0x7f4da8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7fbdaaffd700 1 -- 172.21.15.7:0/867592528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbdb801a460 con 0x7fbdc40af8e0 2024-01-11T07:07:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.238+0000 7fbdaaffd700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdb0066fb0 0x7fbdb0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7fbdaaffd700 1 -- 172.21.15.7:0/867592528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbdb8084260 con 0x7fbdc40af8e0 2024-01-11T07:07:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7fbdc2ffd700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdb0066fb0 0x7fbdb0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f2fbdffb700 1 -- 172.21.15.7:0/3167777428 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fc0053c20 con 0x7f2fc806dd00 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da7fff700 1 --2- 172.21.15.7:0/2613810702 >> v2:172.21.15.7:3300/0 conn(0x7f4da806cb60 0x7f4da8069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 676253445 0 0) 0x7f4d90003570 con 0x7f4da8069670 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 >> v2:172.21.15.113:3300/0 conn(0x7f4da806e800 msgr2=0x7f4da8068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 --2- 172.21.15.7:0/2613810702 >> v2:172.21.15.113:3300/0 conn(0x7f4da806e800 0x7f4da8068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 >> v2:172.21.15.7:3300/0 conn(0x7f4da806cb60 msgr2=0x7f4da8069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 --2- 172.21.15.7:0/2613810702 >> v2:172.21.15.7:3300/0 conn(0x7f4da806cb60 0x7f4da8069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80cab00 con 0x7f4da8069670 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f4d9c001470 con 0x7f4da8069670 2024-01-11T07:07:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 >> v1:172.21.15.113:6789/0 conn(0x7f4da8069670 legacy=0x7f4da8161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d90003ae0 0x7f4d90003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90004810 0x7f4d90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d900051c0 0x7f4d9000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80cb1c0 con 0x7f4d90004810 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80ca820 con 0x7f4d900051c0 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000c100 con 0x7f4d90003ae0 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f7294dc5700 1 -- 172.21.15.7:0/3384199103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900698a0 msgr2=0x7f72900ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f7294dc5700 1 --2- 172.21.15.7:0/3384199103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900698a0 0x7f72900ae7f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7288009e60 tx=0x7f7288009780 comp rx=0 tx=0).stop 2024-01-11T07:07:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f7294dc5700 1 -- 172.21.15.7:0/3384199103 shutdown_connections 2024-01-11T07:07:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f7294dc5700 1 --2- 172.21.15.7:0/3384199103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900698a0 0x7f72900ae7f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f7294dc5700 1 --2- 172.21.15.7:0/3384199103 >> v2:172.21.15.113:3300/0 conn(0x7f729006a2f0 0x7f7290069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.239+0000 7f7294dc5700 1 -- 172.21.15.7:0/3384199103 >> 172.21.15.7:0/3384199103 conn(0x7f7290063230 msgr2=0x7f7290063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f9da67fc700 1 -- 172.21.15.7:0/1105181143 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac01cce0 con 0x7f9db80c6f30 2024-01-11T07:07:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f9da67fc700 1 -- 172.21.15.7:0/1105181143 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac019640 con 0x7f9db80c6f30 2024-01-11T07:07:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4dad2af700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d900051c0 0x7f4d9000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f0d23a92700 1 -- 172.21.15.7:0/2878698874 shutdown_connections 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f0d23a92700 1 --2- 172.21.15.7:0/2878698874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d080052e0 0x7f0d0800ba60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f0d23a92700 1 -- 172.21.15.7:0/2878698874 >> 172.21.15.7:0/2878698874 conn(0x7f0d1c063230 msgr2=0x7f0d1c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f5e43bdd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e3c06e340 0x7f5e3c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f5e43bdd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c15a070 0x7f5e3c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f5e43bdd700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0d18c0 con 0x7f5e3c159ca0 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f5e43bdd700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0d0010 con 0x7f5e3c06e340 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f5e43bdd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0cf400 con 0x7f5e3c15a070 2024-01-11T07:07:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f5e41979700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5e3c159ca0 0x7f5e3c06dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40370/0 (socket says 172.21.15.7:40370) 2024-01-11T07:07:10.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f5e41979700 1 -- 172.21.15.7:0/4043030305 learned_addr learned my addr 172.21.15.7:0/4043030305 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:10.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f0d23a92700 1 -- 172.21.15.7:0/2878698874 shutdown_connections 2024-01-11T07:07:10.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4dacaae700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d90003ae0 0x7f4d90003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4da7fff700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90004810 0x7f4d90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4dacaae700 1 -- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d900051c0 msgr2=0x7f4d9000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4dacaae700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d900051c0 0x7f4d9000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4dacaae700 1 -- 172.21.15.7:0/2613810702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90004810 msgr2=0x7f4d90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4dacaae700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90004810 0x7f4d90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4dacaae700 1 -- 172.21.15.7:0/2613810702 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80cab00 con 0x7f4d90003ae0 2024-01-11T07:07:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f4dad2af700 1 -- 172.21.15.7:0/2613810702 reap_dead start 2024-01-11T07:07:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f4dacaae700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d90003ae0 0x7f4d90003ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4d98006f00 tx=0x7f4d980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d9fc78333718f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f5e4217a700 1 --2- 172.21.15.7:0/4043030305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c15a070 0x7f5e3c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f9da67fc700 1 -- 172.21.15.7:0/1105181143 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9dac019a70 con 0x7f9db80c6f30 2024-01-11T07:07:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db817b620 con 0x7f9db80c6f30 2024-01-11T07:07:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f5e41178700 1 --2- 172.21.15.7:0/4043030305 >> v2:172.21.15.113:3300/0 conn(0x7f5e3c06e340 0x7f5e3c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f5e4217a700 1 -- 172.21.15.7:0/4043030305 >> v2:172.21.15.113:3300/0 conn(0x7f5e3c06e340 msgr2=0x7f5e3c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f5e4217a700 1 --2- 172.21.15.7:0/4043030305 >> v2:172.21.15.113:3300/0 conn(0x7f5e3c06e340 0x7f5e3c0ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f5e4217a700 1 -- 172.21.15.7:0/4043030305 >> v1:172.21.15.113:6789/0 conn(0x7f5e3c159ca0 legacy=0x7f5e3c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f5e4217a700 1 -- 172.21.15.7:0/4043030305 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0cf6e0 con 0x7f5e3c15a070 2024-01-11T07:07:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f9da67fc700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.242+0000 7f5e4217a700 1 --2- 172.21.15.7:0/4043030305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c15a070 0x7f5e3c0ced30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009e60 tx=0x7f5e34009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850ef1639bda054a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.242+0000 7f9da67fc700 1 -- 172.21.15.7:0/1105181143 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9dac085670 con 0x7f9db80c6f30 2024-01-11T07:07:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f7294dc5700 1 -- 172.21.15.7:0/3384199103 shutdown_connections 2024-01-11T07:07:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800e710 con 0x7f4d90003ae0 2024-01-11T07:07:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9800e870 con 0x7f4d90003ae0 2024-01-11T07:07:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.241+0000 7f4da77fe700 1 -- 172.21.15.7:0/2613810702 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800ec30 con 0x7f4d90003ae0 2024-01-11T07:07:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.242+0000 7f4daed12700 1 -- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d90003ae0 msgr2=0x7f4d90003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.242+0000 7f4daed12700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d90003ae0 0x7f4d90003ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4d98006f00 tx=0x7f4d980091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.240+0000 7f0d23a92700 1 -- 172.21.15.7:0/2878698874 wait complete. 2024-01-11T07:07:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.242+0000 7f0d23a92700 1 Processor -- start 2024-01-11T07:07:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.242+0000 7f9db659c700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.243+0000 7f9db659c700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d98066f80 0x7f9d98069430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9da8000c00 tx=0x7f9da8007f10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.244+0000 7fbdc2ffd700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdb0066fb0 0x7fbdb0069460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4001820 tx=0x7fbdb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.242+0000 7f7294dc5700 1 -- 172.21.15.7:0/3384199103 wait complete. 2024-01-11T07:07:10.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.244+0000 7f7294dc5700 1 Processor -- start 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.244+0000 7f0d23a92700 1 -- start start 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.245+0000 7f0d23a92700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d080052e0 0x7f0d1c14c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.245+0000 7f0d23a92700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1c0acd40 0x7f0d1c0ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.245+0000 7f0d23a92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c14e5a0 0x7f0d1c14ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.245+0000 7f0d23a92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c148390 con 0x7f0d1c14e5a0 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.245+0000 7f0d23a92700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0b05a0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.245+0000 7f0d23a92700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0ada00 con 0x7f0d080052e0 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2102d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1c0acd40 0x7f0d1c0ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2102d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1c0acd40 0x7f0d1c0ad1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36442/0 (socket says 172.21.15.7:36442) 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2102d700 1 -- 172.21.15.7:0/1937056258 learned_addr learned my addr 172.21.15.7:0/1937056258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2182e700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d080052e0 0x7f0d1c14c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f9da67fc700 1 -- 172.21.15.7:0/1105181143 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9db817b620 con 0x7f9db80c6f30 2024-01-11T07:07:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2102d700 1 -- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d080052e0 msgr2=0x7f0d1c14c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2102d700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d080052e0 0x7f0d1c14c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2102d700 1 -- 172.21.15.7:0/1937056258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c14e5a0 msgr2=0x7f0d1c14ea40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T07:07:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2102d700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c14e5a0 0x7f0d1c14ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f0d2102d700 1 -- 172.21.15.7:0/1937056258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0adce0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f7294dc5700 1 -- start start 2024-01-11T07:07:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f7294dc5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72900698a0 0x7f729006df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f7294dc5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729006a2f0 0x7f72900c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f7294dc5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900ca5f0 0x7f72900c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f7294dc5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729006dae0 con 0x7f72900698a0 2024-01-11T07:07:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f7294dc5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729006ced0 con 0x7f729006a2f0 2024-01-11T07:07:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f7294dc5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729006d1b0 con 0x7f72900ca5f0 2024-01-11T07:07:10.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f0d2102d700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1c0acd40 0x7f0d1c0ad1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c06cf80 tx=0x7f0d0c009060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f728dd9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729006a2f0 0x7f72900c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f728dd9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729006a2f0 0x7f72900c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36444/0 (socket says 172.21.15.7:36444) 2024-01-11T07:07:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f728dd9b700 1 -- 172.21.15.7:0/1711400935 learned_addr learned my addr 172.21.15.7:0/1711400935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7f728ed9d700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900ca5f0 0x7f72900c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.249+0000 7f728dd9b700 1 -- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900ca5f0 msgr2=0x7f72900c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.249+0000 7f728dd9b700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900ca5f0 0x7f72900c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.249+0000 7f728dd9b700 1 -- 172.21.15.7:0/1711400935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72900698a0 msgr2=0x7f729006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.249+0000 7f728dd9b700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72900698a0 0x7f729006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.249+0000 7f728dd9b700 1 -- 172.21.15.7:0/1711400935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840094f0 con 0x7f729006a2f0 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.251+0000 7f728dd9b700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729006a2f0 0x7f72900c3f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f728400ec50 tx=0x7f7284009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.251+0000 7f727e7fc700 1 -- 172.21.15.7:0/1711400935 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400c920 con 0x7f729006a2f0 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.251+0000 7f727e7fc700 1 -- 172.21.15.7:0/1711400935 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728400ca80 con 0x7f729006a2f0 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.251+0000 7f727e7fc700 1 -- 172.21.15.7:0/1711400935 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400ce00 con 0x7f729006a2f0 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.251+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7288009a20 con 0x7f729006a2f0 2024-01-11T07:07:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.251+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729006d770 con 0x7f729006a2f0 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.252+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729004c3c0 con 0x7f729006a2f0 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.253+0000 7f5e40977700 1 -- 172.21.15.7:0/4043030305 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400e070 con 0x7f5e3c15a070 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.253+0000 7f5e40977700 1 -- 172.21.15.7:0/4043030305 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3400bb60 con 0x7f5e3c15a070 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.253+0000 7f5e40977700 1 -- 172.21.15.7:0/4043030305 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34012430 con 0x7f5e3c15a070 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4043030305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c15a070 msgr2=0x7f5e3c0ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4043030305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c15a070 0x7f5e3c0ced30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009e60 tx=0x7f5e34009780 comp rx=0 tx=0).stop 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4043030305 shutdown_connections 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4043030305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c15a070 0x7f5e3c0ced30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4043030305 >> v2:172.21.15.113:3300/0 conn(0x7f5e3c06e340 0x7f5e3c0ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4043030305 >> 172.21.15.7:0/4043030305 conn(0x7f5e3c063cc0 msgr2=0x7f5e3c0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.248+0000 7fbdaaffd700 1 -- 172.21.15.7:0/867592528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdb80514b0 con 0x7fbdc40af8e0 2024-01-11T07:07:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f4daed12700 1 -- 172.21.15.7:0/2613810702 shutdown_connections 2024-01-11T07:07:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f4daed12700 1 --2- 172.21.15.7:0/2613810702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d90003ae0 0x7f4d90003ef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.246+0000 7f4daed12700 1 -- 172.21.15.7:0/2613810702 >> 172.21.15.7:0/2613810702 conn(0x7f4da8063230 msgr2=0x7f4da8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.249+0000 7f0d11ffb700 1 -- 172.21.15.7:0/1937056258 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00be90 con 0x7f0d1c0acd40 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.249+0000 7f0d11ffb700 1 -- 172.21.15.7:0/1937056258 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00f510 con 0x7f0d1c0acd40 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.249+0000 7f0d11ffb700 1 -- 172.21.15.7:0/1937056258 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00f8d0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.250+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c0adfc0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.250+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c0ad6f0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.251+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c169ea0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.253+0000 7f0d11ffb700 1 -- 172.21.15.7:0/1937056258 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0d0c00fc80 con 0x7f0d1c0acd40 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.253+0000 7f0d11ffb700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d00066f30 0x7f0d000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.253+0000 7f0d11ffb700 1 -- 172.21.15.7:0/1937056258 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d0c0846c0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.253+0000 7f0d2182e700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d00066f30 0x7f0d000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f0d2182e700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d00066f30 0x7f0d000693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800b0b0 tx=0x7f0d1800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f4daed12700 1 -- 172.21.15.7:0/2613810702 shutdown_connections 2024-01-11T07:07:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.254+0000 7f4daed12700 1 -- 172.21.15.7:0/2613810702 wait complete. 2024-01-11T07:07:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.256+0000 7f4daed12700 1 Processor -- start 2024-01-11T07:07:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.257+0000 7f0d11ffb700 1 -- 172.21.15.7:0/1937056258 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d0c051a90 con 0x7f0d1c0acd40 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.257+0000 7f4daed12700 1 -- start start 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4daed12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90003ae0 0x7f4da814d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4daed12700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da80b10b0 0x7f4da80c1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4daed12700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da80b1520 0x7f4da80c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4daed12700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80b2380 con 0x7f4d90003ae0 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4daed12700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80b1ae0 con 0x7f4da80b1520 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4daed12700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80b1dc0 con 0x7f4da80b10b0 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4dacaae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90003ae0 0x7f4da814d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4dacaae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90003ae0 0x7f4da814d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41486/0 (socket says 172.21.15.7:41486) 2024-01-11T07:07:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.258+0000 7f4dacaae700 1 -- 172.21.15.7:0/231722190 learned_addr learned my addr 172.21.15.7:0/231722190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.259+0000 7f4dacaae700 1 -- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da80b10b0 msgr2=0x7f4da80c1620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T07:07:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.259+0000 7f4dacaae700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da80b10b0 0x7f4da80c1620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.259+0000 7f4dacaae700 1 -- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da80b1520 msgr2=0x7f4da80c1b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T07:07:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.259+0000 7f4dacaae700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da80b1520 0x7f4da80c1b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.259+0000 7f4dacaae700 1 -- 172.21.15.7:0/231722190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80b20a0 con 0x7f4d90003ae0 2024-01-11T07:07:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.259+0000 7f4dacaae700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90003ae0 0x7f4da814d8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800eda0 tx=0x7f4d9801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.260+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4043030305 shutdown_connections 2024-01-11T07:07:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.260+0000 7f727e7fc700 1 -- 172.21.15.7:0/1711400935 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7284013680 con 0x7f729006a2f0 2024-01-11T07:07:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.260+0000 7f727e7fc700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7270067080 0x7f7270069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.260+0000 7f727e7fc700 1 -- 172.21.15.7:0/1711400935 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72840850a0 con 0x7f729006a2f0 2024-01-11T07:07:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.261+0000 7f728e59c700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7270067080 0x7f7270069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.260+0000 7f4da4ff9700 1 -- 172.21.15.7:0/231722190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98022070 con 0x7f4d90003ae0 2024-01-11T07:07:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.260+0000 7f4da4ff9700 1 -- 172.21.15.7:0/231722190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98013cb0 con 0x7f4d90003ae0 2024-01-11T07:07:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.260+0000 7f4da4ff9700 1 -- 172.21.15.7:0/231722190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800e710 con 0x7f4d90003ae0 2024-01-11T07:07:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.261+0000 7f728e59c700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7270067080 0x7f7270069530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7280009740 tx=0x7f7280006c60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.262+0000 7f727e7fc700 1 -- 172.21.15.7:0/1711400935 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7284052370 con 0x7f729006a2f0 2024-01-11T07:07:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.261+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da80b2380 con 0x7f4d90003ae0 2024-01-11T07:07:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.261+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da815d0f0 con 0x7f4d90003ae0 2024-01-11T07:07:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.264+0000 7f4da4ff9700 1 -- 172.21.15.7:0/231722190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4d9801c950 con 0x7f4d90003ae0 2024-01-11T07:07:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.264+0000 7f4da4ff9700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d8c067050 0x7f4d8c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.265+0000 7f4da4ff9700 1 -- 172.21.15.7:0/231722190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d98085e10 con 0x7f4d90003ae0 2024-01-11T07:07:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.266+0000 7f4da7fff700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d8c067050 0x7f4d8c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.264+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4043030305 wait complete. 2024-01-11T07:07:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.266+0000 7f5e43bdd700 1 Processor -- start 2024-01-11T07:07:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.266+0000 7f4da7fff700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d8c067050 0x7f4d8c069500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4da80c9b70 tx=0x7f4da0009630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.266+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da804c3c0 con 0x7f4d90003ae0 2024-01-11T07:07:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.270+0000 7f5e43bdd700 1 -- start start 2024-01-11T07:07:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.271+0000 7f5e43bdd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e3c06e340 0x7f5e3c06dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.271+0000 7f5e43bdd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c159ca0 0x7f5e3c17a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.271+0000 7f5e43bdd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e3c15a070 0x7f5e3c17a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.271+0000 7f5e43bdd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c06e130 con 0x7f5e3c15a070 2024-01-11T07:07:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.271+0000 7f5e43bdd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b53e0 con 0x7f5e3c06e340 2024-01-11T07:07:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.271+0000 7f5e43bdd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c17b0d0 con 0x7f5e3c159ca0 2024-01-11T07:07:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c159ca0 0x7f5e3c17a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c159ca0 0x7f5e3c17a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:39128/0 (socket says 172.21.15.7:39128) 2024-01-11T07:07:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 -- 172.21.15.7:0/4218884534 learned_addr learned my addr 172.21.15.7:0/4218884534 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 -- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e3c06e340 msgr2=0x7f5e3c06dbf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T07:07:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e3c06e340 0x7f5e3c06dbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 -- 172.21.15.7:0/4218884534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e3c15a070 msgr2=0x7f5e3c17a840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-11T07:07:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e3c15a070 0x7f5e3c17a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 -- 172.21.15.7:0/4218884534 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34009a20 con 0x7f5e3c159ca0 2024-01-11T07:07:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e41178700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c159ca0 0x7f5e3c17a300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00ec80 tx=0x7f5e2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.272+0000 7f5e31ffb700 1 -- 172.21.15.7:0/4218884534 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00c9d0 con 0x7f5e3c159ca0 2024-01-11T07:07:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.273+0000 7f5e31ffb700 1 -- 172.21.15.7:0/4218884534 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c00cb30 con 0x7f5e3c159ca0 2024-01-11T07:07:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.273+0000 7f5e31ffb700 1 -- 172.21.15.7:0/4218884534 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c013470 con 0x7f5e3c159ca0 2024-01-11T07:07:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.276+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0b4860 con 0x7f5e3c159ca0 2024-01-11T07:07:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.276+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c0b4e20 con 0x7f5e3c159ca0 2024-01-11T07:07:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.277+0000 7f4da4ff9700 1 -- 172.21.15.7:0/231722190 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d98053060 con 0x7f4d90003ae0 2024-01-11T07:07:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.278+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c04c3c0 con 0x7f5e3c159ca0 2024-01-11T07:07:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.280+0000 7f5e31ffb700 1 -- 172.21.15.7:0/4218884534 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5e2c013860 con 0x7f5e3c159ca0 2024-01-11T07:07:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.280+0000 7f5e31ffb700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e20067080 0x7f5e20069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.281+0000 7f5e31ffb700 1 -- 172.21.15.7:0/4218884534 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e2c085050 con 0x7f5e3c159ca0 2024-01-11T07:07:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.283+0000 7f5e31ffb700 1 -- 172.21.15.7:0/4218884534 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e2c052320 con 0x7f5e3c159ca0 2024-01-11T07:07:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.287+0000 7f5e41979700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e20067080 0x7f5e20069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:10.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.314+0000 7f5e41979700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e20067080 0x7f5e20069530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e380058d0 tx=0x7f5e38007ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.599+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7c40005550 con 0x7f7c601592b0 2024-01-11T07:07:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.600+0000 7f7c5cff9700 1 -- 172.21.15.7:0/4159722440 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c58018030 con 0x7f7c601592b0 2024-01-11T07:07:10.602 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-01-11T07:07:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.603+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c48068d50 msgr2=0x7f7c4806b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.603+0000 7f7c66a90700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c48068d50 0x7f7c4806b200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c601675f0 tx=0x7f7c50015000 comp rx=0 tx=0).stop 2024-01-11T07:07:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.603+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c601592b0 msgr2=0x7f7c601679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.603+0000 7f7c66a90700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c601592b0 0x7f7c601679f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009ec0 tx=0x7f7c5800b040 comp rx=0 tx=0).stop 2024-01-11T07:07:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.604+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 shutdown_connections 2024-01-11T07:07:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.604+0000 7f7c66a90700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7c48068d50 0x7f7c4806b200 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.604+0000 7f7c66a90700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c60169a40 0x7f7c600c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.604+0000 7f7c66a90700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c601592b0 0x7f7c601679f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.604+0000 7f7c66a90700 1 --2- 172.21.15.7:0/4159722440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7c60166600 0x7f7c60169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.604+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 >> 172.21.15.7:0/4159722440 conn(0x7f7c60063230 msgr2=0x7f7c6006d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.604+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 shutdown_connections 2024-01-11T07:07:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.605+0000 7f7c66a90700 1 -- 172.21.15.7:0/4159722440 wait complete. 2024-01-11T07:07:10.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-01-11T07:07:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.650+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2fa8005820 con 0x7f2fc806dd00 2024-01-11T07:07:10.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.653+0000 7f2fbdffb700 1 -- 172.21.15.7:0/3167777428 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2fc0057240 con 0x7f2fc806dd00 2024-01-11T07:07:10.652 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-01-11T07:07:10.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2fac0670b0 msgr2=0x7f2fac069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2fac0670b0 0x7f2fac069560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8088010 tx=0x7f2fb8008040 comp rx=0 tx=0).stop 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 msgr2=0x7f2fc80ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80ca8f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2fc815b010 tx=0x7f2fc0017000 comp rx=0 tx=0).stop 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 shutdown_connections 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2fac0670b0 0x7f2fac069560 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8088010 tx=0x7f2fb8008040 comp rx=0 tx=0).stop 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fc8159ff0 0x7f2fc80c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2fc806dd00 0x7f2fc80ca8f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 --2- 172.21.15.7:0/3167777428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc806e0d0 0x7f2fc80c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 >> 172.21.15.7:0/3167777428 conn(0x7f2fc80680e0 msgr2=0x7f2fc8063860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 shutdown_connections 2024-01-11T07:07:10.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.663+0000 7f2fd00b3700 1 -- 172.21.15.7:0/3167777428 wait complete. 2024-01-11T07:07:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.670+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4da80b2600 con 0x7f4d90003ae0 2024-01-11T07:07:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.671+0000 7f4da4ff9700 1 -- 172.21.15.7:0/231722190 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d9802b020 con 0x7f4d90003ae0 2024-01-11T07:07:10.670 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-01-11T07:07:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.673+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9db817b620 con 0x7f9db80c6f30 2024-01-11T07:07:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.673+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d8c067050 msgr2=0x7f4d8c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.673+0000 7f4daed12700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d8c067050 0x7f4d8c069500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4da80c9b70 tx=0x7f4da0009630 comp rx=0 tx=0).stop 2024-01-11T07:07:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.673+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90003ae0 msgr2=0x7f4da814d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.673+0000 7f4daed12700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90003ae0 0x7f4da814d8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800eda0 tx=0x7f4d9801a000 comp rx=0 tx=0).stop 2024-01-11T07:07:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.674+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 shutdown_connections 2024-01-11T07:07:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.674+0000 7f4daed12700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d8c067050 0x7f4d8c069500 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.674+0000 7f4daed12700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da80b1520 0x7f4da80c1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.674+0000 7f4daed12700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d90003ae0 0x7f4da814d8f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.674+0000 7f4daed12700 1 --2- 172.21.15.7:0/231722190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4da80b10b0 0x7f4da80c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.674+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 >> 172.21.15.7:0/231722190 conn(0x7f4da8063230 msgr2=0x7f4da80c8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.674 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-01-11T07:07:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.674+0000 7f9da67fc700 1 -- 172.21.15.7:0/1105181143 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9db817b620 con 0x7f9db80c6f30 2024-01-11T07:07:10.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.676+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 shutdown_connections 2024-01-11T07:07:10.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.676+0000 7f4daed12700 1 -- 172.21.15.7:0/231722190 wait complete. 2024-01-11T07:07:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.679+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d98066f80 msgr2=0x7f9d98069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.679+0000 7f9dbce98700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d98066f80 0x7f9d98069430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9da8000c00 tx=0x7f9da8007f10 comp rx=0 tx=0).stop 2024-01-11T07:07:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.679+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9db80c6f30 msgr2=0x7f9db806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.679+0000 7f9dbce98700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9db80c6f30 0x7f9db806e160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9dac015000 tx=0x7f9dac016000 comp rx=0 tx=0).stop 2024-01-11T07:07:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.679+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbdc40c2e60 con 0x7fbdc40af8e0 2024-01-11T07:07:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.680+0000 7fbdaaffd700 1 -- 172.21.15.7:0/867592528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbdb8054ad0 con 0x7fbdc40af8e0 2024-01-11T07:07:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.680+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 shutdown_connections 2024-01-11T07:07:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.680+0000 7f9dbce98700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.680+0000 7f9dbce98700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9db80cbd10 0x7f9db806e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.680+0000 7f9dbce98700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9db8063d40 0x7f9db80c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.680+0000 7f9dbce98700 1 --2- 172.21.15.7:0/1105181143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9db80c6f30 0x7f9db806e160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.680+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 >> 172.21.15.7:0/1105181143 conn(0x7f9db8063230 msgr2=0x7f9db80c1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.681+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0d1c04c3c0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.682+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 shutdown_connections 2024-01-11T07:07:10.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.683+0000 7f9dbce98700 1 -- 172.21.15.7:0/1105181143 wait complete. 2024-01-11T07:07:10.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.683+0000 7f0d11ffb700 1 -- 172.21.15.7:0/1937056258 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d0c0550b0 con 0x7f0d1c0acd40 2024-01-11T07:07:10.682 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-01-11T07:07:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.684+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f729015f290 con 0x7f729006a2f0 2024-01-11T07:07:10.684 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-01-11T07:07:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.685+0000 7f727e7fc700 1 -- 172.21.15.7:0/1711400935 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7284055990 con 0x7f729006a2f0 2024-01-11T07:07:10.685 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.685+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdb0066fb0 msgr2=0x7fbdb0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.685+0000 7fbdc958a700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdb0066fb0 0x7fbdb0069460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4001820 tx=0x7fbdb4009040 comp rx=0 tx=0).stop 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc40af8e0 msgr2=0x7fbdc416f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc40af8e0 0x7fbdc416f160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800b810 tx=0x7fbdb80091d0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 shutdown_connections 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdb0066fb0 0x7fbdb0069460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdc40cbeb0 0x7fbdc416e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdc406f2e0 0x7fbdc40c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 --2- 172.21.15.7:0/867592528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc40af8e0 0x7fbdc416f160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 >> 172.21.15.7:0/867592528 conn(0x7fbdc4063230 msgr2=0x7fbdc40ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 shutdown_connections 2024-01-11T07:07:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.686+0000 7fbdc958a700 1 -- 172.21.15.7:0/867592528 wait complete. 2024-01-11T07:07:10.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-01-11T07:07:10.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7270067080 msgr2=0x7f7270069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f7294dc5700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7270067080 0x7f7270069530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7280009740 tx=0x7f7280006c60 comp rx=0 tx=0).stop 2024-01-11T07:07:10.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729006a2f0 msgr2=0x7f72900c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f7294dc5700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729006a2f0 0x7f72900c3f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f728400ec50 tx=0x7f7284009250 comp rx=0 tx=0).stop 2024-01-11T07:07:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d00066f30 msgr2=0x7f0d000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d00066f30 0x7f0d000693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800b0b0 tx=0x7f0d1800b040 comp rx=0 tx=0).stop 2024-01-11T07:07:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1c0acd40 msgr2=0x7f0d1c0ad1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1c0acd40 0x7f0d1c0ad1b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c06cf80 tx=0x7f0d0c009060 comp rx=0 tx=0).stop 2024-01-11T07:07:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 shutdown_connections 2024-01-11T07:07:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d00066f30 0x7f0d000693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d1c14e5a0 0x7f0d1c14ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d080052e0 0x7f0d1c14c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 --2- 172.21.15.7:0/1937056258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d1c0acd40 0x7f0d1c0ad1b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 >> 172.21.15.7:0/1937056258 conn(0x7f0d1c063230 msgr2=0x7f0d1c1607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 shutdown_connections 2024-01-11T07:07:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.689+0000 7f0d23a92700 1 -- 172.21.15.7:0/1937056258 wait complete. 2024-01-11T07:07:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.691+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 shutdown_connections 2024-01-11T07:07:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.691+0000 7f7294dc5700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7270067080 0x7f7270069530 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.691+0000 7f7294dc5700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72900ca5f0 0x7f72900c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.691+0000 7f7294dc5700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72900698a0 0x7f729006df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.691+0000 7f7294dc5700 1 --2- 172.21.15.7:0/1711400935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f729006a2f0 0x7f72900c3f30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.691+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 >> 172.21.15.7:0/1711400935 conn(0x7f7290063230 msgr2=0x7f72900c2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.691+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 shutdown_connections 2024-01-11T07:07:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.691+0000 7f7294dc5700 1 -- 172.21.15.7:0/1711400935 wait complete. 2024-01-11T07:07:10.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-01-11T07:07:10.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.697+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5e3c0cf9c0 con 0x7f5e3c159ca0 2024-01-11T07:07:10.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.698+0000 7f5e31ffb700 1 -- 172.21.15.7:0/4218884534 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5e2c055940 con 0x7f5e3c159ca0 2024-01-11T07:07:10.698 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-01-11T07:07:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e20067080 msgr2=0x7f5e20069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e20067080 0x7f5e20069530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e380058d0 tx=0x7f5e38007ac0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c159ca0 msgr2=0x7f5e3c17a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c159ca0 0x7f5e3c17a300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00ec80 tx=0x7f5e2c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 shutdown_connections 2024-01-11T07:07:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e20067080 0x7f5e20069530 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5e380058d0 tx=0x7f5e38007ac0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e3c15a070 0x7f5e3c17a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e3c06e340 0x7f5e3c06dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 --2- 172.21.15.7:0/4218884534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3c159ca0 0x7f5e3c17a300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.701+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 >> 172.21.15.7:0/4218884534 conn(0x7f5e3c063cc0 msgr2=0x7f5e3c0c6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.702+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 shutdown_connections 2024-01-11T07:07:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:10.702+0000 7f5e43bdd700 1 -- 172.21.15.7:0/4218884534 wait complete. 2024-01-11T07:07:10.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-01-11T07:07:10.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-01-11T07:07:10.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-01-11T07:07:10.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-01-11T07:07:10.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-01-11T07:07:11.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-11T07:07:11.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-11T07:07:11.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-11T07:07:11.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-11T07:07:11.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-11T07:07:11.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-11T07:07:11.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-11T07:07:11.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-11T07:07:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.792+0000 7f26126cb700 1 Processor -- start 2024-01-11T07:07:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.792+0000 7f26126cb700 1 -- start start 2024-01-11T07:07:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.794+0000 7f26126cb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7b30 0x7f260c1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.794+0000 7f26126cb700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f260c0c7f00 0x7f260c153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.794+0000 7f26126cb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260c0cbb20 con 0x7f260c0cafb0 2024-01-11T07:07:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.794+0000 7f26126cb700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260c157880 con 0x7f260c0c7f00 2024-01-11T07:07:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.794+0000 7f26126cb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c069a20 con 0x7f260c0c7b30 2024-01-11T07:07:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.795+0000 7f260b7fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f260c0c7f00 0x7f260c153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.795+0000 7f260b7fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f260c0c7f00 0x7f260c153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36448/0 (socket says 172.21.15.7:36448) 2024-01-11T07:07:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.795+0000 7f260b7fe700 1 -- 172.21.15.7:0/668566109 learned_addr learned my addr 172.21.15.7:0/668566109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.795+0000 7f260b7fe700 1 -- 172.21.15.7:0/668566109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7b30 msgr2=0x7f260c1537b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-11T07:07:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.795+0000 7f260b7fe700 1 --2- 172.21.15.7:0/668566109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7b30 0x7f260c1537b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.795+0000 7f260b7fe700 1 -- 172.21.15.7:0/668566109 >> v1:172.21.15.7:6789/0 conn(0x7f260c0cafb0 legacy=0x7f260c0cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.795+0000 7f260b7fe700 1 -- 172.21.15.7:0/668566109 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0cbd50 con 0x7f260c0c7f00 2024-01-11T07:07:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.796+0000 7f260b7fe700 1 --2- 172.21.15.7:0/668566109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7f00 0x7f260c153cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00bc40 tx=0x7f25fc00b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d0cdf4a7bfc0a7c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.796+0000 7f260affd700 1 -- 172.21.15.7:0/668566109 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00db00 con 0x7f260c0c7f00 2024-01-11T07:07:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.796+0000 7f260affd700 1 -- 172.21.15.7:0/668566109 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc00dc60 con 0x7f260c0c7f00 2024-01-11T07:07:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.796+0000 7f260affd700 1 -- 172.21.15.7:0/668566109 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc017410 con 0x7f260c0c7f00 2024-01-11T07:07:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.797+0000 7f26126cb700 1 -- 172.21.15.7:0/668566109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7f00 msgr2=0x7f260c153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.797+0000 7f26126cb700 1 --2- 172.21.15.7:0/668566109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7f00 0x7f260c153cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00bc40 tx=0x7f25fc00b4e0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.805+0000 7f26126cb700 1 -- 172.21.15.7:0/668566109 shutdown_connections 2024-01-11T07:07:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.805+0000 7f26126cb700 1 --2- 172.21.15.7:0/668566109 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7b30 0x7f260c1537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.805+0000 7f26126cb700 1 --2- 172.21.15.7:0/668566109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7f00 0x7f260c153cf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.805+0000 7f26126cb700 1 -- 172.21.15.7:0/668566109 >> 172.21.15.7:0/668566109 conn(0x7f260c0cd090 msgr2=0x7f260c0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.805+0000 7f26126cb700 1 -- 172.21.15.7:0/668566109 shutdown_connections 2024-01-11T07:07:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.805+0000 7f26126cb700 1 -- 172.21.15.7:0/668566109 wait complete. 2024-01-11T07:07:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.807+0000 7f26126cb700 1 Processor -- start 2024-01-11T07:07:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.808+0000 7f26126cb700 1 -- start start 2024-01-11T07:07:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.809+0000 7f26126cb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7b30 0x7f260c172510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.809+0000 7f26126cb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7f00 0x7f260c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.809+0000 7f26126cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f260c0cafb0 0x7f260c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.809+0000 7f26126cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c06daf0 con 0x7f260c0cafb0 2024-01-11T07:07:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.809+0000 7f26126cb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260c06e630 con 0x7f260c0c7b30 2024-01-11T07:07:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.809+0000 7f26126cb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260c1557b0 con 0x7f260c0c7f00 2024-01-11T07:07:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.809+0000 7f260b7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7f00 0x7f260c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260bfff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7b30 0x7f260c172510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260bfff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7b30 0x7f260c172510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36462/0 (socket says 172.21.15.7:36462) 2024-01-11T07:07:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260bfff700 1 -- 172.21.15.7:0/1382059244 learned_addr learned my addr 172.21.15.7:0/1382059244 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f2610c68700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f260c0cafb0 0x7f260c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260b7fe700 1 -- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7b30 msgr2=0x7f260c172510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260b7fe700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7b30 0x7f260c172510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260b7fe700 1 -- 172.21.15.7:0/1382059244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f260c0cafb0 msgr2=0x7f260c151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260b7fe700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f260c0cafb0 0x7f260c151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260b7fe700 1 -- 172.21.15.7:0/1382059244 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc00b780 con 0x7f260c0c7f00 2024-01-11T07:07:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f2610c68700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f260c0cafb0 0x7f260c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.810+0000 7f260b7fe700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7f00 0x7f260c155270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25fc018000 tx=0x7f25fc019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.811+0000 7f25fbfff700 1 -- 172.21.15.7:0/1382059244 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc009040 con 0x7f260c0c7f00 2024-01-11T07:07:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.811+0000 7f25fbfff700 1 -- 172.21.15.7:0/1382059244 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc020470 con 0x7f260c0c7f00 2024-01-11T07:07:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.811+0000 7f25fbfff700 1 -- 172.21.15.7:0/1382059244 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc020870 con 0x7f260c0c7f00 2024-01-11T07:07:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.815+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260c06dd00 con 0x7f260c0c7f00 2024-01-11T07:07:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.815+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260c06e2c0 con 0x7f260c0c7f00 2024-01-11T07:07:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.817+0000 7f25fbfff700 1 -- 172.21.15.7:0/1382059244 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f25fc015070 con 0x7f260c0c7f00 2024-01-11T07:07:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.818+0000 7f25fbfff700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25f4067200 0x7f25f40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.818+0000 7f25fbfff700 1 -- 172.21.15.7:0/1382059244 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f25fc085500 con 0x7f260c0c7f00 2024-01-11T07:07:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.818+0000 7f260bfff700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25f4067200 0x7f25f40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.819+0000 7f260bfff700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25f4067200 0x7f25f40696b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2600009b80 tx=0x7f26000091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.831+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260c04c3c0 con 0x7f260c0c7f00 2024-01-11T07:07:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.842+0000 7f25fbfff700 1 -- 172.21.15.7:0/1382059244 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25fc052800 con 0x7f260c0c7f00 2024-01-11T07:07:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.898+0000 7fec7f66b700 1 Processor -- start 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.899+0000 7fec7f66b700 1 -- start start 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.900+0000 7fec7f66b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fec7806e630 0x7fec7806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.900+0000 7fec7f66b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec780c98b0 0x7fec780c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.900+0000 7fec7f66b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7816a620 con 0x7fec7806ea00 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.900+0000 7fec7f66b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7816b480 con 0x7fec7806e630 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.900+0000 7fec7f66b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7816a870 con 0x7fec780c98b0 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.901+0000 7fec7dc08700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec780c98b0 0x7fec780c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.901+0000 7fec7dc08700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec780c98b0 0x7fec780c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36466/0 (socket says 172.21.15.7:36466) 2024-01-11T07:07:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.901+0000 7fec7dc08700 1 -- 172.21.15.7:0/2404919985 learned_addr learned my addr 172.21.15.7:0/2404919985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.902+0000 7fec7d407700 1 --2- 172.21.15.7:0/2404919985 >> v2:172.21.15.7:3300/0 conn(0x7fec7806e630 0x7fec7806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.903+0000 7fec7d407700 1 -- 172.21.15.7:0/2404919985 >> v2:172.21.15.113:3300/0 conn(0x7fec780c98b0 msgr2=0x7fec780c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.903+0000 7fec7d407700 1 --2- 172.21.15.7:0/2404919985 >> v2:172.21.15.113:3300/0 conn(0x7fec780c98b0 0x7fec780c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.903+0000 7fec7d407700 1 -- 172.21.15.7:0/2404919985 >> v1:172.21.15.113:6789/0 conn(0x7fec7806ea00 legacy=0x7fec780c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.904+0000 7fec7d407700 1 -- 172.21.15.7:0/2404919985 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7816ab50 con 0x7fec7806e630 2024-01-11T07:07:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.904+0000 7fec7d407700 1 --2- 172.21.15.7:0/2404919985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 0x7fec7806cb60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec780b27b0 tx=0x7fec70007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dde69a97290f82 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.904+0000 7fec6ffff700 1 -- 172.21.15.7:0/2404919985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70012310 con 0x7fec7806e630 2024-01-11T07:07:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.904+0000 7fec6ffff700 1 -- 172.21.15.7:0/2404919985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec70012470 con 0x7fec7806e630 2024-01-11T07:07:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.904+0000 7fec6ffff700 1 -- 172.21.15.7:0/2404919985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000c8b0 con 0x7fec7806e630 2024-01-11T07:07:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.905+0000 7fec7f66b700 1 -- 172.21.15.7:0/2404919985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 msgr2=0x7fec7806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.905+0000 7fec7f66b700 1 --2- 172.21.15.7:0/2404919985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 0x7fec7806cb60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec780b27b0 tx=0x7fec70007cc0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.906+0000 7fec7f66b700 1 -- 172.21.15.7:0/2404919985 shutdown_connections 2024-01-11T07:07:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.906+0000 7fec7f66b700 1 --2- 172.21.15.7:0/2404919985 >> v2:172.21.15.113:3300/0 conn(0x7fec780c98b0 0x7fec780c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.906+0000 7fec7f66b700 1 --2- 172.21.15.7:0/2404919985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 0x7fec7806cb60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.906+0000 7fec7f66b700 1 -- 172.21.15.7:0/2404919985 >> 172.21.15.7:0/2404919985 conn(0x7fec78063230 msgr2=0x7fec78063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.906+0000 7fec7f66b700 1 -- 172.21.15.7:0/2404919985 shutdown_connections 2024-01-11T07:07:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.906+0000 7fec7f66b700 1 -- 172.21.15.7:0/2404919985 wait complete. 2024-01-11T07:07:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.908+0000 7fec7f66b700 1 Processor -- start 2024-01-11T07:07:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.908+0000 7fec7f66b700 1 -- start start 2024-01-11T07:07:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.910+0000 7fec7f66b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 0x7fec7817a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.910+0000 7fec7f66b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7806ea00 0x7fec781662f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.910+0000 7fec7f66b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec780c98b0 0x7fec780b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.910+0000 7fec7f66b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7817ad40 con 0x7fec7806e630 2024-01-11T07:07:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.910+0000 7fec7f66b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec78166830 con 0x7fec7806ea00 2024-01-11T07:07:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.910+0000 7fec7f66b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec780c45e0 con 0x7fec780c98b0 2024-01-11T07:07:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.911+0000 7fec7dc08700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec780c98b0 0x7fec780b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.911+0000 7fec7cc06700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7806ea00 0x7fec781662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.911+0000 7fec7cc06700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7806ea00 0x7fec781662f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36478/0 (socket says 172.21.15.7:36478) 2024-01-11T07:07:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.911+0000 7fec7cc06700 1 -- 172.21.15.7:0/3929813957 learned_addr learned my addr 172.21.15.7:0/3929813957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.912+0000 7fec7d407700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 0x7fec7817a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec7d407700 1 -- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec780c98b0 msgr2=0x7fec780b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec7d407700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec780c98b0 0x7fec780b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec7d407700 1 -- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7806ea00 msgr2=0x7fec781662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec7d407700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7806ea00 0x7fec781662f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec7d407700 1 -- 172.21.15.7:0/3929813957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70005d80 con 0x7fec7806e630 2024-01-11T07:07:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec7dc08700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec780c98b0 0x7fec780b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec7d407700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 0x7fec7817a800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec70020000 tx=0x7fec70023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec6d7fa700 1 -- 172.21.15.7:0/3929813957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70017070 con 0x7fec7806e630 2024-01-11T07:07:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec6d7fa700 1 -- 172.21.15.7:0/3929813957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec70003300 con 0x7fec7806e630 2024-01-11T07:07:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.913+0000 7fec6d7fa700 1 -- 172.21.15.7:0/3929813957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000e480 con 0x7fec7806e630 2024-01-11T07:07:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.914+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec780c48c0 con 0x7fec7806e630 2024-01-11T07:07:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.914+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec780c4e80 con 0x7fec7806e630 2024-01-11T07:07:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.917+0000 7fec6d7fa700 1 -- 172.21.15.7:0/3929813957 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fec70003050 con 0x7fec7806e630 2024-01-11T07:07:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.918+0000 7fec6d7fa700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fec640671e0 0x7fec64069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.918+0000 7fec6d7fa700 1 -- 172.21.15.7:0/3929813957 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec70085b00 con 0x7fec7806e630 2024-01-11T07:07:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.919+0000 7fec7cc06700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fec640671e0 0x7fec64069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.919+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec580052a0 con 0x7fec7806e630 2024-01-11T07:07:11.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.920+0000 7fec7cc06700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fec640671e0 0x7fec64069690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec680078a0 tx=0x7fec68007830 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.928+0000 7fec6d7fa700 1 -- 172.21.15.7:0/3929813957 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec70052d50 con 0x7fec7806e630 2024-01-11T07:07:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.946+0000 7fe87f3f2700 1 Processor -- start 2024-01-11T07:07:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.947+0000 7fe87f3f2700 1 -- start start 2024-01-11T07:07:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.948+0000 7fe87f3f2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe878166c00 0x7fe87816a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.948+0000 7fe87f3f2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87816b550 con 0x7fe8780ca9d0 2024-01-11T07:07:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.948+0000 7fe87f3f2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8780c8f50 con 0x7fe8780c8ab0 2024-01-11T07:07:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.948+0000 7fe87f3f2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe878166fd0 con 0x7fe878166c00 2024-01-11T07:07:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.949+0000 7fe87d18e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe878166c00 0x7fe87816a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.949+0000 7fe87d18e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe878166c00 0x7fe87816a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36494/0 (socket says 172.21.15.7:36494) 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.949+0000 7fe87d18e700 1 -- 172.21.15.7:0/971963248 learned_addr learned my addr 172.21.15.7:0/971963248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.949+0000 7fe87d18e700 1 -- 172.21.15.7:0/971963248 >> v1:172.21.15.113:6789/0 conn(0x7fe8780c8ab0 legacy=0x7fe87806cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.949+0000 7fe87d18e700 1 -- 172.21.15.7:0/971963248 >> v1:172.21.15.7:6789/0 conn(0x7fe8780ca9d0 legacy=0x7fe8780c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.949+0000 7fe87d18e700 1 -- 172.21.15.7:0/971963248 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87816acb0 con 0x7fe878166c00 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.950+0000 7fe87d18e700 1 --2- 172.21.15.7:0/971963248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe878166c00 0x7fe87816a620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe87000d1a0 tx=0x7fe870007690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25750c43b94aa6d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.950+0000 7fe86ffff700 1 -- 172.21.15.7:0/971963248 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870010070 con 0x7fe878166c00 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.950+0000 7fe86ffff700 1 -- 172.21.15.7:0/971963248 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000dc70 con 0x7fe878166c00 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.950+0000 7fe86ffff700 1 -- 172.21.15.7:0/971963248 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870007b40 con 0x7fe878166c00 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.951+0000 7fe87f3f2700 1 -- 172.21.15.7:0/971963248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe878166c00 msgr2=0x7fe87816a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.951+0000 7fe87f3f2700 1 --2- 172.21.15.7:0/971963248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe878166c00 0x7fe87816a620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe87000d1a0 tx=0x7fe870007690 comp rx=0 tx=0).stop 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.954+0000 7fe87f3f2700 1 -- 172.21.15.7:0/971963248 shutdown_connections 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.954+0000 7fe87f3f2700 1 --2- 172.21.15.7:0/971963248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe878166c00 0x7fe87816a620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.954+0000 7fe87f3f2700 1 -- 172.21.15.7:0/971963248 >> 172.21.15.7:0/971963248 conn(0x7fe878063230 msgr2=0x7fe878063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.954+0000 7fe87f3f2700 1 -- 172.21.15.7:0/971963248 shutdown_connections 2024-01-11T07:07:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.954+0000 7fe87f3f2700 1 -- 172.21.15.7:0/971963248 wait complete. 2024-01-11T07:07:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.956+0000 7fe87f3f2700 1 Processor -- start 2024-01-11T07:07:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.956+0000 7fe87f3f2700 1 -- start start 2024-01-11T07:07:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.957+0000 7fe87f3f2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8780c8ab0 0x7fe8781889e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.957+0000 7fe87f3f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8780ca9d0 0x7fe8780880d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87f3f2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe878166c00 0x7fe878167110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87f3f2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe878169520 con 0x7fe8780ca9d0 2024-01-11T07:07:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87f3f2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8781699b0 con 0x7fe8780c8ab0 2024-01-11T07:07:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87f3f2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe878168910 con 0x7fe878166c00 2024-01-11T07:07:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87c98d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8780ca9d0 0x7fe8780880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87d18e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8780c8ab0 0x7fe8781889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87c98d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8780ca9d0 0x7fe8780880d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41518/0 (socket says 172.21.15.7:41518) 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87c98d700 1 -- 172.21.15.7:0/2931670355 learned_addr learned my addr 172.21.15.7:0/2931670355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.958+0000 7fe87d98f700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe878166c00 0x7fe878167110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.959+0000 7fe87c98d700 1 -- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe878166c00 msgr2=0x7fe878167110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.959+0000 7fe87c98d700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe878166c00 0x7fe878167110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.959+0000 7fe87c98d700 1 -- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8780c8ab0 msgr2=0x7fe8781889e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.959+0000 7fe87c98d700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8780c8ab0 0x7fe8781889e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.959+0000 7fe87c98d700 1 -- 172.21.15.7:0/2931670355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87000be70 con 0x7fe8780ca9d0 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.959+0000 7fe87d18e700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8780c8ab0 0x7fe8781889e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.959+0000 7fe87c98d700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8780ca9d0 0x7fe8780880d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c590 tx=0x7fe87400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.962+0000 7fe86d7fa700 1 -- 172.21.15.7:0/2931670355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874011590 con 0x7fe8780ca9d0 2024-01-11T07:07:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.962+0000 7fe86d7fa700 1 -- 172.21.15.7:0/2931670355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8740116f0 con 0x7fe8780ca9d0 2024-01-11T07:07:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.962+0000 7fe86d7fa700 1 -- 172.21.15.7:0/2931670355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874011ab0 con 0x7fe8780ca9d0 2024-01-11T07:07:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.962+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe878168bf0 con 0x7fe8780ca9d0 2024-01-11T07:07:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.962+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8781691b0 con 0x7fe8780ca9d0 2024-01-11T07:07:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.964+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe878169340 con 0x7fe8780ca9d0 2024-01-11T07:07:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.965+0000 7fe86d7fa700 1 -- 172.21.15.7:0/2931670355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe874012420 con 0x7fe8780ca9d0 2024-01-11T07:07:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.966+0000 7fe86d7fa700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe85c066f80 0x7fe85c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.966+0000 7fe86d7fa700 1 -- 172.21.15.7:0/2931670355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe8740838a0 con 0x7fe8780ca9d0 2024-01-11T07:07:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.966+0000 7fe87d18e700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe85c066f80 0x7fe85c069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.967+0000 7fe87d18e700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe85c066f80 0x7fe85c069430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe870007260 tx=0x7fe87001a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.970+0000 7fde85b07700 1 Processor -- start 2024-01-11T07:07:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.970+0000 7fde85b07700 1 -- start start 2024-01-11T07:07:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.971+0000 7fde85b07700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cba20 0x7fde8015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.971+0000 7fde85b07700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde800c6f30 0x7fde80155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.971+0000 7fde85b07700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde800cbdf0 0x7fde80155d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fde85b07700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde801562b0 con 0x7fde800c6f30 2024-01-11T07:07:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fde85b07700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde800ca3a0 con 0x7fde800cbdf0 2024-01-11T07:07:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fde85b07700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800cb200 con 0x7fde800cba20 2024-01-11T07:07:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.970+0000 7fb202f18700 1 Processor -- start 2024-01-11T07:07:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.970+0000 7fb202f18700 1 -- start start 2024-01-11T07:07:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.971+0000 7fb202f18700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1fc06e850 0x7fb1fc15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.971+0000 7fb202f18700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc0af5d0 con 0x7fb1fc15af90 2024-01-11T07:07:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.971+0000 7fb202f18700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc0cc3b0 con 0x7fb1fc068bf0 2024-01-11T07:07:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fb202f18700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1fc06dc50 con 0x7fb1fc06e850 2024-01-11T07:07:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fb2014b5700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb1fc068bf0 0x7fb1fc0c8b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40404/0 (socket says 172.21.15.7:40404) 2024-01-11T07:07:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fb2014b5700 1 -- 172.21.15.7:0/2404846487 learned_addr learned my addr 172.21.15.7:0/2404846487 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fde7ffff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde800cbdf0 0x7fde80155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fb1fbfff700 1 --2- 172.21.15.7:0/2404846487 >> v2:172.21.15.113:3300/0 conn(0x7fb1fc06e850 0x7fb1fc15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fde7effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde800c6f30 0x7fde80155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fde7effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde800c6f30 0x7fde80155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41522/0 (socket says 172.21.15.7:41522) 2024-01-11T07:07:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fde7effd700 1 -- 172.21.15.7:0/4049782707 learned_addr learned my addr 172.21.15.7:0/4049782707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.972+0000 7fde7f7fe700 1 --2- 172.21.15.7:0/4049782707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cba20 0x7fde8015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fde7f7fe700 1 -- 172.21.15.7:0/4049782707 >> v2:172.21.15.113:3300/0 conn(0x7fde800cbdf0 msgr2=0x7fde80155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fde7f7fe700 1 --2- 172.21.15.7:0/4049782707 >> v2:172.21.15.113:3300/0 conn(0x7fde800cbdf0 0x7fde80155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fde7f7fe700 1 -- 172.21.15.7:0/4049782707 >> v2:172.21.15.7:3300/0 conn(0x7fde800c6f30 msgr2=0x7fde80155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fde7f7fe700 1 --2- 172.21.15.7:0/4049782707 >> v2:172.21.15.7:3300/0 conn(0x7fde800c6f30 0x7fde80155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fde7f7fe700 1 -- 172.21.15.7:0/4049782707 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8015a8e0 con 0x7fde800cba20 2024-01-11T07:07:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fde7f7fe700 1 --2- 172.21.15.7:0/4049782707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cba20 0x7fde8015a3a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fde800c3830 tx=0x7fde78007620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56e5fceecf26cb9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fb1fb7fe700 1 -- 172.21.15.7:0/2404846487 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130060251 0 0) 0x7fb1fc0cc3b0 con 0x7fb1fc068bf0 2024-01-11T07:07:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fb1fb7fe700 1 -- 172.21.15.7:0/2404846487 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1ec003570 con 0x7fb1fc068bf0 2024-01-11T07:07:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fb1fbfff700 1 -- 172.21.15.7:0/2404846487 >> v1:172.21.15.113:6789/0 conn(0x7fb1fc068bf0 legacy=0x7fb1fc0c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.973+0000 7fb1fbfff700 1 -- 172.21.15.7:0/2404846487 >> v1:172.21.15.7:6789/0 conn(0x7fb1fc15af90 legacy=0x7fb1fc06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fb1fbfff700 1 -- 172.21.15.7:0/2404846487 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc069360 con 0x7fb1fc06e850 2024-01-11T07:07:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fb1fbfff700 1 --2- 172.21.15.7:0/2404846487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc06e850 0x7fb1fc15c1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009910 tx=0x7fb1f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a641b130acabc13f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fe86d7fa700 1 -- 172.21.15.7:0/2931670355 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe87404b5a0 con 0x7fe8780ca9d0 2024-01-11T07:07:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fde7e7fc700 1 -- 172.21.15.7:0/4049782707 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78010070 con 0x7fde800cba20 2024-01-11T07:07:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fde7e7fc700 1 -- 172.21.15.7:0/4049782707 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7800dcd0 con 0x7fde800cba20 2024-01-11T07:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fde7e7fc700 1 -- 172.21.15.7:0/4049782707 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78007ce0 con 0x7fde800cba20 2024-01-11T07:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.975+0000 7fde85b07700 1 -- 172.21.15.7:0/4049782707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cba20 msgr2=0x7fde8015a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.975+0000 7fde85b07700 1 --2- 172.21.15.7:0/4049782707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cba20 0x7fde8015a3a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fde800c3830 tx=0x7fde78007620 comp rx=0 tx=0).stop 2024-01-11T07:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fb1fb7fe700 1 -- 172.21.15.7:0/2404846487 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0011630 con 0x7fb1fc06e850 2024-01-11T07:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fb1fb7fe700 1 -- 172.21.15.7:0/2404846487 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f0011790 con 0x7fb1fc06e850 2024-01-11T07:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.974+0000 7fb1fb7fe700 1 -- 172.21.15.7:0/2404846487 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0011b50 con 0x7fb1fc06e850 2024-01-11T07:07:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.975+0000 7fb202f18700 1 -- 172.21.15.7:0/2404846487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc06e850 msgr2=0x7fb1fc15c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.975+0000 7fb202f18700 1 --2- 172.21.15.7:0/2404846487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc06e850 0x7fb1fc15c1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009910 tx=0x7fb1f00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.975+0000 7fb202f18700 1 -- 172.21.15.7:0/2404846487 shutdown_connections 2024-01-11T07:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.975+0000 7fb202f18700 1 --2- 172.21.15.7:0/2404846487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc06e850 0x7fb1fc15c1e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.975+0000 7fb202f18700 1 -- 172.21.15.7:0/2404846487 >> 172.21.15.7:0/2404846487 conn(0x7fb1fc063230 msgr2=0x7fb1fc063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fde85b07700 1 -- 172.21.15.7:0/4049782707 shutdown_connections 2024-01-11T07:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fde85b07700 1 --2- 172.21.15.7:0/4049782707 >> v2:172.21.15.113:3300/0 conn(0x7fde800cbdf0 0x7fde80155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fde85b07700 1 --2- 172.21.15.7:0/4049782707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cba20 0x7fde8015a3a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fde85b07700 1 --2- 172.21.15.7:0/4049782707 >> v2:172.21.15.7:3300/0 conn(0x7fde800c6f30 0x7fde80155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fde85b07700 1 -- 172.21.15.7:0/4049782707 >> 172.21.15.7:0/4049782707 conn(0x7fde80063230 msgr2=0x7fde80063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fde85b07700 1 -- 172.21.15.7:0/4049782707 shutdown_connections 2024-01-11T07:07:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fb202f18700 1 -- 172.21.15.7:0/2404846487 shutdown_connections 2024-01-11T07:07:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fde85b07700 1 -- 172.21.15.7:0/4049782707 wait complete. 2024-01-11T07:07:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.976+0000 7fb202f18700 1 -- 172.21.15.7:0/2404846487 wait complete. 2024-01-11T07:07:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.978+0000 7fde85b07700 1 Processor -- start 2024-01-11T07:07:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.978+0000 7fb202f18700 1 Processor -- start 2024-01-11T07:07:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.978+0000 7fb202f18700 1 -- start start 2024-01-11T07:07:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fb202f18700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc068bf0 0x7fb1fc149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fb202f18700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1fc06e850 0x7fb1fc0c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fb202f18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1fc15af90 0x7fb1fc0c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fb202f18700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc14a4c0 con 0x7fb1fc15af90 2024-01-11T07:07:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fb202f18700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0a64b0 con 0x7fb1fc068bf0 2024-01-11T07:07:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fb202f18700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0a77a0 con 0x7fb1fc06e850 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.978+0000 7fde85b07700 1 -- start start 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fde85b07700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde800c6f30 0x7fde80150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fde85b07700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde800cba20 0x7fde801421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fde85b07700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cbdf0 0x7fde80157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fde85b07700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800b0a20 con 0x7fde800c6f30 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fde85b07700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800afe10 con 0x7fde800cba20 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.979+0000 7fde85b07700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800b00f0 con 0x7fde800cbdf0 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7ffff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cbdf0 0x7fde80157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7effd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde800cba20 0x7fde801421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7effd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde800cba20 0x7fde801421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36510/0 (socket says 172.21.15.7:36510) 2024-01-11T07:07:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7effd700 1 -- 172.21.15.7:0/1289121483 learned_addr learned my addr 172.21.15.7:0/1289121483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fb2014b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1fc15af90 0x7fb1fc0c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fb2014b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1fc15af90 0x7fb1fc0c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41528/0 (socket says 172.21.15.7:41528) 2024-01-11T07:07:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fb2014b5700 1 -- 172.21.15.7:0/2802786038 learned_addr learned my addr 172.21.15.7:0/2802786038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fb200cb4700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc068bf0 0x7fb1fc149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7ffff700 1 -- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde800cba20 msgr2=0x7fde801421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7ffff700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde800cba20 0x7fde801421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7ffff700 1 -- 172.21.15.7:0/1289121483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde800c6f30 msgr2=0x7fde80150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7ffff700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde800c6f30 0x7fde80150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.980+0000 7fde7ffff700 1 -- 172.21.15.7:0/1289121483 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780078c0 con 0x7fde800cbdf0 2024-01-11T07:07:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7fb2014b5700 1 -- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1fc06e850 msgr2=0x7fb1fc0c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7fb2014b5700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1fc06e850 0x7fb1fc0c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7fb2014b5700 1 -- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc068bf0 msgr2=0x7fb1fc149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7fb2014b5700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc068bf0 0x7fb1fc149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7fb2014b5700 1 -- 172.21.15.7:0/2802786038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f0009420 con 0x7fb1fc15af90 2024-01-11T07:07:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7f4bb533d700 1 Processor -- start 2024-01-11T07:07:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7fde7ffff700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cbdf0 0x7fde80157920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde7000c650 tx=0x7fde70008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7fb2014b5700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1fc15af90 0x7fb1fc0c2f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400cb90 tx=0x7fb1f40019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7f4bb533d700 1 -- start start 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f4bb533d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4bb00c9110 0x7f4bb006d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f4bb533d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb00cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f4bb533d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb00c9860 con 0x7f4bb015e120 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f4bb533d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb0069d90 con 0x7f4bb00c9110 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f4bb533d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00ccb30 con 0x7f4bb00cc1b0 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f4baf7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb00cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f4baf7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb00cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:39198/0 (socket says 172.21.15.7:39198) 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f4baf7fe700 1 -- 172.21.15.7:0/1447914272 learned_addr learned my addr 172.21.15.7:0/1447914272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4bae7fc700 1 --2- 172.21.15.7:0/1447914272 >> v2:172.21.15.113:3300/0 conn(0x7f4bb00c9110 0x7f4bb006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4baf7fe700 1 -- 172.21.15.7:0/1447914272 >> v2:172.21.15.113:3300/0 conn(0x7f4bb00c9110 msgr2=0x7f4bb006d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4badffb700 1 -- 172.21.15.7:0/1447914272 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211818760 0 0) 0x7f4bb00c9860 con 0x7f4bb015e120 2024-01-11T07:07:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4baf7fe700 1 --2- 172.21.15.7:0/1447914272 >> v2:172.21.15.113:3300/0 conn(0x7f4bb00c9110 0x7f4bb006d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4baf7fe700 1 -- 172.21.15.7:0/1447914272 >> v1:172.21.15.113:6789/0 conn(0x7f4bb015e120 legacy=0x7f4bb01626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4baf7fe700 1 -- 172.21.15.7:0/1447914272 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00c9580 con 0x7f4bb00cc1b0 2024-01-11T07:07:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4bae7fc700 1 --2- 172.21.15.7:0/1447914272 >> v2:172.21.15.113:3300/0 conn(0x7f4bb00c9110 0x7f4bb006d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4baf7fe700 1 --2- 172.21.15.7:0/1447914272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb00cc5f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009f20 tx=0x7f4ba8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66896c2ff617874c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7f10295f6700 1 Processor -- start 2024-01-11T07:07:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.981+0000 7f10295f6700 1 -- start start 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f10295f6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f102406e630 0x7f102406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f10295f6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102416a620 con 0x7f102406ea00 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f10295f6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102416b480 con 0x7f10240c98b0 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f10295f6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102416a870 con 0x7f102406e630 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f10227fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f102406ea00 0x7f10240c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40244/0 (socket says 172.21.15.7:40244) 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f1022ffd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f102406e630 0x7f102406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.983+0000 7f10227fc700 1 -- 172.21.15.7:0/1198304631 learned_addr learned my addr 172.21.15.7:0/1198304631 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.984+0000 7f4badffb700 1 -- 172.21.15.7:0/1447914272 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800e070 con 0x7f4bb00cc1b0 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7f4badffb700 1 -- 172.21.15.7:0/1447914272 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba800ba80 con 0x7f4bb00cc1b0 2024-01-11T07:07:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7f4badffb700 1 -- 172.21.15.7:0/1447914272 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80092e0 con 0x7f4bb00cc1b0 2024-01-11T07:07:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7fb1f8ff9700 1 -- 172.21.15.7:0/2802786038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400a8b0 con 0x7fb1fc15af90 2024-01-11T07:07:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7fb1f8ff9700 1 -- 172.21.15.7:0/2802786038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f400aa50 con 0x7fb1fc15af90 2024-01-11T07:07:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7fde6f7fe700 1 -- 172.21.15.7:0/1289121483 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70011600 con 0x7fde800cbdf0 2024-01-11T07:07:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7f4bb533d700 1 -- 172.21.15.7:0/1447914272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 msgr2=0x7f4bb00cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7f4bb533d700 1 --2- 172.21.15.7:0/1447914272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb00cc5f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009f20 tx=0x7f4ba8009840 comp rx=0 tx=0).stop 2024-01-11T07:07:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7f4bb533d700 1 -- 172.21.15.7:0/1447914272 shutdown_connections 2024-01-11T07:07:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7f4bb533d700 1 --2- 172.21.15.7:0/1447914272 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb00cc5f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7f4bb533d700 1 --2- 172.21.15.7:0/1447914272 >> v2:172.21.15.113:3300/0 conn(0x7f4bb00c9110 0x7f4bb006d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7f4bb533d700 1 -- 172.21.15.7:0/1447914272 >> 172.21.15.7:0/1447914272 conn(0x7f4bb00cd090 msgr2=0x7f4bb00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7fde6f7fe700 1 -- 172.21.15.7:0/1289121483 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70011760 con 0x7fde800cbdf0 2024-01-11T07:07:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7fde6f7fe700 1 -- 172.21.15.7:0/1289121483 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70011b00 con 0x7fde800cbdf0 2024-01-11T07:07:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde800b03d0 con 0x7fde800cbdf0 2024-01-11T07:07:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7f4bb533d700 1 -- 172.21.15.7:0/1447914272 shutdown_connections 2024-01-11T07:07:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7f4bb533d700 1 -- 172.21.15.7:0/1447914272 wait complete. 2024-01-11T07:07:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.986+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde80142720 con 0x7fde800cbdf0 2024-01-11T07:07:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc0a6700 con 0x7fb1fc15af90 2024-01-11T07:07:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.985+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc0a6cc0 con 0x7fb1fc15af90 2024-01-11T07:07:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.988+0000 7f4bb533d700 1 Processor -- start 2024-01-11T07:07:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.987+0000 7f1022ffd700 1 -- 172.21.15.7:0/1198304631 >> v1:172.21.15.113:6789/0 conn(0x7f10240c98b0 legacy=0x7f10240c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.987+0000 7f1022ffd700 1 -- 172.21.15.7:0/1198304631 >> v1:172.21.15.7:6789/0 conn(0x7f102406ea00 legacy=0x7f10240c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.987+0000 7f1022ffd700 1 -- 172.21.15.7:0/1198304631 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102416ab50 con 0x7f102406e630 2024-01-11T07:07:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.988+0000 7f4bb533d700 1 -- start start 2024-01-11T07:07:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.988+0000 7fde6f7fe700 1 -- 172.21.15.7:0/1289121483 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fde70018360 con 0x7fde800cbdf0 2024-01-11T07:07:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.989+0000 7fde6f7fe700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde64067050 0x7fde64069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.989+0000 7fde6f7fe700 1 -- 172.21.15.7:0/1289121483 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde700841c0 con 0x7fde800cbdf0 2024-01-11T07:07:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f1022ffd700 1 --2- 172.21.15.7:0/1198304631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102406e630 0x7f102406cb60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f101c009940 tx=0x7f101c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=220ca915522f454b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4bb533d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bb00c9110 0x7f4bb014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4bb533d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb014ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4bb533d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb015e120 0x7f4bb014bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4bb533d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0146c10 con 0x7f4bb015e120 2024-01-11T07:07:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4bb533d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0146ef0 con 0x7f4bb00c9110 2024-01-11T07:07:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4bb533d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb01471d0 con 0x7f4bb00cc1b0 2024-01-11T07:07:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7fde7f7fe700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde64067050 0x7fde64069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4baeffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bb00c9110 0x7f4bb014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4baeffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bb00c9110 0x7f4bb014d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36542/0 (socket says 172.21.15.7:36542) 2024-01-11T07:07:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4baeffd700 1 -- 172.21.15.7:0/964851006 learned_addr learned my addr 172.21.15.7:0/964851006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8004c3c0 con 0x7fde800cbdf0 2024-01-11T07:07:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4baeffd700 1 -- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 msgr2=0x7f4bb014ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4baeffd700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb014ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4baeffd700 1 -- 172.21.15.7:0/964851006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb015e120 msgr2=0x7f4bb014bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4baeffd700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb015e120 0x7f4bb014bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7f4baeffd700 1 -- 172.21.15.7:0/964851006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba8009ae0 con 0x7f4bb00c9110 2024-01-11T07:07:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.989+0000 7fb1f8ff9700 1 -- 172.21.15.7:0/2802786038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400ae10 con 0x7fb1fc15af90 2024-01-11T07:07:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.989+0000 7fb1f8ff9700 1 -- 172.21.15.7:0/2802786038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb1f4007300 con 0x7fb1fc15af90 2024-01-11T07:07:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.989+0000 7fb1f8ff9700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb1e0067100 0x7fb1e00695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.990+0000 7fb1f8ff9700 1 -- 172.21.15.7:0/2802786038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb1f4084960 con 0x7fb1fc15af90 2024-01-11T07:07:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.991+0000 7f4baeffd700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bb00c9110 0x7f4bb014d710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400c530 tx=0x7f4ba400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.991+0000 7fb200cb4700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb1e0067100 0x7fb1e00695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.991+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc0052a0 con 0x7fb1fc15af90 2024-01-11T07:07:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.992+0000 7f4b96ffd700 1 -- 172.21.15.7:0/964851006 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4011590 con 0x7f4bb00c9110 2024-01-11T07:07:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.992+0000 7fb200cb4700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb1e0067100 0x7fb1e00695b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8006fd0 tx=0x7fb1e80090f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.992+0000 7f4b96ffd700 1 -- 172.21.15.7:0/964851006 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba40116f0 con 0x7f4bb00c9110 2024-01-11T07:07:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.992+0000 7f4b96ffd700 1 -- 172.21.15.7:0/964851006 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4011ab0 con 0x7f4bb00c9110 2024-01-11T07:07:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.992+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb00b0940 con 0x7f4bb00c9110 2024-01-11T07:07:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.992+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb014dc50 con 0x7f4bb00c9110 2024-01-11T07:07:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.994+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b900052a0 con 0x7f4bb00c9110 2024-01-11T07:07:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.994+0000 7f4b96ffd700 1 -- 172.21.15.7:0/964851006 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4ba401b3b0 con 0x7f4bb00c9110 2024-01-11T07:07:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.995+0000 7f4b96ffd700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b9c067080 0x7f4b9c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.995+0000 7f4b96ffd700 1 -- 172.21.15.7:0/964851006 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ba4083b80 con 0x7f4bb00c9110 2024-01-11T07:07:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.993+0000 7f1021ffb700 1 -- 172.21.15.7:0/1198304631 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c011600 con 0x7f102406e630 2024-01-11T07:07:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.993+0000 7f1021ffb700 1 -- 172.21.15.7:0/1198304631 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c011760 con 0x7f102406e630 2024-01-11T07:07:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.993+0000 7f1021ffb700 1 -- 172.21.15.7:0/1198304631 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c011b20 con 0x7f102406e630 2024-01-11T07:07:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.995+0000 7f10295f6700 1 -- 172.21.15.7:0/1198304631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102406e630 msgr2=0x7f102406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.995+0000 7f10295f6700 1 --2- 172.21.15.7:0/1198304631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102406e630 0x7f102406cb60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f101c009940 tx=0x7f101c009bd0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.996+0000 7f10295f6700 1 -- 172.21.15.7:0/1198304631 shutdown_connections 2024-01-11T07:07:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.996+0000 7f10295f6700 1 --2- 172.21.15.7:0/1198304631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102406e630 0x7f102406cb60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.996+0000 7f10295f6700 1 -- 172.21.15.7:0/1198304631 >> 172.21.15.7:0/1198304631 conn(0x7f1024063230 msgr2=0x7f1024063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.996+0000 7f4bae7fc700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b9c067080 0x7f4b9c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.997+0000 7f4bae7fc700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b9c067080 0x7f4b9c069530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00c7140 tx=0x7f4ba0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.999+0000 7f4b96ffd700 1 -- 172.21.15.7:0/964851006 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ba4050dd0 con 0x7f4bb00c9110 2024-01-11T07:07:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.996+0000 7fde6f7fe700 1 -- 172.21.15.7:0/1289121483 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde70051380 con 0x7fde800cbdf0 2024-01-11T07:07:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:11.996+0000 7fde7f7fe700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde64067050 0x7fde64069500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde800c3830 tx=0x7fde7801b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.001+0000 7f10295f6700 1 -- 172.21.15.7:0/1198304631 shutdown_connections 2024-01-11T07:07:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.001+0000 7fb1f8ff9700 1 -- 172.21.15.7:0/2802786038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1f4051c30 con 0x7fb1fc15af90 2024-01-11T07:07:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.001+0000 7f10295f6700 1 -- 172.21.15.7:0/1198304631 wait complete. 2024-01-11T07:07:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.003+0000 7f10295f6700 1 Processor -- start 2024-01-11T07:07:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.003+0000 7f10295f6700 1 -- start start 2024-01-11T07:07:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.004+0000 7f10295f6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102406e630 0x7f1024165330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.005+0000 7f10295f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f102406ea00 0x7f10240c2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.005+0000 7f10295f6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10240c98b0 0x7f10240b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.005+0000 7f10295f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240b2630 con 0x7f102406ea00 2024-01-11T07:07:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.005+0000 7f10295f6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024167be0 con 0x7f102406e630 2024-01-11T07:07:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.005+0000 7f10295f6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10241672b0 con 0x7f10240c98b0 2024-01-11T07:07:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.005+0000 7f10227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f102406ea00 0x7f10240c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.005+0000 7f10227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f102406ea00 0x7f10240c2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41564/0 (socket says 172.21.15.7:41564) 2024-01-11T07:07:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.005+0000 7f10227fc700 1 -- 172.21.15.7:0/2231128320 learned_addr learned my addr 172.21.15.7:0/2231128320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.006+0000 7f10237fe700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10240c98b0 0x7f10240b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.006+0000 7f10227fc700 1 -- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10240c98b0 msgr2=0x7f10240b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.006+0000 7f10227fc700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10240c98b0 0x7f10240b1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.006+0000 7f10227fc700 1 -- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102406e630 msgr2=0x7f1024165330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T07:07:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.006+0000 7f10227fc700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102406e630 0x7f1024165330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.006+0000 7f10227fc700 1 -- 172.21.15.7:0/2231128320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c009e70 con 0x7f102406ea00 2024-01-11T07:07:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.006+0000 7f9869253700 1 Processor -- start 2024-01-11T07:07:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.007+0000 7f10227fc700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f102406ea00 0x7f10240c2770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f101800eee0 tx=0x7f1018007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.006+0000 7f9869253700 1 -- start start 2024-01-11T07:07:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.008+0000 7f9869253700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 0x7f98640c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.008+0000 7f9869253700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640c1950 con 0x7f98640c8a10 2024-01-11T07:07:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.008+0000 7f9869253700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98640c1c30 con 0x7f9864155ea0 2024-01-11T07:07:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.008+0000 7f9869253700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98640c1f10 con 0x7f98640c13e0 2024-01-11T07:07:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.008+0000 7f9868a52700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f98640c8a10 0x7f98640c7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40260/0 (socket says 172.21.15.7:40260) 2024-01-11T07:07:12.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.008+0000 7f98637fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 0x7f98640c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.008+0000 7f9868a52700 1 -- 172.21.15.7:0/286990514 learned_addr learned my addr 172.21.15.7:0/286990514 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.009+0000 7f98637fe700 1 -- 172.21.15.7:0/286990514 >> v1:172.21.15.113:6789/0 conn(0x7f9864155ea0 legacy=0x7f98640cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.009+0000 7f98637fe700 1 -- 172.21.15.7:0/286990514 >> v1:172.21.15.7:6789/0 conn(0x7f98640c8a10 legacy=0x7f98640c7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.009+0000 7f98637fe700 1 -- 172.21.15.7:0/286990514 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98640c21f0 con 0x7f98640c13e0 2024-01-11T07:07:12.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.009+0000 7f98637fe700 1 --2- 172.21.15.7:0/286990514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 0x7f98640c6ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f985400cb80 tx=0x7f9854006180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42b232787781581 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.009+0000 7f9862ffd700 1 -- 172.21.15.7:0/286990514 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854011070 con 0x7f98640c13e0 2024-01-11T07:07:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.007+0000 7f100affd700 1 -- 172.21.15.7:0/2231128320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101800ce60 con 0x7f102406ea00 2024-01-11T07:07:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.007+0000 7f100affd700 1 -- 172.21.15.7:0/2231128320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018005b30 con 0x7f102406ea00 2024-01-11T07:07:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.007+0000 7f100affd700 1 -- 172.21.15.7:0/2231128320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018001760 con 0x7f102406ea00 2024-01-11T07:07:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.009+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1024167590 con 0x7f102406ea00 2024-01-11T07:07:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.009+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10240b2240 con 0x7f102406ea00 2024-01-11T07:07:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.010+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10240c2cb0 con 0x7f102406ea00 2024-01-11T07:07:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.011+0000 7f9862ffd700 1 -- 172.21.15.7:0/286990514 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985400c3b0 con 0x7f98640c13e0 2024-01-11T07:07:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.011+0000 7f9862ffd700 1 -- 172.21.15.7:0/286990514 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854007b50 con 0x7f98640c13e0 2024-01-11T07:07:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.012+0000 7f9869253700 1 -- 172.21.15.7:0/286990514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 msgr2=0x7f98640c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.012+0000 7f9869253700 1 --2- 172.21.15.7:0/286990514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 0x7f98640c6ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f985400cb80 tx=0x7f9854006180 comp rx=0 tx=0).stop 2024-01-11T07:07:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.012+0000 7f9869253700 1 -- 172.21.15.7:0/286990514 shutdown_connections 2024-01-11T07:07:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.012+0000 7f9869253700 1 --2- 172.21.15.7:0/286990514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 0x7f98640c6ec0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.012+0000 7f9869253700 1 -- 172.21.15.7:0/286990514 >> 172.21.15.7:0/286990514 conn(0x7f98640cc640 msgr2=0x7f98640cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.013+0000 7f9869253700 1 -- 172.21.15.7:0/286990514 shutdown_connections 2024-01-11T07:07:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.013+0000 7f9869253700 1 -- 172.21.15.7:0/286990514 wait complete. 2024-01-11T07:07:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.015+0000 7f9869253700 1 Processor -- start 2024-01-11T07:07:12.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.016+0000 7f9869253700 1 -- start start 2024-01-11T07:07:12.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.017+0000 7f9869253700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 0x7f9864171170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.017+0000 7f9869253700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98640c8a10 0x7f9864099780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.017+0000 7f9869253700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864155ea0 0x7f9864099cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.017+0000 7f9869253700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640968d0 con 0x7f9864155ea0 2024-01-11T07:07:12.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.017+0000 7f9869253700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640962c0 con 0x7f98640c8a10 2024-01-11T07:07:12.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.017+0000 7f9869253700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98641716b0 con 0x7f98640c13e0 2024-01-11T07:07:12.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.018+0000 7f9868a52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864155ea0 0x7f9864099cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.018+0000 7f9868a52700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864155ea0 0x7f9864099cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41568/0 (socket says 172.21.15.7:41568) 2024-01-11T07:07:12.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.018+0000 7f9868a52700 1 -- 172.21.15.7:0/996020873 learned_addr learned my addr 172.21.15.7:0/996020873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:12.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.019+0000 7f9868a52700 1 -- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 msgr2=0x7f9864171170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T07:07:12.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.019+0000 7f9868a52700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 0x7f9864171170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.019+0000 7f9868a52700 1 -- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98640c8a10 msgr2=0x7f9864099780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T07:07:12.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.019+0000 7f9868a52700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98640c8a10 0x7f9864099780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.019+0000 7f9868a52700 1 -- 172.21.15.7:0/996020873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985400c710 con 0x7f9864155ea0 2024-01-11T07:07:12.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.020+0000 7f9868a52700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864155ea0 0x7f9864099cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f985c00cb20 tx=0x7f985c001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.025+0000 7f100affd700 1 -- 172.21.15.7:0/2231128320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1018001ad0 con 0x7f102406ea00 2024-01-11T07:07:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.025+0000 7f100affd700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1010067080 0x7f1010069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.025+0000 7f100affd700 1 -- 172.21.15.7:0/2231128320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1018017030 con 0x7f102406ea00 2024-01-11T07:07:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.026+0000 7f100affd700 1 -- 172.21.15.7:0/2231128320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1018084920 con 0x7f102406ea00 2024-01-11T07:07:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.027+0000 7f1022ffd700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1010067080 0x7f1010069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.023+0000 7f984bfff700 1 -- 172.21.15.7:0/996020873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00a8b0 con 0x7f9864155ea0 2024-01-11T07:07:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.023+0000 7f984bfff700 1 -- 172.21.15.7:0/996020873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c00aa10 con 0x7f9864155ea0 2024-01-11T07:07:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.024+0000 7f984bfff700 1 -- 172.21.15.7:0/996020873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00adf0 con 0x7f9864155ea0 2024-01-11T07:07:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.024+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9864177dc0 con 0x7f9864155ea0 2024-01-11T07:07:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.024+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9864096560 con 0x7f9864155ea0 2024-01-11T07:07:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.030+0000 7f984bfff700 1 -- 172.21.15.7:0/996020873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f985c012630 con 0x7f9864155ea0 2024-01-11T07:07:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.031+0000 7f984bfff700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9850067080 0x7f9850069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.031+0000 7f984bfff700 1 -- 172.21.15.7:0/996020873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f985c0849f0 con 0x7f9864155ea0 2024-01-11T07:07:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.032+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98440052b0 con 0x7f9864155ea0 2024-01-11T07:07:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.036+0000 7f9863fff700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9850067080 0x7f9850069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.041+0000 7f984bfff700 1 -- 172.21.15.7:0/996020873 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f985c051c40 con 0x7f9864155ea0 2024-01-11T07:07:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.047+0000 7f1022ffd700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1010067080 0x7f1010069530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f101c009940 tx=0x7f101c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.048+0000 7f9863fff700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9850067080 0x7f9850069530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9858007fd0 tx=0x7f9858007b20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.261+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f260c06e770 con 0x7f260c0c7f00 2024-01-11T07:07:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.262+0000 7f25fbfff700 1 -- 172.21.15.7:0/1382059244 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f25fc025030 con 0x7f260c0c7f00 2024-01-11T07:07:12.264 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25f4067200 msgr2=0x7f25f40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25f4067200 0x7f25f40696b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2600009b80 tx=0x7f26000091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7f00 msgr2=0x7f260c155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7f00 0x7f260c155270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25fc018000 tx=0x7f25fc019000 comp rx=0 tx=0).stop 2024-01-11T07:07:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 shutdown_connections 2024-01-11T07:07:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25f4067200 0x7f25f40696b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f260c0cafb0 0x7f260c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f260c0c7b30 0x7f260c172510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 --2- 172.21.15.7:0/1382059244 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f260c0c7f00 0x7f260c155270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.266+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 >> 172.21.15.7:0/1382059244 conn(0x7f260c0cd090 msgr2=0x7f260c0cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.267+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 shutdown_connections 2024-01-11T07:07:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.267+0000 7f26126cb700 1 -- 172.21.15.7:0/1382059244 wait complete. 2024-01-11T07:07:12.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-01-11T07:07:12.287 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:12.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.354+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb1dc005550 con 0x7fb1fc15af90 2024-01-11T07:07:12.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.354+0000 7fb1f8ff9700 1 -- 172.21.15.7:0/2802786038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1f4055250 con 0x7fb1fc15af90 2024-01-11T07:07:12.355 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.356+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fec58005550 con 0x7fec7806e630 2024-01-11T07:07:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.356+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb1e0067100 msgr2=0x7fb1e00695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.356+0000 7fb202f18700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb1e0067100 0x7fb1e00695b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8006fd0 tx=0x7fb1e80090f0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.356+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1fc15af90 msgr2=0x7fb1fc0c2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.356+0000 7fb202f18700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1fc15af90 0x7fb1fc0c2f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400cb90 tx=0x7fb1f40019c0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.357+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 shutdown_connections 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.357+0000 7fb202f18700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb1e0067100 0x7fb1e00695b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.357+0000 7fb202f18700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1fc15af90 0x7fb1fc0c2f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.357+0000 7fb202f18700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1fc068bf0 0x7fb1fc149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.357+0000 7fb202f18700 1 --2- 172.21.15.7:0/2802786038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1fc06e850 0x7fb1fc0c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.357+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 >> 172.21.15.7:0/2802786038 conn(0x7fb1fc063230 msgr2=0x7fb1fc15bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.357+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 shutdown_connections 2024-01-11T07:07:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.357+0000 7fb202f18700 1 -- 172.21.15.7:0/2802786038 wait complete. 2024-01-11T07:07:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.358+0000 7fec6d7fa700 1 -- 172.21.15.7:0/3929813957 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fec70029020 con 0x7fec7806e630 2024-01-11T07:07:12.358 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.361+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fec640671e0 msgr2=0x7fec64069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.361+0000 7fec7f66b700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fec640671e0 0x7fec64069690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec680078a0 tx=0x7fec68007830 comp rx=0 tx=0).stop 2024-01-11T07:07:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.361+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 msgr2=0x7fec7817a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.361+0000 7fec7f66b700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 0x7fec7817a800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec70020000 tx=0x7fec70023000 comp rx=0 tx=0).stop 2024-01-11T07:07:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.362+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 shutdown_connections 2024-01-11T07:07:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.362+0000 7fec7f66b700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fec640671e0 0x7fec64069690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.362+0000 7fec7f66b700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec780c98b0 0x7fec780b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.362+0000 7fec7f66b700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec7806e630 0x7fec7817a800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.362+0000 7fec7f66b700 1 --2- 172.21.15.7:0/3929813957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7806ea00 0x7fec781662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.362+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 >> 172.21.15.7:0/3929813957 conn(0x7fec78063230 msgr2=0x7fec780cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.362+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 shutdown_connections 2024-01-11T07:07:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.362+0000 7fec7f66b700 1 -- 172.21.15.7:0/3929813957 wait complete. 2024-01-11T07:07:12.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-01-11T07:07:12.375 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:12.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-01-11T07:07:12.383 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:12.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.406+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe87804c3c0 con 0x7fe8780ca9d0 2024-01-11T07:07:12.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.407+0000 7fe86d7fa700 1 -- 172.21.15.7:0/2931670355 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe874017020 con 0x7fe8780ca9d0 2024-01-11T07:07:12.406 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe85c066f80 msgr2=0x7fe85c069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe85c066f80 0x7fe85c069430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe870007260 tx=0x7fe87001a000 comp rx=0 tx=0).stop 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8780ca9d0 msgr2=0x7fe8780880d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8780ca9d0 0x7fe8780880d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c590 tx=0x7fe87400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 shutdown_connections 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe85c066f80 0x7fe85c069430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe878166c00 0x7fe878167110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8780c8ab0 0x7fe8781889e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 --2- 172.21.15.7:0/2931670355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8780ca9d0 0x7fe8780880d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.410+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 >> 172.21.15.7:0/2931670355 conn(0x7fe878063230 msgr2=0x7fe878165f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.411+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 shutdown_connections 2024-01-11T07:07:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.411+0000 7fe87f3f2700 1 -- 172.21.15.7:0/2931670355 wait complete. 2024-01-11T07:07:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.428+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9844005560 con 0x7f9864155ea0 2024-01-11T07:07:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.429+0000 7f984bfff700 1 -- 172.21.15.7:0/996020873 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f985c016020 con 0x7f9864155ea0 2024-01-11T07:07:12.428 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:12.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-01-11T07:07:12.429 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9850067080 msgr2=0x7f9850069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9850067080 0x7f9850069530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9858007fd0 tx=0x7f9858007b20 comp rx=0 tx=0).stop 2024-01-11T07:07:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864155ea0 msgr2=0x7f9864099cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864155ea0 0x7f9864099cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f985c00cb20 tx=0x7f985c001950 comp rx=0 tx=0).stop 2024-01-11T07:07:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 shutdown_connections 2024-01-11T07:07:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9850067080 0x7f9850069530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864155ea0 0x7f9864099cc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98640c13e0 0x7f9864171170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 --2- 172.21.15.7:0/996020873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98640c8a10 0x7f9864099780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 >> 172.21.15.7:0/996020873 conn(0x7f98640cc640 msgr2=0x7f98640c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 shutdown_connections 2024-01-11T07:07:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.431+0000 7f9869253700 1 -- 172.21.15.7:0/996020873 wait complete. 2024-01-11T07:07:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.432+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4b90005550 con 0x7f4bb00c9110 2024-01-11T07:07:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.432+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde800b0b90 con 0x7fde800cbdf0 2024-01-11T07:07:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.433+0000 7f4b96ffd700 1 -- 172.21.15.7:0/964851006 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ba4017020 con 0x7f4bb00c9110 2024-01-11T07:07:12.433 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.433+0000 7fde6f7fe700 1 -- 172.21.15.7:0/1289121483 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde7001e090 con 0x7fde800cbdf0 2024-01-11T07:07:12.434 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.435+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b9c067080 msgr2=0x7f4b9c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.435+0000 7f4bb533d700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b9c067080 0x7f4b9c069530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00c7140 tx=0x7f4ba0008040 comp rx=0 tx=0).stop 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.435+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bb00c9110 msgr2=0x7f4bb014d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.435+0000 7f4bb533d700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bb00c9110 0x7f4bb014d710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400c530 tx=0x7f4ba400c8f0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.436+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 shutdown_connections 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.436+0000 7f4bb533d700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b9c067080 0x7f4b9c069530 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.436+0000 7f4bb533d700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bb015e120 0x7f4bb014bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.436+0000 7f4bb533d700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bb00c9110 0x7f4bb014d710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.436+0000 7f4bb533d700 1 --2- 172.21.15.7:0/964851006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4bb00cc1b0 0x7f4bb014ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.436+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 >> 172.21.15.7:0/964851006 conn(0x7f4bb00cd090 msgr2=0x7f4bb00c2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.436+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 shutdown_connections 2024-01-11T07:07:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.436+0000 7f4bb533d700 1 -- 172.21.15.7:0/964851006 wait complete. 2024-01-11T07:07:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde64067050 msgr2=0x7fde64069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde64067050 0x7fde64069500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde800c3830 tx=0x7fde7801b000 comp rx=0 tx=0).stop 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cbdf0 msgr2=0x7fde80157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cbdf0 0x7fde80157920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde7000c650 tx=0x7fde70008040 comp rx=0 tx=0).stop 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 shutdown_connections 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde64067050 0x7fde64069500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde800cbdf0 0x7fde80157920 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde800c6f30 0x7fde80150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 --2- 172.21.15.7:0/1289121483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde800cba20 0x7fde801421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 >> 172.21.15.7:0/1289121483 conn(0x7fde80063230 msgr2=0x7fde80068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 shutdown_connections 2024-01-11T07:07:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.437+0000 7fde85b07700 1 -- 172.21.15.7:0/1289121483 wait complete. 2024-01-11T07:07:12.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-01-11T07:07:12.449 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:12.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-01-11T07:07:12.452 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:12.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.453+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f10240c2cb0 con 0x7f102406ea00 2024-01-11T07:07:12.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.454+0000 7f100affd700 1 -- 172.21.15.7:0/2231128320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10240c2cb0 con 0x7f102406ea00 2024-01-11T07:07:12.453 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2024-01-11T07:07:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1010067080 msgr2=0x7f1010069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1010067080 0x7f1010069530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f101c009940 tx=0x7f101c017000 comp rx=0 tx=0).stop 2024-01-11T07:07:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f102406ea00 msgr2=0x7f10240c2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f102406ea00 0x7f10240c2770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f101800eee0 tx=0x7f1018007e40 comp rx=0 tx=0).stop 2024-01-11T07:07:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 shutdown_connections 2024-01-11T07:07:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1010067080 0x7f1010069530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10240c98b0 0x7f10240b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f102406e630 0x7f1024165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 --2- 172.21.15.7:0/2231128320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f102406ea00 0x7f10240c2770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 >> 172.21.15.7:0/2231128320 conn(0x7f1024063230 msgr2=0x7f102406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 shutdown_connections 2024-01-11T07:07:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.457+0000 7f10295f6700 1 -- 172.21.15.7:0/2231128320 wait complete. 2024-01-11T07:07:12.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-01-11T07:07:12.459 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:12.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-01-11T07:07:12.470 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:12.470 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-11T07:07:12.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.645+0000 7f61f4559700 1 Processor -- start 2024-01-11T07:07:12.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.645+0000 7f61f4559700 1 -- start start 2024-01-11T07:07:12.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f4559700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61ec154740 0x7f61ec0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f4559700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f4559700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ec0b1c10 con 0x7f61ec154370 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f4559700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ec0b1ef0 con 0x7f61ec154740 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f4559700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec15a7d0 con 0x7f61ec157ad0 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f1af4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61ec154740 0x7f61ec0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f1af4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61ec154740 0x7f61ec0b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41572/0 (socket says 172.21.15.7:41572) 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f22f5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f61ec154370 0x7f61ec15afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40274/0 (socket says 172.21.15.7:40274) 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f1af4700 1 -- 172.21.15.7:0/3568784634 learned_addr learned my addr 172.21.15.7:0/3568784634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.646+0000 7f61f2af6700 1 --2- 172.21.15.7:0/3568784634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f1af4700 1 -- 172.21.15.7:0/3568784634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 msgr2=0x7f61ec069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f1af4700 1 --2- 172.21.15.7:0/3568784634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f1af4700 1 -- 172.21.15.7:0/3568784634 >> v1:172.21.15.7:6789/0 conn(0x7f61ec154370 legacy=0x7f61ec15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f1af4700 1 -- 172.21.15.7:0/3568784634 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec15aab0 con 0x7f61ec154740 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f12f3700 1 -- 172.21.15.7:0/3568784634 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779118951 0 0) 0x7f61ec0b1c10 con 0x7f61ec154370 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f2af6700 1 --2- 172.21.15.7:0/3568784634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f1af4700 1 --2- 172.21.15.7:0/3568784634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154740 0x7f61ec0b1530 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f61e800bc50 tx=0x7f61e800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ba29dc613c5690 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f12f3700 1 -- 172.21.15.7:0/3568784634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e800be60 con 0x7f61ec154740 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f12f3700 1 -- 172.21.15.7:0/3568784634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e800de00 con 0x7f61ec154740 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f12f3700 1 -- 172.21.15.7:0/3568784634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80196b0 con 0x7f61ec154740 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f4559700 1 -- 172.21.15.7:0/3568784634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154740 msgr2=0x7f61ec0b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.647+0000 7f61f4559700 1 --2- 172.21.15.7:0/3568784634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154740 0x7f61ec0b1530 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f61e800bc50 tx=0x7f61e800b4f0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.648+0000 7f61f4559700 1 -- 172.21.15.7:0/3568784634 shutdown_connections 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.648+0000 7f61f4559700 1 --2- 172.21.15.7:0/3568784634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.648+0000 7f61f4559700 1 --2- 172.21.15.7:0/3568784634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154740 0x7f61ec0b1530 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.648+0000 7f61f4559700 1 -- 172.21.15.7:0/3568784634 >> 172.21.15.7:0/3568784634 conn(0x7f61ec0cd910 msgr2=0x7f61ec0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.648+0000 7f61f4559700 1 -- 172.21.15.7:0/3568784634 shutdown_connections 2024-01-11T07:07:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.648+0000 7f61f4559700 1 -- 172.21.15.7:0/3568784634 wait complete. 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.649+0000 7f61f4559700 1 Processor -- start 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.649+0000 7f61f4559700 1 -- start start 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f4559700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154370 0x7f61ec0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f4559700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61ec154740 0x7f61ec16b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f4559700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f4559700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec154c10 con 0x7f61ec154370 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f4559700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec155790 con 0x7f61ec154740 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f4559700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec0c96b0 con 0x7f61ec157ad0 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154370 0x7f61ec0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154370 0x7f61ec0c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41574/0 (socket says 172.21.15.7:41574) 2024-01-11T07:07:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 -- 172.21.15.7:0/420546985 learned_addr learned my addr 172.21.15.7:0/420546985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f1af4700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61ec154740 0x7f61ec16b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f2af6700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 -- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 msgr2=0x7f61ec155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 -- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61ec154740 msgr2=0x7f61ec16b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61ec154740 0x7f61ec16b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 -- 172.21.15.7:0/420546985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e800b790 con 0x7f61ec154370 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f22f5700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154370 0x7f61ec0c9170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00c7e0 tx=0x7f61dc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61e27fc700 1 -- 172.21.15.7:0/420546985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc005a70 con 0x7f61ec154370 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61e27fc700 1 -- 172.21.15.7:0/420546985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc005bd0 con 0x7f61ec154370 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec154e60 con 0x7f61ec154370 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.650+0000 7f61e27fc700 1 -- 172.21.15.7:0/420546985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0070d0 con 0x7f61ec154370 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.651+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec155420 con 0x7f61ec154370 2024-01-11T07:07:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.651+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ec04c3c0 con 0x7f61ec154370 2024-01-11T07:07:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.652+0000 7f61e27fc700 1 -- 172.21.15.7:0/420546985 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f61dc007440 con 0x7f61ec154370 2024-01-11T07:07:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.652+0000 7f61e27fc700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61d0066f30 0x7f61d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.653+0000 7f61e27fc700 1 -- 172.21.15.7:0/420546985 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61dc083a00 con 0x7f61ec154370 2024-01-11T07:07:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.653+0000 7f61f1af4700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61d0066f30 0x7f61d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.653+0000 7f61f1af4700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61d0066f30 0x7f61d00693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61e800bc50 tx=0x7f61e801b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.657+0000 7f61e27fc700 1 -- 172.21.15.7:0/420546985 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61dc050c70 con 0x7f61ec154370 2024-01-11T07:07:12.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.857+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61ec1558d0 con 0x7f61d0066f30 2024-01-11T07:07:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.865+0000 7f61e27fc700 1 -- 172.21.15.7:0/420546985 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260149 (secure 0 0 0) 0x7f61e800be60 con 0x7f61d0066f30 2024-01-11T07:07:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.868+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61d0066f30 msgr2=0x7f61d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.868+0000 7f61f4559700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61d0066f30 0x7f61d00693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61e800bc50 tx=0x7f61e801b000 comp rx=0 tx=0).stop 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.868+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154370 msgr2=0x7f61ec0c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.868+0000 7f61f4559700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154370 0x7f61ec0c9170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00c7e0 tx=0x7f61dc007e10 comp rx=0 tx=0).stop 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.869+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 shutdown_connections 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.869+0000 7f61f4559700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61d0066f30 0x7f61d00693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.869+0000 7f61f4559700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61ec157ad0 0x7f61ec155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.869+0000 7f61f4559700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ec154370 0x7f61ec0c9170 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.869+0000 7f61f4559700 1 --2- 172.21.15.7:0/420546985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61ec154740 0x7f61ec16b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.869+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 >> 172.21.15.7:0/420546985 conn(0x7f61ec0cd910 msgr2=0x7f61ec150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.869+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 shutdown_connections 2024-01-11T07:07:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:12.869+0000 7f61f4559700 1 -- 172.21.15.7:0/420546985 wait complete. 2024-01-11T07:07:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:12.881 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-11T07:07:11.841937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560715,"statfs":{"total":1600000000,"available":1598173068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.875097"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978080+0000","last_change":"2024-01-11T07:06:58.977426+0000","last_active":"2024-01-11T07:06:58.978080+0000","last_peered":"2024-01-11T07:06:58.978080+0000","last_clean":"2024-01-11T07:06:58.978080+0000","last_became_active":"2024-01-11T07:06:58.977173+0000","last_became_peered":"2024-01-11T07:06:58.977173+0000","last_unstale":"2024-01-11T07:06:58.978080+0000","last_undegraded":"2024-01-11T07:06:58.978080+0000","last_fullsized":"2024-01-11T07:06:58.978080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:27:13.892155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982635+0000","last_change":"2024-01-11T07:06:58.982509+0000","last_active":"2024-01-11T07:06:58.982635+0000","last_peered":"2024-01-11T07:06:58.982635+0000","last_clean":"2024-01-11T07:06:58.982635+0000","last_became_active":"2024-01-11T07:06:58.982234+0000","last_became_peered":"2024-01-11T07:06:58.982234+0000","last_unstale":"2024-01-11T07:06:58.982635+0000","last_undegraded":"2024-01-11T07:06:58.982635+0000","last_fullsized":"2024-01-11T07:06:58.982635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981566+0000","last_change":"2024-01-11T07:06:58.981034+0000","last_active":"2024-01-11T07:06:58.981566+0000","last_peered":"2024-01-11T07:06:58.981566+0000","last_clean":"2024-01-11T07:06:58.981566+0000","last_became_active":"2024-01-11T07:06:58.979996+0000","last_became_peered":"2024-01-11T07:06:58.979996+0000","last_unstale":"2024-01-11T07:06:58.981566+0000","last_undegraded":"2024-01-11T07:06:58.981566+0000","last_fullsized":"2024-01-11T07:06:58.981566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979116+0000","last_change":"2024-01-11T07:06:58.979053+0000","last_active":"2024-01-11T07:06:58.979116+0000","last_peered":"2024-01-11T07:06:58.979116+0000","last_clean":"2024-01-11T07:06:58.979116+0000","last_became_active":"2024-01-11T07:06:58.978648+0000","last_became_peered":"2024-01-11T07:06:58.978648+0000","last_unstale":"2024-01-11T07:06:58.979116+0000","last_undegraded":"2024-01-11T07:06:58.979116+0000","last_fullsized":"2024-01-11T07:06:58.979116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:11:14.542059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978059+0000","last_change":"2024-01-11T07:06:58.977393+0000","last_active":"2024-01-11T07:06:58.978059+0000","last_peered":"2024-01-11T07:06:58.978059+0000","last_clean":"2024-01-11T07:06:58.978059+0000","last_became_active":"2024-01-11T07:06:58.977145+0000","last_became_peered":"2024-01-11T07:06:58.977145+0000","last_unstale":"2024-01-11T07:06:58.978059+0000","last_undegraded":"2024-01-11T07:06:58.978059+0000","last_fullsized":"2024-01-11T07:06:58.978059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:07:39.492394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972053+0000","last_change":"2024-01-11T07:06:58.971961+0000","last_active":"2024-01-11T07:06:58.972053+0000","last_peered":"2024-01-11T07:06:58.972053+0000","last_clean":"2024-01-11T07:06:58.972053+0000","last_became_active":"2024-01-11T07:06:58.971577+0000","last_became_peered":"2024-01-11T07:06:58.971577+0000","last_unstale":"2024-01-11T07:06:58.972053+0000","last_undegraded":"2024-01-11T07:06:58.972053+0000","last_fullsized":"2024-01-11T07:06:58.972053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:27:13.892155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.984886+0000","last_change":"2024-01-11T07:06:58.984793+0000","last_active":"2024-01-11T07:06:58.984886+0000","last_peered":"2024-01-11T07:06:58.984886+0000","last_clean":"2024-01-11T07:06:58.984886+0000","last_became_active":"2024-01-11T07:06:58.984465+0000","last_became_peered":"2024-01-11T07:06:58.984465+0000","last_unstale":"2024-01-11T07:06:58.984886+0000","last_undegraded":"2024-01-11T07:06:58.984886+0000","last_fullsized":"2024-01-11T07:06:58.984886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:11:14.542059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977071+0000","last_change":"2024-01-11T07:06:58.976965+0000","last_active":"2024-01-11T07:06:58.977071+0000","last_peered":"2024-01-11T07:06:58.977071+0000","last_clean":"2024-01-11T07:06:58.977071+0000","last_became_active":"2024-01-11T07:06:58.976113+0000","last_became_peered":"2024-01-11T07:06:58.976113+0000","last_unstale":"2024-01-11T07:06:58.977071+0000","last_undegraded":"2024-01-11T07:06:58.977071+0000","last_fullsized":"2024-01-11T07:06:58.977071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978104+0000","last_change":"2024-01-11T07:06:58.977843+0000","last_active":"2024-01-11T07:06:58.978104+0000","last_peered":"2024-01-11T07:06:58.978104+0000","last_clean":"2024-01-11T07:06:58.978104+0000","last_became_active":"2024-01-11T07:06:58.976923+0000","last_became_peered":"2024-01-11T07:06:58.976923+0000","last_unstale":"2024-01-11T07:06:58.978104+0000","last_undegraded":"2024-01-11T07:06:58.978104+0000","last_fullsized":"2024-01-11T07:06:58.978104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:04:53.068090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975533+0000","last_change":"2024-01-11T07:06:58.975456+0000","last_active":"2024-01-11T07:06:58.975533+0000","last_peered":"2024-01-11T07:06:58.975533+0000","last_clean":"2024-01-11T07:06:58.975533+0000","last_became_active":"2024-01-11T07:06:58.974898+0000","last_became_peered":"2024-01-11T07:06:58.974898+0000","last_unstale":"2024-01-11T07:06:58.975533+0000","last_undegraded":"2024-01-11T07:06:58.975533+0000","last_fullsized":"2024-01-11T07:06:58.975533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975115+0000","last_change":"2024-01-11T07:06:58.975019+0000","last_active":"2024-01-11T07:06:58.975115+0000","last_peered":"2024-01-11T07:06:58.975115+0000","last_clean":"2024-01-11T07:06:58.975115+0000","last_became_active":"2024-01-11T07:06:58.974797+0000","last_became_peered":"2024-01-11T07:06:58.974797+0000","last_unstale":"2024-01-11T07:06:58.975115+0000","last_undegraded":"2024-01-11T07:06:58.975115+0000","last_fullsized":"2024-01-11T07:06:58.975115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975231+0000","last_change":"2024-01-11T07:06:58.974746+0000","last_active":"2024-01-11T07:06:58.975231+0000","last_peered":"2024-01-11T07:06:58.975231+0000","last_clean":"2024-01-11T07:06:58.975231+0000","last_became_active":"2024-01-11T07:06:58.974520+0000","last_became_peered":"2024-01-11T07:06:58.974520+0000","last_unstale":"2024-01-11T07:06:58.975231+0000","last_undegraded":"2024-01-11T07:06:58.975231+0000","last_fullsized":"2024-01-11T07:06:58.975231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982637+0000","last_change":"2024-01-11T07:06:58.982409+0000","last_active":"2024-01-11T07:06:58.982637+0000","last_peered":"2024-01-11T07:06:58.982637+0000","last_clean":"2024-01-11T07:06:58.982637+0000","last_became_active":"2024-01-11T07:06:58.981807+0000","last_became_peered":"2024-01-11T07:06:58.981807+0000","last_unstale":"2024-01-11T07:06:58.982637+0000","last_undegraded":"2024-01-11T07:06:58.982637+0000","last_fullsized":"2024-01-11T07:06:58.982637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978084+0000","last_change":"2024-01-11T07:06:58.977853+0000","last_active":"2024-01-11T07:06:58.978084+0000","last_peered":"2024-01-11T07:06:58.978084+0000","last_clean":"2024-01-11T07:06:58.978084+0000","last_became_active":"2024-01-11T07:06:58.977594+0000","last_became_peered":"2024-01-11T07:06:58.977594+0000","last_unstale":"2024-01-11T07:06:58.978084+0000","last_undegraded":"2024-01-11T07:06:58.978084+0000","last_fullsized":"2024-01-11T07:06:58.978084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982542+0000","last_change":"2024-01-11T07:06:58.982470+0000","last_active":"2024-01-11T07:06:58.982542+0000","last_peered":"2024-01-11T07:06:58.982542+0000","last_clean":"2024-01-11T07:06:58.982542+0000","last_became_active":"2024-01-11T07:06:58.982209+0000","last_became_peered":"2024-01-11T07:06:58.982209+0000","last_unstale":"2024-01-11T07:06:58.982542+0000","last_undegraded":"2024-01-11T07:06:58.982542+0000","last_fullsized":"2024-01-11T07:06:58.982542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971907+0000","last_change":"2024-01-11T07:06:58.971802+0000","last_active":"2024-01-11T07:06:58.971907+0000","last_peered":"2024-01-11T07:06:58.971907+0000","last_clean":"2024-01-11T07:06:58.971907+0000","last_became_active":"2024-01-11T07:06:58.971251+0000","last_became_peered":"2024-01-11T07:06:58.971251+0000","last_unstale":"2024-01-11T07:06:58.971907+0000","last_undegraded":"2024-01-11T07:06:58.971907+0000","last_fullsized":"2024-01-11T07:06:58.971907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:19.418614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983779+0000","last_change":"2024-01-11T07:06:58.983363+0000","last_active":"2024-01-11T07:06:58.983779+0000","last_peered":"2024-01-11T07:06:58.983779+0000","last_clean":"2024-01-11T07:06:58.983779+0000","last_became_active":"2024-01-11T07:06:58.982003+0000","last_became_peered":"2024-01-11T07:06:58.982003+0000","last_unstale":"2024-01-11T07:06:58.983779+0000","last_undegraded":"2024-01-11T07:06:58.983779+0000","last_fullsized":"2024-01-11T07:06:58.983779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:22:00.354437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975549+0000","last_change":"2024-01-11T07:06:58.975439+0000","last_active":"2024-01-11T07:06:58.975549+0000","last_peered":"2024-01-11T07:06:58.975549+0000","last_clean":"2024-01-11T07:06:58.975549+0000","last_became_active":"2024-01-11T07:06:58.975201+0000","last_became_peered":"2024-01-11T07:06:58.975201+0000","last_unstale":"2024-01-11T07:06:58.975549+0000","last_undegraded":"2024-01-11T07:06:58.975549+0000","last_fullsized":"2024-01-11T07:06:58.975549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978122+0000","last_change":"2024-01-11T07:06:58.977996+0000","last_active":"2024-01-11T07:06:58.978122+0000","last_peered":"2024-01-11T07:06:58.978122+0000","last_clean":"2024-01-11T07:06:58.978122+0000","last_became_active":"2024-01-11T07:06:58.977268+0000","last_became_peered":"2024-01-11T07:06:58.977268+0000","last_unstale":"2024-01-11T07:06:58.978122+0000","last_undegraded":"2024-01-11T07:06:58.978122+0000","last_fullsized":"2024-01-11T07:06:58.978122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:19.418614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.158069+0000","last_change":"2024-01-11T07:06:57.964441+0000","last_active":"2024-01-11T07:06:59.158069+0000","last_peered":"2024-01-11T07:06:59.158069+0000","last_clean":"2024-01-11T07:06:59.158069+0000","last_became_active":"2024-01-11T07:06:57.963971+0000","last_became_peered":"2024-01-11T07:06:57.963971+0000","last_unstale":"2024-01-11T07:06:59.158069+0000","last_undegraded":"2024-01-11T07:06:59.158069+0000","last_fullsized":"2024-01-11T07:06:59.158069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980035+0000","last_change":"2024-01-11T07:06:58.979891+0000","last_active":"2024-01-11T07:06:58.980035+0000","last_peered":"2024-01-11T07:06:58.980035+0000","last_clean":"2024-01-11T07:06:58.980035+0000","last_became_active":"2024-01-11T07:06:58.979638+0000","last_became_peered":"2024-01-11T07:06:58.979638+0000","last_unstale":"2024-01-11T07:06:58.980035+0000","last_undegraded":"2024-01-11T07:06:58.980035+0000","last_fullsized":"2024-01-11T07:06:58.980035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157701+0000","last_change":"2024-01-11T07:06:57.958809+0000","last_active":"2024-01-11T07:06:59.157701+0000","last_peered":"2024-01-11T07:06:59.157701+0000","last_clean":"2024-01-11T07:06:59.157701+0000","last_became_active":"2024-01-11T07:06:57.958478+0000","last_became_peered":"2024-01-11T07:06:57.958478+0000","last_unstale":"2024-01-11T07:06:59.157701+0000","last_undegraded":"2024-01-11T07:06:59.157701+0000","last_fullsized":"2024-01-11T07:06:59.157701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.969856+0000","last_change":"2024-01-11T07:06:58.969781+0000","last_active":"2024-01-11T07:06:58.969856+0000","last_peered":"2024-01-11T07:06:58.969856+0000","last_clean":"2024-01-11T07:06:58.969856+0000","last_became_active":"2024-01-11T07:06:58.969505+0000","last_became_peered":"2024-01-11T07:06:58.969505+0000","last_unstale":"2024-01-11T07:06:58.969856+0000","last_undegraded":"2024-01-11T07:06:58.969856+0000","last_fullsized":"2024-01-11T07:06:58.969856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156412+0000","last_change":"2024-01-11T07:06:57.967660+0000","last_active":"2024-01-11T07:06:59.156412+0000","last_peered":"2024-01-11T07:06:59.156412+0000","last_clean":"2024-01-11T07:06:59.156412+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:59.156412+0000","last_undegraded":"2024-01-11T07:06:59.156412+0000","last_fullsized":"2024-01-11T07:06:59.156412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970064+0000","last_change":"2024-01-11T07:06:58.969973+0000","last_active":"2024-01-11T07:06:58.970064+0000","last_peered":"2024-01-11T07:06:58.970064+0000","last_clean":"2024-01-11T07:06:58.970064+0000","last_became_active":"2024-01-11T07:06:58.969722+0000","last_became_peered":"2024-01-11T07:06:58.969722+0000","last_unstale":"2024-01-11T07:06:58.970064+0000","last_undegraded":"2024-01-11T07:06:58.970064+0000","last_fullsized":"2024-01-11T07:06:58.970064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.154688+0000","last_change":"2024-01-11T07:06:57.967196+0000","last_active":"2024-01-11T07:06:59.154688+0000","last_peered":"2024-01-11T07:06:59.154688+0000","last_clean":"2024-01-11T07:06:59.154688+0000","last_became_active":"2024-01-11T07:06:57.966964+0000","last_became_peered":"2024-01-11T07:06:57.966964+0000","last_unstale":"2024-01-11T07:06:59.154688+0000","last_undegraded":"2024-01-11T07:06:59.154688+0000","last_fullsized":"2024-01-11T07:06:59.154688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:27.038629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970044+0000","last_change":"2024-01-11T07:06:58.969967+0000","last_active":"2024-01-11T07:06:58.970044+0000","last_peered":"2024-01-11T07:06:58.970044+0000","last_clean":"2024-01-11T07:06:58.970044+0000","last_became_active":"2024-01-11T07:06:58.969708+0000","last_became_peered":"2024-01-11T07:06:58.969708+0000","last_unstale":"2024-01-11T07:06:58.970044+0000","last_undegraded":"2024-01-11T07:06:58.970044+0000","last_fullsized":"2024-01-11T07:06:58.970044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157494+0000","last_change":"2024-01-11T07:06:57.964730+0000","last_active":"2024-01-11T07:06:59.157494+0000","last_peered":"2024-01-11T07:06:59.157494+0000","last_clean":"2024-01-11T07:06:59.157494+0000","last_became_active":"2024-01-11T07:06:57.964464+0000","last_became_peered":"2024-01-11T07:06:57.964464+0000","last_unstale":"2024-01-11T07:06:59.157494+0000","last_undegraded":"2024-01-11T07:06:59.157494+0000","last_fullsized":"2024-01-11T07:06:59.157494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979054+0000","last_change":"2024-01-11T07:06:58.978898+0000","last_active":"2024-01-11T07:06:58.979054+0000","last_peered":"2024-01-11T07:06:58.979054+0000","last_clean":"2024-01-11T07:06:58.979054+0000","last_became_active":"2024-01-11T07:06:58.978593+0000","last_became_peered":"2024-01-11T07:06:58.978593+0000","last_unstale":"2024-01-11T07:06:58.979054+0000","last_undegraded":"2024-01-11T07:06:58.979054+0000","last_fullsized":"2024-01-11T07:06:58.979054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156518+0000","last_change":"2024-01-11T07:06:57.961158+0000","last_active":"2024-01-11T07:06:59.156518+0000","last_peered":"2024-01-11T07:06:59.156518+0000","last_clean":"2024-01-11T07:06:59.156518+0000","last_became_active":"2024-01-11T07:06:57.960836+0000","last_became_peered":"2024-01-11T07:06:57.960836+0000","last_unstale":"2024-01-11T07:06:59.156518+0000","last_undegraded":"2024-01-11T07:06:59.156518+0000","last_fullsized":"2024-01-11T07:06:59.156518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968566+0000","last_change":"2024-01-11T07:06:58.968497+0000","last_active":"2024-01-11T07:06:58.968566+0000","last_peered":"2024-01-11T07:06:58.968566+0000","last_clean":"2024-01-11T07:06:58.968566+0000","last_became_active":"2024-01-11T07:06:58.968280+0000","last_became_peered":"2024-01-11T07:06:58.968280+0000","last_unstale":"2024-01-11T07:06:58.968566+0000","last_undegraded":"2024-01-11T07:06:58.968566+0000","last_fullsized":"2024-01-11T07:06:58.968566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975862+0000","last_change":"2024-01-11T07:06:57.963583+0000","last_active":"2024-01-11T07:06:58.975862+0000","last_peered":"2024-01-11T07:06:58.975862+0000","last_clean":"2024-01-11T07:06:58.975862+0000","last_became_active":"2024-01-11T07:06:57.962966+0000","last_became_peered":"2024-01-11T07:06:57.962966+0000","last_unstale":"2024-01-11T07:06:58.975862+0000","last_undegraded":"2024-01-11T07:06:58.975862+0000","last_fullsized":"2024-01-11T07:06:58.975862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980008+0000","last_change":"2024-01-11T07:06:58.979636+0000","last_active":"2024-01-11T07:06:58.980008+0000","last_peered":"2024-01-11T07:06:58.980008+0000","last_clean":"2024-01-11T07:06:58.980008+0000","last_became_active":"2024-01-11T07:06:58.979362+0000","last_became_peered":"2024-01-11T07:06:58.979362+0000","last_unstale":"2024-01-11T07:06:58.980008+0000","last_undegraded":"2024-01-11T07:06:58.980008+0000","last_fullsized":"2024-01-11T07:06:58.980008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968179+0000","last_change":"2024-01-11T07:06:57.965145+0000","last_active":"2024-01-11T07:06:58.968179+0000","last_peered":"2024-01-11T07:06:58.968179+0000","last_clean":"2024-01-11T07:06:58.968179+0000","last_became_active":"2024-01-11T07:06:57.964890+0000","last_became_peered":"2024-01-11T07:06:57.964890+0000","last_unstale":"2024-01-11T07:06:58.968179+0000","last_undegraded":"2024-01-11T07:06:58.968179+0000","last_fullsized":"2024-01-11T07:06:58.968179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982521+0000","last_change":"2024-01-11T07:06:58.982437+0000","last_active":"2024-01-11T07:06:58.982521+0000","last_peered":"2024-01-11T07:06:58.982521+0000","last_clean":"2024-01-11T07:06:58.982521+0000","last_became_active":"2024-01-11T07:06:58.982170+0000","last_became_peered":"2024-01-11T07:06:58.982170+0000","last_unstale":"2024-01-11T07:06:58.982521+0000","last_undegraded":"2024-01-11T07:06:58.982521+0000","last_fullsized":"2024-01-11T07:06:58.982521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972373+0000","last_change":"2024-01-11T07:06:57.967613+0000","last_active":"2024-01-11T07:06:58.972373+0000","last_peered":"2024-01-11T07:06:58.972373+0000","last_clean":"2024-01-11T07:06:58.972373+0000","last_became_active":"2024-01-11T07:06:57.967356+0000","last_became_peered":"2024-01-11T07:06:57.967356+0000","last_unstale":"2024-01-11T07:06:58.972373+0000","last_undegraded":"2024-01-11T07:06:58.972373+0000","last_fullsized":"2024-01-11T07:06:58.972373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983217+0000","last_change":"2024-01-11T07:06:58.983040+0000","last_active":"2024-01-11T07:06:58.983217+0000","last_peered":"2024-01-11T07:06:58.983217+0000","last_clean":"2024-01-11T07:06:58.983217+0000","last_became_active":"2024-01-11T07:06:58.981241+0000","last_became_peered":"2024-01-11T07:06:58.981241+0000","last_unstale":"2024-01-11T07:06:58.983217+0000","last_undegraded":"2024-01-11T07:06:58.983217+0000","last_fullsized":"2024-01-11T07:06:58.983217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:57:17.525152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.967926+0000","last_change":"2024-01-11T07:06:57.966453+0000","last_active":"2024-01-11T07:06:58.967926+0000","last_peered":"2024-01-11T07:06:58.967926+0000","last_clean":"2024-01-11T07:06:58.967926+0000","last_became_active":"2024-01-11T07:06:57.966083+0000","last_became_peered":"2024-01-11T07:06:57.966083+0000","last_unstale":"2024-01-11T07:06:58.967926+0000","last_undegraded":"2024-01-11T07:06:58.967926+0000","last_fullsized":"2024-01-11T07:06:58.967926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970715+0000","last_change":"2024-01-11T07:06:58.970636+0000","last_active":"2024-01-11T07:06:58.970715+0000","last_peered":"2024-01-11T07:06:58.970715+0000","last_clean":"2024-01-11T07:06:58.970715+0000","last_became_active":"2024-01-11T07:06:58.970180+0000","last_became_peered":"2024-01-11T07:06:58.970180+0000","last_unstale":"2024-01-11T07:06:58.970715+0000","last_undegraded":"2024-01-11T07:06:58.970715+0000","last_fullsized":"2024-01-11T07:06:58.970715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156721+0000","last_change":"2024-01-11T07:06:57.966021+0000","last_active":"2024-01-11T07:06:59.156721+0000","last_peered":"2024-01-11T07:06:59.156721+0000","last_clean":"2024-01-11T07:06:59.156721+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:59.156721+0000","last_undegraded":"2024-01-11T07:06:59.156721+0000","last_fullsized":"2024-01-11T07:06:59.156721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980149+0000","last_change":"2024-01-11T07:06:58.980082+0000","last_active":"2024-01-11T07:06:58.980149+0000","last_peered":"2024-01-11T07:06:58.980149+0000","last_clean":"2024-01-11T07:06:58.980149+0000","last_became_active":"2024-01-11T07:06:58.979407+0000","last_became_peered":"2024-01-11T07:06:58.979407+0000","last_unstale":"2024-01-11T07:06:58.980149+0000","last_undegraded":"2024-01-11T07:06:58.980149+0000","last_fullsized":"2024-01-11T07:06:58.980149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.158497+0000","last_change":"2024-01-11T07:06:57.962108+0000","last_active":"2024-01-11T07:06:59.158497+0000","last_peered":"2024-01-11T07:06:59.158497+0000","last_clean":"2024-01-11T07:06:59.158497+0000","last_became_active":"2024-01-11T07:06:57.961848+0000","last_became_peered":"2024-01-11T07:06:57.961848+0000","last_unstale":"2024-01-11T07:06:59.158497+0000","last_undegraded":"2024-01-11T07:06:59.158497+0000","last_fullsized":"2024-01-11T07:06:59.158497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980308+0000","last_change":"2024-01-11T07:06:58.980239+0000","last_active":"2024-01-11T07:06:58.980308+0000","last_peered":"2024-01-11T07:06:58.980308+0000","last_clean":"2024-01-11T07:06:58.980308+0000","last_became_active":"2024-01-11T07:06:58.979974+0000","last_became_peered":"2024-01-11T07:06:58.979974+0000","last_unstale":"2024-01-11T07:06:58.980308+0000","last_undegraded":"2024-01-11T07:06:58.980308+0000","last_fullsized":"2024-01-11T07:06:58.980308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:57:17.525152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155524+0000","last_change":"2024-01-11T07:06:57.960489+0000","last_active":"2024-01-11T07:06:59.155524+0000","last_peered":"2024-01-11T07:06:59.155524+0000","last_clean":"2024-01-11T07:06:59.155524+0000","last_became_active":"2024-01-11T07:06:57.960225+0000","last_became_peered":"2024-01-11T07:06:57.960225+0000","last_unstale":"2024-01-11T07:06:59.155524+0000","last_undegraded":"2024-01-11T07:06:59.155524+0000","last_fullsized":"2024-01-11T07:06:59.155524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978545+0000","last_change":"2024-01-11T07:06:58.978471+0000","last_active":"2024-01-11T07:06:58.978545+0000","last_peered":"2024-01-11T07:06:58.978545+0000","last_clean":"2024-01-11T07:06:58.978545+0000","last_became_active":"2024-01-11T07:06:58.977884+0000","last_became_peered":"2024-01-11T07:06:58.977884+0000","last_unstale":"2024-01-11T07:06:58.978545+0000","last_undegraded":"2024-01-11T07:06:58.978545+0000","last_fullsized":"2024-01-11T07:06:58.978545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968025+0000","last_change":"2024-01-11T07:06:57.965935+0000","last_active":"2024-01-11T07:06:58.968025+0000","last_peered":"2024-01-11T07:06:58.968025+0000","last_clean":"2024-01-11T07:06:58.968025+0000","last_became_active":"2024-01-11T07:06:57.965303+0000","last_became_peered":"2024-01-11T07:06:57.965303+0000","last_unstale":"2024-01-11T07:06:58.968025+0000","last_undegraded":"2024-01-11T07:06:58.968025+0000","last_fullsized":"2024-01-11T07:06:58.968025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981838+0000","last_change":"2024-01-11T07:06:58.981581+0000","last_active":"2024-01-11T07:06:58.981838+0000","last_peered":"2024-01-11T07:06:58.981838+0000","last_clean":"2024-01-11T07:06:58.981838+0000","last_became_active":"2024-01-11T07:06:58.981032+0000","last_became_peered":"2024-01-11T07:06:58.981032+0000","last_unstale":"2024-01-11T07:06:58.981838+0000","last_undegraded":"2024-01-11T07:06:58.981838+0000","last_fullsized":"2024-01-11T07:06:58.981838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:05:09.439404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973296+0000","last_change":"2024-01-11T07:06:57.960488+0000","last_active":"2024-01-11T07:06:58.973296+0000","last_peered":"2024-01-11T07:06:58.973296+0000","last_clean":"2024-01-11T07:06:58.973296+0000","last_became_active":"2024-01-11T07:06:57.959963+0000","last_became_peered":"2024-01-11T07:06:57.959963+0000","last_unstale":"2024-01-11T07:06:58.973296+0000","last_undegraded":"2024-01-11T07:06:58.973296+0000","last_fullsized":"2024-01-11T07:06:58.973296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982184+0000","last_change":"2024-01-11T07:06:58.982110+0000","last_active":"2024-01-11T07:06:58.982184+0000","last_peered":"2024-01-11T07:06:58.982184+0000","last_clean":"2024-01-11T07:06:58.982184+0000","last_became_active":"2024-01-11T07:06:58.981828+0000","last_became_peered":"2024-01-11T07:06:58.981828+0000","last_unstale":"2024-01-11T07:06:58.982184+0000","last_undegraded":"2024-01-11T07:06:58.982184+0000","last_fullsized":"2024-01-11T07:06:58.982184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.159786+0000","last_change":"2024-01-11T07:06:57.961526+0000","last_active":"2024-01-11T07:06:59.159786+0000","last_peered":"2024-01-11T07:06:59.159786+0000","last_clean":"2024-01-11T07:06:59.159786+0000","last_became_active":"2024-01-11T07:06:57.961256+0000","last_became_peered":"2024-01-11T07:06:57.961256+0000","last_unstale":"2024-01-11T07:06:59.159786+0000","last_undegraded":"2024-01-11T07:06:59.159786+0000","last_fullsized":"2024-01-11T07:06:59.159786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980041+0000","last_change":"2024-01-11T07:06:58.979973+0000","last_active":"2024-01-11T07:06:58.980041+0000","last_peered":"2024-01-11T07:06:58.980041+0000","last_clean":"2024-01-11T07:06:58.980041+0000","last_became_active":"2024-01-11T07:06:58.979738+0000","last_became_peered":"2024-01-11T07:06:58.979738+0000","last_unstale":"2024-01-11T07:06:58.980041+0000","last_undegraded":"2024-01-11T07:06:58.980041+0000","last_fullsized":"2024-01-11T07:06:58.980041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:05.350780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155041+0000","last_change":"2024-01-11T07:06:57.967501+0000","last_active":"2024-01-11T07:06:59.155041+0000","last_peered":"2024-01-11T07:06:59.155041+0000","last_clean":"2024-01-11T07:06:59.155041+0000","last_became_active":"2024-01-11T07:06:57.966742+0000","last_became_peered":"2024-01-11T07:06:57.966742+0000","last_unstale":"2024-01-11T07:06:59.155041+0000","last_undegraded":"2024-01-11T07:06:59.155041+0000","last_fullsized":"2024-01-11T07:06:59.155041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:43:51.416880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981871+0000","last_change":"2024-01-11T07:06:58.981779+0000","last_active":"2024-01-11T07:06:58.981871+0000","last_peered":"2024-01-11T07:06:58.981871+0000","last_clean":"2024-01-11T07:06:58.981871+0000","last_became_active":"2024-01-11T07:06:58.981332+0000","last_became_peered":"2024-01-11T07:06:58.981332+0000","last_unstale":"2024-01-11T07:06:58.981871+0000","last_undegraded":"2024-01-11T07:06:58.981871+0000","last_fullsized":"2024-01-11T07:06:58.981871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:04:40.488711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.976544+0000","last_change":"2024-01-11T07:06:57.960497+0000","last_active":"2024-01-11T07:06:58.976544+0000","last_peered":"2024-01-11T07:06:58.976544+0000","last_clean":"2024-01-11T07:06:58.976544+0000","last_became_active":"2024-01-11T07:06:57.959939+0000","last_became_peered":"2024-01-11T07:06:57.959939+0000","last_unstale":"2024-01-11T07:06:58.976544+0000","last_undegraded":"2024-01-11T07:06:58.976544+0000","last_fullsized":"2024-01-11T07:06:58.976544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982201+0000","last_change":"2024-01-11T07:06:58.982135+0000","last_active":"2024-01-11T07:06:58.982201+0000","last_peered":"2024-01-11T07:06:58.982201+0000","last_clean":"2024-01-11T07:06:58.982201+0000","last_became_active":"2024-01-11T07:06:58.981852+0000","last_became_peered":"2024-01-11T07:06:58.981852+0000","last_unstale":"2024-01-11T07:06:58.982201+0000","last_undegraded":"2024-01-11T07:06:58.982201+0000","last_fullsized":"2024-01-11T07:06:58.982201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:05.350780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968680+0000","last_change":"2024-01-11T07:06:57.966310+0000","last_active":"2024-01-11T07:06:58.968680+0000","last_peered":"2024-01-11T07:06:58.968680+0000","last_clean":"2024-01-11T07:06:58.968680+0000","last_became_active":"2024-01-11T07:06:57.965787+0000","last_became_peered":"2024-01-11T07:06:57.965787+0000","last_unstale":"2024-01-11T07:06:58.968680+0000","last_undegraded":"2024-01-11T07:06:58.968680+0000","last_fullsized":"2024-01-11T07:06:58.968680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970968+0000","last_change":"2024-01-11T07:06:58.970832+0000","last_active":"2024-01-11T07:06:58.970968+0000","last_peered":"2024-01-11T07:06:58.970968+0000","last_clean":"2024-01-11T07:06:58.970968+0000","last_became_active":"2024-01-11T07:06:58.970599+0000","last_became_peered":"2024-01-11T07:06:58.970599+0000","last_unstale":"2024-01-11T07:06:58.970968+0000","last_undegraded":"2024-01-11T07:06:58.970968+0000","last_fullsized":"2024-01-11T07:06:58.970968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977892+0000","last_change":"2024-01-11T07:06:58.977802+0000","last_active":"2024-01-11T07:06:58.977892+0000","last_peered":"2024-01-11T07:06:58.977892+0000","last_clean":"2024-01-11T07:06:58.977892+0000","last_became_active":"2024-01-11T07:06:58.977558+0000","last_became_peered":"2024-01-11T07:06:58.977558+0000","last_unstale":"2024-01-11T07:06:58.977892+0000","last_undegraded":"2024-01-11T07:06:58.977892+0000","last_fullsized":"2024-01-11T07:06:58.977892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973773+0000","last_change":"2024-01-11T07:06:57.963281+0000","last_active":"2024-01-11T07:06:58.973773+0000","last_peered":"2024-01-11T07:06:58.973773+0000","last_clean":"2024-01-11T07:06:58.973773+0000","last_became_active":"2024-01-11T07:06:57.962993+0000","last_became_peered":"2024-01-11T07:06:57.962993+0000","last_unstale":"2024-01-11T07:06:58.973773+0000","last_undegraded":"2024-01-11T07:06:58.973773+0000","last_fullsized":"2024-01-11T07:06:58.973773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982425+0000","last_change":"2024-01-11T07:06:58.982334+0000","last_active":"2024-01-11T07:06:58.982425+0000","last_peered":"2024-01-11T07:06:58.982425+0000","last_clean":"2024-01-11T07:06:58.982425+0000","last_became_active":"2024-01-11T07:06:58.981895+0000","last_became_peered":"2024-01-11T07:06:58.981895+0000","last_unstale":"2024-01-11T07:06:58.982425+0000","last_undegraded":"2024-01-11T07:06:58.982425+0000","last_fullsized":"2024-01-11T07:06:58.982425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968343+0000","last_change":"2024-01-11T07:06:57.965590+0000","last_active":"2024-01-11T07:06:58.968343+0000","last_peered":"2024-01-11T07:06:58.968343+0000","last_clean":"2024-01-11T07:06:58.968343+0000","last_became_active":"2024-01-11T07:06:57.964910+0000","last_became_peered":"2024-01-11T07:06:57.964910+0000","last_unstale":"2024-01-11T07:06:58.968343+0000","last_undegraded":"2024-01-11T07:06:58.968343+0000","last_fullsized":"2024-01-11T07:06:58.968343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972745+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:06:58.972745+0000","last_peered":"2024-01-11T07:06:58.972745+0000","last_clean":"2024-01-11T07:06:58.972745+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:06:58.972745+0000","last_undegraded":"2024-01-11T07:06:58.972745+0000","last_fullsized":"2024-01-11T07:06:58.972745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.969817+0000","last_change":"2024-01-11T07:06:58.969690+0000","last_active":"2024-01-11T07:06:58.969817+0000","last_peered":"2024-01-11T07:06:58.969817+0000","last_clean":"2024-01-11T07:06:58.969817+0000","last_became_active":"2024-01-11T07:06:58.969269+0000","last_became_peered":"2024-01-11T07:06:58.969269+0000","last_unstale":"2024-01-11T07:06:58.969817+0000","last_undegraded":"2024-01-11T07:06:58.969817+0000","last_fullsized":"2024-01-11T07:06:58.969817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971432+0000","last_change":"2024-01-11T07:06:57.964928+0000","last_active":"2024-01-11T07:06:58.971432+0000","last_peered":"2024-01-11T07:06:58.971432+0000","last_clean":"2024-01-11T07:06:58.971432+0000","last_became_active":"2024-01-11T07:06:57.964692+0000","last_became_peered":"2024-01-11T07:06:57.964692+0000","last_unstale":"2024-01-11T07:06:58.971432+0000","last_undegraded":"2024-01-11T07:06:58.971432+0000","last_fullsized":"2024-01-11T07:06:58.971432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971975+0000","last_change":"2024-01-11T07:06:57.967422+0000","last_active":"2024-01-11T07:06:58.971975+0000","last_peered":"2024-01-11T07:06:58.971975+0000","last_clean":"2024-01-11T07:06:58.971975+0000","last_became_active":"2024-01-11T07:06:57.967213+0000","last_became_peered":"2024-01-11T07:06:57.967213+0000","last_unstale":"2024-01-11T07:06:58.971975+0000","last_undegraded":"2024-01-11T07:06:58.971975+0000","last_fullsized":"2024-01-11T07:06:58.971975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:16.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977579+0000","last_change":"2024-01-11T07:06:58.977325+0000","last_active":"2024-01-11T07:06:58.977579+0000","last_peered":"2024-01-11T07:06:58.977579+0000","last_clean":"2024-01-11T07:06:58.977579+0000","last_became_active":"2024-01-11T07:06:58.977041+0000","last_became_peered":"2024-01-11T07:06:58.977041+0000","last_unstale":"2024-01-11T07:06:58.977579+0000","last_undegraded":"2024-01-11T07:06:58.977579+0000","last_fullsized":"2024-01-11T07:06:58.977579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-11T07:07:00.222933+0000","last_change":"2024-01-11T07:06:57.958705+0000","last_active":"2024-01-11T07:07:00.222933+0000","last_peered":"2024-01-11T07:07:00.222933+0000","last_clean":"2024-01-11T07:07:00.222933+0000","last_became_active":"2024-01-11T07:06:57.958421+0000","last_became_peered":"2024-01-11T07:06:57.958421+0000","last_unstale":"2024-01-11T07:07:00.222933+0000","last_undegraded":"2024-01-11T07:07:00.222933+0000","last_fullsized":"2024-01-11T07:07:00.222933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980832+0000","last_change":"2024-01-11T07:06:58.980504+0000","last_active":"2024-01-11T07:06:58.980832+0000","last_peered":"2024-01-11T07:06:58.980832+0000","last_clean":"2024-01-11T07:06:58.980832+0000","last_became_active":"2024-01-11T07:06:58.980255+0000","last_became_peered":"2024-01-11T07:06:58.980255+0000","last_unstale":"2024-01-11T07:06:58.980832+0000","last_undegraded":"2024-01-11T07:06:58.980832+0000","last_fullsized":"2024-01-11T07:06:58.980832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157334+0000","last_change":"2024-01-11T07:06:57.964975+0000","last_active":"2024-01-11T07:06:59.157334+0000","last_peered":"2024-01-11T07:06:59.157334+0000","last_clean":"2024-01-11T07:06:59.157334+0000","last_became_active":"2024-01-11T07:06:57.964709+0000","last_became_peered":"2024-01-11T07:06:57.964709+0000","last_unstale":"2024-01-11T07:06:59.157334+0000","last_undegraded":"2024-01-11T07:06:59.157334+0000","last_fullsized":"2024-01-11T07:06:59.157334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981582+0000","last_change":"2024-01-11T07:06:58.981191+0000","last_active":"2024-01-11T07:06:58.981582+0000","last_peered":"2024-01-11T07:06:58.981582+0000","last_clean":"2024-01-11T07:06:58.981582+0000","last_became_active":"2024-01-11T07:06:58.980786+0000","last_became_peered":"2024-01-11T07:06:58.980786+0000","last_unstale":"2024-01-11T07:06:58.981582+0000","last_undegraded":"2024-01-11T07:06:58.981582+0000","last_fullsized":"2024-01-11T07:06:58.981582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975354+0000","last_change":"2024-01-11T07:06:57.963735+0000","last_active":"2024-01-11T07:06:58.975354+0000","last_peered":"2024-01-11T07:06:58.975354+0000","last_clean":"2024-01-11T07:06:58.975354+0000","last_became_active":"2024-01-11T07:06:57.963471+0000","last_became_peered":"2024-01-11T07:06:57.963471+0000","last_unstale":"2024-01-11T07:06:58.975354+0000","last_undegraded":"2024-01-11T07:06:58.975354+0000","last_fullsized":"2024-01-11T07:06:58.975354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978396+0000","last_change":"2024-01-11T07:06:58.978278+0000","last_active":"2024-01-11T07:06:58.978396+0000","last_peered":"2024-01-11T07:06:58.978396+0000","last_clean":"2024-01-11T07:06:58.978396+0000","last_became_active":"2024-01-11T07:06:58.976004+0000","last_became_peered":"2024-01-11T07:06:58.976004+0000","last_unstale":"2024-01-11T07:06:58.978396+0000","last_undegraded":"2024-01-11T07:06:58.978396+0000","last_fullsized":"2024-01-11T07:06:58.978396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.976655+0000","last_change":"2024-01-11T07:06:57.965092+0000","last_active":"2024-01-11T07:06:58.976655+0000","last_peered":"2024-01-11T07:06:58.976655+0000","last_clean":"2024-01-11T07:06:58.976655+0000","last_became_active":"2024-01-11T07:06:57.964872+0000","last_became_peered":"2024-01-11T07:06:57.964872+0000","last_unstale":"2024-01-11T07:06:58.976655+0000","last_undegraded":"2024-01-11T07:06:58.976655+0000","last_fullsized":"2024-01-11T07:06:58.976655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981559+0000","last_change":"2024-01-11T07:06:58.981370+0000","last_active":"2024-01-11T07:06:58.981559+0000","last_peered":"2024-01-11T07:06:58.981559+0000","last_clean":"2024-01-11T07:06:58.981559+0000","last_became_active":"2024-01-11T07:06:58.981071+0000","last_became_peered":"2024-01-11T07:06:58.981071+0000","last_unstale":"2024-01-11T07:06:58.981559+0000","last_undegraded":"2024-01-11T07:06:58.981559+0000","last_fullsized":"2024-01-11T07:06:58.981559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973250+0000","last_change":"2024-01-11T07:06:57.962256+0000","last_active":"2024-01-11T07:06:58.973250+0000","last_peered":"2024-01-11T07:06:58.973250+0000","last_clean":"2024-01-11T07:06:58.973250+0000","last_became_active":"2024-01-11T07:06:57.961595+0000","last_became_peered":"2024-01-11T07:06:57.961595+0000","last_unstale":"2024-01-11T07:06:58.973250+0000","last_undegraded":"2024-01-11T07:06:58.973250+0000","last_fullsized":"2024-01-11T07:06:58.973250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.984461+0000","last_change":"2024-01-11T07:06:58.984384+0000","last_active":"2024-01-11T07:06:58.984461+0000","last_peered":"2024-01-11T07:06:58.984461+0000","last_clean":"2024-01-11T07:06:58.984461+0000","last_became_active":"2024-01-11T07:06:58.984107+0000","last_became_peered":"2024-01-11T07:06:58.984107+0000","last_unstale":"2024-01-11T07:06:58.984461+0000","last_undegraded":"2024-01-11T07:06:58.984461+0000","last_fullsized":"2024-01-11T07:06:58.984461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.154946+0000","last_change":"2024-01-11T07:06:57.963683+0000","last_active":"2024-01-11T07:06:59.154946+0000","last_peered":"2024-01-11T07:06:59.154946+0000","last_clean":"2024-01-11T07:06:59.154946+0000","last_became_active":"2024-01-11T07:06:57.963452+0000","last_became_peered":"2024-01-11T07:06:57.963452+0000","last_unstale":"2024-01-11T07:06:59.154946+0000","last_undegraded":"2024-01-11T07:06:59.154946+0000","last_fullsized":"2024-01-11T07:06:59.154946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978185+0000","last_change":"2024-01-11T07:06:58.978001+0000","last_active":"2024-01-11T07:06:58.978185+0000","last_peered":"2024-01-11T07:06:58.978185+0000","last_clean":"2024-01-11T07:06:58.978185+0000","last_became_active":"2024-01-11T07:06:58.977325+0000","last_became_peered":"2024-01-11T07:06:58.977325+0000","last_unstale":"2024-01-11T07:06:58.978185+0000","last_undegraded":"2024-01-11T07:06:58.978185+0000","last_fullsized":"2024-01-11T07:06:58.978185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973136+0000","last_change":"2024-01-11T07:06:57.966946+0000","last_active":"2024-01-11T07:06:58.973136+0000","last_peered":"2024-01-11T07:06:58.973136+0000","last_clean":"2024-01-11T07:06:58.973136+0000","last_became_active":"2024-01-11T07:06:57.966738+0000","last_became_peered":"2024-01-11T07:06:57.966738+0000","last_unstale":"2024-01-11T07:06:58.973136+0000","last_undegraded":"2024-01-11T07:06:58.973136+0000","last_fullsized":"2024-01-11T07:06:58.973136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981855+0000","last_change":"2024-01-11T07:06:58.981768+0000","last_active":"2024-01-11T07:06:58.981855+0000","last_peered":"2024-01-11T07:06:58.981855+0000","last_clean":"2024-01-11T07:06:58.981855+0000","last_became_active":"2024-01-11T07:06:58.981467+0000","last_became_peered":"2024-01-11T07:06:58.981467+0000","last_unstale":"2024-01-11T07:06:58.981855+0000","last_undegraded":"2024-01-11T07:06:58.981855+0000","last_fullsized":"2024-01-11T07:06:58.981855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:22:00.354437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156189+0000","last_change":"2024-01-11T07:06:57.960096+0000","last_active":"2024-01-11T07:06:59.156189+0000","last_peered":"2024-01-11T07:06:59.156189+0000","last_clean":"2024-01-11T07:06:59.156189+0000","last_became_active":"2024-01-11T07:06:57.959880+0000","last_became_peered":"2024-01-11T07:06:57.959880+0000","last_unstale":"2024-01-11T07:06:59.156189+0000","last_undegraded":"2024-01-11T07:06:59.156189+0000","last_fullsized":"2024-01-11T07:06:59.156189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983031+0000","last_change":"2024-01-11T07:06:58.982845+0000","last_active":"2024-01-11T07:06:58.983031+0000","last_peered":"2024-01-11T07:06:58.983031+0000","last_clean":"2024-01-11T07:06:58.983031+0000","last_became_active":"2024-01-11T07:06:58.980928+0000","last_became_peered":"2024-01-11T07:06:58.980928+0000","last_unstale":"2024-01-11T07:06:58.983031+0000","last_undegraded":"2024-01-11T07:06:58.983031+0000","last_fullsized":"2024-01-11T07:06:58.983031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155763+0000","last_change":"2024-01-11T07:06:57.963359+0000","last_active":"2024-01-11T07:06:59.155763+0000","last_peered":"2024-01-11T07:06:59.155763+0000","last_clean":"2024-01-11T07:06:59.155763+0000","last_became_active":"2024-01-11T07:06:57.963116+0000","last_became_peered":"2024-01-11T07:06:57.963116+0000","last_unstale":"2024-01-11T07:06:59.155763+0000","last_undegraded":"2024-01-11T07:06:59.155763+0000","last_fullsized":"2024-01-11T07:06:59.155763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978190+0000","last_change":"2024-01-11T07:06:58.978133+0000","last_active":"2024-01-11T07:06:58.978190+0000","last_peered":"2024-01-11T07:06:58.978190+0000","last_clean":"2024-01-11T07:06:58.978190+0000","last_became_active":"2024-01-11T07:06:58.977605+0000","last_became_peered":"2024-01-11T07:06:58.977605+0000","last_unstale":"2024-01-11T07:06:58.978190+0000","last_undegraded":"2024-01-11T07:06:58.978190+0000","last_fullsized":"2024-01-11T07:06:58.978190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982813+0000","last_change":"2024-01-11T07:06:58.982744+0000","last_active":"2024-01-11T07:06:58.982813+0000","last_peered":"2024-01-11T07:06:58.982813+0000","last_clean":"2024-01-11T07:06:58.982813+0000","last_became_active":"2024-01-11T07:06:58.982442+0000","last_became_peered":"2024-01-11T07:06:58.982442+0000","last_unstale":"2024-01-11T07:06:58.982813+0000","last_undegraded":"2024-01-11T07:06:58.982813+0000","last_fullsized":"2024-01-11T07:06:58.982813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975116+0000","last_change":"2024-01-11T07:06:58.974838+0000","last_active":"2024-01-11T07:06:58.975116+0000","last_peered":"2024-01-11T07:06:58.975116+0000","last_clean":"2024-01-11T07:06:58.975116+0000","last_became_active":"2024-01-11T07:06:58.974628+0000","last_became_peered":"2024-01-11T07:06:58.974628+0000","last_unstale":"2024-01-11T07:06:58.975116+0000","last_undegraded":"2024-01-11T07:06:58.975116+0000","last_fullsized":"2024-01-11T07:06:58.975116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980086+0000","last_change":"2024-01-11T07:06:58.979993+0000","last_active":"2024-01-11T07:06:58.980086+0000","last_peered":"2024-01-11T07:06:58.980086+0000","last_clean":"2024-01-11T07:06:58.980086+0000","last_became_active":"2024-01-11T07:06:58.979782+0000","last_became_peered":"2024-01-11T07:06:58.979782+0000","last_unstale":"2024-01-11T07:06:58.980086+0000","last_undegraded":"2024-01-11T07:06:58.980086+0000","last_fullsized":"2024-01-11T07:06:58.980086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979408+0000","last_change":"2024-01-11T07:06:58.979256+0000","last_active":"2024-01-11T07:06:58.979408+0000","last_peered":"2024-01-11T07:06:58.979408+0000","last_clean":"2024-01-11T07:06:58.979408+0000","last_became_active":"2024-01-11T07:06:58.979025+0000","last_became_peered":"2024-01-11T07:06:58.979025+0000","last_unstale":"2024-01-11T07:06:58.979408+0000","last_undegraded":"2024-01-11T07:06:58.979408+0000","last_fullsized":"2024-01-11T07:06:58.979408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978671+0000","last_change":"2024-01-11T07:06:58.978602+0000","last_active":"2024-01-11T07:06:58.978671+0000","last_peered":"2024-01-11T07:06:58.978671+0000","last_clean":"2024-01-11T07:06:58.978671+0000","last_became_active":"2024-01-11T07:06:58.978265+0000","last_became_peered":"2024-01-11T07:06:58.978265+0000","last_unstale":"2024-01-11T07:06:58.978671+0000","last_undegraded":"2024-01-11T07:06:58.978671+0000","last_fullsized":"2024-01-11T07:06:58.978671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978598+0000","last_change":"2024-01-11T07:06:58.978524+0000","last_active":"2024-01-11T07:06:58.978598+0000","last_peered":"2024-01-11T07:06:58.978598+0000","last_clean":"2024-01-11T07:06:58.978598+0000","last_became_active":"2024-01-11T07:06:58.978266+0000","last_became_peered":"2024-01-11T07:06:58.978266+0000","last_unstale":"2024-01-11T07:06:58.978598+0000","last_undegraded":"2024-01-11T07:06:58.978598+0000","last_fullsized":"2024-01-11T07:06:58.978598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977723+0000","last_change":"2024-01-11T07:06:58.977673+0000","last_active":"2024-01-11T07:06:58.977723+0000","last_peered":"2024-01-11T07:06:58.977723+0000","last_clean":"2024-01-11T07:06:58.977723+0000","last_became_active":"2024-01-11T07:06:58.977490+0000","last_became_peered":"2024-01-11T07:06:58.977490+0000","last_unstale":"2024-01-11T07:06:58.977723+0000","last_undegraded":"2024-01-11T07:06:58.977723+0000","last_fullsized":"2024-01-11T07:06:58.977723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982405+0000","last_change":"2024-01-11T07:06:58.982014+0000","last_active":"2024-01-11T07:06:58.982405+0000","last_peered":"2024-01-11T07:06:58.982405+0000","last_clean":"2024-01-11T07:06:58.982405+0000","last_became_active":"2024-01-11T07:06:58.981670+0000","last_became_peered":"2024-01-11T07:06:58.981670+0000","last_unstale":"2024-01-11T07:06:58.982405+0000","last_undegraded":"2024-01-11T07:06:58.982405+0000","last_fullsized":"2024-01-11T07:06:58.982405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979096+0000","last_change":"2024-01-11T07:06:58.978874+0000","last_active":"2024-01-11T07:06:58.979096+0000","last_peered":"2024-01-11T07:06:58.979096+0000","last_clean":"2024-01-11T07:06:58.979096+0000","last_became_active":"2024-01-11T07:06:58.978319+0000","last_became_peered":"2024-01-11T07:06:58.978319+0000","last_unstale":"2024-01-11T07:06:58.979096+0000","last_undegraded":"2024-01-11T07:06:58.979096+0000","last_fullsized":"2024-01-11T07:06:58.979096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977051+0000","last_change":"2024-01-11T07:06:58.976808+0000","last_active":"2024-01-11T07:06:58.977051+0000","last_peered":"2024-01-11T07:06:58.977051+0000","last_clean":"2024-01-11T07:06:58.977051+0000","last_became_active":"2024-01-11T07:06:58.975696+0000","last_became_peered":"2024-01-11T07:06:58.975696+0000","last_unstale":"2024-01-11T07:06:58.977051+0000","last_undegraded":"2024-01-11T07:06:58.977051+0000","last_fullsized":"2024-01-11T07:06:58.977051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983760+0000","last_change":"2024-01-11T07:06:58.983202+0000","last_active":"2024-01-11T07:06:58.983760+0000","last_peered":"2024-01-11T07:06:58.983760+0000","last_clean":"2024-01-11T07:06:58.983760+0000","last_became_active":"2024-01-11T07:06:58.981685+0000","last_became_peered":"2024-01-11T07:06:58.981685+0000","last_unstale":"2024-01-11T07:06:58.983760+0000","last_undegraded":"2024-01-11T07:06:58.983760+0000","last_fullsized":"2024-01-11T07:06:58.983760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:04:53.068090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978684+0000","last_change":"2024-01-11T07:06:58.978620+0000","last_active":"2024-01-11T07:06:58.978684+0000","last_peered":"2024-01-11T07:06:58.978684+0000","last_clean":"2024-01-11T07:06:58.978684+0000","last_became_active":"2024-01-11T07:06:58.978229+0000","last_became_peered":"2024-01-11T07:06:58.978229+0000","last_unstale":"2024-01-11T07:06:58.978684+0000","last_undegraded":"2024-01-11T07:06:58.978684+0000","last_fullsized":"2024-01-11T07:06:58.978684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981445+0000","last_change":"2024-01-11T07:06:58.981078+0000","last_active":"2024-01-11T07:06:58.981445+0000","last_peered":"2024-01-11T07:06:58.981445+0000","last_clean":"2024-01-11T07:06:58.981445+0000","last_became_active":"2024-01-11T07:06:58.980821+0000","last_became_peered":"2024-01-11T07:06:58.980821+0000","last_unstale":"2024-01-11T07:06:58.981445+0000","last_undegraded":"2024-01-11T07:06:58.981445+0000","last_fullsized":"2024-01-11T07:06:58.981445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":655,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199328880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":79,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195232,"statfs":{"total":200000000,"available":199918266,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":655,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199328824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":79,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195232,"statfs":{"total":200000000,"available":199918266,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919676,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919712,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:12.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.055+0000 7f6707b4a700 1 Processor -- start 2024-01-11T07:07:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.055+0000 7f6707b4a700 1 -- start start 2024-01-11T07:07:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f6707b4a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f670006e0d0 0x7f6700159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f6707b4a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670015b770 con 0x7f670006dd00 2024-01-11T07:07:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f6707b4a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670015abf0 con 0x7f6700159ff0 2024-01-11T07:07:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f6707b4a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670015aed0 con 0x7f670006e0d0 2024-01-11T07:07:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f67058e6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f670006dd00 0x7f67000c98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40280/0 (socket says 172.21.15.7:40280) 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f67058e6700 1 -- 172.21.15.7:0/518567028 learned_addr learned my addr 172.21.15.7:0/518567028 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f67050e5700 1 --2- 172.21.15.7:0/518567028 >> v2:172.21.15.113:3300/0 conn(0x7f670006e0d0 0x7f6700159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986742027 0 0) 0x7f670015b770 con 0x7f670006dd00 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.056+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66ec003570 con 0x7f670006dd00 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1299379110 0 0) 0x7f66ec003570 con 0x7f670006dd00 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 >> v2:172.21.15.113:3300/0 conn(0x7f670006e0d0 msgr2=0x7f6700159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 --2- 172.21.15.7:0/518567028 >> v2:172.21.15.113:3300/0 conn(0x7f670006e0d0 0x7f6700159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 >> v1:172.21.15.113:6789/0 conn(0x7f6700159ff0 legacy=0x7f6700069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670015b1b0 con 0x7f670006dd00 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f66fc001470 con 0x7f670006dd00 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 >> v1:172.21.15.7:6789/0 conn(0x7f670006dd00 legacy=0x7f67000c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec003a60 0x7f66ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66ec004810 0x7f66ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec0051c0 0x7f66ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670015aed0 con 0x7f66ec003a60 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec00c160 con 0x7f66ec0051c0 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec00c320 con 0x7f66ec004810 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67058e6700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec003a60 0x7f66ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67060e7700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec0051c0 0x7f66ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67050e5700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66ec004810 0x7f66ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67058e6700 1 -- 172.21.15.7:0/518567028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66ec004810 msgr2=0x7f66ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67058e6700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66ec004810 0x7f66ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67058e6700 1 -- 172.21.15.7:0/518567028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec0051c0 msgr2=0x7f66ec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67058e6700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec0051c0 0x7f66ec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67060e7700 1 -- 172.21.15.7:0/518567028 reap_dead start 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67058e6700 1 -- 172.21.15.7:0/518567028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec003570 con 0x7f66ec003a60 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.057+0000 7f67058e6700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec003a60 0x7f66ec003e70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00c3c0 tx=0x7f66fc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2118a6dd2ebc62d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc00cc90 con 0x7f66ec003a60 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc00ce10 con 0x7f66ec003a60 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f67048e4700 1 -- 172.21.15.7:0/518567028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc005b70 con 0x7f66ec003a60 2024-01-11T07:07:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f6707b4a700 1 -- 172.21.15.7:0/518567028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec003a60 msgr2=0x7f66ec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f6707b4a700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec003a60 0x7f66ec003e70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00c3c0 tx=0x7f66fc001e50 comp rx=0 tx=0).stop 2024-01-11T07:07:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f6707b4a700 1 -- 172.21.15.7:0/518567028 shutdown_connections 2024-01-11T07:07:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f6707b4a700 1 --2- 172.21.15.7:0/518567028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec003a60 0x7f66ec003e70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f6707b4a700 1 -- 172.21.15.7:0/518567028 >> 172.21.15.7:0/518567028 conn(0x7f67000680e0 msgr2=0x7f67000cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f6707b4a700 1 -- 172.21.15.7:0/518567028 shutdown_connections 2024-01-11T07:07:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.058+0000 7f6707b4a700 1 -- 172.21.15.7:0/518567028 wait complete. 2024-01-11T07:07:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.059+0000 7f6707b4a700 1 Processor -- start 2024-01-11T07:07:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.059+0000 7f6707b4a700 1 -- start start 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f6707b4a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec003a60 0x7f6700157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f6707b4a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67000b2fd0 0x7f67000c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f6707b4a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67000aba60 0x7f67000abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f6707b4a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000ac420 con 0x7f67000aba60 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f6707b4a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000c5df0 con 0x7f66ec003a60 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f6707b4a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67001584f0 con 0x7f67000b2fd0 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f67060e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67000aba60 0x7f67000abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f67060e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67000aba60 0x7f67000abee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41598/0 (socket says 172.21.15.7:41598) 2024-01-11T07:07:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f67060e7700 1 -- 172.21.15.7:0/2839611824 learned_addr learned my addr 172.21.15.7:0/2839611824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f67050e5700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67000b2fd0 0x7f67000c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.060+0000 7f67058e6700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec003a60 0x7f6700157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f67060e7700 1 -- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67000b2fd0 msgr2=0x7f67000c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f67060e7700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67000b2fd0 0x7f67000c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f67060e7700 1 -- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec003a60 msgr2=0x7f6700157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f67060e7700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec003a60 0x7f6700157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f67060e7700 1 -- 172.21.15.7:0/2839611824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000c51e0 con 0x7f67000aba60 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f67058e6700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec003a60 0x7f6700157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f67060e7700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67000aba60 0x7f67000abee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67000ca310 tx=0x7f66f8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f66f5ffb700 1 -- 172.21.15.7:0/2839611824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8006e80 con 0x7f67000aba60 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f66f5ffb700 1 -- 172.21.15.7:0/2839611824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f800c5d0 con 0x7f67000aba60 2024-01-11T07:07:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67000c54c0 con 0x7f67000aba60 2024-01-11T07:07:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f66f5ffb700 1 -- 172.21.15.7:0/2839611824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800c970 con 0x7f67000aba60 2024-01-11T07:07:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.061+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67000c5a80 con 0x7f67000aba60 2024-01-11T07:07:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.062+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6700064850 con 0x7f67000aba60 2024-01-11T07:07:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.063+0000 7f66f5ffb700 1 -- 172.21.15.7:0/2839611824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f66f800cd20 con 0x7f67000aba60 2024-01-11T07:07:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.063+0000 7f66f5ffb700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.063+0000 7f66f5ffb700 1 -- 172.21.15.7:0/2839611824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f66f8083f30 con 0x7f67000aba60 2024-01-11T07:07:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.063+0000 7f67058e6700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.064+0000 7f67058e6700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66fc007cd0 tx=0x7f66fc015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.067+0000 7f66f5ffb700 1 -- 172.21.15.7:0/2839611824 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66f80511a0 con 0x7f67000aba60 2024-01-11T07:07:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.266+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f670004c3c0 con 0x7f66e4066f30 2024-01-11T07:07:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.273+0000 7f66f5ffb700 1 -- 172.21.15.7:0/2839611824 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260149 (secure 0 0 0) 0x7f670004c3c0 con 0x7f66e4066f30 2024-01-11T07:07:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 msgr2=0x7f66e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66fc007cd0 tx=0x7f66fc015000 comp rx=0 tx=0).stop 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67000aba60 msgr2=0x7f67000abee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67000aba60 0x7f67000abee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67000ca310 tx=0x7f66f8001640 comp rx=0 tx=0).stop 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 shutdown_connections 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67000aba60 0x7f67000abee0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66ec003a60 0x7f6700157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 --2- 172.21.15.7:0/2839611824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67000b2fd0 0x7f67000c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 >> 172.21.15.7:0/2839611824 conn(0x7f67000680e0 msgr2=0x7f6700158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 shutdown_connections 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.276+0000 7f6707b4a700 1 -- 172.21.15.7:0/2839611824 wait complete. 2024-01-11T07:07:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:13.289 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-01-11T07:07:11.841937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560715,"statfs":{"total":1600000000,"available":1598173068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.875097"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978080+0000","last_change":"2024-01-11T07:06:58.977426+0000","last_active":"2024-01-11T07:06:58.978080+0000","last_peered":"2024-01-11T07:06:58.978080+0000","last_clean":"2024-01-11T07:06:58.978080+0000","last_became_active":"2024-01-11T07:06:58.977173+0000","last_became_peered":"2024-01-11T07:06:58.977173+0000","last_unstale":"2024-01-11T07:06:58.978080+0000","last_undegraded":"2024-01-11T07:06:58.978080+0000","last_fullsized":"2024-01-11T07:06:58.978080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:27:13.892155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982635+0000","last_change":"2024-01-11T07:06:58.982509+0000","last_active":"2024-01-11T07:06:58.982635+0000","last_peered":"2024-01-11T07:06:58.982635+0000","last_clean":"2024-01-11T07:06:58.982635+0000","last_became_active":"2024-01-11T07:06:58.982234+0000","last_became_peered":"2024-01-11T07:06:58.982234+0000","last_unstale":"2024-01-11T07:06:58.982635+0000","last_undegraded":"2024-01-11T07:06:58.982635+0000","last_fullsized":"2024-01-11T07:06:58.982635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981566+0000","last_change":"2024-01-11T07:06:58.981034+0000","last_active":"2024-01-11T07:06:58.981566+0000","last_peered":"2024-01-11T07:06:58.981566+0000","last_clean":"2024-01-11T07:06:58.981566+0000","last_became_active":"2024-01-11T07:06:58.979996+0000","last_became_peered":"2024-01-11T07:06:58.979996+0000","last_unstale":"2024-01-11T07:06:58.981566+0000","last_undegraded":"2024-01-11T07:06:58.981566+0000","last_fullsized":"2024-01-11T07:06:58.981566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979116+0000","last_change":"2024-01-11T07:06:58.979053+0000","last_active":"2024-01-11T07:06:58.979116+0000","last_peered":"2024-01-11T07:06:58.979116+0000","last_clean":"2024-01-11T07:06:58.979116+0000","last_became_active":"2024-01-11T07:06:58.978648+0000","last_became_peered":"2024-01-11T07:06:58.978648+0000","last_unstale":"2024-01-11T07:06:58.979116+0000","last_undegraded":"2024-01-11T07:06:58.979116+0000","last_fullsized":"2024-01-11T07:06:58.979116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:11:14.542059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978059+0000","last_change":"2024-01-11T07:06:58.977393+0000","last_active":"2024-01-11T07:06:58.978059+0000","last_peered":"2024-01-11T07:06:58.978059+0000","last_clean":"2024-01-11T07:06:58.978059+0000","last_became_active":"2024-01-11T07:06:58.977145+0000","last_became_peered":"2024-01-11T07:06:58.977145+0000","last_unstale":"2024-01-11T07:06:58.978059+0000","last_undegraded":"2024-01-11T07:06:58.978059+0000","last_fullsized":"2024-01-11T07:06:58.978059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:07:39.492394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972053+0000","last_change":"2024-01-11T07:06:58.971961+0000","last_active":"2024-01-11T07:06:58.972053+0000","last_peered":"2024-01-11T07:06:58.972053+0000","last_clean":"2024-01-11T07:06:58.972053+0000","last_became_active":"2024-01-11T07:06:58.971577+0000","last_became_peered":"2024-01-11T07:06:58.971577+0000","last_unstale":"2024-01-11T07:06:58.972053+0000","last_undegraded":"2024-01-11T07:06:58.972053+0000","last_fullsized":"2024-01-11T07:06:58.972053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:27:13.892155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.984886+0000","last_change":"2024-01-11T07:06:58.984793+0000","last_active":"2024-01-11T07:06:58.984886+0000","last_peered":"2024-01-11T07:06:58.984886+0000","last_clean":"2024-01-11T07:06:58.984886+0000","last_became_active":"2024-01-11T07:06:58.984465+0000","last_became_peered":"2024-01-11T07:06:58.984465+0000","last_unstale":"2024-01-11T07:06:58.984886+0000","last_undegraded":"2024-01-11T07:06:58.984886+0000","last_fullsized":"2024-01-11T07:06:58.984886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:11:14.542059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977071+0000","last_change":"2024-01-11T07:06:58.976965+0000","last_active":"2024-01-11T07:06:58.977071+0000","last_peered":"2024-01-11T07:06:58.977071+0000","last_clean":"2024-01-11T07:06:58.977071+0000","last_became_active":"2024-01-11T07:06:58.976113+0000","last_became_peered":"2024-01-11T07:06:58.976113+0000","last_unstale":"2024-01-11T07:06:58.977071+0000","last_undegraded":"2024-01-11T07:06:58.977071+0000","last_fullsized":"2024-01-11T07:06:58.977071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978104+0000","last_change":"2024-01-11T07:06:58.977843+0000","last_active":"2024-01-11T07:06:58.978104+0000","last_peered":"2024-01-11T07:06:58.978104+0000","last_clean":"2024-01-11T07:06:58.978104+0000","last_became_active":"2024-01-11T07:06:58.976923+0000","last_became_peered":"2024-01-11T07:06:58.976923+0000","last_unstale":"2024-01-11T07:06:58.978104+0000","last_undegraded":"2024-01-11T07:06:58.978104+0000","last_fullsized":"2024-01-11T07:06:58.978104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:04:53.068090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975533+0000","last_change":"2024-01-11T07:06:58.975456+0000","last_active":"2024-01-11T07:06:58.975533+0000","last_peered":"2024-01-11T07:06:58.975533+0000","last_clean":"2024-01-11T07:06:58.975533+0000","last_became_active":"2024-01-11T07:06:58.974898+0000","last_became_peered":"2024-01-11T07:06:58.974898+0000","last_unstale":"2024-01-11T07:06:58.975533+0000","last_undegraded":"2024-01-11T07:06:58.975533+0000","last_fullsized":"2024-01-11T07:06:58.975533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975115+0000","last_change":"2024-01-11T07:06:58.975019+0000","last_active":"2024-01-11T07:06:58.975115+0000","last_peered":"2024-01-11T07:06:58.975115+0000","last_clean":"2024-01-11T07:06:58.975115+0000","last_became_active":"2024-01-11T07:06:58.974797+0000","last_became_peered":"2024-01-11T07:06:58.974797+0000","last_unstale":"2024-01-11T07:06:58.975115+0000","last_undegraded":"2024-01-11T07:06:58.975115+0000","last_fullsized":"2024-01-11T07:06:58.975115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975231+0000","last_change":"2024-01-11T07:06:58.974746+0000","last_active":"2024-01-11T07:06:58.975231+0000","last_peered":"2024-01-11T07:06:58.975231+0000","last_clean":"2024-01-11T07:06:58.975231+0000","last_became_active":"2024-01-11T07:06:58.974520+0000","last_became_peered":"2024-01-11T07:06:58.974520+0000","last_unstale":"2024-01-11T07:06:58.975231+0000","last_undegraded":"2024-01-11T07:06:58.975231+0000","last_fullsized":"2024-01-11T07:06:58.975231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:59.240388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982637+0000","last_change":"2024-01-11T07:06:58.982409+0000","last_active":"2024-01-11T07:06:58.982637+0000","last_peered":"2024-01-11T07:06:58.982637+0000","last_clean":"2024-01-11T07:06:58.982637+0000","last_became_active":"2024-01-11T07:06:58.981807+0000","last_became_peered":"2024-01-11T07:06:58.981807+0000","last_unstale":"2024-01-11T07:06:58.982637+0000","last_undegraded":"2024-01-11T07:06:58.982637+0000","last_fullsized":"2024-01-11T07:06:58.982637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978084+0000","last_change":"2024-01-11T07:06:58.977853+0000","last_active":"2024-01-11T07:06:58.978084+0000","last_peered":"2024-01-11T07:06:58.978084+0000","last_clean":"2024-01-11T07:06:58.978084+0000","last_became_active":"2024-01-11T07:06:58.977594+0000","last_became_peered":"2024-01-11T07:06:58.977594+0000","last_unstale":"2024-01-11T07:06:58.978084+0000","last_undegraded":"2024-01-11T07:06:58.978084+0000","last_fullsized":"2024-01-11T07:06:58.978084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982542+0000","last_change":"2024-01-11T07:06:58.982470+0000","last_active":"2024-01-11T07:06:58.982542+0000","last_peered":"2024-01-11T07:06:58.982542+0000","last_clean":"2024-01-11T07:06:58.982542+0000","last_became_active":"2024-01-11T07:06:58.982209+0000","last_became_peered":"2024-01-11T07:06:58.982209+0000","last_unstale":"2024-01-11T07:06:58.982542+0000","last_undegraded":"2024-01-11T07:06:58.982542+0000","last_fullsized":"2024-01-11T07:06:58.982542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971907+0000","last_change":"2024-01-11T07:06:58.971802+0000","last_active":"2024-01-11T07:06:58.971907+0000","last_peered":"2024-01-11T07:06:58.971907+0000","last_clean":"2024-01-11T07:06:58.971907+0000","last_became_active":"2024-01-11T07:06:58.971251+0000","last_became_peered":"2024-01-11T07:06:58.971251+0000","last_unstale":"2024-01-11T07:06:58.971907+0000","last_undegraded":"2024-01-11T07:06:58.971907+0000","last_fullsized":"2024-01-11T07:06:58.971907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:19.418614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983779+0000","last_change":"2024-01-11T07:06:58.983363+0000","last_active":"2024-01-11T07:06:58.983779+0000","last_peered":"2024-01-11T07:06:58.983779+0000","last_clean":"2024-01-11T07:06:58.983779+0000","last_became_active":"2024-01-11T07:06:58.982003+0000","last_became_peered":"2024-01-11T07:06:58.982003+0000","last_unstale":"2024-01-11T07:06:58.983779+0000","last_undegraded":"2024-01-11T07:06:58.983779+0000","last_fullsized":"2024-01-11T07:06:58.983779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:22:00.354437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975549+0000","last_change":"2024-01-11T07:06:58.975439+0000","last_active":"2024-01-11T07:06:58.975549+0000","last_peered":"2024-01-11T07:06:58.975549+0000","last_clean":"2024-01-11T07:06:58.975549+0000","last_became_active":"2024-01-11T07:06:58.975201+0000","last_became_peered":"2024-01-11T07:06:58.975201+0000","last_unstale":"2024-01-11T07:06:58.975549+0000","last_undegraded":"2024-01-11T07:06:58.975549+0000","last_fullsized":"2024-01-11T07:06:58.975549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978122+0000","last_change":"2024-01-11T07:06:58.977996+0000","last_active":"2024-01-11T07:06:58.978122+0000","last_peered":"2024-01-11T07:06:58.978122+0000","last_clean":"2024-01-11T07:06:58.978122+0000","last_became_active":"2024-01-11T07:06:58.977268+0000","last_became_peered":"2024-01-11T07:06:58.977268+0000","last_unstale":"2024-01-11T07:06:58.978122+0000","last_undegraded":"2024-01-11T07:06:58.978122+0000","last_fullsized":"2024-01-11T07:06:58.978122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:19.418614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.158069+0000","last_change":"2024-01-11T07:06:57.964441+0000","last_active":"2024-01-11T07:06:59.158069+0000","last_peered":"2024-01-11T07:06:59.158069+0000","last_clean":"2024-01-11T07:06:59.158069+0000","last_became_active":"2024-01-11T07:06:57.963971+0000","last_became_peered":"2024-01-11T07:06:57.963971+0000","last_unstale":"2024-01-11T07:06:59.158069+0000","last_undegraded":"2024-01-11T07:06:59.158069+0000","last_fullsized":"2024-01-11T07:06:59.158069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980035+0000","last_change":"2024-01-11T07:06:58.979891+0000","last_active":"2024-01-11T07:06:58.980035+0000","last_peered":"2024-01-11T07:06:58.980035+0000","last_clean":"2024-01-11T07:06:58.980035+0000","last_became_active":"2024-01-11T07:06:58.979638+0000","last_became_peered":"2024-01-11T07:06:58.979638+0000","last_unstale":"2024-01-11T07:06:58.980035+0000","last_undegraded":"2024-01-11T07:06:58.980035+0000","last_fullsized":"2024-01-11T07:06:58.980035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157701+0000","last_change":"2024-01-11T07:06:57.958809+0000","last_active":"2024-01-11T07:06:59.157701+0000","last_peered":"2024-01-11T07:06:59.157701+0000","last_clean":"2024-01-11T07:06:59.157701+0000","last_became_active":"2024-01-11T07:06:57.958478+0000","last_became_peered":"2024-01-11T07:06:57.958478+0000","last_unstale":"2024-01-11T07:06:59.157701+0000","last_undegraded":"2024-01-11T07:06:59.157701+0000","last_fullsized":"2024-01-11T07:06:59.157701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.969856+0000","last_change":"2024-01-11T07:06:58.969781+0000","last_active":"2024-01-11T07:06:58.969856+0000","last_peered":"2024-01-11T07:06:58.969856+0000","last_clean":"2024-01-11T07:06:58.969856+0000","last_became_active":"2024-01-11T07:06:58.969505+0000","last_became_peered":"2024-01-11T07:06:58.969505+0000","last_unstale":"2024-01-11T07:06:58.969856+0000","last_undegraded":"2024-01-11T07:06:58.969856+0000","last_fullsized":"2024-01-11T07:06:58.969856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156412+0000","last_change":"2024-01-11T07:06:57.967660+0000","last_active":"2024-01-11T07:06:59.156412+0000","last_peered":"2024-01-11T07:06:59.156412+0000","last_clean":"2024-01-11T07:06:59.156412+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:59.156412+0000","last_undegraded":"2024-01-11T07:06:59.156412+0000","last_fullsized":"2024-01-11T07:06:59.156412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970064+0000","last_change":"2024-01-11T07:06:58.969973+0000","last_active":"2024-01-11T07:06:58.970064+0000","last_peered":"2024-01-11T07:06:58.970064+0000","last_clean":"2024-01-11T07:06:58.970064+0000","last_became_active":"2024-01-11T07:06:58.969722+0000","last_became_peered":"2024-01-11T07:06:58.969722+0000","last_unstale":"2024-01-11T07:06:58.970064+0000","last_undegraded":"2024-01-11T07:06:58.970064+0000","last_fullsized":"2024-01-11T07:06:58.970064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:52.883593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.154688+0000","last_change":"2024-01-11T07:06:57.967196+0000","last_active":"2024-01-11T07:06:59.154688+0000","last_peered":"2024-01-11T07:06:59.154688+0000","last_clean":"2024-01-11T07:06:59.154688+0000","last_became_active":"2024-01-11T07:06:57.966964+0000","last_became_peered":"2024-01-11T07:06:57.966964+0000","last_unstale":"2024-01-11T07:06:59.154688+0000","last_undegraded":"2024-01-11T07:06:59.154688+0000","last_fullsized":"2024-01-11T07:06:59.154688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:27.038629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970044+0000","last_change":"2024-01-11T07:06:58.969967+0000","last_active":"2024-01-11T07:06:58.970044+0000","last_peered":"2024-01-11T07:06:58.970044+0000","last_clean":"2024-01-11T07:06:58.970044+0000","last_became_active":"2024-01-11T07:06:58.969708+0000","last_became_peered":"2024-01-11T07:06:58.969708+0000","last_unstale":"2024-01-11T07:06:58.970044+0000","last_undegraded":"2024-01-11T07:06:58.970044+0000","last_fullsized":"2024-01-11T07:06:58.970044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157494+0000","last_change":"2024-01-11T07:06:57.964730+0000","last_active":"2024-01-11T07:06:59.157494+0000","last_peered":"2024-01-11T07:06:59.157494+0000","last_clean":"2024-01-11T07:06:59.157494+0000","last_became_active":"2024-01-11T07:06:57.964464+0000","last_became_peered":"2024-01-11T07:06:57.964464+0000","last_unstale":"2024-01-11T07:06:59.157494+0000","last_undegraded":"2024-01-11T07:06:59.157494+0000","last_fullsized":"2024-01-11T07:06:59.157494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979054+0000","last_change":"2024-01-11T07:06:58.978898+0000","last_active":"2024-01-11T07:06:58.979054+0000","last_peered":"2024-01-11T07:06:58.979054+0000","last_clean":"2024-01-11T07:06:58.979054+0000","last_became_active":"2024-01-11T07:06:58.978593+0000","last_became_peered":"2024-01-11T07:06:58.978593+0000","last_unstale":"2024-01-11T07:06:58.979054+0000","last_undegraded":"2024-01-11T07:06:58.979054+0000","last_fullsized":"2024-01-11T07:06:58.979054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:42.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156518+0000","last_change":"2024-01-11T07:06:57.961158+0000","last_active":"2024-01-11T07:06:59.156518+0000","last_peered":"2024-01-11T07:06:59.156518+0000","last_clean":"2024-01-11T07:06:59.156518+0000","last_became_active":"2024-01-11T07:06:57.960836+0000","last_became_peered":"2024-01-11T07:06:57.960836+0000","last_unstale":"2024-01-11T07:06:59.156518+0000","last_undegraded":"2024-01-11T07:06:59.156518+0000","last_fullsized":"2024-01-11T07:06:59.156518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968566+0000","last_change":"2024-01-11T07:06:58.968497+0000","last_active":"2024-01-11T07:06:58.968566+0000","last_peered":"2024-01-11T07:06:58.968566+0000","last_clean":"2024-01-11T07:06:58.968566+0000","last_became_active":"2024-01-11T07:06:58.968280+0000","last_became_peered":"2024-01-11T07:06:58.968280+0000","last_unstale":"2024-01-11T07:06:58.968566+0000","last_undegraded":"2024-01-11T07:06:58.968566+0000","last_fullsized":"2024-01-11T07:06:58.968566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975862+0000","last_change":"2024-01-11T07:06:57.963583+0000","last_active":"2024-01-11T07:06:58.975862+0000","last_peered":"2024-01-11T07:06:58.975862+0000","last_clean":"2024-01-11T07:06:58.975862+0000","last_became_active":"2024-01-11T07:06:57.962966+0000","last_became_peered":"2024-01-11T07:06:57.962966+0000","last_unstale":"2024-01-11T07:06:58.975862+0000","last_undegraded":"2024-01-11T07:06:58.975862+0000","last_fullsized":"2024-01-11T07:06:58.975862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:01:43.405601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980008+0000","last_change":"2024-01-11T07:06:58.979636+0000","last_active":"2024-01-11T07:06:58.980008+0000","last_peered":"2024-01-11T07:06:58.980008+0000","last_clean":"2024-01-11T07:06:58.980008+0000","last_became_active":"2024-01-11T07:06:58.979362+0000","last_became_peered":"2024-01-11T07:06:58.979362+0000","last_unstale":"2024-01-11T07:06:58.980008+0000","last_undegraded":"2024-01-11T07:06:58.980008+0000","last_fullsized":"2024-01-11T07:06:58.980008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:55.497646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968179+0000","last_change":"2024-01-11T07:06:57.965145+0000","last_active":"2024-01-11T07:06:58.968179+0000","last_peered":"2024-01-11T07:06:58.968179+0000","last_clean":"2024-01-11T07:06:58.968179+0000","last_became_active":"2024-01-11T07:06:57.964890+0000","last_became_peered":"2024-01-11T07:06:57.964890+0000","last_unstale":"2024-01-11T07:06:58.968179+0000","last_undegraded":"2024-01-11T07:06:58.968179+0000","last_fullsized":"2024-01-11T07:06:58.968179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982521+0000","last_change":"2024-01-11T07:06:58.982437+0000","last_active":"2024-01-11T07:06:58.982521+0000","last_peered":"2024-01-11T07:06:58.982521+0000","last_clean":"2024-01-11T07:06:58.982521+0000","last_became_active":"2024-01-11T07:06:58.982170+0000","last_became_peered":"2024-01-11T07:06:58.982170+0000","last_unstale":"2024-01-11T07:06:58.982521+0000","last_undegraded":"2024-01-11T07:06:58.982521+0000","last_fullsized":"2024-01-11T07:06:58.982521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972373+0000","last_change":"2024-01-11T07:06:57.967613+0000","last_active":"2024-01-11T07:06:58.972373+0000","last_peered":"2024-01-11T07:06:58.972373+0000","last_clean":"2024-01-11T07:06:58.972373+0000","last_became_active":"2024-01-11T07:06:57.967356+0000","last_became_peered":"2024-01-11T07:06:57.967356+0000","last_unstale":"2024-01-11T07:06:58.972373+0000","last_undegraded":"2024-01-11T07:06:58.972373+0000","last_fullsized":"2024-01-11T07:06:58.972373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983217+0000","last_change":"2024-01-11T07:06:58.983040+0000","last_active":"2024-01-11T07:06:58.983217+0000","last_peered":"2024-01-11T07:06:58.983217+0000","last_clean":"2024-01-11T07:06:58.983217+0000","last_became_active":"2024-01-11T07:06:58.981241+0000","last_became_peered":"2024-01-11T07:06:58.981241+0000","last_unstale":"2024-01-11T07:06:58.983217+0000","last_undegraded":"2024-01-11T07:06:58.983217+0000","last_fullsized":"2024-01-11T07:06:58.983217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:57:17.525152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.967926+0000","last_change":"2024-01-11T07:06:57.966453+0000","last_active":"2024-01-11T07:06:58.967926+0000","last_peered":"2024-01-11T07:06:58.967926+0000","last_clean":"2024-01-11T07:06:58.967926+0000","last_became_active":"2024-01-11T07:06:57.966083+0000","last_became_peered":"2024-01-11T07:06:57.966083+0000","last_unstale":"2024-01-11T07:06:58.967926+0000","last_undegraded":"2024-01-11T07:06:58.967926+0000","last_fullsized":"2024-01-11T07:06:58.967926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970715+0000","last_change":"2024-01-11T07:06:58.970636+0000","last_active":"2024-01-11T07:06:58.970715+0000","last_peered":"2024-01-11T07:06:58.970715+0000","last_clean":"2024-01-11T07:06:58.970715+0000","last_became_active":"2024-01-11T07:06:58.970180+0000","last_became_peered":"2024-01-11T07:06:58.970180+0000","last_unstale":"2024-01-11T07:06:58.970715+0000","last_undegraded":"2024-01-11T07:06:58.970715+0000","last_fullsized":"2024-01-11T07:06:58.970715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156721+0000","last_change":"2024-01-11T07:06:57.966021+0000","last_active":"2024-01-11T07:06:59.156721+0000","last_peered":"2024-01-11T07:06:59.156721+0000","last_clean":"2024-01-11T07:06:59.156721+0000","last_became_active":"2024-01-11T07:06:57.965763+0000","last_became_peered":"2024-01-11T07:06:57.965763+0000","last_unstale":"2024-01-11T07:06:59.156721+0000","last_undegraded":"2024-01-11T07:06:59.156721+0000","last_fullsized":"2024-01-11T07:06:59.156721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980149+0000","last_change":"2024-01-11T07:06:58.980082+0000","last_active":"2024-01-11T07:06:58.980149+0000","last_peered":"2024-01-11T07:06:58.980149+0000","last_clean":"2024-01-11T07:06:58.980149+0000","last_became_active":"2024-01-11T07:06:58.979407+0000","last_became_peered":"2024-01-11T07:06:58.979407+0000","last_unstale":"2024-01-11T07:06:58.980149+0000","last_undegraded":"2024-01-11T07:06:58.980149+0000","last_fullsized":"2024-01-11T07:06:58.980149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:20.167583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.158497+0000","last_change":"2024-01-11T07:06:57.962108+0000","last_active":"2024-01-11T07:06:59.158497+0000","last_peered":"2024-01-11T07:06:59.158497+0000","last_clean":"2024-01-11T07:06:59.158497+0000","last_became_active":"2024-01-11T07:06:57.961848+0000","last_became_peered":"2024-01-11T07:06:57.961848+0000","last_unstale":"2024-01-11T07:06:59.158497+0000","last_undegraded":"2024-01-11T07:06:59.158497+0000","last_fullsized":"2024-01-11T07:06:59.158497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980308+0000","last_change":"2024-01-11T07:06:58.980239+0000","last_active":"2024-01-11T07:06:58.980308+0000","last_peered":"2024-01-11T07:06:58.980308+0000","last_clean":"2024-01-11T07:06:58.980308+0000","last_became_active":"2024-01-11T07:06:58.979974+0000","last_became_peered":"2024-01-11T07:06:58.979974+0000","last_unstale":"2024-01-11T07:06:58.980308+0000","last_undegraded":"2024-01-11T07:06:58.980308+0000","last_fullsized":"2024-01-11T07:06:58.980308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:57:17.525152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155524+0000","last_change":"2024-01-11T07:06:57.960489+0000","last_active":"2024-01-11T07:06:59.155524+0000","last_peered":"2024-01-11T07:06:59.155524+0000","last_clean":"2024-01-11T07:06:59.155524+0000","last_became_active":"2024-01-11T07:06:57.960225+0000","last_became_peered":"2024-01-11T07:06:57.960225+0000","last_unstale":"2024-01-11T07:06:59.155524+0000","last_undegraded":"2024-01-11T07:06:59.155524+0000","last_fullsized":"2024-01-11T07:06:59.155524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:00:35.469371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978545+0000","last_change":"2024-01-11T07:06:58.978471+0000","last_active":"2024-01-11T07:06:58.978545+0000","last_peered":"2024-01-11T07:06:58.978545+0000","last_clean":"2024-01-11T07:06:58.978545+0000","last_became_active":"2024-01-11T07:06:58.977884+0000","last_became_peered":"2024-01-11T07:06:58.977884+0000","last_unstale":"2024-01-11T07:06:58.978545+0000","last_undegraded":"2024-01-11T07:06:58.978545+0000","last_fullsized":"2024-01-11T07:06:58.978545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:00:28.043324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968025+0000","last_change":"2024-01-11T07:06:57.965935+0000","last_active":"2024-01-11T07:06:58.968025+0000","last_peered":"2024-01-11T07:06:58.968025+0000","last_clean":"2024-01-11T07:06:58.968025+0000","last_became_active":"2024-01-11T07:06:57.965303+0000","last_became_peered":"2024-01-11T07:06:57.965303+0000","last_unstale":"2024-01-11T07:06:58.968025+0000","last_undegraded":"2024-01-11T07:06:58.968025+0000","last_fullsized":"2024-01-11T07:06:58.968025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981838+0000","last_change":"2024-01-11T07:06:58.981581+0000","last_active":"2024-01-11T07:06:58.981838+0000","last_peered":"2024-01-11T07:06:58.981838+0000","last_clean":"2024-01-11T07:06:58.981838+0000","last_became_active":"2024-01-11T07:06:58.981032+0000","last_became_peered":"2024-01-11T07:06:58.981032+0000","last_unstale":"2024-01-11T07:06:58.981838+0000","last_undegraded":"2024-01-11T07:06:58.981838+0000","last_fullsized":"2024-01-11T07:06:58.981838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:05:09.439404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973296+0000","last_change":"2024-01-11T07:06:57.960488+0000","last_active":"2024-01-11T07:06:58.973296+0000","last_peered":"2024-01-11T07:06:58.973296+0000","last_clean":"2024-01-11T07:06:58.973296+0000","last_became_active":"2024-01-11T07:06:57.959963+0000","last_became_peered":"2024-01-11T07:06:57.959963+0000","last_unstale":"2024-01-11T07:06:58.973296+0000","last_undegraded":"2024-01-11T07:06:58.973296+0000","last_fullsized":"2024-01-11T07:06:58.973296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982184+0000","last_change":"2024-01-11T07:06:58.982110+0000","last_active":"2024-01-11T07:06:58.982184+0000","last_peered":"2024-01-11T07:06:58.982184+0000","last_clean":"2024-01-11T07:06:58.982184+0000","last_became_active":"2024-01-11T07:06:58.981828+0000","last_became_peered":"2024-01-11T07:06:58.981828+0000","last_unstale":"2024-01-11T07:06:58.982184+0000","last_undegraded":"2024-01-11T07:06:58.982184+0000","last_fullsized":"2024-01-11T07:06:58.982184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.159786+0000","last_change":"2024-01-11T07:06:57.961526+0000","last_active":"2024-01-11T07:06:59.159786+0000","last_peered":"2024-01-11T07:06:59.159786+0000","last_clean":"2024-01-11T07:06:59.159786+0000","last_became_active":"2024-01-11T07:06:57.961256+0000","last_became_peered":"2024-01-11T07:06:57.961256+0000","last_unstale":"2024-01-11T07:06:59.159786+0000","last_undegraded":"2024-01-11T07:06:59.159786+0000","last_fullsized":"2024-01-11T07:06:59.159786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980041+0000","last_change":"2024-01-11T07:06:58.979973+0000","last_active":"2024-01-11T07:06:58.980041+0000","last_peered":"2024-01-11T07:06:58.980041+0000","last_clean":"2024-01-11T07:06:58.980041+0000","last_became_active":"2024-01-11T07:06:58.979738+0000","last_became_peered":"2024-01-11T07:06:58.979738+0000","last_unstale":"2024-01-11T07:06:58.980041+0000","last_undegraded":"2024-01-11T07:06:58.980041+0000","last_fullsized":"2024-01-11T07:06:58.980041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:05.350780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155041+0000","last_change":"2024-01-11T07:06:57.967501+0000","last_active":"2024-01-11T07:06:59.155041+0000","last_peered":"2024-01-11T07:06:59.155041+0000","last_clean":"2024-01-11T07:06:59.155041+0000","last_became_active":"2024-01-11T07:06:57.966742+0000","last_became_peered":"2024-01-11T07:06:57.966742+0000","last_unstale":"2024-01-11T07:06:59.155041+0000","last_undegraded":"2024-01-11T07:06:59.155041+0000","last_fullsized":"2024-01-11T07:06:59.155041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:43:51.416880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981871+0000","last_change":"2024-01-11T07:06:58.981779+0000","last_active":"2024-01-11T07:06:58.981871+0000","last_peered":"2024-01-11T07:06:58.981871+0000","last_clean":"2024-01-11T07:06:58.981871+0000","last_became_active":"2024-01-11T07:06:58.981332+0000","last_became_peered":"2024-01-11T07:06:58.981332+0000","last_unstale":"2024-01-11T07:06:58.981871+0000","last_undegraded":"2024-01-11T07:06:58.981871+0000","last_fullsized":"2024-01-11T07:06:58.981871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:04:40.488711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.976544+0000","last_change":"2024-01-11T07:06:57.960497+0000","last_active":"2024-01-11T07:06:58.976544+0000","last_peered":"2024-01-11T07:06:58.976544+0000","last_clean":"2024-01-11T07:06:58.976544+0000","last_became_active":"2024-01-11T07:06:57.959939+0000","last_became_peered":"2024-01-11T07:06:57.959939+0000","last_unstale":"2024-01-11T07:06:58.976544+0000","last_undegraded":"2024-01-11T07:06:58.976544+0000","last_fullsized":"2024-01-11T07:06:58.976544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982201+0000","last_change":"2024-01-11T07:06:58.982135+0000","last_active":"2024-01-11T07:06:58.982201+0000","last_peered":"2024-01-11T07:06:58.982201+0000","last_clean":"2024-01-11T07:06:58.982201+0000","last_became_active":"2024-01-11T07:06:58.981852+0000","last_became_peered":"2024-01-11T07:06:58.981852+0000","last_unstale":"2024-01-11T07:06:58.982201+0000","last_undegraded":"2024-01-11T07:06:58.982201+0000","last_fullsized":"2024-01-11T07:06:58.982201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:05.350780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968680+0000","last_change":"2024-01-11T07:06:57.966310+0000","last_active":"2024-01-11T07:06:58.968680+0000","last_peered":"2024-01-11T07:06:58.968680+0000","last_clean":"2024-01-11T07:06:58.968680+0000","last_became_active":"2024-01-11T07:06:57.965787+0000","last_became_peered":"2024-01-11T07:06:57.965787+0000","last_unstale":"2024-01-11T07:06:58.968680+0000","last_undegraded":"2024-01-11T07:06:58.968680+0000","last_fullsized":"2024-01-11T07:06:58.968680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.970968+0000","last_change":"2024-01-11T07:06:58.970832+0000","last_active":"2024-01-11T07:06:58.970968+0000","last_peered":"2024-01-11T07:06:58.970968+0000","last_clean":"2024-01-11T07:06:58.970968+0000","last_became_active":"2024-01-11T07:06:58.970599+0000","last_became_peered":"2024-01-11T07:06:58.970599+0000","last_unstale":"2024-01-11T07:06:58.970968+0000","last_undegraded":"2024-01-11T07:06:58.970968+0000","last_fullsized":"2024-01-11T07:06:58.970968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:32.002044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977892+0000","last_change":"2024-01-11T07:06:58.977802+0000","last_active":"2024-01-11T07:06:58.977892+0000","last_peered":"2024-01-11T07:06:58.977892+0000","last_clean":"2024-01-11T07:06:58.977892+0000","last_became_active":"2024-01-11T07:06:58.977558+0000","last_became_peered":"2024-01-11T07:06:58.977558+0000","last_unstale":"2024-01-11T07:06:58.977892+0000","last_undegraded":"2024-01-11T07:06:58.977892+0000","last_fullsized":"2024-01-11T07:06:58.977892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973773+0000","last_change":"2024-01-11T07:06:57.963281+0000","last_active":"2024-01-11T07:06:58.973773+0000","last_peered":"2024-01-11T07:06:58.973773+0000","last_clean":"2024-01-11T07:06:58.973773+0000","last_became_active":"2024-01-11T07:06:57.962993+0000","last_became_peered":"2024-01-11T07:06:57.962993+0000","last_unstale":"2024-01-11T07:06:58.973773+0000","last_undegraded":"2024-01-11T07:06:58.973773+0000","last_fullsized":"2024-01-11T07:06:58.973773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982425+0000","last_change":"2024-01-11T07:06:58.982334+0000","last_active":"2024-01-11T07:06:58.982425+0000","last_peered":"2024-01-11T07:06:58.982425+0000","last_clean":"2024-01-11T07:06:58.982425+0000","last_became_active":"2024-01-11T07:06:58.981895+0000","last_became_peered":"2024-01-11T07:06:58.981895+0000","last_unstale":"2024-01-11T07:06:58.982425+0000","last_undegraded":"2024-01-11T07:06:58.982425+0000","last_fullsized":"2024-01-11T07:06:58.982425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.968343+0000","last_change":"2024-01-11T07:06:57.965590+0000","last_active":"2024-01-11T07:06:58.968343+0000","last_peered":"2024-01-11T07:06:58.968343+0000","last_clean":"2024-01-11T07:06:58.968343+0000","last_became_active":"2024-01-11T07:06:57.964910+0000","last_became_peered":"2024-01-11T07:06:57.964910+0000","last_unstale":"2024-01-11T07:06:58.968343+0000","last_undegraded":"2024-01-11T07:06:58.968343+0000","last_fullsized":"2024-01-11T07:06:58.968343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:13.314633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.972745+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:06:58.972745+0000","last_peered":"2024-01-11T07:06:58.972745+0000","last_clean":"2024-01-11T07:06:58.972745+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:06:58.972745+0000","last_undegraded":"2024-01-11T07:06:58.972745+0000","last_fullsized":"2024-01-11T07:06:58.972745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.969817+0000","last_change":"2024-01-11T07:06:58.969690+0000","last_active":"2024-01-11T07:06:58.969817+0000","last_peered":"2024-01-11T07:06:58.969817+0000","last_clean":"2024-01-11T07:06:58.969817+0000","last_became_active":"2024-01-11T07:06:58.969269+0000","last_became_peered":"2024-01-11T07:06:58.969269+0000","last_unstale":"2024-01-11T07:06:58.969817+0000","last_undegraded":"2024-01-11T07:06:58.969817+0000","last_fullsized":"2024-01-11T07:06:58.969817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971432+0000","last_change":"2024-01-11T07:06:57.964928+0000","last_active":"2024-01-11T07:06:58.971432+0000","last_peered":"2024-01-11T07:06:58.971432+0000","last_clean":"2024-01-11T07:06:58.971432+0000","last_became_active":"2024-01-11T07:06:57.964692+0000","last_became_peered":"2024-01-11T07:06:57.964692+0000","last_unstale":"2024-01-11T07:06:58.971432+0000","last_undegraded":"2024-01-11T07:06:58.971432+0000","last_fullsized":"2024-01-11T07:06:58.971432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:58.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.971975+0000","last_change":"2024-01-11T07:06:57.967422+0000","last_active":"2024-01-11T07:06:58.971975+0000","last_peered":"2024-01-11T07:06:58.971975+0000","last_clean":"2024-01-11T07:06:58.971975+0000","last_became_active":"2024-01-11T07:06:57.967213+0000","last_became_peered":"2024-01-11T07:06:57.967213+0000","last_unstale":"2024-01-11T07:06:58.971975+0000","last_undegraded":"2024-01-11T07:06:58.971975+0000","last_fullsized":"2024-01-11T07:06:58.971975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:16.164276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977579+0000","last_change":"2024-01-11T07:06:58.977325+0000","last_active":"2024-01-11T07:06:58.977579+0000","last_peered":"2024-01-11T07:06:58.977579+0000","last_clean":"2024-01-11T07:06:58.977579+0000","last_became_active":"2024-01-11T07:06:58.977041+0000","last_became_peered":"2024-01-11T07:06:58.977041+0000","last_unstale":"2024-01-11T07:06:58.977579+0000","last_undegraded":"2024-01-11T07:06:58.977579+0000","last_fullsized":"2024-01-11T07:06:58.977579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-11T07:07:00.222933+0000","last_change":"2024-01-11T07:06:57.958705+0000","last_active":"2024-01-11T07:07:00.222933+0000","last_peered":"2024-01-11T07:07:00.222933+0000","last_clean":"2024-01-11T07:07:00.222933+0000","last_became_active":"2024-01-11T07:06:57.958421+0000","last_became_peered":"2024-01-11T07:06:57.958421+0000","last_unstale":"2024-01-11T07:07:00.222933+0000","last_undegraded":"2024-01-11T07:07:00.222933+0000","last_fullsized":"2024-01-11T07:07:00.222933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980832+0000","last_change":"2024-01-11T07:06:58.980504+0000","last_active":"2024-01-11T07:06:58.980832+0000","last_peered":"2024-01-11T07:06:58.980832+0000","last_clean":"2024-01-11T07:06:58.980832+0000","last_became_active":"2024-01-11T07:06:58.980255+0000","last_became_peered":"2024-01-11T07:06:58.980255+0000","last_unstale":"2024-01-11T07:06:58.980832+0000","last_undegraded":"2024-01-11T07:06:58.980832+0000","last_fullsized":"2024-01-11T07:06:58.980832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:34.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.157334+0000","last_change":"2024-01-11T07:06:57.964975+0000","last_active":"2024-01-11T07:06:59.157334+0000","last_peered":"2024-01-11T07:06:59.157334+0000","last_clean":"2024-01-11T07:06:59.157334+0000","last_became_active":"2024-01-11T07:06:57.964709+0000","last_became_peered":"2024-01-11T07:06:57.964709+0000","last_unstale":"2024-01-11T07:06:59.157334+0000","last_undegraded":"2024-01-11T07:06:59.157334+0000","last_fullsized":"2024-01-11T07:06:59.157334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:57.287923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981582+0000","last_change":"2024-01-11T07:06:58.981191+0000","last_active":"2024-01-11T07:06:58.981582+0000","last_peered":"2024-01-11T07:06:58.981582+0000","last_clean":"2024-01-11T07:06:58.981582+0000","last_became_active":"2024-01-11T07:06:58.980786+0000","last_became_peered":"2024-01-11T07:06:58.980786+0000","last_unstale":"2024-01-11T07:06:58.981582+0000","last_undegraded":"2024-01-11T07:06:58.981582+0000","last_fullsized":"2024-01-11T07:06:58.981582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975354+0000","last_change":"2024-01-11T07:06:57.963735+0000","last_active":"2024-01-11T07:06:58.975354+0000","last_peered":"2024-01-11T07:06:58.975354+0000","last_clean":"2024-01-11T07:06:58.975354+0000","last_became_active":"2024-01-11T07:06:57.963471+0000","last_became_peered":"2024-01-11T07:06:57.963471+0000","last_unstale":"2024-01-11T07:06:58.975354+0000","last_undegraded":"2024-01-11T07:06:58.975354+0000","last_fullsized":"2024-01-11T07:06:58.975354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978396+0000","last_change":"2024-01-11T07:06:58.978278+0000","last_active":"2024-01-11T07:06:58.978396+0000","last_peered":"2024-01-11T07:06:58.978396+0000","last_clean":"2024-01-11T07:06:58.978396+0000","last_became_active":"2024-01-11T07:06:58.976004+0000","last_became_peered":"2024-01-11T07:06:58.976004+0000","last_unstale":"2024-01-11T07:06:58.978396+0000","last_undegraded":"2024-01-11T07:06:58.978396+0000","last_fullsized":"2024-01-11T07:06:58.978396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:58.292618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.976655+0000","last_change":"2024-01-11T07:06:57.965092+0000","last_active":"2024-01-11T07:06:58.976655+0000","last_peered":"2024-01-11T07:06:58.976655+0000","last_clean":"2024-01-11T07:06:58.976655+0000","last_became_active":"2024-01-11T07:06:57.964872+0000","last_became_peered":"2024-01-11T07:06:57.964872+0000","last_unstale":"2024-01-11T07:06:58.976655+0000","last_undegraded":"2024-01-11T07:06:58.976655+0000","last_fullsized":"2024-01-11T07:06:58.976655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:51:41.448368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981559+0000","last_change":"2024-01-11T07:06:58.981370+0000","last_active":"2024-01-11T07:06:58.981559+0000","last_peered":"2024-01-11T07:06:58.981559+0000","last_clean":"2024-01-11T07:06:58.981559+0000","last_became_active":"2024-01-11T07:06:58.981071+0000","last_became_peered":"2024-01-11T07:06:58.981071+0000","last_unstale":"2024-01-11T07:06:58.981559+0000","last_undegraded":"2024-01-11T07:06:58.981559+0000","last_fullsized":"2024-01-11T07:06:58.981559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:51.450792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973250+0000","last_change":"2024-01-11T07:06:57.962256+0000","last_active":"2024-01-11T07:06:58.973250+0000","last_peered":"2024-01-11T07:06:58.973250+0000","last_clean":"2024-01-11T07:06:58.973250+0000","last_became_active":"2024-01-11T07:06:57.961595+0000","last_became_peered":"2024-01-11T07:06:57.961595+0000","last_unstale":"2024-01-11T07:06:58.973250+0000","last_undegraded":"2024-01-11T07:06:58.973250+0000","last_fullsized":"2024-01-11T07:06:58.973250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:30.997349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.984461+0000","last_change":"2024-01-11T07:06:58.984384+0000","last_active":"2024-01-11T07:06:58.984461+0000","last_peered":"2024-01-11T07:06:58.984461+0000","last_clean":"2024-01-11T07:06:58.984461+0000","last_became_active":"2024-01-11T07:06:58.984107+0000","last_became_peered":"2024-01-11T07:06:58.984107+0000","last_unstale":"2024-01-11T07:06:58.984461+0000","last_undegraded":"2024-01-11T07:06:58.984461+0000","last_fullsized":"2024-01-11T07:06:58.984461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.154946+0000","last_change":"2024-01-11T07:06:57.963683+0000","last_active":"2024-01-11T07:06:59.154946+0000","last_peered":"2024-01-11T07:06:59.154946+0000","last_clean":"2024-01-11T07:06:59.154946+0000","last_became_active":"2024-01-11T07:06:57.963452+0000","last_became_peered":"2024-01-11T07:06:57.963452+0000","last_unstale":"2024-01-11T07:06:59.154946+0000","last_undegraded":"2024-01-11T07:06:59.154946+0000","last_fullsized":"2024-01-11T07:06:59.154946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:19.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978185+0000","last_change":"2024-01-11T07:06:58.978001+0000","last_active":"2024-01-11T07:06:58.978185+0000","last_peered":"2024-01-11T07:06:58.978185+0000","last_clean":"2024-01-11T07:06:58.978185+0000","last_became_active":"2024-01-11T07:06:58.977325+0000","last_became_peered":"2024-01-11T07:06:58.977325+0000","last_unstale":"2024-01-11T07:06:58.978185+0000","last_undegraded":"2024-01-11T07:06:58.978185+0000","last_fullsized":"2024-01-11T07:06:58.978185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.973136+0000","last_change":"2024-01-11T07:06:57.966946+0000","last_active":"2024-01-11T07:06:58.973136+0000","last_peered":"2024-01-11T07:06:58.973136+0000","last_clean":"2024-01-11T07:06:58.973136+0000","last_became_active":"2024-01-11T07:06:57.966738+0000","last_became_peered":"2024-01-11T07:06:57.966738+0000","last_unstale":"2024-01-11T07:06:58.973136+0000","last_undegraded":"2024-01-11T07:06:58.973136+0000","last_fullsized":"2024-01-11T07:06:58.973136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981855+0000","last_change":"2024-01-11T07:06:58.981768+0000","last_active":"2024-01-11T07:06:58.981855+0000","last_peered":"2024-01-11T07:06:58.981855+0000","last_clean":"2024-01-11T07:06:58.981855+0000","last_became_active":"2024-01-11T07:06:58.981467+0000","last_became_peered":"2024-01-11T07:06:58.981467+0000","last_unstale":"2024-01-11T07:06:58.981855+0000","last_undegraded":"2024-01-11T07:06:58.981855+0000","last_fullsized":"2024-01-11T07:06:58.981855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:22:00.354437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.156189+0000","last_change":"2024-01-11T07:06:57.960096+0000","last_active":"2024-01-11T07:06:59.156189+0000","last_peered":"2024-01-11T07:06:59.156189+0000","last_clean":"2024-01-11T07:06:59.156189+0000","last_became_active":"2024-01-11T07:06:57.959880+0000","last_became_peered":"2024-01-11T07:06:57.959880+0000","last_unstale":"2024-01-11T07:06:59.156189+0000","last_undegraded":"2024-01-11T07:06:59.156189+0000","last_fullsized":"2024-01-11T07:06:59.156189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:36.081296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983031+0000","last_change":"2024-01-11T07:06:58.982845+0000","last_active":"2024-01-11T07:06:58.983031+0000","last_peered":"2024-01-11T07:06:58.983031+0000","last_clean":"2024-01-11T07:06:58.983031+0000","last_became_active":"2024-01-11T07:06:58.980928+0000","last_became_peered":"2024-01-11T07:06:58.980928+0000","last_unstale":"2024-01-11T07:06:58.983031+0000","last_undegraded":"2024-01-11T07:06:58.983031+0000","last_fullsized":"2024-01-11T07:06:58.983031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:39:48.993236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:59.155763+0000","last_change":"2024-01-11T07:06:57.963359+0000","last_active":"2024-01-11T07:06:59.155763+0000","last_peered":"2024-01-11T07:06:59.155763+0000","last_clean":"2024-01-11T07:06:59.155763+0000","last_became_active":"2024-01-11T07:06:57.963116+0000","last_became_peered":"2024-01-11T07:06:57.963116+0000","last_unstale":"2024-01-11T07:06:59.155763+0000","last_undegraded":"2024-01-11T07:06:59.155763+0000","last_fullsized":"2024-01-11T07:06:59.155763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:56.943319+0000","last_clean_scrub_stamp":"2024-01-11T07:06:56.943319+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:19.906878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978190+0000","last_change":"2024-01-11T07:06:58.978133+0000","last_active":"2024-01-11T07:06:58.978190+0000","last_peered":"2024-01-11T07:06:58.978190+0000","last_clean":"2024-01-11T07:06:58.978190+0000","last_became_active":"2024-01-11T07:06:58.977605+0000","last_became_peered":"2024-01-11T07:06:58.977605+0000","last_unstale":"2024-01-11T07:06:58.978190+0000","last_undegraded":"2024-01-11T07:06:58.978190+0000","last_fullsized":"2024-01-11T07:06:58.978190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982813+0000","last_change":"2024-01-11T07:06:58.982744+0000","last_active":"2024-01-11T07:06:58.982813+0000","last_peered":"2024-01-11T07:06:58.982813+0000","last_clean":"2024-01-11T07:06:58.982813+0000","last_became_active":"2024-01-11T07:06:58.982442+0000","last_became_peered":"2024-01-11T07:06:58.982442+0000","last_unstale":"2024-01-11T07:06:58.982813+0000","last_undegraded":"2024-01-11T07:06:58.982813+0000","last_fullsized":"2024-01-11T07:06:58.982813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:47:11.719208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.975116+0000","last_change":"2024-01-11T07:06:58.974838+0000","last_active":"2024-01-11T07:06:58.975116+0000","last_peered":"2024-01-11T07:06:58.975116+0000","last_clean":"2024-01-11T07:06:58.975116+0000","last_became_active":"2024-01-11T07:06:58.974628+0000","last_became_peered":"2024-01-11T07:06:58.974628+0000","last_unstale":"2024-01-11T07:06:58.975116+0000","last_undegraded":"2024-01-11T07:06:58.975116+0000","last_fullsized":"2024-01-11T07:06:58.975116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:17.168970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.980086+0000","last_change":"2024-01-11T07:06:58.979993+0000","last_active":"2024-01-11T07:06:58.980086+0000","last_peered":"2024-01-11T07:06:58.980086+0000","last_clean":"2024-01-11T07:06:58.980086+0000","last_became_active":"2024-01-11T07:06:58.979782+0000","last_became_peered":"2024-01-11T07:06:58.979782+0000","last_unstale":"2024-01-11T07:06:58.980086+0000","last_undegraded":"2024-01-11T07:06:58.980086+0000","last_fullsized":"2024-01-11T07:06:58.980086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979408+0000","last_change":"2024-01-11T07:06:58.979256+0000","last_active":"2024-01-11T07:06:58.979408+0000","last_peered":"2024-01-11T07:06:58.979408+0000","last_clean":"2024-01-11T07:06:58.979408+0000","last_became_active":"2024-01-11T07:06:58.979025+0000","last_became_peered":"2024-01-11T07:06:58.979025+0000","last_unstale":"2024-01-11T07:06:58.979408+0000","last_undegraded":"2024-01-11T07:06:58.979408+0000","last_fullsized":"2024-01-11T07:06:58.979408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978671+0000","last_change":"2024-01-11T07:06:58.978602+0000","last_active":"2024-01-11T07:06:58.978671+0000","last_peered":"2024-01-11T07:06:58.978671+0000","last_clean":"2024-01-11T07:06:58.978671+0000","last_became_active":"2024-01-11T07:06:58.978265+0000","last_became_peered":"2024-01-11T07:06:58.978265+0000","last_unstale":"2024-01-11T07:06:58.978671+0000","last_undegraded":"2024-01-11T07:06:58.978671+0000","last_fullsized":"2024-01-11T07:06:58.978671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:35:17.678645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978598+0000","last_change":"2024-01-11T07:06:58.978524+0000","last_active":"2024-01-11T07:06:58.978598+0000","last_peered":"2024-01-11T07:06:58.978598+0000","last_clean":"2024-01-11T07:06:58.978598+0000","last_became_active":"2024-01-11T07:06:58.978266+0000","last_became_peered":"2024-01-11T07:06:58.978266+0000","last_unstale":"2024-01-11T07:06:58.978598+0000","last_undegraded":"2024-01-11T07:06:58.978598+0000","last_fullsized":"2024-01-11T07:06:58.978598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:07.482069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977723+0000","last_change":"2024-01-11T07:06:58.977673+0000","last_active":"2024-01-11T07:06:58.977723+0000","last_peered":"2024-01-11T07:06:58.977723+0000","last_clean":"2024-01-11T07:06:58.977723+0000","last_became_active":"2024-01-11T07:06:58.977490+0000","last_became_peered":"2024-01-11T07:06:58.977490+0000","last_unstale":"2024-01-11T07:06:58.977723+0000","last_undegraded":"2024-01-11T07:06:58.977723+0000","last_fullsized":"2024-01-11T07:06:58.977723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:25.471130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.982405+0000","last_change":"2024-01-11T07:06:58.982014+0000","last_active":"2024-01-11T07:06:58.982405+0000","last_peered":"2024-01-11T07:06:58.982405+0000","last_clean":"2024-01-11T07:06:58.982405+0000","last_became_active":"2024-01-11T07:06:58.981670+0000","last_became_peered":"2024-01-11T07:06:58.981670+0000","last_unstale":"2024-01-11T07:06:58.982405+0000","last_undegraded":"2024-01-11T07:06:58.982405+0000","last_fullsized":"2024-01-11T07:06:58.982405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:53.790298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.979096+0000","last_change":"2024-01-11T07:06:58.978874+0000","last_active":"2024-01-11T07:06:58.979096+0000","last_peered":"2024-01-11T07:06:58.979096+0000","last_clean":"2024-01-11T07:06:58.979096+0000","last_became_active":"2024-01-11T07:06:58.978319+0000","last_became_peered":"2024-01-11T07:06:58.978319+0000","last_unstale":"2024-01-11T07:06:58.979096+0000","last_undegraded":"2024-01-11T07:06:58.979096+0000","last_fullsized":"2024-01-11T07:06:58.979096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.977051+0000","last_change":"2024-01-11T07:06:58.976808+0000","last_active":"2024-01-11T07:06:58.977051+0000","last_peered":"2024-01-11T07:06:58.977051+0000","last_clean":"2024-01-11T07:06:58.977051+0000","last_became_active":"2024-01-11T07:06:58.975696+0000","last_became_peered":"2024-01-11T07:06:58.975696+0000","last_unstale":"2024-01-11T07:06:58.977051+0000","last_undegraded":"2024-01-11T07:06:58.977051+0000","last_fullsized":"2024-01-11T07:06:58.977051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:35.224847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.983760+0000","last_change":"2024-01-11T07:06:58.983202+0000","last_active":"2024-01-11T07:06:58.983760+0000","last_peered":"2024-01-11T07:06:58.983760+0000","last_clean":"2024-01-11T07:06:58.983760+0000","last_became_active":"2024-01-11T07:06:58.981685+0000","last_became_peered":"2024-01-11T07:06:58.981685+0000","last_unstale":"2024-01-11T07:06:58.983760+0000","last_undegraded":"2024-01-11T07:06:58.983760+0000","last_fullsized":"2024-01-11T07:06:58.983760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:04:53.068090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.978684+0000","last_change":"2024-01-11T07:06:58.978620+0000","last_active":"2024-01-11T07:06:58.978684+0000","last_peered":"2024-01-11T07:06:58.978684+0000","last_clean":"2024-01-11T07:06:58.978684+0000","last_became_active":"2024-01-11T07:06:58.978229+0000","last_became_peered":"2024-01-11T07:06:58.978229+0000","last_unstale":"2024-01-11T07:06:58.978684+0000","last_undegraded":"2024-01-11T07:06:58.978684+0000","last_fullsized":"2024-01-11T07:06:58.978684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:21:36.159743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-11T07:06:58.981445+0000","last_change":"2024-01-11T07:06:58.981078+0000","last_active":"2024-01-11T07:06:58.981445+0000","last_peered":"2024-01-11T07:06:58.981445+0000","last_clean":"2024-01-11T07:06:58.981445+0000","last_became_active":"2024-01-11T07:06:58.980821+0000","last_became_peered":"2024-01-11T07:06:58.980821+0000","last_unstale":"2024-01-11T07:06:58.981445+0000","last_undegraded":"2024-01-11T07:06:58.981445+0000","last_fullsized":"2024-01-11T07:06:58.981445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:57.948014+0000","last_clean_scrub_stamp":"2024-01-11T07:06:57.948014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:46:24.575175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":655,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199328880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":79,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195232,"statfs":{"total":200000000,"available":199918266,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":655,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199328824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":79,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195232,"statfs":{"total":200000000,"available":199918266,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919676,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":78,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195234,"statfs":{"total":200000000,"available":199919712,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:13.293 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-11T07:07:13.294 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-11T07:07:13.294 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-11T07:07:13.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-11T07:07:13.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.467+0000 7fcbe5d1f700 1 Processor -- start 2024-01-11T07:07:13.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.467+0000 7fcbe5d1f700 1 -- start start 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbe5d1f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbe0159c90 0x7fcbe00b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbe5d1f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe00ce9b0 con 0x7fcbe015a060 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbe5d1f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe00cf530 con 0x7fcbe0069320 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbe5d1f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe00cf9c0 con 0x7fcbe0159c90 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbdf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbe0159c90 0x7fcbe00b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbdeffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcbe015a060 0x7fcbe00b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40292/0 (socket says 172.21.15.7:40292) 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbdeffd700 1 -- 172.21.15.7:0/2341020833 learned_addr learned my addr 172.21.15.7:0/2341020833 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbdf7fe700 1 -- 172.21.15.7:0/2341020833 >> v1:172.21.15.113:6789/0 conn(0x7fcbe0069320 legacy=0x7fcbe00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbdf7fe700 1 -- 172.21.15.7:0/2341020833 >> v1:172.21.15.7:6789/0 conn(0x7fcbe015a060 legacy=0x7fcbe00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbdf7fe700 1 -- 172.21.15.7:0/2341020833 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe00cec00 con 0x7fcbe0159c90 2024-01-11T07:07:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbdf7fe700 1 --2- 172.21.15.7:0/2341020833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe00b1820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00098e0 tx=0x7fcbd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67926bcfddb561e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.468+0000 7fcbde7fc700 1 -- 172.21.15.7:0/2341020833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0009e80 con 0x7fcbe0159c90 2024-01-11T07:07:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbde7fc700 1 -- 172.21.15.7:0/2341020833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd000b860 con 0x7fcbe0159c90 2024-01-11T07:07:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbde7fc700 1 -- 172.21.15.7:0/2341020833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000bbe0 con 0x7fcbe0159c90 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/2341020833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 msgr2=0x7fcbe00b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbe5d1f700 1 --2- 172.21.15.7:0/2341020833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe00b1820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00098e0 tx=0x7fcbd00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/2341020833 shutdown_connections 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbe5d1f700 1 --2- 172.21.15.7:0/2341020833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe00b1820 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/2341020833 >> 172.21.15.7:0/2341020833 conn(0x7fcbe0071100 msgr2=0x7fcbe006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/2341020833 shutdown_connections 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.469+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/2341020833 wait complete. 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbe5d1f700 1 Processor -- start 2024-01-11T07:07:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbe5d1f700 1 -- start start 2024-01-11T07:07:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbe5d1f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbe0069320 0x7fcbe016cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbe5d1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe0156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbe5d1f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbe015a060 0x7fcbe00b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbe5d1f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0157fd0 con 0x7fcbe0159c90 2024-01-11T07:07:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbe5d1f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe01573c0 con 0x7fcbe015a060 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbe5d1f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe01576a0 con 0x7fcbe0069320 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbdeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe0156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.471+0000 7fcbdeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe0156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41614/0 (socket says 172.21.15.7:41614) 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdeffd700 1 -- 172.21.15.7:0/1063031940 learned_addr learned my addr 172.21.15.7:0/1063031940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdf7fe700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbe0069320 0x7fcbe016cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdffff700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbe015a060 0x7fcbe00b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdeffd700 1 -- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbe0069320 msgr2=0x7fcbe016cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdeffd700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbe0069320 0x7fcbe016cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdeffd700 1 -- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbe015a060 msgr2=0x7fcbe00b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdeffd700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbe015a060 0x7fcbe00b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdeffd700 1 -- 172.21.15.7:0/1063031940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009420 con 0x7fcbe0159c90 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdf7fe700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbe0069320 0x7fcbe016cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdffff700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbe015a060 0x7fcbe00b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbdeffd700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe0156a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c7c0 tx=0x7fcbd4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbc77fe700 1 -- 172.21.15.7:0/1063031940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4005b30 con 0x7fcbe0159c90 2024-01-11T07:07:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbc77fe700 1 -- 172.21.15.7:0/1063031940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd4005cb0 con 0x7fcbe0159c90 2024-01-11T07:07:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe0157980 con 0x7fcbe0159c90 2024-01-11T07:07:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbc77fe700 1 -- 172.21.15.7:0/1063031940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4004a80 con 0x7fcbe0159c90 2024-01-11T07:07:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.472+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe016d470 con 0x7fcbe0159c90 2024-01-11T07:07:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.473+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe004c3c0 con 0x7fcbe0159c90 2024-01-11T07:07:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.474+0000 7fcbc77fe700 1 -- 172.21.15.7:0/1063031940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcbd40070f0 con 0x7fcbe0159c90 2024-01-11T07:07:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.474+0000 7fcbc77fe700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbcc066f30 0x7fcbcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.474+0000 7fcbc77fe700 1 -- 172.21.15.7:0/1063031940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcbd4083940 con 0x7fcbe0159c90 2024-01-11T07:07:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.474+0000 7fcbdf7fe700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbcc066f30 0x7fcbcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.475+0000 7fcbdf7fe700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbcc066f30 0x7fcbcc0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00093f0 tx=0x7fcbd0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.479+0000 7fcbc77fe700 1 -- 172.21.15.7:0/1063031940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbd4050c30 con 0x7fcbe0159c90 2024-01-11T07:07:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.729+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcbe015ec20 con 0x7fcbe0159c90 2024-01-11T07:07:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.729+0000 7fcbc77fe700 1 -- 172.21.15.7:0/1063031940 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcbd4018020 con 0x7fcbe0159c90 2024-01-11T07:07:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:13.729 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-11T07:07:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbcc066f30 msgr2=0x7fcbcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbcc066f30 0x7fcbcc0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00093f0 tx=0x7fcbd0017000 comp rx=0 tx=0).stop 2024-01-11T07:07:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 msgr2=0x7fcbe0156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe0156a00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c7c0 tx=0x7fcbd4007f70 comp rx=0 tx=0).stop 2024-01-11T07:07:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 shutdown_connections 2024-01-11T07:07:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbcc066f30 0x7fcbcc0693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbe015a060 0x7fcbe00b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcbe0069320 0x7fcbe016cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 --2- 172.21.15.7:0/1063031940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbe0159c90 0x7fcbe0156a00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 >> 172.21.15.7:0/1063031940 conn(0x7fcbe0071100 msgr2=0x7fcbe015cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 shutdown_connections 2024-01-11T07:07:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.731+0000 7fcbe5d1f700 1 -- 172.21.15.7:0/1063031940 wait complete. 2024-01-11T07:07:13.744 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-11T07:07:13.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.876+0000 7ff360358700 1 Processor -- start 2024-01-11T07:07:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.876+0000 7ff360358700 1 -- start start 2024-01-11T07:07:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff360358700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3580ac3a0 0x7ff3580cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff360358700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff35815d920 0x7ff35815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff360358700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 0x7ff35815a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff360358700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35815e2d0 con 0x7ff3580ac3a0 2024-01-11T07:07:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff360358700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35806ac30 con 0x7ff35815d920 2024-01-11T07:07:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff360358700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580ac770 con 0x7ff35815a2c0 2024-01-11T07:07:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3580ac3a0 0x7ff3580cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3580ac3a0 0x7ff3580cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41616/0 (socket says 172.21.15.7:41616) 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 -- 172.21.15.7:0/4261065535 learned_addr learned my addr 172.21.15.7:0/4261065535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35d8f3700 1 --2- 172.21.15.7:0/4261065535 >> v2:172.21.15.113:3300/0 conn(0x7ff35815d920 0x7ff35815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e8f5700 1 --2- 172.21.15.7:0/4261065535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 0x7ff35815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 -- 172.21.15.7:0/4261065535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 msgr2=0x7ff35815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 --2- 172.21.15.7:0/4261065535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 0x7ff35815a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 -- 172.21.15.7:0/4261065535 >> v2:172.21.15.113:3300/0 conn(0x7ff35815d920 msgr2=0x7ff35815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 --2- 172.21.15.7:0/4261065535 >> v2:172.21.15.113:3300/0 conn(0x7ff35815d920 0x7ff35815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 -- 172.21.15.7:0/4261065535 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35806a300 con 0x7ff3580ac3a0 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35d8f3700 1 --2- 172.21.15.7:0/4261065535 >> v2:172.21.15.113:3300/0 conn(0x7ff35815d920 0x7ff35815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.877+0000 7ff35e0f4700 1 --2- 172.21.15.7:0/4261065535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3580ac3a0 0x7ff3580cac60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff35806ce10 tx=0x7ff34c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbc7d75a0b21581 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff35d0f2700 1 -- 172.21.15.7:0/4261065535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c00b860 con 0x7ff3580ac3a0 2024-01-11T07:07:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff35d0f2700 1 -- 172.21.15.7:0/4261065535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c00b9c0 con 0x7ff3580ac3a0 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff35d0f2700 1 -- 172.21.15.7:0/4261065535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c00bd60 con 0x7ff3580ac3a0 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 -- 172.21.15.7:0/4261065535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3580ac3a0 msgr2=0x7ff3580cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 --2- 172.21.15.7:0/4261065535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3580ac3a0 0x7ff3580cac60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff35806ce10 tx=0x7ff34c009250 comp rx=0 tx=0).stop 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 -- 172.21.15.7:0/4261065535 shutdown_connections 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 --2- 172.21.15.7:0/4261065535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3580ac3a0 0x7ff3580cac60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 --2- 172.21.15.7:0/4261065535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 0x7ff35815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 --2- 172.21.15.7:0/4261065535 >> v2:172.21.15.113:3300/0 conn(0x7ff35815d920 0x7ff35815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 -- 172.21.15.7:0/4261065535 >> 172.21.15.7:0/4261065535 conn(0x7ff3580cd090 msgr2=0x7ff3580cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 -- 172.21.15.7:0/4261065535 shutdown_connections 2024-01-11T07:07:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.878+0000 7ff360358700 1 -- 172.21.15.7:0/4261065535 wait complete. 2024-01-11T07:07:13.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.880+0000 7ff360358700 1 Processor -- start 2024-01-11T07:07:13.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.880+0000 7ff360358700 1 -- start start 2024-01-11T07:07:13.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.880+0000 7ff360358700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3580ac3a0 0x7ff3580af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.880+0000 7ff360358700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 0x7ff35813b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.880+0000 7ff360358700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff35815d920 0x7ff3580ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.880+0000 7ff360358700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580c8bb0 con 0x7ff35815d920 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.880+0000 7ff360358700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580af020 con 0x7ff3580ac3a0 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.880+0000 7ff360358700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580b0db0 con 0x7ff35815a2c0 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff35815d920 0x7ff3580ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e0f4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3580ac3a0 0x7ff3580af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35d8f3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 0x7ff35813b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff35815d920 0x7ff3580ad380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41620/0 (socket says 172.21.15.7:41620) 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e0f4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3580ac3a0 0x7ff3580af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36626/0 (socket says 172.21.15.7:36626) 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 -- 172.21.15.7:0/3007685647 learned_addr learned my addr 172.21.15.7:0/3007685647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 -- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 msgr2=0x7ff35813b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 0x7ff35813b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 -- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3580ac3a0 msgr2=0x7ff3580af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3580ac3a0 0x7ff3580af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 -- 172.21.15.7:0/3007685647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c0094f0 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff35e8f5700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff35815d920 0x7ff3580ad380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff350007f20 tx=0x7ff350009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff34a7fc700 1 -- 172.21.15.7:0/3007685647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35000eb90 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff34a7fc700 1 -- 172.21.15.7:0/3007685647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35000ed10 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35813b700 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff34a7fc700 1 -- 172.21.15.7:0/3007685647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350014670 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.881+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3580ae410 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.882+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3580af1b0 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.883+0000 7ff34a7fc700 1 -- 172.21.15.7:0/3007685647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff3500149e0 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.883+0000 7ff34a7fc700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff33c066f60 0x7ff33c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.883+0000 7ff34a7fc700 1 -- 172.21.15.7:0/3007685647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff350084ae0 con 0x7ff35815d920 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.883+0000 7ff35e0f4700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff33c066f60 0x7ff33c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.884+0000 7ff35e0f4700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff33c066f60 0x7ff33c069410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff35806ce10 tx=0x7ff34c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:13.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:13.888+0000 7ff34a7fc700 1 -- 172.21.15.7:0/3007685647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff350051d50 con 0x7ff35815d920 2024-01-11T07:07:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.094+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff35804c3c0 con 0x7ff35815d920 2024-01-11T07:07:14.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.095+0000 7ff34a7fc700 1 -- 172.21.15.7:0/3007685647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7ff350055370 con 0x7ff35815d920 2024-01-11T07:07:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:14.095 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff33c066f60 msgr2=0x7ff33c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff33c066f60 0x7ff33c069410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff35806ce10 tx=0x7ff34c017000 comp rx=0 tx=0).stop 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff35815d920 msgr2=0x7ff3580ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff35815d920 0x7ff3580ad380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff350007f20 tx=0x7ff350009860 comp rx=0 tx=0).stop 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 shutdown_connections 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff33c066f60 0x7ff33c069410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff35815d920 0x7ff3580ad380 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3580ac3a0 0x7ff3580af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 --2- 172.21.15.7:0/3007685647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff35815a2c0 0x7ff35813b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 >> 172.21.15.7:0/3007685647 conn(0x7ff3580cd090 msgr2=0x7ff3580c1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 shutdown_connections 2024-01-11T07:07:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.098+0000 7ff360358700 1 -- 172.21.15.7:0/3007685647 wait complete. 2024-01-11T07:07:14.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:06:56.038804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:06:57.365958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:07:14.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:07:14.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.247+0000 7fd711391700 1 Processor -- start 2024-01-11T07:07:14.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.247+0000 7fd711391700 1 -- start start 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.247+0000 7fd711391700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c0aee00 0x7fd70c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.247+0000 7fd711391700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd70c15af90 0x7fd70c0c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.247+0000 7fd711391700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd70c0c2c80 0x7fd70c0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.247+0000 7fd711391700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70c0c3100 con 0x7fd70c15af90 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.247+0000 7fd711391700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70c1573c0 con 0x7fd70c0c2c80 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.247+0000 7fd711391700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c158200 con 0x7fd70c0aee00 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd70c15af90 0x7fd70c0c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd70c15af90 0x7fd70c0c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41628/0 (socket says 172.21.15.7:41628) 2024-01-11T07:07:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 -- 172.21.15.7:0/4231143628 learned_addr learned my addr 172.21.15.7:0/4231143628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70affd700 1 --2- 172.21.15.7:0/4231143628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c0aee00 0x7fd70c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70b7fe700 1 --2- 172.21.15.7:0/4231143628 >> v2:172.21.15.113:3300/0 conn(0x7fd70c0c2c80 0x7fd70c0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 -- 172.21.15.7:0/4231143628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c0aee00 msgr2=0x7fd70c156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 --2- 172.21.15.7:0/4231143628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c0aee00 0x7fd70c156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 -- 172.21.15.7:0/4231143628 >> v2:172.21.15.113:3300/0 conn(0x7fd70c0c2c80 msgr2=0x7fd70c0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 --2- 172.21.15.7:0/4231143628 >> v2:172.21.15.113:3300/0 conn(0x7fd70c0c2c80 0x7fd70c0c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 -- 172.21.15.7:0/4231143628 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c1575f0 con 0x7fd70c15af90 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70affd700 1 --2- 172.21.15.7:0/4231143628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c0aee00 0x7fd70c156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd70a7fc700 1 --2- 172.21.15.7:0/4231143628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c15af90 0x7fd70c0c2740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0ca4b0 tx=0x7fd6f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3f757b01774eda server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd709ffb700 1 -- 172.21.15.7:0/4231143628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400b820 con 0x7fd70c15af90 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd709ffb700 1 -- 172.21.15.7:0/4231143628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400b980 con 0x7fd70c15af90 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.248+0000 7fd709ffb700 1 -- 172.21.15.7:0/4231143628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400bd20 con 0x7fd70c15af90 2024-01-11T07:07:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 -- 172.21.15.7:0/4231143628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c15af90 msgr2=0x7fd70c0c2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 --2- 172.21.15.7:0/4231143628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c15af90 0x7fd70c0c2740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0ca4b0 tx=0x7fd6f4009250 comp rx=0 tx=0).stop 2024-01-11T07:07:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 -- 172.21.15.7:0/4231143628 shutdown_connections 2024-01-11T07:07:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 --2- 172.21.15.7:0/4231143628 >> v2:172.21.15.113:3300/0 conn(0x7fd70c0c2c80 0x7fd70c0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 --2- 172.21.15.7:0/4231143628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c0aee00 0x7fd70c156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 --2- 172.21.15.7:0/4231143628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c15af90 0x7fd70c0c2740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 -- 172.21.15.7:0/4231143628 >> 172.21.15.7:0/4231143628 conn(0x7fd70c063230 msgr2=0x7fd70c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 -- 172.21.15.7:0/4231143628 shutdown_connections 2024-01-11T07:07:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.249+0000 7fd711391700 1 -- 172.21.15.7:0/4231143628 wait complete. 2024-01-11T07:07:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.250+0000 7fd711391700 1 Processor -- start 2024-01-11T07:07:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.250+0000 7fd711391700 1 -- start start 2024-01-11T07:07:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd711391700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c0aee00 0x7fd70c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd711391700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c0c2c80 0x7fd70c147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd711391700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c15af90 0x7fd70c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd711391700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c16b270 con 0x7fd70c0c2c80 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd711391700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c16b510 con 0x7fd70c0aee00 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd711391700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c16b7f0 con 0x7fd70c15af90 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd70a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c0c2c80 0x7fd70c147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd70a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c0c2c80 0x7fd70c147160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41630/0 (socket says 172.21.15.7:41630) 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.251+0000 7fd70a7fc700 1 -- 172.21.15.7:0/46355253 learned_addr learned my addr 172.21.15.7:0/46355253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd70b7fe700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c15af90 0x7fd70c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd70affd700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c0aee00 0x7fd70c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd70a7fc700 1 -- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c15af90 msgr2=0x7fd70c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd70a7fc700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c15af90 0x7fd70c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd70a7fc700 1 -- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c0aee00 msgr2=0x7fd70c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd70a7fc700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c0aee00 0x7fd70c0b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd70a7fc700 1 -- 172.21.15.7:0/46355253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f40094f0 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd70a7fc700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c0c2c80 0x7fd70c147160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40094c0 tx=0x7fd6f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd702ffd700 1 -- 172.21.15.7:0/46355253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f401a450 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd702ffd700 1 -- 172.21.15.7:0/46355253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f401a5b0 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70c16bad0 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd702ffd700 1 -- 172.21.15.7:0/46355253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f401a950 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.252+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70c0b2470 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.253+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70c16c120 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.254+0000 7fd702ffd700 1 -- 172.21.15.7:0/46355253 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd6f401ad00 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.254+0000 7fd702ffd700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.254+0000 7fd702ffd700 1 -- 172.21.15.7:0/46355253 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd6f4085f40 con 0x7fd70c0c2c80 2024-01-11T07:07:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.254+0000 7fd70affd700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.255+0000 7fd70affd700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd6ec066f30 0x7fd6ec0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc006f00 tx=0x7fd6fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.258+0000 7fd702ffd700 1 -- 172.21.15.7:0/46355253 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6f40531b0 con 0x7fd70c0c2c80 2024-01-11T07:07:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.463+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd70c04c3c0 con 0x7fd70c0c2c80 2024-01-11T07:07:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.463+0000 7fd702ffd700 1 -- 172.21.15.7:0/46355253 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6f40567d0 con 0x7fd70c0c2c80 2024-01-11T07:07:14.463 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:07:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.465+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd6ec066f30 msgr2=0x7fd6ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.465+0000 7fd711391700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd6ec066f30 0x7fd6ec0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc006f00 tx=0x7fd6fc008040 comp rx=0 tx=0).stop 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.465+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c0c2c80 msgr2=0x7fd70c147160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.465+0000 7fd711391700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c0c2c80 0x7fd70c147160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40094c0 tx=0x7fd6f4017000 comp rx=0 tx=0).stop 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.466+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 shutdown_connections 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.466+0000 7fd711391700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd6ec066f30 0x7fd6ec0693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.466+0000 7fd711391700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd70c15af90 0x7fd70c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.466+0000 7fd711391700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd70c0aee00 0x7fd70c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.466+0000 7fd711391700 1 --2- 172.21.15.7:0/46355253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd70c0c2c80 0x7fd70c147160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.466+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 >> 172.21.15.7:0/46355253 conn(0x7fd70c063230 msgr2=0x7fd70c0cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.466+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 shutdown_connections 2024-01-11T07:07:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.466+0000 7fd711391700 1 -- 172.21.15.7:0/46355253 wait complete. 2024-01-11T07:07:14.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:07:14.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.613+0000 7f9fd87b7700 1 Processor -- start 2024-01-11T07:07:14.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.613+0000 7f9fd87b7700 1 -- start start 2024-01-11T07:07:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd87b7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fd006ae90 0x7f9fd006b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd87b7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd006b840 con 0x7f9fd00afdc0 2024-01-11T07:07:14.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd87b7700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd006df10 con 0x7f9fd015bd10 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd87b7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd006eb20 con 0x7f9fd006ae90 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd5d52700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fd006ae90 0x7f9fd006b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd6553700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9fd00afdc0 0x7f9fd00b0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40300/0 (socket says 172.21.15.7:40300) 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd6553700 1 -- 172.21.15.7:0/4224953649 learned_addr learned my addr 172.21.15.7:0/4224953649 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd5d52700 1 -- 172.21.15.7:0/4224953649 >> v1:172.21.15.113:6789/0 conn(0x7f9fd015bd10 legacy=0x7f9fd015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd5d52700 1 -- 172.21.15.7:0/4224953649 >> v1:172.21.15.7:6789/0 conn(0x7f9fd00afdc0 legacy=0x7f9fd00b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd5d52700 1 -- 172.21.15.7:0/4224953649 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd006e1f0 con 0x7f9fd006ae90 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.614+0000 7f9fd5551700 1 -- 172.21.15.7:0/4224953649 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632624998 0 0) 0x7f9fd006b840 con 0x7f9fd00afdc0 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd5d52700 1 --2- 172.21.15.7:0/4224953649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd006ae90 0x7f9fd006b300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00098e0 tx=0x7f9fc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3113f6c3673f1cc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd5551700 1 -- 172.21.15.7:0/4224953649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0009e80 con 0x7f9fd006ae90 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd5551700 1 -- 172.21.15.7:0/4224953649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc000b820 con 0x7f9fd006ae90 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd5551700 1 -- 172.21.15.7:0/4224953649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000bba0 con 0x7f9fd006ae90 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd87b7700 1 -- 172.21.15.7:0/4224953649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd006ae90 msgr2=0x7f9fd006b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd87b7700 1 --2- 172.21.15.7:0/4224953649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd006ae90 0x7f9fd006b300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00098e0 tx=0x7f9fc00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd87b7700 1 -- 172.21.15.7:0/4224953649 shutdown_connections 2024-01-11T07:07:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd87b7700 1 --2- 172.21.15.7:0/4224953649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd006ae90 0x7f9fd006b300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd87b7700 1 -- 172.21.15.7:0/4224953649 >> 172.21.15.7:0/4224953649 conn(0x7f9fd00680e0 msgr2=0x7f9fd00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd87b7700 1 -- 172.21.15.7:0/4224953649 shutdown_connections 2024-01-11T07:07:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.615+0000 7f9fd87b7700 1 -- 172.21.15.7:0/4224953649 wait complete. 2024-01-11T07:07:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.616+0000 7f9fd87b7700 1 Processor -- start 2024-01-11T07:07:14.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd87b7700 1 -- start start 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd87b7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd006ae90 0x7f9fd015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd87b7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd00afdc0 0x7f9fd015dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd87b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd015bd10 0x7f9fd01686d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd87b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd016a710 con 0x7f9fd015bd10 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd87b7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0169860 con 0x7f9fd006ae90 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd87b7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0169b00 con 0x7f9fd00afdc0 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd6d54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd015bd10 0x7f9fd01686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd6d54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd015bd10 0x7f9fd01686d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41658/0 (socket says 172.21.15.7:41658) 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.617+0000 7f9fd6d54700 1 -- 172.21.15.7:0/2552249667 learned_addr learned my addr 172.21.15.7:0/2552249667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd6553700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd006ae90 0x7f9fd015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd5d52700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd00afdc0 0x7f9fd015dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd6d54700 1 -- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd00afdc0 msgr2=0x7f9fd015dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd6d54700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd00afdc0 0x7f9fd015dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd6d54700 1 -- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd006ae90 msgr2=0x7f9fd015d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd6d54700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd006ae90 0x7f9fd015d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd6d54700 1 -- 172.21.15.7:0/2552249667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc0009420 con 0x7f9fd015bd10 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd5d52700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd00afdc0 0x7f9fd015dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd6553700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd006ae90 0x7f9fd015d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd6d54700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd015bd10 0x7f9fd01686d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800bc30 tx=0x7f9fc8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fc67fc700 1 -- 172.21.15.7:0/2552249667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800c9b0 con 0x7f9fd015bd10 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fc67fc700 1 -- 172.21.15.7:0/2552249667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc800cb50 con 0x7f9fd015bd10 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd0169de0 con 0x7f9fd015bd10 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fc67fc700 1 -- 172.21.15.7:0/2552249667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8014410 con 0x7f9fd015bd10 2024-01-11T07:07:14.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.618+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd016a3a0 con 0x7f9fd015bd10 2024-01-11T07:07:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.619+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd0064a40 con 0x7f9fd015bd10 2024-01-11T07:07:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.620+0000 7f9fc67fc700 1 -- 172.21.15.7:0/2552249667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9fc8014800 con 0x7f9fd015bd10 2024-01-11T07:07:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.620+0000 7f9fc67fc700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.620+0000 7f9fc67fc700 1 -- 172.21.15.7:0/2552249667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9fc80845e0 con 0x7f9fd015bd10 2024-01-11T07:07:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.621+0000 7f9fd6553700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.621+0000 7f9fd6553700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fb4066f30 0x7f9fb40693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0011e0 tx=0x7f9fcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.625+0000 7f9fc67fc700 1 -- 172.21.15.7:0/2552249667 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fc804c5a0 con 0x7f9fd015bd10 2024-01-11T07:07:14.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.826+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fd004c3c0 con 0x7f9fd015bd10 2024-01-11T07:07:14.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.827+0000 7f9fc67fc700 1 -- 172.21.15.7:0/2552249667 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fc8051850 con 0x7f9fd015bd10 2024-01-11T07:07:14.827 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fb4066f30 msgr2=0x7f9fb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fb4066f30 0x7f9fb40693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0011e0 tx=0x7f9fcc009040 comp rx=0 tx=0).stop 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd015bd10 msgr2=0x7f9fd01686d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd015bd10 0x7f9fd01686d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800bc30 tx=0x7f9fc8009820 comp rx=0 tx=0).stop 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 shutdown_connections 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd015bd10 0x7f9fd01686d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd006ae90 0x7f9fd015d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 --2- 172.21.15.7:0/2552249667 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd00afdc0 0x7f9fd015dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 >> 172.21.15.7:0/2552249667 conn(0x7f9fd00680e0 msgr2=0x7f9fd015b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 shutdown_connections 2024-01-11T07:07:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.829+0000 7f9fd87b7700 1 -- 172.21.15.7:0/2552249667 wait complete. 2024-01-11T07:07:14.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:07:14.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.977+0000 7feef87ad700 1 Processor -- start 2024-01-11T07:07:14.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.977+0000 7feef87ad700 1 -- start start 2024-01-11T07:07:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef87ad700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 0x7feef0157e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef87ad700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feef00cd500 0x7feef00b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef87ad700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef00ca3d0 con 0x7feef00cf1b0 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef87ad700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef00b1790 con 0x7feef00cd500 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef87ad700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef00cd9a0 con 0x7feef0157a90 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef6d4a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feef00cd500 0x7feef00b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef6d4a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feef00cd500 0x7feef00b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41672/0 (socket says 172.21.15.7:41672) 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef6d4a700 1 -- 172.21.15.7:0/2113743178 learned_addr learned my addr 172.21.15.7:0/2113743178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef5d48700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feef00cf1b0 0x7feef00c8f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:40450/0 (socket says 172.21.15.7:40450) 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.978+0000 7feef6549700 1 --2- 172.21.15.7:0/2113743178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 0x7feef0157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.979+0000 7feef6d4a700 1 -- 172.21.15.7:0/2113743178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 msgr2=0x7feef0157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.979+0000 7feef6d4a700 1 --2- 172.21.15.7:0/2113743178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 0x7feef0157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.979+0000 7feef6d4a700 1 -- 172.21.15.7:0/2113743178 >> v1:172.21.15.113:6789/0 conn(0x7feef00cf1b0 legacy=0x7feef00c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.979+0000 7feef6d4a700 1 -- 172.21.15.7:0/2113743178 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef00cdc80 con 0x7feef00cd500 2024-01-11T07:07:14.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.979+0000 7feef6d4a700 1 --2- 172.21.15.7:0/2113743178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef00b21e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feee8009f50 tx=0x7feee8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9eea3b4dcd8c47 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.979+0000 7feef5547700 1 -- 172.21.15.7:0/2113743178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee800e070 con 0x7feef00cd500 2024-01-11T07:07:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.979+0000 7feef5547700 1 -- 172.21.15.7:0/2113743178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee800b9c0 con 0x7feef00cd500 2024-01-11T07:07:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.979+0000 7feef5547700 1 -- 172.21.15.7:0/2113743178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee80092e0 con 0x7feef00cd500 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.980+0000 7feef87ad700 1 -- 172.21.15.7:0/2113743178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 msgr2=0x7feef00b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.980+0000 7feef87ad700 1 --2- 172.21.15.7:0/2113743178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef00b21e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feee8009f50 tx=0x7feee8009840 comp rx=0 tx=0).stop 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.980+0000 7feef87ad700 1 -- 172.21.15.7:0/2113743178 shutdown_connections 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.980+0000 7feef87ad700 1 --2- 172.21.15.7:0/2113743178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef00b21e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.980+0000 7feef87ad700 1 --2- 172.21.15.7:0/2113743178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 0x7feef0157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.980+0000 7feef87ad700 1 -- 172.21.15.7:0/2113743178 >> 172.21.15.7:0/2113743178 conn(0x7feef0097a50 msgr2=0x7feef00ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.980+0000 7feef87ad700 1 -- 172.21.15.7:0/2113743178 shutdown_connections 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.980+0000 7feef87ad700 1 -- 172.21.15.7:0/2113743178 wait complete. 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.981+0000 7feef87ad700 1 Processor -- start 2024-01-11T07:07:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.981+0000 7feef87ad700 1 -- start start 2024-01-11T07:07:14.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef87ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef0154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef87ad700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef00cf1b0 0x7feef0169c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef87ad700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 0x7feef00684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef87ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c2dc0 con 0x7feef00cd500 2024-01-11T07:07:14.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef87ad700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c21b0 con 0x7feef00cf1b0 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef87ad700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c2490 con 0x7feef0157a90 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef6549700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef0154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef6549700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef0154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41676/0 (socket says 172.21.15.7:41676) 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.982+0000 7feef6549700 1 -- 172.21.15.7:0/3305047839 learned_addr learned my addr 172.21.15.7:0/3305047839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef5d48700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef00cf1b0 0x7feef0169c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef6d4a700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 0x7feef00684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef6549700 1 -- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 msgr2=0x7feef00684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef6549700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 0x7feef00684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef6549700 1 -- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef00cf1b0 msgr2=0x7feef0169c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef6549700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef00cf1b0 0x7feef0169c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef6549700 1 -- 172.21.15.7:0/3305047839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8009ae0 con 0x7feef00cd500 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef6549700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef0154fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feeec00ea00 tx=0x7feeec009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feee67fc700 1 -- 172.21.15.7:0/3305047839 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec00c6a0 con 0x7feef00cd500 2024-01-11T07:07:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feee67fc700 1 -- 172.21.15.7:0/3305047839 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec00c800 con 0x7feef00cd500 2024-01-11T07:07:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef00c2770 con 0x7feef00cd500 2024-01-11T07:07:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feee67fc700 1 -- 172.21.15.7:0/3305047839 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec00cb80 con 0x7feef00cd500 2024-01-11T07:07:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.983+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef0155500 con 0x7feef00cd500 2024-01-11T07:07:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.984+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef004c3c0 con 0x7feef00cd500 2024-01-11T07:07:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.985+0000 7feee67fc700 1 -- 172.21.15.7:0/3305047839 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feeec013470 con 0x7feef00cd500 2024-01-11T07:07:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.985+0000 7feee67fc700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feed4066f30 0x7feed40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.985+0000 7feee67fc700 1 -- 172.21.15.7:0/3305047839 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feeec084a60 con 0x7feef00cd500 2024-01-11T07:07:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.985+0000 7feef5d48700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feed4066f30 0x7feed40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.986+0000 7feef5d48700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feed4066f30 0x7feed40693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feef015d2e0 tx=0x7feee0007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:14.990+0000 7feee67fc700 1 -- 172.21.15.7:0/3305047839 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeec051cd0 con 0x7feef00cd500 2024-01-11T07:07:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.190+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feef00b19e0 con 0x7feef00cd500 2024-01-11T07:07:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.191+0000 7feee67fc700 1 -- 172.21.15.7:0/3305047839 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7feeec0210a0 con 0x7feef00cd500 2024-01-11T07:07:15.190 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feed4066f30 msgr2=0x7feed40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feed4066f30 0x7feed40693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feef015d2e0 tx=0x7feee0007eb0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 msgr2=0x7feef0154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef0154fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feeec00ea00 tx=0x7feeec009120 comp rx=0 tx=0).stop 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 shutdown_connections 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feed4066f30 0x7feed40693e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feef0157a90 0x7feef00684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef00cd500 0x7feef0154fc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 --2- 172.21.15.7:0/3305047839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef00cf1b0 0x7feef0169c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 >> 172.21.15.7:0/3305047839 conn(0x7feef0097a50 msgr2=0x7feef00b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 shutdown_connections 2024-01-11T07:07:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.193+0000 7feef87ad700 1 -- 172.21.15.7:0/3305047839 wait complete. 2024-01-11T07:07:15.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.339+0000 7f1a2721d700 1 Processor -- start 2024-01-11T07:07:15.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.339+0000 7f1a2721d700 1 -- start start 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a2721d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a20159c90 0x7f1a200b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a2721d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a200ce9b0 con 0x7f1a2015a060 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a2721d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a200cf530 con 0x7f1a20069320 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a2721d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a200cf9c0 con 0x7f1a20159c90 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a24fb9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a20159c90 0x7f1a200b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a1ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1a2015a060 0x7f1a200b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40312/0 (socket says 172.21.15.7:40312) 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a24fb9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a20159c90 0x7f1a200b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41684/0 (socket says 172.21.15.7:41684) 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a1ffff700 1 -- 172.21.15.7:0/3319980265 learned_addr learned my addr 172.21.15.7:0/3319980265 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a24fb9700 1 -- 172.21.15.7:0/3319980265 >> v1:172.21.15.113:6789/0 conn(0x7f1a20069320 legacy=0x7f1a200b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a24fb9700 1 -- 172.21.15.7:0/3319980265 >> v1:172.21.15.7:6789/0 conn(0x7f1a2015a060 legacy=0x7f1a200b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a24fb9700 1 -- 172.21.15.7:0/3319980265 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a200cec00 con 0x7f1a20159c90 2024-01-11T07:07:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a24fb9700 1 --2- 172.21.15.7:0/3319980265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20159c90 0x7f1a200b1820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a100098e0 tx=0x7f1a100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd665dbb2191e0c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.340+0000 7f1a1f7fe700 1 -- 172.21.15.7:0/3319980265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10009e80 con 0x7f1a20159c90 2024-01-11T07:07:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a1f7fe700 1 -- 172.21.15.7:0/3319980265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1000b760 con 0x7f1a20159c90 2024-01-11T07:07:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a1f7fe700 1 -- 172.21.15.7:0/3319980265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000bae0 con 0x7f1a20159c90 2024-01-11T07:07:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a2721d700 1 -- 172.21.15.7:0/3319980265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20159c90 msgr2=0x7f1a200b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a2721d700 1 --2- 172.21.15.7:0/3319980265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20159c90 0x7f1a200b1820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a100098e0 tx=0x7f1a100091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a2721d700 1 -- 172.21.15.7:0/3319980265 shutdown_connections 2024-01-11T07:07:15.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a2721d700 1 --2- 172.21.15.7:0/3319980265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20159c90 0x7f1a200b1820 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a2721d700 1 -- 172.21.15.7:0/3319980265 >> 172.21.15.7:0/3319980265 conn(0x7f1a20071100 msgr2=0x7f1a2006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:15.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a2721d700 1 -- 172.21.15.7:0/3319980265 shutdown_connections 2024-01-11T07:07:15.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.341+0000 7f1a2721d700 1 -- 172.21.15.7:0/3319980265 wait complete. 2024-01-11T07:07:15.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.342+0000 7f1a2721d700 1 Processor -- start 2024-01-11T07:07:15.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.343+0000 7f1a2721d700 1 -- start start 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.343+0000 7f1a2721d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20069320 0x7f1a2015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.343+0000 7f1a2721d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a20159c90 0x7f1a2016d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.343+0000 7f1a2721d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a2015a060 0x7f1a20154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.343+0000 7f1a2721d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2016e890 con 0x7f1a20069320 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.343+0000 7f1a2721d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2016e2d0 con 0x7f1a2015a060 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.343+0000 7f1a2721d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2016e5b0 con 0x7f1a20159c90 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.343+0000 7f1a24fb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20069320 0x7f1a2015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a24fb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20069320 0x7f1a2015bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41688/0 (socket says 172.21.15.7:41688) 2024-01-11T07:07:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a24fb9700 1 -- 172.21.15.7:0/607404730 learned_addr learned my addr 172.21.15.7:0/607404730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a1ffff700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a20159c90 0x7f1a2016d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a257ba700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a2015a060 0x7f1a20154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a24fb9700 1 -- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a20159c90 msgr2=0x7f1a2016d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a24fb9700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a20159c90 0x7f1a2016d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a24fb9700 1 -- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a2015a060 msgr2=0x7f1a20154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a24fb9700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a2015a060 0x7f1a20154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a24fb9700 1 -- 172.21.15.7:0/607404730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10009420 con 0x7f1a20069320 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a24fb9700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20069320 0x7f1a2015bfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1a100093f0 tx=0x7f1a10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a1cff9700 1 -- 172.21.15.7:0/607404730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000bee0 con 0x7f1a20069320 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a1cff9700 1 -- 172.21.15.7:0/607404730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1000bae0 con 0x7f1a20069320 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a200af6e0 con 0x7f1a20069320 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a1cff9700 1 -- 172.21.15.7:0/607404730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1001c6b0 con 0x7f1a20069320 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.344+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a200c8440 con 0x7f1a20069320 2024-01-11T07:07:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.345+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a200afb80 con 0x7f1a20069320 2024-01-11T07:07:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.346+0000 7f1a1cff9700 1 -- 172.21.15.7:0/607404730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1a1001caa0 con 0x7f1a20069320 2024-01-11T07:07:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.346+0000 7f1a1cff9700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a00066f30 0x7f1a000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.346+0000 7f1a1cff9700 1 -- 172.21.15.7:0/607404730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a100868b0 con 0x7f1a20069320 2024-01-11T07:07:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.347+0000 7f1a1ffff700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a00066f30 0x7f1a000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.347+0000 7f1a1ffff700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a00066f30 0x7f1a000693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a20159010 tx=0x7f1a08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:15.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.351+0000 7f1a1cff9700 1 -- 172.21.15.7:0/607404730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a10053a80 con 0x7f1a20069320 2024-01-11T07:07:15.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.605+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a200af540 con 0x7f1a20069320 2024-01-11T07:07:15.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.606+0000 7f1a1cff9700 1 -- 172.21.15.7:0/607404730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f1a200af540 con 0x7f1a20069320 2024-01-11T07:07:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:15.606 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:06:58.385324+0000","modified":"2024-01-11T07:07:02.162269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-11T07:07:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a00066f30 msgr2=0x7f1a000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a00066f30 0x7f1a000693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a20159010 tx=0x7f1a08009040 comp rx=0 tx=0).stop 2024-01-11T07:07:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20069320 msgr2=0x7f1a2015bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20069320 0x7f1a2015bfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1a100093f0 tx=0x7f1a10016000 comp rx=0 tx=0).stop 2024-01-11T07:07:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 shutdown_connections 2024-01-11T07:07:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1a00066f30 0x7f1a000693e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a2015a060 0x7f1a20154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a20069320 0x7f1a2015bfa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 --2- 172.21.15.7:0/607404730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a20159c90 0x7f1a2016d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 >> 172.21.15.7:0/607404730 conn(0x7f1a20071100 msgr2=0x7f1a2015f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 shutdown_connections 2024-01-11T07:07:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.608+0000 7f1a2721d700 1 -- 172.21.15.7:0/607404730 wait complete. 2024-01-11T07:07:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T07:07:15.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:15.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.752+0000 7fd4c7a3a700 1 Processor -- start 2024-01-11T07:07:15.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.753+0000 7fd4c7a3a700 1 -- start start 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c7a3a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adb40 0x7fd4c00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c7a3a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4c00b0b40 0x7fd4c00cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c7a3a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c00cc2a0 con 0x7fd4c00adf10 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c7a3a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c00b2930 con 0x7fd4c00b0b40 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c7a3a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00b3b50 con 0x7fd4c00adb40 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c5fd7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4c00b0b40 0x7fd4c00cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c5fd7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4c00b0b40 0x7fd4c00cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41692/0 (socket says 172.21.15.7:41692) 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c5fd7700 1 -- 172.21.15.7:0/3337157535 learned_addr learned my addr 172.21.15.7:0/3337157535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c57d6700 1 --2- 172.21.15.7:0/3337157535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adb40 0x7fd4c00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c5fd7700 1 -- 172.21.15.7:0/3337157535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adb40 msgr2=0x7fd4c00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c5fd7700 1 --2- 172.21.15.7:0/3337157535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adb40 0x7fd4c00b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c5fd7700 1 -- 172.21.15.7:0/3337157535 >> v1:172.21.15.113:6789/0 conn(0x7fd4c00adf10 legacy=0x7fd4c00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c5fd7700 1 -- 172.21.15.7:0/3337157535 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00b3e30 con 0x7fd4c00b0b40 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c57d6700 1 --2- 172.21.15.7:0/3337157535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adb40 0x7fd4c00b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4c5fd7700 1 --2- 172.21.15.7:0/3337157535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00b0b40 0x7fd4c00cca60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009f20 tx=0x7fd4b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0adaced51b3f583 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.754+0000 7fd4b7fff700 1 -- 172.21.15.7:0/3337157535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800e070 con 0x7fd4c00b0b40 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4b7fff700 1 -- 172.21.15.7:0/3337157535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b800ba80 con 0x7fd4c00b0b40 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4b7fff700 1 -- 172.21.15.7:0/3337157535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80092e0 con 0x7fd4c00b0b40 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/3337157535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00b0b40 msgr2=0x7fd4c00cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/3337157535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00b0b40 0x7fd4c00cca60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009f20 tx=0x7fd4b8009840 comp rx=0 tx=0).stop 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/3337157535 shutdown_connections 2024-01-11T07:07:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/3337157535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00b0b40 0x7fd4c00cca60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/3337157535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adb40 0x7fd4c00b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/3337157535 >> 172.21.15.7:0/3337157535 conn(0x7fd4c0071160 msgr2=0x7fd4c006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:15.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/3337157535 shutdown_connections 2024-01-11T07:07:15.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.755+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/3337157535 wait complete. 2024-01-11T07:07:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.756+0000 7fd4c7a3a700 1 Processor -- start 2024-01-11T07:07:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.756+0000 7fd4c7a3a700 1 -- start start 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.757+0000 7fd4c7a3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00adb40 0x7fd4c016af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.757+0000 7fd4c7a3a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adf10 0x7fd4c00ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.757+0000 7fd4c7a3a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b0b40 0x7fd4c015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.757+0000 7fd4c7a3a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c016db40 con 0x7fd4c00adb40 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.757+0000 7fd4c7a3a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c016e9a0 con 0x7fd4c00b0b40 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.757+0000 7fd4c7a3a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c016b4b0 con 0x7fd4c00adf10 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00adb40 0x7fd4c016af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00adb40 0x7fd4c016af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41698/0 (socket says 172.21.15.7:41698) 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 -- 172.21.15.7:0/1966291745 learned_addr learned my addr 172.21.15.7:0/1966291745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c5fd7700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b0b40 0x7fd4c015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c4fd5700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adf10 0x7fd4c00ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 -- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adf10 msgr2=0x7fd4c00ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adf10 0x7fd4c00ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 -- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b0b40 msgr2=0x7fd4c015c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b0b40 0x7fd4c015c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 -- 172.21.15.7:0/1966291745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b8009ae0 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c57d6700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00adb40 0x7fd4c016af70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00e8d0 tx=0x7fd4bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4b57fa700 1 -- 172.21.15.7:0/1966291745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0093b0 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4b57fa700 1 -- 172.21.15.7:0/1966291745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc013410 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c00ad3d0 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4b57fa700 1 -- 172.21.15.7:0/1966291745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0137b0 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.758+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c016e070 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.760+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c006e9a0 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.760+0000 7fd4b57fa700 1 -- 172.21.15.7:0/1966291745 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd4bc013be0 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.760+0000 7fd4b57fa700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd4a4066f30 0x7fd4a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.761+0000 7fd4b57fa700 1 -- 172.21.15.7:0/1966291745 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd4bc0844d0 con 0x7fd4c00adb40 2024-01-11T07:07:15.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.761+0000 7fd4c4fd5700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd4a4066f30 0x7fd4a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.761+0000 7fd4c4fd5700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd4a4066f30 0x7fd4a40693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0157f80 tx=0x7fd4b0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.765+0000 7fd4b57fa700 1 -- 172.21.15.7:0/1966291745 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4bc051740 con 0x7fd4c00adb40 2024-01-11T07:07:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.971+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4c004c3c0 con 0x7fd4c00adb40 2024-01-11T07:07:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.972+0000 7fd4b57fa700 1 -- 172.21.15.7:0/1966291745 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fd4bc054d60 con 0x7fd4c00adb40 2024-01-11T07:07:15.971 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:15.971 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd4a4066f30 msgr2=0x7fd4a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd4a4066f30 0x7fd4a40693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0157f80 tx=0x7fd4b0007ec0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00adb40 msgr2=0x7fd4c016af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00adb40 0x7fd4c016af70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00e8d0 tx=0x7fd4bc008040 comp rx=0 tx=0).stop 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 shutdown_connections 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd4a4066f30 0x7fd4a40693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4c00b0b40 0x7fd4c015c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4c00adb40 0x7fd4c016af70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 --2- 172.21.15.7:0/1966291745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4c00adf10 0x7fd4c00ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 >> 172.21.15.7:0/1966291745 conn(0x7fd4c0071160 msgr2=0x7fd4c0156650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 shutdown_connections 2024-01-11T07:07:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:15.974+0000 7fd4c7a3a700 1 -- 172.21.15.7:0/1966291745 wait complete. 2024-01-11T07:07:15.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:06:56.038804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:06:57.365958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:07:15.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:07:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.122+0000 7f32cd75c700 1 Processor -- start 2024-01-11T07:07:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.122+0000 7f32cd75c700 1 -- start start 2024-01-11T07:07:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32cd75c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f32c8063480 0x7f32c815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32cd75c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c806c1c0 0x7f32c8157680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32cd75c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c80ca180 con 0x7f32c8063850 2024-01-11T07:07:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32cd75c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c80c9ea0 con 0x7f32c8063480 2024-01-11T07:07:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32cd75c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c815b9d0 con 0x7f32c806c1c0 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32be7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f32c8063850 0x7f32c80c9650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40316/0 (socket says 172.21.15.7:40316) 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32be7fc700 1 -- 172.21.15.7:0/2954711020 learned_addr learned my addr 172.21.15.7:0/2954711020 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32c77fe700 1 --2- 172.21.15.7:0/2954711020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c806c1c0 0x7f32c8157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32c6ffd700 1 --2- 172.21.15.7:0/2954711020 >> v2:172.21.15.113:3300/0 conn(0x7f32c8063480 0x7f32c815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32c67fc700 1 -- 172.21.15.7:0/2954711020 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211632105 0 0) 0x7f32c80ca180 con 0x7f32c8063850 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.123+0000 7f32c67fc700 1 -- 172.21.15.7:0/2954711020 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b4003570 con 0x7f32c8063850 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c77fe700 1 -- 172.21.15.7:0/2954711020 >> v2:172.21.15.113:3300/0 conn(0x7f32c8063480 msgr2=0x7f32c815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c77fe700 1 --2- 172.21.15.7:0/2954711020 >> v2:172.21.15.113:3300/0 conn(0x7f32c8063480 0x7f32c815c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c77fe700 1 -- 172.21.15.7:0/2954711020 >> v1:172.21.15.7:6789/0 conn(0x7f32c8063850 legacy=0x7f32c80c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c67fc700 1 -- 172.21.15.7:0/2954711020 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1666415406 0 0) 0x7f32b4003570 con 0x7f32c8063850 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c77fe700 1 -- 172.21.15.7:0/2954711020 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c815bcb0 con 0x7f32c806c1c0 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c77fe700 1 --2- 172.21.15.7:0/2954711020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c806c1c0 0x7f32c8157680 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009e60 tx=0x7f32c0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ce9f23c72c47a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c67fc700 1 -- 172.21.15.7:0/2954711020 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000e070 con 0x7f32c806c1c0 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c67fc700 1 -- 172.21.15.7:0/2954711020 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000b7a0 con 0x7f32c806c1c0 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.124+0000 7f32c67fc700 1 -- 172.21.15.7:0/2954711020 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0001470 con 0x7f32c806c1c0 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.125+0000 7f32cd75c700 1 -- 172.21.15.7:0/2954711020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c806c1c0 msgr2=0x7f32c8157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.125+0000 7f32cd75c700 1 --2- 172.21.15.7:0/2954711020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c806c1c0 0x7f32c8157680 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009e60 tx=0x7f32c0009780 comp rx=0 tx=0).stop 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.125+0000 7f32cd75c700 1 -- 172.21.15.7:0/2954711020 shutdown_connections 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.125+0000 7f32cd75c700 1 --2- 172.21.15.7:0/2954711020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c806c1c0 0x7f32c8157680 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.125+0000 7f32cd75c700 1 --2- 172.21.15.7:0/2954711020 >> v2:172.21.15.113:3300/0 conn(0x7f32c8063480 0x7f32c815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.125+0000 7f32cd75c700 1 -- 172.21.15.7:0/2954711020 >> 172.21.15.7:0/2954711020 conn(0x7f32c80cd8e0 msgr2=0x7f32c80cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.125+0000 7f32cd75c700 1 -- 172.21.15.7:0/2954711020 shutdown_connections 2024-01-11T07:07:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.125+0000 7f32cd75c700 1 -- 172.21.15.7:0/2954711020 wait complete. 2024-01-11T07:07:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.126+0000 7f32cd75c700 1 Processor -- start 2024-01-11T07:07:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.126+0000 7f32cd75c700 1 -- start start 2024-01-11T07:07:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32cd75c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c8063480 0x7f32c8150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32cd75c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32c8063850 0x7f32c80cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32cd75c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32c806c1c0 0x7f32c80cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32cd75c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80c4fe0 con 0x7f32c8063850 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32cd75c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80ab4a0 con 0x7f32c806c1c0 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32cd75c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c80cca30 con 0x7f32c8063480 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32be7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32c8063850 0x7f32c80cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32be7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32c8063850 0x7f32c80cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41704/0 (socket says 172.21.15.7:41704) 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32be7fc700 1 -- 172.21.15.7:0/4281007149 learned_addr learned my addr 172.21.15.7:0/4281007149 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32c6ffd700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c8063480 0x7f32c8150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32c77fe700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32c806c1c0 0x7f32c80cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32be7fc700 1 -- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c8063480 msgr2=0x7f32c8150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32be7fc700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c8063480 0x7f32c8150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32be7fc700 1 -- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32c806c1c0 msgr2=0x7f32c80cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32be7fc700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32c806c1c0 0x7f32c80cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.127+0000 7f32be7fc700 1 -- 172.21.15.7:0/4281007149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c0009a20 con 0x7f32c8063850 2024-01-11T07:07:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.128+0000 7f32be7fc700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32c8063850 0x7f32c80cbec0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f32b000cb90 tx=0x7f32b0001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.128+0000 7f32bf7fe700 1 -- 172.21.15.7:0/4281007149 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0007360 con 0x7f32c8063850 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.128+0000 7f32bf7fe700 1 -- 172.21.15.7:0/4281007149 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b00074e0 con 0x7f32c8063850 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.128+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c80ccd10 con 0x7f32c8063850 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.128+0000 7f32bf7fe700 1 -- 172.21.15.7:0/4281007149 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0004830 con 0x7f32c8063850 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.128+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c80cd2d0 con 0x7f32c8063850 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.129+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c80c4db0 con 0x7f32c8063850 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.129+0000 7f32bf7fe700 1 -- 172.21.15.7:0/4281007149 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f32b0004bc0 con 0x7f32c8063850 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.130+0000 7f32bf7fe700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.130+0000 7f32bf7fe700 1 -- 172.21.15.7:0/4281007149 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32b0083dd0 con 0x7f32c8063850 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.130+0000 7f32c6ffd700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.130+0000 7f32c6ffd700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f32a8066f30 0x7f32a80693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009db0 tx=0x7f32b80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.134+0000 7f32bf7fe700 1 -- 172.21.15.7:0/4281007149 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32c80c4db0 con 0x7f32c8063850 2024-01-11T07:07:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.338+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32c80c4db0 con 0x7f32c8063850 2024-01-11T07:07:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.338+0000 7f32bf7fe700 1 -- 172.21.15.7:0/4281007149 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32c80c4db0 con 0x7f32c8063850 2024-01-11T07:07:16.338 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:07:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f32a8066f30 msgr2=0x7f32a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f32a8066f30 0x7f32a80693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009db0 tx=0x7f32b80091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32c8063850 msgr2=0x7f32c80cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32c8063850 0x7f32c80cbec0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f32b000cb90 tx=0x7f32b0001b50 comp rx=0 tx=0).stop 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 shutdown_connections 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f32a8066f30 0x7f32a80693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32c806c1c0 0x7f32c80cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32c8063480 0x7f32c8150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 --2- 172.21.15.7:0/4281007149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32c8063850 0x7f32c80cbec0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.340+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 >> 172.21.15.7:0/4281007149 conn(0x7f32c80cd8e0 msgr2=0x7f32c80c35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.341+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 shutdown_connections 2024-01-11T07:07:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.341+0000 7f32cd75c700 1 -- 172.21.15.7:0/4281007149 wait complete. 2024-01-11T07:07:16.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:07:16.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.489+0000 7f0720d36700 1 Processor -- start 2024-01-11T07:07:16.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.489+0000 7f0720d36700 1 -- start start 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.489+0000 7f0720d36700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f071c157ae0 0x7f071c0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.489+0000 7f0720d36700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f071c06e5a0 0x7f071c157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.489+0000 7f0720d36700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c06ea80 con 0x7f071c063230 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.489+0000 7f0720d36700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c06bab0 con 0x7f071c157ae0 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.489+0000 7f0720d36700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c06b340 con 0x7f071c06e5a0 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f071a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f071c157ae0 0x7f071c0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f071a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f071c157ae0 0x7f071c0b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41714/0 (socket says 172.21.15.7:41714) 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f071a59c700 1 -- 172.21.15.7:0/1398274577 learned_addr learned my addr 172.21.15.7:0/1398274577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f0711d9b700 1 --2- 172.21.15.7:0/1398274577 >> v2:172.21.15.113:3300/0 conn(0x7f071c06e5a0 0x7f071c157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f071a59c700 1 -- 172.21.15.7:0/1398274577 >> v2:172.21.15.113:3300/0 conn(0x7f071c06e5a0 msgr2=0x7f071c157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f071a59c700 1 --2- 172.21.15.7:0/1398274577 >> v2:172.21.15.113:3300/0 conn(0x7f071c06e5a0 0x7f071c157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f071a59c700 1 -- 172.21.15.7:0/1398274577 >> v1:172.21.15.113:6789/0 conn(0x7f071c063230 legacy=0x7f071c069bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:16.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f071a59c700 1 -- 172.21.15.7:0/1398274577 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c06b620 con 0x7f071c157ae0 2024-01-11T07:07:16.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f071a59c700 1 --2- 172.21.15.7:0/1398274577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0b0760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f071c06f080 tx=0x7f070c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d83562f7d130138 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f0719d9b700 1 -- 172.21.15.7:0/1398274577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00b820 con 0x7f071c157ae0 2024-01-11T07:07:16.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f0719d9b700 1 -- 172.21.15.7:0/1398274577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c00b980 con 0x7f071c157ae0 2024-01-11T07:07:16.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.490+0000 7f0719d9b700 1 -- 172.21.15.7:0/1398274577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00bd20 con 0x7f071c157ae0 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.491+0000 7f0720d36700 1 -- 172.21.15.7:0/1398274577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 msgr2=0x7f071c0b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.491+0000 7f0720d36700 1 --2- 172.21.15.7:0/1398274577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0b0760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f071c06f080 tx=0x7f070c009250 comp rx=0 tx=0).stop 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.491+0000 7f0720d36700 1 -- 172.21.15.7:0/1398274577 shutdown_connections 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.491+0000 7f0720d36700 1 --2- 172.21.15.7:0/1398274577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0b0760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.491+0000 7f0720d36700 1 --2- 172.21.15.7:0/1398274577 >> v2:172.21.15.113:3300/0 conn(0x7f071c06e5a0 0x7f071c157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.491+0000 7f0720d36700 1 -- 172.21.15.7:0/1398274577 >> 172.21.15.7:0/1398274577 conn(0x7f071c0680e0 msgr2=0x7f071c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.491+0000 7f0720d36700 1 -- 172.21.15.7:0/1398274577 shutdown_connections 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.491+0000 7f0720d36700 1 -- 172.21.15.7:0/1398274577 wait complete. 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.492+0000 7f0720d36700 1 Processor -- start 2024-01-11T07:07:16.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.492+0000 7f0720d36700 1 -- start start 2024-01-11T07:07:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f0720d36700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c063230 0x7f071c14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f0720d36700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c06e5a0 0x7f071c0c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f0720d36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f0720d36700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c16c730 con 0x7f071c157ae0 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f0720d36700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c16bb20 con 0x7f071c06e5a0 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f0720d36700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c16be00 con 0x7f071c063230 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f071ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f071ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41718/0 (socket says 172.21.15.7:41718) 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f071ad9d700 1 -- 172.21.15.7:0/2586186905 learned_addr learned my addr 172.21.15.7:0/2586186905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f071a59c700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c063230 0x7f071c14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f0711d9b700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c06e5a0 0x7f071c0c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.493+0000 7f071ad9d700 1 -- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c063230 msgr2=0x7f071c14f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f071ad9d700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c063230 0x7f071c14f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f071ad9d700 1 -- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c06e5a0 msgr2=0x7f071c0c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f071ad9d700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c06e5a0 0x7f071c0c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f071ad9d700 1 -- 172.21.15.7:0/2586186905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c0094f0 con 0x7f071c157ae0 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f071ad9d700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0c3a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f071400c930 tx=0x7f07140026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f0712ffd700 1 -- 172.21.15.7:0/2586186905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714012630 con 0x7f071c157ae0 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f0712ffd700 1 -- 172.21.15.7:0/2586186905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07140127d0 con 0x7f071c157ae0 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c16c0e0 con 0x7f071c157ae0 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f0712ffd700 1 -- 172.21.15.7:0/2586186905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714012b90 con 0x7f071c157ae0 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.494+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c14fc20 con 0x7f071c157ae0 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.495+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c064a40 con 0x7f071c157ae0 2024-01-11T07:07:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.495+0000 7f0712ffd700 1 -- 172.21.15.7:0/2586186905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f071401a3b0 con 0x7f071c157ae0 2024-01-11T07:07:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.496+0000 7f0712ffd700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f06fc066f30 0x7f06fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.496+0000 7f0712ffd700 1 -- 172.21.15.7:0/2586186905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0714084180 con 0x7f071c157ae0 2024-01-11T07:07:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.496+0000 7f071a59c700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f06fc066f30 0x7f06fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.497+0000 7f071a59c700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f06fc066f30 0x7f06fc0693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f071c06f080 tx=0x7f070c016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.500+0000 7f0712ffd700 1 -- 172.21.15.7:0/2586186905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0714054e80 con 0x7f071c157ae0 2024-01-11T07:07:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.705+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f071c04c3c0 con 0x7f071c157ae0 2024-01-11T07:07:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.706+0000 7f0712ffd700 1 -- 172.21.15.7:0/2586186905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0714054a10 con 0x7f071c157ae0 2024-01-11T07:07:16.705 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:07:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f06fc066f30 msgr2=0x7f06fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f06fc066f30 0x7f06fc0693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f071c06f080 tx=0x7f070c016000 comp rx=0 tx=0).stop 2024-01-11T07:07:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 msgr2=0x7f071c0c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0c3a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f071400c930 tx=0x7f07140026e0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 shutdown_connections 2024-01-11T07:07:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f06fc066f30 0x7f06fc0693e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071c157ae0 0x7f071c0c3a60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f071c063230 0x7f071c14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 --2- 172.21.15.7:0/2586186905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f071c06e5a0 0x7f071c0c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 >> 172.21.15.7:0/2586186905 conn(0x7f071c0680e0 msgr2=0x7f071c156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 shutdown_connections 2024-01-11T07:07:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.708+0000 7f0720d36700 1 -- 172.21.15.7:0/2586186905 wait complete. 2024-01-11T07:07:16.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:07:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.856+0000 7f45b7d45700 1 Processor -- start 2024-01-11T07:07:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.856+0000 7f45b7d45700 1 -- start start 2024-01-11T07:07:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b7d45700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45b015e540 0x7f45b00cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b7d45700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45b00ccc40 0x7f45b0068620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b7d45700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b00c9980 con 0x7f45b00cf370 2024-01-11T07:07:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b7d45700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b015e910 con 0x7f45b00ccc40 2024-01-11T07:07:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b7d45700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b0068f70 con 0x7f45b015e540 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b62e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45b00ccc40 0x7f45b0068620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b62e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45b00ccc40 0x7f45b0068620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41728/0 (socket says 172.21.15.7:41728) 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b62e2700 1 -- 172.21.15.7:0/2288599194 learned_addr learned my addr 172.21.15.7:0/2288599194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b5ae1700 1 --2- 172.21.15.7:0/2288599194 >> v2:172.21.15.113:3300/0 conn(0x7f45b015e540 0x7f45b00cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b62e2700 1 -- 172.21.15.7:0/2288599194 >> v2:172.21.15.113:3300/0 conn(0x7f45b015e540 msgr2=0x7f45b00cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b62e2700 1 --2- 172.21.15.7:0/2288599194 >> v2:172.21.15.113:3300/0 conn(0x7f45b015e540 0x7f45b00cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b62e2700 1 -- 172.21.15.7:0/2288599194 >> v1:172.21.15.113:6789/0 conn(0x7f45b00cf370 legacy=0x7f45b00c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b62e2700 1 -- 172.21.15.7:0/2288599194 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00c9090 con 0x7f45b00ccc40 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.857+0000 7f45b62e2700 1 --2- 172.21.15.7:0/2288599194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00ccc40 0x7f45b0068620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45b006faf0 tx=0x7f45a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e7a6cd8952af3b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b4adf700 1 -- 172.21.15.7:0/2288599194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a800e070 con 0x7f45b00ccc40 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b4adf700 1 -- 172.21.15.7:0/2288599194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a800ba20 con 0x7f45b00ccc40 2024-01-11T07:07:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b4adf700 1 -- 172.21.15.7:0/2288599194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a80092e0 con 0x7f45b00ccc40 2024-01-11T07:07:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b7d45700 1 -- 172.21.15.7:0/2288599194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00ccc40 msgr2=0x7f45b0068620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b7d45700 1 --2- 172.21.15.7:0/2288599194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00ccc40 0x7f45b0068620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45b006faf0 tx=0x7f45a8009840 comp rx=0 tx=0).stop 2024-01-11T07:07:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b7d45700 1 -- 172.21.15.7:0/2288599194 shutdown_connections 2024-01-11T07:07:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b7d45700 1 --2- 172.21.15.7:0/2288599194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00ccc40 0x7f45b0068620 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b7d45700 1 --2- 172.21.15.7:0/2288599194 >> v2:172.21.15.113:3300/0 conn(0x7f45b015e540 0x7f45b00cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b7d45700 1 -- 172.21.15.7:0/2288599194 >> 172.21.15.7:0/2288599194 conn(0x7f45b0097a50 msgr2=0x7f45b00ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:16.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b7d45700 1 -- 172.21.15.7:0/2288599194 shutdown_connections 2024-01-11T07:07:16.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.858+0000 7f45b7d45700 1 -- 172.21.15.7:0/2288599194 wait complete. 2024-01-11T07:07:16.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.860+0000 7f45b7d45700 1 Processor -- start 2024-01-11T07:07:16.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.860+0000 7f45b7d45700 1 -- start start 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b7d45700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b00ccc40 0x7f45b0161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b7d45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00cf370 0x7f45b015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b7d45700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b015e540 0x7f45b00c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b7d45700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0161570 con 0x7f45b00cf370 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b7d45700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0171cb0 con 0x7f45b00ccc40 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b7d45700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0160d30 con 0x7f45b015e540 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b52e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00cf370 0x7f45b015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b52e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00cf370 0x7f45b015b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41742/0 (socket says 172.21.15.7:41742) 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b52e0700 1 -- 172.21.15.7:0/1144169435 learned_addr learned my addr 172.21.15.7:0/1144169435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b62e2700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b015e540 0x7f45b00c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b5ae1700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b00ccc40 0x7f45b0161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b52e0700 1 -- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b015e540 msgr2=0x7f45b00c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.861+0000 7f45b52e0700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b015e540 0x7f45b00c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45b52e0700 1 -- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b00ccc40 msgr2=0x7f45b0161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45b52e0700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b00ccc40 0x7f45b0161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45b52e0700 1 -- 172.21.15.7:0/1144169435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a8009ae0 con 0x7f45b00cf370 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45b52e0700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00cf370 0x7f45b015b0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f45a000c5c0 tx=0x7f45a0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45a5ffb700 1 -- 172.21.15.7:0/1144169435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a0011590 con 0x7f45b00cf370 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45a5ffb700 1 -- 172.21.15.7:0/1144169435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a0011710 con 0x7f45b00cf370 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b015b600 con 0x7f45b00cf370 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45a5ffb700 1 -- 172.21.15.7:0/1144169435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a0011ad0 con 0x7f45b00cf370 2024-01-11T07:07:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.862+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b015f260 con 0x7f45b00cf370 2024-01-11T07:07:16.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.863+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b0063740 con 0x7f45b00cf370 2024-01-11T07:07:16.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.864+0000 7f45a5ffb700 1 -- 172.21.15.7:0/1144169435 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f45a001b3b0 con 0x7f45b00cf370 2024-01-11T07:07:16.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.864+0000 7f45a5ffb700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4594066f30 0x7f45940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:16.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.864+0000 7f45a5ffb700 1 -- 172.21.15.7:0/1144169435 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f45a0083860 con 0x7f45b00cf370 2024-01-11T07:07:16.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.864+0000 7f45b5ae1700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4594066f30 0x7f45940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:16.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.865+0000 7f45b5ae1700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4594066f30 0x7f45940693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009d00 tx=0x7f45ac009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:16.868+0000 7f45a5ffb700 1 -- 172.21.15.7:0/1144169435 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45a0050ad0 con 0x7f45b00cf370 2024-01-11T07:07:17.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.076+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45b004c3c0 con 0x7f45b00cf370 2024-01-11T07:07:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.076+0000 7f45a5ffb700 1 -- 172.21.15.7:0/1144169435 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45a0017070 con 0x7f45b00cf370 2024-01-11T07:07:17.077 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:07:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4594066f30 msgr2=0x7f45940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4594066f30 0x7f45940693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009d00 tx=0x7f45ac009120 comp rx=0 tx=0).stop 2024-01-11T07:07:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00cf370 msgr2=0x7f45b015b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00cf370 0x7f45b015b0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f45a000c5c0 tx=0x7f45a0007f50 comp rx=0 tx=0).stop 2024-01-11T07:07:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 shutdown_connections 2024-01-11T07:07:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4594066f30 0x7f45940693e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b015e540 0x7f45b00c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b00ccc40 0x7f45b0161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 --2- 172.21.15.7:0/1144169435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b00cf370 0x7f45b015b0c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 >> 172.21.15.7:0/1144169435 conn(0x7f45b0097a50 msgr2=0x7f45b015c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.078+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 shutdown_connections 2024-01-11T07:07:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.079+0000 7f45b7d45700 1 -- 172.21.15.7:0/1144169435 wait complete. 2024-01-11T07:07:17.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.223+0000 7f93156ff700 1 Processor -- start 2024-01-11T07:07:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.223+0000 7f93156ff700 1 -- start start 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f93156ff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100cd6d0 0x7f93100cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f93156ff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f931015d8a0 0x7f931015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f93156ff700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931015e690 con 0x7f931015e250 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f93156ff700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931015acf0 con 0x7f931015d8a0 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f93156ff700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931015afd0 con 0x7f93100cd6d0 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f930effd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100cd6d0 0x7f93100cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f930e7fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f931015d8a0 0x7f931015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f930effd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100cd6d0 0x7f93100cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:39424/0 (socket says 172.21.15.7:39424) 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f930e7fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f931015d8a0 0x7f931015dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:36742/0 (socket says 172.21.15.7:36742) 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.224+0000 7f930effd700 1 -- 172.21.15.7:0/2558832423 learned_addr learned my addr 172.21.15.7:0/2558832423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930effd700 1 -- 172.21.15.7:0/2558832423 >> v2:172.21.15.113:3300/0 conn(0x7f931015d8a0 msgr2=0x7f931015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930effd700 1 --2- 172.21.15.7:0/2558832423 >> v2:172.21.15.113:3300/0 conn(0x7f931015d8a0 0x7f931015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930effd700 1 -- 172.21.15.7:0/2558832423 >> v1:172.21.15.113:6789/0 conn(0x7f931015e250 legacy=0x7f93100cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930effd700 1 -- 172.21.15.7:0/2558832423 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100ce980 con 0x7f93100cd6d0 2024-01-11T07:07:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930dffb700 1 -- 172.21.15.7:0/2558832423 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211127842 0 0) 0x7f931015e690 con 0x7f931015e250 2024-01-11T07:07:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930e7fc700 1 --2- 172.21.15.7:0/2558832423 >> v2:172.21.15.113:3300/0 conn(0x7f931015d8a0 0x7f931015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930effd700 1 --2- 172.21.15.7:0/2558832423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100cd6d0 0x7f93100cdaa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93000099b0 tx=0x7f9300009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12dd04618d217daf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930dffb700 1 -- 172.21.15.7:0/2558832423 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300009ea0 con 0x7f93100cd6d0 2024-01-11T07:07:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930dffb700 1 -- 172.21.15.7:0/2558832423 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930000b860 con 0x7f93100cd6d0 2024-01-11T07:07:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.225+0000 7f930dffb700 1 -- 172.21.15.7:0/2558832423 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930000bc00 con 0x7f93100cd6d0 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.226+0000 7f93156ff700 1 -- 172.21.15.7:0/2558832423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100cd6d0 msgr2=0x7f93100cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.226+0000 7f93156ff700 1 --2- 172.21.15.7:0/2558832423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100cd6d0 0x7f93100cdaa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93000099b0 tx=0x7f9300009250 comp rx=0 tx=0).stop 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.226+0000 7f93156ff700 1 -- 172.21.15.7:0/2558832423 shutdown_connections 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.226+0000 7f93156ff700 1 --2- 172.21.15.7:0/2558832423 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93100cd6d0 0x7f93100cdaa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.226+0000 7f93156ff700 1 --2- 172.21.15.7:0/2558832423 >> v2:172.21.15.113:3300/0 conn(0x7f931015d8a0 0x7f931015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.226+0000 7f93156ff700 1 -- 172.21.15.7:0/2558832423 >> 172.21.15.7:0/2558832423 conn(0x7f9310071120 msgr2=0x7f931006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.226+0000 7f93156ff700 1 -- 172.21.15.7:0/2558832423 shutdown_connections 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.226+0000 7f93156ff700 1 -- 172.21.15.7:0/2558832423 wait complete. 2024-01-11T07:07:17.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.227+0000 7f93156ff700 1 Processor -- start 2024-01-11T07:07:17.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.227+0000 7f93156ff700 1 -- start start 2024-01-11T07:07:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f93156ff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93100cd6d0 0x7f9310148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f93156ff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f931015d8a0 0x7f93100b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f93156ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931015e250 0x7f93100cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f93156ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100b09f0 con 0x7f931015e250 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f93156ff700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100b1570 con 0x7f93100cd6d0 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f93156ff700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93100b0c40 con 0x7f931015d8a0 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f930f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931015e250 0x7f93100cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f930f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931015e250 0x7f93100cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41748/0 (socket says 172.21.15.7:41748) 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f930f7fe700 1 -- 172.21.15.7:0/2340612696 learned_addr learned my addr 172.21.15.7:0/2340612696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f930e7fc700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f931015d8a0 0x7f93100b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.228+0000 7f930effd700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93100cd6d0 0x7f9310148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f930f7fe700 1 -- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f931015d8a0 msgr2=0x7f93100b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f930f7fe700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f931015d8a0 0x7f93100b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f930f7fe700 1 -- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93100cd6d0 msgr2=0x7f9310148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f930f7fe700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93100cd6d0 0x7f9310148640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f930f7fe700 1 -- 172.21.15.7:0/2340612696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f8009420 con 0x7f931015e250 2024-01-11T07:07:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f930f7fe700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931015e250 0x7f93100cf4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f930800c8c0 tx=0x7f9308001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f9306ffd700 1 -- 172.21.15.7:0/2340612696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308012630 con 0x7f931015e250 2024-01-11T07:07:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f9306ffd700 1 -- 172.21.15.7:0/2340612696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9308012790 con 0x7f931015e250 2024-01-11T07:07:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93000094f0 con 0x7f931015e250 2024-01-11T07:07:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f9306ffd700 1 -- 172.21.15.7:0/2340612696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308012b50 con 0x7f931015e250 2024-01-11T07:07:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.229+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93100b1200 con 0x7f931015e250 2024-01-11T07:07:17.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.230+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931004c3c0 con 0x7f931015e250 2024-01-11T07:07:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.230+0000 7f9306ffd700 1 -- 172.21.15.7:0/2340612696 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f930801a3a0 con 0x7f931015e250 2024-01-11T07:07:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.231+0000 7f9306ffd700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92f0066f30 0x7f92f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.231+0000 7f9306ffd700 1 -- 172.21.15.7:0/2340612696 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93080846e0 con 0x7f931015e250 2024-01-11T07:07:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.231+0000 7f930effd700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92f0066f30 0x7f92f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.232+0000 7f930effd700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92f0066f30 0x7f92f00693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f93000099b0 tx=0x7f9300017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:17.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.235+0000 7f9306ffd700 1 -- 172.21.15.7:0/2340612696 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9308051950 con 0x7f931015e250 2024-01-11T07:07:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.492+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f931015a7b0 con 0x7f931015e250 2024-01-11T07:07:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.493+0000 7f9306ffd700 1 -- 172.21.15.7:0/2340612696 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f9308054f70 con 0x7f931015e250 2024-01-11T07:07:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:06:58.385324+0000","modified":"2024-01-11T07:07:02.162269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-11T07:07:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92f0066f30 msgr2=0x7f92f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92f0066f30 0x7f92f00693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f93000099b0 tx=0x7f9300017000 comp rx=0 tx=0).stop 2024-01-11T07:07:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931015e250 msgr2=0x7f93100cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931015e250 0x7f93100cf4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f930800c8c0 tx=0x7f9308001670 comp rx=0 tx=0).stop 2024-01-11T07:07:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 shutdown_connections 2024-01-11T07:07:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92f0066f30 0x7f92f00693e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f931015e250 0x7f93100cf4c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93100cd6d0 0x7f9310148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 --2- 172.21.15.7:0/2340612696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f931015d8a0 0x7f93100b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 >> 172.21.15.7:0/2340612696 conn(0x7f9310071120 msgr2=0x7f9310158970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.495+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 shutdown_connections 2024-01-11T07:07:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.496+0000 7f93156ff700 1 -- 172.21.15.7:0/2340612696 wait complete. 2024-01-11T07:07:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T07:07:17.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.640+0000 7f8026da1700 1 Processor -- start 2024-01-11T07:07:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.640+0000 7f8026da1700 1 -- start start 2024-01-11T07:07:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.641+0000 7f8026da1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80201599a0 0x7f80200cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.641+0000 7f8026da1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80200ac1c0 0x7f80200aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.641+0000 7f8026da1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80200692a0 con 0x7f8020159d70 2024-01-11T07:07:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.641+0000 7f8026da1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8020069860 con 0x7f80201599a0 2024-01-11T07:07:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.641+0000 7f8026da1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200cac50 con 0x7f80200ac1c0 2024-01-11T07:07:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8025d9f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80201599a0 0x7f80200cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8025d9f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80201599a0 0x7f80200cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44186/0 (socket says 172.21.15.7:44186) 2024-01-11T07:07:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8025d9f700 1 -- 172.21.15.7:0/257896726 learned_addr learned my addr 172.21.15.7:0/257896726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f802559e700 1 --2- 172.21.15.7:0/257896726 >> v2:172.21.15.113:3300/0 conn(0x7f80200ac1c0 0x7f80200aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8025d9f700 1 -- 172.21.15.7:0/257896726 >> v2:172.21.15.113:3300/0 conn(0x7f80200ac1c0 msgr2=0x7f80200aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8025d9f700 1 --2- 172.21.15.7:0/257896726 >> v2:172.21.15.113:3300/0 conn(0x7f80200ac1c0 0x7f80200aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8025d9f700 1 -- 172.21.15.7:0/257896726 >> v1:172.21.15.113:6789/0 conn(0x7f8020159d70 legacy=0x7f802015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8025d9f700 1 -- 172.21.15.7:0/257896726 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80200caf30 con 0x7f80201599a0 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8025d9f700 1 --2- 172.21.15.7:0/257896726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f80200cc610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f80200cdfd0 tx=0x7f8014009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5141d9502fc26e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8024d9d700 1 -- 172.21.15.7:0/257896726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801400b820 con 0x7f80201599a0 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8024d9d700 1 -- 172.21.15.7:0/257896726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801400b980 con 0x7f80201599a0 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.642+0000 7f8024d9d700 1 -- 172.21.15.7:0/257896726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801400bd20 con 0x7f80201599a0 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.643+0000 7f8026da1700 1 -- 172.21.15.7:0/257896726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 msgr2=0x7f80200cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.643+0000 7f8026da1700 1 --2- 172.21.15.7:0/257896726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f80200cc610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f80200cdfd0 tx=0x7f8014009250 comp rx=0 tx=0).stop 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.643+0000 7f8026da1700 1 -- 172.21.15.7:0/257896726 shutdown_connections 2024-01-11T07:07:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.643+0000 7f8026da1700 1 --2- 172.21.15.7:0/257896726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f80200cc610 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.643+0000 7f8026da1700 1 --2- 172.21.15.7:0/257896726 >> v2:172.21.15.113:3300/0 conn(0x7f80200ac1c0 0x7f80200aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.643+0000 7f8026da1700 1 -- 172.21.15.7:0/257896726 >> 172.21.15.7:0/257896726 conn(0x7f80200680e0 msgr2=0x7f80200ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.643+0000 7f8026da1700 1 -- 172.21.15.7:0/257896726 shutdown_connections 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.643+0000 7f8026da1700 1 -- 172.21.15.7:0/257896726 wait complete. 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.644+0000 7f8026da1700 1 Processor -- start 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.644+0000 7f8026da1700 1 -- start start 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f8026da1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80200ac1c0 0x7f80200b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f8026da1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f802016ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f8026da1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8020159d70 0x7f802016e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f8026da1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200afe40 con 0x7f80201599a0 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f8026da1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200b0120 con 0x7f80200ac1c0 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f8026da1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80200b0400 con 0x7f8020159d70 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f802559e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f802016ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f802559e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f802016ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44190/0 (socket says 172.21.15.7:44190) 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f802559e700 1 -- 172.21.15.7:0/4121890888 learned_addr learned my addr 172.21.15.7:0/4121890888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f80265a0700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8020159d70 0x7f802016e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f8025d9f700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80200ac1c0 0x7f80200b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f802559e700 1 -- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8020159d70 msgr2=0x7f802016e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f802559e700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8020159d70 0x7f802016e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f802559e700 1 -- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80200ac1c0 msgr2=0x7f80200b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f802559e700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80200ac1c0 0x7f80200b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.645+0000 7f802559e700 1 -- 172.21.15.7:0/4121890888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140094f0 con 0x7f80201599a0 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.646+0000 7f802559e700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f802016ddb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f801c00d860 tx=0x7f801c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.646+0000 7f8011ffb700 1 -- 172.21.15.7:0/4121890888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c0125f0 con 0x7f80201599a0 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.646+0000 7f8011ffb700 1 -- 172.21.15.7:0/4121890888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c012750 con 0x7f80201599a0 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.646+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80200b06e0 con 0x7f80201599a0 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.646+0000 7f8011ffb700 1 -- 172.21.15.7:0/4121890888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c012b10 con 0x7f80201599a0 2024-01-11T07:07:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.646+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80200b1420 con 0x7f80201599a0 2024-01-11T07:07:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.647+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802004c3c0 con 0x7f80201599a0 2024-01-11T07:07:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.647+0000 7f8011ffb700 1 -- 172.21.15.7:0/4121890888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f801c01e460 con 0x7f80201599a0 2024-01-11T07:07:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.648+0000 7f8011ffb700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.648+0000 7f8011ffb700 1 -- 172.21.15.7:0/4121890888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f801c084a60 con 0x7f80201599a0 2024-01-11T07:07:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.648+0000 7f8025d9f700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.648+0000 7f8025d9f700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8004066f30 0x7f80040693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f80200cdfd0 tx=0x7f8014016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.652+0000 7f8011ffb700 1 -- 172.21.15.7:0/4121890888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f801c055760 con 0x7f80201599a0 2024-01-11T07:07:17.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.855+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f802004bb80 con 0x7f80201599a0 2024-01-11T07:07:17.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.855+0000 7f8011ffb700 1 -- 172.21.15.7:0/4121890888 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f801c0552f0 con 0x7f80201599a0 2024-01-11T07:07:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:17.856 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8004066f30 msgr2=0x7f80040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8004066f30 0x7f80040693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f80200cdfd0 tx=0x7f8014016000 comp rx=0 tx=0).stop 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 msgr2=0x7f802016ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f802016ddb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f801c00d860 tx=0x7f801c008040 comp rx=0 tx=0).stop 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 shutdown_connections 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8020159d70 0x7f802016e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80200ac1c0 0x7f80200b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 --2- 172.21.15.7:0/4121890888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80201599a0 0x7f802016ddb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 >> 172.21.15.7:0/4121890888 conn(0x7f80200680e0 msgr2=0x7f80200b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 shutdown_connections 2024-01-11T07:07:17.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:17.858+0000 7f8026da1700 1 -- 172.21.15.7:0/4121890888 wait complete. 2024-01-11T07:07:17.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.003+0000 7f431a3d3700 1 Processor -- start 2024-01-11T07:07:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.003+0000 7f431a3d3700 1 -- start start 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f431a3d3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4314159ff0 0x7f43140b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f431a3d3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f431406f4d0 0x7f43140b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f431a3d3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4314067de0 con 0x7f4314159c20 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f431a3d3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4314069570 con 0x7f4314159ff0 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f431a3d3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43140b20e0 con 0x7f431406f4d0 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f43137fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4314159ff0 0x7f43140b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f4318970700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f431406f4d0 0x7f43140b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f43137fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4314159ff0 0x7f43140b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:39658/0 (socket says 172.21.15.7:39658) 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f43137fe700 1 -- 172.21.15.7:0/321404711 learned_addr learned my addr 172.21.15.7:0/321404711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.004+0000 7f4313fff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4314159c20 0x7f4314068270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:52314/0 (socket says 172.21.15.7:52314) 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f43137fe700 1 -- 172.21.15.7:0/321404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f431406f4d0 msgr2=0x7f43140b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f43137fe700 1 --2- 172.21.15.7:0/321404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f431406f4d0 0x7f43140b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f43137fe700 1 -- 172.21.15.7:0/321404711 >> v1:172.21.15.113:6789/0 conn(0x7f4314159c20 legacy=0x7f4314068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f43137fe700 1 -- 172.21.15.7:0/321404711 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431406f950 con 0x7f4314159ff0 2024-01-11T07:07:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f4312ffd700 1 -- 172.21.15.7:0/321404711 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685802984 0 0) 0x7f4314067de0 con 0x7f4314159c20 2024-01-11T07:07:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f4318970700 1 --2- 172.21.15.7:0/321404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f431406f4d0 0x7f43140b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f43137fe700 1 --2- 172.21.15.7:0/321404711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159ff0 0x7f43140b1820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0099b0 tx=0x7f42fc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a3c28641e48c79 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f4312ffd700 1 -- 172.21.15.7:0/321404711 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc009ea0 con 0x7f4314159ff0 2024-01-11T07:07:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f4312ffd700 1 -- 172.21.15.7:0/321404711 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc00b860 con 0x7f4314159ff0 2024-01-11T07:07:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.005+0000 7f4312ffd700 1 -- 172.21.15.7:0/321404711 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc00bc00 con 0x7f4314159ff0 2024-01-11T07:07:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.006+0000 7f431a3d3700 1 -- 172.21.15.7:0/321404711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159ff0 msgr2=0x7f43140b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.006+0000 7f431a3d3700 1 --2- 172.21.15.7:0/321404711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159ff0 0x7f43140b1820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0099b0 tx=0x7f42fc009250 comp rx=0 tx=0).stop 2024-01-11T07:07:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.006+0000 7f431a3d3700 1 -- 172.21.15.7:0/321404711 shutdown_connections 2024-01-11T07:07:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.006+0000 7f431a3d3700 1 --2- 172.21.15.7:0/321404711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f431406f4d0 0x7f43140b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.006+0000 7f431a3d3700 1 --2- 172.21.15.7:0/321404711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159ff0 0x7f43140b1820 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.006+0000 7f431a3d3700 1 -- 172.21.15.7:0/321404711 >> 172.21.15.7:0/321404711 conn(0x7f4314071160 msgr2=0x7f431406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:18.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.006+0000 7f431a3d3700 1 -- 172.21.15.7:0/321404711 shutdown_connections 2024-01-11T07:07:18.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.006+0000 7f431a3d3700 1 -- 172.21.15.7:0/321404711 wait complete. 2024-01-11T07:07:18.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.007+0000 7f431a3d3700 1 Processor -- start 2024-01-11T07:07:18.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.007+0000 7f431a3d3700 1 -- start start 2024-01-11T07:07:18.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f431a3d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431406f4d0 0x7f4314142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f431a3d3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159c20 0x7f43140ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f431a3d3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4314159ff0 0x7f431415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f431a3d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4314143380 con 0x7f431406f4d0 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f431a3d3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43140cbc00 con 0x7f4314159c20 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f431a3d3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43140caff0 con 0x7f4314159ff0 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f4313fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431406f4d0 0x7f4314142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f4313fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431406f4d0 0x7f4314142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44200/0 (socket says 172.21.15.7:44200) 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f4313fff700 1 -- 172.21.15.7:0/3809900596 learned_addr learned my addr 172.21.15.7:0/3809900596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f43137fe700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159c20 0x7f43140ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.008+0000 7f4318970700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4314159ff0 0x7f431415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f4313fff700 1 -- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4314159ff0 msgr2=0x7f431415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f4313fff700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4314159ff0 0x7f431415e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f4313fff700 1 -- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159c20 msgr2=0x7f43140ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f4313fff700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159c20 0x7f43140ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f4313fff700 1 -- 172.21.15.7:0/3809900596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c009a20 con 0x7f431406f4d0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f43137fe700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159c20 0x7f43140ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f4313fff700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431406f4d0 0x7f4314142e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f43040011e0 tx=0x7f4304007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f430bfff700 1 -- 172.21.15.7:0/3809900596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304005a70 con 0x7f431406f4d0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f430bfff700 1 -- 172.21.15.7:0/3809900596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4304005bd0 con 0x7f431406f4d0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc0094f0 con 0x7f431406f4d0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f430bfff700 1 -- 172.21.15.7:0/3809900596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040070d0 con 0x7f431406f4d0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.009+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43140cb5b0 con 0x7f431406f4d0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.010+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431404c3c0 con 0x7f431406f4d0 2024-01-11T07:07:18.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.010+0000 7f430bfff700 1 -- 172.21.15.7:0/3809900596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4304007480 con 0x7f431406f4d0 2024-01-11T07:07:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.011+0000 7f430bfff700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f42f4066f30 0x7f42f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.011+0000 7f430bfff700 1 -- 172.21.15.7:0/3809900596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f43040838c0 con 0x7f431406f4d0 2024-01-11T07:07:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.011+0000 7f43137fe700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f42f4066f30 0x7f42f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.012+0000 7f43137fe700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f42f4066f30 0x7f42f40693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0099b0 tx=0x7f42fc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.015+0000 7f430bfff700 1 -- 172.21.15.7:0/3809900596 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f430404b5a0 con 0x7f431406f4d0 2024-01-11T07:07:18.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.265+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43140b1e00 con 0x7f431406f4d0 2024-01-11T07:07:18.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.266+0000 7f430bfff700 1 -- 172.21.15.7:0/3809900596 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f4304017070 con 0x7f431406f4d0 2024-01-11T07:07:18.266 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:18.266 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:06:58.385324+0000","modified":"2024-01-11T07:07:02.162269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-11T07:07:18.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f42f4066f30 msgr2=0x7f42f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f42f4066f30 0x7f42f40693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0099b0 tx=0x7f42fc017000 comp rx=0 tx=0).stop 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431406f4d0 msgr2=0x7f4314142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431406f4d0 0x7f4314142e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f43040011e0 tx=0x7f4304007e10 comp rx=0 tx=0).stop 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 shutdown_connections 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f42f4066f30 0x7f42f40693e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4314159ff0 0x7f431415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431406f4d0 0x7f4314142e40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 --2- 172.21.15.7:0/3809900596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4314159c20 0x7f43140ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 >> 172.21.15.7:0/3809900596 conn(0x7f4314071160 msgr2=0x7f43140b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 shutdown_connections 2024-01-11T07:07:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.268+0000 7f431a3d3700 1 -- 172.21.15.7:0/3809900596 wait complete. 2024-01-11T07:07:18.270 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T07:07:18.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:06:58.385324+0000', 'modified': '2024-01-11T07:07:02.162269+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4257}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4257': {'gid': 4257, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6836/153691742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 153691742}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 153691742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:07:18.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:07:18.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-11T07:07:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.825+0000 7f590051e700 1 Processor -- start 2024-01-11T07:07:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.825+0000 7f590051e700 1 -- start start 2024-01-11T07:07:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f590051e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58f81579b0 0x7f58f815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f590051e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f8157d80 0x7f58f8063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f590051e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f8063bb0 con 0x7f58f8063770 2024-01-11T07:07:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f590051e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f80aeab0 con 0x7f58f81579b0 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f590051e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80aed90 con 0x7f58f8157d80 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f58feabb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f58f8063770 0x7f58f806dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51662/0 (socket says 172.21.15.7:51662) 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f58feabb700 1 -- 172.21.15.7:0/2009734179 learned_addr learned my addr 172.21.15.7:0/2009734179 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/2009734179 >> v2:172.21.15.113:3300/0 conn(0x7f58f81579b0 0x7f58f815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.826+0000 7f58fdab9700 1 --2- 172.21.15.7:0/2009734179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f8157d80 0x7f58f8063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fd2b8700 1 -- 172.21.15.7:0/2009734179 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429004148 0 0) 0x7f58f8063bb0 con 0x7f58f8063770 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fd2b8700 1 -- 172.21.15.7:0/2009734179 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e0003570 con 0x7f58f8063770 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fe2ba700 1 -- 172.21.15.7:0/2009734179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f8157d80 msgr2=0x7f58f8063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/2009734179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f8157d80 0x7f58f8063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fe2ba700 1 -- 172.21.15.7:0/2009734179 >> v1:172.21.15.7:6789/0 conn(0x7f58f8063770 legacy=0x7f58f806dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fe2ba700 1 -- 172.21.15.7:0/2009734179 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f80af070 con 0x7f58f81579b0 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/2009734179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f81579b0 0x7f58f815bf50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009e60 tx=0x7f58e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fc5be5b5f9341c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fd2b8700 1 -- 172.21.15.7:0/2009734179 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800e070 con 0x7f58f81579b0 2024-01-11T07:07:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fd2b8700 1 -- 172.21.15.7:0/2009734179 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e800bab0 con 0x7f58f81579b0 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.827+0000 7f58fd2b8700 1 -- 172.21.15.7:0/2009734179 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800be70 con 0x7f58f81579b0 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.828+0000 7f590051e700 1 -- 172.21.15.7:0/2009734179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f81579b0 msgr2=0x7f58f815bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.828+0000 7f590051e700 1 --2- 172.21.15.7:0/2009734179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f81579b0 0x7f58f815bf50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009e60 tx=0x7f58e8009780 comp rx=0 tx=0).stop 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.828+0000 7f590051e700 1 -- 172.21.15.7:0/2009734179 shutdown_connections 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.828+0000 7f590051e700 1 --2- 172.21.15.7:0/2009734179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f81579b0 0x7f58f815bf50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.828+0000 7f590051e700 1 --2- 172.21.15.7:0/2009734179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f8157d80 0x7f58f8063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.828+0000 7f590051e700 1 -- 172.21.15.7:0/2009734179 >> 172.21.15.7:0/2009734179 conn(0x7f58f80680e0 msgr2=0x7f58f80cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.828+0000 7f590051e700 1 -- 172.21.15.7:0/2009734179 shutdown_connections 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.828+0000 7f590051e700 1 -- 172.21.15.7:0/2009734179 wait complete. 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.829+0000 7f590051e700 1 Processor -- start 2024-01-11T07:07:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.829+0000 7f590051e700 1 -- start start 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f590051e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f8063770 0x7f58f80c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f590051e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f81579b0 0x7f58f80c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f590051e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8157d80 0x7f58f8167940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f590051e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80c1120 con 0x7f58f8157d80 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f590051e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8168190 con 0x7f58f8063770 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f590051e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8168470 con 0x7f58f81579b0 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58feabb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8157d80 0x7f58f8167940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58fe2ba700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f8063770 0x7f58f80c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58feabb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8157d80 0x7f58f8167940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44208/0 (socket says 172.21.15.7:44208) 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58feabb700 1 -- 172.21.15.7:0/726826455 learned_addr learned my addr 172.21.15.7:0/726826455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58fdab9700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f81579b0 0x7f58f80c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58feabb700 1 -- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f81579b0 msgr2=0x7f58f80c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58feabb700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f81579b0 0x7f58f80c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58feabb700 1 -- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f8063770 msgr2=0x7f58f80c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58feabb700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f8063770 0x7f58f80c3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58feabb700 1 -- 172.21.15.7:0/726826455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e8009a20 con 0x7f58f8157d80 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.830+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f8063770 0x7f58f80c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.831+0000 7f58feabb700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8157d80 0x7f58f8167940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58f00010e0 tx=0x7f58f0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.831+0000 7f58f67fc700 1 -- 172.21.15.7:0/726826455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0005c00 con 0x7f58f8157d80 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.831+0000 7f58f67fc700 1 -- 172.21.15.7:0/726826455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0005d60 con 0x7f58f8157d80 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.831+0000 7f590051e700 1 -- 172.21.15.7:0/726826455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f8168750 con 0x7f58f8157d80 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.831+0000 7f58f67fc700 1 -- 172.21.15.7:0/726826455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f00072e0 con 0x7f58f8157d80 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.831+0000 7f590051e700 1 -- 172.21.15.7:0/726826455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f80c4e20 con 0x7f58f8157d80 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.833+0000 7f58f67fc700 1 -- 172.21.15.7:0/726826455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f58f0012400 con 0x7f58f8157d80 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.833+0000 7f58f67fc700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58e4066f30 0x7f58e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.833+0000 7f58f67fc700 1 -- 172.21.15.7:0/726826455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f58f0083990 con 0x7f58f8157d80 2024-01-11T07:07:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.833+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58e4066f30 0x7f58e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.834+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58e4066f30 0x7f58e40693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f58e80097c0 tx=0x7f58e8019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.835+0000 7f590981eb80 1 Processor -- start 2024-01-11T07:07:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.835+0000 7f590981eb80 1 -- start start 2024-01-11T07:07:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f590981eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63bfa0 0x563f2e63c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f590981eb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e637c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f590981eb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 0x563f2e658360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f590981eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2e56a3e0 con 0x563f2e63bfa0 2024-01-11T07:07:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f590981eb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2e567be0 con 0x563f2e63c8b0 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f590981eb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f2e5639d0 con 0x563f2e638170 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f58fe2ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63bfa0 0x563f2e63c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f58fe2ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63bfa0 0x563f2e63c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44222/0 (socket says 172.21.15.7:44222) 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f58fe2ba700 1 -- 172.21.15.7:0/4005486064 learned_addr learned my addr 172.21.15.7:0/4005486064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f58feabb700 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 0x563f2e658360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.836+0000 7f58fdab9700 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e637c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58fe2ba700 1 -- 172.21.15.7:0/4005486064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 msgr2=0x563f2e658360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 0x563f2e658360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58fe2ba700 1 -- 172.21.15.7:0/4005486064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 msgr2=0x563f2e637c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e637c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58fe2ba700 1 -- 172.21.15.7:0/4005486064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f2e3f7360 con 0x563f2e63bfa0 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63bfa0 0x563f2e63c370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58e8012670 tx=0x7f58e8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb426861bcd79ab server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58df7fe700 1 -- 172.21.15.7:0/4005486064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8028480 con 0x563f2e63bfa0 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58df7fe700 1 -- 172.21.15.7:0/4005486064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e80285e0 con 0x563f2e63bfa0 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.837+0000 7f58df7fe700 1 -- 172.21.15.7:0/4005486064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e80289a0 con 0x563f2e63bfa0 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 -- 172.21.15.7:0/4005486064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63bfa0 msgr2=0x563f2e63c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63bfa0 0x563f2e63c370 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58e8012670 tx=0x7f58e8021040 comp rx=0 tx=0).stop 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 -- 172.21.15.7:0/4005486064 shutdown_connections 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 0x563f2e658360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63bfa0 0x563f2e63c370 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 --2- 172.21.15.7:0/4005486064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e637c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 -- 172.21.15.7:0/4005486064 >> 172.21.15.7:0/4005486064 conn(0x563f2e612440 msgr2=0x563f2e65ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 -- 172.21.15.7:0/4005486064 shutdown_connections 2024-01-11T07:07:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 -- 172.21.15.7:0/4005486064 wait complete. 2024-01-11T07:07:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 -- start start 2024-01-11T07:07:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.838+0000 7f590981eb80 1 Processor -- start 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.839+0000 7f590981eb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e638170 0x563f2e652aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.839+0000 7f590981eb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e659620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.839+0000 7f590981eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e638e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.839+0000 7f590981eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2e5101b0 con 0x563f2e63c8b0 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.839+0000 7f590981eb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2e50fe50 con 0x563f2e638170 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.839+0000 7f590981eb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f2e45ce40 con 0x563f2e63bfa0 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f590981eb80 10 client.0 _ll_register_callbacks cb 0x563f2e45f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e638e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e638e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44230/0 (socket says 172.21.15.7:44230) 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 -- 172.21.15.7:0/3489181907 learned_addr learned my addr 172.21.15.7:0/3489181907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e638170 0x563f2e652aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58fdab9700 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e659620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 -- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 msgr2=0x563f2e659620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e659620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 -- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e638170 msgr2=0x563f2e652aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e638170 0x563f2e652aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 -- 172.21.15.7:0/3489181907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f80af070 con 0x563f2e63c8b0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58fdab9700 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e659620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58feabb700 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e638e80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007fd0 tx=0x7f58f0081d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05cf7cfedb18229 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58dcff9700 1 -- 172.21.15.7:0/3489181907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0057a70 con 0x563f2e63c8b0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58dcff9700 1 -- 172.21.15.7:0/3489181907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f58f006ae40 con 0x563f2e63c8b0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f590981eb80 1 -- 172.21.15.7:0/3489181907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f2e5e4eb0 con 0x563f2e63c8b0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f58dcff9700 1 -- 172.21.15.7:0/3489181907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58f005b7c0 con 0x563f2e63c8b0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f590981eb80 10 client.4626 fetch_fsmap learned FSMap version 8 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.840+0000 7f590981eb80 1 -- 172.21.15.7:0/3489181907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563f2e3a3b10 con 0x563f2e63c8b0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f58dcff9700 1 -- 172.21.15.7:0/3489181907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563f2e5e4eb0 con 0x563f2e63c8b0 2024-01-11T07:07:18.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f58dcff9700 1 -- 172.21.15.7:0/3489181907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f58f0070270 con 0x563f2e63c8b0 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f590981eb80 10 client.4626 fetch_fsmap finished waiting for FSMap version 8 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f590981eb80 10 client.4626 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f590981eb80 20 client.4626 populate_metadata read hostname 'smithi007' 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f590981eb80 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] conn(0x563f2e66d860 0x563f2e66dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f590981eb80 4 client.4626 mds_command: new command op to 4257 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f590981eb80 1 -- 172.21.15.7:0/3489181907 --> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f2e53a880 con 0x563f2e66d860 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] conn(0x563f2e66d860 0x563f2e66dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f58dcff9700 10 client.4626 ms_handle_connect on v2:172.21.15.113:6834/153691742 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.841+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] conn(0x563f2e66d860 0x563f2e66dc30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e6908d18a0ab39 server_cookie=eab40d74faf285b4 in_seq=0 out_seq=0 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f58dcff9700 1 -- 172.21.15.7:0/3489181907 <== mds.0 v2:172.21.15.113:6834/153691742 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563f2e53a880 con 0x563f2e66d860 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f58dcff9700 10 client.4626 handle_command_reply: tid=0 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 client.4626 shutdown 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 -- 172.21.15.7:0/3489181907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 msgr2=0x563f2e638e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e638e80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007fd0 tx=0x7f58f0081d70 comp rx=0 tx=0).stop 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 -- 172.21.15.7:0/3489181907 shutdown_connections 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e638e80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e638170 0x563f2e652aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] conn(0x563f2e66d860 0x563f2e66dc30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 --2- 172.21.15.7:0/3489181907 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e659620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f58dcff9700 0 client.4626 ms_handle_reset on v2:172.21.15.113:6834/153691742 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 -- 172.21.15.7:0/3489181907 >> 172.21.15.7:0/3489181907 conn(0x563f2e612440 msgr2=0x563f2e612810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 -- 172.21.15.7:0/3489181907 shutdown_connections 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.842+0000 7f590981eb80 1 -- 172.21.15.7:0/3489181907 wait complete. 2024-01-11T07:07:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.843+0000 7f590981eb80 20 client.4626 trim_cache size 0 max 16384 2024-01-11T07:07:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.861+0000 7f590981eb80 1 Processor -- start 2024-01-11T07:07:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.861+0000 7f590981eb80 1 -- start start 2024-01-11T07:07:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.861+0000 7f590981eb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 0x563f2e6436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.861+0000 7f590981eb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63bfa0 0x563f2e614200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.861+0000 7f590981eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e60a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.861+0000 7f590981eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2e637410 con 0x563f2e63c8b0 2024-01-11T07:07:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.861+0000 7f590981eb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2e60c270 con 0x563f2e63bfa0 2024-01-11T07:07:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.861+0000 7f590981eb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f2e63c370 con 0x563f2e638170 2024-01-11T07:07:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e60a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e60a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44236/0 (socket says 172.21.15.7:44236) 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 -- 172.21.15.7:0/2176247425 learned_addr learned my addr 172.21.15.7:0/2176247425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58fdab9700 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63bfa0 0x563f2e614200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 0x563f2e6436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 -- 172.21.15.7:0/2176247425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 msgr2=0x563f2e6436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 0x563f2e6436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 -- 172.21.15.7:0/2176247425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63bfa0 msgr2=0x563f2e614200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63bfa0 0x563f2e614200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 -- 172.21.15.7:0/2176247425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f2e532290 con 0x563f2e63c8b0 2024-01-11T07:07:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58feabb700 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e60a380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007b90 tx=0x7f58f006f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b7c385782c311f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58dffff700 1 -- 172.21.15.7:0/2176247425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0057a70 con 0x563f2e63c8b0 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58dffff700 1 -- 172.21.15.7:0/2176247425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f005b7c0 con 0x563f2e63c8b0 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.862+0000 7f58dffff700 1 -- 172.21.15.7:0/2176247425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0070270 con 0x563f2e63c8b0 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 -- 172.21.15.7:0/2176247425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 msgr2=0x563f2e60a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e60a380 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007b90 tx=0x7f58f006f000 comp rx=0 tx=0).stop 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 -- 172.21.15.7:0/2176247425 shutdown_connections 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e63c8b0 0x563f2e60a380 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e638170 0x563f2e6436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 --2- 172.21.15.7:0/2176247425 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63bfa0 0x563f2e614200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 -- 172.21.15.7:0/2176247425 >> 172.21.15.7:0/2176247425 conn(0x563f2e612440 msgr2=0x563f2e63b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 -- 172.21.15.7:0/2176247425 shutdown_connections 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 -- 172.21.15.7:0/2176247425 wait complete. 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 -- start start 2024-01-11T07:07:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.863+0000 7f590981eb80 1 Processor -- start 2024-01-11T07:07:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.864+0000 7f590981eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e638170 0x563f2e63ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.864+0000 7f590981eb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e6c3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.864+0000 7f590981eb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e60f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.864+0000 7f590981eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2e6c5400 con 0x563f2e638170 2024-01-11T07:07:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.864+0000 7f590981eb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2e610730 con 0x563f2e63c8b0 2024-01-11T07:07:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.864+0000 7f590981eb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f2e610480 con 0x563f2e63bfa0 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f590981eb80 10 client.0 _ll_register_callbacks cb 0x563f2e45f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e638170 0x563f2e63ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e638170 0x563f2e63ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44246/0 (socket says 172.21.15.7:44246) 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 -- 172.21.15.7:0/3074290056 learned_addr learned my addr 172.21.15.7:0/3074290056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fdab9700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e6c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58feabb700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e60f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 -- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 msgr2=0x563f2e6c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e6c3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 -- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 msgr2=0x563f2e60f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e60f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 -- 172.21.15.7:0/3074290056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f2e3a3b10 con 0x563f2e638170 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fdab9700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e6c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58feabb700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e60f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58fe2ba700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e638170 0x563f2e63ddd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e8035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d5f4179fc35f0d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58dcff9700 1 -- 172.21.15.7:0/3074290056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8039040 con 0x563f2e638170 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58dcff9700 1 -- 172.21.15.7:0/3074290056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f58e8028690 con 0x563f2e638170 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f590981eb80 1 -- 172.21.15.7:0/3074290056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f2e608490 con 0x563f2e638170 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f58dcff9700 1 -- 172.21.15.7:0/3074290056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58e803d250 con 0x563f2e638170 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.865+0000 7f590981eb80 10 client.4632 fetch_fsmap learned FSMap version 8 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f590981eb80 1 -- 172.21.15.7:0/3074290056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563f2e3fbc80 con 0x563f2e638170 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f58dcff9700 1 -- 172.21.15.7:0/3074290056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563f2e608490 con 0x563f2e638170 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f58dcff9700 1 -- 172.21.15.7:0/3074290056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f58e8048450 con 0x563f2e638170 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f590981eb80 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f590981eb80 10 client.4632 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f590981eb80 20 client.4632 populate_metadata read hostname 'smithi007' 2024-01-11T07:07:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f590981eb80 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] conn(0x563f2e65cb40 0x563f2e65cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f590981eb80 4 client.4632 mds_command: new command op to 4257 tid=0[{"prefix": "status"}] 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f590981eb80 1 -- 172.21.15.7:0/3074290056 --> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563f2e506910 con 0x563f2e65cb40 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f58fdab9700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] conn(0x563f2e65cb40 0x563f2e65cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f58dcff9700 10 client.4632 ms_handle_connect on v2:172.21.15.113:6834/153691742 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.866+0000 7f58fdab9700 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] conn(0x563f2e65cb40 0x563f2e65cf10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195b95fe323af2bb server_cookie=2f970f4d8e7fc904 in_seq=0 out_seq=0 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f58dcff9700 1 -- 172.21.15.7:0/3074290056 <== mds.0 v2:172.21.15.113:6834/153691742 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x563f2e506910 con 0x563f2e65cb40 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f58dcff9700 10 client.4632 handle_command_reply: tid=0 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 client.4632 shutdown 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 -- 172.21.15.7:0/3074290056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e638170 msgr2=0x563f2e63ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e638170 0x563f2e63ddd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e8035040 comp rx=0 tx=0).stop 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 -- 172.21.15.7:0/3074290056 shutdown_connections 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563f2e63c8b0 0x563f2e60f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563f2e638170 0x563f2e63ddd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:6834/153691742,v1:172.21.15.113:6836/153691742] conn(0x563f2e65cb40 0x563f2e65cf10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 --2- 172.21.15.7:0/3074290056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563f2e63bfa0 0x563f2e6c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f58dcff9700 0 client.4632 ms_handle_reset on v2:172.21.15.113:6834/153691742 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 -- 172.21.15.7:0/3074290056 >> 172.21.15.7:0/3074290056 conn(0x563f2e612440 msgr2=0x563f2e63b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 -- 172.21.15.7:0/3074290056 shutdown_connections 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 1 -- 172.21.15.7:0/3074290056 wait complete. 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.867+0000 7f590981eb80 20 client.4632 trim_cache size 0 max 16384 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:07:18.876 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4257, 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 19.718651560000001, 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 8, 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 22, 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 22, 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 25.395335203999998 2024-01-11T07:07:18.877 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:07:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 -- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58e4066f30 msgr2=0x7f58e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58e4066f30 0x7f58e40693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f58e80097c0 tx=0x7f58e8019000 comp rx=0 tx=0).stop 2024-01-11T07:07:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 -- 172.21.15.7:0/726826455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8157d80 msgr2=0x7f58f8167940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8157d80 0x7f58f8167940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58f00010e0 tx=0x7f58f0007dd0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 -- 172.21.15.7:0/726826455 shutdown_connections 2024-01-11T07:07:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58e4066f30 0x7f58e40693e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8157d80 0x7f58f8167940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58f8063770 0x7f58f80c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 --2- 172.21.15.7:0/726826455 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58f81579b0 0x7f58f80c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:18.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 -- 172.21.15.7:0/726826455 >> 172.21.15.7:0/726826455 conn(0x7f58f80680e0 msgr2=0x7f58f8155f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:18.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 -- 172.21.15.7:0/726826455 shutdown_connections 2024-01-11T07:07:18.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:18.868+0000 7f590051e700 1 -- 172.21.15.7:0/726826455 wait complete. 2024-01-11T07:07:18.881 INFO:teuthology.run_tasks:Running task kclient... 2024-01-11T07:07:18.903 INFO:tasks.kclient:Mounting kernel clients... 2024-01-11T07:07:18.903 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-11T07:07:18.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:18.903 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:07:18.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:18.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-11T07:07:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-01-11T07:07:18.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:07:19.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.000 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:07:19.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-11T07:07:19.139 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-01-11T07:07:19.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:07:19.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.141 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:07:19.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.163 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-01-11T07:07:19.238 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-01-11T07:07:19.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:07:19.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.240 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:07:19.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.301 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-01-11T07:07:19.373 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-01-11T07:07:19.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:07:19.374 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-01-11T07:07:19.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:07:19.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.515+0000 7f4479013700 1 Processor -- start 2024-01-11T07:07:19.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.515+0000 7f4479013700 1 -- start start 2024-01-11T07:07:19.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.516+0000 7f4479013700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44740b0b40 0x7f4474162770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f4479013700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4474159300 con 0x7f4474158280 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f4479013700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44740cf000 con 0x7f44740cafb0 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f4479013700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740cf260 con 0x7f44740b0b40 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f447259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4474158280 0x7f44740b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51678/0 (socket says 172.21.15.7:51678) 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f447259c700 1 -- 172.21.15.7:0/2899113132 learned_addr learned my addr 172.21.15.7:0/2899113132 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f447359e700 1 --2- 172.21.15.7:0/2899113132 >> v2:172.21.15.7:3300/0 conn(0x7f44740b0b40 0x7f4474162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f447359e700 1 -- 172.21.15.7:0/2899113132 >> v1:172.21.15.113:6789/0 conn(0x7f44740cafb0 legacy=0x7f44740ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f447359e700 1 -- 172.21.15.7:0/2899113132 >> v1:172.21.15.7:6789/0 conn(0x7f4474158280 legacy=0x7f44740b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f447359e700 1 -- 172.21.15.7:0/2899113132 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740cf540 con 0x7f44740b0b40 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f4471d9b700 1 -- 172.21.15.7:0/2899113132 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762874738 0 0) 0x7f4474159300 con 0x7f4474158280 2024-01-11T07:07:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f447359e700 1 --2- 172.21.15.7:0/2899113132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740b0b40 0x7f4474162770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f446c009e60 tx=0x7f446c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7989f745bb6c1126 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f4471d9b700 1 -- 172.21.15.7:0/2899113132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00e070 con 0x7f44740b0b40 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f4471d9b700 1 -- 172.21.15.7:0/2899113132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00bc30 con 0x7f44740b0b40 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.517+0000 7f4471d9b700 1 -- 172.21.15.7:0/2899113132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c012430 con 0x7f44740b0b40 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.518+0000 7f4479013700 1 -- 172.21.15.7:0/2899113132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740b0b40 msgr2=0x7f4474162770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.518+0000 7f4479013700 1 --2- 172.21.15.7:0/2899113132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740b0b40 0x7f4474162770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f446c009e60 tx=0x7f446c009780 comp rx=0 tx=0).stop 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.518+0000 7f4479013700 1 -- 172.21.15.7:0/2899113132 shutdown_connections 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.518+0000 7f4479013700 1 --2- 172.21.15.7:0/2899113132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740b0b40 0x7f4474162770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.518+0000 7f4479013700 1 -- 172.21.15.7:0/2899113132 >> 172.21.15.7:0/2899113132 conn(0x7f4474071100 msgr2=0x7f447406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.518+0000 7f4479013700 1 -- 172.21.15.7:0/2899113132 shutdown_connections 2024-01-11T07:07:19.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.518+0000 7f4479013700 1 -- 172.21.15.7:0/2899113132 wait complete. 2024-01-11T07:07:19.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.520+0000 7f4479013700 1 Processor -- start 2024-01-11T07:07:19.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.520+0000 7f4479013700 1 -- start start 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f4479013700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44740b0b40 0x7f4474142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f4479013700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740cafb0 0x7f44740b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f4479013700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4474158280 0x7f44740b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f4479013700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4474155010 con 0x7f44740cafb0 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f4479013700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4474154400 con 0x7f4474158280 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f4479013700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44741546e0 con 0x7f44740b0b40 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740cafb0 0x7f44740b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740cafb0 0x7f44740b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44266/0 (socket says 172.21.15.7:44266) 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447259c700 1 -- 172.21.15.7:0/1923017157 learned_addr learned my addr 172.21.15.7:0/1923017157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f4472d9d700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44740b0b40 0x7f4474142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447359e700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4474158280 0x7f44740b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447259c700 1 -- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44740b0b40 msgr2=0x7f4474142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447259c700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44740b0b40 0x7f4474142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447259c700 1 -- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4474158280 msgr2=0x7f44740b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447259c700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4474158280 0x7f44740b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.521+0000 7f447259c700 1 -- 172.21.15.7:0/1923017157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c009a20 con 0x7f44740cafb0 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.522+0000 7f447259c700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740cafb0 0x7f44740b2350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c5e0 tx=0x7f445c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.522+0000 7f446affd700 1 -- 172.21.15.7:0/1923017157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c005b30 con 0x7f44740cafb0 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.522+0000 7f446affd700 1 -- 172.21.15.7:0/1923017157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c005c90 con 0x7f44740cafb0 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.522+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44741549c0 con 0x7f44740cafb0 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.522+0000 7f446affd700 1 -- 172.21.15.7:0/1923017157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c007300 con 0x7f44740cafb0 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.522+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44741554a0 con 0x7f44740cafb0 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.523+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4474155180 con 0x7f44740cafb0 2024-01-11T07:07:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.523+0000 7f446affd700 1 -- 172.21.15.7:0/1923017157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f445c01b3b0 con 0x7f44740cafb0 2024-01-11T07:07:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.524+0000 7f446affd700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4454066f30 0x7f44540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.524+0000 7f446affd700 1 -- 172.21.15.7:0/1923017157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f445c083930 con 0x7f44740cafb0 2024-01-11T07:07:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.524+0000 7f4472d9d700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4454066f30 0x7f44540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.524+0000 7f4472d9d700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4454066f30 0x7f44540693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4464003a80 tx=0x7f4464007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.528+0000 7f446affd700 1 -- 172.21.15.7:0/1923017157 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f445c050ba0 con 0x7f44740cafb0 2024-01-11T07:07:19.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.778+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f447404c3c0 con 0x7f44740cafb0 2024-01-11T07:07:19.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.779+0000 7f446affd700 1 -- 172.21.15.7:0/1923017157 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f445c017070 con 0x7f44740cafb0 2024-01-11T07:07:19.780 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:07:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4454066f30 msgr2=0x7f44540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4454066f30 0x7f44540693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4464003a80 tx=0x7f4464007ec0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740cafb0 msgr2=0x7f44740b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740cafb0 0x7f44740b2350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c5e0 tx=0x7f445c000fe0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 shutdown_connections 2024-01-11T07:07:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4454066f30 0x7f44540693e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4474158280 0x7f44740b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f44740b0b40 0x7f4474142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 --2- 172.21.15.7:0/1923017157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44740cafb0 0x7f44740b2350 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 >> 172.21.15.7:0/1923017157 conn(0x7f4474071100 msgr2=0x7f44740ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 shutdown_connections 2024-01-11T07:07:19.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:19.781+0000 7f4479013700 1 -- 172.21.15.7:0/1923017157 wait complete. 2024-01-11T07:07:19.795 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-11T07:07:19.795 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:07:19.795 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:07:19.796 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:07:19.796 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:07:19.796 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-11T07:07:19.796 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-11T07:07:19.796 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:07:19.796 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:07:19.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.796 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:07:19.815 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42378sec preferred_lft 42378sec 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:07:19.817 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-11T07:07:19.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.817 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:19.817 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:19.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-01-11T07:07:19.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-01-11T07:07:19.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-01-11T07:07:19.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-11T07:07:19.818 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:19.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:19.996 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:07:20.029 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:07:20.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:20.031 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:07:20.120 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:07:20.120 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:07:20.120 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:07:20.121 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:07:20.121 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:07:20.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:20.121 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:20.122 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:20.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:07:20.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:07:20.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:07:20.122 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:20.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:20.359 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:07:20.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:20.382 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-01-11T07:07:20.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:20.443 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:20.444 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:20.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:07:20.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-11T07:07:20.444 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:20.554 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-11T07:07:20.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:20.554 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:20.554 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:20.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-11T07:07:20.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-11T07:07:20.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-11T07:07:20.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-11T07:07:20.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-11T07:07:20.555 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:20.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:20.771 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:20.771 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:20.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:07:20.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-01-11T07:07:20.771 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:20.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:20.880 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:07:20.899 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:07:20.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:20.900 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:07:20.960 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:07:20.960 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:07:20.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:07:21.054 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:07:21.054 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:07:21.054 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:21.136 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:07:21.136 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:07:21.136 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:21.136 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:21.136 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:21.153 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:07:21.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:07:21.154 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:21.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:21.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:21.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:07:21.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:07:21.154 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:07:21.156 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:07:21.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:21.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:07:21.188 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-01-11T07:07:21.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:07:21.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.364+0000 7fae1ccd2700 1 Processor -- start 2024-01-11T07:07:21.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.364+0000 7fae1ccd2700 1 -- start start 2024-01-11T07:07:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae1ccd2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fae180ac3a0 0x7fae180cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae1ccd2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae1815d920 0x7fae1815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae1ccd2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae1815a2c0 0x7fae1815a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae1ccd2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1815e2d0 con 0x7fae1815a2c0 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae1ccd2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1806ac30 con 0x7fae180ac3a0 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae1ccd2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae180ac770 con 0x7fae1815d920 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae16d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae1815a2c0 0x7fae1815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae16d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae1815a2c0 0x7fae1815a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44274/0 (socket says 172.21.15.7:44274) 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae16d9d700 1 -- 172.21.15.7:0/2107972975 learned_addr learned my addr 172.21.15.7:0/2107972975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae0ffff700 1 --2- 172.21.15.7:0/2107972975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae1815d920 0x7fae1815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.365+0000 7fae1659c700 1 --2- 172.21.15.7:0/2107972975 >> v2:172.21.15.113:3300/0 conn(0x7fae180ac3a0 0x7fae180cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae16d9d700 1 -- 172.21.15.7:0/2107972975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae1815d920 msgr2=0x7fae1815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae16d9d700 1 --2- 172.21.15.7:0/2107972975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae1815d920 0x7fae1815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae16d9d700 1 -- 172.21.15.7:0/2107972975 >> v2:172.21.15.113:3300/0 conn(0x7fae180ac3a0 msgr2=0x7fae180cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae16d9d700 1 --2- 172.21.15.7:0/2107972975 >> v2:172.21.15.113:3300/0 conn(0x7fae180ac3a0 0x7fae180cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae16d9d700 1 -- 172.21.15.7:0/2107972975 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1806a300 con 0x7fae1815a2c0 2024-01-11T07:07:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae16d9d700 1 --2- 172.21.15.7:0/2107972975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815a2c0 0x7fae1815a740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae180ccff0 tx=0x7fae10009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e1d0122cc0a76e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae15d9b700 1 -- 172.21.15.7:0/2107972975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1000e070 con 0x7fae1815a2c0 2024-01-11T07:07:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae15d9b700 1 -- 172.21.15.7:0/2107972975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1000b970 con 0x7fae1815a2c0 2024-01-11T07:07:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.366+0000 7fae15d9b700 1 -- 172.21.15.7:0/2107972975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae100092e0 con 0x7fae1815a2c0 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 -- 172.21.15.7:0/2107972975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815a2c0 msgr2=0x7fae1815a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/2107972975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815a2c0 0x7fae1815a740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae180ccff0 tx=0x7fae10009840 comp rx=0 tx=0).stop 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 -- 172.21.15.7:0/2107972975 shutdown_connections 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/2107972975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815a2c0 0x7fae1815a740 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/2107972975 >> v2:172.21.15.113:3300/0 conn(0x7fae180ac3a0 0x7fae180cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/2107972975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae1815d920 0x7fae1815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 -- 172.21.15.7:0/2107972975 >> 172.21.15.7:0/2107972975 conn(0x7fae180cd090 msgr2=0x7fae180cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 -- 172.21.15.7:0/2107972975 shutdown_connections 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.367+0000 7fae1ccd2700 1 -- 172.21.15.7:0/2107972975 wait complete. 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.368+0000 7fae1ccd2700 1 Processor -- start 2024-01-11T07:07:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.368+0000 7fae1ccd2700 1 -- start start 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae1ccd2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae180ac3a0 0x7fae1815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae1ccd2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae1815a2c0 0x7fae1815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae1ccd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815d920 0x7fae1806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae1ccd2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1806e000 con 0x7fae1815d920 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae1ccd2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae180ac970 con 0x7fae1815a2c0 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae1ccd2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae180acc50 con 0x7fae180ac3a0 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815d920 0x7fae1806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae1659c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae180ac3a0 0x7fae1815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae0ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae1815a2c0 0x7fae1815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815d920 0x7fae1806e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44284/0 (socket says 172.21.15.7:44284) 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae1659c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae180ac3a0 0x7fae1815a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:57308/0 (socket says 172.21.15.7:57308) 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 -- 172.21.15.7:0/3573542655 learned_addr learned my addr 172.21.15.7:0/3573542655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 -- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae180ac3a0 msgr2=0x7fae1815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae180ac3a0 0x7fae1815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 -- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae1815a2c0 msgr2=0x7fae1815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae1815a2c0 0x7fae1815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 -- 172.21.15.7:0/3573542655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae10009ae0 con 0x7fae1815d920 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.369+0000 7fae16d9d700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815d920 0x7fae1806e490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae1806fdf0 tx=0x7fae10018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.370+0000 7fae0e7fc700 1 -- 172.21.15.7:0/3573542655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10008ad0 con 0x7fae1815d920 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.370+0000 7fae0e7fc700 1 -- 172.21.15.7:0/3573542655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae10008c50 con 0x7fae1815d920 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.370+0000 7fae0e7fc700 1 -- 172.21.15.7:0/3573542655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1001c410 con 0x7fae1815d920 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.370+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1815cff0 con 0x7fae1815d920 2024-01-11T07:07:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.370+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae180af750 con 0x7fae1815d920 2024-01-11T07:07:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.371+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1806ddd0 con 0x7fae1815d920 2024-01-11T07:07:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.371+0000 7fae0e7fc700 1 -- 172.21.15.7:0/3573542655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fae10003040 con 0x7fae1815d920 2024-01-11T07:07:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.372+0000 7fae0e7fc700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fadf8066f30 0x7fadf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.372+0000 7fae0e7fc700 1 -- 172.21.15.7:0/3573542655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae10085b60 con 0x7fae1815d920 2024-01-11T07:07:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.372+0000 7fae1659c700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fadf8066f30 0x7fadf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.372+0000 7fae1659c700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fadf8066f30 0x7fadf80693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae08009d10 tx=0x7fae080091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.376+0000 7fae0e7fc700 1 -- 172.21.15.7:0/3573542655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae1806ddd0 con 0x7fae1815d920 2024-01-11T07:07:21.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.627+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fae1806ddd0 con 0x7fae1815d920 2024-01-11T07:07:21.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.627+0000 7fae0e7fc700 1 -- 172.21.15.7:0/3573542655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fae1806ddd0 con 0x7fae1815d920 2024-01-11T07:07:21.627 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fadf8066f30 msgr2=0x7fadf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fadf8066f30 0x7fadf80693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae08009d10 tx=0x7fae080091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815d920 msgr2=0x7fae1806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815d920 0x7fae1806e490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae1806fdf0 tx=0x7fae10018000 comp rx=0 tx=0).stop 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 shutdown_connections 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fadf8066f30 0x7fadf80693e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1815d920 0x7fae1806e490 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae180ac3a0 0x7fae1815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 --2- 172.21.15.7:0/3573542655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae1815a2c0 0x7fae1815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 >> 172.21.15.7:0/3573542655 conn(0x7fae180cd090 msgr2=0x7fae180aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 shutdown_connections 2024-01-11T07:07:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:21.629+0000 7fae1ccd2700 1 -- 172.21.15.7:0/3573542655 wait complete. 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:07:21.643 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-11T07:07:21.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:21.644 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:07:21.663 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:21.663 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:21.663 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:07:21.664 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42376sec preferred_lft 42376sec 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:07:21.665 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:07:21.666 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link tentative 2024-01-11T07:07:21.666 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:21.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:21.666 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:07:21.734 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:07:21.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:21.735 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:07:21.800 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:07:21.800 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:07:21.800 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:07:21.800 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:07:21.801 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:07:21.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:21.801 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:21.801 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:21.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:07:21.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:07:21.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:07:21.801 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:21.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:21.925 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:07:21.944 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:07:21.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:21.945 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-01-11T07:07:22.007 INFO:teuthology.orchestra.run.smithi007.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-11T07:07:22.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:22.007 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:22.007 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:22.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:07:22.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-11T07:07:22.007 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:22.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:22.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether c2:09:34:2e:67:62 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-11T07:07:22.162 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:22.163 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c009:34ff:fe2e:6762/64 scope link tentative 2024-01-11T07:07:22.163 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:22.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:22.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-11T07:07:22.225 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:22.225 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether c2:09:34:2e:67:62 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c009:34ff:fe2e:6762/64 scope link tentative 2024-01-11T07:07:22.226 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:22.238 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-11T07:07:22.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:22.238 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:22.238 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:22.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-11T07:07:22.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-11T07:07:22.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-11T07:07:22.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-11T07:07:22.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-11T07:07:22.239 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:22.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:22.491 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:22.491 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:22.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.1 up 2024-01-11T07:07:22.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.1 master ceph-brx 2024-01-11T07:07:22.491 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:22.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:22.591 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-01-11T07:07:22.610 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-01-11T07:07:22.610 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:22.610 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-01-11T07:07:22.671 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:07:22.671 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:07:22.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-11T07:07:22.764 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-11T07:07:22.764 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-11T07:07:22.764 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:22.774 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-11T07:07:22.774 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-11T07:07:22.774 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:22.774 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:22.774 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:22.792 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-11T07:07:22.792 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-11T07:07:22.792 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:22.792 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:22.793 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:22.793 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:07:22.793 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:07:22.793 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:07:22.795 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:07:22.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:22.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-11T07:07:22.827 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-01-11T07:07:22.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:07:23.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.007+0000 7f0b50061700 1 Processor -- start 2024-01-11T07:07:23.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.007+0000 7f0b50061700 1 -- start start 2024-01-11T07:07:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b50061700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b480c1cc0 0x7f0b480b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b50061700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b480ca570 0x7f0b480cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:23.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b50061700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b480b1350 con 0x7f0b480cc060 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b50061700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4815e850 con 0x7f0b480c1cc0 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b50061700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b480cb3f0 con 0x7f0b480ca570 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b4ddfd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b480c1cc0 0x7f0b480b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b4ddfd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b480c1cc0 0x7f0b480b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44292/0 (socket says 172.21.15.7:44292) 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b4ddfd700 1 -- 172.21.15.7:0/244773063 learned_addr learned my addr 172.21.15.7:0/244773063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.008+0000 7f0b4d5fc700 1 --2- 172.21.15.7:0/244773063 >> v2:172.21.15.113:3300/0 conn(0x7f0b480ca570 0x7f0b480cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.009+0000 7f0b4ddfd700 1 -- 172.21.15.7:0/244773063 >> v2:172.21.15.113:3300/0 conn(0x7f0b480ca570 msgr2=0x7f0b480cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.009+0000 7f0b4ddfd700 1 --2- 172.21.15.7:0/244773063 >> v2:172.21.15.113:3300/0 conn(0x7f0b480ca570 0x7f0b480cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.009+0000 7f0b4ddfd700 1 -- 172.21.15.7:0/244773063 >> v1:172.21.15.113:6789/0 conn(0x7f0b480cc060 legacy=0x7f0b480cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.009+0000 7f0b4ddfd700 1 -- 172.21.15.7:0/244773063 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b480cb690 con 0x7f0b480c1cc0 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.009+0000 7f0b4ddfd700 1 --2- 172.21.15.7:0/244773063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b480b0e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b440099b0 tx=0x7f0b44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488f205036e99826 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.009+0000 7f0b4cdfb700 1 -- 172.21.15.7:0/244773063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44009ea0 con 0x7f0b480c1cc0 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.009+0000 7f0b4cdfb700 1 -- 172.21.15.7:0/244773063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4400b8b0 con 0x7f0b480c1cc0 2024-01-11T07:07:23.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.009+0000 7f0b4cdfb700 1 -- 172.21.15.7:0/244773063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400bc50 con 0x7f0b480c1cc0 2024-01-11T07:07:23.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.010+0000 7f0b50061700 1 -- 172.21.15.7:0/244773063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 msgr2=0x7f0b480b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:23.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.010+0000 7f0b50061700 1 --2- 172.21.15.7:0/244773063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b480b0e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b440099b0 tx=0x7f0b44009250 comp rx=0 tx=0).stop 2024-01-11T07:07:23.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.010+0000 7f0b50061700 1 -- 172.21.15.7:0/244773063 shutdown_connections 2024-01-11T07:07:23.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.010+0000 7f0b50061700 1 --2- 172.21.15.7:0/244773063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b480b0e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.010+0000 7f0b50061700 1 --2- 172.21.15.7:0/244773063 >> v2:172.21.15.113:3300/0 conn(0x7f0b480ca570 0x7f0b480cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.010+0000 7f0b50061700 1 -- 172.21.15.7:0/244773063 >> 172.21.15.7:0/244773063 conn(0x7f0b48063230 msgr2=0x7f0b48063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:23.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.010+0000 7f0b50061700 1 -- 172.21.15.7:0/244773063 shutdown_connections 2024-01-11T07:07:23.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.010+0000 7f0b50061700 1 -- 172.21.15.7:0/244773063 wait complete. 2024-01-11T07:07:23.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.011+0000 7f0b50061700 1 Processor -- start 2024-01-11T07:07:23.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.011+0000 7f0b50061700 1 -- start start 2024-01-11T07:07:23.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.012+0000 7f0b50061700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b4815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:23.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.012+0000 7f0b50061700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b480ca570 0x7f0b480c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:23.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.012+0000 7f0b50061700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b480cc060 0x7f0b480c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:23.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.012+0000 7f0b50061700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480c2a40 con 0x7f0b480c1cc0 2024-01-11T07:07:23.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.012+0000 7f0b50061700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480c5a90 con 0x7f0b480cc060 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.012+0000 7f0b50061700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4815cba0 con 0x7f0b480ca570 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b4815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b4815c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44296/0 (socket says 172.21.15.7:44296) 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 -- 172.21.15.7:0/3434144686 learned_addr learned my addr 172.21.15.7:0/3434144686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4d5fc700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b480ca570 0x7f0b480c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4e5fe700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b480cc060 0x7f0b480c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 -- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b480ca570 msgr2=0x7f0b480c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b480ca570 0x7f0b480c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 -- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b480cc060 msgr2=0x7f0b480c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b480cc060 0x7f0b480c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 -- 172.21.15.7:0/3434144686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b440094f0 con 0x7f0b480c1cc0 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4d5fc700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b480ca570 0x7f0b480c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4e5fe700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b480cc060 0x7f0b480c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:23.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b4ddfd700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b4815c660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0b440094c0 tx=0x7f0b44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:23.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b39ffb700 1 -- 172.21.15.7:0/3434144686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44024440 con 0x7f0b480c1cc0 2024-01-11T07:07:23.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b39ffb700 1 -- 172.21.15.7:0/3434144686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b440245a0 con 0x7f0b480c1cc0 2024-01-11T07:07:23.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b480c2c20 con 0x7f0b480c1cc0 2024-01-11T07:07:23.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.013+0000 7f0b39ffb700 1 -- 172.21.15.7:0/3434144686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44024960 con 0x7f0b480c1cc0 2024-01-11T07:07:23.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.014+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b480c31e0 con 0x7f0b480c1cc0 2024-01-11T07:07:23.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.015+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4804c3c0 con 0x7f0b480c1cc0 2024-01-11T07:07:23.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.015+0000 7f0b39ffb700 1 -- 172.21.15.7:0/3434144686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0b4401a420 con 0x7f0b480c1cc0 2024-01-11T07:07:23.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.015+0000 7f0b39ffb700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b2c066f30 0x7f0b2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:23.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.015+0000 7f0b39ffb700 1 -- 172.21.15.7:0/3434144686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0b440861b0 con 0x7f0b480c1cc0 2024-01-11T07:07:23.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.016+0000 7f0b4d5fc700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b2c066f30 0x7f0b2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:23.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.016+0000 7f0b4d5fc700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b2c066f30 0x7f0b2c0693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b480ca980 tx=0x7f0b3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:23.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.020+0000 7f0b39ffb700 1 -- 172.21.15.7:0/3434144686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b44053420 con 0x7f0b480c1cc0 2024-01-11T07:07:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.274+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0b48069e50 con 0x7f0b480c1cc0 2024-01-11T07:07:23.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.275+0000 7f0b39ffb700 1 -- 172.21.15.7:0/3434144686 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0b44056a40 con 0x7f0b480c1cc0 2024-01-11T07:07:23.275 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:07:23.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b2c066f30 msgr2=0x7f0b2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b2c066f30 0x7f0b2c0693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b480ca980 tx=0x7f0b3c008040 comp rx=0 tx=0).stop 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 msgr2=0x7f0b4815c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b4815c660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0b440094c0 tx=0x7f0b44017000 comp rx=0 tx=0).stop 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 shutdown_connections 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b2c066f30 0x7f0b2c0693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b480cc060 0x7f0b480c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b480c1cc0 0x7f0b4815c660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 --2- 172.21.15.7:0/3434144686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b480ca570 0x7f0b480c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 >> 172.21.15.7:0/3434144686 conn(0x7f0b48063230 msgr2=0x7f0b480cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 shutdown_connections 2024-01-11T07:07:23.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:23.277+0000 7f0b50061700 1 -- 172.21.15.7:0/3434144686 wait complete. 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:07:23.291 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:07:23.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.292 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:07:23.312 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:07:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:07:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42417sec preferred_lft 42417sec 2024-01-11T07:07:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:07:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:23.314 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-11T07:07:23.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.314 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:23.314 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:23.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add name ceph-brx type bridge 2024-01-11T07:07:23.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr flush dev ceph-brx 2024-01-11T07:07:23.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set ceph-brx up 2024-01-11T07:07:23.314 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-11T07:07:23.315 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:23.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.483 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:07:23.511 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:07:23.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.513 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:07:23.584 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:07:23.584 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:07:23.584 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:07:23.584 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:07:23.585 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:07:23.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.585 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:23.585 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:23.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:07:23.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:07:23.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:07:23.585 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:23.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.816 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:07:23.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.833 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-01-11T07:07:23.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.889 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:23.889 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:23.890 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:07:23.890 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-11T07:07:23.890 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:23.987 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-11T07:07:23.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:23.988 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:23.988 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:23.988 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-11T07:07:23.988 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-11T07:07:23.988 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-11T07:07:23.988 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-11T07:07:23.988 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-11T07:07:23.988 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:24.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:24.220 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:24.220 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:24.220 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:07:24.220 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.0 master ceph-brx 2024-01-11T07:07:24.221 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:24.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:24.318 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:07:24.335 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:07:24.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:24.336 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:07:24.392 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:07:24.392 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:07:24.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:07:24.474 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:07:24.475 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:07:24.475 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:24.559 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:07:24.559 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:07:24.559 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:24.559 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:24.559 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:24.576 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:07:24.576 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:07:24.576 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:24.576 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:24.576 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:24.576 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:07:24.576 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:07:24.576 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:07:24.579 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:07:24.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:24.579 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:07:24.645 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-01-11T07:07:24.645 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:07:24.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.784+0000 7f4d8d0b7700 1 Processor -- start 2024-01-11T07:07:24.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.784+0000 7f4d8d0b7700 1 -- start start 2024-01-11T07:07:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.784+0000 7f4d8d0b7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806afc0 0x7f4d8815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.784+0000 7f4d8d0b7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8815b770 con 0x7f4d8806e0d0 2024-01-11T07:07:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.784+0000 7f4d8d0b7700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8815abf0 con 0x7f4d8806dd00 2024-01-11T07:07:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.784+0000 7f4d8d0b7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8815aed0 con 0x7f4d8806afc0 2024-01-11T07:07:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d8659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4d8806e0d0 0x7f4d88159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51694/0 (socket says 172.21.15.7:51694) 2024-01-11T07:07:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d8659c700 1 -- 172.21.15.7:0/2193218315 learned_addr learned my addr 172.21.15.7:0/2193218315 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d8759e700 1 --2- 172.21.15.7:0/2193218315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806afc0 0x7f4d8815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d85d9b700 1 -- 172.21.15.7:0/2193218315 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320435422 0 0) 0x7f4d8815b770 con 0x7f4d8806e0d0 2024-01-11T07:07:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d85d9b700 1 -- 172.21.15.7:0/2193218315 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d70003570 con 0x7f4d8806e0d0 2024-01-11T07:07:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d8759e700 1 -- 172.21.15.7:0/2193218315 >> v1:172.21.15.113:6789/0 conn(0x7f4d8806dd00 legacy=0x7f4d880c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d8759e700 1 -- 172.21.15.7:0/2193218315 >> v1:172.21.15.7:6789/0 conn(0x7f4d8806e0d0 legacy=0x7f4d88159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d8759e700 1 -- 172.21.15.7:0/2193218315 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8815b1b0 con 0x7f4d8806afc0 2024-01-11T07:07:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.785+0000 7f4d8759e700 1 --2- 172.21.15.7:0/2193218315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806afc0 0x7f4d8815a1c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009e90 tx=0x7f4d80009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b52a187aea1318 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d85d9b700 1 -- 172.21.15.7:0/2193218315 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000e070 con 0x7f4d8806afc0 2024-01-11T07:07:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d85d9b700 1 -- 172.21.15.7:0/2193218315 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8000bc30 con 0x7f4d8806afc0 2024-01-11T07:07:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d85d9b700 1 -- 172.21.15.7:0/2193218315 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d80012430 con 0x7f4d8806afc0 2024-01-11T07:07:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2193218315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806afc0 msgr2=0x7f4d8815a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d8d0b7700 1 --2- 172.21.15.7:0/2193218315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806afc0 0x7f4d8815a1c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009e90 tx=0x7f4d80009780 comp rx=0 tx=0).stop 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2193218315 shutdown_connections 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d8d0b7700 1 --2- 172.21.15.7:0/2193218315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806afc0 0x7f4d8815a1c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2193218315 >> 172.21.15.7:0/2193218315 conn(0x7f4d880680e0 msgr2=0x7f4d880cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2193218315 shutdown_connections 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.786+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2193218315 wait complete. 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.787+0000 7f4d8d0b7700 1 Processor -- start 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8d0b7700 1 -- start start 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8d0b7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8806afc0 0x7f4d8814f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8d0b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d8806dd00 0x7f4d8814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8d0b7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806e0d0 0x7f4d881584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8d0b7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8814ef20 con 0x7f4d8806dd00 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8d0b7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88153dc0 con 0x7f4d8806afc0 2024-01-11T07:07:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8d0b7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8814e910 con 0x7f4d8806e0d0 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d8806dd00 0x7f4d8814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d8806dd00 0x7f4d8814cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44300/0 (socket says 172.21.15.7:44300) 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.788+0000 7f4d8659c700 1 -- 172.21.15.7:0/2085525243 learned_addr learned my addr 172.21.15.7:0/2085525243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8759e700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806e0d0 0x7f4d881584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d86d9d700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8806afc0 0x7f4d8814f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8659c700 1 -- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806e0d0 msgr2=0x7f4d881584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8659c700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806e0d0 0x7f4d881584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8659c700 1 -- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8806afc0 msgr2=0x7f4d8814f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8659c700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8806afc0 0x7f4d8814f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8659c700 1 -- 172.21.15.7:0/2085525243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80009a20 con 0x7f4d8806dd00 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8659c700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d8806dd00 0x7f4d8814cdd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00c590 tx=0x7f4d7c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d6effd700 1 -- 172.21.15.7:0/2085525243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c005b30 con 0x7f4d8806dd00 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d6effd700 1 -- 172.21.15.7:0/2085525243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c005c90 con 0x7f4d8806dd00 2024-01-11T07:07:24.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d88153ae0 con 0x7f4d8806dd00 2024-01-11T07:07:24.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d6effd700 1 -- 172.21.15.7:0/2085525243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c007320 con 0x7f4d8806dd00 2024-01-11T07:07:24.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.789+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8814d310 con 0x7f4d8806dd00 2024-01-11T07:07:24.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.790+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8814f090 con 0x7f4d8806dd00 2024-01-11T07:07:24.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.791+0000 7f4d6effd700 1 -- 172.21.15.7:0/2085525243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4d7c01b3b0 con 0x7f4d8806dd00 2024-01-11T07:07:24.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.791+0000 7f4d6effd700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d74066f30 0x7f4d740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.791+0000 7f4d6effd700 1 -- 172.21.15.7:0/2085525243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d7c083910 con 0x7f4d8806dd00 2024-01-11T07:07:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.791+0000 7f4d86d9d700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d74066f30 0x7f4d740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.792+0000 7f4d86d9d700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d74066f30 0x7f4d740693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d78003ac0 tx=0x7f4d78007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:24.796+0000 7f4d6effd700 1 -- 172.21.15.7:0/2085525243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d7c050b80 con 0x7f4d8806dd00 2024-01-11T07:07:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.047+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d8804c3c0 con 0x7f4d8806dd00 2024-01-11T07:07:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.047+0000 7f4d6effd700 1 -- 172.21.15.7:0/2085525243 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d7c017020 con 0x7f4d8806dd00 2024-01-11T07:07:25.048 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d74066f30 msgr2=0x7f4d740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d74066f30 0x7f4d740693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d78003ac0 tx=0x7f4d78007ea0 comp rx=0 tx=0).stop 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d8806dd00 msgr2=0x7f4d8814cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d8806dd00 0x7f4d8814cdd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00c590 tx=0x7f4d7c007e00 comp rx=0 tx=0).stop 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 shutdown_connections 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d74066f30 0x7f4d740693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d8806e0d0 0x7f4d881584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d8806afc0 0x7f4d8814f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 --2- 172.21.15.7:0/2085525243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d8806dd00 0x7f4d8814cdd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 >> 172.21.15.7:0/2085525243 conn(0x7f4d880680e0 msgr2=0x7f4d88063c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 shutdown_connections 2024-01-11T07:07:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:25.050+0000 7f4d8d0b7700 1 -- 172.21.15.7:0/2085525243 wait complete. 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:07:25.064 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-11T07:07:25.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.064 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:07:25.083 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:07:25.084 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42415sec preferred_lft 42415sec 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:07:25.085 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.086 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:07:25.086 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:07:25.086 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link tentative 2024-01-11T07:07:25.086 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.086 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:07:25.150 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:07:25.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.151 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:07:25.211 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:07:25.211 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:07:25.211 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:07:25.211 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:07:25.211 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:07:25.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.211 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:25.211 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:25.211 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:07:25.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:07:25.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:07:25.212 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:25.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.318 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:07:25.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:07:25.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.336 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-01-11T07:07:25.392 INFO:teuthology.orchestra.run.smithi113.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-11T07:07:25.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.393 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:25.393 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:25.393 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:07:25.393 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-11T07:07:25.393 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:25.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.486 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-11T07:07:25.524 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:25.524 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether d2:75:70:a2:80:e8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d075:70ff:fea2:80e8/64 scope link tentative 2024-01-11T07:07:25.525 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.537 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-11T07:07:25.618 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:25.618 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:25.618 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:25.618 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.618 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:25.618 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.618 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:07:25.619 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether d2:75:70:a2:80:e8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-11T07:07:25.619 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-11T07:07:25.619 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.619 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::d075:70ff:fea2:80e8/64 scope link tentative 2024-01-11T07:07:25.619 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:25.631 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-11T07:07:25.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.631 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:25.631 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:25.632 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-11T07:07:25.632 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-11T07:07:25.632 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-11T07:07:25.632 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-11T07:07:25.632 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-11T07:07:25.632 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:25.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:25.870 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:25.870 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:25.870 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.1 up 2024-01-11T07:07:25.870 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.1 master ceph-brx 2024-01-11T07:07:25.870 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:25.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:25.956 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-01-11T07:07:25.972 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-01-11T07:07:25.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:25.973 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-01-11T07:07:26.029 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:07:26.029 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:07:26.029 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-11T07:07:26.111 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-11T07:07:26.112 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-11T07:07:26.112 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:26.120 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-11T07:07:26.120 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-11T07:07:26.120 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:26.120 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:26.120 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:26.137 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-11T07:07:26.137 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-11T07:07:26.137 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:26.137 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:26.137 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:26.137 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:07:26.138 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:07:26.138 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:07:26.139 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:07:26.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:26.140 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-11T07:07:26.206 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-11T07:07:26.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:26.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.392+0000 7eff70c93700 1 Processor -- start 2024-01-11T07:07:26.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.392+0000 7eff70c93700 1 -- start start 2024-01-11T07:07:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff70c93700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff6c0aff20 0x7eff6c0aab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff70c93700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff6c063680 0x7eff6c06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff70c93700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6c0ab040 con 0x7eff6c063230 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff70c93700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c0b02f0 con 0x7eff6c063680 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff70c93700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c154860 con 0x7eff6c0aff20 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff69d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7eff6c063230 0x7eff6c06dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51706/0 (socket says 172.21.15.7:51706) 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff69d9b700 1 -- 172.21.15.7:0/265647582 learned_addr learned my addr 172.21.15.7:0/265647582 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff6ad9d700 1 --2- 172.21.15.7:0/265647582 >> v2:172.21.15.7:3300/0 conn(0x7eff6c063680 0x7eff6c06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.393+0000 7eff6a59c700 1 --2- 172.21.15.7:0/265647582 >> v2:172.21.15.113:3300/0 conn(0x7eff6c0aff20 0x7eff6c0aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6959a700 1 -- 172.21.15.7:0/265647582 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038196969 0 0) 0x7eff6c0ab040 con 0x7eff6c063230 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6959a700 1 -- 172.21.15.7:0/265647582 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff54003570 con 0x7eff6c063230 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6ad9d700 1 -- 172.21.15.7:0/265647582 >> v2:172.21.15.113:3300/0 conn(0x7eff6c0aff20 msgr2=0x7eff6c0aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6ad9d700 1 --2- 172.21.15.7:0/265647582 >> v2:172.21.15.113:3300/0 conn(0x7eff6c0aff20 0x7eff6c0aab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6ad9d700 1 -- 172.21.15.7:0/265647582 >> v1:172.21.15.7:6789/0 conn(0x7eff6c063230 legacy=0x7eff6c06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6ad9d700 1 -- 172.21.15.7:0/265647582 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c15d910 con 0x7eff6c063680 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6a59c700 1 --2- 172.21.15.7:0/265647582 >> v2:172.21.15.113:3300/0 conn(0x7eff6c0aff20 0x7eff6c0aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6ad9d700 1 --2- 172.21.15.7:0/265647582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c06e1f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06d9e0 tx=0x7eff64009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1861f1252c57bc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6959a700 1 -- 172.21.15.7:0/265647582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6400e070 con 0x7eff6c063680 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6959a700 1 -- 172.21.15.7:0/265647582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6400bab0 con 0x7eff6c063680 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.394+0000 7eff6959a700 1 -- 172.21.15.7:0/265647582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6400be70 con 0x7eff6c063680 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.395+0000 7eff70c93700 1 -- 172.21.15.7:0/265647582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 msgr2=0x7eff6c06e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.395+0000 7eff70c93700 1 --2- 172.21.15.7:0/265647582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c06e1f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06d9e0 tx=0x7eff64009780 comp rx=0 tx=0).stop 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.395+0000 7eff70c93700 1 -- 172.21.15.7:0/265647582 shutdown_connections 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.395+0000 7eff70c93700 1 --2- 172.21.15.7:0/265647582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c06e1f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.395+0000 7eff70c93700 1 --2- 172.21.15.7:0/265647582 >> v2:172.21.15.113:3300/0 conn(0x7eff6c0aff20 0x7eff6c0aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.395+0000 7eff70c93700 1 -- 172.21.15.7:0/265647582 >> 172.21.15.7:0/265647582 conn(0x7eff6c068090 msgr2=0x7eff6c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:26.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.395+0000 7eff70c93700 1 -- 172.21.15.7:0/265647582 shutdown_connections 2024-01-11T07:07:26.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.395+0000 7eff70c93700 1 -- 172.21.15.7:0/265647582 wait complete. 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.396+0000 7eff70c93700 1 Processor -- start 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.396+0000 7eff70c93700 1 -- start start 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff70c93700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6c063230 0x7eff6c0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff70c93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff70c93700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff6c0aff20 0x7eff6c06b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff70c93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0b24a0 con 0x7eff6c063680 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff70c93700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0b1890 con 0x7eff6c0aff20 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff70c93700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0b1b70 con 0x7eff6c063230 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff69d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6c063230 0x7eff6c0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff69d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44314/0 (socket says 172.21.15.7:44314) 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff69d9b700 1 -- 172.21.15.7:0/2740684919 learned_addr learned my addr 172.21.15.7:0/2740684919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff6ad9d700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff6c0aff20 0x7eff6c06b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff6a59c700 1 -- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff6c0aff20 msgr2=0x7eff6c06b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff6a59c700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff6c0aff20 0x7eff6c06b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff6a59c700 1 -- 172.21.15.7:0/2740684919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 msgr2=0x7eff6c153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff6a59c700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff6a59c700 1 -- 172.21.15.7:0/2740684919 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c009420 con 0x7eff6c063230 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff6ad9d700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff6c0aff20 0x7eff6c06b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.397+0000 7eff69d9b700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.398+0000 7eff6a59c700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6c063230 0x7eff6c0b1060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00ea30 tx=0x7eff5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.398+0000 7eff5a7fc700 1 -- 172.21.15.7:0/2740684919 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00ee70 con 0x7eff6c063230 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.398+0000 7eff5a7fc700 1 -- 172.21.15.7:0/2740684919 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c00c850 con 0x7eff6c063230 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.398+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff64009a20 con 0x7eff6c063230 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.398+0000 7eff5a7fc700 1 -- 172.21.15.7:0/2740684919 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00cbf0 con 0x7eff6c063230 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.398+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c0b2130 con 0x7eff6c063230 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.399+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c04c3c0 con 0x7eff6c063230 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.399+0000 7eff5a7fc700 1 -- 172.21.15.7:0/2740684919 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7eff5c013470 con 0x7eff6c063230 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.400+0000 7eff5a7fc700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff4c066f30 0x7eff4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.400+0000 7eff5a7fc700 1 -- 172.21.15.7:0/2740684919 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff5c084690 con 0x7eff6c063230 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.400+0000 7eff69d9b700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff4c066f30 0x7eff4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.400+0000 7eff69d9b700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff4c066f30 0x7eff4c0693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06be20 tx=0x7eff60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.405+0000 7eff5a7fc700 1 -- 172.21.15.7:0/2740684919 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff5c051900 con 0x7eff6c063230 2024-01-11T07:07:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.610+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff6c15dbf0 con 0x7eff6c063230 2024-01-11T07:07:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.611+0000 7eff5a7fc700 1 -- 172.21.15.7:0/2740684919 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7eff5c054f20 con 0x7eff6c063230 2024-01-11T07:07:26.612 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:26.612 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff4c066f30 msgr2=0x7eff4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff4c066f30 0x7eff4c0693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06be20 tx=0x7eff60009040 comp rx=0 tx=0).stop 2024-01-11T07:07:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6c063230 msgr2=0x7eff6c0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6c063230 0x7eff6c0b1060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00ea30 tx=0x7eff5c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 shutdown_connections 2024-01-11T07:07:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff4c066f30 0x7eff4c0693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff6c0aff20 0x7eff6c06b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff6c063230 0x7eff6c0b1060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 --2- 172.21.15.7:0/2740684919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff6c063680 0x7eff6c153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 >> 172.21.15.7:0/2740684919 conn(0x7eff6c068090 msgr2=0x7eff6c0c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 shutdown_connections 2024-01-11T07:07:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.613+0000 7eff70c93700 1 -- 172.21.15.7:0/2740684919 wait complete. 2024-01-11T07:07:26.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:06:56.038804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:06:57.365958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:07:26.628 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:07:26.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.762+0000 7fb81dde4700 1 Processor -- start 2024-01-11T07:07:26.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.762+0000 7fb81dde4700 1 -- start start 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.763+0000 7fb81dde4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb818159d10 0x7fb818068c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.763+0000 7fb81dde4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.763+0000 7fb81dde4700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81815dbb0 con 0x7fb81806f1d0 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.763+0000 7fb81dde4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81815d9a0 con 0x7fb818159d10 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.763+0000 7fb81dde4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81806f650 con 0x7fb8180691d0 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.763+0000 7fb8177fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb818159d10 0x7fb818068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.763+0000 7fb816ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb817fff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb81806f1d0 0x7fb8180b3b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:52348/0 (socket says 172.21.15.7:52348) 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb817fff700 1 -- 172.21.15.7:0/1312891871 learned_addr learned my addr 172.21.15.7:0/1312891871 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb816ffd700 1 -- 172.21.15.7:0/1312891871 >> v2:172.21.15.113:3300/0 conn(0x7fb818159d10 msgr2=0x7fb818068c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb816ffd700 1 --2- 172.21.15.7:0/1312891871 >> v2:172.21.15.113:3300/0 conn(0x7fb818159d10 0x7fb818068c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb816ffd700 1 -- 172.21.15.7:0/1312891871 >> v1:172.21.15.113:6789/0 conn(0x7fb81806f1d0 legacy=0x7fb8180b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb816ffd700 1 -- 172.21.15.7:0/1312891871 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81815de90 con 0x7fb8180691d0 2024-01-11T07:07:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb8177fe700 1 --2- 172.21.15.7:0/1312891871 >> v2:172.21.15.113:3300/0 conn(0x7fb818159d10 0x7fb818068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb816ffd700 1 --2- 172.21.15.7:0/1312891871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81806ec90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0098e0 tx=0x7fb80c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fcc2df52c8b5254 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb8167fc700 1 -- 172.21.15.7:0/1312891871 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00b640 con 0x7fb8180691d0 2024-01-11T07:07:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb8167fc700 1 -- 172.21.15.7:0/1312891871 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c00b7a0 con 0x7fb8180691d0 2024-01-11T07:07:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.764+0000 7fb8167fc700 1 -- 172.21.15.7:0/1312891871 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00bb40 con 0x7fb8180691d0 2024-01-11T07:07:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.765+0000 7fb81dde4700 1 -- 172.21.15.7:0/1312891871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 msgr2=0x7fb81806ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.765+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1312891871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81806ec90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0098e0 tx=0x7fb80c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.765+0000 7fb81dde4700 1 -- 172.21.15.7:0/1312891871 shutdown_connections 2024-01-11T07:07:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.765+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1312891871 >> v2:172.21.15.113:3300/0 conn(0x7fb818159d10 0x7fb818068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.765+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1312891871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81806ec90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.765+0000 7fb81dde4700 1 -- 172.21.15.7:0/1312891871 >> 172.21.15.7:0/1312891871 conn(0x7fb8180ce9f0 msgr2=0x7fb8180cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.765+0000 7fb81dde4700 1 -- 172.21.15.7:0/1312891871 shutdown_connections 2024-01-11T07:07:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.765+0000 7fb81dde4700 1 -- 172.21.15.7:0/1312891871 wait complete. 2024-01-11T07:07:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.767+0000 7fb81dde4700 1 Processor -- start 2024-01-11T07:07:26.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.767+0000 7fb81dde4700 1 -- start start 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb81dde4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb81dde4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb81806f1d0 0x7fb8181580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb81dde4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb818159d10 0x7fb81815ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb81dde4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180c7250 con 0x7fb818159d10 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb81dde4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818158600 con 0x7fb81806f1d0 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb81dde4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81815d3e0 con 0x7fb8180691d0 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb817fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb818159d10 0x7fb81815ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb8177fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb8177fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81815c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:57360/0 (socket says 172.21.15.7:57360) 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb8177fe700 1 -- 172.21.15.7:0/1418548399 learned_addr learned my addr 172.21.15.7:0/1418548399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb816ffd700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb81806f1d0 0x7fb8181580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb8177fe700 1 -- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb81806f1d0 msgr2=0x7fb8181580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb8177fe700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb81806f1d0 0x7fb8181580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb8177fe700 1 -- 172.21.15.7:0/1418548399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb818159d10 msgr2=0x7fb81815ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb8177fe700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb818159d10 0x7fb81815ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb8177fe700 1 -- 172.21.15.7:0/1418548399 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb808009420 con 0x7fb8180691d0 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.768+0000 7fb817fff700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb818159d10 0x7fb81815ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.769+0000 7fb8177fe700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81815c7b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb80800ec80 tx=0x7fb8080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.769+0000 7fb7ff7fe700 1 -- 172.21.15.7:0/1418548399 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80800c850 con 0x7fb8180691d0 2024-01-11T07:07:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.769+0000 7fb7ff7fe700 1 -- 172.21.15.7:0/1418548399 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80800c9b0 con 0x7fb8180691d0 2024-01-11T07:07:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.769+0000 7fb7ff7fe700 1 -- 172.21.15.7:0/1418548399 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80800cd30 con 0x7fb8180691d0 2024-01-11T07:07:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.769+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c009420 con 0x7fb8180691d0 2024-01-11T07:07:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.769+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81815d100 con 0x7fb8180691d0 2024-01-11T07:07:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.770+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb818069770 con 0x7fb8180691d0 2024-01-11T07:07:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.770+0000 7fb7ff7fe700 1 -- 172.21.15.7:0/1418548399 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb808013680 con 0x7fb8180691d0 2024-01-11T07:07:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.771+0000 7fb7ff7fe700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb804066f30 0x7fb8040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.771+0000 7fb7ff7fe700 1 -- 172.21.15.7:0/1418548399 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb808084740 con 0x7fb8180691d0 2024-01-11T07:07:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.771+0000 7fb816ffd700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb804066f30 0x7fb8040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.771+0000 7fb816ffd700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb804066f30 0x7fb8040693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0093f0 tx=0x7fb80c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:26.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.776+0000 7fb7ff7fe700 1 -- 172.21.15.7:0/1418548399 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8080519b0 con 0x7fb8180691d0 2024-01-11T07:07:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.978+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb81804c3c0 con 0x7fb8180691d0 2024-01-11T07:07:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.979+0000 7fb7ff7fe700 1 -- 172.21.15.7:0/1418548399 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb808054fd0 con 0x7fb8180691d0 2024-01-11T07:07:26.979 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb804066f30 msgr2=0x7fb8040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb804066f30 0x7fb8040693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0093f0 tx=0x7fb80c017000 comp rx=0 tx=0).stop 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 msgr2=0x7fb81815c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81815c7b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb80800ec80 tx=0x7fb8080091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 shutdown_connections 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb804066f30 0x7fb8040693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb818159d10 0x7fb81815ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8180691d0 0x7fb81815c7b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 --2- 172.21.15.7:0/1418548399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb81806f1d0 0x7fb8181580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 >> 172.21.15.7:0/1418548399 conn(0x7fb8180ce9f0 msgr2=0x7fb818158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 shutdown_connections 2024-01-11T07:07:26.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:26.981+0000 7fb81dde4700 1 -- 172.21.15.7:0/1418548399 wait complete. 2024-01-11T07:07:26.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:07:27.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.130+0000 7fc1bce00700 1 Processor -- start 2024-01-11T07:07:27.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.130+0000 7fc1bce00700 1 -- start start 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1bce00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1b815db90 0x7fc1b815df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1bce00700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1b80c3690 0x7fc1b806e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1bce00700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b815e4a0 con 0x7fc1b806ec70 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1bce00700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b806d3b0 con 0x7fc1b815db90 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1bce00700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b806d690 con 0x7fc1b80c3690 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1b815db90 0x7fc1b815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1b815db90 0x7fc1b815df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44326/0 (socket says 172.21.15.7:44326) 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b6d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc1b806ec70 0x7fc1b80c9010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51714/0 (socket says 172.21.15.7:51714) 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b659c700 1 -- 172.21.15.7:0/806809648 learned_addr learned my addr 172.21.15.7:0/806809648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b5d9b700 1 --2- 172.21.15.7:0/806809648 >> v2:172.21.15.113:3300/0 conn(0x7fc1b80c3690 0x7fc1b806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b659c700 1 -- 172.21.15.7:0/806809648 >> v2:172.21.15.113:3300/0 conn(0x7fc1b80c3690 msgr2=0x7fc1b806e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b659c700 1 --2- 172.21.15.7:0/806809648 >> v2:172.21.15.113:3300/0 conn(0x7fc1b80c3690 0x7fc1b806e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b659c700 1 -- 172.21.15.7:0/806809648 >> v1:172.21.15.7:6789/0 conn(0x7fc1b806ec70 legacy=0x7fc1b80c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b659c700 1 -- 172.21.15.7:0/806809648 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b806d970 con 0x7fc1b815db90 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b559a700 1 -- 172.21.15.7:0/806809648 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495962484 0 0) 0x7fc1b815e4a0 con 0x7fc1b806ec70 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b659c700 1 --2- 172.21.15.7:0/806809648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b815db90 0x7fc1b815df60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8070ef0 tx=0x7fc1a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67977e65f77a6dfd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.131+0000 7fc1b559a700 1 -- 172.21.15.7:0/806809648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800b860 con 0x7fc1b815db90 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1b559a700 1 -- 172.21.15.7:0/806809648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a800b9c0 con 0x7fc1b815db90 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1b559a700 1 -- 172.21.15.7:0/806809648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800bd60 con 0x7fc1b815db90 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1bce00700 1 -- 172.21.15.7:0/806809648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b815db90 msgr2=0x7fc1b815df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1bce00700 1 --2- 172.21.15.7:0/806809648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b815db90 0x7fc1b815df60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8070ef0 tx=0x7fc1a8009250 comp rx=0 tx=0).stop 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1bce00700 1 -- 172.21.15.7:0/806809648 shutdown_connections 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1bce00700 1 --2- 172.21.15.7:0/806809648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b815db90 0x7fc1b815df60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1bce00700 1 --2- 172.21.15.7:0/806809648 >> v2:172.21.15.113:3300/0 conn(0x7fc1b80c3690 0x7fc1b806e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1bce00700 1 -- 172.21.15.7:0/806809648 >> 172.21.15.7:0/806809648 conn(0x7fc1b80cd070 msgr2=0x7fc1b80cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1bce00700 1 -- 172.21.15.7:0/806809648 shutdown_connections 2024-01-11T07:07:27.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.132+0000 7fc1bce00700 1 -- 172.21.15.7:0/806809648 wait complete. 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.133+0000 7fc1bce00700 1 Processor -- start 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1bce00700 1 -- start start 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1bce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b806ec70 0x7fc1b80c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1bce00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80c3690 0x7fc1b814ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1bce00700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b815db90 0x7fc1b809a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1bce00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80c4e60 con 0x7fc1b806ec70 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1bce00700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80c4250 con 0x7fc1b80c3690 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1bce00700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80c4530 con 0x7fc1b815db90 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b806ec70 0x7fc1b80c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b806ec70 0x7fc1b80c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44338/0 (socket says 172.21.15.7:44338) 2024-01-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.134+0000 7fc1b659c700 1 -- 172.21.15.7:0/3110902449 learned_addr learned my addr 172.21.15.7:0/3110902449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1b5d9b700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80c3690 0x7fc1b814ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1b6d9d700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b815db90 0x7fc1b809a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1b659c700 1 -- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b815db90 msgr2=0x7fc1b809a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1b659c700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b815db90 0x7fc1b809a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1b659c700 1 -- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80c3690 msgr2=0x7fc1b814ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1b659c700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80c3690 0x7fc1b814ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1b659c700 1 -- 172.21.15.7:0/3110902449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a80094f0 con 0x7fc1b806ec70 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1b659c700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b806ec70 0x7fc1b80c52f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009290 tx=0x7fc1a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1a67fc700 1 -- 172.21.15.7:0/3110902449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8011e40 con 0x7fc1b806ec70 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1a67fc700 1 -- 172.21.15.7:0/3110902449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8026450 con 0x7fc1b806ec70 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1a67fc700 1 -- 172.21.15.7:0/3110902449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8026810 con 0x7fc1b806ec70 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b80c4810 con 0x7fc1b806ec70 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.135+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b8099c90 con 0x7fc1b806ec70 2024-01-11T07:07:27.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.136+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b804c3c0 con 0x7fc1b806ec70 2024-01-11T07:07:27.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.137+0000 7fc1a67fc700 1 -- 172.21.15.7:0/3110902449 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc1a8026c40 con 0x7fc1b806ec70 2024-01-11T07:07:27.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.137+0000 7fc1a67fc700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc198066f30 0x7fc1980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.137+0000 7fc1a67fc700 1 -- 172.21.15.7:0/3110902449 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc1a80861d0 con 0x7fc1b806ec70 2024-01-11T07:07:27.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.137+0000 7fc1b5d9b700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc198066f30 0x7fc1980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.138+0000 7fc1b5d9b700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc198066f30 0x7fc1980693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1b815e360 tx=0x7fc1ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.142+0000 7fc1a67fc700 1 -- 172.21.15.7:0/3110902449 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1a8053420 con 0x7fc1b806ec70 2024-01-11T07:07:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.344+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1b80b2e40 con 0x7fc1b806ec70 2024-01-11T07:07:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.344+0000 7fc1a67fc700 1 -- 172.21.15.7:0/3110902449 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1a8056a40 con 0x7fc1b806ec70 2024-01-11T07:07:27.345 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:07:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.346+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc198066f30 msgr2=0x7fc1980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc198066f30 0x7fc1980693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1b815e360 tx=0x7fc1ac008040 comp rx=0 tx=0).stop 2024-01-11T07:07:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b806ec70 msgr2=0x7fc1b80c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b806ec70 0x7fc1b80c52f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009290 tx=0x7fc1a8017000 comp rx=0 tx=0).stop 2024-01-11T07:07:27.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 shutdown_connections 2024-01-11T07:07:27.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc198066f30 0x7fc1980693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b815db90 0x7fc1b809a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b806ec70 0x7fc1b80c52f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 --2- 172.21.15.7:0/3110902449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b80c3690 0x7fc1b814ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 >> 172.21.15.7:0/3110902449 conn(0x7fc1b80cd070 msgr2=0x7fc1b80c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:27.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 shutdown_connections 2024-01-11T07:07:27.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.347+0000 7fc1bce00700 1 -- 172.21.15.7:0/3110902449 wait complete. 2024-01-11T07:07:27.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:07:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.495+0000 7f526f534700 1 Processor -- start 2024-01-11T07:07:27.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.495+0000 7f526f534700 1 -- start start 2024-01-11T07:07:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.496+0000 7f526f534700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5268157bc0 0x7f52680cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.496+0000 7f526f534700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f52680b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.496+0000 7f526f534700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52680ca080 con 0x7f5268157f90 2024-01-11T07:07:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.496+0000 7f526f534700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52680ca640 con 0x7f5268157bc0 2024-01-11T07:07:27.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.496+0000 7f526f534700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680b1ca0 con 0x7f52680ccc00 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.496+0000 7f526d2d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5268157bc0 0x7f52680cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.496+0000 7f526d2d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5268157bc0 0x7f52680cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34112/0 (socket says 172.21.15.7:34112) 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.496+0000 7f526d2d0700 1 -- 172.21.15.7:0/2478827566 learned_addr learned my addr 172.21.15.7:0/2478827566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f526dad1700 1 --2- 172.21.15.7:0/2478827566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f52680b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f526d2d0700 1 -- 172.21.15.7:0/2478827566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 msgr2=0x7f52680b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f526d2d0700 1 --2- 172.21.15.7:0/2478827566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f52680b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f526d2d0700 1 -- 172.21.15.7:0/2478827566 >> v1:172.21.15.113:6789/0 conn(0x7f5268157f90 legacy=0x7f52680cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f526d2d0700 1 -- 172.21.15.7:0/2478827566 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52680ca360 con 0x7f5268157bc0 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f526dad1700 1 --2- 172.21.15.7:0/2478827566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f52680b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f526d2d0700 1 --2- 172.21.15.7:0/2478827566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f52680cf270 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5264009990 tx=0x7f5264009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e88d4ba586a54fa server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f525ffff700 1 -- 172.21.15.7:0/2478827566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264005c10 con 0x7f5268157bc0 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f525ffff700 1 -- 172.21.15.7:0/2478827566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5264005d70 con 0x7f5268157bc0 2024-01-11T07:07:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.497+0000 7f525ffff700 1 -- 172.21.15.7:0/2478827566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400b840 con 0x7f5268157bc0 2024-01-11T07:07:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.498+0000 7f526f534700 1 -- 172.21.15.7:0/2478827566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 msgr2=0x7f52680cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.498+0000 7f526f534700 1 --2- 172.21.15.7:0/2478827566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f52680cf270 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5264009990 tx=0x7f5264009220 comp rx=0 tx=0).stop 2024-01-11T07:07:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.498+0000 7f526f534700 1 -- 172.21.15.7:0/2478827566 shutdown_connections 2024-01-11T07:07:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.498+0000 7f526f534700 1 --2- 172.21.15.7:0/2478827566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f52680b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.498+0000 7f526f534700 1 --2- 172.21.15.7:0/2478827566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f52680cf270 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.498+0000 7f526f534700 1 -- 172.21.15.7:0/2478827566 >> 172.21.15.7:0/2478827566 conn(0x7f5268097a50 msgr2=0x7f52680ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.498+0000 7f526f534700 1 -- 172.21.15.7:0/2478827566 shutdown_connections 2024-01-11T07:07:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.498+0000 7f526f534700 1 -- 172.21.15.7:0/2478827566 wait complete. 2024-01-11T07:07:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.499+0000 7f526f534700 1 Processor -- start 2024-01-11T07:07:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.499+0000 7f526f534700 1 -- start start 2024-01-11T07:07:27.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526f534700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f5268167300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526f534700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f5268168300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526f534700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5268157f90 0x7f5268168aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526f534700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680c5400 con 0x7f5268157bc0 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526f534700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5268168840 con 0x7f5268157f90 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526f534700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680c24c0 con 0x7f52680ccc00 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526cacf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f5268168300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526cacf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f5268168300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34126/0 (socket says 172.21.15.7:34126) 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526cacf700 1 -- 172.21.15.7:0/300063723 learned_addr learned my addr 172.21.15.7:0/300063723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526d2d0700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f5268167300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.500+0000 7f526dad1700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5268157f90 0x7f5268168aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f526cacf700 1 -- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 msgr2=0x7f5268167300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f526cacf700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f5268167300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f526cacf700 1 -- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5268157f90 msgr2=0x7f5268168aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f526cacf700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5268157f90 0x7f5268168aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f526cacf700 1 -- 172.21.15.7:0/300063723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5264009480 con 0x7f5268157bc0 2024-01-11T07:07:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f526cacf700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f5268168300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f525800b5e0 tx=0x7f5258009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f525d7fa700 1 -- 172.21.15.7:0/300063723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800f510 con 0x7f5268157bc0 2024-01-11T07:07:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f525d7fa700 1 -- 172.21.15.7:0/300063723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525800f670 con 0x7f5268157bc0 2024-01-11T07:07:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52680c27a0 con 0x7f5268157bc0 2024-01-11T07:07:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f525d7fa700 1 -- 172.21.15.7:0/300063723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800fa30 con 0x7f5268157bc0 2024-01-11T07:07:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.501+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52680c2d60 con 0x7f5268157bc0 2024-01-11T07:07:27.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.502+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526804c3c0 con 0x7f5268157bc0 2024-01-11T07:07:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.502+0000 7f525d7fa700 1 -- 172.21.15.7:0/300063723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f525801a400 con 0x7f5268157bc0 2024-01-11T07:07:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.503+0000 7f525d7fa700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.503+0000 7f525d7fa700 1 -- 172.21.15.7:0/300063723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5258083ed0 con 0x7f5268157bc0 2024-01-11T07:07:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.503+0000 7f526d2d0700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.504+0000 7f526d2d0700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52640030c0 tx=0x7f5264015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.508+0000 7f525d7fa700 1 -- 172.21.15.7:0/300063723 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52580511c0 con 0x7f5268157bc0 2024-01-11T07:07:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.714+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52680b18c0 con 0x7f5268157bc0 2024-01-11T07:07:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.714+0000 7f525d7fa700 1 -- 172.21.15.7:0/300063723 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52580547e0 con 0x7f5268157bc0 2024-01-11T07:07:27.714 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 msgr2=0x7f524c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52640030c0 tx=0x7f5264015000 comp rx=0 tx=0).stop 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 msgr2=0x7f5268168300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f5268168300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f525800b5e0 tx=0x7f5258009120 comp rx=0 tx=0).stop 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 shutdown_connections 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f524c066f30 0x7f524c0693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5268157f90 0x7f5268168aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52680ccc00 0x7f5268167300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 --2- 172.21.15.7:0/300063723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5268157bc0 0x7f5268168300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 >> 172.21.15.7:0/300063723 conn(0x7f5268097a50 msgr2=0x7f5268156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 shutdown_connections 2024-01-11T07:07:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.717+0000 7f526f534700 1 -- 172.21.15.7:0/300063723 wait complete. 2024-01-11T07:07:27.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:27.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.861+0000 7f3d3a4d2700 1 Processor -- start 2024-01-11T07:07:27.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.861+0000 7f3d3a4d2700 1 -- start start 2024-01-11T07:07:27.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.862+0000 7f3d3a4d2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d34157f20 0x7f3d340cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.862+0000 7f3d3a4d2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3406a580 con 0x7f3d3415b4b0 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.862+0000 7f3d3a4d2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3415a9d0 con 0x7f3d340cce70 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.862+0000 7f3d3a4d2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3406a860 con 0x7f3d34157f20 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.862+0000 7f3d337fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d34157f20 0x7f3d340cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d337fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d34157f20 0x7f3d340cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34134/0 (socket says 172.21.15.7:34134) 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d38a6f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3d3415b4b0 0x7f3d340b1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46706/0 (socket says 172.21.15.7:46706) 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d38a6f700 1 -- 172.21.15.7:0/2836662177 learned_addr learned my addr 172.21.15.7:0/2836662177 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d337fe700 1 -- 172.21.15.7:0/2836662177 >> v1:172.21.15.113:6789/0 conn(0x7f3d340cce70 legacy=0x7f3d34069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d337fe700 1 -- 172.21.15.7:0/2836662177 >> v1:172.21.15.7:6789/0 conn(0x7f3d3415b4b0 legacy=0x7f3d340b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d337fe700 1 -- 172.21.15.7:0/2836662177 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3415acb0 con 0x7f3d34157f20 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d32ffd700 1 -- 172.21.15.7:0/2836662177 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301825313 0 0) 0x7f3d3406a580 con 0x7f3d3415b4b0 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d337fe700 1 --2- 172.21.15.7:0/2836662177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d34157f20 0x7f3d340cd240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3d280098e0 tx=0x7f3d280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b10c7ad49099932 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d32ffd700 1 -- 172.21.15.7:0/2836662177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28009e80 con 0x7f3d34157f20 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d32ffd700 1 -- 172.21.15.7:0/2836662177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2800b860 con 0x7f3d34157f20 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.863+0000 7f3d32ffd700 1 -- 172.21.15.7:0/2836662177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2800bbe0 con 0x7f3d34157f20 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.864+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/2836662177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d34157f20 msgr2=0x7f3d340cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.864+0000 7f3d3a4d2700 1 --2- 172.21.15.7:0/2836662177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d34157f20 0x7f3d340cd240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3d280098e0 tx=0x7f3d280091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.864+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/2836662177 shutdown_connections 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.864+0000 7f3d3a4d2700 1 --2- 172.21.15.7:0/2836662177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d34157f20 0x7f3d340cd240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.864+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/2836662177 >> 172.21.15.7:0/2836662177 conn(0x7f3d340cd910 msgr2=0x7f3d340cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.864+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/2836662177 shutdown_connections 2024-01-11T07:07:27.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.864+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/2836662177 wait complete. 2024-01-11T07:07:27.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.865+0000 7f3d3a4d2700 1 Processor -- start 2024-01-11T07:07:27.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.865+0000 7f3d3a4d2700 1 -- start start 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d3a4d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d340cce70 0x7f3d34153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d3a4d2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d34157f20 0x7f3d340a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d3a4d2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d3415b4b0 0x7f3d34158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d3a4d2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340a9e80 con 0x7f3d340cce70 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d3a4d2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340a98c0 con 0x7f3d34157f20 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d3a4d2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340a9ba0 con 0x7f3d3415b4b0 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d340cce70 0x7f3d34153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d340cce70 0x7f3d34153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34138/0 (socket says 172.21.15.7:34138) 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d33fff700 1 -- 172.21.15.7:0/4029687655 learned_addr learned my addr 172.21.15.7:0/4029687655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d337fe700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d34157f20 0x7f3d340a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.866+0000 7f3d38a6f700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d3415b4b0 0x7f3d34158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d33fff700 1 -- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d3415b4b0 msgr2=0x7f3d34158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d33fff700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d3415b4b0 0x7f3d34158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d33fff700 1 -- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d34157f20 msgr2=0x7f3d340a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d33fff700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d34157f20 0x7f3d340a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d33fff700 1 -- 172.21.15.7:0/4029687655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d28009420 con 0x7f3d340cce70 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d33fff700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d340cce70 0x7f3d34153130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400c590 tx=0x7f3d2400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d1bfff700 1 -- 172.21.15.7:0/4029687655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24011590 con 0x7f3d340cce70 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d1bfff700 1 -- 172.21.15.7:0/4029687655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d24011710 con 0x7f3d340cce70 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d34158430 con 0x7f3d340cce70 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d1bfff700 1 -- 172.21.15.7:0/4029687655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24011ad0 con 0x7f3d340cce70 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.867+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d34153670 con 0x7f3d340cce70 2024-01-11T07:07:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.868+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d341588d0 con 0x7f3d340cce70 2024-01-11T07:07:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.869+0000 7f3d1bfff700 1 -- 172.21.15.7:0/4029687655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3d2401b3b0 con 0x7f3d340cce70 2024-01-11T07:07:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.869+0000 7f3d1bfff700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3d20066f30 0x7f3d200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.869+0000 7f3d1bfff700 1 -- 172.21.15.7:0/4029687655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3d240839a0 con 0x7f3d340cce70 2024-01-11T07:07:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.869+0000 7f3d337fe700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3d20066f30 0x7f3d200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.870+0000 7f3d337fe700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3d20066f30 0x7f3d200693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d34158360 tx=0x7f3d28017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:27.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:27.873+0000 7f3d1bfff700 1 -- 172.21.15.7:0/4029687655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d24050c10 con 0x7f3d340cce70 2024-01-11T07:07:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.082+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d34064a40 con 0x7f3d340cce70 2024-01-11T07:07:28.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.082+0000 7f3d1bfff700 1 -- 172.21.15.7:0/4029687655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f3d24017070 con 0x7f3d340cce70 2024-01-11T07:07:28.082 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:28.082 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3d20066f30 msgr2=0x7f3d200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3d20066f30 0x7f3d200693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d34158360 tx=0x7f3d28017000 comp rx=0 tx=0).stop 2024-01-11T07:07:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d340cce70 msgr2=0x7f3d34153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d340cce70 0x7f3d34153130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400c590 tx=0x7f3d2400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 shutdown_connections 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3d20066f30 0x7f3d200693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d3415b4b0 0x7f3d34158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d340cce70 0x7f3d34153130 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 --2- 172.21.15.7:0/4029687655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d34157f20 0x7f3d340a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 >> 172.21.15.7:0/4029687655 conn(0x7f3d340cd910 msgr2=0x7f3d34154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 shutdown_connections 2024-01-11T07:07:28.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.085+0000 7f3d3a4d2700 1 -- 172.21.15.7:0/4029687655 wait complete. 2024-01-11T07:07:28.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:06:56.038804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:06:57.365958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:07:28.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:07:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.234+0000 7f549fb36700 1 Processor -- start 2024-01-11T07:07:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.234+0000 7f549fb36700 1 -- start start 2024-01-11T07:07:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549fb36700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5498159e00 0x7f549806a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549fb36700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f549815a1d0 0x7f54980ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549fb36700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549806aad0 con 0x7f54980ad630 2024-01-11T07:07:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549fb36700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54980adfa0 con 0x7f5498159e00 2024-01-11T07:07:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549fb36700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5498070830 con 0x7f549815a1d0 2024-01-11T07:07:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549d8d2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5498159e00 0x7f549806a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549e0d3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54980ad630 0x7f54980aa8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46710/0 (socket says 172.21.15.7:46710) 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549e0d3700 1 -- 172.21.15.7:0/1377586671 learned_addr learned my addr 172.21.15.7:0/1377586671 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549d0d1700 1 --2- 172.21.15.7:0/1377586671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f549815a1d0 0x7f54980ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549d8d2700 1 -- 172.21.15.7:0/1377586671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f549815a1d0 msgr2=0x7f54980ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549d8d2700 1 --2- 172.21.15.7:0/1377586671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f549815a1d0 0x7f54980ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549d8d2700 1 -- 172.21.15.7:0/1377586671 >> v1:172.21.15.7:6789/0 conn(0x7f54980ad630 legacy=0x7f54980aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549d8d2700 1 -- 172.21.15.7:0/1377586671 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5498070b10 con 0x7f5498159e00 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549c8d0700 1 -- 172.21.15.7:0/1377586671 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458215348 0 0) 0x7f5490001490 con 0x7f54980ad630 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.235+0000 7f549d0d1700 1 --2- 172.21.15.7:0/1377586671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f549815a1d0 0x7f54980ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549d8d2700 1 --2- 172.21.15.7:0/1377586671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5498159e00 0x7f549806a590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f54940098e0 tx=0x7f54940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b947c76f49d3d57 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549c8d0700 1 -- 172.21.15.7:0/1377586671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494009e80 con 0x7f5498159e00 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549c8d0700 1 -- 172.21.15.7:0/1377586671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549400b860 con 0x7f5498159e00 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549c8d0700 1 -- 172.21.15.7:0/1377586671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400bbe0 con 0x7f5498159e00 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549fb36700 1 -- 172.21.15.7:0/1377586671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5498159e00 msgr2=0x7f549806a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549fb36700 1 --2- 172.21.15.7:0/1377586671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5498159e00 0x7f549806a590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f54940098e0 tx=0x7f54940091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549fb36700 1 -- 172.21.15.7:0/1377586671 shutdown_connections 2024-01-11T07:07:28.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549fb36700 1 --2- 172.21.15.7:0/1377586671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5498159e00 0x7f549806a590 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549fb36700 1 --2- 172.21.15.7:0/1377586671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f549815a1d0 0x7f54980ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.236+0000 7f549fb36700 1 -- 172.21.15.7:0/1377586671 >> 172.21.15.7:0/1377586671 conn(0x7f5498068080 msgr2=0x7f54980cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:28.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.237+0000 7f549fb36700 1 -- 172.21.15.7:0/1377586671 shutdown_connections 2024-01-11T07:07:28.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.237+0000 7f549fb36700 1 -- 172.21.15.7:0/1377586671 wait complete. 2024-01-11T07:07:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.238+0000 7f549fb36700 1 Processor -- start 2024-01-11T07:07:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.238+0000 7f549fb36700 1 -- start start 2024-01-11T07:07:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549fb36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54980ad630 0x7f54980b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549fb36700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5498159e00 0x7f549815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549fb36700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f549815a1d0 0x7f5498142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549fb36700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980b11f0 con 0x7f54980ad630 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549fb36700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549815ce10 con 0x7f549815a1d0 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549fb36700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549815d0f0 con 0x7f5498159e00 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54980ad630 0x7f54980b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54980ad630 0x7f54980b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34156/0 (socket says 172.21.15.7:34156) 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 -- 172.21.15.7:0/1844487243 learned_addr learned my addr 172.21.15.7:0/1844487243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d0d1700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5498159e00 0x7f549815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549e0d3700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f549815a1d0 0x7f5498142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 -- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5498159e00 msgr2=0x7f549815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5498159e00 0x7f549815c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 -- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f549815a1d0 msgr2=0x7f5498142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f549815a1d0 0x7f5498142d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 -- 172.21.15.7:0/1844487243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494009420 con 0x7f54980ad630 2024-01-11T07:07:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549e0d3700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f549815a1d0 0x7f5498142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549d8d2700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54980ad630 0x7f54980b0770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f54980685c0 tx=0x7f5494017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f548dffb700 1 -- 172.21.15.7:0/1844487243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400bbe0 con 0x7f54980ad630 2024-01-11T07:07:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f548dffb700 1 -- 172.21.15.7:0/1844487243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5494000cc0 con 0x7f54980ad630 2024-01-11T07:07:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549815d3d0 con 0x7f54980ad630 2024-01-11T07:07:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.239+0000 7f548dffb700 1 -- 172.21.15.7:0/1844487243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494011670 con 0x7f54980ad630 2024-01-11T07:07:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.240+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549815cb00 con 0x7f54980ad630 2024-01-11T07:07:28.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.241+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549804c3c0 con 0x7f54980ad630 2024-01-11T07:07:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.241+0000 7f548dffb700 1 -- 172.21.15.7:0/1844487243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5494011a20 con 0x7f54980ad630 2024-01-11T07:07:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.242+0000 7f548dffb700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.242+0000 7f548dffb700 1 -- 172.21.15.7:0/1844487243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5494085b50 con 0x7f54980ad630 2024-01-11T07:07:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.242+0000 7f549d0d1700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.242+0000 7f549d0d1700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f547c066f30 0x7f547c0693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54980b0460 tx=0x7f5488008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.246+0000 7f548dffb700 1 -- 172.21.15.7:0/1844487243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5494052d20 con 0x7f54980ad630 2024-01-11T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.449+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54980cd980 con 0x7f54980ad630 2024-01-11T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.450+0000 7f548dffb700 1 -- 172.21.15.7:0/1844487243 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5494056340 con 0x7f54980ad630 2024-01-11T07:07:28.449 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:07:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f547c066f30 msgr2=0x7f547c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f547c066f30 0x7f547c0693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54980b0460 tx=0x7f5488008040 comp rx=0 tx=0).stop 2024-01-11T07:07:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54980ad630 msgr2=0x7f54980b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54980ad630 0x7f54980b0770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f54980685c0 tx=0x7f5494017000 comp rx=0 tx=0).stop 2024-01-11T07:07:28.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 shutdown_connections 2024-01-11T07:07:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f549815a1d0 0x7f5498142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54980ad630 0x7f54980b0770 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 --2- 172.21.15.7:0/1844487243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5498159e00 0x7f549815c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 >> 172.21.15.7:0/1844487243 conn(0x7f5498068080 msgr2=0x7f54980aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 shutdown_connections 2024-01-11T07:07:28.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.452+0000 7f549fb36700 1 -- 172.21.15.7:0/1844487243 wait complete. 2024-01-11T07:07:28.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:07:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.626+0000 7ff8eb8cc700 1 Processor -- start 2024-01-11T07:07:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.626+0000 7ff8eb8cc700 1 -- start start 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.627+0000 7ff8eb8cc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 0x7ff8e40b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.627+0000 7ff8eb8cc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8e406e200 0x7ff8e40c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.627+0000 7ff8eb8cc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8e40c91c0 0x7ff8e40af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.627+0000 7ff8eb8cc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e40b1540 con 0x7ff8e40c91c0 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.627+0000 7ff8eb8cc700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e4069c70 con 0x7ff8e406e200 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.627+0000 7ff8eb8cc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e4069f50 con 0x7ff8e406de30 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.627+0000 7ff8e9e69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8e40c91c0 0x7ff8e40af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9e69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8e40c91c0 0x7ff8e40af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34160/0 (socket says 172.21.15.7:34160) 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9e69700 1 -- 172.21.15.7:0/290011670 learned_addr learned my addr 172.21.15.7:0/290011670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e8e67700 1 --2- 172.21.15.7:0/290011670 >> v2:172.21.15.113:3300/0 conn(0x7ff8e406e200 0x7ff8e40c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9668700 1 --2- 172.21.15.7:0/290011670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 0x7ff8e40b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9e69700 1 -- 172.21.15.7:0/290011670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 msgr2=0x7ff8e40b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9e69700 1 --2- 172.21.15.7:0/290011670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 0x7ff8e40b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9e69700 1 -- 172.21.15.7:0/290011670 >> v2:172.21.15.113:3300/0 conn(0x7ff8e406e200 msgr2=0x7ff8e40c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9e69700 1 --2- 172.21.15.7:0/290011670 >> v2:172.21.15.113:3300/0 conn(0x7ff8e406e200 0x7ff8e40c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9e69700 1 -- 172.21.15.7:0/290011670 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e406a230 con 0x7ff8e40c91c0 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e8e67700 1 --2- 172.21.15.7:0/290011670 >> v2:172.21.15.113:3300/0 conn(0x7ff8e406e200 0x7ff8e40c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8e9e69700 1 --2- 172.21.15.7:0/290011670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e40c91c0 0x7ff8e40af9b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4068820 tx=0x7ff8dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c845edd1afc97a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8dbfff700 1 -- 172.21.15.7:0/290011670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00e070 con 0x7ff8e40c91c0 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8dbfff700 1 -- 172.21.15.7:0/290011670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc00ba50 con 0x7ff8e40c91c0 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.628+0000 7ff8dbfff700 1 -- 172.21.15.7:0/290011670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0092e0 con 0x7ff8e40c91c0 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/290011670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e40c91c0 msgr2=0x7ff8e40af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/290011670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e40c91c0 0x7ff8e40af9b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4068820 tx=0x7ff8dc009840 comp rx=0 tx=0).stop 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/290011670 shutdown_connections 2024-01-11T07:07:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/290011670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e40c91c0 0x7ff8e40af9b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/290011670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 0x7ff8e40b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/290011670 >> v2:172.21.15.113:3300/0 conn(0x7ff8e406e200 0x7ff8e40c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/290011670 >> 172.21.15.7:0/290011670 conn(0x7ff8e40680e0 msgr2=0x7ff8e40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/290011670 shutdown_connections 2024-01-11T07:07:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.629+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/290011670 wait complete. 2024-01-11T07:07:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.630+0000 7ff8eb8cc700 1 Processor -- start 2024-01-11T07:07:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.630+0000 7ff8eb8cc700 1 -- start start 2024-01-11T07:07:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8eb8cc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 0x7ff8e40c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8eb8cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e406e200 0x7ff8e40cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8eb8cc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e40c91c0 0x7ff8e40cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8eb8cc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e4166cf0 con 0x7ff8e406e200 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8eb8cc700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e4166fd0 con 0x7ff8e40c91c0 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8eb8cc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e41672b0 con 0x7ff8e406de30 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8e8e67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e406e200 0x7ff8e40cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8e8e67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e406e200 0x7ff8e40cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34162/0 (socket says 172.21.15.7:34162) 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.631+0000 7ff8e8e67700 1 -- 172.21.15.7:0/83775730 learned_addr learned my addr 172.21.15.7:0/83775730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8e9668700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 0x7ff8e40c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8e9e69700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e40c91c0 0x7ff8e40cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8e8e67700 1 -- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 msgr2=0x7ff8e40c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8e8e67700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 0x7ff8e40c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8e8e67700 1 -- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e40c91c0 msgr2=0x7ff8e40cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8e8e67700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e40c91c0 0x7ff8e40cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8e8e67700 1 -- 172.21.15.7:0/83775730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc009ae0 con 0x7ff8e406e200 2024-01-11T07:07:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8e8e67700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e406e200 0x7ff8e40cc370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e8d0 tx=0x7ff8e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8d97fa700 1 -- 172.21.15.7:0/83775730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0013410 con 0x7ff8e406e200 2024-01-11T07:07:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8d97fa700 1 -- 172.21.15.7:0/83775730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e0013570 con 0x7ff8e406e200 2024-01-11T07:07:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e4167590 con 0x7ff8e406e200 2024-01-11T07:07:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8d97fa700 1 -- 172.21.15.7:0/83775730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0013910 con 0x7ff8e406e200 2024-01-11T07:07:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.632+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e4169a30 con 0x7ff8e406e200 2024-01-11T07:07:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.633+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e4064a40 con 0x7ff8e406e200 2024-01-11T07:07:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.634+0000 7ff8d97fa700 1 -- 172.21.15.7:0/83775730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff8e0013d00 con 0x7ff8e406e200 2024-01-11T07:07:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.634+0000 7ff8d97fa700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8c8066f30 0x7ff8c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.634+0000 7ff8d97fa700 1 -- 172.21.15.7:0/83775730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff8e0084250 con 0x7ff8e406e200 2024-01-11T07:07:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.635+0000 7ff8e9668700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8c8066f30 0x7ff8c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.635+0000 7ff8e9668700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8c8066f30 0x7ff8c80693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4006fd0 tx=0x7ff8d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.639+0000 7ff8d97fa700 1 -- 172.21.15.7:0/83775730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8e0054f50 con 0x7ff8e406e200 2024-01-11T07:07:28.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.840+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8e404c3c0 con 0x7ff8e406e200 2024-01-11T07:07:28.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.841+0000 7ff8d97fa700 1 -- 172.21.15.7:0/83775730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8e0054ae0 con 0x7ff8e406e200 2024-01-11T07:07:28.841 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:07:28.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8c8066f30 msgr2=0x7ff8c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8c8066f30 0x7ff8c80693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4006fd0 tx=0x7ff8d4008040 comp rx=0 tx=0).stop 2024-01-11T07:07:28.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e406e200 msgr2=0x7ff8e40cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e406e200 0x7ff8e40cc370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e8d0 tx=0x7ff8e0008040 comp rx=0 tx=0).stop 2024-01-11T07:07:28.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 shutdown_connections 2024-01-11T07:07:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8c8066f30 0x7ff8c80693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e40c91c0 0x7ff8e40cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8e406de30 0x7ff8e40c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 --2- 172.21.15.7:0/83775730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8e406e200 0x7ff8e40cc370 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 >> 172.21.15.7:0/83775730 conn(0x7ff8e40680e0 msgr2=0x7ff8e40af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 shutdown_connections 2024-01-11T07:07:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.843+0000 7ff8eb8cc700 1 -- 172.21.15.7:0/83775730 wait complete. 2024-01-11T07:07:28.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:07:28.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.991+0000 7ff85a9e7700 1 Processor -- start 2024-01-11T07:07:28.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.991+0000 7ff85a9e7700 1 -- start start 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.991+0000 7ff85a9e7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 0x7ff85415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.991+0000 7ff85a9e7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8540adf30 0x7ff85406a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff85a9e7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8540ae460 con 0x7ff854157c30 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff85a9e7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8540af0e0 con 0x7ff8540adf30 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff85a9e7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540ae740 con 0x7ff854157860 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff8537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8540adf30 0x7ff85406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff858f84700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff854157c30 0x7ff8540b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46718/0 (socket says 172.21.15.7:46718) 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff8537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8540adf30 0x7ff85406a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34178/0 (socket says 172.21.15.7:34178) 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff858f84700 1 -- 172.21.15.7:0/791798566 learned_addr learned my addr 172.21.15.7:0/791798566 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff853fff700 1 --2- 172.21.15.7:0/791798566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 0x7ff85415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff8537fe700 1 -- 172.21.15.7:0/791798566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 msgr2=0x7ff85415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff8537fe700 1 --2- 172.21.15.7:0/791798566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 0x7ff85415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff8537fe700 1 -- 172.21.15.7:0/791798566 >> v1:172.21.15.7:6789/0 conn(0x7ff854157c30 legacy=0x7ff8540b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff8537fe700 1 -- 172.21.15.7:0/791798566 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540aea20 con 0x7ff8540adf30 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff852ffd700 1 -- 172.21.15.7:0/791798566 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305121728 0 0) 0x7ff8540ae460 con 0x7ff854157c30 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff8537fe700 1 --2- 172.21.15.7:0/791798566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8540adf30 0x7ff85406a600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff8480098e0 tx=0x7ff8480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4eb712ec6fc8b5b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff852ffd700 1 -- 172.21.15.7:0/791798566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84800b640 con 0x7ff8540adf30 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.992+0000 7ff852ffd700 1 -- 172.21.15.7:0/791798566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84800b7a0 con 0x7ff8540adf30 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff852ffd700 1 -- 172.21.15.7:0/791798566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84800bb40 con 0x7ff8540adf30 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff85a9e7700 1 -- 172.21.15.7:0/791798566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8540adf30 msgr2=0x7ff85406a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/791798566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8540adf30 0x7ff85406a600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff8480098e0 tx=0x7ff8480091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff85a9e7700 1 -- 172.21.15.7:0/791798566 shutdown_connections 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/791798566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 0x7ff85415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/791798566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8540adf30 0x7ff85406a600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff85a9e7700 1 -- 172.21.15.7:0/791798566 >> 172.21.15.7:0/791798566 conn(0x7ff8540cd070 msgr2=0x7ff8540cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:28.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff85a9e7700 1 -- 172.21.15.7:0/791798566 shutdown_connections 2024-01-11T07:07:28.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.993+0000 7ff85a9e7700 1 -- 172.21.15.7:0/791798566 wait complete. 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.994+0000 7ff85a9e7700 1 Processor -- start 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff85a9e7700 1 -- start start 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff85a9e7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8540adf30 0x7ff854154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff85a9e7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 0x7ff85406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff85a9e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff854157c30 0x7ff8540b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff85a9e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540b08f0 con 0x7ff854157c30 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff85a9e7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540c1e10 con 0x7ff8540adf30 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff85a9e7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540b0e10 con 0x7ff854157860 2024-01-11T07:07:28.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff858f84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff854157c30 0x7ff8540b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.995+0000 7ff858f84700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff854157c30 0x7ff8540b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34180/0 (socket says 172.21.15.7:34180) 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff858f84700 1 -- 172.21.15.7:0/615808178 learned_addr learned my addr 172.21.15.7:0/615808178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff853fff700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8540adf30 0x7ff854154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff8537fe700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 0x7ff85406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff858f84700 1 -- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 msgr2=0x7ff85406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff858f84700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 0x7ff85406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff858f84700 1 -- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8540adf30 msgr2=0x7ff854154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff858f84700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8540adf30 0x7ff854154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff858f84700 1 -- 172.21.15.7:0/615808178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff848009420 con 0x7ff854157c30 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff858f84700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff854157c30 0x7ff8540b12a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00cc20 tx=0x7ff84c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff83bfff700 1 -- 172.21.15.7:0/615808178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00a870 con 0x7ff854157c30 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff83bfff700 1 -- 172.21.15.7:0/615808178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c00a9d0 con 0x7ff854157c30 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8540b0ae0 con 0x7ff854157c30 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff83bfff700 1 -- 172.21.15.7:0/615808178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00ad90 con 0x7ff854157c30 2024-01-11T07:07:28.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.996+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8540c2020 con 0x7ff854157c30 2024-01-11T07:07:28.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.997+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85404c3c0 con 0x7ff854157c30 2024-01-11T07:07:28.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.998+0000 7ff83bfff700 1 -- 172.21.15.7:0/615808178 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff84c007300 con 0x7ff854157c30 2024-01-11T07:07:28.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.998+0000 7ff83bfff700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff840066f30 0x7ff8400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:28.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.998+0000 7ff83bfff700 1 -- 172.21.15.7:0/615808178 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff84c084800 con 0x7ff854157c30 2024-01-11T07:07:28.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.998+0000 7ff853fff700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff840066f30 0x7ff8400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:28.999+0000 7ff853fff700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff840066f30 0x7ff8400693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff844009db0 tx=0x7ff8440091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.003+0000 7ff83bfff700 1 -- 172.21.15.7:0/615808178 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff84c051af0 con 0x7ff854157c30 2024-01-11T07:07:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.205+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8540b0f50 con 0x7ff854157c30 2024-01-11T07:07:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.206+0000 7ff83bfff700 1 -- 172.21.15.7:0/615808178 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff84c055110 con 0x7ff854157c30 2024-01-11T07:07:29.206 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:07:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff840066f30 msgr2=0x7ff8400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff840066f30 0x7ff8400693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff844009db0 tx=0x7ff8440091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff854157c30 msgr2=0x7ff8540b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff854157c30 0x7ff8540b12a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00cc20 tx=0x7ff84c0019b0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 shutdown_connections 2024-01-11T07:07:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff840066f30 0x7ff8400693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff854157c30 0x7ff8540b12a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8540adf30 0x7ff854154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 --2- 172.21.15.7:0/615808178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff854157860 0x7ff85406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 >> 172.21.15.7:0/615808178 conn(0x7ff8540cd070 msgr2=0x7ff8541554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 shutdown_connections 2024-01-11T07:07:29.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.208+0000 7ff85a9e7700 1 -- 172.21.15.7:0/615808178 wait complete. 2024-01-11T07:07:29.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.356+0000 7fe5d699c700 1 Processor -- start 2024-01-11T07:07:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.356+0000 7fe5d699c700 1 -- start start 2024-01-11T07:07:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5d699c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d01560d0 0x7fe5d0069e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5d699c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5d00c7520 0x7fe5d00cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5d699c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d00699a0 con 0x7fe5d00cadf0 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5d699c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d00cc2f0 con 0x7fe5d00c7520 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5d699c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0068d90 con 0x7fe5d01560d0 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5d4f39700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5d00c7520 0x7fe5d00cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5cf7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe5d00cadf0 0x7fe5d00c6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46730/0 (socket says 172.21.15.7:46730) 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5cf7fe700 1 -- 172.21.15.7:0/3327161518 learned_addr learned my addr 172.21.15.7:0/3327161518 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.357+0000 7fe5cffff700 1 --2- 172.21.15.7:0/3327161518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d01560d0 0x7fe5d0069e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5d4f39700 1 -- 172.21.15.7:0/3327161518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d01560d0 msgr2=0x7fe5d0069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5d4f39700 1 --2- 172.21.15.7:0/3327161518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d01560d0 0x7fe5d0069e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5d4f39700 1 -- 172.21.15.7:0/3327161518 >> v1:172.21.15.7:6789/0 conn(0x7fe5d00cadf0 legacy=0x7fe5d00c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5d4f39700 1 -- 172.21.15.7:0/3327161518 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0069070 con 0x7fe5d00c7520 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5ceffd700 1 -- 172.21.15.7:0/3327161518 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955603066 0 0) 0x7fe5d00699a0 con 0x7fe5d00cadf0 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5d4f39700 1 --2- 172.21.15.7:0/3327161518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d00c7520 0x7fe5d00cb260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800d0d0 tx=0x7fe5c800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5532b32d9280ade server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5ceffd700 1 -- 172.21.15.7:0/3327161518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8010070 con 0x7fe5d00c7520 2024-01-11T07:07:29.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5ceffd700 1 -- 172.21.15.7:0/3327161518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c800dbe0 con 0x7fe5d00c7520 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5ceffd700 1 -- 172.21.15.7:0/3327161518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8016400 con 0x7fe5d00c7520 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5d699c700 1 -- 172.21.15.7:0/3327161518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d00c7520 msgr2=0x7fe5d00cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.358+0000 7fe5d699c700 1 --2- 172.21.15.7:0/3327161518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d00c7520 0x7fe5d00cb260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800d0d0 tx=0x7fe5c800b990 comp rx=0 tx=0).stop 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.359+0000 7fe5d699c700 1 -- 172.21.15.7:0/3327161518 shutdown_connections 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.359+0000 7fe5d699c700 1 --2- 172.21.15.7:0/3327161518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d00c7520 0x7fe5d00cb260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.359+0000 7fe5d699c700 1 --2- 172.21.15.7:0/3327161518 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d01560d0 0x7fe5d0069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.359+0000 7fe5d699c700 1 -- 172.21.15.7:0/3327161518 >> 172.21.15.7:0/3327161518 conn(0x7fe5d0063230 msgr2=0x7fe5d0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.359+0000 7fe5d699c700 1 -- 172.21.15.7:0/3327161518 shutdown_connections 2024-01-11T07:07:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.359+0000 7fe5d699c700 1 -- 172.21.15.7:0/3327161518 wait complete. 2024-01-11T07:07:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.361+0000 7fe5d699c700 1 Processor -- start 2024-01-11T07:07:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.361+0000 7fe5d699c700 1 -- start start 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d699c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d00c7520 0x7fe5d00a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d699c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d00cadf0 0x7fe5d00adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d01560d0 0x7fe5d00aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d699c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d00ae500 con 0x7fe5d01560d0 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d699c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d00ae7e0 con 0x7fe5d00c7520 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d699c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d00aeac0 con 0x7fe5d00cadf0 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d4f39700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d01560d0 0x7fe5d00aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d4f39700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d01560d0 0x7fe5d00aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34194/0 (socket says 172.21.15.7:34194) 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d4f39700 1 -- 172.21.15.7:0/530970973 learned_addr learned my addr 172.21.15.7:0/530970973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5cf7fe700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d00cadf0 0x7fe5d00adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5cffff700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d00c7520 0x7fe5d00a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d4f39700 1 -- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d00cadf0 msgr2=0x7fe5d00adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d4f39700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d00cadf0 0x7fe5d00adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d4f39700 1 -- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d00c7520 msgr2=0x7fe5d00a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d4f39700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d00c7520 0x7fe5d00a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.362+0000 7fe5d4f39700 1 -- 172.21.15.7:0/530970973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c800bc30 con 0x7fe5d01560d0 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.363+0000 7fe5d4f39700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d01560d0 0x7fe5d00aa4e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5d006ac50 tx=0x7fe5c801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.363+0000 7fe5b7fff700 1 -- 172.21.15.7:0/530970973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800b020 con 0x7fe5d01560d0 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.363+0000 7fe5b7fff700 1 -- 172.21.15.7:0/530970973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c800b180 con 0x7fe5d01560d0 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.363+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d00aeda0 con 0x7fe5d01560d0 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.363+0000 7fe5b7fff700 1 -- 172.21.15.7:0/530970973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80206f0 con 0x7fe5d01560d0 2024-01-11T07:07:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.363+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d00a6c40 con 0x7fe5d01560d0 2024-01-11T07:07:29.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.364+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d004c3c0 con 0x7fe5d01560d0 2024-01-11T07:07:29.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.365+0000 7fe5b7fff700 1 -- 172.21.15.7:0/530970973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe5c8003040 con 0x7fe5d01560d0 2024-01-11T07:07:29.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.365+0000 7fe5b7fff700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe5bc066f30 0x7fe5bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.365+0000 7fe5b7fff700 1 -- 172.21.15.7:0/530970973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5c8088780 con 0x7fe5d01560d0 2024-01-11T07:07:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.365+0000 7fe5cffff700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe5bc066f30 0x7fe5bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.366+0000 7fe5cffff700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe5bc066f30 0x7fe5bc0693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe5d006cbe0 tx=0x7fe5c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:29.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.369+0000 7fe5b7fff700 1 -- 172.21.15.7:0/530970973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5c8055aa0 con 0x7fe5d01560d0 2024-01-11T07:07:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.574+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5d0166bd0 con 0x7fe5d01560d0 2024-01-11T07:07:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.575+0000 7fe5b7fff700 1 -- 172.21.15.7:0/530970973 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fe5d0166bd0 con 0x7fe5d01560d0 2024-01-11T07:07:29.575 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:29.575 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe5bc066f30 msgr2=0x7fe5bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe5bc066f30 0x7fe5bc0693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe5d006cbe0 tx=0x7fe5c0008040 comp rx=0 tx=0).stop 2024-01-11T07:07:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d01560d0 msgr2=0x7fe5d00aa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d01560d0 0x7fe5d00aa4e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5d006ac50 tx=0x7fe5c801c000 comp rx=0 tx=0).stop 2024-01-11T07:07:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 shutdown_connections 2024-01-11T07:07:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe5bc066f30 0x7fe5bc0693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5d01560d0 0x7fe5d00aa4e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5d00c7520 0x7fe5d00a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 --2- 172.21.15.7:0/530970973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5d00cadf0 0x7fe5d00adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 >> 172.21.15.7:0/530970973 conn(0x7fe5d0063230 msgr2=0x7fe5d006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 shutdown_connections 2024-01-11T07:07:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.577+0000 7fe5d699c700 1 -- 172.21.15.7:0/530970973 wait complete. 2024-01-11T07:07:29.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:06:56.038804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:06:57.365958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:07:29.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:07:29.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.726+0000 7f02d3cb7700 1 Processor -- start 2024-01-11T07:07:29.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.727+0000 7f02d3cb7700 1 -- start start 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.727+0000 7f02d3cb7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.727+0000 7f02d3cb7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02cc157f90 0x7f02cc0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.727+0000 7f02d3cb7700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02cc0ca080 con 0x7f02cc0cca30 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.727+0000 7f02d3cb7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02cc0ca640 con 0x7f02cc157f90 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.727+0000 7f02d3cb7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc0b1ca0 con 0x7f02cc157bc0 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1a53700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1252700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02cc157f90 0x7f02cc0cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1a53700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:46782/0 (socket says 172.21.15.7:46782) 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1252700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02cc157f90 0x7f02cc0cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:33770/0 (socket says 172.21.15.7:33770) 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1a53700 1 -- 172.21.15.7:0/4073428068 learned_addr learned my addr 172.21.15.7:0/4073428068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d2254700 1 --1- 172.21.15.7:0/4073428068 >> v1:172.21.15.113:6789/0 conn(0x7f02cc0cca30 0x7f02cc0c8f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:38004/0 (socket says 172.21.15.7:38004) 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1a53700 1 -- 172.21.15.7:0/4073428068 >> v2:172.21.15.113:3300/0 conn(0x7f02cc157f90 msgr2=0x7f02cc0cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1a53700 1 --2- 172.21.15.7:0/4073428068 >> v2:172.21.15.113:3300/0 conn(0x7f02cc157f90 0x7f02cc0cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1a53700 1 -- 172.21.15.7:0/4073428068 >> v1:172.21.15.113:6789/0 conn(0x7f02cc0cca30 legacy=0x7f02cc0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1a53700 1 -- 172.21.15.7:0/4073428068 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc0ca360 con 0x7f02cc157bc0 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1252700 1 --2- 172.21.15.7:0/4073428068 >> v2:172.21.15.113:3300/0 conn(0x7f02cc157f90 0x7f02cc0cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d1a53700 1 --2- 172.21.15.7:0/4073428068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0cf270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009990 tx=0x7f02c8009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f974bfb559dd1cfb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.728+0000 7f02d0a51700 1 -- 172.21.15.7:0/4073428068 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8009e80 con 0x7f02cc157bc0 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d0a51700 1 -- 172.21.15.7:0/4073428068 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c8005c10 con 0x7f02cc157bc0 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d0a51700 1 -- 172.21.15.7:0/4073428068 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c800b650 con 0x7f02cc157bc0 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d3cb7700 1 -- 172.21.15.7:0/4073428068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 msgr2=0x7f02cc0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/4073428068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0cf270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009990 tx=0x7f02c8009220 comp rx=0 tx=0).stop 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d3cb7700 1 -- 172.21.15.7:0/4073428068 shutdown_connections 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/4073428068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0cf270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/4073428068 >> v2:172.21.15.113:3300/0 conn(0x7f02cc157f90 0x7f02cc0cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d3cb7700 1 -- 172.21.15.7:0/4073428068 >> 172.21.15.7:0/4073428068 conn(0x7f02cc097a50 msgr2=0x7f02cc0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d3cb7700 1 -- 172.21.15.7:0/4073428068 shutdown_connections 2024-01-11T07:07:29.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.729+0000 7f02d3cb7700 1 -- 172.21.15.7:0/4073428068 wait complete. 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.730+0000 7f02d3cb7700 1 Processor -- start 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.730+0000 7f02d3cb7700 1 -- start start 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d3cb7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02cc0cca30 0x7f02cc0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d3cb7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d3cb7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02cc157f90 0x7f02cc156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d3cb7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc0c5f10 con 0x7f02cc157f90 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d3cb7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc159770 con 0x7f02cc0cca30 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d3cb7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc15bd70 con 0x7f02cc157bc0 2024-01-11T07:07:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d2254700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02cc157f90 0x7f02cc156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d2254700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02cc157f90 0x7f02cc156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34210/0 (socket says 172.21.15.7:34210) 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.731+0000 7f02d2254700 1 -- 172.21.15.7:0/388602645 learned_addr learned my addr 172.21.15.7:0/388602645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d1a53700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02cc0cca30 0x7f02cc0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d1252700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d2254700 1 -- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 msgr2=0x7f02cc0c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d2254700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d2254700 1 -- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02cc0cca30 msgr2=0x7f02cc0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d2254700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02cc0cca30 0x7f02cc0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d2254700 1 -- 172.21.15.7:0/388602645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc009420 con 0x7f02cc157f90 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d2254700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02cc157f90 0x7f02cc156e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f02c400cd00 tx=0x7f02c4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02c1ffb700 1 -- 172.21.15.7:0/388602645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c400ce60 con 0x7f02cc157f90 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02c1ffb700 1 -- 172.21.15.7:0/388602645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c4005df0 con 0x7f02cc157f90 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c8009480 con 0x7f02cc157f90 2024-01-11T07:07:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02c1ffb700 1 -- 172.21.15.7:0/388602645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c400ab00 con 0x7f02cc157f90 2024-01-11T07:07:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.732+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc159400 con 0x7f02cc157f90 2024-01-11T07:07:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.733+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc159900 con 0x7f02cc157f90 2024-01-11T07:07:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.734+0000 7f02c1ffb700 1 -- 172.21.15.7:0/388602645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f02c40070f0 con 0x7f02cc157f90 2024-01-11T07:07:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.734+0000 7f02c1ffb700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b0066f30 0x7f02b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.734+0000 7f02c1ffb700 1 -- 172.21.15.7:0/388602645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f02c4084540 con 0x7f02cc157f90 2024-01-11T07:07:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.734+0000 7f02d1a53700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b0066f30 0x7f02b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:29.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.735+0000 7f02d1a53700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b0066f30 0x7f02b00693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009260 tx=0x7f02c8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:29.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.739+0000 7f02c1ffb700 1 -- 172.21.15.7:0/388602645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02c4051830 con 0x7f02cc157f90 2024-01-11T07:07:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.938+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02cc04c3c0 con 0x7f02cc157f90 2024-01-11T07:07:29.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.939+0000 7f02c1ffb700 1 -- 172.21.15.7:0/388602645 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02c4054e50 con 0x7f02cc157f90 2024-01-11T07:07:29.938 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b0066f30 msgr2=0x7f02b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b0066f30 0x7f02b00693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009260 tx=0x7f02c8016000 comp rx=0 tx=0).stop 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02cc157f90 msgr2=0x7f02cc156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02cc157f90 0x7f02cc156e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f02c400cd00 tx=0x7f02c4001b10 comp rx=0 tx=0).stop 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 shutdown_connections 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b0066f30 0x7f02b00693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02cc157f90 0x7f02cc156e50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02cc0cca30 0x7f02cc0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 --2- 172.21.15.7:0/388602645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02cc157bc0 0x7f02cc0c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 >> 172.21.15.7:0/388602645 conn(0x7f02cc097a50 msgr2=0x7f02cc0cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 shutdown_connections 2024-01-11T07:07:29.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:29.941+0000 7f02d3cb7700 1 -- 172.21.15.7:0/388602645 wait complete. 2024-01-11T07:07:29.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:07:30.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.090+0000 7f12b0a8a700 1 Processor -- start 2024-01-11T07:07:30.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.090+0000 7f12b0a8a700 1 -- start start 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.091+0000 7f12b0a8a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac156e80 0x7f12ac0c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.091+0000 7f12b0a8a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12ac0c2c80 0x7f12ac0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.091+0000 7f12b0a8a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac158200 con 0x7f12ac0aee00 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.091+0000 7f12b0a8a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ac0c3100 con 0x7f12ac0c2c80 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.091+0000 7f12b0a8a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac1575f0 con 0x7f12ac156e80 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12aad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12ac0c2c80 0x7f12ac0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12aad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12ac0c2c80 0x7f12ac0cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34220/0 (socket says 172.21.15.7:34220) 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12aad9d700 1 -- 172.21.15.7:0/611690752 learned_addr learned my addr 172.21.15.7:0/611690752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12a9d9b700 1 --2- 172.21.15.7:0/611690752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac156e80 0x7f12ac0c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12aad9d700 1 -- 172.21.15.7:0/611690752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac156e80 msgr2=0x7f12ac0c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12aad9d700 1 --2- 172.21.15.7:0/611690752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac156e80 0x7f12ac0c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12aad9d700 1 -- 172.21.15.7:0/611690752 >> v1:172.21.15.113:6789/0 conn(0x7f12ac0aee00 legacy=0x7f12ac15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12aad9d700 1 -- 172.21.15.7:0/611690752 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac1578d0 con 0x7f12ac0c2c80 2024-01-11T07:07:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12a9d9b700 1 --2- 172.21.15.7:0/611690752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac156e80 0x7f12ac0c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12aad9d700 1 --2- 172.21.15.7:0/611690752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac0c2c80 0x7f12ac0cb0f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009f20 tx=0x7f12a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7517e7b914b819c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12a959a700 1 -- 172.21.15.7:0/611690752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a400e070 con 0x7f12ac0c2c80 2024-01-11T07:07:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12a959a700 1 -- 172.21.15.7:0/611690752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a400ba80 con 0x7f12ac0c2c80 2024-01-11T07:07:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.092+0000 7f12a959a700 1 -- 172.21.15.7:0/611690752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40092e0 con 0x7f12ac0c2c80 2024-01-11T07:07:30.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.093+0000 7f12b0a8a700 1 -- 172.21.15.7:0/611690752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac0c2c80 msgr2=0x7f12ac0cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.093+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/611690752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac0c2c80 0x7f12ac0cb0f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009f20 tx=0x7f12a4009840 comp rx=0 tx=0).stop 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.093+0000 7f12b0a8a700 1 -- 172.21.15.7:0/611690752 shutdown_connections 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.093+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/611690752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac0c2c80 0x7f12ac0cb0f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.093+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/611690752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac156e80 0x7f12ac0c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.093+0000 7f12b0a8a700 1 -- 172.21.15.7:0/611690752 >> 172.21.15.7:0/611690752 conn(0x7f12ac063230 msgr2=0x7f12ac063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.093+0000 7f12b0a8a700 1 -- 172.21.15.7:0/611690752 shutdown_connections 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.093+0000 7f12b0a8a700 1 -- 172.21.15.7:0/611690752 wait complete. 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.094+0000 7f12b0a8a700 1 Processor -- start 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.094+0000 7f12b0a8a700 1 -- start start 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12b0a8a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12ac0aee00 0x7f12ac1412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12b0a8a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac0c2c80 0x7f12ac1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12b0a8a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac156e80 0x7f12ac158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12b0a8a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac1417e0 con 0x7f12ac156e80 2024-01-11T07:07:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12b0a8a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0b0e80 con 0x7f12ac0aee00 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12b0a8a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0b0270 con 0x7f12ac0c2c80 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac156e80 0x7f12ac158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac156e80 0x7f12ac158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34232/0 (socket says 172.21.15.7:34232) 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aad9d700 1 -- 172.21.15.7:0/3037835382 learned_addr learned my addr 172.21.15.7:0/3037835382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aa59c700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12ac0aee00 0x7f12ac1412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12a9d9b700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac0c2c80 0x7f12ac1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aad9d700 1 -- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac0c2c80 msgr2=0x7f12ac1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aad9d700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac0c2c80 0x7f12ac1584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aad9d700 1 -- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12ac0aee00 msgr2=0x7f12ac1412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aad9d700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12ac0aee00 0x7f12ac1412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.095+0000 7f12aad9d700 1 -- 172.21.15.7:0/3037835382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a4009ae0 con 0x7f12ac156e80 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.096+0000 7f12aad9d700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac156e80 0x7f12ac158a10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f12a4008950 tx=0x7f12a4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.096+0000 7f12a27fc700 1 -- 172.21.15.7:0/3037835382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4008ea0 con 0x7f12ac156e80 2024-01-11T07:07:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.096+0000 7f12a27fc700 1 -- 172.21.15.7:0/3037835382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a40032f0 con 0x7f12ac156e80 2024-01-11T07:07:30.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.096+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac0b0550 con 0x7f12ac156e80 2024-01-11T07:07:30.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.096+0000 7f12a27fc700 1 -- 172.21.15.7:0/3037835382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40266b0 con 0x7f12ac156e80 2024-01-11T07:07:30.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.096+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac0b0b10 con 0x7f12ac156e80 2024-01-11T07:07:30.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.097+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ac04c3c0 con 0x7f12ac156e80 2024-01-11T07:07:30.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.097+0000 7f12a27fc700 1 -- 172.21.15.7:0/3037835382 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f12a4003040 con 0x7f12ac156e80 2024-01-11T07:07:30.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.098+0000 7f12a27fc700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f128c066f30 0x7f128c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.098+0000 7f12a27fc700 1 -- 172.21.15.7:0/3037835382 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12a4086430 con 0x7f12ac156e80 2024-01-11T07:07:30.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.098+0000 7f12aa59c700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f128c066f30 0x7f128c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.099+0000 7f12aa59c700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f128c066f30 0x7f128c0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f129c003860 tx=0x7f129c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.102+0000 7f12a27fc700 1 -- 172.21.15.7:0/3037835382 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12a40536a0 con 0x7f12ac156e80 2024-01-11T07:07:30.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:07:30.263 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:07:30.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.308+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12ac0b0fc0 con 0x7f12ac156e80 2024-01-11T07:07:30.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.308+0000 7f12a27fc700 1 -- 172.21.15.7:0/3037835382 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12a4056cc0 con 0x7f12ac156e80 2024-01-11T07:07:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:07:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.310+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f128c066f30 msgr2=0x7f128c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f128c066f30 0x7f128c0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f129c003860 tx=0x7f129c009040 comp rx=0 tx=0).stop 2024-01-11T07:07:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac156e80 msgr2=0x7f12ac158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac156e80 0x7f12ac158a10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f12a4008950 tx=0x7f12a4018000 comp rx=0 tx=0).stop 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 shutdown_connections 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f128c066f30 0x7f128c0693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ac156e80 0x7f12ac158a10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12ac0aee00 0x7f12ac1412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 --2- 172.21.15.7:0/3037835382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12ac0c2c80 0x7f12ac1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 >> 172.21.15.7:0/3037835382 conn(0x7f12ac063230 msgr2=0x7f12ac06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 shutdown_connections 2024-01-11T07:07:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.311+0000 7f12b0a8a700 1 -- 172.21.15.7:0/3037835382 wait complete. 2024-01-11T07:07:30.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:07:30.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.501+0000 7f02d661e700 1 Processor -- start 2024-01-11T07:07:30.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.501+0000 7f02d661e700 1 -- start start 2024-01-11T07:07:30.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.501+0000 7f02d661e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02d006de30 0x7f02d0154210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.501+0000 7f02d661e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d0154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.501+0000 7f02d661e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d015a790 con 0x7f02d006e200 2024-01-11T07:07:30.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.501+0000 7f02d661e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d015b7a0 con 0x7f02d006de30 2024-01-11T07:07:30.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.501+0000 7f02d661e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d015a9e0 con 0x7f02d006b450 2024-01-11T07:07:30.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02d006de30 0x7f02d0154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02cf7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f02d006e200 0x7f02d006ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46734/0 (socket says 172.21.15.7:46734) 2024-01-11T07:07:30.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02cf7fe700 1 -- 172.21.15.7:0/995126171 learned_addr learned my addr 172.21.15.7:0/995126171 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02d4bbb700 1 --2- 172.21.15.7:0/995126171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d0154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02cffff700 1 -- 172.21.15.7:0/995126171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 msgr2=0x7f02d0154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02cffff700 1 --2- 172.21.15.7:0/995126171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d0154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02cffff700 1 -- 172.21.15.7:0/995126171 >> v1:172.21.15.7:6789/0 conn(0x7f02d006e200 legacy=0x7f02d006ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02cffff700 1 -- 172.21.15.7:0/995126171 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d015acc0 con 0x7f02d006de30 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02d4bbb700 1 --2- 172.21.15.7:0/995126171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d0154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02ceffd700 1 -- 172.21.15.7:0/995126171 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60153731 0 0) 0x7f02d015a790 con 0x7f02d006e200 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02cffff700 1 --2- 172.21.15.7:0/995126171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006de30 0x7f02d0154210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f02c00098e0 tx=0x7f02c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b425d7097019c58 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02ceffd700 1 -- 172.21.15.7:0/995126171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0009e80 con 0x7f02d006de30 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02ceffd700 1 -- 172.21.15.7:0/995126171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c000b860 con 0x7f02d006de30 2024-01-11T07:07:30.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.502+0000 7f02ceffd700 1 -- 172.21.15.7:0/995126171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c000bbe0 con 0x7f02d006de30 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.503+0000 7f02d661e700 1 -- 172.21.15.7:0/995126171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006de30 msgr2=0x7f02d0154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.503+0000 7f02d661e700 1 --2- 172.21.15.7:0/995126171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006de30 0x7f02d0154210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f02c00098e0 tx=0x7f02c00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.503+0000 7f02d661e700 1 -- 172.21.15.7:0/995126171 shutdown_connections 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.503+0000 7f02d661e700 1 --2- 172.21.15.7:0/995126171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d0154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.503+0000 7f02d661e700 1 --2- 172.21.15.7:0/995126171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006de30 0x7f02d0154210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.503+0000 7f02d661e700 1 -- 172.21.15.7:0/995126171 >> 172.21.15.7:0/995126171 conn(0x7f02d00cd8f0 msgr2=0x7f02d00cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.503+0000 7f02d661e700 1 -- 172.21.15.7:0/995126171 shutdown_connections 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.503+0000 7f02d661e700 1 -- 172.21.15.7:0/995126171 wait complete. 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.504+0000 7f02d661e700 1 Processor -- start 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.505+0000 7f02d661e700 1 -- start start 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.505+0000 7f02d661e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d01572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.505+0000 7f02d661e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d006de30 0x7f02d00c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.505+0000 7f02d661e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006e200 0x7f02d00c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.505+0000 7f02d661e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0157830 con 0x7f02d006e200 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.505+0000 7f02d661e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d00adb60 con 0x7f02d006de30 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.505+0000 7f02d661e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d00ae6e0 con 0x7f02d006b450 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006e200 0x7f02d00c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006e200 0x7f02d00c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34256/0 (socket says 172.21.15.7:34256) 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 -- 172.21.15.7:0/2711663760 learned_addr learned my addr 172.21.15.7:0/2711663760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02cffff700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d01572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02cf7fe700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d006de30 0x7f02d00c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 -- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 msgr2=0x7f02d01572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d01572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 -- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d006de30 msgr2=0x7f02d00c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d006de30 0x7f02d00c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 -- 172.21.15.7:0/2711663760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0009420 con 0x7f02d006e200 2024-01-11T07:07:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d4bbb700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006e200 0x7f02d00c7130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f02c800d180 tx=0x7f02c8009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02b7fff700 1 -- 172.21.15.7:0/2711663760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c800dcc0 con 0x7f02d006e200 2024-01-11T07:07:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02b7fff700 1 -- 172.21.15.7:0/2711663760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c800f040 con 0x7f02d006e200 2024-01-11T07:07:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d00addb0 con 0x7f02d006e200 2024-01-11T07:07:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02b7fff700 1 -- 172.21.15.7:0/2711663760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8015690 con 0x7f02d006e200 2024-01-11T07:07:30.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.506+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d00ae370 con 0x7f02d006e200 2024-01-11T07:07:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.507+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d00c7d00 con 0x7f02d006e200 2024-01-11T07:07:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.508+0000 7f02b7fff700 1 -- 172.21.15.7:0/2711663760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f02c801e3b0 con 0x7f02d006e200 2024-01-11T07:07:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.508+0000 7f02b7fff700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02bc066f30 0x7f02bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.508+0000 7f02b7fff700 1 -- 172.21.15.7:0/2711663760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f02c8084a90 con 0x7f02d006e200 2024-01-11T07:07:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.508+0000 7f02cffff700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02bc066f30 0x7f02bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.509+0000 7f02cffff700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02bc066f30 0x7f02bc0693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02c00091e0 tx=0x7f02c0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.513+0000 7f02b7fff700 1 -- 172.21.15.7:0/2711663760 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02c8051c60 con 0x7f02d006e200 2024-01-11T07:07:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.715+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02d0064a40 con 0x7f02d006e200 2024-01-11T07:07:30.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.715+0000 7f02b7fff700 1 -- 172.21.15.7:0/2711663760 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02c801a0a0 con 0x7f02d006e200 2024-01-11T07:07:30.715 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:07:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02bc066f30 msgr2=0x7f02bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02bc066f30 0x7f02bc0693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02c00091e0 tx=0x7f02c0017000 comp rx=0 tx=0).stop 2024-01-11T07:07:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006e200 msgr2=0x7f02d00c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006e200 0x7f02d00c7130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f02c800d180 tx=0x7f02c8009a50 comp rx=0 tx=0).stop 2024-01-11T07:07:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 shutdown_connections 2024-01-11T07:07:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02bc066f30 0x7f02bc0693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d006e200 0x7f02d00c7130 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d006b450 0x7f02d01572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 --2- 172.21.15.7:0/2711663760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d006de30 0x7f02d00c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.717+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 >> 172.21.15.7:0/2711663760 conn(0x7f02d00cd8f0 msgr2=0x7f02d0151eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.718+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 shutdown_connections 2024-01-11T07:07:30.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.718+0000 7f02d661e700 1 -- 172.21.15.7:0/2711663760 wait complete. 2024-01-11T07:07:30.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:30.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.863+0000 7f68d6e64700 1 Processor -- start 2024-01-11T07:07:30.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.863+0000 7f68d6e64700 1 -- start start 2024-01-11T07:07:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d6e64700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68d00cafb0 0x7f68d00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d6e64700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d0159300 con 0x7f68d00b0b40 2024-01-11T07:07:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d6e64700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d00cf260 con 0x7f68d0158280 2024-01-11T07:07:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d6e64700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00cf540 con 0x7f68d00cafb0 2024-01-11T07:07:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d4c00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68d00cafb0 0x7f68d00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d4c00700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68d00cafb0 0x7f68d00ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34262/0 (socket says 172.21.15.7:34262) 2024-01-11T07:07:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d4c00700 1 -- 172.21.15.7:0/1194094137 learned_addr learned my addr 172.21.15.7:0/1194094137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d4c00700 1 -- 172.21.15.7:0/1194094137 >> v1:172.21.15.113:6789/0 conn(0x7f68d0158280 legacy=0x7f68d00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d4c00700 1 -- 172.21.15.7:0/1194094137 >> v1:172.21.15.7:6789/0 conn(0x7f68d00b0b40 legacy=0x7f68d0161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.864+0000 7f68d4c00700 1 -- 172.21.15.7:0/1194094137 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00cf820 con 0x7f68d00cafb0 2024-01-11T07:07:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.865+0000 7f68d4c00700 1 --2- 172.21.15.7:0/1194094137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d00cafb0 0x7f68d00ce8f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f68c40099b0 tx=0x7f68c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf697219aa090bd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.865+0000 7f68cf7fe700 1 -- 172.21.15.7:0/1194094137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c400b860 con 0x7f68d00cafb0 2024-01-11T07:07:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.865+0000 7f68cf7fe700 1 -- 172.21.15.7:0/1194094137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c400b9c0 con 0x7f68d00cafb0 2024-01-11T07:07:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.865+0000 7f68cf7fe700 1 -- 172.21.15.7:0/1194094137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c400bd60 con 0x7f68d00cafb0 2024-01-11T07:07:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.865+0000 7f68d6e64700 1 -- 172.21.15.7:0/1194094137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d00cafb0 msgr2=0x7f68d00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.865+0000 7f68d6e64700 1 --2- 172.21.15.7:0/1194094137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d00cafb0 0x7f68d00ce8f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f68c40099b0 tx=0x7f68c4009250 comp rx=0 tx=0).stop 2024-01-11T07:07:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.866+0000 7f68d6e64700 1 -- 172.21.15.7:0/1194094137 shutdown_connections 2024-01-11T07:07:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.866+0000 7f68d6e64700 1 --2- 172.21.15.7:0/1194094137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d00cafb0 0x7f68d00ce8f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.866+0000 7f68d6e64700 1 -- 172.21.15.7:0/1194094137 >> 172.21.15.7:0/1194094137 conn(0x7f68d0071100 msgr2=0x7f68d006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.866+0000 7f68d6e64700 1 -- 172.21.15.7:0/1194094137 shutdown_connections 2024-01-11T07:07:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.866+0000 7f68d6e64700 1 -- 172.21.15.7:0/1194094137 wait complete. 2024-01-11T07:07:30.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.867+0000 7f68d6e64700 1 Processor -- start 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.867+0000 7f68d6e64700 1 -- start start 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d6e64700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68d00b0b40 0x7f68d0151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d6e64700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00cafb0 0x7f68d015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d6e64700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0158280 0x7f68d015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d6e64700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0153330 con 0x7f68d0158280 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d6e64700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0152720 con 0x7f68d00cafb0 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d6e64700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0152a00 con 0x7f68d00b0b40 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0158280 0x7f68d015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0158280 0x7f68d015bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34274/0 (socket says 172.21.15.7:34274) 2024-01-11T07:07:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 -- 172.21.15.7:0/1209101859 learned_addr learned my addr 172.21.15.7:0/1209101859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d4c00700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68d00b0b40 0x7f68d0151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68cffff700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00cafb0 0x7f68d015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 -- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68d00b0b40 msgr2=0x7f68d0151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68d00b0b40 0x7f68d0151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 -- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00cafb0 msgr2=0x7f68d015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00cafb0 0x7f68d015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 -- 172.21.15.7:0/1209101859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c40094f0 con 0x7f68d0158280 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.868+0000 7f68d5401700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0158280 0x7f68d015bc00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c8e0 tx=0x7f68c8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.869+0000 7f68ccff9700 1 -- 172.21.15.7:0/1209101859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800ce80 con 0x7f68d0158280 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.869+0000 7f68ccff9700 1 -- 172.21.15.7:0/1209101859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c800a7e0 con 0x7f68d0158280 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.869+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d0152ce0 con 0x7f68d0158280 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.869+0000 7f68ccff9700 1 -- 172.21.15.7:0/1209101859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800ab80 con 0x7f68d0158280 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.869+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d01537c0 con 0x7f68d0158280 2024-01-11T07:07:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.870+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d004c3c0 con 0x7f68d0158280 2024-01-11T07:07:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.870+0000 7f68ccff9700 1 -- 172.21.15.7:0/1209101859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f68c8012420 con 0x7f68d0158280 2024-01-11T07:07:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.871+0000 7f68ccff9700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f68b4066f30 0x7f68b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.871+0000 7f68ccff9700 1 -- 172.21.15.7:0/1209101859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68c8084820 con 0x7f68d0158280 2024-01-11T07:07:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.871+0000 7f68d4c00700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f68b4066f30 0x7f68b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.871+0000 7f68d4c00700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f68b4066f30 0x7f68b40693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f68c40099b0 tx=0x7f68c4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:30.875+0000 7f68ccff9700 1 -- 172.21.15.7:0/1209101859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68c8051a90 con 0x7f68d0158280 2024-01-11T07:07:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.083+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68d0159550 con 0x7f68d0158280 2024-01-11T07:07:31.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.084+0000 7f68ccff9700 1 -- 172.21.15.7:0/1209101859 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f68c80550b0 con 0x7f68d0158280 2024-01-11T07:07:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:31.084 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f68b4066f30 msgr2=0x7f68b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f68b4066f30 0x7f68b40693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f68c40099b0 tx=0x7f68c4017000 comp rx=0 tx=0).stop 2024-01-11T07:07:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0158280 msgr2=0x7f68d015bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0158280 0x7f68d015bc00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c8e0 tx=0x7f68c8001710 comp rx=0 tx=0).stop 2024-01-11T07:07:31.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 shutdown_connections 2024-01-11T07:07:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f68b4066f30 0x7f68b40693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68d0158280 0x7f68d015bc00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68d00b0b40 0x7f68d0151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 --2- 172.21.15.7:0/1209101859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68d00cafb0 0x7f68d015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.086+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 >> 172.21.15.7:0/1209101859 conn(0x7f68d0071100 msgr2=0x7f68d0155320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.087+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 shutdown_connections 2024-01-11T07:07:31.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.087+0000 7f68d6e64700 1 -- 172.21.15.7:0/1209101859 wait complete. 2024-01-11T07:07:31.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:06:56.038804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:06:57.365958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:07:31.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:07:31.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.236+0000 7f630d05b700 1 Processor -- start 2024-01-11T07:07:31.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.236+0000 7f630d05b700 1 -- start start 2024-01-11T07:07:31.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f630d05b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157bc0 0x7f63080cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f630d05b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63080ca080 con 0x7f6308157f90 2024-01-11T07:07:31.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f630d05b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63080ca360 con 0x7f63080ccc00 2024-01-11T07:07:31.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f630d05b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63080ca640 con 0x7f6308157bc0 2024-01-11T07:07:31.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f630659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6308157f90 0x7f63080cc4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46760/0 (socket says 172.21.15.7:46760) 2024-01-11T07:07:31.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f630659c700 1 -- 172.21.15.7:0/2382194965 learned_addr learned my addr 172.21.15.7:0/2382194965 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f6306d9d700 1 --2- 172.21.15.7:0/2382194965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157bc0 0x7f63080cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f6305d9b700 1 -- 172.21.15.7:0/2382194965 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604013063 0 0) 0x7f63080ca080 con 0x7f6308157f90 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f6305d9b700 1 -- 172.21.15.7:0/2382194965 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62f0003570 con 0x7f6308157f90 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f6305d9b700 1 -- 172.21.15.7:0/2382194965 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280394689 0 0) 0x7f63080ca360 con 0x7f63080ccc00 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f6305d9b700 1 -- 172.21.15.7:0/2382194965 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63080ca080 con 0x7f63080ccc00 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f6306d9d700 1 -- 172.21.15.7:0/2382194965 >> v1:172.21.15.113:6789/0 conn(0x7f63080ccc00 legacy=0x7f63080b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f6306d9d700 1 -- 172.21.15.7:0/2382194965 >> v1:172.21.15.7:6789/0 conn(0x7f6308157f90 legacy=0x7f63080cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.237+0000 7f6306d9d700 1 -- 172.21.15.7:0/2382194965 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63080b18c0 con 0x7f6308157bc0 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f6306d9d700 1 --2- 172.21.15.7:0/2382194965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157bc0 0x7f63080cf270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009900 tx=0x7f62fc009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=361fafd1dc24941 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f6305d9b700 1 -- 172.21.15.7:0/2382194965 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc005c10 con 0x7f6308157bc0 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f6305d9b700 1 -- 172.21.15.7:0/2382194965 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc005d70 con 0x7f6308157bc0 2024-01-11T07:07:31.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f6305d9b700 1 -- 172.21.15.7:0/2382194965 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc006240 con 0x7f6308157bc0 2024-01-11T07:07:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f630d05b700 1 -- 172.21.15.7:0/2382194965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157bc0 msgr2=0x7f63080cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f630d05b700 1 --2- 172.21.15.7:0/2382194965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157bc0 0x7f63080cf270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009900 tx=0x7f62fc009120 comp rx=0 tx=0).stop 2024-01-11T07:07:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f630d05b700 1 -- 172.21.15.7:0/2382194965 shutdown_connections 2024-01-11T07:07:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f630d05b700 1 --2- 172.21.15.7:0/2382194965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157bc0 0x7f63080cf270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f630d05b700 1 -- 172.21.15.7:0/2382194965 >> 172.21.15.7:0/2382194965 conn(0x7f6308097a50 msgr2=0x7f63080ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f630d05b700 1 -- 172.21.15.7:0/2382194965 shutdown_connections 2024-01-11T07:07:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.238+0000 7f630d05b700 1 -- 172.21.15.7:0/2382194965 wait complete. 2024-01-11T07:07:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.240+0000 7f630d05b700 1 Processor -- start 2024-01-11T07:07:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.240+0000 7f630d05b700 1 -- start start 2024-01-11T07:07:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630d05b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63080ccc00 0x7f63080c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630d05b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308157bc0 0x7f630816a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630d05b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157f90 0x7f630816d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630d05b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63080b1ca0 con 0x7f6308157bc0 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630d05b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63080c5fb0 con 0x7f63080ccc00 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630d05b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63080c6290 con 0x7f6308157f90 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308157bc0 0x7f630816a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308157bc0 0x7f630816a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34288/0 (socket says 172.21.15.7:34288) 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630659c700 1 -- 172.21.15.7:0/3851286307 learned_addr learned my addr 172.21.15.7:0/3851286307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f6306d9d700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63080ccc00 0x7f63080c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630759e700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157f90 0x7f630816d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630659c700 1 -- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157f90 msgr2=0x7f630816d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630659c700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157f90 0x7f630816d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630659c700 1 -- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63080ccc00 msgr2=0x7f63080c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630659c700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63080ccc00 0x7f63080c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.241+0000 7f630659c700 1 -- 172.21.15.7:0/3851286307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0093c0 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.242+0000 7f630659c700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308157bc0 0x7f630816a2c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c790 tx=0x7f62f8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.242+0000 7f62eeffd700 1 -- 172.21.15.7:0/3851286307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8005b30 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.242+0000 7f62eeffd700 1 -- 172.21.15.7:0/3851286307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8005c90 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.242+0000 7f62eeffd700 1 -- 172.21.15.7:0/3851286307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8007300 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.242+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63080c6570 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.242+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63080c6850 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.243+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630816a110 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.243+0000 7f62eeffd700 1 -- 172.21.15.7:0/3851286307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f62f801b3b0 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.244+0000 7f62eeffd700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f62f4066f30 0x7f62f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.244+0000 7f62eeffd700 1 -- 172.21.15.7:0/3851286307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f62f8083960 con 0x7f6308157bc0 2024-01-11T07:07:31.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.244+0000 7f6306d9d700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f62f4066f30 0x7f62f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.244+0000 7f6306d9d700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f62f4066f30 0x7f62f40693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009160 tx=0x7f62fc016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.248+0000 7f62eeffd700 1 -- 172.21.15.7:0/3851286307 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62f8054660 con 0x7f6308157bc0 2024-01-11T07:07:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.450+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f630804c3c0 con 0x7f6308157bc0 2024-01-11T07:07:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.450+0000 7f62eeffd700 1 -- 172.21.15.7:0/3851286307 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f62f8017020 con 0x7f6308157bc0 2024-01-11T07:07:31.450 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:07:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.452+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f62f4066f30 msgr2=0x7f62f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.452+0000 7f630d05b700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f62f4066f30 0x7f62f40693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009160 tx=0x7f62fc016000 comp rx=0 tx=0).stop 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.452+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308157bc0 msgr2=0x7f630816a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.452+0000 7f630d05b700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308157bc0 0x7f630816a2c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c790 tx=0x7f62f8007ea0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.453+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 shutdown_connections 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.453+0000 7f630d05b700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f62f4066f30 0x7f62f40693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.453+0000 7f630d05b700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6308157f90 0x7f630816d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.453+0000 7f630d05b700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63080ccc00 0x7f63080c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.453+0000 7f630d05b700 1 --2- 172.21.15.7:0/3851286307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6308157bc0 0x7f630816a2c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.453+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 >> 172.21.15.7:0/3851286307 conn(0x7f6308097a50 msgr2=0x7f63081557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.453+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 shutdown_connections 2024-01-11T07:07:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.453+0000 7f630d05b700 1 -- 172.21.15.7:0/3851286307 wait complete. 2024-01-11T07:07:31.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:07:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.600+0000 7f4874a42700 1 Processor -- start 2024-01-11T07:07:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.600+0000 7f4874a42700 1 -- start start 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.601+0000 7f4874a42700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f487006de30 0x7f4870154210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.601+0000 7f4874a42700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f487006b450 0x7f4870154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.601+0000 7f4874a42700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487015a790 con 0x7f487006e200 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.601+0000 7f4874a42700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487015b7a0 con 0x7f487006de30 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.601+0000 7f4874a42700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487015a9e0 con 0x7f487006b450 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.601+0000 7f486e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f487006de30 0x7f4870154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f487006e200 0x7f487006ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46762/0 (socket says 172.21.15.7:46762) 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486dd9b700 1 -- 172.21.15.7:0/3246241358 learned_addr learned my addr 172.21.15.7:0/3246241358 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486ed9d700 1 --2- 172.21.15.7:0/3246241358 >> v2:172.21.15.113:3300/0 conn(0x7f487006b450 0x7f4870154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486e59c700 1 -- 172.21.15.7:0/3246241358 >> v2:172.21.15.113:3300/0 conn(0x7f487006b450 msgr2=0x7f4870154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486e59c700 1 --2- 172.21.15.7:0/3246241358 >> v2:172.21.15.113:3300/0 conn(0x7f487006b450 0x7f4870154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486e59c700 1 -- 172.21.15.7:0/3246241358 >> v1:172.21.15.7:6789/0 conn(0x7f487006e200 legacy=0x7f487006ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486e59c700 1 -- 172.21.15.7:0/3246241358 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487015acc0 con 0x7f487006de30 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486d59a700 1 -- 172.21.15.7:0/3246241358 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334476704 0 0) 0x7f487015a790 con 0x7f487006e200 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486e59c700 1 --2- 172.21.15.7:0/3246241358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f4870154210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48700c7ec0 tx=0x7f486000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6a90a8a8d6c947 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486d59a700 1 -- 172.21.15.7:0/3246241358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486000db70 con 0x7f487006de30 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486d59a700 1 -- 172.21.15.7:0/3246241358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486000dcd0 con 0x7f487006de30 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.602+0000 7f486d59a700 1 -- 172.21.15.7:0/3246241358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48600196b0 con 0x7f487006de30 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.603+0000 7f4874a42700 1 -- 172.21.15.7:0/3246241358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 msgr2=0x7f4870154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.603+0000 7f4874a42700 1 --2- 172.21.15.7:0/3246241358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f4870154210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48700c7ec0 tx=0x7f486000b430 comp rx=0 tx=0).stop 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.603+0000 7f4874a42700 1 -- 172.21.15.7:0/3246241358 shutdown_connections 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.603+0000 7f4874a42700 1 --2- 172.21.15.7:0/3246241358 >> v2:172.21.15.113:3300/0 conn(0x7f487006b450 0x7f4870154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.603+0000 7f4874a42700 1 --2- 172.21.15.7:0/3246241358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f4870154210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.603+0000 7f4874a42700 1 -- 172.21.15.7:0/3246241358 >> 172.21.15.7:0/3246241358 conn(0x7f48700cd8f0 msgr2=0x7f48700cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.603+0000 7f4874a42700 1 -- 172.21.15.7:0/3246241358 shutdown_connections 2024-01-11T07:07:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.603+0000 7f4874a42700 1 -- 172.21.15.7:0/3246241358 wait complete. 2024-01-11T07:07:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.604+0000 7f4874a42700 1 Processor -- start 2024-01-11T07:07:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.604+0000 7f4874a42700 1 -- start start 2024-01-11T07:07:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f4874a42700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f487006b450 0x7f487016a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f4874a42700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f487016e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f4874a42700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f487006e200 0x7f487006a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f4874a42700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487016aee0 con 0x7f487006de30 2024-01-11T07:07:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f4874a42700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4870152690 con 0x7f487006e200 2024-01-11T07:07:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f4874a42700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487016b970 con 0x7f487006b450 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f487016e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f487016e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34306/0 (socket says 172.21.15.7:34306) 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486dd9b700 1 -- 172.21.15.7:0/407140634 learned_addr learned my addr 172.21.15.7:0/407140634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486ed9d700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f487006e200 0x7f487006a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486e59c700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f487006b450 0x7f487016a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486dd9b700 1 -- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f487006b450 msgr2=0x7f487016a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486dd9b700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f487006b450 0x7f487016a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486dd9b700 1 -- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f487006e200 msgr2=0x7f487006a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486dd9b700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f487006e200 0x7f487006a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.605+0000 7f486dd9b700 1 -- 172.21.15.7:0/407140634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486000b6d0 con 0x7f487006de30 2024-01-11T07:07:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.606+0000 7f486dd9b700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f487016e960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f485800c5b0 tx=0x7f4858001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.606+0000 7f48667fc700 1 -- 172.21.15.7:0/407140634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858005b30 con 0x7f487006de30 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.606+0000 7f48667fc700 1 -- 172.21.15.7:0/407140634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4858005c90 con 0x7f487006de30 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.606+0000 7f48667fc700 1 -- 172.21.15.7:0/407140634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858007300 con 0x7f487006de30 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.606+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487016b0d0 con 0x7f487006de30 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.606+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487016b460 con 0x7f487006de30 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.607+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4870064a40 con 0x7f487006de30 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.607+0000 7f48667fc700 1 -- 172.21.15.7:0/407140634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f485801b3b0 con 0x7f487006de30 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.608+0000 7f48667fc700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4850066f30 0x7f48500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.608+0000 7f48667fc700 1 -- 172.21.15.7:0/407140634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4858083a90 con 0x7f487006de30 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.608+0000 7f486e59c700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4850066f30 0x7f48500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.609+0000 7f486e59c700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4850066f30 0x7f48500693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f487015a3f0 tx=0x7f486001b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.612+0000 7f48667fc700 1 -- 172.21.15.7:0/407140634 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4858050c60 con 0x7f487006de30 2024-01-11T07:07:31.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.817+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f487016eea0 con 0x7f487006de30 2024-01-11T07:07:31.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.817+0000 7f48667fc700 1 -- 172.21.15.7:0/407140634 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4858017020 con 0x7f487006de30 2024-01-11T07:07:31.817 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:07:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.819+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4850066f30 msgr2=0x7f48500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.819+0000 7f4874a42700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4850066f30 0x7f48500693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f487015a3f0 tx=0x7f486001b000 comp rx=0 tx=0).stop 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.819+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 msgr2=0x7f487016e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.819+0000 7f4874a42700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f487016e960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f485800c5b0 tx=0x7f4858001070 comp rx=0 tx=0).stop 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.820+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 shutdown_connections 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.820+0000 7f4874a42700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4850066f30 0x7f48500693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.820+0000 7f4874a42700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f487006e200 0x7f487006a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.820+0000 7f4874a42700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f487006b450 0x7f487016a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.820+0000 7f4874a42700 1 --2- 172.21.15.7:0/407140634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f487006de30 0x7f487016e960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.820+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 >> 172.21.15.7:0/407140634 conn(0x7f48700cd8f0 msgr2=0x7f48700c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.820+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 shutdown_connections 2024-01-11T07:07:31.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.820+0000 7f4874a42700 1 -- 172.21.15.7:0/407140634 wait complete. 2024-01-11T07:07:31.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:07:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.967+0000 7f1f6b05e700 1 Processor -- start 2024-01-11T07:07:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.967+0000 7f1f6b05e700 1 -- start start 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.968+0000 7f1f6b05e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f640b0390 0x7f1f6406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.968+0000 7f1f6b05e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f6415a2a0 0x7f1f6406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.968+0000 7f1f6b05e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640d0130 con 0x7f1f64159ed0 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.968+0000 7f1f6b05e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640cc5d0 con 0x7f1f640b0390 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.968+0000 7f1f6b05e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f640cfae0 con 0x7f1f6415a2a0 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.968+0000 7f1f63fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f640b0390 0x7f1f6406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f68dfa700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f64159ed0 0x7f1f640cbba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46768/0 (socket says 172.21.15.7:46768) 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f68dfa700 1 -- 172.21.15.7:0/2502646598 learned_addr learned my addr 172.21.15.7:0/2502646598 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f63fff700 1 --2- 172.21.15.7:0/2502646598 >> v2:172.21.15.7:3300/0 conn(0x7f1f640b0390 0x7f1f6406dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34322/0 (socket says 172.21.15.7:34322) 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f695fb700 1 --2- 172.21.15.7:0/2502646598 >> v2:172.21.15.113:3300/0 conn(0x7f1f6415a2a0 0x7f1f6406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f63fff700 1 -- 172.21.15.7:0/2502646598 >> v2:172.21.15.113:3300/0 conn(0x7f1f6415a2a0 msgr2=0x7f1f6406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f63fff700 1 --2- 172.21.15.7:0/2502646598 >> v2:172.21.15.113:3300/0 conn(0x7f1f6415a2a0 0x7f1f6406e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f63fff700 1 -- 172.21.15.7:0/2502646598 >> v1:172.21.15.7:6789/0 conn(0x7f1f64159ed0 legacy=0x7f1f640cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f63fff700 1 -- 172.21.15.7:0/2502646598 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640cfdc0 con 0x7f1f640b0390 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f637fe700 1 -- 172.21.15.7:0/2502646598 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227605739 0 0) 0x7f1f640d0130 con 0x7f1f64159ed0 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f695fb700 1 --2- 172.21.15.7:0/2502646598 >> v2:172.21.15.113:3300/0 conn(0x7f1f6415a2a0 0x7f1f6406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f63fff700 1 --2- 172.21.15.7:0/2502646598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640b0390 0x7f1f6406dc30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f64089c80 tx=0x7f1f4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99879cf9468975a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f637fe700 1 -- 172.21.15.7:0/2502646598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00b6b0 con 0x7f1f640b0390 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f637fe700 1 -- 172.21.15.7:0/2502646598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00b810 con 0x7f1f640b0390 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.969+0000 7f1f637fe700 1 -- 172.21.15.7:0/2502646598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00bbb0 con 0x7f1f640b0390 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.970+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2502646598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640b0390 msgr2=0x7f1f6406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.970+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2502646598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640b0390 0x7f1f6406dc30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1f64089c80 tx=0x7f1f4c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.970+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2502646598 shutdown_connections 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.970+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2502646598 >> v2:172.21.15.113:3300/0 conn(0x7f1f6415a2a0 0x7f1f6406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.970+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2502646598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f640b0390 0x7f1f6406dc30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.970+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2502646598 >> 172.21.15.7:0/2502646598 conn(0x7f1f64063d00 msgr2=0x7f1f640d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:31.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.970+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2502646598 shutdown_connections 2024-01-11T07:07:31.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.970+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2502646598 wait complete. 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.971+0000 7f1f6b05e700 1 Processor -- start 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.971+0000 7f1f6b05e700 1 -- start start 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f6b05e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f640b0390 0x7f1f640ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f6b05e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f64159ed0 0x7f1f6414ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f6b05e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6415a2a0 0x7f1f6415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f6b05e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640cac70 con 0x7f1f6415a2a0 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f6b05e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640c8f20 con 0x7f1f64159ed0 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f6b05e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640ca9b0 con 0x7f1f640b0390 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f695fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6415a2a0 0x7f1f6415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f695fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6415a2a0 0x7f1f6415a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34334/0 (socket says 172.21.15.7:34334) 2024-01-11T07:07:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f695fb700 1 -- 172.21.15.7:0/2371583614 learned_addr learned my addr 172.21.15.7:0/2371583614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f68dfa700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f640b0390 0x7f1f640ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.972+0000 7f1f63fff700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f64159ed0 0x7f1f6414ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f695fb700 1 -- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f640b0390 msgr2=0x7f1f640ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f695fb700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f640b0390 0x7f1f640ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f695fb700 1 -- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f64159ed0 msgr2=0x7f1f6414ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f695fb700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f64159ed0 0x7f1f6414ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f695fb700 1 -- 172.21.15.7:0/2371583614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c009420 con 0x7f1f6415a2a0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f695fb700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6415a2a0 0x7f1f6415a6e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009fa0 tx=0x7f1f5c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f60ff9700 1 -- 172.21.15.7:0/2371583614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00dd10 con 0x7f1f6415a2a0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f60ff9700 1 -- 172.21.15.7:0/2371583614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c00f040 con 0x7f1f6415a2a0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6414f390 con 0x7f1f6415a2a0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f60ff9700 1 -- 172.21.15.7:0/2371583614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c015650 con 0x7f1f6415a2a0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.973+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640b4e50 con 0x7f1f6415a2a0 2024-01-11T07:07:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.974+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6406f660 con 0x7f1f6415a2a0 2024-01-11T07:07:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.974+0000 7f1f60ff9700 1 -- 172.21.15.7:0/2371583614 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f5c015a00 con 0x7f1f6415a2a0 2024-01-11T07:07:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.975+0000 7f1f60ff9700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f44066f30 0x7f1f440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.975+0000 7f1f60ff9700 1 -- 172.21.15.7:0/2371583614 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f5c084d50 con 0x7f1f6415a2a0 2024-01-11T07:07:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.975+0000 7f1f68dfa700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f44066f30 0x7f1f440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.976+0000 7f1f68dfa700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f44066f30 0x7f1f440693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f540011e0 tx=0x7f1f54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:31.979+0000 7f1f60ff9700 1 -- 172.21.15.7:0/2371583614 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f5c088030 con 0x7f1f6415a2a0 2024-01-11T07:07:32.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.183+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f6404c3c0 con 0x7f1f6415a2a0 2024-01-11T07:07:32.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.183+0000 7f1f60ff9700 1 -- 172.21.15.7:0/2371583614 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f5c051fc0 con 0x7f1f6415a2a0 2024-01-11T07:07:32.183 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:07:32.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.185+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f44066f30 msgr2=0x7f1f440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:32.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.185+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f44066f30 0x7f1f440693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f540011e0 tx=0x7f1f54009040 comp rx=0 tx=0).stop 2024-01-11T07:07:32.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.185+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6415a2a0 msgr2=0x7f1f6415a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.185+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6415a2a0 0x7f1f6415a6e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009fa0 tx=0x7f1f5c0099b0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.186+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 shutdown_connections 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.186+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f44066f30 0x7f1f440693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.186+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f6415a2a0 0x7f1f6415a6e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.186+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f640b0390 0x7f1f640ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.186+0000 7f1f6b05e700 1 --2- 172.21.15.7:0/2371583614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f64159ed0 0x7f1f6414ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.186+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 >> 172.21.15.7:0/2371583614 conn(0x7f1f64063d00 msgr2=0x7f1f640c5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.186+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 shutdown_connections 2024-01-11T07:07:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.186+0000 7f1f6b05e700 1 -- 172.21.15.7:0/2371583614 wait complete. 2024-01-11T07:07:32.220 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-01-11T07:07:32.220 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2024-01-11T07:07:32.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-01-11T07:07:32.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.355+0000 7f65ad1fe700 1 Processor -- start 2024-01-11T07:07:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.355+0000 7f65ad1fe700 1 -- start start 2024-01-11T07:07:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65ad1fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a806dd00 0x7f65a80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65ad1fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f65a806e0d0 0x7f65a8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65ad1fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a8159ff0 0x7f65a80c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65ad1fe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a815b770 con 0x7f65a806dd00 2024-01-11T07:07:32.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65ad1fe700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a815abf0 con 0x7f65a806e0d0 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65ad1fe700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a815aed0 con 0x7f65a8159ff0 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a806dd00 0x7f65a80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a806dd00 0x7f65a80c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34336/0 (socket says 172.21.15.7:34336) 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 -- 172.21.15.7:0/3670849855 learned_addr learned my addr 172.21.15.7:0/3670849855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a659c700 1 --2- 172.21.15.7:0/3670849855 >> v2:172.21.15.113:3300/0 conn(0x7f65a806e0d0 0x7f65a8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a759e700 1 --2- 172.21.15.7:0/3670849855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a8159ff0 0x7f65a80c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 -- 172.21.15.7:0/3670849855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a8159ff0 msgr2=0x7f65a80c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 --2- 172.21.15.7:0/3670849855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a8159ff0 0x7f65a80c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 -- 172.21.15.7:0/3670849855 >> v2:172.21.15.113:3300/0 conn(0x7f65a806e0d0 msgr2=0x7f65a8159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 --2- 172.21.15.7:0/3670849855 >> v2:172.21.15.113:3300/0 conn(0x7f65a806e0d0 0x7f65a8159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 -- 172.21.15.7:0/3670849855 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a815b1b0 con 0x7f65a806dd00 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a659c700 1 --2- 172.21.15.7:0/3670849855 >> v2:172.21.15.113:3300/0 conn(0x7f65a806e0d0 0x7f65a8159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a759e700 1 --2- 172.21.15.7:0/3670849855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a8159ff0 0x7f65a80c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.356+0000 7f65a6d9d700 1 --2- 172.21.15.7:0/3670849855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a806dd00 0x7f65a80c98d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f65a806d2e0 tx=0x7f6598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9e0061cbe21589 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65a5d9b700 1 -- 172.21.15.7:0/3670849855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800b860 con 0x7f65a806dd00 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65a5d9b700 1 -- 172.21.15.7:0/3670849855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659800b9c0 con 0x7f65a806dd00 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65a5d9b700 1 -- 172.21.15.7:0/3670849855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800bd60 con 0x7f65a806dd00 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 -- 172.21.15.7:0/3670849855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a806dd00 msgr2=0x7f65a80c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/3670849855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a806dd00 0x7f65a80c98d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f65a806d2e0 tx=0x7f6598009250 comp rx=0 tx=0).stop 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 -- 172.21.15.7:0/3670849855 shutdown_connections 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/3670849855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a806dd00 0x7f65a80c98d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/3670849855 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a8159ff0 0x7f65a80c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/3670849855 >> v2:172.21.15.113:3300/0 conn(0x7f65a806e0d0 0x7f65a8159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 -- 172.21.15.7:0/3670849855 >> 172.21.15.7:0/3670849855 conn(0x7f65a80680e0 msgr2=0x7f65a80cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 -- 172.21.15.7:0/3670849855 shutdown_connections 2024-01-11T07:07:32.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.357+0000 7f65ad1fe700 1 -- 172.21.15.7:0/3670849855 wait complete. 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.359+0000 7f65ad1fe700 1 Processor -- start 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.359+0000 7f65ad1fe700 1 -- start start 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65ad1fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a806dd00 0x7f65a8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65ad1fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a806e0d0 0x7f65a8156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65ad1fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a8159ff0 0x7f65a8156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65ad1fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8155450 con 0x7f65a8159ff0 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65ad1fe700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80c5860 con 0x7f65a806dd00 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65ad1fe700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80c4c50 con 0x7f65a806e0d0 2024-01-11T07:07:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a8159ff0 0x7f65a8156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a8159ff0 0x7f65a8156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34338/0 (socket says 172.21.15.7:34338) 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 -- 172.21.15.7:0/2850099185 learned_addr learned my addr 172.21.15.7:0/2850099185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a659c700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a806e0d0 0x7f65a8156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a6d9d700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a806dd00 0x7f65a8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 -- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a806e0d0 msgr2=0x7f65a8156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a806e0d0 0x7f65a8156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 -- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a806dd00 msgr2=0x7f65a8154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a806dd00 0x7f65a8154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 -- 172.21.15.7:0/2850099185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980094f0 con 0x7f65a8159ff0 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a659c700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a806e0d0 0x7f65a8156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f65a759e700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a8159ff0 0x7f65a8156d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65a0007f30 tx=0x7f65a0009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.360+0000 7f658effd700 1 -- 172.21.15.7:0/2850099185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000dce0 con 0x7f65a8159ff0 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.361+0000 7f658effd700 1 -- 172.21.15.7:0/2850099185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a000f040 con 0x7f65a8159ff0 2024-01-11T07:07:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.361+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a80c4f30 con 0x7f65a8159ff0 2024-01-11T07:07:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.361+0000 7f658effd700 1 -- 172.21.15.7:0/2850099185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0015690 con 0x7f65a8159ff0 2024-01-11T07:07:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.361+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a80c54f0 con 0x7f65a8159ff0 2024-01-11T07:07:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.361+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a804c3c0 con 0x7f65a8159ff0 2024-01-11T07:07:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.362+0000 7f658effd700 1 -- 172.21.15.7:0/2850099185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f65a001e3b0 con 0x7f65a8159ff0 2024-01-11T07:07:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.362+0000 7f658effd700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6594066f30 0x7f65940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.363+0000 7f658effd700 1 -- 172.21.15.7:0/2850099185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f65a0084a70 con 0x7f65a8159ff0 2024-01-11T07:07:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.363+0000 7f65a6d9d700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6594066f30 0x7f65940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.363+0000 7f65a6d9d700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6594066f30 0x7f65940693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65a806d2e0 tx=0x7f6598017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.367+0000 7f658effd700 1 -- 172.21.15.7:0/2850099185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65a0051ce0 con 0x7f65a8159ff0 2024-01-11T07:07:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:32.656+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f65a815b490 con 0x7f65a8159ff0 2024-01-11T07:07:33.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.310+0000 7f658effd700 1 -- 172.21.15.7:0/2850099185 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f65a001a0c0 con 0x7f65a8159ff0 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6594066f30 msgr2=0x7f65940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6594066f30 0x7f65940693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f65a806d2e0 tx=0x7f6598017000 comp rx=0 tx=0).stop 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a8159ff0 msgr2=0x7f65a8156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a8159ff0 0x7f65a8156d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65a0007f30 tx=0x7f65a0009930 comp rx=0 tx=0).stop 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 shutdown_connections 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6594066f30 0x7f65940693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a8159ff0 0x7f65a8156d60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65a806dd00 0x7f65a8154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 --2- 172.21.15.7:0/2850099185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65a806e0d0 0x7f65a8156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 >> 172.21.15.7:0/2850099185 conn(0x7f65a80680e0 msgr2=0x7f65a806abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:33.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 shutdown_connections 2024-01-11T07:07:33.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.313+0000 7f65ad1fe700 1 -- 172.21.15.7:0/2850099185 wait complete. 2024-01-11T07:07:33.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-11T07:07:33.479 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-11T07:07:33.479 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-11T07:07:33.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:07:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.615+0000 7f8631d44700 1 Processor -- start 2024-01-11T07:07:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.615+0000 7f8631d44700 1 -- start start 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.615+0000 7f8631d44700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0b2ed0 0x7f862c15e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f8631d44700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c06adf0 con 0x7f862c0b32a0 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f8631d44700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c06b0d0 con 0x7f862c063940 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f8631d44700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0ce390 con 0x7f862c0b2ed0 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f862c0b32a0 0x7f862c063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46780/0 (socket says 172.21.15.7:46780) 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862affd700 1 -- 172.21.15.7:0/1173771552 learned_addr learned my addr 172.21.15.7:0/1173771552 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862b7fe700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0b2ed0 0x7f862c15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409790103 0 0) 0x7f862c06adf0 con 0x7f862c0b32a0 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8614003570 con 0x7f862c0b32a0 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393547980 0 0) 0x7f862c06b0d0 con 0x7f862c063940 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f862c06adf0 con 0x7f862c063940 2024-01-11T07:07:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 426899999 0 0) 0x7f8614003570 con 0x7f862c0b32a0 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0b2ed0 msgr2=0x7f862c15e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0b2ed0 0x7f862c15e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 >> v1:172.21.15.113:6789/0 conn(0x7f862c063940 legacy=0x7f862c06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0ce670 con 0x7f862c0b32a0 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f8620001470 con 0x7f862c0b32a0 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 >> v1:172.21.15.7:6789/0 conn(0x7f862c0b32a0 legacy=0x7f862c063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.616+0000 7f862a7fc700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0b2ed0 0x7f8614003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862a7fc700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86140041d0 0x7f8614004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862a7fc700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614004d50 0x7f86140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0ce390 con 0x7f8614004d50 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861400bbf0 con 0x7f86140041d0 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861400bdb0 con 0x7f862c0b2ed0 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862bfff700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614004d50 0x7f86140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862affd700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86140041d0 0x7f8614004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862b7fe700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0b2ed0 0x7f8614003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862bfff700 1 -- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0b2ed0 msgr2=0x7f8614003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862bfff700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0b2ed0 0x7f8614003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862bfff700 1 -- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86140041d0 msgr2=0x7f8614004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862bfff700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86140041d0 0x7f8614004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862bfff700 1 -- 172.21.15.7:0/1173771552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8614003570 con 0x7f8614004d50 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862bfff700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614004d50 0x7f86140051c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f862c063d40 tx=0x7f86240017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40edb52e686e327e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624001ab0 con 0x7f8614004d50 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862400c880 con 0x7f8614004d50 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.617+0000 7f862a7fc700 1 -- 172.21.15.7:0/1173771552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400cc20 con 0x7f8614004d50 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.618+0000 7f8631d44700 1 -- 172.21.15.7:0/1173771552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614004d50 msgr2=0x7f86140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.618+0000 7f8631d44700 1 --2- 172.21.15.7:0/1173771552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614004d50 0x7f86140051c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f862c063d40 tx=0x7f86240017a0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.618+0000 7f862bfff700 1 -- 172.21.15.7:0/1173771552 reap_dead start 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.618+0000 7f8631d44700 1 -- 172.21.15.7:0/1173771552 shutdown_connections 2024-01-11T07:07:33.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.618+0000 7f8631d44700 1 -- 172.21.15.7:0/1173771552 >> 172.21.15.7:0/1173771552 conn(0x7f862c0680e0 msgr2=0x7f862c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.618+0000 7f8631d44700 1 -- 172.21.15.7:0/1173771552 shutdown_connections 2024-01-11T07:07:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.618+0000 7f8631d44700 1 -- 172.21.15.7:0/1173771552 wait complete. 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.619+0000 7f8631d44700 1 Processor -- start 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.619+0000 7f8631d44700 1 -- start start 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f8631d44700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8614004d50 0x7f862c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f8631d44700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0c5220 0x7f862c0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f8631d44700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f862c0b0f30 0x7f862c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f8631d44700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0c7cd0 con 0x7f862c0b0f30 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f8631d44700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0b18e0 con 0x7f8614004d50 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f8631d44700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0c73a0 con 0x7f862c0c5220 2024-01-11T07:07:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f862bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f862c0b0f30 0x7f862c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f862bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f862c0b0f30 0x7f862c0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34356/0 (socket says 172.21.15.7:34356) 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f862bfff700 1 -- 172.21.15.7:0/2247341327 learned_addr learned my addr 172.21.15.7:0/2247341327 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f862affd700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0c5220 0x7f862c0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.620+0000 7f862b7fe700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8614004d50 0x7f862c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f862bfff700 1 -- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0c5220 msgr2=0x7f862c0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f862bfff700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0c5220 0x7f862c0c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f862bfff700 1 -- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8614004d50 msgr2=0x7f862c0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f862bfff700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8614004d50 0x7f862c0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f862bfff700 1 -- 172.21.15.7:0/2247341327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0c7680 con 0x7f862c0b0f30 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f862bfff700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f862c0b0f30 0x7f862c0b13a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f862c06ef30 tx=0x7f8624016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f86137fe700 1 -- 172.21.15.7:0/2247341327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862400c6a0 con 0x7f862c0b0f30 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f86137fe700 1 -- 172.21.15.7:0/2247341327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862400c9f0 con 0x7f862c0b0f30 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c0c7960 con 0x7f862c0b0f30 2024-01-11T07:07:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f86137fe700 1 -- 172.21.15.7:0/2247341327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624006ba0 con 0x7f862c0b0f30 2024-01-11T07:07:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.621+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c157e10 con 0x7f862c0b0f30 2024-01-11T07:07:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.622+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862c064a40 con 0x7f862c0b0f30 2024-01-11T07:07:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.623+0000 7f86137fe700 1 -- 172.21.15.7:0/2247341327 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8624003030 con 0x7f862c0b0f30 2024-01-11T07:07:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.624+0000 7f86137fe700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618066f30 0x7f86180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.624+0000 7f86137fe700 1 -- 172.21.15.7:0/2247341327 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8624086bf0 con 0x7f862c0b0f30 2024-01-11T07:07:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.624+0000 7f862b7fe700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618066f30 0x7f86180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.624+0000 7f862b7fe700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618066f30 0x7f86180693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f861c006f00 tx=0x7f861c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:33.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.628+0000 7f86137fe700 1 -- 172.21.15.7:0/2247341327 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8624053e80 con 0x7f862c0b0f30 2024-01-11T07:07:33.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.831+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f862c04c3c0 con 0x7f862c0b0f30 2024-01-11T07:07:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.835+0000 7f86137fe700 1 -- 172.21.15.7:0/2247341327 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8624007dd0 con 0x7f862c0b0f30 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618066f30 msgr2=0x7f86180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618066f30 0x7f86180693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f861c006f00 tx=0x7f861c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f862c0b0f30 msgr2=0x7f862c0b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f862c0b0f30 0x7f862c0b13a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f862c06ef30 tx=0x7f8624016000 comp rx=0 tx=0).stop 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 shutdown_connections 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618066f30 0x7f86180693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f862c0b0f30 0x7f862c0b13a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8614004d50 0x7f862c0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 --2- 172.21.15.7:0/2247341327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f862c0c5220 0x7f862c0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 >> 172.21.15.7:0/2247341327 conn(0x7f862c0680e0 msgr2=0x7f862c06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 shutdown_connections 2024-01-11T07:07:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:33.837+0000 7f8631d44700 1 -- 172.21.15.7:0/2247341327 wait complete. 2024-01-11T07:07:33.850 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:07:33.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:07:33.900 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:07:33.952 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:07:33.952 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:07:33.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:07:33.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-01-11T07:07:33.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:07:33.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:33.996 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:07:34.061 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:07:34.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:07:34.061 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-01-11T07:07:34.110 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:07:34.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:34.111 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:07:34.161 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-01-11T07:07:34.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:07:34.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-01-11T07:07:34.208 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:07:34.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:34.209 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-11T07:07:34.230 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-01-11T07:07:34.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:07:34.230 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-01-11T07:07:34.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:07:34.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:34.271 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-11T07:07:34.330 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:07:34.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.471+0000 7f2bac9fa700 1 Processor -- start 2024-01-11T07:07:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.471+0000 7f2bac9fa700 1 -- start start 2024-01-11T07:07:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2bac9fa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ba815a060 0x7f2ba80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2bac9fa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8069320 0x7f2ba80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2bac9fa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba80ce9b0 con 0x7f2ba8159c90 2024-01-11T07:07:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2bac9fa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba80b1f30 con 0x7f2ba815a060 2024-01-11T07:07:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2bac9fa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba80cf530 con 0x7f2ba8069320 2024-01-11T07:07:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2b9dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ba815a060 0x7f2ba80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2ba659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2ba8159c90 0x7f2ba80b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46790/0 (socket says 172.21.15.7:46790) 2024-01-11T07:07:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2b9dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ba815a060 0x7f2ba80b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34366/0 (socket says 172.21.15.7:34366) 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2ba659c700 1 -- 172.21.15.7:0/2842314101 learned_addr learned my addr 172.21.15.7:0/2842314101 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2ba6d9d700 1 --2- 172.21.15.7:0/2842314101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8069320 0x7f2ba80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.472+0000 7f2b9dd9b700 1 -- 172.21.15.7:0/2842314101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8069320 msgr2=0x7f2ba80b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2b9dd9b700 1 --2- 172.21.15.7:0/2842314101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8069320 0x7f2ba80b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2b9dd9b700 1 -- 172.21.15.7:0/2842314101 >> v1:172.21.15.7:6789/0 conn(0x7f2ba8159c90 legacy=0x7f2ba80b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2ba5d9b700 1 -- 172.21.15.7:0/2842314101 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337902998 0 0) 0x7f2ba80ce9b0 con 0x7f2ba8159c90 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2b9dd9b700 1 -- 172.21.15.7:0/2842314101 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba80cf9c0 con 0x7f2ba815a060 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2b9dd9b700 1 --2- 172.21.15.7:0/2842314101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba815a060 0x7f2ba80b0430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098e0 tx=0x7f2b900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12df8e4c5a709ca0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2ba5d9b700 1 -- 172.21.15.7:0/2842314101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000b640 con 0x7f2ba815a060 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2ba5d9b700 1 -- 172.21.15.7:0/2842314101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9000b7a0 con 0x7f2ba815a060 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2ba5d9b700 1 -- 172.21.15.7:0/2842314101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000bb40 con 0x7f2ba815a060 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2842314101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba815a060 msgr2=0x7f2ba80b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2842314101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba815a060 0x7f2ba80b0430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098e0 tx=0x7f2b900091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2842314101 shutdown_connections 2024-01-11T07:07:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2842314101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8069320 0x7f2ba80b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2842314101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba815a060 0x7f2ba80b0430 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.473+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2842314101 >> 172.21.15.7:0/2842314101 conn(0x7f2ba8071100 msgr2=0x7f2ba806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.474+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2842314101 shutdown_connections 2024-01-11T07:07:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.474+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2842314101 wait complete. 2024-01-11T07:07:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.475+0000 7f2bac9fa700 1 Processor -- start 2024-01-11T07:07:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.475+0000 7f2bac9fa700 1 -- start start 2024-01-11T07:07:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2bac9fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba8069320 0x7f2ba8155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2bac9fa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8159c90 0x7f2ba8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2bac9fa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba815a060 0x7f2ba81562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2bac9fa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba816a0f0 con 0x7f2ba8069320 2024-01-11T07:07:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2bac9fa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba8155b40 con 0x7f2ba815a060 2024-01-11T07:07:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2bac9fa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba8169da0 con 0x7f2ba8159c90 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba8069320 0x7f2ba8155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba8069320 0x7f2ba8155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34374/0 (socket says 172.21.15.7:34374) 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 -- 172.21.15.7:0/2095742316 learned_addr learned my addr 172.21.15.7:0/2095742316 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2b9dd9b700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8159c90 0x7f2ba8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba6d9d700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba815a060 0x7f2ba81562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 -- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8159c90 msgr2=0x7f2ba8155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8159c90 0x7f2ba8155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 -- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba815a060 msgr2=0x7f2ba81562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba815a060 0x7f2ba81562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 -- 172.21.15.7:0/2095742316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90009420 con 0x7f2ba8069320 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2b9dd9b700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8159c90 0x7f2ba8155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba6d9d700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba815a060 0x7f2ba81562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2ba659c700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba8069320 0x7f2ba8155600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c7e0 tx=0x7f2b98007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.476+0000 7f2b9effd700 1 -- 172.21.15.7:0/2095742316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98005b30 con 0x7f2ba8069320 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.477+0000 7f2b9effd700 1 -- 172.21.15.7:0/2095742316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b98005c90 con 0x7f2ba8069320 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.477+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba815fd80 con 0x7f2ba8069320 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.477+0000 7f2b9effd700 1 -- 172.21.15.7:0/2095742316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98007300 con 0x7f2ba8069320 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.477+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba8160340 con 0x7f2ba8069320 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.478+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba804c3c0 con 0x7f2ba8069320 2024-01-11T07:07:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.478+0000 7f2b9effd700 1 -- 172.21.15.7:0/2095742316 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2b9801b400 con 0x7f2ba8069320 2024-01-11T07:07:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.478+0000 7f2b9effd700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b88066f30 0x7f2b880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.479+0000 7f2b9effd700 1 -- 172.21.15.7:0/2095742316 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b98084040 con 0x7f2ba8069320 2024-01-11T07:07:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.479+0000 7f2b9dd9b700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b88066f30 0x7f2b880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.479+0000 7f2b9dd9b700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b88066f30 0x7f2b880693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80b32e0 tx=0x7f2b90017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.483+0000 7f2b9effd700 1 -- 172.21.15.7:0/2095742316 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b98051330 con 0x7f2ba8069320 2024-01-11T07:07:34.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.734+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ba80b2b00 con 0x7f2ba8069320 2024-01-11T07:07:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.734+0000 7f2b9effd700 1 -- 172.21.15.7:0/2095742316 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f2b98054950 con 0x7f2ba8069320 2024-01-11T07:07:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:34.734 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:06:58.385324+0000","modified":"2024-01-11T07:07:02.162269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b88066f30 msgr2=0x7f2b880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b88066f30 0x7f2b880693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80b32e0 tx=0x7f2b90017000 comp rx=0 tx=0).stop 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba8069320 msgr2=0x7f2ba8155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba8069320 0x7f2ba8155600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c7e0 tx=0x7f2b98007df0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 shutdown_connections 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b88066f30 0x7f2b880693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ba815a060 0x7f2ba81562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ba8069320 0x7f2ba8155600 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 --2- 172.21.15.7:0/2095742316 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ba8159c90 0x7f2ba8155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 >> 172.21.15.7:0/2095742316 conn(0x7f2ba8071100 msgr2=0x7f2ba80688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 shutdown_connections 2024-01-11T07:07:34.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.737+0000 7f2bac9fa700 1 -- 172.21.15.7:0/2095742316 wait complete. 2024-01-11T07:07:34.738 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T07:07:34.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.883+0000 7f1c24ae3700 1 Processor -- start 2024-01-11T07:07:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.883+0000 7f1c24ae3700 1 -- start start 2024-01-11T07:07:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c24ae3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c20158280 0x7f1c200b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c24ae3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c24ae3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c20159300 con 0x7f1c200cafb0 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c24ae3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c200cf000 con 0x7f1c20158280 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c24ae3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200cf260 con 0x7f1c2015aa80 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1dd9b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c20158280 0x7f1c200b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1ed9d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1ed9d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:46954/0 (socket says 172.21.15.7:46954) 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1dd9b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c20158280 0x7f1c200b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:33896/0 (socket says 172.21.15.7:33896) 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1ed9d700 1 -- 172.21.15.7:0/3359177387 learned_addr learned my addr 172.21.15.7:0/3359177387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1ed9d700 1 -- 172.21.15.7:0/3359177387 >> v2:172.21.15.113:3300/0 conn(0x7f1c20158280 msgr2=0x7f1c200b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1ed9d700 1 --2- 172.21.15.7:0/3359177387 >> v2:172.21.15.113:3300/0 conn(0x7f1c20158280 0x7f1c200b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1ed9d700 1 -- 172.21.15.7:0/3359177387 >> v1:172.21.15.113:6789/0 conn(0x7f1c200cafb0 legacy=0x7f1c200ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1ed9d700 1 -- 172.21.15.7:0/3359177387 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200cf540 con 0x7f1c2015aa80 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.884+0000 7f1c1dd9b700 1 --2- 172.21.15.7:0/3359177387 >> v2:172.21.15.113:3300/0 conn(0x7f1c20158280 0x7f1c200b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.885+0000 7f1c1ed9d700 1 --2- 172.21.15.7:0/3359177387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200b0970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009f20 tx=0x7f1c18009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c3d12d6ca5c709 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.885+0000 7f1c1d59a700 1 -- 172.21.15.7:0/3359177387 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800e070 con 0x7f1c2015aa80 2024-01-11T07:07:34.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.885+0000 7f1c1d59a700 1 -- 172.21.15.7:0/3359177387 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1800b9d0 con 0x7f1c2015aa80 2024-01-11T07:07:34.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.885+0000 7f1c1d59a700 1 -- 172.21.15.7:0/3359177387 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c180092e0 con 0x7f1c2015aa80 2024-01-11T07:07:34.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.885+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3359177387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 msgr2=0x7f1c200b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.885+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3359177387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200b0970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009f20 tx=0x7f1c18009840 comp rx=0 tx=0).stop 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.886+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3359177387 shutdown_connections 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.886+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3359177387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200b0970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.886+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3359177387 >> v2:172.21.15.113:3300/0 conn(0x7f1c20158280 0x7f1c200b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.886+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3359177387 >> 172.21.15.7:0/3359177387 conn(0x7f1c20071100 msgr2=0x7f1c2006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.886+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3359177387 shutdown_connections 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.886+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3359177387 wait complete. 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.887+0000 7f1c24ae3700 1 Processor -- start 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.887+0000 7f1c24ae3700 1 -- start start 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c24ae3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c200cafb0 0x7f1c2014f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c24ae3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c20158280 0x7f1c201699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c24ae3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c24ae3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c9260 con 0x7f1c200cafb0 2024-01-11T07:07:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c24ae3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200aeb40 con 0x7f1c20158280 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c24ae3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200aee20 con 0x7f1c2015aa80 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c200cafb0 0x7f1c2014f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c200cafb0 0x7f1c2014f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34380/0 (socket says 172.21.15.7:34380) 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 -- 172.21.15.7:0/3751956635 learned_addr learned my addr 172.21.15.7:0/3751956635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1ed9d700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1dd9b700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c20158280 0x7f1c201699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 -- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 msgr2=0x7f1c200ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 -- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c20158280 msgr2=0x7f1c201699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c20158280 0x7f1c201699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 -- 172.21.15.7:0/3751956635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c14009420 con 0x7f1c200cafb0 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.888+0000 7f1c1e59c700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c200cafb0 0x7f1c2014f930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000c590 tx=0x7f1c1000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.889+0000 7f1c0e7fc700 1 -- 172.21.15.7:0/3751956635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10011590 con 0x7f1c200cafb0 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.889+0000 7f1c0e7fc700 1 -- 172.21.15.7:0/3751956635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c10011710 con 0x7f1c200cafb0 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.889+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c18009ae0 con 0x7f1c200cafb0 2024-01-11T07:07:34.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.889+0000 7f1c0e7fc700 1 -- 172.21.15.7:0/3751956635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10011ad0 con 0x7f1c200cafb0 2024-01-11T07:07:34.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.889+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c200c94e0 con 0x7f1c200cafb0 2024-01-11T07:07:34.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.890+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c200af340 con 0x7f1c200cafb0 2024-01-11T07:07:34.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.890+0000 7f1c0e7fc700 1 -- 172.21.15.7:0/3751956635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1c1001b3b0 con 0x7f1c200cafb0 2024-01-11T07:07:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.890+0000 7f1c0e7fc700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c00066f30 0x7f1c000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.891+0000 7f1c0e7fc700 1 -- 172.21.15.7:0/3751956635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c100839e0 con 0x7f1c200cafb0 2024-01-11T07:07:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.891+0000 7f1c1dd9b700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c00066f30 0x7f1c000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.891+0000 7f1c1dd9b700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c00066f30 0x7f1c000693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1c20155910 tx=0x7f1c140091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:34.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:34.895+0000 7f1c0e7fc700 1 -- 172.21.15.7:0/3751956635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c200af340 con 0x7f1c200cafb0 2024-01-11T07:07:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.100+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c200af340 con 0x7f1c200cafb0 2024-01-11T07:07:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.101+0000 7f1c0e7fc700 1 -- 172.21.15.7:0/3751956635 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f1c200af340 con 0x7f1c200cafb0 2024-01-11T07:07:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:35.101 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c00066f30 msgr2=0x7f1c000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c00066f30 0x7f1c000693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1c20155910 tx=0x7f1c140091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c200cafb0 msgr2=0x7f1c2014f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c200cafb0 0x7f1c2014f930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000c590 tx=0x7f1c1000c8a0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 shutdown_connections 2024-01-11T07:07:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c00066f30 0x7f1c000693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c2015aa80 0x7f1c200ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c200cafb0 0x7f1c2014f930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 --2- 172.21.15.7:0/3751956635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c20158280 0x7f1c201699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.103+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 >> 172.21.15.7:0/3751956635 conn(0x7f1c20071100 msgr2=0x7f1c200ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.104+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 shutdown_connections 2024-01-11T07:07:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.104+0000 7f1c24ae3700 1 -- 172.21.15.7:0/3751956635 wait complete. 2024-01-11T07:07:35.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:06:56.038804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:06:57.365958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:07:35.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:07:35.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.253+0000 7fe2bd799700 1 Processor -- start 2024-01-11T07:07:35.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.253+0000 7fe2bd799700 1 -- start start 2024-01-11T07:07:35.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2bd799700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 0x7fe2b80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2bd799700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe2b80cc0d0 0x7fe2b80cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2bd799700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b80cee70 con 0x7fe2b80cdf10 2024-01-11T07:07:35.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2bd799700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b80ce860 con 0x7fe2b80cc0d0 2024-01-11T07:07:35.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2bd799700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b0100 con 0x7fe2b8159ea0 2024-01-11T07:07:35.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b67fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe2b80cdf10 0x7fe2b80cb9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46804/0 (socket says 172.21.15.7:46804) 2024-01-11T07:07:35.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b67fc700 1 -- 172.21.15.7:0/909381110 learned_addr learned my addr 172.21.15.7:0/909381110 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:35.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b77fe700 1 --2- 172.21.15.7:0/909381110 >> v2:172.21.15.113:3300/0 conn(0x7fe2b80cc0d0 0x7fe2b80cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b6ffd700 1 --2- 172.21.15.7:0/909381110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 0x7fe2b80af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b5ffb700 1 -- 172.21.15.7:0/909381110 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243846025 0 0) 0x7fe2b80cee70 con 0x7fe2b80cdf10 2024-01-11T07:07:35.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b5ffb700 1 -- 172.21.15.7:0/909381110 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a0003570 con 0x7fe2b80cdf10 2024-01-11T07:07:35.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b77fe700 1 -- 172.21.15.7:0/909381110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 msgr2=0x7fe2b80af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:35.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b77fe700 1 --2- 172.21.15.7:0/909381110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 0x7fe2b80af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b77fe700 1 -- 172.21.15.7:0/909381110 >> v1:172.21.15.7:6789/0 conn(0x7fe2b80cdf10 legacy=0x7fe2b80cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.254+0000 7fe2b77fe700 1 -- 172.21.15.7:0/909381110 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80ceb00 con 0x7fe2b80cc0d0 2024-01-11T07:07:35.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2b77fe700 1 --2- 172.21.15.7:0/909381110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 0x7fe2b80cf300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009e60 tx=0x7fe2b0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c4d1d3f6467a17 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:35.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2b5ffb700 1 -- 172.21.15.7:0/909381110 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000e070 con 0x7fe2b80cc0d0 2024-01-11T07:07:35.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2b5ffb700 1 -- 172.21.15.7:0/909381110 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000bab0 con 0x7fe2b80cc0d0 2024-01-11T07:07:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2b5ffb700 1 -- 172.21.15.7:0/909381110 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000be70 con 0x7fe2b80cc0d0 2024-01-11T07:07:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2bd799700 1 -- 172.21.15.7:0/909381110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 msgr2=0x7fe2b80cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2bd799700 1 --2- 172.21.15.7:0/909381110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 0x7fe2b80cf300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009e60 tx=0x7fe2b0009780 comp rx=0 tx=0).stop 2024-01-11T07:07:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2bd799700 1 -- 172.21.15.7:0/909381110 shutdown_connections 2024-01-11T07:07:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2bd799700 1 --2- 172.21.15.7:0/909381110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 0x7fe2b80cf300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2bd799700 1 --2- 172.21.15.7:0/909381110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 0x7fe2b80af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2bd799700 1 -- 172.21.15.7:0/909381110 >> 172.21.15.7:0/909381110 conn(0x7fe2b8071120 msgr2=0x7fe2b806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2bd799700 1 -- 172.21.15.7:0/909381110 shutdown_connections 2024-01-11T07:07:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.255+0000 7fe2bd799700 1 -- 172.21.15.7:0/909381110 wait complete. 2024-01-11T07:07:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.257+0000 7fe2bd799700 1 Processor -- start 2024-01-11T07:07:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.257+0000 7fe2bd799700 1 -- start start 2024-01-11T07:07:35.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2bd799700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 0x7fe2b80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2bd799700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b80cdf10 0x7fe2b8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2bd799700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 0x7fe2b80c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2bd799700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b4c00 con 0x7fe2b80cdf10 2024-01-11T07:07:35.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2bd799700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b3f70 con 0x7fe2b80cc0d0 2024-01-11T07:07:35.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2bd799700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b4250 con 0x7fe2b8159ea0 2024-01-11T07:07:35.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b80cdf10 0x7fe2b8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b80cdf10 0x7fe2b8156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34388/0 (socket says 172.21.15.7:34388) 2024-01-11T07:07:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 -- 172.21.15.7:0/2380305899 learned_addr learned my addr 172.21.15.7:0/2380305899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b77fe700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 0x7fe2b80c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b6ffd700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 0x7fe2b80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 -- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 msgr2=0x7fe2b80c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 0x7fe2b80c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 -- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 msgr2=0x7fe2b80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 0x7fe2b80b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 -- 172.21.15.7:0/2380305899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0009a20 con 0x7fe2b80cdf10 2024-01-11T07:07:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2b67fc700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b80cdf10 0x7fe2b8156290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00c7c0 tx=0x7fe2ac007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe29effd700 1 -- 172.21.15.7:0/2380305899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac005b30 con 0x7fe2b80cdf10 2024-01-11T07:07:35.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe29effd700 1 -- 172.21.15.7:0/2380305899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac005c90 con 0x7fe2b80cdf10 2024-01-11T07:07:35.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe29effd700 1 -- 172.21.15.7:0/2380305899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac007300 con 0x7fe2b80cdf10 2024-01-11T07:07:35.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.258+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b80b4530 con 0x7fe2b80cdf10 2024-01-11T07:07:35.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.259+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81567d0 con 0x7fe2b80cdf10 2024-01-11T07:07:35.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.259+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b80695e0 con 0x7fe2b80cdf10 2024-01-11T07:07:35.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.260+0000 7fe29effd700 1 -- 172.21.15.7:0/2380305899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe2ac01b3b0 con 0x7fe2b80cdf10 2024-01-11T07:07:35.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.260+0000 7fe29effd700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2a4066f30 0x7fe2a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.261+0000 7fe29effd700 1 -- 172.21.15.7:0/2380305899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2ac083a40 con 0x7fe2b80cdf10 2024-01-11T07:07:35.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.261+0000 7fe2b6ffd700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2a4066f30 0x7fe2a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.261+0000 7fe2b6ffd700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2a4066f30 0x7fe2a40693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009d10 tx=0x7fe2a80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:35.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.265+0000 7fe29effd700 1 -- 172.21.15.7:0/2380305899 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2ac085e60 con 0x7fe2b80cdf10 2024-01-11T07:07:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.471+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2b804c3c0 con 0x7fe2b80cdf10 2024-01-11T07:07:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.472+0000 7fe29effd700 1 -- 172.21.15.7:0/2380305899 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2ac017020 con 0x7fe2b80cdf10 2024-01-11T07:07:35.472 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:07:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2a4066f30 msgr2=0x7fe2a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2a4066f30 0x7fe2a40693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009d10 tx=0x7fe2a80091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b80cdf10 msgr2=0x7fe2b8156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b80cdf10 0x7fe2b8156290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00c7c0 tx=0x7fe2ac007e00 comp rx=0 tx=0).stop 2024-01-11T07:07:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 shutdown_connections 2024-01-11T07:07:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2a4066f30 0x7fe2a40693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2b8159ea0 0x7fe2b80c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2b80cc0d0 0x7fe2b80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 --2- 172.21.15.7:0/2380305899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b80cdf10 0x7fe2b8156290 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 >> 172.21.15.7:0/2380305899 conn(0x7fe2b8071120 msgr2=0x7fe2b80cb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 shutdown_connections 2024-01-11T07:07:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.474+0000 7fe2bd799700 1 -- 172.21.15.7:0/2380305899 wait complete. 2024-01-11T07:07:35.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:07:35.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.623+0000 7eff8cad6700 1 Processor -- start 2024-01-11T07:07:35.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.623+0000 7eff8cad6700 1 -- start start 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff8cad6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff88159fb0 0x7eff880c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff8cad6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff880c7760 0x7eff88063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff8cad6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff880c98f0 con 0x7eff88159be0 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff8cad6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff880c8140 con 0x7eff880c7760 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff8cad6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff880c8420 con 0x7eff88159fb0 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff86d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff880c7760 0x7eff88063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff86d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff880c7760 0x7eff88063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34394/0 (socket says 172.21.15.7:34394) 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff86d9d700 1 -- 172.21.15.7:0/3150700542 learned_addr learned my addr 172.21.15.7:0/3150700542 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.624+0000 7eff8659c700 1 --1- 172.21.15.7:0/3150700542 >> v1:172.21.15.7:6789/0 conn(0x7eff88159be0 0x7eff880c91e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46810/0 (socket says 172.21.15.7:46810) 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff85d9b700 1 --2- 172.21.15.7:0/3150700542 >> v2:172.21.15.113:3300/0 conn(0x7eff88159fb0 0x7eff880c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff86d9d700 1 -- 172.21.15.7:0/3150700542 >> v2:172.21.15.113:3300/0 conn(0x7eff88159fb0 msgr2=0x7eff880c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:35.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff86d9d700 1 --2- 172.21.15.7:0/3150700542 >> v2:172.21.15.113:3300/0 conn(0x7eff88159fb0 0x7eff880c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff86d9d700 1 -- 172.21.15.7:0/3150700542 >> v1:172.21.15.7:6789/0 conn(0x7eff88159be0 legacy=0x7eff880c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff86d9d700 1 -- 172.21.15.7:0/3150700542 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff880c8700 con 0x7eff880c7760 2024-01-11T07:07:35.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff8559a700 1 -- 172.21.15.7:0/3150700542 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161970238 0 0) 0x7eff880c98f0 con 0x7eff88159be0 2024-01-11T07:07:35.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff86d9d700 1 --2- 172.21.15.7:0/3150700542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880c7760 0x7eff88063230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7eff8806f030 tx=0x7eff80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4305d951dde6d167 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:35.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff8559a700 1 -- 172.21.15.7:0/3150700542 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000e070 con 0x7eff880c7760 2024-01-11T07:07:35.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff8559a700 1 -- 172.21.15.7:0/3150700542 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8000ba20 con 0x7eff880c7760 2024-01-11T07:07:35.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.625+0000 7eff8559a700 1 -- 172.21.15.7:0/3150700542 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800092e0 con 0x7eff880c7760 2024-01-11T07:07:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.626+0000 7eff8cad6700 1 -- 172.21.15.7:0/3150700542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880c7760 msgr2=0x7eff88063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.626+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3150700542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880c7760 0x7eff88063230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7eff8806f030 tx=0x7eff80009840 comp rx=0 tx=0).stop 2024-01-11T07:07:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.626+0000 7eff8cad6700 1 -- 172.21.15.7:0/3150700542 shutdown_connections 2024-01-11T07:07:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.626+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3150700542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880c7760 0x7eff88063230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.626+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3150700542 >> v2:172.21.15.113:3300/0 conn(0x7eff88159fb0 0x7eff880c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.626+0000 7eff8cad6700 1 -- 172.21.15.7:0/3150700542 >> 172.21.15.7:0/3150700542 conn(0x7eff880cd8f0 msgr2=0x7eff880cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.626+0000 7eff8cad6700 1 -- 172.21.15.7:0/3150700542 shutdown_connections 2024-01-11T07:07:35.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.626+0000 7eff8cad6700 1 -- 172.21.15.7:0/3150700542 wait complete. 2024-01-11T07:07:35.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.627+0000 7eff8cad6700 1 Processor -- start 2024-01-11T07:07:35.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.627+0000 7eff8cad6700 1 -- start start 2024-01-11T07:07:35.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff8cad6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff880c7760 0x7eff88151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff8cad6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff88159be0 0x7eff8814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff8cad6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff8815b360 0x7eff8815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff8cad6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8815bd20 con 0x7eff88159be0 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff8cad6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8816dec0 con 0x7eff8815b360 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff8cad6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8814c430 con 0x7eff880c7760 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff85d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff88159be0 0x7eff8814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff85d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff88159be0 0x7eff8814b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34402/0 (socket says 172.21.15.7:34402) 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff85d9b700 1 -- 172.21.15.7:0/3314201410 learned_addr learned my addr 172.21.15.7:0/3314201410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff8659c700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff880c7760 0x7eff88151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff86d9d700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff8815b360 0x7eff8815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff85d9b700 1 -- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff880c7760 msgr2=0x7eff88151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff85d9b700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff880c7760 0x7eff88151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff85d9b700 1 -- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff8815b360 msgr2=0x7eff8815b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff85d9b700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff8815b360 0x7eff8815b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff85d9b700 1 -- 172.21.15.7:0/3314201410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80009ae0 con 0x7eff88159be0 2024-01-11T07:07:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.628+0000 7eff86d9d700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff8815b360 0x7eff8815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.629+0000 7eff85d9b700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff88159be0 0x7eff8814b940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7eff7000e960 tx=0x7eff700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.629+0000 7eff7e7fc700 1 -- 172.21.15.7:0/3314201410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70013680 con 0x7eff88159be0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.629+0000 7eff7e7fc700 1 -- 172.21.15.7:0/3314201410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff700137e0 con 0x7eff88159be0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.629+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8814be80 con 0x7eff88159be0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.629+0000 7eff7e7fc700 1 -- 172.21.15.7:0/3314201410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70013b80 con 0x7eff88159be0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.629+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8816d870 con 0x7eff88159be0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.630+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8816e050 con 0x7eff88159be0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.630+0000 7eff7e7fc700 1 -- 172.21.15.7:0/3314201410 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7eff7000c490 con 0x7eff88159be0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.631+0000 7eff7e7fc700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff68066fb0 0x7eff68069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.631+0000 7eff7e7fc700 1 -- 172.21.15.7:0/3314201410 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff70084740 con 0x7eff88159be0 2024-01-11T07:07:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.631+0000 7eff8659c700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff68066fb0 0x7eff68069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.632+0000 7eff8659c700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff68066fb0 0x7eff68069460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff780012a0 tx=0x7eff78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.635+0000 7eff7e7fc700 1 -- 172.21.15.7:0/3314201410 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff70051910 con 0x7eff88159be0 2024-01-11T07:07:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.839+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff8804c3c0 con 0x7eff88159be0 2024-01-11T07:07:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.840+0000 7eff7e7fc700 1 -- 172.21.15.7:0/3314201410 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff70054f30 con 0x7eff88159be0 2024-01-11T07:07:35.840 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:07:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff68066fb0 msgr2=0x7eff68069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff68066fb0 0x7eff68069460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff780012a0 tx=0x7eff78009040 comp rx=0 tx=0).stop 2024-01-11T07:07:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff88159be0 msgr2=0x7eff8814b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff88159be0 0x7eff8814b940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7eff7000e960 tx=0x7eff700091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 shutdown_connections 2024-01-11T07:07:35.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff68066fb0 0x7eff68069460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff8815b360 0x7eff8815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff880c7760 0x7eff88151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 --2- 172.21.15.7:0/3314201410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff88159be0 0x7eff8814b940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 >> 172.21.15.7:0/3314201410 conn(0x7eff880cd8f0 msgr2=0x7eff8814c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 shutdown_connections 2024-01-11T07:07:35.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.842+0000 7eff8cad6700 1 -- 172.21.15.7:0/3314201410 wait complete. 2024-01-11T07:07:35.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:07:35.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.990+0000 7f5af1baf700 1 Processor -- start 2024-01-11T07:07:35.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.990+0000 7f5af1baf700 1 -- start start 2024-01-11T07:07:35.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5af1baf700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cc240 0x7f5aec15d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5af1baf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5af1baf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec15bf50 con 0x7f5aec0cdda0 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5af1baf700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec15e540 con 0x7f5aec0cc240 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5af1baf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec15b900 con 0x7f5aec15ddc0 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5aeb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5aec0cdda0 0x7f5aec0cbb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46822/0 (socket says 172.21.15.7:46822) 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5aeb7fe700 1 -- 172.21.15.7:0/4058483100 learned_addr learned my addr 172.21.15.7:0/4058483100 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5aeaffd700 1 --2- 172.21.15.7:0/4058483100 >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cc240 0x7f5aec15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.991+0000 7f5aebfff700 1 --2- 172.21.15.7:0/4058483100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aea7fc700 1 -- 172.21.15.7:0/4058483100 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306800069 0 0) 0x7f5aec15bf50 con 0x7f5aec0cdda0 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aea7fc700 1 -- 172.21.15.7:0/4058483100 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad4003570 con 0x7f5aec0cdda0 2024-01-11T07:07:35.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aebfff700 1 -- 172.21.15.7:0/4058483100 >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cc240 msgr2=0x7f5aec15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:35.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aebfff700 1 --2- 172.21.15.7:0/4058483100 >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cc240 0x7f5aec15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aebfff700 1 -- 172.21.15.7:0/4058483100 >> v1:172.21.15.7:6789/0 conn(0x7f5aec0cdda0 legacy=0x7f5aec0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aebfff700 1 -- 172.21.15.7:0/4058483100 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec15bbe0 con 0x7f5aec15ddc0 2024-01-11T07:07:35.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aebfff700 1 --2- 172.21.15.7:0/4058483100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec15c3e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009e60 tx=0x7f5ae4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e50addb6c52191 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:35.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aea7fc700 1 -- 172.21.15.7:0/4058483100 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400e070 con 0x7f5aec15ddc0 2024-01-11T07:07:35.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aea7fc700 1 -- 172.21.15.7:0/4058483100 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae400bab0 con 0x7f5aec15ddc0 2024-01-11T07:07:35.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.992+0000 7f5aea7fc700 1 -- 172.21.15.7:0/4058483100 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400be70 con 0x7f5aec15ddc0 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.993+0000 7f5af1baf700 1 -- 172.21.15.7:0/4058483100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 msgr2=0x7f5aec15c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.993+0000 7f5af1baf700 1 --2- 172.21.15.7:0/4058483100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec15c3e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009e60 tx=0x7f5ae4009780 comp rx=0 tx=0).stop 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.993+0000 7f5af1baf700 1 -- 172.21.15.7:0/4058483100 shutdown_connections 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.993+0000 7f5af1baf700 1 --2- 172.21.15.7:0/4058483100 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec15c3e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.993+0000 7f5af1baf700 1 --2- 172.21.15.7:0/4058483100 >> v2:172.21.15.113:3300/0 conn(0x7f5aec0cc240 0x7f5aec15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.993+0000 7f5af1baf700 1 -- 172.21.15.7:0/4058483100 >> 172.21.15.7:0/4058483100 conn(0x7f5aec071160 msgr2=0x7f5aec06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.993+0000 7f5af1baf700 1 -- 172.21.15.7:0/4058483100 shutdown_connections 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.993+0000 7f5af1baf700 1 -- 172.21.15.7:0/4058483100 wait complete. 2024-01-11T07:07:35.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.994+0000 7f5af1baf700 1 Processor -- start 2024-01-11T07:07:35.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.994+0000 7f5af1baf700 1 -- start start 2024-01-11T07:07:35.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5af1baf700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aec0cc240 0x7f5aec0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5af1baf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aec0cdda0 0x7f5aec0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5af1baf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5af1baf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0b3ec0 con 0x7f5aec0cdda0 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5af1baf700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0b4970 con 0x7f5aec0cc240 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5af1baf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0c7e50 con 0x7f5aec15ddc0 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5aeaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aec0cdda0 0x7f5aec0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5aeaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aec0cdda0 0x7f5aec0d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34408/0 (socket says 172.21.15.7:34408) 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5aeaffd700 1 -- 172.21.15.7:0/2430481826 learned_addr learned my addr 172.21.15.7:0/2430481826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5aeb7fe700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aec0cc240 0x7f5aec0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.995+0000 7f5aebfff700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5aeaffd700 1 -- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 msgr2=0x7f5aec169e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5aeaffd700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec169e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5aeaffd700 1 -- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aec0cc240 msgr2=0x7f5aec0b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5aeaffd700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aec0cc240 0x7f5aec0b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5aeaffd700 1 -- 172.21.15.7:0/2430481826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009a20 con 0x7f5aec0cdda0 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5aeb7fe700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aec0cc240 0x7f5aec0b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:35.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5aeaffd700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aec0cdda0 0x7f5aec0d0b60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000ec80 tx=0x7f5ae00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5ad37fe700 1 -- 172.21.15.7:0/2430481826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000c9d0 con 0x7f5aec0cdda0 2024-01-11T07:07:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5ad37fe700 1 -- 172.21.15.7:0/2430481826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae000cb30 con 0x7f5aec0cdda0 2024-01-11T07:07:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec0b40d0 con 0x7f5aec0cdda0 2024-01-11T07:07:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5ad37fe700 1 -- 172.21.15.7:0/2430481826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0013470 con 0x7f5aec0cdda0 2024-01-11T07:07:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.996+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec0b4690 con 0x7f5aec0cdda0 2024-01-11T07:07:35.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.997+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec04c3c0 con 0x7f5aec0cdda0 2024-01-11T07:07:35.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.997+0000 7f5ad37fe700 1 -- 172.21.15.7:0/2430481826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5ae0013820 con 0x7f5aec0cdda0 2024-01-11T07:07:35.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.998+0000 7f5ad37fe700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ad8066f30 0x7f5ad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:35.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.998+0000 7f5ad37fe700 1 -- 172.21.15.7:0/2430481826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ae0084920 con 0x7f5aec0cdda0 2024-01-11T07:07:35.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.998+0000 7f5aeb7fe700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ad8066f30 0x7f5ad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:35.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:35.999+0000 7f5aeb7fe700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ad8066f30 0x7f5ad80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0058d0 tx=0x7f5adc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:36.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.003+0000 7f5ad37fe700 1 -- 172.21.15.7:0/2430481826 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ae0051b90 con 0x7f5aec0cdda0 2024-01-11T07:07:36.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.207+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5aec0c7fe0 con 0x7f5aec0cdda0 2024-01-11T07:07:36.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.208+0000 7f5ad37fe700 1 -- 172.21.15.7:0/2430481826 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ae00551b0 con 0x7f5aec0cdda0 2024-01-11T07:07:36.208 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:07:36.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ad8066f30 msgr2=0x7f5ad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ad8066f30 0x7f5ad80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0058d0 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2024-01-11T07:07:36.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aec0cdda0 msgr2=0x7f5aec0d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aec0cdda0 0x7f5aec0d0b60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000ec80 tx=0x7f5ae00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 shutdown_connections 2024-01-11T07:07:36.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ad8066f30 0x7f5ad80693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5aec15ddc0 0x7f5aec169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5aec0cc240 0x7f5aec0b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 --2- 172.21.15.7:0/2430481826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aec0cdda0 0x7f5aec0d0b60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 >> 172.21.15.7:0/2430481826 conn(0x7f5aec071160 msgr2=0x7f5aec1596b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:36.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 shutdown_connections 2024-01-11T07:07:36.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.210+0000 7f5af1baf700 1 -- 172.21.15.7:0/2430481826 wait complete. 2024-01-11T07:07:36.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.356+0000 7f6fc70ec700 1 Processor -- start 2024-01-11T07:07:36.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.356+0000 7f6fc70ec700 1 -- start start 2024-01-11T07:07:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fc70ec700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc00ac3a0 0x7f6fc00cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fc70ec700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6fc015d920 0x7f6fc015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fc70ec700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc015e2d0 con 0x7f6fc015a2c0 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fc70ec700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc006ac30 con 0x7f6fc015d920 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fc70ec700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc00ac770 con 0x7f6fc00ac3a0 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fbffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6fc015d920 0x7f6fc015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fbffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6fc015d920 0x7f6fc015dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34414/0 (socket says 172.21.15.7:34414) 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fbffff700 1 -- 172.21.15.7:0/2562783842 learned_addr learned my addr 172.21.15.7:0/2562783842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.357+0000 7f6fc4e88700 1 --2- 172.21.15.7:0/2562783842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc00ac3a0 0x7f6fc00cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fbffff700 1 -- 172.21.15.7:0/2562783842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc00ac3a0 msgr2=0x7f6fc00cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fbffff700 1 --2- 172.21.15.7:0/2562783842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc00ac3a0 0x7f6fc00cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fbffff700 1 -- 172.21.15.7:0/2562783842 >> v1:172.21.15.113:6789/0 conn(0x7f6fc015a2c0 legacy=0x7f6fc015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fbffff700 1 -- 172.21.15.7:0/2562783842 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc006a300 con 0x7f6fc015d920 2024-01-11T07:07:36.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fbffff700 1 --2- 172.21.15.7:0/2562783842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc015dd90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40099b0 tx=0x7f6fb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf2e650601536df server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fbf7fe700 1 -- 172.21.15.7:0/2562783842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400b820 con 0x7f6fc015d920 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fbf7fe700 1 -- 172.21.15.7:0/2562783842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb400b980 con 0x7f6fc015d920 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fbf7fe700 1 -- 172.21.15.7:0/2562783842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400bd20 con 0x7f6fc015d920 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2562783842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 msgr2=0x7f6fc015dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.358+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2562783842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc015dd90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40099b0 tx=0x7f6fb4009250 comp rx=0 tx=0).stop 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.359+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2562783842 shutdown_connections 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.359+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2562783842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc00ac3a0 0x7f6fc00cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.359+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2562783842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc015dd90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.359+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2562783842 >> 172.21.15.7:0/2562783842 conn(0x7f6fc00cd090 msgr2=0x7f6fc00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.359+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2562783842 shutdown_connections 2024-01-11T07:07:36.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.359+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2562783842 wait complete. 2024-01-11T07:07:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.360+0000 7f6fc70ec700 1 Processor -- start 2024-01-11T07:07:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.360+0000 7f6fc70ec700 1 -- start start 2024-01-11T07:07:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc70ec700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00ac3a0 0x7f6fc00c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc70ec700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc015a2c0 0x7f6fc0141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc70ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc00c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc70ec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc0141300 con 0x7f6fc015d920 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc70ec700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc0169fd0 con 0x7f6fc00ac3a0 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc70ec700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc0140730 con 0x7f6fc015a2c0 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc5689700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc00c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc4e88700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00ac3a0 0x7f6fc00c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc4e88700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00ac3a0 0x7f6fc00c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:33964/0 (socket says 172.21.15.7:33964) 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fbffff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc015a2c0 0x7f6fc0141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc5689700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc00c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34422/0 (socket says 172.21.15.7:34422) 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc4e88700 1 -- 172.21.15.7:0/2302447406 learned_addr learned my addr 172.21.15.7:0/2302447406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc5689700 1 -- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc015a2c0 msgr2=0x7f6fc0141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc5689700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc015a2c0 0x7f6fc0141790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc5689700 1 -- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00ac3a0 msgr2=0x7f6fc00c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc5689700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00ac3a0 0x7f6fc00c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc5689700 1 -- 172.21.15.7:0/2302447406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40094f0 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.361+0000 7f6fc5689700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc00c9360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800bad0 tx=0x7f6fb8009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.362+0000 7f6fbcff9700 1 -- 172.21.15.7:0/2302447406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800c760 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.362+0000 7f6fbcff9700 1 -- 172.21.15.7:0/2302447406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb800c8c0 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.362+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc01409d0 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.362+0000 7f6fbcff9700 1 -- 172.21.15.7:0/2302447406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800cc80 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.362+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc0140f90 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.363+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc0141470 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.363+0000 7f6fbcff9700 1 -- 172.21.15.7:0/2302447406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6fb8014620 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.364+0000 7f6fbcff9700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.364+0000 7f6fbcff9700 1 -- 172.21.15.7:0/2302447406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6fb8084320 con 0x7f6fc015d920 2024-01-11T07:07:36.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.364+0000 7f6fc4e88700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.364+0000 7f6fc4e88700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fac066f30 0x7f6fac0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0006fd0 tx=0x7f6fb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.368+0000 7f6fbcff9700 1 -- 172.21.15.7:0/2302447406 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fb804c5a0 con 0x7f6fc015d920 2024-01-11T07:07:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.618+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fc004c3c0 con 0x7f6fc015d920 2024-01-11T07:07:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.619+0000 7f6fbcff9700 1 -- 172.21.15.7:0/2302447406 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f6fb8051590 con 0x7f6fc015d920 2024-01-11T07:07:36.619 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:36.619 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:06:58.385324+0000","modified":"2024-01-11T07:07:02.162269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-11T07:07:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fac066f30 msgr2=0x7f6fac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fac066f30 0x7f6fac0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0006fd0 tx=0x7f6fb0008040 comp rx=0 tx=0).stop 2024-01-11T07:07:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 msgr2=0x7f6fc00c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc00c9360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800bad0 tx=0x7f6fb8009760 comp rx=0 tx=0).stop 2024-01-11T07:07:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 shutdown_connections 2024-01-11T07:07:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fac066f30 0x7f6fac0693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc015d920 0x7f6fc00c9360 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc00ac3a0 0x7f6fc00c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 --2- 172.21.15.7:0/2302447406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc015a2c0 0x7f6fc0141790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 >> 172.21.15.7:0/2302447406 conn(0x7f6fc00cd090 msgr2=0x7f6fc0069610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 shutdown_connections 2024-01-11T07:07:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.621+0000 7f6fc70ec700 1 -- 172.21.15.7:0/2302447406 wait complete. 2024-01-11T07:07:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T07:07:36.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:36.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.768+0000 7fd0ca975700 1 Processor -- start 2024-01-11T07:07:36.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.768+0000 7fd0ca975700 1 -- start start 2024-01-11T07:07:36.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0ca975700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd0c4158280 0x7fd0c40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0ca975700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c4159300 con 0x7fd0c415aa80 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0ca975700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c40cf000 con 0x7fd0c40cafb0 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0ca975700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c40cf260 con 0x7fd0c4158280 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0c8f12700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0c415aa80 0x7fd0c4161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46836/0 (socket says 172.21.15.7:46836) 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0c8f12700 1 -- 172.21.15.7:0/3661154509 learned_addr learned my addr 172.21.15.7:0/3661154509 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0c37fe700 1 --2- 172.21.15.7:0/3661154509 >> v2:172.21.15.113:3300/0 conn(0x7fd0c4158280 0x7fd0c40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0c2ffd700 1 -- 172.21.15.7:0/3661154509 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109166946 0 0) 0x7fd0c4159300 con 0x7fd0c415aa80 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0c2ffd700 1 -- 172.21.15.7:0/3661154509 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b0003570 con 0x7fd0c415aa80 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.769+0000 7fd0c37fe700 1 -- 172.21.15.7:0/3661154509 >> v1:172.21.15.113:6789/0 conn(0x7fd0c40cafb0 legacy=0x7fd0c40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.770+0000 7fd0c37fe700 1 -- 172.21.15.7:0/3661154509 >> v1:172.21.15.7:6789/0 conn(0x7fd0c415aa80 legacy=0x7fd0c4161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.770+0000 7fd0c37fe700 1 -- 172.21.15.7:0/3661154509 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c40cf540 con 0x7fd0c4158280 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.770+0000 7fd0c37fe700 1 --2- 172.21.15.7:0/3661154509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c4158280 0x7fd0c40b0430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009860 tx=0x7fd0ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90e65f1245405306 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.770+0000 7fd0c2ffd700 1 -- 172.21.15.7:0/3661154509 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac011630 con 0x7fd0c4158280 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.770+0000 7fd0c2ffd700 1 -- 172.21.15.7:0/3661154509 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac011790 con 0x7fd0c4158280 2024-01-11T07:07:36.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.770+0000 7fd0c2ffd700 1 -- 172.21.15.7:0/3661154509 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac011b30 con 0x7fd0c4158280 2024-01-11T07:07:36.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.770+0000 7fd0ca975700 1 -- 172.21.15.7:0/3661154509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c4158280 msgr2=0x7fd0c40b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.770+0000 7fd0ca975700 1 --2- 172.21.15.7:0/3661154509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c4158280 0x7fd0c40b0430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009860 tx=0x7fd0ac0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.771+0000 7fd0ca975700 1 -- 172.21.15.7:0/3661154509 shutdown_connections 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.771+0000 7fd0ca975700 1 --2- 172.21.15.7:0/3661154509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c4158280 0x7fd0c40b0430 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.771+0000 7fd0ca975700 1 -- 172.21.15.7:0/3661154509 >> 172.21.15.7:0/3661154509 conn(0x7fd0c4071100 msgr2=0x7fd0c406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.771+0000 7fd0ca975700 1 -- 172.21.15.7:0/3661154509 shutdown_connections 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.771+0000 7fd0ca975700 1 -- 172.21.15.7:0/3661154509 wait complete. 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.772+0000 7fd0ca975700 1 Processor -- start 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.772+0000 7fd0ca975700 1 -- start start 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0ca975700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c40cafb0 0x7fd0c40c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0ca975700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0c4158280 0x7fd0c4153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0ca975700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c415aa80 0x7fd0c40ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0ca975700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40c61b0 con 0x7fd0c40cafb0 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0ca975700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40c5220 con 0x7fd0c415aa80 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0ca975700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40c57e0 con 0x7fd0c4158280 2024-01-11T07:07:36.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c40cafb0 0x7fd0c40c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c40cafb0 0x7fd0c40c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34438/0 (socket says 172.21.15.7:34438) 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 -- 172.21.15.7:0/1089908678 learned_addr learned my addr 172.21.15.7:0/1089908678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c37fe700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0c4158280 0x7fd0c4153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c8f12700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c415aa80 0x7fd0c40ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 -- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0c4158280 msgr2=0x7fd0c4153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0c4158280 0x7fd0c4153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 -- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c415aa80 msgr2=0x7fd0c40ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c415aa80 0x7fd0c40ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 -- 172.21.15.7:0/1089908678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac009420 con 0x7fd0c40cafb0 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.773+0000 7fd0c3fff700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c40cafb0 0x7fd0c40c5c70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c810 tx=0x7fd0b4007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.774+0000 7fd0bbfff700 1 -- 172.21.15.7:0/1089908678 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4011590 con 0x7fd0c40cafb0 2024-01-11T07:07:36.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.774+0000 7fd0bbfff700 1 -- 172.21.15.7:0/1089908678 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4011710 con 0x7fd0c40cafb0 2024-01-11T07:07:36.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.774+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c40aec60 con 0x7fd0c40cafb0 2024-01-11T07:07:36.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.774+0000 7fd0bbfff700 1 -- 172.21.15.7:0/1089908678 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4011ad0 con 0x7fd0c40cafb0 2024-01-11T07:07:36.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.774+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c40c5470 con 0x7fd0c40cafb0 2024-01-11T07:07:36.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.775+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c40c5ac0 con 0x7fd0c40cafb0 2024-01-11T07:07:36.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.775+0000 7fd0bbfff700 1 -- 172.21.15.7:0/1089908678 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd0b401b3b0 con 0x7fd0c40cafb0 2024-01-11T07:07:36.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.776+0000 7fd0bbfff700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd0a4066f60 0x7fd0a4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:36.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.776+0000 7fd0bbfff700 1 -- 172.21.15.7:0/1089908678 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd0b4083ad0 con 0x7fd0c40cafb0 2024-01-11T07:07:36.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.776+0000 7fd0c37fe700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd0a4066f60 0x7fd0a4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:36.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.776+0000 7fd0c37fe700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd0a4066f60 0x7fd0a4069410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4158650 tx=0x7fd0ac016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:36.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.780+0000 7fd0bbfff700 1 -- 172.21.15.7:0/1089908678 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0b4050d40 con 0x7fd0c40cafb0 2024-01-11T07:07:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.985+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0c404c3c0 con 0x7fd0c40cafb0 2024-01-11T07:07:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.986+0000 7fd0bbfff700 1 -- 172.21.15.7:0/1089908678 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fd0b4017070 con 0x7fd0c40cafb0 2024-01-11T07:07:36.986 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:36.986 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd0a4066f60 msgr2=0x7fd0a4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd0a4066f60 0x7fd0a4069410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4158650 tx=0x7fd0ac016000 comp rx=0 tx=0).stop 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c40cafb0 msgr2=0x7fd0c40c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c40cafb0 0x7fd0c40c5c70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c810 tx=0x7fd0b4007f30 comp rx=0 tx=0).stop 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 shutdown_connections 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd0a4066f60 0x7fd0a4069410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0c415aa80 0x7fd0c40ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c40cafb0 0x7fd0c40c5c70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 --2- 172.21.15.7:0/1089908678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0c4158280 0x7fd0c4153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 >> 172.21.15.7:0/1089908678 conn(0x7fd0c4071100 msgr2=0x7fd0c4155200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 shutdown_connections 2024-01-11T07:07:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:36.989+0000 7fd0ca975700 1 -- 172.21.15.7:0/1089908678 wait complete. 2024-01-11T07:07:37.002 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:07:37.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:37.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.134+0000 7f733dbd6700 1 Processor -- start 2024-01-11T07:07:37.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.134+0000 7f733dbd6700 1 -- start start 2024-01-11T07:07:37.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.134+0000 7f733dbd6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73380cafb0 0x7f73380ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f733dbd6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7338158280 0x7f73380b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f733dbd6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7338159300 con 0x7f73380b0970 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f733dbd6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73380b0dc0 con 0x7f7338158280 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f733dbd6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73380cf260 con 0x7f73380cafb0 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f7336ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7338158280 0x7f73380b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f7337fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73380b0970 0x7f7338159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46838/0 (socket says 172.21.15.7:46838) 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f7337fff700 1 -- 172.21.15.7:0/1189519006 learned_addr learned my addr 172.21.15.7:0/1189519006 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f73377fe700 1 --2- 172.21.15.7:0/1189519006 >> v2:172.21.15.113:3300/0 conn(0x7f73380cafb0 0x7f73380ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f7336ffd700 1 -- 172.21.15.7:0/1189519006 >> v2:172.21.15.113:3300/0 conn(0x7f73380cafb0 msgr2=0x7f73380ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f7336ffd700 1 --2- 172.21.15.7:0/1189519006 >> v2:172.21.15.113:3300/0 conn(0x7f73380cafb0 0x7f73380ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f7336ffd700 1 -- 172.21.15.7:0/1189519006 >> v1:172.21.15.7:6789/0 conn(0x7f73380b0970 legacy=0x7f7338159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f73367fc700 1 -- 172.21.15.7:0/1189519006 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571002218 0 0) 0x7f7338159300 con 0x7f73380b0970 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f7336ffd700 1 -- 172.21.15.7:0/1189519006 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73380cf540 con 0x7f7338158280 2024-01-11T07:07:37.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.135+0000 7f7336ffd700 1 --2- 172.21.15.7:0/1189519006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7338158280 0x7f73380b0430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73380cfb70 tx=0x7f732c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f54ecf2e407e22 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f73367fc700 1 -- 172.21.15.7:0/1189519006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c011630 con 0x7f7338158280 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f73367fc700 1 -- 172.21.15.7:0/1189519006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c011790 con 0x7f7338158280 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f73367fc700 1 -- 172.21.15.7:0/1189519006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c011b30 con 0x7f7338158280 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f733dbd6700 1 -- 172.21.15.7:0/1189519006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7338158280 msgr2=0x7f73380b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f733dbd6700 1 --2- 172.21.15.7:0/1189519006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7338158280 0x7f73380b0430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73380cfb70 tx=0x7f732c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f733dbd6700 1 -- 172.21.15.7:0/1189519006 shutdown_connections 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f733dbd6700 1 --2- 172.21.15.7:0/1189519006 >> v2:172.21.15.113:3300/0 conn(0x7f73380cafb0 0x7f73380ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f733dbd6700 1 --2- 172.21.15.7:0/1189519006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7338158280 0x7f73380b0430 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f733dbd6700 1 -- 172.21.15.7:0/1189519006 >> 172.21.15.7:0/1189519006 conn(0x7f7338071100 msgr2=0x7f733806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f733dbd6700 1 -- 172.21.15.7:0/1189519006 shutdown_connections 2024-01-11T07:07:37.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.136+0000 7f733dbd6700 1 -- 172.21.15.7:0/1189519006 wait complete. 2024-01-11T07:07:37.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.138+0000 7f733dbd6700 1 Processor -- start 2024-01-11T07:07:37.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.138+0000 7f733dbd6700 1 -- start start 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f733dbd6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73380b0970 0x7f7338155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f733dbd6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73380cafb0 0x7f73380c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f733dbd6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7338158280 0x7f73380c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f733dbd6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7338069420 con 0x7f73380cafb0 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f733dbd6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733815cf70 con 0x7f73380b0970 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f733dbd6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73380c87f0 con 0x7f7338158280 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f7336ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73380cafb0 0x7f73380c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f7336ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73380cafb0 0x7f73380c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34454/0 (socket says 172.21.15.7:34454) 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f7336ffd700 1 -- 172.21.15.7:0/493260766 learned_addr learned my addr 172.21.15.7:0/493260766 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f73377fe700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73380b0970 0x7f7338155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.139+0000 7f7337fff700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7338158280 0x7f73380c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f7336ffd700 1 -- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7338158280 msgr2=0x7f73380c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f7336ffd700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7338158280 0x7f73380c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f7336ffd700 1 -- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73380b0970 msgr2=0x7f7338155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f7336ffd700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73380b0970 0x7f7338155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f7336ffd700 1 -- 172.21.15.7:0/493260766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c009420 con 0x7f73380cafb0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f7336ffd700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73380cafb0 0x7f73380c7c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f732c013000 tx=0x7f732c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f731f7fe700 1 -- 172.21.15.7:0/493260766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c000d10 con 0x7f73380cafb0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f731f7fe700 1 -- 172.21.15.7:0/493260766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c011b10 con 0x7f73380cafb0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73380c8ad0 con 0x7f73380cafb0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f731f7fe700 1 -- 172.21.15.7:0/493260766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c025640 con 0x7f73380cafb0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.140+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733815cb90 con 0x7f73380cafb0 2024-01-11T07:07:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.141+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73380cc190 con 0x7f73380cafb0 2024-01-11T07:07:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.142+0000 7f731f7fe700 1 -- 172.21.15.7:0/493260766 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f732c025a30 con 0x7f73380cafb0 2024-01-11T07:07:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.142+0000 7f731f7fe700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.142+0000 7f731f7fe700 1 -- 172.21.15.7:0/493260766 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f732c085ce0 con 0x7f73380cafb0 2024-01-11T07:07:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.142+0000 7f73377fe700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.143+0000 7f73377fe700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7324066f30 0x7f73240693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7328009d20 tx=0x7f73280091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.146+0000 7f731f7fe700 1 -- 172.21.15.7:0/493260766 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f732c052f50 con 0x7f73380cafb0 2024-01-11T07:07:37.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.398+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f733804c3c0 con 0x7f73380cafb0 2024-01-11T07:07:37.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.398+0000 7f731f7fe700 1 -- 172.21.15.7:0/493260766 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f732c056570 con 0x7f73380cafb0 2024-01-11T07:07:37.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:37.398 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:06:58.385324+0000","modified":"2024-01-11T07:07:02.162269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7324066f30 msgr2=0x7f73240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7324066f30 0x7f73240693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7328009d20 tx=0x7f73280091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73380cafb0 msgr2=0x7f73380c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73380cafb0 0x7f73380c7c10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f732c013000 tx=0x7f732c017000 comp rx=0 tx=0).stop 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 shutdown_connections 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7338158280 0x7f73380c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73380b0970 0x7f7338155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 --2- 172.21.15.7:0/493260766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73380cafb0 0x7f73380c7c10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 >> 172.21.15.7:0/493260766 conn(0x7f7338071100 msgr2=0x7f73380cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 shutdown_connections 2024-01-11T07:07:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.401+0000 7f733dbd6700 1 -- 172.21.15.7:0/493260766 wait complete. 2024-01-11T07:07:37.402 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T07:07:37.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:37.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.546+0000 7f39b8d4b700 1 Processor -- start 2024-01-11T07:07:37.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.546+0000 7f39b8d4b700 1 -- start start 2024-01-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.547+0000 7f39b8d4b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b406f3c0 0x7f39b40ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.547+0000 7f39b8d4b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b40c8f80 con 0x7f39b4157ab0 2024-01-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.547+0000 7f39b8d4b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b40c9de0 con 0x7f39b4157e80 2024-01-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.547+0000 7f39b8d4b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40c91d0 con 0x7f39b406f3c0 2024-01-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.547+0000 7f39b259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f39b4157ab0 0x7f39b406ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38962/0 (socket says 172.21.15.7:38962) 2024-01-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.547+0000 7f39b259c700 1 -- 172.21.15.7:0/237571963 learned_addr learned my addr 172.21.15.7:0/237571963 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.547+0000 7f39b2d9d700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b406f3c0 0x7f39b40ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034966262 0 0) 0x7f39b40c8f80 con 0x7f39b4157ab0 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a0003570 con 0x7f39b4157ab0 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165665266 0 0) 0x7f39a0003570 con 0x7f39b4157ab0 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b406f3c0 msgr2=0x7f39b40ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b406f3c0 0x7f39b40ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 >> v1:172.21.15.113:6789/0 conn(0x7f39b4157e80 legacy=0x7f39b40b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40c94b0 con 0x7f39b4157ab0 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f39a4001470 con 0x7f39b4157ab0 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 >> v1:172.21.15.7:6789/0 conn(0x7f39b4157ab0 legacy=0x7f39b406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0003a90 0x7f39a0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39a0004810 0x7f39a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b406f3c0 0x7f39a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40c91d0 con 0x7f39a0003a90 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a000bb10 con 0x7f39a0004810 2024-01-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a000bcd0 con 0x7f39b406f3c0 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b259c700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0003a90 0x7f39a0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b1d9b700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39a0004810 0x7f39a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b2d9d700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b406f3c0 0x7f39a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b259c700 1 -- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b406f3c0 msgr2=0x7f39a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b259c700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39b406f3c0 0x7f39a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b259c700 1 -- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39a0004810 msgr2=0x7f39a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b259c700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39a0004810 0x7f39a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.548+0000 7f39b259c700 1 -- 172.21.15.7:0/237571963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a0003570 con 0x7f39a0003a90 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b259c700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0003a90 0x7f39a0003e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c390 tx=0x7f39b40c9d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d120b474afcf2e2d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a400cc60 con 0x7f39a0003a90 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a400cde0 con 0x7f39a0003a90 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b159a700 1 -- 172.21.15.7:0/237571963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4005b70 con 0x7f39a0003a90 2024-01-11T07:07:37.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b8d4b700 1 -- 172.21.15.7:0/237571963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0003a90 msgr2=0x7f39a0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b8d4b700 1 --2- 172.21.15.7:0/237571963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0003a90 0x7f39a0003e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c390 tx=0x7f39b40c9d50 comp rx=0 tx=0).stop 2024-01-11T07:07:37.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b2d9d700 1 -- 172.21.15.7:0/237571963 reap_dead start 2024-01-11T07:07:37.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b8d4b700 1 -- 172.21.15.7:0/237571963 shutdown_connections 2024-01-11T07:07:37.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.549+0000 7f39b8d4b700 1 -- 172.21.15.7:0/237571963 >> 172.21.15.7:0/237571963 conn(0x7f39b4070690 msgr2=0x7f39b4070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:37.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.550+0000 7f39b8d4b700 1 -- 172.21.15.7:0/237571963 shutdown_connections 2024-01-11T07:07:37.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.550+0000 7f39b8d4b700 1 -- 172.21.15.7:0/237571963 wait complete. 2024-01-11T07:07:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.551+0000 7f39b8d4b700 1 Processor -- start 2024-01-11T07:07:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.551+0000 7f39b8d4b700 1 -- start start 2024-01-11T07:07:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.551+0000 7f39b8d4b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a0003a90 0x7f39b416e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.551+0000 7f39b8d4b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b416a0f0 0x7f39b414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.551+0000 7f39b8d4b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b40679b0 0x7f39b415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.551+0000 7f39b8d4b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40ccc60 con 0x7f39b416a0f0 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.551+0000 7f39b8d4b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b416a8d0 con 0x7f39b40679b0 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.551+0000 7f39b8d4b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b416abb0 con 0x7f39a0003a90 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b416a0f0 0x7f39b414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b416a0f0 0x7f39b414ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56698/0 (socket says 172.21.15.7:56698) 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 -- 172.21.15.7:0/147418328 learned_addr learned my addr 172.21.15.7:0/147418328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b259c700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a0003a90 0x7f39b416e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b2d9d700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b40679b0 0x7f39b415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 -- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a0003a90 msgr2=0x7f39b416e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a0003a90 0x7f39b416e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 -- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b40679b0 msgr2=0x7f39b415c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b40679b0 0x7f39b415c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 -- 172.21.15.7:0/147418328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b416ae90 con 0x7f39b416a0f0 2024-01-11T07:07:37.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b1d9b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b416a0f0 0x7f39b414ee10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f399c002b90 tx=0x7f399c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39aa7fc700 1 -- 172.21.15.7:0/147418328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c00f850 con 0x7f39b416a0f0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39aa7fc700 1 -- 172.21.15.7:0/147418328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c00f9d0 con 0x7f39b416a0f0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b414f350 con 0x7f39b416a0f0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39aa7fc700 1 -- 172.21.15.7:0/147418328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c00fd90 con 0x7f39b416a0f0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.552+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b4067e20 con 0x7f39b416a0f0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.553+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b40680a0 con 0x7f39b416a0f0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.554+0000 7f39aa7fc700 1 -- 172.21.15.7:0/147418328 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f399c013600 con 0x7f39b416a0f0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.554+0000 7f39aa7fc700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3994066f30 0x7f39940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.554+0000 7f39aa7fc700 1 -- 172.21.15.7:0/147418328 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f399c084ae0 con 0x7f39b416a0f0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.554+0000 7f39b259c700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3994066f30 0x7f39940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.555+0000 7f39b259c700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3994066f30 0x7f39940693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f39a4007d10 tx=0x7f39a4015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.559+0000 7f39aa7fc700 1 -- 172.21.15.7:0/147418328 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39b40680a0 con 0x7f39b416a0f0 2024-01-11T07:07:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.809+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39b40680a0 con 0x7f39b416a0f0 2024-01-11T07:07:37.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.810+0000 7f39aa7fc700 1 -- 172.21.15.7:0/147418328 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4588 (secure 0 0 0) 0x7f39b40680a0 con 0x7f39b416a0f0 2024-01-11T07:07:37.810 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:37.810 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3952109348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3952109348},{"type":"v1","addr":"172.21.15.113:6837","nonce":3952109348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/4092952892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":4092952892},{"type":"v1","addr":"172.21.15.7:6834","nonce":4092952892}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2977774932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2977774932},{"type":"v1","addr":"172.21.15.7:6835","nonce":2977774932}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:06:58.385324+0000","modified":"2024-01-11T07:07:02.162269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6836/153691742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":153691742},{"type":"v1","addr":"172.21.15.113:6836","nonce":153691742}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-01-11T07:07:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3994066f30 msgr2=0x7f39940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3994066f30 0x7f39940693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f39a4007d10 tx=0x7f39a4015000 comp rx=0 tx=0).stop 2024-01-11T07:07:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b416a0f0 msgr2=0x7f39b414ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b416a0f0 0x7f39b414ee10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f399c002b90 tx=0x7f399c007d10 comp rx=0 tx=0).stop 2024-01-11T07:07:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 shutdown_connections 2024-01-11T07:07:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3994066f30 0x7f39940693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b40679b0 0x7f39b415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39a0003a90 0x7f39b416e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 --2- 172.21.15.7:0/147418328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b416a0f0 0x7f39b414ee10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 >> 172.21.15.7:0/147418328 conn(0x7f39b4070690 msgr2=0x7f39b4156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 shutdown_connections 2024-01-11T07:07:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.812+0000 7f39b8d4b700 1 -- 172.21.15.7:0/147418328 wait complete. 2024-01-11T07:07:37.813 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-01-11T07:07:37.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:37.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.958+0000 7fb82e172700 1 Processor -- start 2024-01-11T07:07:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.958+0000 7fb82e172700 1 -- start start 2024-01-11T07:07:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.959+0000 7fb82e172700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb82806afc0 0x7fb82815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.959+0000 7fb82e172700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82815b770 con 0x7fb82806e0d0 2024-01-11T07:07:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.959+0000 7fb82e172700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82815abf0 con 0x7fb82806dd00 2024-01-11T07:07:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.959+0000 7fb82e172700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82815aed0 con 0x7fb82806afc0 2024-01-11T07:07:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.959+0000 7fb826ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb82806e0d0 0x7fb828159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38966/0 (socket says 172.21.15.7:38966) 2024-01-11T07:07:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.959+0000 7fb826ffd700 1 -- 172.21.15.7:0/1579856831 learned_addr learned my addr 172.21.15.7:0/1579856831 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.959+0000 7fb827fff700 1 --2- 172.21.15.7:0/1579856831 >> v2:172.21.15.113:3300/0 conn(0x7fb82806afc0 0x7fb82815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.960+0000 7fb827fff700 1 -- 172.21.15.7:0/1579856831 >> v1:172.21.15.113:6789/0 conn(0x7fb82806dd00 legacy=0x7fb8280c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.960+0000 7fb827fff700 1 -- 172.21.15.7:0/1579856831 >> v1:172.21.15.7:6789/0 conn(0x7fb82806e0d0 legacy=0x7fb828159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.960+0000 7fb8267fc700 1 -- 172.21.15.7:0/1579856831 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684391601 0 0) 0x7fb82815b770 con 0x7fb82806e0d0 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.960+0000 7fb827fff700 1 -- 172.21.15.7:0/1579856831 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82815b1b0 con 0x7fb82806afc0 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.960+0000 7fb827fff700 1 --2- 172.21.15.7:0/1579856831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806afc0 0x7fb82815a1c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb820009de0 tx=0x7fb820009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff4111d714f446c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.960+0000 7fb8267fc700 1 -- 172.21.15.7:0/1579856831 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000e070 con 0x7fb82806afc0 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.960+0000 7fb8267fc700 1 -- 172.21.15.7:0/1579856831 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82000bbb0 con 0x7fb82806afc0 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.960+0000 7fb8267fc700 1 -- 172.21.15.7:0/1579856831 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820012430 con 0x7fb82806afc0 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.961+0000 7fb82e172700 1 -- 172.21.15.7:0/1579856831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806afc0 msgr2=0x7fb82815a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.961+0000 7fb82e172700 1 --2- 172.21.15.7:0/1579856831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806afc0 0x7fb82815a1c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb820009de0 tx=0x7fb820009780 comp rx=0 tx=0).stop 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.961+0000 7fb82e172700 1 -- 172.21.15.7:0/1579856831 shutdown_connections 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.961+0000 7fb82e172700 1 --2- 172.21.15.7:0/1579856831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806afc0 0x7fb82815a1c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.961+0000 7fb82e172700 1 -- 172.21.15.7:0/1579856831 >> 172.21.15.7:0/1579856831 conn(0x7fb8280680e0 msgr2=0x7fb8280cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.961+0000 7fb82e172700 1 -- 172.21.15.7:0/1579856831 shutdown_connections 2024-01-11T07:07:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.961+0000 7fb82e172700 1 -- 172.21.15.7:0/1579856831 wait complete. 2024-01-11T07:07:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.962+0000 7fb82e172700 1 Processor -- start 2024-01-11T07:07:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.963+0000 7fb82e172700 1 -- start start 2024-01-11T07:07:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.963+0000 7fb82e172700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82806afc0 0x7fb828157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.963+0000 7fb82e172700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb82806dd00 0x7fb8281686f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.963+0000 7fb82e172700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806e0d0 0x7fb8280a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.963+0000 7fb82e172700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb828168c30 con 0x7fb82806afc0 2024-01-11T07:07:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.963+0000 7fb82e172700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280a6bb0 con 0x7fb82806e0d0 2024-01-11T07:07:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.963+0000 7fb82e172700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280c8400 con 0x7fb82806dd00 2024-01-11T07:07:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82806afc0 0x7fb828157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82806afc0 0x7fb828157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56712/0 (socket says 172.21.15.7:56712) 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb826ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb82806dd00 0x7fb8281686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 -- 172.21.15.7:0/1166004576 learned_addr learned my addr 172.21.15.7:0/1166004576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb827fff700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806e0d0 0x7fb8280a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 -- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb82806dd00 msgr2=0x7fb8281686f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb82806dd00 0x7fb8281686f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 -- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806e0d0 msgr2=0x7fb8280a7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806e0d0 0x7fb8280a7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 -- 172.21.15.7:0/1166004576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820009a20 con 0x7fb82806afc0 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb8277fe700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82806afc0 0x7fb828157760 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c810 tx=0x7fb818007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb80f7fe700 1 -- 172.21.15.7:0/1166004576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818011590 con 0x7fb82806afc0 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb80f7fe700 1 -- 172.21.15.7:0/1166004576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818011710 con 0x7fb82806afc0 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8280c86e0 con 0x7fb82806afc0 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb80f7fe700 1 -- 172.21.15.7:0/1166004576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818011ad0 con 0x7fb82806afc0 2024-01-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.964+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8280a6840 con 0x7fb82806afc0 2024-01-11T07:07:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.966+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82804c3c0 con 0x7fb82806afc0 2024-01-11T07:07:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.966+0000 7fb80f7fe700 1 -- 172.21.15.7:0/1166004576 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb81801b3b0 con 0x7fb82806afc0 2024-01-11T07:07:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.967+0000 7fb80f7fe700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb814066f30 0x7fb8140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.967+0000 7fb80f7fe700 1 -- 172.21.15.7:0/1166004576 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb818083ac0 con 0x7fb82806afc0 2024-01-11T07:07:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.967+0000 7fb826ffd700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb814066f30 0x7fb8140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:37.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.967+0000 7fb826ffd700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb814066f30 0x7fb8140693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb8280c9b60 tx=0x7fb81c007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:37.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:37.971+0000 7fb80f7fe700 1 -- 172.21.15.7:0/1166004576 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb818050d30 con 0x7fb82806afc0 2024-01-11T07:07:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.176+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb82815b490 con 0x7fb82806afc0 2024-01-11T07:07:38.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.177+0000 7fb80f7fe700 1 -- 172.21.15.7:0/1166004576 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fb818083e60 con 0x7fb82806afc0 2024-01-11T07:07:38.177 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:38.177 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:06:59.964564+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:06:56.038804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-11T07:06:57.365958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:38.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb814066f30 msgr2=0x7fb8140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb814066f30 0x7fb8140693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb8280c9b60 tx=0x7fb81c007f30 comp rx=0 tx=0).stop 2024-01-11T07:07:38.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82806afc0 msgr2=0x7fb828157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82806afc0 0x7fb828157760 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c810 tx=0x7fb818007ee0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 shutdown_connections 2024-01-11T07:07:38.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb814066f30 0x7fb8140693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb82806e0d0 0x7fb8280a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb82806afc0 0x7fb828157760 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 --2- 172.21.15.7:0/1166004576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb82806dd00 0x7fb8281686f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 >> 172.21.15.7:0/1166004576 conn(0x7fb8280680e0 msgr2=0x7fb8280ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:38.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.179+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 shutdown_connections 2024-01-11T07:07:38.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.180+0000 7fb82e172700 1 -- 172.21.15.7:0/1166004576 wait complete. 2024-01-11T07:07:38.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:07:38.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.328+0000 7f72f7949700 1 Processor -- start 2024-01-11T07:07:38.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.328+0000 7f72f7949700 1 -- start start 2024-01-11T07:07:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f7949700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f00aed00 0x7f72f015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f7949700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72f006a410 0x7f72f00cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f7949700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f0158010 con 0x7f72f0156e90 2024-01-11T07:07:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f7949700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f015b530 con 0x7f72f006a410 2024-01-11T07:07:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f7949700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0157400 con 0x7f72f00aed00 2024-01-11T07:07:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f5ee6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72f006a410 0x7f72f00cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f5ee6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72f006a410 0x7f72f00cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56716/0 (socket says 172.21.15.7:56716) 2024-01-11T07:07:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f5ee6700 1 -- 172.21.15.7:0/4053059641 learned_addr learned my addr 172.21.15.7:0/4053059641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f56e5700 1 --2- 172.21.15.7:0/4053059641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f00aed00 0x7f72f015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f5ee6700 1 -- 172.21.15.7:0/4053059641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f00aed00 msgr2=0x7f72f015aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f5ee6700 1 --2- 172.21.15.7:0/4053059641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f00aed00 0x7f72f015aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f5ee6700 1 -- 172.21.15.7:0/4053059641 >> v1:172.21.15.113:6789/0 conn(0x7f72f0156e90 legacy=0x7f72f0069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72f5ee6700 1 -- 172.21.15.7:0/4053059641 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f01576e0 con 0x7f72f006a410 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.329+0000 7f72e7fff700 1 -- 172.21.15.7:0/4053059641 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405527611 0 0) 0x7f72f0158010 con 0x7f72f0156e90 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72f5ee6700 1 --2- 172.21.15.7:0/4053059641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f006a410 0x7f72f00cafc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009f20 tx=0x7f72e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931c03525bce8fca server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72e7fff700 1 -- 172.21.15.7:0/4053059641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e800e070 con 0x7f72f006a410 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72e7fff700 1 -- 172.21.15.7:0/4053059641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e800b9d0 con 0x7f72f006a410 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72e7fff700 1 -- 172.21.15.7:0/4053059641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e80092e0 con 0x7f72f006a410 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72f7949700 1 -- 172.21.15.7:0/4053059641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f006a410 msgr2=0x7f72f00cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72f7949700 1 --2- 172.21.15.7:0/4053059641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f006a410 0x7f72f00cafc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009f20 tx=0x7f72e8009840 comp rx=0 tx=0).stop 2024-01-11T07:07:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72f7949700 1 -- 172.21.15.7:0/4053059641 shutdown_connections 2024-01-11T07:07:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72f7949700 1 --2- 172.21.15.7:0/4053059641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f006a410 0x7f72f00cafc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72f7949700 1 --2- 172.21.15.7:0/4053059641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f00aed00 0x7f72f015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.330+0000 7f72f7949700 1 -- 172.21.15.7:0/4053059641 >> 172.21.15.7:0/4053059641 conn(0x7f72f0063230 msgr2=0x7f72f0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.331+0000 7f72f7949700 1 -- 172.21.15.7:0/4053059641 shutdown_connections 2024-01-11T07:07:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.331+0000 7f72f7949700 1 -- 172.21.15.7:0/4053059641 wait complete. 2024-01-11T07:07:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.332+0000 7f72f7949700 1 Processor -- start 2024-01-11T07:07:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.332+0000 7f72f7949700 1 -- start start 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.333+0000 7f72f7949700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f006a410 0x7f72f00c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.333+0000 7f72f7949700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f00aed00 0x7f72f00b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.333+0000 7f72f7949700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72f0156e90 0x7f72f00c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.333+0000 7f72f7949700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f00c6750 con 0x7f72f00aed00 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.333+0000 7f72f7949700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f00b2270 con 0x7f72f0156e90 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.333+0000 7f72f7949700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0152e50 con 0x7f72f006a410 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f00aed00 0x7f72f00b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f00aed00 0x7f72f00b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56724/0 (socket says 172.21.15.7:56724) 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 -- 172.21.15.7:0/3916592594 learned_addr learned my addr 172.21.15.7:0/3916592594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f56e5700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f006a410 0x7f72f00c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f5ee6700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72f0156e90 0x7f72f00c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 -- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f006a410 msgr2=0x7f72f00c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f006a410 0x7f72f00c2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 -- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72f0156e90 msgr2=0x7f72f00c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72f0156e90 0x7f72f00c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 -- 172.21.15.7:0/3916592594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8009ae0 con 0x7f72f00aed00 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f5ee6700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72f0156e90 0x7f72f00c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f4ee4700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f00aed00 0x7f72f00b1d30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f72e000c530 tx=0x7f72e000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72e57fa700 1 -- 172.21.15.7:0/3916592594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0011590 con 0x7f72f00aed00 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72e57fa700 1 -- 172.21.15.7:0/3916592594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e0011710 con 0x7f72f00aed00 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f01589f0 con 0x7f72f00aed00 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72e57fa700 1 -- 172.21.15.7:0/3916592594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0011ad0 con 0x7f72f00aed00 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.334+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f00c64f0 con 0x7f72f00aed00 2024-01-11T07:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.335+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f01584e0 con 0x7f72f00aed00 2024-01-11T07:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.336+0000 7f72e57fa700 1 -- 172.21.15.7:0/3916592594 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f72e001b3b0 con 0x7f72f00aed00 2024-01-11T07:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.336+0000 7f72e57fa700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.336+0000 7f72e57fa700 1 -- 172.21.15.7:0/3916592594 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72e0083a70 con 0x7f72f00aed00 2024-01-11T07:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.337+0000 7f72f56e5700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.337+0000 7f72f56e5700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009db0 tx=0x7f72ec0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.341+0000 7f72e57fa700 1 -- 172.21.15.7:0/3916592594 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72f01584e0 con 0x7f72f00aed00 2024-01-11T07:07:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.590+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f72f01584e0 con 0x7f72f00aed00 2024-01-11T07:07:38.590 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:07:38.591+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:07:38.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.594+0000 7f72e57fa700 1 -- 172.21.15.7:0/3916592594 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f72f01584e0 con 0x7f72f00aed00 2024-01-11T07:07:38.595 INFO:tasks.ceph.mds.b.smithi113.stderr: -1839> 2024-01-11T07:07:04.489+0000 7f5638400700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:07:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 msgr2=0x7f72d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009db0 tx=0x7f72ec0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f00aed00 msgr2=0x7f72f00b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f00aed00 0x7f72f00b1d30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f72e000c530 tx=0x7f72e000c840 comp rx=0 tx=0).stop 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 shutdown_connections 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72f0156e90 0x7f72f00c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f006a410 0x7f72f00c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 --2- 172.21.15.7:0/3916592594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f00aed00 0x7f72f00b1d30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 >> 172.21.15.7:0/3916592594 conn(0x7f72f0063230 msgr2=0x7f72f00cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 shutdown_connections 2024-01-11T07:07:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.597+0000 7f72f7949700 1 -- 172.21.15.7:0/3916592594 wait complete. 2024-01-11T07:07:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:07:38.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:07:38.619 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:07:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.745+0000 7f060c7aa700 1 Processor -- start 2024-01-11T07:07:38.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.745+0000 7f060c7aa700 1 -- start start 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.746+0000 7f060c7aa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0604063230 0x7f0604063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.746+0000 7f060c7aa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040ab100 con 0x7f06040ac5b0 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060c7aa700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060415d880 con 0x7f060415a280 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060c7aa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060406c5b0 con 0x7f0604063230 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060a546700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0604063230 0x7f0604063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060ad47700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f06040ac5b0 0x7f06040ac9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38978/0 (socket says 172.21.15.7:38978) 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060ad47700 1 -- 172.21.15.7:0/3855223087 learned_addr learned my addr 172.21.15.7:0/3855223087 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060a546700 1 -- 172.21.15.7:0/3855223087 >> v1:172.21.15.113:6789/0 conn(0x7f060415a280 legacy=0x7f06040aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060a546700 1 -- 172.21.15.7:0/3855223087 >> v1:172.21.15.7:6789/0 conn(0x7f06040ac5b0 legacy=0x7f06040ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060a546700 1 -- 172.21.15.7:0/3855223087 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604063b40 con 0x7f0604063230 2024-01-11T07:07:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f0609544700 1 -- 172.21.15.7:0/3855223087 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957318569 0 0) 0x7f06040ab100 con 0x7f06040ac5b0 2024-01-11T07:07:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f060a546700 1 --2- 172.21.15.7:0/3855223087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0604063230 0x7f0604063600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06000098e0 tx=0x7f06000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77abe4870da2e705 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f0609544700 1 -- 172.21.15.7:0/3855223087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600009e80 con 0x7f0604063230 2024-01-11T07:07:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f0609544700 1 -- 172.21.15.7:0/3855223087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060000b860 con 0x7f0604063230 2024-01-11T07:07:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.747+0000 7f0609544700 1 -- 172.21.15.7:0/3855223087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060000bbe0 con 0x7f0604063230 2024-01-11T07:07:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.748+0000 7f060c7aa700 1 -- 172.21.15.7:0/3855223087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0604063230 msgr2=0x7f0604063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.748+0000 7f060c7aa700 1 --2- 172.21.15.7:0/3855223087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0604063230 0x7f0604063600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06000098e0 tx=0x7f06000091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.748+0000 7f060c7aa700 1 -- 172.21.15.7:0/3855223087 shutdown_connections 2024-01-11T07:07:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.748+0000 7f060c7aa700 1 --2- 172.21.15.7:0/3855223087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0604063230 0x7f0604063600 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.748+0000 7f060c7aa700 1 -- 172.21.15.7:0/3855223087 >> 172.21.15.7:0/3855223087 conn(0x7f06040680e0 msgr2=0x7f06040cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.748+0000 7f060c7aa700 1 -- 172.21.15.7:0/3855223087 shutdown_connections 2024-01-11T07:07:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.748+0000 7f060c7aa700 1 -- 172.21.15.7:0/3855223087 wait complete. 2024-01-11T07:07:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.749+0000 7f060c7aa700 1 Processor -- start 2024-01-11T07:07:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.749+0000 7f060c7aa700 1 -- start start 2024-01-11T07:07:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060c7aa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0604063230 0x7f0604146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060c7aa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06040ac5b0 0x7f06040c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060c7aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060415a280 0x7f06040ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060c7aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06041472d0 con 0x7f060415a280 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060c7aa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060416cb00 con 0x7f06040ac5b0 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060c7aa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060406b420 con 0x7f0604063230 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060ad47700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060415a280 0x7f06040ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060ad47700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060415a280 0x7f06040ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56742/0 (socket says 172.21.15.7:56742) 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060ad47700 1 -- 172.21.15.7:0/1640557961 learned_addr learned my addr 172.21.15.7:0/1640557961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f060a546700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0604063230 0x7f0604146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.750+0000 7f0609d45700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06040ac5b0 0x7f06040c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f060ad47700 1 -- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0604063230 msgr2=0x7f0604146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f060ad47700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0604063230 0x7f0604146d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f060ad47700 1 -- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06040ac5b0 msgr2=0x7f06040c12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f060ad47700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06040ac5b0 0x7f06040c12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f060ad47700 1 -- 172.21.15.7:0/1640557961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0600009420 con 0x7f060415a280 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f060ad47700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060415a280 0x7f06040ade00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00cc90 tx=0x7f05fc001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f05fa7fc700 1 -- 172.21.15.7:0/1640557961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00a8b0 con 0x7f060415a280 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f05fa7fc700 1 -- 172.21.15.7:0/1640557961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc00aa10 con 0x7f060415a280 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060416c1d0 con 0x7f060415a280 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f05fa7fc700 1 -- 172.21.15.7:0/1640557961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00add0 con 0x7f060415a280 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.751+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060416c790 con 0x7f060415a280 2024-01-11T07:07:38.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.752+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060416cc90 con 0x7f060415a280 2024-01-11T07:07:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.753+0000 7f05fa7fc700 1 -- 172.21.15.7:0/1640557961 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f05fc007300 con 0x7f060415a280 2024-01-11T07:07:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.753+0000 7f05fa7fc700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f05e8066f30 0x7f05e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.753+0000 7f05fa7fc700 1 -- 172.21.15.7:0/1640557961 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f05fc084860 con 0x7f060415a280 2024-01-11T07:07:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.753+0000 7f060a546700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f05e8066f30 0x7f05e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:38.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.754+0000 7f060a546700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f05e8066f30 0x7f05e80693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f06000093f0 tx=0x7f0600017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:38.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:38.757+0000 7f05fa7fc700 1 -- 172.21.15.7:0/1640557961 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05fc051b00 con 0x7f060415a280 2024-01-11T07:07:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.013+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f060404c3c0 con 0x7f060415a280 2024-01-11T07:07:39.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.595+0000 7f05fa7fc700 1 -- 172.21.15.7:0/1640557961 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05fc055120 con 0x7f060415a280 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f05e8066f30 msgr2=0x7f05e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f05e8066f30 0x7f05e80693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f06000093f0 tx=0x7f0600017000 comp rx=0 tx=0).stop 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060415a280 msgr2=0x7f06040ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060415a280 0x7f06040ade00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00cc90 tx=0x7f05fc001a80 comp rx=0 tx=0).stop 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 shutdown_connections 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f05e8066f30 0x7f05e80693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f060415a280 0x7f06040ade00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0604063230 0x7f0604146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 --2- 172.21.15.7:0/1640557961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06040ac5b0 0x7f06040c12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 >> 172.21.15.7:0/1640557961 conn(0x7f06040680e0 msgr2=0x7f06040cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 shutdown_connections 2024-01-11T07:07:39.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.598+0000 7f060c7aa700 1 -- 172.21.15.7:0/1640557961 wait complete. 2024-01-11T07:07:39.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:07:39.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.752+0000 7ff88361a700 1 Processor -- start 2024-01-11T07:07:39.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.752+0000 7ff88361a700 1 -- start start 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.753+0000 7ff88361a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff87c0cac20 0x7ff87c162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.753+0000 7ff88361a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c0cd4c0 con 0x7ff87c15b2b0 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.753+0000 7ff88361a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c0ced00 con 0x7ff87c15aee0 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.753+0000 7ff88361a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c0cc5d0 con 0x7ff87c0cac20 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.753+0000 7ff881bb7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff87c0cac20 0x7ff87c162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.753+0000 7ff880bb5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff87c15b2b0 0x7ff87c0ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38984/0 (socket says 172.21.15.7:38984) 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.753+0000 7ff880bb5700 1 -- 172.21.15.7:0/1464057675 learned_addr learned my addr 172.21.15.7:0/1464057675 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.754+0000 7ff881bb7700 1 -- 172.21.15.7:0/1464057675 >> v1:172.21.15.113:6789/0 conn(0x7ff87c15aee0 legacy=0x7ff87c15c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.754+0000 7ff881bb7700 1 -- 172.21.15.7:0/1464057675 >> v1:172.21.15.7:6789/0 conn(0x7ff87c15b2b0 legacy=0x7ff87c0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.754+0000 7ff881bb7700 1 -- 172.21.15.7:0/1464057675 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c0cc8b0 con 0x7ff87c0cac20 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.754+0000 7ff873fff700 1 -- 172.21.15.7:0/1464057675 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311046910 0 0) 0x7ff87c0cd4c0 con 0x7ff87c15b2b0 2024-01-11T07:07:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.754+0000 7ff881bb7700 1 --2- 172.21.15.7:0/1464057675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c0cac20 0x7ff87c162ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff874009e60 tx=0x7ff874009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105b2fdce3da3f50 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.754+0000 7ff873fff700 1 -- 172.21.15.7:0/1464057675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87400e070 con 0x7ff87c0cac20 2024-01-11T07:07:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.754+0000 7ff873fff700 1 -- 172.21.15.7:0/1464057675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87400bbf0 con 0x7ff87c0cac20 2024-01-11T07:07:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.754+0000 7ff873fff700 1 -- 172.21.15.7:0/1464057675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff874012430 con 0x7ff87c0cac20 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.755+0000 7ff88361a700 1 -- 172.21.15.7:0/1464057675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c0cac20 msgr2=0x7ff87c162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.755+0000 7ff88361a700 1 --2- 172.21.15.7:0/1464057675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c0cac20 0x7ff87c162ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff874009e60 tx=0x7ff874009780 comp rx=0 tx=0).stop 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.755+0000 7ff88361a700 1 -- 172.21.15.7:0/1464057675 shutdown_connections 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.755+0000 7ff88361a700 1 --2- 172.21.15.7:0/1464057675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c0cac20 0x7ff87c162ab0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.755+0000 7ff88361a700 1 -- 172.21.15.7:0/1464057675 >> 172.21.15.7:0/1464057675 conn(0x7ff87c070690 msgr2=0x7ff87c070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.755+0000 7ff88361a700 1 -- 172.21.15.7:0/1464057675 shutdown_connections 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.755+0000 7ff88361a700 1 -- 172.21.15.7:0/1464057675 wait complete. 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.756+0000 7ff88361a700 1 Processor -- start 2024-01-11T07:07:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.756+0000 7ff88361a700 1 -- start start 2024-01-11T07:07:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff88361a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87c0cac20 0x7ff87c0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff88361a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff87c15aee0 0x7ff87c0b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff88361a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c15b2b0 0x7ff87c0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff88361a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c156f10 con 0x7ff87c15b2b0 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff88361a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0c79c0 con 0x7ff87c0cac20 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff88361a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0b23a0 con 0x7ff87c15aee0 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff881bb7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c15b2b0 0x7ff87c0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff881bb7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c15b2b0 0x7ff87c0b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56764/0 (socket says 172.21.15.7:56764) 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff881bb7700 1 -- 172.21.15.7:0/1496483246 learned_addr learned my addr 172.21.15.7:0/1496483246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff880bb5700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff87c15aee0 0x7ff87c0b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.757+0000 7ff8813b6700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87c0cac20 0x7ff87c0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff881bb7700 1 -- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff87c15aee0 msgr2=0x7ff87c0b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff881bb7700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff87c15aee0 0x7ff87c0b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff881bb7700 1 -- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87c0cac20 msgr2=0x7ff87c0b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff881bb7700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87c0cac20 0x7ff87c0b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff881bb7700 1 -- 172.21.15.7:0/1496483246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff874009a20 con 0x7ff87c15b2b0 2024-01-11T07:07:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff881bb7700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c15b2b0 0x7ff87c0b1d70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff87c06dc90 tx=0x7ff874018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff8717fa700 1 -- 172.21.15.7:0/1496483246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87401e640 con 0x7ff87c15b2b0 2024-01-11T07:07:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff8717fa700 1 -- 172.21.15.7:0/1496483246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87401e7a0 con 0x7ff87c15b2b0 2024-01-11T07:07:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c067720 con 0x7ff87c15b2b0 2024-01-11T07:07:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff8717fa700 1 -- 172.21.15.7:0/1496483246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87401eb60 con 0x7ff87c15b2b0 2024-01-11T07:07:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.758+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c067ce0 con 0x7ff87c15b2b0 2024-01-11T07:07:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.759+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c04c3c0 con 0x7ff87c15b2b0 2024-01-11T07:07:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.760+0000 7ff8717fa700 1 -- 172.21.15.7:0/1496483246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff874024070 con 0x7ff87c15b2b0 2024-01-11T07:07:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.760+0000 7ff8717fa700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.760+0000 7ff8717fa700 1 -- 172.21.15.7:0/1496483246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ff874086f70 con 0x7ff87c15b2b0 2024-01-11T07:07:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.760+0000 7ff8813b6700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.761+0000 7ff8813b6700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff860066f30 0x7ff8600693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff86c003a70 tx=0x7ff86c007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:39.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.765+0000 7ff8717fa700 1 -- 172.21.15.7:0/1496483246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8740b85a0 con 0x7ff87c15b2b0 2024-01-11T07:07:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:39.969+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff87c06ca80 con 0x7ff87c15b2b0 2024-01-11T07:07:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.601+0000 7ff8717fa700 1 -- 172.21.15.7:0/1496483246 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff874057c20 con 0x7ff87c15b2b0 2024-01-11T07:07:40.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.604+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff860066f30 msgr2=0x7ff8600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:40.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.604+0000 7ff88361a700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff860066f30 0x7ff8600693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff86c003a70 tx=0x7ff86c007ef0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.604+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c15b2b0 msgr2=0x7ff87c0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.604+0000 7ff88361a700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c15b2b0 0x7ff87c0b1d70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff87c06dc90 tx=0x7ff874018000 comp rx=0 tx=0).stop 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.605+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 shutdown_connections 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.605+0000 7ff88361a700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff860066f30 0x7ff8600693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.605+0000 7ff88361a700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff87c15b2b0 0x7ff87c0b1d70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.605+0000 7ff88361a700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87c0cac20 0x7ff87c0b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.605+0000 7ff88361a700 1 --2- 172.21.15.7:0/1496483246 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff87c15aee0 0x7ff87c0b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.605+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 >> 172.21.15.7:0/1496483246 conn(0x7ff87c070690 msgr2=0x7ff87c156790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.605+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 shutdown_connections 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.605+0000 7ff88361a700 1 -- 172.21.15.7:0/1496483246 wait complete. 2024-01-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:07:40.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:07:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.761+0000 7f1e99fd5700 1 Processor -- start 2024-01-11T07:07:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.761+0000 7f1e99fd5700 1 -- start start 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e99fd5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e94157a40 0x7f1e9406ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e99fd5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e9406f390 0x7f1e940cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e99fd5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9406d750 con 0x7f1e94069540 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e99fd5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9406a6c0 con 0x7f1e94157a40 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e99fd5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e94069ab0 con 0x7f1e9406f390 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e94157a40 0x7f1e9406ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e94157a40 0x7f1e9406ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56770/0 (socket says 172.21.15.7:56770) 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e937fe700 1 -- 172.21.15.7:0/1636915057 learned_addr learned my addr 172.21.15.7:0/1636915057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e93fff700 1 --2- 172.21.15.7:0/1636915057 >> v2:172.21.15.113:3300/0 conn(0x7f1e9406f390 0x7f1e940cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e937fe700 1 -- 172.21.15.7:0/1636915057 >> v2:172.21.15.113:3300/0 conn(0x7f1e9406f390 msgr2=0x7f1e940cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e937fe700 1 --2- 172.21.15.7:0/1636915057 >> v2:172.21.15.113:3300/0 conn(0x7f1e9406f390 0x7f1e940cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e937fe700 1 -- 172.21.15.7:0/1636915057 >> v1:172.21.15.113:6789/0 conn(0x7f1e94069540 legacy=0x7f1e9406ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e937fe700 1 -- 172.21.15.7:0/1636915057 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e94069d90 con 0x7f1e94157a40 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.763+0000 7f1e937fe700 1 --2- 172.21.15.7:0/1636915057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e9406ab50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e94070a60 tx=0x7f1e84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286875d6227f3e9b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e927fc700 1 -- 172.21.15.7:0/1636915057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84009e70 con 0x7f1e94157a40 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e927fc700 1 -- 172.21.15.7:0/1636915057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8400b860 con 0x7f1e94157a40 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e927fc700 1 -- 172.21.15.7:0/1636915057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400bc00 con 0x7f1e94157a40 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e99fd5700 1 -- 172.21.15.7:0/1636915057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 msgr2=0x7f1e9406ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/1636915057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e9406ab50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1e94070a60 tx=0x7f1e84009250 comp rx=0 tx=0).stop 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e99fd5700 1 -- 172.21.15.7:0/1636915057 shutdown_connections 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/1636915057 >> v2:172.21.15.113:3300/0 conn(0x7f1e9406f390 0x7f1e940cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/1636915057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e9406ab50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e99fd5700 1 -- 172.21.15.7:0/1636915057 >> 172.21.15.7:0/1636915057 conn(0x7f1e940680b0 msgr2=0x7f1e940cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e99fd5700 1 -- 172.21.15.7:0/1636915057 shutdown_connections 2024-01-11T07:07:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.764+0000 7f1e99fd5700 1 -- 172.21.15.7:0/1636915057 wait complete. 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.766+0000 7f1e99fd5700 1 Processor -- start 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.766+0000 7f1e99fd5700 1 -- start start 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e99fd5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e94069540 0x7f1e940c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e99fd5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e9406f390 0x7f1e9414bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e99fd5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e940cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e99fd5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940c4c60 con 0x7f1e94157a40 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e99fd5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9414c0f0 con 0x7f1e94069540 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e99fd5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940cca70 con 0x7f1e9406f390 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e940cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e940cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56780/0 (socket says 172.21.15.7:56780) 2024-01-11T07:07:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 -- 172.21.15.7:0/3160906250 learned_addr learned my addr 172.21.15.7:0/3160906250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e937fe700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e94069540 0x7f1e940c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e92ffd700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e9406f390 0x7f1e9414bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 -- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e9406f390 msgr2=0x7f1e9414bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e9406f390 0x7f1e9414bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 -- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e94069540 msgr2=0x7f1e940c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e94069540 0x7f1e940c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 -- 172.21.15.7:0/3160906250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e840094f0 con 0x7f1e94157a40 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e93fff700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e940cac60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009ec0 tx=0x7f1e8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e8b7fe700 1 -- 172.21.15.7:0/3160906250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c00cc20 con 0x7f1e94157a40 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e8b7fe700 1 -- 172.21.15.7:0/3160906250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c00f040 con 0x7f1e94157a40 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e940cb7d0 con 0x7f1e94157a40 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.767+0000 7f1e8b7fe700 1 -- 172.21.15.7:0/3160906250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c014480 con 0x7f1e94157a40 2024-01-11T07:07:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.768+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e940c4610 con 0x7f1e94157a40 2024-01-11T07:07:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.769+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9404c3c0 con 0x7f1e94157a40 2024-01-11T07:07:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.769+0000 7f1e8b7fe700 1 -- 172.21.15.7:0/3160906250 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1e8c014830 con 0x7f1e94157a40 2024-01-11T07:07:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.769+0000 7f1e8b7fe700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e74066f30 0x7f1e740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.769+0000 7f1e8b7fe700 1 -- 172.21.15.7:0/3160906250 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f1e8c085110 con 0x7f1e94157a40 2024-01-11T07:07:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.770+0000 7f1e937fe700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e74066f30 0x7f1e740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:40.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.770+0000 7f1e937fe700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e74066f30 0x7f1e740693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e94070a60 tx=0x7f1e84017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.774+0000 7f1e8b7fe700 1 -- 172.21.15.7:0/3160906250 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e8c055f70 con 0x7f1e94157a40 2024-01-11T07:07:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:40.977+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1e9406a070 con 0x7f1e94157a40 2024-01-11T07:07:41.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.605+0000 7f1e8b7fe700 1 -- 172.21.15.7:0/3160906250 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1e8c055b00 con 0x7f1e94157a40 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e74066f30 msgr2=0x7f1e740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e74066f30 0x7f1e740693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e94070a60 tx=0x7f1e84017000 comp rx=0 tx=0).stop 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 msgr2=0x7f1e940cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e940cac60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009ec0 tx=0x7f1e8c00b040 comp rx=0 tx=0).stop 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 shutdown_connections 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1e74066f30 0x7f1e740693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e94157a40 0x7f1e940cac60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e94069540 0x7f1e940c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 --2- 172.21.15.7:0/3160906250 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e9406f390 0x7f1e9414bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 >> 172.21.15.7:0/3160906250 conn(0x7f1e940680b0 msgr2=0x7f1e940b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 shutdown_connections 2024-01-11T07:07:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.609+0000 7f1e99fd5700 1 -- 172.21.15.7:0/3160906250 wait complete. 2024-01-11T07:07:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:07:41.625 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:07:41.625 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:07:41.625 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:07:41.626 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:07:41.626 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:07:41.626 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:07:41.626 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:07:41.626 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:07:41.627 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:07:41.627 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:07:41.627 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:07:41.627 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:07:41.627 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:07:41.627 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:07:41.627 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:07:41.628 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:07:41.636 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:07:41.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:07:41.637 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:07:41.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:07:41.638 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:07:41.638 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:07:41.639 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:07:41.639 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:07:41.679 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:07:41.679 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:41.680 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:07:41.680 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:41.681 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:07:41.681 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:41.682 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:07:41.682 DEBUG:teuthology.parallel:result is None 2024-01-11T07:07:41.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:07:41.770 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:07:41.770 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:07:41.776 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:07:41.776 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:07:41.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.830+0000 7f02d6e94700 1 Processor -- start 2024-01-11T07:07:41.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.830+0000 7f02d6e94700 1 -- start start 2024-01-11T07:07:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02d6e94700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7b30 0x7f02d01537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02d6e94700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02d00c7f00 0x7f02d0153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02d6e94700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d00cbb20 con 0x7f02d00cafb0 2024-01-11T07:07:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02d6e94700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d0157880 con 0x7f02d00c7f00 2024-01-11T07:07:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02d6e94700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0069a20 con 0x7f02d00c7b30 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02d00c7f00 0x7f02d0153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02d5431700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f02d00cafb0 0x7f02d00cb410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39012/0 (socket says 172.21.15.7:39012) 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02d5431700 1 -- 172.21.15.7:0/4174632166 learned_addr learned my addr 172.21.15.7:0/4174632166 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02d4c30700 1 --2- 172.21.15.7:0/4174632166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7b30 0x7f02d01537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02cffff700 1 -- 172.21.15.7:0/4174632166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7b30 msgr2=0x7f02d01537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02cffff700 1 --2- 172.21.15.7:0/4174632166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7b30 0x7f02d01537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02cffff700 1 -- 172.21.15.7:0/4174632166 >> v1:172.21.15.7:6789/0 conn(0x7f02d00cafb0 legacy=0x7f02d00cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02cffff700 1 -- 172.21.15.7:0/4174632166 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d00cbd50 con 0x7f02d00c7f00 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.831+0000 7f02cf7fe700 1 -- 172.21.15.7:0/4174632166 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216598542 0 0) 0x7f02d00cbb20 con 0x7f02d00cafb0 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02cffff700 1 --2- 172.21.15.7:0/4174632166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00c7f00 0x7f02d0153cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098e0 tx=0x7f02bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ea57976091476e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02cf7fe700 1 -- 172.21.15.7:0/4174632166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00b640 con 0x7f02d00c7f00 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02cf7fe700 1 -- 172.21.15.7:0/4174632166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc00b7a0 con 0x7f02d00c7f00 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02cf7fe700 1 -- 172.21.15.7:0/4174632166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00bb40 con 0x7f02d00c7f00 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02d6e94700 1 -- 172.21.15.7:0/4174632166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00c7f00 msgr2=0x7f02d0153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02d6e94700 1 --2- 172.21.15.7:0/4174632166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00c7f00 0x7f02d0153cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098e0 tx=0x7f02bc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02d6e94700 1 -- 172.21.15.7:0/4174632166 shutdown_connections 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02d6e94700 1 --2- 172.21.15.7:0/4174632166 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7b30 0x7f02d01537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02d6e94700 1 --2- 172.21.15.7:0/4174632166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00c7f00 0x7f02d0153cf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02d6e94700 1 -- 172.21.15.7:0/4174632166 >> 172.21.15.7:0/4174632166 conn(0x7f02d00cd090 msgr2=0x7f02d00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02d6e94700 1 -- 172.21.15.7:0/4174632166 shutdown_connections 2024-01-11T07:07:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.832+0000 7f02d6e94700 1 -- 172.21.15.7:0/4174632166 wait complete. 2024-01-11T07:07:41.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.834+0000 7f02d6e94700 1 Processor -- start 2024-01-11T07:07:41.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.834+0000 7f02d6e94700 1 -- start start 2024-01-11T07:07:41.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d6e94700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d00c7b30 0x7f02d0171750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:41.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d6e94700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7f00 0x7f02d016ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:41.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d6e94700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00cafb0 0x7f02d0151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:41.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d6e94700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d01522c0 con 0x7f02d00cafb0 2024-01-11T07:07:41.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d6e94700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0171de0 con 0x7f02d00c7b30 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d6e94700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d01720c0 con 0x7f02d00c7f00 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00cafb0 0x7f02d0151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00cafb0 0x7f02d0151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56838/0 (socket says 172.21.15.7:56838) 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 -- 172.21.15.7:0/1126400038 learned_addr learned my addr 172.21.15.7:0/1126400038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02cffff700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7f00 0x7f02d016ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d4c30700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d00c7b30 0x7f02d0171750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 -- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7f00 msgr2=0x7f02d016ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7f00 0x7f02d016ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 -- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d00c7b30 msgr2=0x7f02d0171750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d00c7b30 0x7f02d0171750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 -- 172.21.15.7:0/1126400038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc009420 con 0x7f02d00cafb0 2024-01-11T07:07:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02d5431700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00cafb0 0x7f02d0151d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f02c800cbf0 tx=0x7f02c80019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.835+0000 7f02ccff9700 1 -- 172.21.15.7:0/1126400038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c800a8b0 con 0x7f02d00cafb0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.836+0000 7f02ccff9700 1 -- 172.21.15.7:0/1126400038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c800aa10 con 0x7f02d00cafb0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.836+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d01723a0 con 0x7f02d00cafb0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.836+0000 7f02ccff9700 1 -- 172.21.15.7:0/1126400038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c800add0 con 0x7f02d00cafb0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.836+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d0147a10 con 0x7f02d00cafb0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.836+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d014bd60 con 0x7f02d00cafb0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.837+0000 7f02ccff9700 1 -- 172.21.15.7:0/1126400038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f02c8007300 con 0x7f02d00cafb0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.837+0000 7f02ccff9700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b4066f30 0x7f02b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.838+0000 7f02ccff9700 1 -- 172.21.15.7:0/1126400038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f02c80844a0 con 0x7f02d00cafb0 2024-01-11T07:07:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.838+0000 7f02d4c30700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b4066f30 0x7f02b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:41.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.838+0000 7f02d4c30700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b4066f30 0x7f02b40693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009db0 tx=0x7f02c40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:41.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:41.842+0000 7f02ccff9700 1 -- 172.21.15.7:0/1126400038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02d014bd60 con 0x7f02d00cafb0 2024-01-11T07:07:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:42.060+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f02d014bd60 con 0x7f02d00cafb0 2024-01-11T07:07:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.182+0000 7f02ccff9700 1 -- 172.21.15.7:0/1126400038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f02d014bd60 con 0x7f02d00cafb0 2024-01-11T07:07:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b4066f30 msgr2=0x7f02b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b4066f30 0x7f02b40693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009db0 tx=0x7f02c40091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00cafb0 msgr2=0x7f02d0151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00cafb0 0x7f02d0151d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f02c800cbf0 tx=0x7f02c80019b0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 shutdown_connections 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02b4066f30 0x7f02b40693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d00cafb0 0x7f02d0151d80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d00c7b30 0x7f02d0171750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 --2- 172.21.15.7:0/1126400038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02d00c7f00 0x7f02d016ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 >> 172.21.15.7:0/1126400038 conn(0x7f02d00cd090 msgr2=0x7f02d00ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 shutdown_connections 2024-01-11T07:07:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.185+0000 7f02d6e94700 1 -- 172.21.15.7:0/1126400038 wait complete. 2024-01-11T07:07:43.186 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:07:43.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:07:43.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.335+0000 7fc1cf29c700 1 Processor -- start 2024-01-11T07:07:43.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.336+0000 7fc1cf29c700 1 -- start start 2024-01-11T07:07:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cf29c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80cf530 0x7fc1c80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cf29c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c8161b90 con 0x7fc1c80b0430 2024-01-11T07:07:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cf29c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c80b36c0 con 0x7fc1c81617c0 2024-01-11T07:07:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cf29c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80b1d60 con 0x7fc1c80cf530 2024-01-11T07:07:43.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cd839700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc1c80b0430 0x7fc1c8067610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39016/0 (socket says 172.21.15.7:39016) 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cd839700 1 -- 172.21.15.7:0/4097101675 learned_addr learned my addr 172.21.15.7:0/4097101675 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cc837700 1 --2- 172.21.15.7:0/4097101675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80cf530 0x7fc1c80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1bbfff700 1 -- 172.21.15.7:0/4097101675 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487447355 0 0) 0x7fc1c8161b90 con 0x7fc1c80b0430 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1bbfff700 1 -- 172.21.15.7:0/4097101675 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b4003570 con 0x7fc1c80b0430 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1bbfff700 1 -- 172.21.15.7:0/4097101675 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623706291 0 0) 0x7fc1c80b36c0 con 0x7fc1c81617c0 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1bbfff700 1 -- 172.21.15.7:0/4097101675 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c8161b90 con 0x7fc1c81617c0 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cc837700 1 -- 172.21.15.7:0/4097101675 >> v1:172.21.15.113:6789/0 conn(0x7fc1c81617c0 legacy=0x7fc1c80b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cc837700 1 -- 172.21.15.7:0/4097101675 >> v1:172.21.15.7:6789/0 conn(0x7fc1c80b0430 legacy=0x7fc1c8067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.337+0000 7fc1cc837700 1 -- 172.21.15.7:0/4097101675 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80cf980 con 0x7fc1c80cf530 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1cc837700 1 --2- 172.21.15.7:0/4097101675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80cf530 0x7fc1c80b1820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009940 tx=0x7fc1bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a46a98fa133cb5c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1bbfff700 1 -- 172.21.15.7:0/4097101675 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc011630 con 0x7fc1c80cf530 2024-01-11T07:07:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1bbfff700 1 -- 172.21.15.7:0/4097101675 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc011790 con 0x7fc1c80cf530 2024-01-11T07:07:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1bbfff700 1 -- 172.21.15.7:0/4097101675 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc011b30 con 0x7fc1c80cf530 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4097101675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80cf530 msgr2=0x7fc1c80b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1cf29c700 1 --2- 172.21.15.7:0/4097101675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80cf530 0x7fc1c80b1820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009940 tx=0x7fc1bc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4097101675 shutdown_connections 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1cf29c700 1 --2- 172.21.15.7:0/4097101675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80cf530 0x7fc1c80b1820 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.338+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4097101675 >> 172.21.15.7:0/4097101675 conn(0x7fc1c8071100 msgr2=0x7fc1c806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.339+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4097101675 shutdown_connections 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.339+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4097101675 wait complete. 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.340+0000 7fc1cf29c700 1 Processor -- start 2024-01-11T07:07:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.340+0000 7fc1cf29c700 1 -- start start 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cf29c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80b0430 0x7fc1c8149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cf29c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c80cf530 0x7fc1c806edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cf29c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c81617c0 0x7fc1c806f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cf29c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c816fb70 con 0x7fc1c81617c0 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cf29c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80acf60 con 0x7fc1c80cf530 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cf29c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80ad240 con 0x7fc1c80b0430 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd839700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c81617c0 0x7fc1c806f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd839700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c81617c0 0x7fc1c806f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56852/0 (socket says 172.21.15.7:56852) 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd839700 1 -- 172.21.15.7:0/4112319639 learned_addr learned my addr 172.21.15.7:0/4112319639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd038700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80b0430 0x7fc1c8149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cc837700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c80cf530 0x7fc1c806edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd839700 1 -- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80b0430 msgr2=0x7fc1c8149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd839700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80b0430 0x7fc1c8149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd839700 1 -- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c80cf530 msgr2=0x7fc1c806edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd839700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c80cf530 0x7fc1c806edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.341+0000 7fc1cd839700 1 -- 172.21.15.7:0/4112319639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc009420 con 0x7fc1c81617c0 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.342+0000 7fc1cd839700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c81617c0 0x7fc1c806f300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000cbc0 tx=0x7fc1c0001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.342+0000 7fc1b97fa700 1 -- 172.21.15.7:0/4112319639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000a7e0 con 0x7fc1c81617c0 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.342+0000 7fc1b97fa700 1 -- 172.21.15.7:0/4112319639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c000a940 con 0x7fc1c81617c0 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.342+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c816f520 con 0x7fc1c81617c0 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.342+0000 7fc1b97fa700 1 -- 172.21.15.7:0/4112319639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000ad00 con 0x7fc1c81617c0 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.342+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c8149b80 con 0x7fc1c81617c0 2024-01-11T07:07:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.343+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c80af8c0 con 0x7fc1c81617c0 2024-01-11T07:07:43.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.343+0000 7fc1b97fa700 1 -- 172.21.15.7:0/4112319639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc1c0007300 con 0x7fc1c81617c0 2024-01-11T07:07:43.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.344+0000 7fc1b97fa700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1ac066f30 0x7fc1ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.344+0000 7fc1b97fa700 1 -- 172.21.15.7:0/4112319639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc1c00844c0 con 0x7fc1c81617c0 2024-01-11T07:07:43.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.344+0000 7fc1cd038700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1ac066f30 0x7fc1ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.344+0000 7fc1cd038700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1ac066f30 0x7fc1ac0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4007fd0 tx=0x7fc1c4007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:43.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.348+0000 7fc1b97fa700 1 -- 172.21.15.7:0/4112319639 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1c0051b00 con 0x7fc1c81617c0 2024-01-11T07:07:43.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.633+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc1c804c3c0 con 0x7fc1c81617c0 2024-01-11T07:07:43.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.633+0000 7fc1b97fa700 1 -- 172.21.15.7:0/4112319639 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc1c000ae60 con 0x7fc1c81617c0 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:07:43.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.635 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:07:43.636 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.637 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:07:43.638 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.639 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.640 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.642 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.643 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:07:43.644 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mds", 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCCk59lpxP1EhAAG1HjTNEjWPpku1AKVi4rXw==", 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mds" 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mgr", 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCCk59lSCP1EhAACNLjMxCpN37l9ELOdXhxOw==", 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-osd", 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCCk59lqCz1EhAAJsV0Mscr2Il6y2eyjDlD0A==", 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-osd" 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd", 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCCk59lxzX1EhAA01bNWkwyx8yT7yjfMfmYlA==", 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCCk59luT71EhAATmITtnz1UilnU1Ys9uLPMg==", 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rgw", 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCCk59lXkj1EhAAbusQ71Ox7eRLxAKQZVT+jA==", 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:07:43.649 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:07:43.650 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:07:43.650 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:07:43.650 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:07:43.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:43.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:07:43.650 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.635+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1ac066f30 msgr2=0x7fc1ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.635+0000 7fc1cf29c700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1ac066f30 0x7fc1ac0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4007fd0 tx=0x7fc1c4007d50 comp rx=0 tx=0).stop 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.635+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c81617c0 msgr2=0x7fc1c806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.635+0000 7fc1cf29c700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c81617c0 0x7fc1c806f300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000cbc0 tx=0x7fc1c0001a20 comp rx=0 tx=0).stop 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.635+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 shutdown_connections 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.635+0000 7fc1cf29c700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1ac066f30 0x7fc1ac0693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.636+0000 7fc1cf29c700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c81617c0 0x7fc1c806f300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.636+0000 7fc1cf29c700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1c80b0430 0x7fc1c8149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.636+0000 7fc1cf29c700 1 --2- 172.21.15.7:0/4112319639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c80cf530 0x7fc1c806edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.636+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 >> 172.21.15.7:0/4112319639 conn(0x7fc1c8071100 msgr2=0x7fc1c80af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:43.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.636+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 shutdown_connections 2024-01-11T07:07:43.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.636+0000 7fc1cf29c700 1 -- 172.21.15.7:0/4112319639 wait complete. 2024-01-11T07:07:43.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-11T07:07:43.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.786+0000 7f457fcc3700 1 Processor -- start 2024-01-11T07:07:43.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.786+0000 7f457fcc3700 1 -- start start 2024-01-11T07:07:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.787+0000 7f457fcc3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f4578164c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.787+0000 7f457fcc3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45780cae90 con 0x7f4578159fd0 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.787+0000 7f457fcc3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457806bb90 con 0x7f4578159c00 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.787+0000 7f457fcc3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780ca840 con 0x7f4578069320 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.787+0000 7f457d25e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4578159fd0 0x7f4578068c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39018/0 (socket says 172.21.15.7:39018) 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.787+0000 7f457d25e700 1 -- 172.21.15.7:0/687688844 learned_addr learned my addr 172.21.15.7:0/687688844 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.787+0000 7f457e260700 1 --2- 172.21.15.7:0/687688844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f4578164c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457ca5d700 1 -- 172.21.15.7:0/687688844 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951049257 0 0) 0x7f45780cae90 con 0x7f4578159fd0 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457ca5d700 1 -- 172.21.15.7:0/687688844 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4564003570 con 0x7f4578159fd0 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457ca5d700 1 -- 172.21.15.7:0/687688844 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961334344 0 0) 0x7f457806bb90 con 0x7f4578159c00 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457ca5d700 1 -- 172.21.15.7:0/687688844 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45780cae90 con 0x7f4578159c00 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457e260700 1 -- 172.21.15.7:0/687688844 >> v1:172.21.15.113:6789/0 conn(0x7f4578159c00 legacy=0x7f457806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457e260700 1 -- 172.21.15.7:0/687688844 >> v1:172.21.15.7:6789/0 conn(0x7f4578159fd0 legacy=0x7f4578068c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457e260700 1 -- 172.21.15.7:0/687688844 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780cab20 con 0x7f4578069320 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457e260700 1 --2- 172.21.15.7:0/687688844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f4578164c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4570009e90 tx=0x7f4570009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5759a54b1f8088e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457ca5d700 1 -- 172.21.15.7:0/687688844 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457000e070 con 0x7f4578069320 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457ca5d700 1 -- 172.21.15.7:0/687688844 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457000bb80 con 0x7f4578069320 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.788+0000 7f457ca5d700 1 -- 172.21.15.7:0/687688844 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570012430 con 0x7f4578069320 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.789+0000 7f457fcc3700 1 -- 172.21.15.7:0/687688844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 msgr2=0x7f4578164c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.789+0000 7f457fcc3700 1 --2- 172.21.15.7:0/687688844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f4578164c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4570009e90 tx=0x7f4570009780 comp rx=0 tx=0).stop 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.789+0000 7f457fcc3700 1 -- 172.21.15.7:0/687688844 shutdown_connections 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.789+0000 7f457fcc3700 1 --2- 172.21.15.7:0/687688844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f4578164c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.789+0000 7f457fcc3700 1 -- 172.21.15.7:0/687688844 >> 172.21.15.7:0/687688844 conn(0x7f4578063230 msgr2=0x7f4578063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.789+0000 7f457fcc3700 1 -- 172.21.15.7:0/687688844 shutdown_connections 2024-01-11T07:07:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.789+0000 7f457fcc3700 1 -- 172.21.15.7:0/687688844 wait complete. 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.790+0000 7f457fcc3700 1 Processor -- start 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.790+0000 7f457fcc3700 1 -- start start 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457fcc3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f45781571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457fcc3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4578159c00 0x7f45780b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457fcc3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4578159fd0 0x7f4578157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457fcc3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780b39b0 con 0x7f4578159c00 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457fcc3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780b2da0 con 0x7f4578159fd0 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457fcc3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780b3080 con 0x7f4578069320 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457d25e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4578159c00 0x7f45780b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457d25e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4578159c00 0x7f45780b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56862/0 (socket says 172.21.15.7:56862) 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457d25e700 1 -- 172.21.15.7:0/539689896 learned_addr learned my addr 172.21.15.7:0/539689896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457e260700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4578159fd0 0x7f4578157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457da5f700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f45781571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457d25e700 1 -- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 msgr2=0x7f45781571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457d25e700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f45781571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457d25e700 1 -- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4578159fd0 msgr2=0x7f4578157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457d25e700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4578159fd0 0x7f4578157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.791+0000 7f457d25e700 1 -- 172.21.15.7:0/539689896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570009a20 con 0x7f4578159c00 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.792+0000 7f457d25e700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4578159c00 0x7f45780b3e40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f456800c580 tx=0x7f4568007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.792+0000 7f456dffb700 1 -- 172.21.15.7:0/539689896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568005b30 con 0x7f4578159c00 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.792+0000 7f456dffb700 1 -- 172.21.15.7:0/539689896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4568005c90 con 0x7f4578159c00 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.792+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45780b3360 con 0x7f4578159c00 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.792+0000 7f456dffb700 1 -- 172.21.15.7:0/539689896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568007320 con 0x7f4578159c00 2024-01-11T07:07:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.792+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45781576e0 con 0x7f4578159c00 2024-01-11T07:07:43.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.793+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45780b3b20 con 0x7f4578159c00 2024-01-11T07:07:43.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.794+0000 7f456dffb700 1 -- 172.21.15.7:0/539689896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f456801b3b0 con 0x7f4578159c00 2024-01-11T07:07:43.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.794+0000 7f456dffb700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f455c066f30 0x7f455c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:43.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.794+0000 7f456dffb700 1 -- 172.21.15.7:0/539689896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4568083640 con 0x7f4578159c00 2024-01-11T07:07:43.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.794+0000 7f457da5f700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f455c066f30 0x7f455c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:43.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.795+0000 7f457da5f700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f455c066f30 0x7f455c0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4574007fd0 tx=0x7f4574007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:43.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:43.799+0000 7f456dffb700 1 -- 172.21.15.7:0/539689896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4568050c00 con 0x7f4578159c00 2024-01-11T07:07:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.085+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f457804c3c0 con 0x7f4578159c00 2024-01-11T07:07:44.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.089+0000 7f456dffb700 1 -- 172.21.15.7:0/539689896 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f45680170a0 con 0x7f4578159c00 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f455c066f30 msgr2=0x7f455c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f455c066f30 0x7f455c0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4574007fd0 tx=0x7f4574007d50 comp rx=0 tx=0).stop 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4578159c00 msgr2=0x7f45780b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4578159c00 0x7f45780b3e40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f456800c580 tx=0x7f4568007e40 comp rx=0 tx=0).stop 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 shutdown_connections 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f455c066f30 0x7f455c0693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4578159fd0 0x7f4578157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4578069320 0x7f45781571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 --2- 172.21.15.7:0/539689896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4578159c00 0x7f45780b3e40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 >> 172.21.15.7:0/539689896 conn(0x7f4578063230 msgr2=0x7f45780b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 shutdown_connections 2024-01-11T07:07:44.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.091+0000 7f457fcc3700 1 -- 172.21.15.7:0/539689896 wait complete. 2024-01-11T07:07:44.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-11T07:07:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.241+0000 7f91f1d7d700 1 Processor -- start 2024-01-11T07:07:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.242+0000 7f91f1d7d700 1 -- start start 2024-01-11T07:07:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.242+0000 7f91f1d7d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f91ec159c90 0x7f91ec0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.242+0000 7f91f1d7d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91ec15a060 0x7f91ec0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.242+0000 7f91f1d7d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ec0ce9b0 con 0x7f91ec069320 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.242+0000 7f91f1d7d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ec0cf530 con 0x7f91ec15a060 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.242+0000 7f91f1d7d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ec0cf9c0 con 0x7f91ec159c90 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91ec15a060 0x7f91ec0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91ec15a060 0x7f91ec0b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56872/0 (socket says 172.21.15.7:56872) 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eaffd700 1 -- 172.21.15.7:0/826711808 learned_addr learned my addr 172.21.15.7:0/826711808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91ebfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91ec069320 0x7f91ec067610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39034/0 (socket says 172.21.15.7:39034) 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eb7fe700 1 --2- 172.21.15.7:0/826711808 >> v2:172.21.15.113:3300/0 conn(0x7f91ec159c90 0x7f91ec0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eaffd700 1 -- 172.21.15.7:0/826711808 >> v2:172.21.15.113:3300/0 conn(0x7f91ec159c90 msgr2=0x7f91ec0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eaffd700 1 --2- 172.21.15.7:0/826711808 >> v2:172.21.15.113:3300/0 conn(0x7f91ec159c90 0x7f91ec0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eaffd700 1 -- 172.21.15.7:0/826711808 >> v1:172.21.15.7:6789/0 conn(0x7f91ec069320 legacy=0x7f91ec067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eaffd700 1 -- 172.21.15.7:0/826711808 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec0cec00 con 0x7f91ec15a060 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91ea7fc700 1 -- 172.21.15.7:0/826711808 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626709391 0 0) 0x7f91ec0ce9b0 con 0x7f91ec069320 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91eaffd700 1 --2- 172.21.15.7:0/826711808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec0b0430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f91ec05f2e0 tx=0x7f91e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef15578f080015e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:44.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91ea7fc700 1 -- 172.21.15.7:0/826711808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0009e70 con 0x7f91ec15a060 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91ea7fc700 1 -- 172.21.15.7:0/826711808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e000b860 con 0x7f91ec15a060 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.243+0000 7f91ea7fc700 1 -- 172.21.15.7:0/826711808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000bc00 con 0x7f91ec15a060 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.244+0000 7f91f1d7d700 1 -- 172.21.15.7:0/826711808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 msgr2=0x7f91ec0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.244+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/826711808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec0b0430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f91ec05f2e0 tx=0x7f91e0009250 comp rx=0 tx=0).stop 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.244+0000 7f91f1d7d700 1 -- 172.21.15.7:0/826711808 shutdown_connections 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.244+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/826711808 >> v2:172.21.15.113:3300/0 conn(0x7f91ec159c90 0x7f91ec0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.244+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/826711808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec0b0430 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.244+0000 7f91f1d7d700 1 -- 172.21.15.7:0/826711808 >> 172.21.15.7:0/826711808 conn(0x7f91ec071100 msgr2=0x7f91ec06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.244+0000 7f91f1d7d700 1 -- 172.21.15.7:0/826711808 shutdown_connections 2024-01-11T07:07:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.244+0000 7f91f1d7d700 1 -- 172.21.15.7:0/826711808 wait complete. 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.245+0000 7f91f1d7d700 1 Processor -- start 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.245+0000 7f91f1d7d700 1 -- start start 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91f1d7d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec069320 0x7f91ec1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91f1d7d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91ec159c90 0x7f91ec0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91f1d7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91f1d7d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0b0740 con 0x7f91ec15a060 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91f1d7d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0c8240 con 0x7f91ec069320 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91f1d7d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec154e30 con 0x7f91ec159c90 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec15bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56878/0 (socket says 172.21.15.7:56878) 2024-01-11T07:07:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.246+0000 7f91ebfff700 1 -- 172.21.15.7:0/3248383656 learned_addr learned my addr 172.21.15.7:0/3248383656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91eaffd700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91ec159c90 0x7f91ec0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91eb7fe700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec069320 0x7f91ec1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91ebfff700 1 -- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91ec159c90 msgr2=0x7f91ec0af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91ebfff700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91ec159c90 0x7f91ec0af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91ebfff700 1 -- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec069320 msgr2=0x7f91ec1548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91ebfff700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec069320 0x7f91ec1548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91ebfff700 1 -- 172.21.15.7:0/3248383656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00094f0 con 0x7f91ec15a060 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91ebfff700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec15bc60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f91e400cd50 tx=0x7f91e4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91d37fe700 1 -- 172.21.15.7:0/3248383656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4005df0 con 0x7f91ec15a060 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91d37fe700 1 -- 172.21.15.7:0/3248383656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e400a8b0 con 0x7f91ec15a060 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec0c7c10 con 0x7f91ec15a060 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91d37fe700 1 -- 172.21.15.7:0/3248383656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e400ac70 con 0x7f91ec15a060 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.247+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec0aed60 con 0x7f91ec15a060 2024-01-11T07:07:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.248+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ec04c3c0 con 0x7f91ec15a060 2024-01-11T07:07:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.249+0000 7f91d37fe700 1 -- 172.21.15.7:0/3248383656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f91e4007300 con 0x7f91ec15a060 2024-01-11T07:07:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.249+0000 7f91d37fe700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.249+0000 7f91d37fe700 1 -- 172.21.15.7:0/3248383656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f91e4084310 con 0x7f91ec15a060 2024-01-11T07:07:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.250+0000 7f91eb7fe700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.250+0000 7f91eb7fe700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91d8066f30 0x7f91d80693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009db0 tx=0x7f91dc0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:44.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.254+0000 7f91d37fe700 1 -- 172.21.15.7:0/3248383656 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91e4051950 con 0x7f91ec15a060 2024-01-11T07:07:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.541+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f91ec0b48c0 con 0x7f91ec15a060 2024-01-11T07:07:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.544+0000 7f91d37fe700 1 -- 172.21.15.7:0/3248383656 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f91e4054f70 con 0x7f91ec15a060 2024-01-11T07:07:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91d8066f30 msgr2=0x7f91d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91d8066f30 0x7f91d80693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009db0 tx=0x7f91dc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 msgr2=0x7f91ec15bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec15bc60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f91e400cd50 tx=0x7f91e4001b10 comp rx=0 tx=0).stop 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 shutdown_connections 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91d8066f30 0x7f91d80693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ec15a060 0x7f91ec15bc60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91ec069320 0x7f91ec1548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 --2- 172.21.15.7:0/3248383656 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f91ec159c90 0x7f91ec0af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 >> 172.21.15.7:0/3248383656 conn(0x7f91ec071100 msgr2=0x7f91ec1566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 shutdown_connections 2024-01-11T07:07:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.547+0000 7f91f1d7d700 1 -- 172.21.15.7:0/3248383656 wait complete. 2024-01-11T07:07:44.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-11T07:07:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.693+0000 7fc243324700 1 Processor -- start 2024-01-11T07:07:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.693+0000 7fc243324700 1 -- start start 2024-01-11T07:07:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc243324700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc23c06dd00 0x7fc23c0c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc243324700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c15a1c0 0x7fc23c069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc243324700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23c15a640 con 0x7fc23c06e0d0 2024-01-11T07:07:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc243324700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23c15b770 con 0x7fc23c06dd00 2024-01-11T07:07:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc243324700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c15abf0 con 0x7fc23c15a1c0 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2410c0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc23c06dd00 0x7fc23c0c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2408bf700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc23c06e0d0 0x7fc23c159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39038/0 (socket says 172.21.15.7:39038) 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2408bf700 1 -- 172.21.15.7:0/514385181 learned_addr learned my addr 172.21.15.7:0/514385181 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2418c1700 1 --2- 172.21.15.7:0/514385181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c15a1c0 0x7fc23c069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2410c0700 1 -- 172.21.15.7:0/514385181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c15a1c0 msgr2=0x7fc23c069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2410c0700 1 --2- 172.21.15.7:0/514385181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c15a1c0 0x7fc23c069fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2410c0700 1 -- 172.21.15.7:0/514385181 >> v1:172.21.15.7:6789/0 conn(0x7fc23c06e0d0 legacy=0x7fc23c159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2410c0700 1 -- 172.21.15.7:0/514385181 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c15aed0 con 0x7fc23c06dd00 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc233fff700 1 -- 172.21.15.7:0/514385181 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848432991 0 0) 0x7fc22c001110 con 0x7fc23c06e0d0 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2418c1700 1 --2- 172.21.15.7:0/514385181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c15a1c0 0x7fc23c069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.694+0000 7fc2410c0700 1 --2- 172.21.15.7:0/514385181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c06dd00 0x7fc23c0c98d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc2380098e0 tx=0x7fc2380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58941ecd6695b31 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc233fff700 1 -- 172.21.15.7:0/514385181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238009e80 con 0x7fc23c06dd00 2024-01-11T07:07:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc233fff700 1 -- 172.21.15.7:0/514385181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800b860 con 0x7fc23c06dd00 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc233fff700 1 -- 172.21.15.7:0/514385181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800bbe0 con 0x7fc23c06dd00 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc243324700 1 -- 172.21.15.7:0/514385181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c06dd00 msgr2=0x7fc23c0c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc243324700 1 --2- 172.21.15.7:0/514385181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c06dd00 0x7fc23c0c98d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc2380098e0 tx=0x7fc2380091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc243324700 1 -- 172.21.15.7:0/514385181 shutdown_connections 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc243324700 1 --2- 172.21.15.7:0/514385181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c15a1c0 0x7fc23c069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc243324700 1 --2- 172.21.15.7:0/514385181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c06dd00 0x7fc23c0c98d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc243324700 1 -- 172.21.15.7:0/514385181 >> 172.21.15.7:0/514385181 conn(0x7fc23c0680e0 msgr2=0x7fc23c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc243324700 1 -- 172.21.15.7:0/514385181 shutdown_connections 2024-01-11T07:07:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.695+0000 7fc243324700 1 -- 172.21.15.7:0/514385181 wait complete. 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.696+0000 7fc243324700 1 Processor -- start 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.697+0000 7fc243324700 1 -- start start 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.697+0000 7fc243324700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c06dd00 0x7fc23c0c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.697+0000 7fc243324700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c06e0d0 0x7fc23c0aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.697+0000 7fc243324700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15a1c0 0x7fc23c0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.697+0000 7fc243324700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c154830 con 0x7fc23c15a1c0 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.697+0000 7fc243324700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c0abbe0 con 0x7fc23c06e0d0 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.697+0000 7fc243324700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c0aafd0 con 0x7fc23c06dd00 2024-01-11T07:07:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15a1c0 0x7fc23c0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15a1c0 0x7fc23c0c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56882/0 (socket says 172.21.15.7:56882) 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 -- 172.21.15.7:0/3530095631 learned_addr learned my addr 172.21.15.7:0/3530095631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2410c0700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c06dd00 0x7fc23c0c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2408bf700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c06e0d0 0x7fc23c0aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 -- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c06dd00 msgr2=0x7fc23c0c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c06dd00 0x7fc23c0c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 -- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c06e0d0 msgr2=0x7fc23c0aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c06e0d0 0x7fc23c0aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 -- 172.21.15.7:0/3530095631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc238009420 con 0x7fc23c15a1c0 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2408bf700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c06e0d0 0x7fc23c0aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2410c0700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c06dd00 0x7fc23c0c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2418c1700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15a1c0 0x7fc23c0c8080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc23400d100 tx=0x7fc2340099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2317fa700 1 -- 172.21.15.7:0/3530095631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23400dda0 con 0x7fc23c15a1c0 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2317fa700 1 -- 172.21.15.7:0/3530095631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23400f040 con 0x7fc23c15a1c0 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc2317fa700 1 -- 172.21.15.7:0/3530095631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc234015690 con 0x7fc23c15a1c0 2024-01-11T07:07:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c0ab2b0 con 0x7fc23c15a1c0 2024-01-11T07:07:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.698+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c0ab870 con 0x7fc23c15a1c0 2024-01-11T07:07:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.699+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23c15a7e0 con 0x7fc23c15a1c0 2024-01-11T07:07:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.700+0000 7fc2317fa700 1 -- 172.21.15.7:0/3530095631 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc23401e3b0 con 0x7fc23c15a1c0 2024-01-11T07:07:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.700+0000 7fc2317fa700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc220066f30 0x7fc2200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.700+0000 7fc2317fa700 1 -- 172.21.15.7:0/3530095631 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc234084770 con 0x7fc23c15a1c0 2024-01-11T07:07:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.700+0000 7fc2410c0700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc220066f30 0x7fc2200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.701+0000 7fc2410c0700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc220066f30 0x7fc2200693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2380091e0 tx=0x7fc238017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.705+0000 7fc2317fa700 1 -- 172.21.15.7:0/3530095631 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc234051d30 con 0x7fc23c15a1c0 2024-01-11T07:07:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.995+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fc23c04c3c0 con 0x7fc23c15a1c0 2024-01-11T07:07:44.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:44.999+0000 7fc2317fa700 1 -- 172.21.15.7:0/3530095631 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc23401a0a0 con 0x7fc23c15a1c0 2024-01-11T07:07:45.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc220066f30 msgr2=0x7fc2200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc220066f30 0x7fc2200693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2380091e0 tx=0x7fc238017000 comp rx=0 tx=0).stop 2024-01-11T07:07:45.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15a1c0 msgr2=0x7fc23c0c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15a1c0 0x7fc23c0c8080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc23400d100 tx=0x7fc2340099b0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 shutdown_connections 2024-01-11T07:07:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc220066f30 0x7fc2200693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15a1c0 0x7fc23c0c8080 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc23c06dd00 0x7fc23c0c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.001+0000 7fc243324700 1 --2- 172.21.15.7:0/3530095631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc23c06e0d0 0x7fc23c0aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.002+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 >> 172.21.15.7:0/3530095631 conn(0x7fc23c0680e0 msgr2=0x7fc23c0cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.002+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 shutdown_connections 2024-01-11T07:07:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.002+0000 7fc243324700 1 -- 172.21.15.7:0/3530095631 wait complete. 2024-01-11T07:07:45.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-11T07:07:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.147+0000 7fcee56d7700 1 Processor -- start 2024-01-11T07:07:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.147+0000 7fcee56d7700 1 -- start start 2024-01-11T07:07:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcee56d7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcee006e340 0x7fcee00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcee56d7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcee015a070 0x7fcee00ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcee56d7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee00d18c0 con 0x7fcee0159ca0 2024-01-11T07:07:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcee56d7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00d0010 con 0x7fcee015a070 2024-01-11T07:07:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcee56d7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00cf400 con 0x7fcee006e340 2024-01-11T07:07:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcedf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcee015a070 0x7fcee00ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcedeffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcee0159ca0 0x7fcee006dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39052/0 (socket says 172.21.15.7:39052) 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcedeffd700 1 -- 172.21.15.7:0/2118933080 learned_addr learned my addr 172.21.15.7:0/2118933080 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcede7fc700 1 --2- 172.21.15.7:0/2118933080 >> v2:172.21.15.113:3300/0 conn(0x7fcee006e340 0x7fcee00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcedf7fe700 1 -- 172.21.15.7:0/2118933080 >> v2:172.21.15.113:3300/0 conn(0x7fcee006e340 msgr2=0x7fcee00ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcedf7fe700 1 --2- 172.21.15.7:0/2118933080 >> v2:172.21.15.113:3300/0 conn(0x7fcee006e340 0x7fcee00ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcedf7fe700 1 -- 172.21.15.7:0/2118933080 >> v1:172.21.15.7:6789/0 conn(0x7fcee0159ca0 legacy=0x7fcee006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.148+0000 7fcedf7fe700 1 -- 172.21.15.7:0/2118933080 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee00cf6e0 con 0x7fcee015a070 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fcedf7fe700 1 --2- 172.21.15.7:0/2118933080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee015a070 0x7fcee00ced30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fced8009e60 tx=0x7fced8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e551b6951ac0f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fceddffb700 1 -- 172.21.15.7:0/2118933080 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800e070 con 0x7fcee015a070 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fceddffb700 1 -- 172.21.15.7:0/2118933080 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced800bbf0 con 0x7fcee015a070 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fceddffb700 1 -- 172.21.15.7:0/2118933080 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8012430 con 0x7fcee015a070 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fcee56d7700 1 -- 172.21.15.7:0/2118933080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee015a070 msgr2=0x7fcee00ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2118933080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee015a070 0x7fcee00ced30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fced8009e60 tx=0x7fced8009780 comp rx=0 tx=0).stop 2024-01-11T07:07:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fcee56d7700 1 -- 172.21.15.7:0/2118933080 shutdown_connections 2024-01-11T07:07:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2118933080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee015a070 0x7fcee00ced30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2118933080 >> v2:172.21.15.113:3300/0 conn(0x7fcee006e340 0x7fcee00ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.149+0000 7fcee56d7700 1 -- 172.21.15.7:0/2118933080 >> 172.21.15.7:0/2118933080 conn(0x7fcee0063cc0 msgr2=0x7fcee00d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.150+0000 7fcee56d7700 1 -- 172.21.15.7:0/2118933080 shutdown_connections 2024-01-11T07:07:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.150+0000 7fcee56d7700 1 -- 172.21.15.7:0/2118933080 wait complete. 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.151+0000 7fcee56d7700 1 Processor -- start 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.151+0000 7fcee56d7700 1 -- start start 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcee56d7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee006e340 0x7fcee0149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcee56d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee0159ca0 0x7fcee015e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcee56d7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee015a070 0x7fcee016a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcee56d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0169d80 con 0x7fcee0159ca0 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcee56d7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015cfb0 con 0x7fcee015a070 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcee56d7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015d200 con 0x7fcee006e340 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee0159ca0 0x7fcee015e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee0159ca0 0x7fcee015e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56898/0 (socket says 172.21.15.7:56898) 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 -- 172.21.15.7:0/2291495825 learned_addr learned my addr 172.21.15.7:0/2291495825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcedeffd700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee006e340 0x7fcee0149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcedf7fe700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee015a070 0x7fcee016a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 -- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee006e340 msgr2=0x7fcee0149850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee006e340 0x7fcee0149850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 -- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee015a070 msgr2=0x7fcee016a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee015a070 0x7fcee016a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 -- 172.21.15.7:0/2291495825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009a20 con 0x7fcee0159ca0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcede7fc700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee0159ca0 0x7fcee015e220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fced400d860 tx=0x7fced4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.152+0000 7fcec6ffd700 1 -- 172.21.15.7:0/2291495825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40125f0 con 0x7fcee0159ca0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.153+0000 7fcec6ffd700 1 -- 172.21.15.7:0/2291495825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4012750 con 0x7fcee0159ca0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.153+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee015d4e0 con 0x7fcee0159ca0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.153+0000 7fcec6ffd700 1 -- 172.21.15.7:0/2291495825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4012af0 con 0x7fcee0159ca0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.153+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee016a2a0 con 0x7fcee0159ca0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.154+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee015d6d0 con 0x7fcee0159ca0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.154+0000 7fcec6ffd700 1 -- 172.21.15.7:0/2291495825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fced4013480 con 0x7fcee0159ca0 2024-01-11T07:07:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.154+0000 7fcec6ffd700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcecc066f30 0x7fcecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.155+0000 7fcec6ffd700 1 -- 172.21.15.7:0/2291495825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fced4084560 con 0x7fcee0159ca0 2024-01-11T07:07:45.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.155+0000 7fcedeffd700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcecc066f30 0x7fcecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.155+0000 7fcedeffd700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcecc066f30 0x7fcecc0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcee00cc450 tx=0x7fced0007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:45.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.159+0000 7fcec6ffd700 1 -- 172.21.15.7:0/2291495825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fced40555b0 con 0x7fcee0159ca0 2024-01-11T07:07:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.447+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fcee016a4c0 con 0x7fcee0159ca0 2024-01-11T07:07:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.451+0000 7fcec6ffd700 1 -- 172.21.15.7:0/2291495825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fced4055140 con 0x7fcee0159ca0 2024-01-11T07:07:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcecc066f30 msgr2=0x7fcecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcecc066f30 0x7fcecc0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcee00cc450 tx=0x7fced0007eb0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee0159ca0 msgr2=0x7fcee015e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee0159ca0 0x7fcee015e220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fced400d860 tx=0x7fced4008040 comp rx=0 tx=0).stop 2024-01-11T07:07:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 shutdown_connections 2024-01-11T07:07:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcecc066f30 0x7fcecc0693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee015a070 0x7fcee016a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee006e340 0x7fcee0149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 --2- 172.21.15.7:0/2291495825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcee0159ca0 0x7fcee015e220 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 >> 172.21.15.7:0/2291495825 conn(0x7fcee0063cc0 msgr2=0x7fcee0157a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 shutdown_connections 2024-01-11T07:07:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.453+0000 7fcee56d7700 1 -- 172.21.15.7:0/2291495825 wait complete. 2024-01-11T07:07:45.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-11T07:07:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.600+0000 7f8ff93a7700 1 Processor -- start 2024-01-11T07:07:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.601+0000 7f8ff93a7700 1 -- start start 2024-01-11T07:07:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.601+0000 7f8ff93a7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff406de80 0x7f8ff40afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.601+0000 7f8ff93a7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff40d0730 con 0x7f8ff406e250 2024-01-11T07:07:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.601+0000 7f8ff93a7700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff40b0420 con 0x7f8ff415a230 2024-01-11T07:07:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.601+0000 7f8ff93a7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40cfb20 con 0x7f8ff406de80 2024-01-11T07:07:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff27fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8ff406e250 0x7f8ff4159b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39054/0 (socket says 172.21.15.7:39054) 2024-01-11T07:07:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff27fc700 1 -- 172.21.15.7:0/165284386 learned_addr learned my addr 172.21.15.7:0/165284386 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff2ffd700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff406de80 0x7f8ff40afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583150497 0 0) 0x7f8ff40d0730 con 0x7f8ff406e250 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fe0003570 con 0x7f8ff406e250 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998478477 0 0) 0x7f8ff40b0420 con 0x7f8ff415a230 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff40d0730 con 0x7f8ff415a230 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2201506805 0 0) 0x7f8fe0003570 con 0x7f8ff406e250 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff406de80 msgr2=0x7f8ff40afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff406de80 0x7f8ff40afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 >> v1:172.21.15.113:6789/0 conn(0x7f8ff415a230 legacy=0x7f8ff40b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40cfe00 con 0x7f8ff406e250 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f8fdc001470 con 0x7f8ff406e250 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 >> v1:172.21.15.7:6789/0 conn(0x7f8ff406e250 legacy=0x7f8ff4159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0003c20 0x7f8fe0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8fe0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff406de80 0x7f8fe0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40cfb20 con 0x7f8fe0004810 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe000bbf0 con 0x7f8fe0003c20 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe000bdb0 con 0x7f8ff406de80 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.602+0000 7f8ff27fc700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8fe0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff2ffd700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0003c20 0x7f8fe0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff37fe700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff406de80 0x7f8fe0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff27fc700 1 -- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff406de80 msgr2=0x7f8fe0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff27fc700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff406de80 0x7f8fe0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff27fc700 1 -- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0003c20 msgr2=0x7f8fe0004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff27fc700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fe0003c20 0x7f8fe0004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff27fc700 1 -- 172.21.15.7:0/165284386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0003570 con 0x7f8fe0004810 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff27fc700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8fe0004c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00c1e0 tx=0x7f8fdc001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333e6424bd48998f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00cb10 con 0x7f8fe0004810 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc00cc90 con 0x7f8fe0004810 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.603+0000 7f8ff1ffb700 1 -- 172.21.15.7:0/165284386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc005b70 con 0x7f8fe0004810 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.604+0000 7f8ff93a7700 1 -- 172.21.15.7:0/165284386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 msgr2=0x7f8fe0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.604+0000 7f8ff93a7700 1 --2- 172.21.15.7:0/165284386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8fe0004c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00c1e0 tx=0x7f8fdc001d40 comp rx=0 tx=0).stop 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.604+0000 7f8ff37fe700 1 -- 172.21.15.7:0/165284386 reap_dead start 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.604+0000 7f8ff93a7700 1 -- 172.21.15.7:0/165284386 shutdown_connections 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.604+0000 7f8ff93a7700 1 -- 172.21.15.7:0/165284386 >> 172.21.15.7:0/165284386 conn(0x7f8ff4071120 msgr2=0x7f8ff40b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.604+0000 7f8ff93a7700 1 -- 172.21.15.7:0/165284386 shutdown_connections 2024-01-11T07:07:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.604+0000 7f8ff93a7700 1 -- 172.21.15.7:0/165284386 wait complete. 2024-01-11T07:07:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.605+0000 7f8ff93a7700 1 Processor -- start 2024-01-11T07:07:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.605+0000 7f8ff93a7700 1 -- start start 2024-01-11T07:07:45.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff93a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8ff4153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff93a7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff416db50 0x7f8ff416dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff93a7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff416e500 0x7f8ff40c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff93a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff415eba0 con 0x7f8fe0004810 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff93a7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff415b0c0 con 0x7f8ff416db50 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff93a7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff415bbf0 con 0x7f8ff416e500 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8ff4153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8ff4153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56914/0 (socket says 172.21.15.7:56914) 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff2ffd700 1 -- 172.21.15.7:0/1979295920 learned_addr learned my addr 172.21.15.7:0/1979295920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff27fc700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff416db50 0x7f8ff416dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff37fe700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff416e500 0x7f8ff40c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff2ffd700 1 -- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff416e500 msgr2=0x7f8ff40c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff2ffd700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff416e500 0x7f8ff40c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff2ffd700 1 -- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff416db50 msgr2=0x7f8ff416dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff2ffd700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff416db50 0x7f8ff416dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.606+0000 7f8ff2ffd700 1 -- 172.21.15.7:0/1979295920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4154e00 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.607+0000 7f8ff2ffd700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8ff4153d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4011d40 tx=0x7f8fe40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.607+0000 7f8feaffd700 1 -- 172.21.15.7:0/1979295920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400fa10 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.607+0000 7f8feaffd700 1 -- 172.21.15.7:0/1979295920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400fb70 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.607+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff416e970 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.607+0000 7f8feaffd700 1 -- 172.21.15.7:0/1979295920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4014440 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.607+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff415b2c0 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.608+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff4089350 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.608+0000 7f8feaffd700 1 -- 172.21.15.7:0/1979295920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8fe4014870 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.609+0000 7f8feaffd700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8fd4066f30 0x7f8fd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.609+0000 7f8feaffd700 1 -- 172.21.15.7:0/1979295920 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8fe4018030 con 0x7f8fe0004810 2024-01-11T07:07:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.609+0000 7f8ff27fc700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8fd4066f30 0x7f8fd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.609+0000 7f8ff27fc700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8fd4066f30 0x7f8fd40693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ff415c4b0 tx=0x7f8fdc015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.613+0000 7f8feaffd700 1 -- 172.21.15.7:0/1979295920 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fe40520f0 con 0x7f8fe0004810 2024-01-11T07:07:45.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.900+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8ff40b3240 con 0x7f8fe0004810 2024-01-11T07:07:45.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.904+0000 7f8feaffd700 1 -- 172.21.15.7:0/1979295920 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8fe40220a0 con 0x7f8fe0004810 2024-01-11T07:07:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8fd4066f30 msgr2=0x7f8fd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8fd4066f30 0x7f8fd40693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ff415c4b0 tx=0x7f8fdc015000 comp rx=0 tx=0).stop 2024-01-11T07:07:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 msgr2=0x7f8ff4153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8ff4153d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4011d40 tx=0x7f8fe40091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 shutdown_connections 2024-01-11T07:07:45.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8fd4066f30 0x7f8fd40693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ff416e500 0x7f8ff40c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fe0004810 0x7f8ff4153d80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 --2- 172.21.15.7:0/1979295920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ff416db50 0x7f8ff416dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 >> 172.21.15.7:0/1979295920 conn(0x7f8ff4071120 msgr2=0x7f8ff41564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 shutdown_connections 2024-01-11T07:07:45.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:45.906+0000 7f8ff93a7700 1 -- 172.21.15.7:0/1979295920 wait complete. 2024-01-11T07:07:45.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-11T07:07:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.051+0000 7fe810220700 1 Processor -- start 2024-01-11T07:07:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.051+0000 7fe810220700 1 -- start start 2024-01-11T07:07:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.052+0000 7fe810220700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 0x7fe8080aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.052+0000 7fe810220700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8080ac5b0 0x7fe8080aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.052+0000 7fe810220700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80815d880 con 0x7fe808063230 2024-01-11T07:07:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.052+0000 7fe810220700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80806c5b0 con 0x7fe8080ac5b0 2024-01-11T07:07:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.052+0000 7fe810220700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80815a7f0 con 0x7fe80815a280 2024-01-11T07:07:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.052+0000 7fe80e7bd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8080ac5b0 0x7fe8080aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80e7bd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8080ac5b0 0x7fe8080aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56920/0 (socket says 172.21.15.7:56920) 2024-01-11T07:07:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80e7bd700 1 -- 172.21.15.7:0/1935529421 learned_addr learned my addr 172.21.15.7:0/1935529421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80d7bb700 1 --2- 172.21.15.7:0/1935529421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 0x7fe8080aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80e7bd700 1 -- 172.21.15.7:0/1935529421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 msgr2=0x7fe8080aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80e7bd700 1 --2- 172.21.15.7:0/1935529421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 0x7fe8080aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80e7bd700 1 -- 172.21.15.7:0/1935529421 >> v1:172.21.15.113:6789/0 conn(0x7fe808063230 legacy=0x7fe808063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80e7bd700 1 -- 172.21.15.7:0/1935529421 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80815aad0 con 0x7fe8080ac5b0 2024-01-11T07:07:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80e7bd700 1 --2- 172.21.15.7:0/1935529421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8080ac5b0 0x7fe8080aca30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe800009f20 tx=0x7fe800009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f19e62e7e5ce52 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80cfba700 1 -- 172.21.15.7:0/1935529421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000e070 con 0x7fe8080ac5b0 2024-01-11T07:07:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80cfba700 1 -- 172.21.15.7:0/1935529421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80000b9d0 con 0x7fe8080ac5b0 2024-01-11T07:07:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.053+0000 7fe80cfba700 1 -- 172.21.15.7:0/1935529421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8000092e0 con 0x7fe8080ac5b0 2024-01-11T07:07:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.054+0000 7fe810220700 1 -- 172.21.15.7:0/1935529421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8080ac5b0 msgr2=0x7fe8080aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.054+0000 7fe810220700 1 --2- 172.21.15.7:0/1935529421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8080ac5b0 0x7fe8080aca30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe800009f20 tx=0x7fe800009840 comp rx=0 tx=0).stop 2024-01-11T07:07:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.054+0000 7fe810220700 1 -- 172.21.15.7:0/1935529421 shutdown_connections 2024-01-11T07:07:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.054+0000 7fe810220700 1 --2- 172.21.15.7:0/1935529421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8080ac5b0 0x7fe8080aca30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.054+0000 7fe810220700 1 --2- 172.21.15.7:0/1935529421 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 0x7fe8080aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.054+0000 7fe810220700 1 -- 172.21.15.7:0/1935529421 >> 172.21.15.7:0/1935529421 conn(0x7fe8080680e0 msgr2=0x7fe8080cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.054+0000 7fe810220700 1 -- 172.21.15.7:0/1935529421 shutdown_connections 2024-01-11T07:07:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.054+0000 7fe810220700 1 -- 172.21.15.7:0/1935529421 wait complete. 2024-01-11T07:07:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.055+0000 7fe810220700 1 Processor -- start 2024-01-11T07:07:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.055+0000 7fe810220700 1 -- start start 2024-01-11T07:07:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe810220700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe808063230 0x7fe8081530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe810220700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8080ac5b0 0x7fe8080b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe810220700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 0x7fe8080b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe810220700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080aeb60 con 0x7fe808063230 2024-01-11T07:07:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe810220700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080aeff0 con 0x7fe8080ac5b0 2024-01-11T07:07:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe810220700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080adf50 con 0x7fe80815a280 2024-01-11T07:07:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80dfbc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe808063230 0x7fe8081530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80dfbc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe808063230 0x7fe8081530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56926/0 (socket says 172.21.15.7:56926) 2024-01-11T07:07:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80dfbc700 1 -- 172.21.15.7:0/1038166309 learned_addr learned my addr 172.21.15.7:0/1038166309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80e7bd700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 0x7fe8080b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80dfbc700 1 -- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 msgr2=0x7fe8080b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80dfbc700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 0x7fe8080b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80d7bb700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8080ac5b0 0x7fe8080b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80dfbc700 1 -- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8080ac5b0 msgr2=0x7fe8080b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.056+0000 7fe80dfbc700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8080ac5b0 0x7fe8080b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.057+0000 7fe80dfbc700 1 -- 172.21.15.7:0/1038166309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009ae0 con 0x7fe808063230 2024-01-11T07:07:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.057+0000 7fe80dfbc700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe808063230 0x7fe8081530f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe80400c4e0 tx=0x7fe80400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.057+0000 7fe7fdffb700 1 -- 172.21.15.7:0/1038166309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804011590 con 0x7fe808063230 2024-01-11T07:07:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.057+0000 7fe7fdffb700 1 -- 172.21.15.7:0/1038166309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe804011710 con 0x7fe808063230 2024-01-11T07:07:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.057+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8080ae230 con 0x7fe808063230 2024-01-11T07:07:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.057+0000 7fe7fdffb700 1 -- 172.21.15.7:0/1038166309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804011ad0 con 0x7fe808063230 2024-01-11T07:07:46.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.057+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8080ae7f0 con 0x7fe808063230 2024-01-11T07:07:46.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.058+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80804c3c0 con 0x7fe808063230 2024-01-11T07:07:46.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.058+0000 7fe7fdffb700 1 -- 172.21.15.7:0/1038166309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe80401b3b0 con 0x7fe808063230 2024-01-11T07:07:46.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.059+0000 7fe7fdffb700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7ec066f30 0x7fe7ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.059+0000 7fe7fdffb700 1 -- 172.21.15.7:0/1038166309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe804083770 con 0x7fe808063230 2024-01-11T07:07:46.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.059+0000 7fe80d7bb700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7ec066f30 0x7fe7ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.060+0000 7fe80d7bb700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7ec066f30 0x7fe7ec0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80093f0 tx=0x7fe7f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.063+0000 7fe7fdffb700 1 -- 172.21.15.7:0/1038166309 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe804050d30 con 0x7fe808063230 2024-01-11T07:07:46.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.349+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fe80815adb0 con 0x7fe808063230 2024-01-11T07:07:46.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.353+0000 7fe7fdffb700 1 -- 172.21.15.7:0/1038166309 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe804017020 con 0x7fe808063230 2024-01-11T07:07:46.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7ec066f30 msgr2=0x7fe7ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7ec066f30 0x7fe7ec0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80093f0 tx=0x7fe7f8008040 comp rx=0 tx=0).stop 2024-01-11T07:07:46.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe808063230 msgr2=0x7fe8081530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe808063230 0x7fe8081530f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe80400c4e0 tx=0x7fe80400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 shutdown_connections 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7ec066f30 0x7fe7ec0693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe80815a280 0x7fe8080b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe808063230 0x7fe8081530f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 --2- 172.21.15.7:0/1038166309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8080ac5b0 0x7fe8080b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 >> 172.21.15.7:0/1038166309 conn(0x7fe8080680e0 msgr2=0x7fe80815b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 shutdown_connections 2024-01-11T07:07:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.356+0000 7fe810220700 1 -- 172.21.15.7:0/1038166309 wait complete. 2024-01-11T07:07:46.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:46.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.499+0000 7f52c4bd9700 1 Processor -- start 2024-01-11T07:07:46.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.500+0000 7f52c4bd9700 1 -- start start 2024-01-11T07:07:46.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52c4bd9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52c01579d0 0x7f52c015c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52c4bd9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c00cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52c4bd9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c006d270 con 0x7f52c00ce220 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52c4bd9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00ce6a0 con 0x7f52c01579d0 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52c4bd9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00cda40 con 0x7f52c0157da0 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52be59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52c01579d0 0x7f52c015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52bed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f52c00ce220 0x7f52c006ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39070/0 (socket says 172.21.15.7:39070) 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52bed9d700 1 -- 172.21.15.7:0/316775968 learned_addr learned my addr 172.21.15.7:0/316775968 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52bdd9b700 1 --2- 172.21.15.7:0/316775968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c00cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52be59c700 1 -- 172.21.15.7:0/316775968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 msgr2=0x7f52c00cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52be59c700 1 --2- 172.21.15.7:0/316775968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c00cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52be59c700 1 -- 172.21.15.7:0/316775968 >> v1:172.21.15.7:6789/0 conn(0x7f52c00ce220 legacy=0x7f52c006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52be59c700 1 -- 172.21.15.7:0/316775968 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00cd5b0 con 0x7f52c01579d0 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52bd59a700 1 -- 172.21.15.7:0/316775968 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481077306 0 0) 0x7f52c006d270 con 0x7f52c00ce220 2024-01-11T07:07:46.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52bdd9b700 1 --2- 172.21.15.7:0/316775968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c00cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.501+0000 7f52be59c700 1 --2- 172.21.15.7:0/316775968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c01579d0 0x7f52c015c100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f52b00098e0 tx=0x7f52b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8ea4ac3f824b7f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52bd59a700 1 -- 172.21.15.7:0/316775968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b0009e80 con 0x7f52c01579d0 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52bd59a700 1 -- 172.21.15.7:0/316775968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b000b860 con 0x7f52c01579d0 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52bd59a700 1 -- 172.21.15.7:0/316775968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000bbe0 con 0x7f52c01579d0 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52c4bd9700 1 -- 172.21.15.7:0/316775968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c01579d0 msgr2=0x7f52c015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/316775968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c01579d0 0x7f52c015c100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f52b00098e0 tx=0x7f52b00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52c4bd9700 1 -- 172.21.15.7:0/316775968 shutdown_connections 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/316775968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c01579d0 0x7f52c015c100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/316775968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c00cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52c4bd9700 1 -- 172.21.15.7:0/316775968 >> 172.21.15.7:0/316775968 conn(0x7f52c006ae00 msgr2=0x7f52c006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52c4bd9700 1 -- 172.21.15.7:0/316775968 shutdown_connections 2024-01-11T07:07:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.502+0000 7f52c4bd9700 1 -- 172.21.15.7:0/316775968 wait complete. 2024-01-11T07:07:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.503+0000 7f52c4bd9700 1 Processor -- start 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.504+0000 7f52c4bd9700 1 -- start start 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.504+0000 7f52c4bd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c00ce220 0x7f52c00c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.504+0000 7f52c4bd9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c01579d0 0x7f52c015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.504+0000 7f52c4bd9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c0168660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.504+0000 7f52c4bd9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c015b4c0 con 0x7f52c00ce220 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.504+0000 7f52c4bd9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c016bad0 con 0x7f52c01579d0 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.504+0000 7f52c4bd9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c016bdb0 con 0x7f52c0157da0 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c00ce220 0x7f52c00c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c00ce220 0x7f52c00c3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56948/0 (socket says 172.21.15.7:56948) 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 -- 172.21.15.7:0/2758209206 learned_addr learned my addr 172.21.15.7:0/2758209206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52bed9d700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c0168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52bdd9b700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c01579d0 0x7f52c015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 -- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 msgr2=0x7f52c0168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c0168660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 -- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c01579d0 msgr2=0x7f52c015a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c01579d0 0x7f52c015a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 -- 172.21.15.7:0/2758209206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0009420 con 0x7f52c00ce220 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52be59c700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c00ce220 0x7f52c00c3a70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f52c006e9f0 tx=0x7f52b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52ae7fc700 1 -- 172.21.15.7:0/2758209206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b000bbe0 con 0x7f52c00ce220 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52ae7fc700 1 -- 172.21.15.7:0/2758209206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b0011420 con 0x7f52c00ce220 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c016c090 con 0x7f52c00ce220 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52ae7fc700 1 -- 172.21.15.7:0/2758209206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00117e0 con 0x7f52c00ce220 2024-01-11T07:07:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.505+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c00c3fb0 con 0x7f52c00ce220 2024-01-11T07:07:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.506+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c006d480 con 0x7f52c00ce220 2024-01-11T07:07:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.507+0000 7f52ae7fc700 1 -- 172.21.15.7:0/2758209206 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f52b0011bd0 con 0x7f52c00ce220 2024-01-11T07:07:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.508+0000 7f52ae7fc700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f52a0066fb0 0x7f52a0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.508+0000 7f52ae7fc700 1 -- 172.21.15.7:0/2758209206 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f52b0086610 con 0x7f52c00ce220 2024-01-11T07:07:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.508+0000 7f52bdd9b700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f52a0066fb0 0x7f52a0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.509+0000 7f52bdd9b700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f52a0066fb0 0x7f52a0069460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006fd0 tx=0x7f52b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.512+0000 7f52ae7fc700 1 -- 172.21.15.7:0/2758209206 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52b0053b30 con 0x7f52c00ce220 2024-01-11T07:07:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.717+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52c00c4200 con 0x7f52c00ce220 2024-01-11T07:07:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.718+0000 7f52ae7fc700 1 -- 172.21.15.7:0/2758209206 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12086 (secure 0 0 0) 0x7f52b0015090 con 0x7f52c00ce220 2024-01-11T07:07:46.717 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:46.718 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:07:43.178850+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f52a0066fb0 msgr2=0x7f52a0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f52a0066fb0 0x7f52a0069460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006fd0 tx=0x7f52b4008040 comp rx=0 tx=0).stop 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c00ce220 msgr2=0x7f52c00c3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c00ce220 0x7f52c00c3a70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f52c006e9f0 tx=0x7f52b0017000 comp rx=0 tx=0).stop 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 shutdown_connections 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f52a0066fb0 0x7f52a0069460 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52c0157da0 0x7f52c0168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52c00ce220 0x7f52c00c3a70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 --2- 172.21.15.7:0/2758209206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52c01579d0 0x7f52c015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 >> 172.21.15.7:0/2758209206 conn(0x7f52c006ae00 msgr2=0x7f52c0154650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:46.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 shutdown_connections 2024-01-11T07:07:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.720+0000 7f52c4bd9700 1 -- 172.21.15.7:0/2758209206 wait complete. 2024-01-11T07:07:46.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:07:46.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:07:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.868+0000 7f47cf08c700 1 Processor -- start 2024-01-11T07:07:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.868+0000 7f47cf08c700 1 -- start start 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cf08c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d0e0 0x7f47c80acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cf08c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47c8063230 0x7f47c8063630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cf08c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c806a2b0 con 0x7f47c815d8b0 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cf08c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c8063b70 con 0x7f47c8063230 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cf08c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c815e070 con 0x7f47c815d0e0 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cd629700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47c8063230 0x7f47c8063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cd629700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47c8063230 0x7f47c8063630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56954/0 (socket says 172.21.15.7:56954) 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cd629700 1 -- 172.21.15.7:0/241015916 learned_addr learned my addr 172.21.15.7:0/241015916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.869+0000 7f47cce28700 1 --2- 172.21.15.7:0/241015916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d0e0 0x7f47c80acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47cd629700 1 -- 172.21.15.7:0/241015916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d0e0 msgr2=0x7f47c80acbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47cd629700 1 --2- 172.21.15.7:0/241015916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d0e0 0x7f47c80acbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47cd629700 1 -- 172.21.15.7:0/241015916 >> v1:172.21.15.113:6789/0 conn(0x7f47c815d8b0 legacy=0x7f47c8069ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47cd629700 1 -- 172.21.15.7:0/241015916 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c815e350 con 0x7f47c8063230 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47c77fe700 1 -- 172.21.15.7:0/241015916 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067222219 0 0) 0x7f47c806a2b0 con 0x7f47c815d8b0 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47cd629700 1 --2- 172.21.15.7:0/241015916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c8063630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009f20 tx=0x7f47c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79a0775a45af7c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47c77fe700 1 -- 172.21.15.7:0/241015916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000e070 con 0x7f47c8063230 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47c77fe700 1 -- 172.21.15.7:0/241015916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c000b9d0 con 0x7f47c8063230 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.870+0000 7f47c77fe700 1 -- 172.21.15.7:0/241015916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c00092e0 con 0x7f47c8063230 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.871+0000 7f47cf08c700 1 -- 172.21.15.7:0/241015916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 msgr2=0x7f47c8063630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.871+0000 7f47cf08c700 1 --2- 172.21.15.7:0/241015916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c8063630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009f20 tx=0x7f47c0009840 comp rx=0 tx=0).stop 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.871+0000 7f47cf08c700 1 -- 172.21.15.7:0/241015916 shutdown_connections 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.871+0000 7f47cf08c700 1 --2- 172.21.15.7:0/241015916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c8063630 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.871+0000 7f47cf08c700 1 --2- 172.21.15.7:0/241015916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d0e0 0x7f47c80acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.871+0000 7f47cf08c700 1 -- 172.21.15.7:0/241015916 >> 172.21.15.7:0/241015916 conn(0x7f47c80680e0 msgr2=0x7f47c80cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.871+0000 7f47cf08c700 1 -- 172.21.15.7:0/241015916 shutdown_connections 2024-01-11T07:07:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.871+0000 7f47cf08c700 1 -- 172.21.15.7:0/241015916 wait complete. 2024-01-11T07:07:46.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.872+0000 7f47cf08c700 1 Processor -- start 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.873+0000 7f47cf08c700 1 -- start start 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cf08c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c80b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cf08c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c815d0e0 0x7f47c80c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cf08c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d8b0 0x7f47c806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cf08c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c816a2d0 con 0x7f47c8063230 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cf08c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c816a5b0 con 0x7f47c815d0e0 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cf08c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c816a890 con 0x7f47c815d8b0 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c80b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c80b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56960/0 (socket says 172.21.15.7:56960) 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 -- 172.21.15.7:0/707980545 learned_addr learned my addr 172.21.15.7:0/707980545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47c7fff700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c815d0e0 0x7f47c80c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cd629700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d8b0 0x7f47c806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 -- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d8b0 msgr2=0x7f47c806ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d8b0 0x7f47c806ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 -- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c815d0e0 msgr2=0x7f47c80c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c815d0e0 0x7f47c80c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 -- 172.21.15.7:0/707980545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0009ae0 con 0x7f47c8063230 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.874+0000 7f47cce28700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c80b20d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47b800ec80 tx=0x7f47b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.875+0000 7f47c4ff9700 1 -- 172.21.15.7:0/707980545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b800c850 con 0x7f47c8063230 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.875+0000 7f47c4ff9700 1 -- 172.21.15.7:0/707980545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b800c9b0 con 0x7f47c8063230 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.875+0000 7f47c4ff9700 1 -- 172.21.15.7:0/707980545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b800cd50 con 0x7f47c8063230 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.875+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c816ab70 con 0x7f47c8063230 2024-01-11T07:07:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.875+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c80b0650 con 0x7f47c8063230 2024-01-11T07:07:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.876+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c816b010 con 0x7f47c8063230 2024-01-11T07:07:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.876+0000 7f47c4ff9700 1 -- 172.21.15.7:0/707980545 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f47b8013680 con 0x7f47c8063230 2024-01-11T07:07:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.876+0000 7f47c4ff9700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47a8066f30 0x7f47a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.877+0000 7f47c4ff9700 1 -- 172.21.15.7:0/707980545 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f47b8084410 con 0x7f47c8063230 2024-01-11T07:07:46.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.877+0000 7f47c7fff700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47a8066f30 0x7f47a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:46.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.877+0000 7f47c7fff700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47a8066f30 0x7f47a80693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47c80cb4e0 tx=0x7f47b0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:46.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:46.881+0000 7f47c4ff9700 1 -- 172.21.15.7:0/707980545 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47b80519d0 con 0x7f47c8063230 2024-01-11T07:07:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.085+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47c816b1c0 con 0x7f47c8063230 2024-01-11T07:07:47.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.086+0000 7f47c4ff9700 1 -- 172.21.15.7:0/707980545 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47b8054ff0 con 0x7f47c8063230 2024-01-11T07:07:47.086 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47a8066f30 msgr2=0x7f47a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47a8066f30 0x7f47a80693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47c80cb4e0 tx=0x7f47b0007ec0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 msgr2=0x7f47c80b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c80b20d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47b800ec80 tx=0x7f47b80091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 shutdown_connections 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47a8066f30 0x7f47a80693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815d8b0 0x7f47c806ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c8063230 0x7f47c80b20d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 --2- 172.21.15.7:0/707980545 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c815d0e0 0x7f47c80c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 >> 172.21.15.7:0/707980545 conn(0x7f47c80680e0 msgr2=0x7f47c806dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 shutdown_connections 2024-01-11T07:07:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.088+0000 7f47cf08c700 1 -- 172.21.15.7:0/707980545 wait complete. 2024-01-11T07:07:47.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:07:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.235+0000 7fc7b7cb4700 1 Processor -- start 2024-01-11T07:07:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.235+0000 7fc7b7cb4700 1 -- start start 2024-01-11T07:07:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b7cb4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7b015b260 0x7fc7b00b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b7cb4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b006e3f0 con 0x7fc7b0069c80 2024-01-11T07:07:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b7cb4700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b00b1f10 con 0x7fc7b0156760 2024-01-11T07:07:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b7cb4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b006ef90 con 0x7fc7b015b260 2024-01-11T07:07:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b6251700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc7b0069c80 0x7fc7b0156b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39080/0 (socket says 172.21.15.7:39080) 2024-01-11T07:07:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b6251700 1 -- 172.21.15.7:0/3651517448 learned_addr learned my addr 172.21.15.7:0/3651517448 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:47.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b524f700 1 --2- 172.21.15.7:0/3651517448 >> v2:172.21.15.113:3300/0 conn(0x7fc7b015b260 0x7fc7b00b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838174695 0 0) 0x7fc7b006e3f0 con 0x7fc7b0069c80 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.236+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c003570 con 0x7fc7b0069c80 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 828528499 0 0) 0x7fc79c003570 con 0x7fc7b0069c80 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 >> v2:172.21.15.113:3300/0 conn(0x7fc7b015b260 msgr2=0x7fc7b00b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 --2- 172.21.15.7:0/3651517448 >> v2:172.21.15.113:3300/0 conn(0x7fc7b015b260 0x7fc7b00b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 >> v1:172.21.15.113:6789/0 conn(0x7fc7b0156760 legacy=0x7fc7b006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b006dda0 con 0x7fc7b0069c80 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fc7a80017f0 con 0x7fc7b0069c80 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b524f700 1 --2- 172.21.15.7:0/3651517448 >> v2:172.21.15.113:3300/0 conn(0x7fc7b015b260 0x7fc7b00b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 >> v1:172.21.15.7:6789/0 conn(0x7fc7b0069c80 legacy=0x7fc7b0156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc79c003a60 0x7fc79c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc79c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc79c0051c0 0x7fc79c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b006ef90 con 0x7fc79c004810 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c00c0d0 con 0x7fc79c0051c0 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c00c290 con 0x7fc79c003a60 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b524f700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc79c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b5a50700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc79c003a60 0x7fc79c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b6251700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc79c0051c0 0x7fc79c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b524f700 1 -- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc79c003a60 msgr2=0x7fc79c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b524f700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc79c003a60 0x7fc79c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b524f700 1 -- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc79c0051c0 msgr2=0x7fc79c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b524f700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc79c0051c0 0x7fc79c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b524f700 1 -- 172.21.15.7:0/3651517448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c003570 con 0x7fc79c004810 2024-01-11T07:07:47.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b5a50700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc79c003a60 0x7fc79c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b6251700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc79c0051c0 0x7fc79c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b6251700 1 -- 172.21.15.7:0/3651517448 reap_dead start 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.237+0000 7fc7b524f700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc79c004c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009df0 tx=0x7fc7a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1dca6409310fcb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0009740 con 0x7fc79c004810 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000e920 con 0x7fc79c004810 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b4a4e700 1 -- 172.21.15.7:0/3651517448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000ece0 con 0x7fc79c004810 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/3651517448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 msgr2=0x7fc79c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b7cb4700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc79c004c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009df0 tx=0x7fc7a00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/3651517448 shutdown_connections 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b7cb4700 1 --2- 172.21.15.7:0/3651517448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc79c004c80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/3651517448 >> 172.21.15.7:0/3651517448 conn(0x7fc7b00cd910 msgr2=0x7fc7b00cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/3651517448 shutdown_connections 2024-01-11T07:07:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.238+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/3651517448 wait complete. 2024-01-11T07:07:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.240+0000 7fc7b7cb4700 1 Processor -- start 2024-01-11T07:07:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.240+0000 7fc7b7cb4700 1 -- start start 2024-01-11T07:07:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b7cb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc7b014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b7cb4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b00c3610 0x7fc7b0063230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b7cb4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7b00c3a80 0x7fc7b0063770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b7cb4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b01528c0 con 0x7fc79c004810 2024-01-11T07:07:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b7cb4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b01536e0 con 0x7fc7b00c3610 2024-01-11T07:07:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b7cb4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b014e300 con 0x7fc7b00c3a80 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b5a50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc7b014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b5a50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc7b014ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56990/0 (socket says 172.21.15.7:56990) 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b5a50700 1 -- 172.21.15.7:0/4018277733 learned_addr learned my addr 172.21.15.7:0/4018277733 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b524f700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b00c3610 0x7fc7b0063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b6251700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7b00c3a80 0x7fc7b0063770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b5a50700 1 -- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7b00c3a80 msgr2=0x7fc7b0063770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b5a50700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7b00c3a80 0x7fc7b0063770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b5a50700 1 -- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b00c3610 msgr2=0x7fc7b0063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b5a50700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b00c3610 0x7fc7b0063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.241+0000 7fc7b5a50700 1 -- 172.21.15.7:0/4018277733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b0152ad0 con 0x7fc79c004810 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.242+0000 7fc7b524f700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b00c3610 0x7fc7b0063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.242+0000 7fc7b6251700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7b00c3a80 0x7fc7b0063770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.242+0000 7fc7b5a50700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc7b014ddc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac000c00 tx=0x7fc7ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.242+0000 7fc7a5ffb700 1 -- 172.21.15.7:0/4018277733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0115c0 con 0x7fc79c004810 2024-01-11T07:07:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.242+0000 7fc7a5ffb700 1 -- 172.21.15.7:0/4018277733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac011720 con 0x7fc79c004810 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.242+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b0152db0 con 0x7fc79c004810 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.242+0000 7fc7a5ffb700 1 -- 172.21.15.7:0/4018277733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac011ac0 con 0x7fc79c004810 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.242+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b0153370 con 0x7fc79c004810 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.243+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b004c3c0 con 0x7fc79c004810 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.244+0000 7fc7a5ffb700 1 -- 172.21.15.7:0/4018277733 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc7ac014480 con 0x7fc79c004810 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.244+0000 7fc7a5ffb700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc794066f30 0x7fc7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.244+0000 7fc7a5ffb700 1 -- 172.21.15.7:0/4018277733 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc7ac083b50 con 0x7fc79c004810 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.244+0000 7fc7b524f700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc794066f30 0x7fc7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.245+0000 7fc7b524f700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc794066f30 0x7fc7940693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00cd890 tx=0x7fc7a001a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.249+0000 7fc7a5ffb700 1 -- 172.21.15.7:0/4018277733 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7ac04d5a0 con 0x7fc79c004810 2024-01-11T07:07:47.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.457+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc7b0153500 con 0x7fc79c004810 2024-01-11T07:07:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.457+0000 7fc7a5ffb700 1 -- 172.21.15.7:0/4018277733 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc7ac019070 con 0x7fc79c004810 2024-01-11T07:07:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:47.457 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:07:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:07:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:07:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:07:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:07:47.457 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:07:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.459+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc794066f30 msgr2=0x7fc7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.459+0000 7fc7b7cb4700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc794066f30 0x7fc7940693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00cd890 tx=0x7fc7a001a000 comp rx=0 tx=0).stop 2024-01-11T07:07:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.459+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 msgr2=0x7fc7b014ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.459+0000 7fc7b7cb4700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc7b014ddc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac000c00 tx=0x7fc7ac009040 comp rx=0 tx=0).stop 2024-01-11T07:07:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.460+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 shutdown_connections 2024-01-11T07:07:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.460+0000 7fc7b7cb4700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc794066f30 0x7fc7940693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.460+0000 7fc7b7cb4700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7b00c3a80 0x7fc7b0063770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.460+0000 7fc7b7cb4700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc79c004810 0x7fc7b014ddc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.460+0000 7fc7b7cb4700 1 --2- 172.21.15.7:0/4018277733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b00c3610 0x7fc7b0063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.460+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 >> 172.21.15.7:0/4018277733 conn(0x7fc7b00cd910 msgr2=0x7fc7b014fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.460+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 shutdown_connections 2024-01-11T07:07:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.460+0000 7fc7b7cb4700 1 -- 172.21.15.7:0/4018277733 wait complete. 2024-01-11T07:07:47.472 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:07:47.473 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:07:47.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.608+0000 7f37d11f9700 1 Processor -- start 2024-01-11T07:07:47.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.608+0000 7f37d11f9700 1 -- start start 2024-01-11T07:07:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37d11f9700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f37cc063480 0x7f37cc06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37d11f9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc06ac80 con 0x7f37cc063850 2024-01-11T07:07:47.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37d11f9700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc06e630 con 0x7f37cc154540 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37d11f9700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37cc06a9a0 con 0x7f37cc063480 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37ca59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f37cc063850 0x7f37cc153e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44996/0 (socket says 172.21.15.7:44996) 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37ca59c700 1 -- 172.21.15.7:0/1267149035 learned_addr learned my addr 172.21.15.7:0/1267149035 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37cad9d700 1 --2- 172.21.15.7:0/1267149035 >> v2:172.21.15.113:3300/0 conn(0x7f37cc063480 0x7f37cc06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891673724 0 0) 0x7f37cc06ac80 con 0x7f37cc063850 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.609+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37b8003570 con 0x7f37cc063850 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 576352892 0 0) 0x7f37b8003570 con 0x7f37cc063850 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 >> v2:172.21.15.113:3300/0 conn(0x7f37cc063480 msgr2=0x7f37cc06eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 --2- 172.21.15.7:0/1267149035 >> v2:172.21.15.113:3300/0 conn(0x7f37cc063480 0x7f37cc06eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 >> v1:172.21.15.113:6789/0 conn(0x7f37cc154540 legacy=0x7f37cc069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc06dfe0 con 0x7f37cc063850 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f37bc001470 con 0x7f37cc063850 2024-01-11T07:07:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 >> v1:172.21.15.7:6789/0 conn(0x7f37cc063850 legacy=0x7f37cc153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37b8003ab0 0x7f37b8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b8004810 0x7f37b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc06a9a0 con 0x7f37b8005210 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b800c130 con 0x7f37b8004810 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b800c2f0 con 0x7f37b8003ab0 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37ca59c700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b8004810 0x7f37b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37cad9d700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37b8003ab0 0x7f37b8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37cb59e700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37cb59e700 1 -- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37b8003ab0 msgr2=0x7f37b8003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37cb59e700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37b8003ab0 0x7f37b8003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37cb59e700 1 -- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b8004810 msgr2=0x7f37b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37cb59e700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b8004810 0x7f37b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37cb59e700 1 -- 172.21.15.7:0/1267149035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8003570 con 0x7f37b8005210 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37ca59c700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b8004810 0x7f37b8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.610+0000 7f37cb59e700 1 -- 172.21.15.7:0/1267149035 reap_dead start 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37cb59e700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37b800ba60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f37c4001fc0 tx=0x7f37c4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27b0d624f6eae5d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4001950 con 0x7f37b8005210 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c400c8b0 con 0x7f37b8005210 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37c9d9b700 1 -- 172.21.15.7:0/1267149035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c400cc50 con 0x7f37b8005210 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37d11f9700 1 -- 172.21.15.7:0/1267149035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 msgr2=0x7f37b800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37d11f9700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37b800ba60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f37c4001fc0 tx=0x7f37c4001640 comp rx=0 tx=0).stop 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37d11f9700 1 -- 172.21.15.7:0/1267149035 shutdown_connections 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37d11f9700 1 --2- 172.21.15.7:0/1267149035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37b800ba60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37d11f9700 1 -- 172.21.15.7:0/1267149035 >> 172.21.15.7:0/1267149035 conn(0x7f37cc06bce0 msgr2=0x7f37cc06c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37d11f9700 1 -- 172.21.15.7:0/1267149035 shutdown_connections 2024-01-11T07:07:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.611+0000 7f37d11f9700 1 -- 172.21.15.7:0/1267149035 wait complete. 2024-01-11T07:07:47.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.613+0000 7f37d11f9700 1 Processor -- start 2024-01-11T07:07:47.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.613+0000 7f37d11f9700 1 -- start start 2024-01-11T07:07:47.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37d11f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37cc151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37d11f9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37cc16a220 0x7f37cc16a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37d11f9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37cc156bc0 0x7f37cc157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37d11f9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc156730 con 0x7f37b8005210 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37d11f9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc155b20 con 0x7f37cc16a220 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37d11f9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc155e00 con 0x7f37cc156bc0 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37cc151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37cc151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50448/0 (socket says 172.21.15.7:50448) 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 -- 172.21.15.7:0/2372314694 learned_addr learned my addr 172.21.15.7:0/2372314694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37ca59c700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37cc16a220 0x7f37cc16a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cb59e700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37cc156bc0 0x7f37cc157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 -- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37cc156bc0 msgr2=0x7f37cc157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37cc156bc0 0x7f37cc157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 -- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37cc16a220 msgr2=0x7f37cc16a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37cc16a220 0x7f37cc16a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 -- 172.21.15.7:0/2372314694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc1560e0 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37cad9d700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37cc151d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f37b4006ac0 tx=0x7f37b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37c2ffd700 1 -- 172.21.15.7:0/2372314694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b400fa10 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37c2ffd700 1 -- 172.21.15.7:0/2372314694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b400fb70 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc1563c0 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.614+0000 7f37c2ffd700 1 -- 172.21.15.7:0/2372314694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4014440 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.615+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc0c92e0 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.616+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc04c3c0 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.616+0000 7f37c2ffd700 1 -- 172.21.15.7:0/2372314694 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f37b4014830 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.616+0000 7f37c2ffd700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f37ac066f30 0x7f37ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.617+0000 7f37c2ffd700 1 -- 172.21.15.7:0/2372314694 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f37b4018030 con 0x7f37b8005210 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.617+0000 7f37ca59c700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f37ac066f30 0x7f37ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.617+0000 7f37ca59c700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f37ac066f30 0x7f37ac0693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f37cc06a7c0 tx=0x7f37bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.621+0000 7f37c2ffd700 1 -- 172.21.15.7:0/2372314694 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37b4052140 con 0x7f37b8005210 2024-01-11T07:07:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:47.824+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f37cc06e2c0 con 0x7f37b8005210 2024-01-11T07:07:48.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.636+0000 7f37c2ffd700 1 -- 172.21.15.7:0/2372314694 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f37b40220b0 con 0x7f37b8005210 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f37ac066f30 msgr2=0x7f37ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f37ac066f30 0x7f37ac0693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f37cc06a7c0 tx=0x7f37bc009040 comp rx=0 tx=0).stop 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 msgr2=0x7f37cc151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37cc151d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f37b4006ac0 tx=0x7f37b40091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 shutdown_connections 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f37ac066f30 0x7f37ac0693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37cc156bc0 0x7f37cc157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8005210 0x7f37cc151d90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 --2- 172.21.15.7:0/2372314694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37cc16a220 0x7f37cc16a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 >> 172.21.15.7:0/2372314694 conn(0x7f37cc06bce0 msgr2=0x7f37cc0b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 shutdown_connections 2024-01-11T07:07:48.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.641+0000 7f37d11f9700 1 -- 172.21.15.7:0/2372314694 wait complete. 2024-01-11T07:07:48.642 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:07:48.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:07:48.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.801+0000 7f3d058ac700 1 Processor -- start 2024-01-11T07:07:48.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.801+0000 7f3d058ac700 1 -- start start 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.802+0000 7f3d058ac700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d000cdaa0 0x7f3d0015a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.802+0000 7f3d058ac700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0015e6c0 con 0x7f3d000cd6d0 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.802+0000 7f3d058ac700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0015acf0 con 0x7f3d000d0170 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.802+0000 7f3d058ac700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d000cdf00 con 0x7f3d000cdaa0 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.802+0000 7f3cfeffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3d000cd6d0 0x7f3d0015d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45010/0 (socket says 172.21.15.7:45010) 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.802+0000 7f3cfeffd700 1 -- 172.21.15.7:0/4003691249 learned_addr learned my addr 172.21.15.7:0/4003691249 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cff7fe700 1 --2- 172.21.15.7:0/4003691249 >> v2:172.21.15.113:3300/0 conn(0x7f3d000cdaa0 0x7f3d0015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284821850 0 0) 0x7f3d0015e6c0 con 0x7f3d000cd6d0 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cec003570 con 0x7f3d000cd6d0 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361410714 0 0) 0x7f3d0015acf0 con 0x7f3d000d0170 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d0015e6c0 con 0x7f3d000d0170 2024-01-11T07:07:48.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574456100 0 0) 0x7f3cec003570 con 0x7f3d000cd6d0 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 >> v2:172.21.15.113:3300/0 conn(0x7f3d000cdaa0 msgr2=0x7f3d0015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 --2- 172.21.15.7:0/4003691249 >> v2:172.21.15.113:3300/0 conn(0x7f3d000cdaa0 0x7f3d0015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 >> v1:172.21.15.113:6789/0 conn(0x7f3d000d0170 legacy=0x7f3d0015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0015afd0 con 0x7f3d000cd6d0 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f3cf0001470 con 0x7f3d000cd6d0 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 >> v1:172.21.15.7:6789/0 conn(0x7f3d000cd6d0 legacy=0x7f3d0015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cec003bf0 0x7f3cec003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec004810 0x7f3cec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cec0051c0 0x7f3cec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000cdf00 con 0x7f3cec003bf0 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec00c160 con 0x7f3cec0051c0 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec00c320 con 0x7f3cec004810 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfeffd700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cec003bf0 0x7f3cec003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cff7fe700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cec0051c0 0x7f3cec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.803+0000 7f3cfe7fc700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec004810 0x7f3cec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfeffd700 1 -- 172.21.15.7:0/4003691249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec004810 msgr2=0x7f3cec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfeffd700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec004810 0x7f3cec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfeffd700 1 -- 172.21.15.7:0/4003691249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cec0051c0 msgr2=0x7f3cec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfeffd700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cec0051c0 0x7f3cec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfeffd700 1 -- 172.21.15.7:0/4003691249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec003570 con 0x7f3cec003bf0 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cff7fe700 1 -- 172.21.15.7:0/4003691249 reap_dead start 2024-01-11T07:07:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfeffd700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cec003bf0 0x7f3cec003fe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000c2a0 tx=0x7f3cf0001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd5a5ac298a8f30 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000cb70 con 0x7f3cec003bf0 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000ccd0 con 0x7f3cec003bf0 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3cfdffb700 1 -- 172.21.15.7:0/4003691249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0005b70 con 0x7f3cec003bf0 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3d058ac700 1 -- 172.21.15.7:0/4003691249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cec003bf0 msgr2=0x7f3cec003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.804+0000 7f3d058ac700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cec003bf0 0x7f3cec003fe0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000c2a0 tx=0x7f3cf0001e10 comp rx=0 tx=0).stop 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.805+0000 7f3d058ac700 1 -- 172.21.15.7:0/4003691249 shutdown_connections 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.805+0000 7f3d058ac700 1 --2- 172.21.15.7:0/4003691249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cec003bf0 0x7f3cec003fe0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.805+0000 7f3d058ac700 1 -- 172.21.15.7:0/4003691249 >> 172.21.15.7:0/4003691249 conn(0x7f3d00071120 msgr2=0x7f3d0006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.805+0000 7f3d058ac700 1 -- 172.21.15.7:0/4003691249 shutdown_connections 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.805+0000 7f3d058ac700 1 -- 172.21.15.7:0/4003691249 wait complete. 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.806+0000 7f3d058ac700 1 Processor -- start 2024-01-11T07:07:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.806+0000 7f3d058ac700 1 -- start start 2024-01-11T07:07:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3d058ac700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec003bf0 0x7f3d00150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3d058ac700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d000b4370 0x7f3d00159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3d058ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0015a240 0x7f3d0015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3d058ac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0015a6b0 con 0x7f3d0015a240 2024-01-11T07:07:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3d058ac700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000b5c30 con 0x7f3d000b4370 2024-01-11T07:07:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3d058ac700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000b5020 con 0x7f3cec003bf0 2024-01-11T07:07:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0015a240 0x7f3d0015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:48.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0015a240 0x7f3d0015b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50478/0 (socket says 172.21.15.7:50478) 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 -- 172.21.15.7:0/4039079054 learned_addr learned my addr 172.21.15.7:0/4039079054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cfe7fc700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d000b4370 0x7f3d00159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cfeffd700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec003bf0 0x7f3d00150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 -- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec003bf0 msgr2=0x7f3d00150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec003bf0 0x7f3d00150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 -- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d000b4370 msgr2=0x7f3d00159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d000b4370 0x7f3d00159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 -- 172.21.15.7:0/4039079054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000b5300 con 0x7f3d0015a240 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cfe7fc700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d000b4370 0x7f3d00159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.807+0000 7f3cff7fe700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0015a240 0x7f3d0015b3d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3d000d1d20 tx=0x7f3cf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.808+0000 7f3cf6ffd700 1 -- 172.21.15.7:0/4039079054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800e040 con 0x7f3d0015a240 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.808+0000 7f3cf6ffd700 1 -- 172.21.15.7:0/4039079054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf800f040 con 0x7f3d0015a240 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.808+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d000b55e0 con 0x7f3d0015a240 2024-01-11T07:07:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.808+0000 7f3cf6ffd700 1 -- 172.21.15.7:0/4039079054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800fd70 con 0x7f3d0015a240 2024-01-11T07:07:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.808+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d000b4cf0 con 0x7f3d0015a240 2024-01-11T07:07:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.809+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0014e8e0 con 0x7f3d0015a240 2024-01-11T07:07:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.809+0000 7f3cf6ffd700 1 -- 172.21.15.7:0/4039079054 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3cf8015640 con 0x7f3d0015a240 2024-01-11T07:07:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.810+0000 7f3cf6ffd700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.810+0000 7f3cf6ffd700 1 -- 172.21.15.7:0/4039079054 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f3cf8084b20 con 0x7f3d0015a240 2024-01-11T07:07:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.810+0000 7f3cfeffd700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.810+0000 7f3cfeffd700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3ce0066f30 0x7f3ce00693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0007c90 tx=0x7f3cf0015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:48.814+0000 7f3cf6ffd700 1 -- 172.21.15.7:0/4039079054 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cf8051f70 con 0x7f3d0015a240 2024-01-11T07:07:49.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.020+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3d0004c3c0 con 0x7f3d0015a240 2024-01-11T07:07:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.643+0000 7f3cf6ffd700 1 -- 172.21.15.7:0/4039079054 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3cf8055590 con 0x7f3d0015a240 2024-01-11T07:07:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3ce0066f30 msgr2=0x7f3ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3ce0066f30 0x7f3ce00693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0007c90 tx=0x7f3cf0015000 comp rx=0 tx=0).stop 2024-01-11T07:07:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0015a240 msgr2=0x7f3d0015b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0015a240 0x7f3d0015b3d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3d000d1d20 tx=0x7f3cf800b040 comp rx=0 tx=0).stop 2024-01-11T07:07:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 shutdown_connections 2024-01-11T07:07:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0015a240 0x7f3d0015b3d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3cec003bf0 0x7f3d00150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 --2- 172.21.15.7:0/4039079054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d000b4370 0x7f3d00159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 >> 172.21.15.7:0/4039079054 conn(0x7f3d00071120 msgr2=0x7f3d001585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 shutdown_connections 2024-01-11T07:07:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.652+0000 7f3d058ac700 1 -- 172.21.15.7:0/4039079054 wait complete. 2024-01-11T07:07:49.653 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:07:49.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:07:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.807+0000 7f148cdaa700 1 Processor -- start 2024-01-11T07:07:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.807+0000 7f148cdaa700 1 -- start start 2024-01-11T07:07:49.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f148cdaa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 0x7f148815af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:49.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f148cdaa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1488154030 0x7f1488154490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f148cdaa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14881549d0 con 0x7f148815daf0 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f148cdaa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148815b460 con 0x7f1488154030 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f148cdaa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14880cb6c0 con 0x7f1488063230 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f1486d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1488154030 0x7f1488154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f1485d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f148815daf0 0x7f1488063600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45022/0 (socket says 172.21.15.7:45022) 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f1486d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1488154030 0x7f1488154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50484/0 (socket says 172.21.15.7:50484) 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.808+0000 7f1485d9b700 1 -- 172.21.15.7:0/3612956097 learned_addr learned my addr 172.21.15.7:0/3612956097 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f148659c700 1 --2- 172.21.15.7:0/3612956097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 0x7f148815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f1486d9d700 1 -- 172.21.15.7:0/3612956097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 msgr2=0x7f148815af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f1486d9d700 1 --2- 172.21.15.7:0/3612956097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 0x7f148815af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f1486d9d700 1 -- 172.21.15.7:0/3612956097 >> v1:172.21.15.7:6789/0 conn(0x7f148815daf0 legacy=0x7f1488063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f1486d9d700 1 -- 172.21.15.7:0/3612956097 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880ccb20 con 0x7f1488154030 2024-01-11T07:07:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f148559a700 1 -- 172.21.15.7:0/3612956097 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718394913 0 0) 0x7f14881549d0 con 0x7f148815daf0 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f1486d9d700 1 --2- 172.21.15.7:0/3612956097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f1488154490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1480009e60 tx=0x7f1480009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7ff885af2c2fa9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f148559a700 1 -- 172.21.15.7:0/3612956097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148000e070 con 0x7f1488154030 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f148559a700 1 -- 172.21.15.7:0/3612956097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148000bc30 con 0x7f1488154030 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.809+0000 7f148559a700 1 -- 172.21.15.7:0/3612956097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480012430 con 0x7f1488154030 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.810+0000 7f148cdaa700 1 -- 172.21.15.7:0/3612956097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 msgr2=0x7f1488154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.810+0000 7f148cdaa700 1 --2- 172.21.15.7:0/3612956097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f1488154490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1480009e60 tx=0x7f1480009780 comp rx=0 tx=0).stop 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.810+0000 7f148cdaa700 1 -- 172.21.15.7:0/3612956097 shutdown_connections 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.810+0000 7f148cdaa700 1 --2- 172.21.15.7:0/3612956097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f1488154490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.810+0000 7f148cdaa700 1 --2- 172.21.15.7:0/3612956097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 0x7f148815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.810+0000 7f148cdaa700 1 -- 172.21.15.7:0/3612956097 >> 172.21.15.7:0/3612956097 conn(0x7f14880ca560 msgr2=0x7f14880ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.810+0000 7f148cdaa700 1 -- 172.21.15.7:0/3612956097 shutdown_connections 2024-01-11T07:07:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.810+0000 7f148cdaa700 1 -- 172.21.15.7:0/3612956097 wait complete. 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.811+0000 7f148cdaa700 1 Processor -- start 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.811+0000 7f148cdaa700 1 -- start start 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f148cdaa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 0x7f14881523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f148cdaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f148816a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f148cdaa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148815daf0 0x7f148806ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f148cdaa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1488159510 con 0x7f1488154030 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f148cdaa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148815aa80 con 0x7f148815daf0 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f148cdaa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148816add0 con 0x7f1488063230 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1485d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f148816a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1485d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f148816a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50500/0 (socket says 172.21.15.7:50500) 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1485d9b700 1 -- 172.21.15.7:0/1466582853 learned_addr learned my addr 172.21.15.7:0/1466582853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f148659c700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 0x7f14881523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1486d9d700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148815daf0 0x7f148806ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1485d9b700 1 -- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 msgr2=0x7f14881523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1485d9b700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 0x7f14881523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1485d9b700 1 -- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148815daf0 msgr2=0x7f148806ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1485d9b700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148815daf0 0x7f148806ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.812+0000 7f1485d9b700 1 -- 172.21.15.7:0/1466582853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480009a20 con 0x7f1488154030 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.813+0000 7f1485d9b700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f148816a890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f147800c810 tx=0x7f1478007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.813+0000 7f147e7fc700 1 -- 172.21.15.7:0/1466582853 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478005b30 con 0x7f1488154030 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.813+0000 7f147e7fc700 1 -- 172.21.15.7:0/1466582853 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1478005c90 con 0x7f1488154030 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.813+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1488159e70 con 0x7f1488154030 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.813+0000 7f147e7fc700 1 -- 172.21.15.7:0/1466582853 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478007300 con 0x7f1488154030 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.813+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148815a430 con 0x7f1488154030 2024-01-11T07:07:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.814+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148815ac30 con 0x7f1488154030 2024-01-11T07:07:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.814+0000 7f147e7fc700 1 -- 172.21.15.7:0/1466582853 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f14780076b0 con 0x7f1488154030 2024-01-11T07:07:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.815+0000 7f147e7fc700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1468066f30 0x7f14680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.815+0000 7f147e7fc700 1 -- 172.21.15.7:0/1466582853 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f1478083f50 con 0x7f1488154030 2024-01-11T07:07:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.815+0000 7f148659c700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1468066f30 0x7f14680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.816+0000 7f148659c700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1468066f30 0x7f14680693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1470006fd0 tx=0x7f1470008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:49.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:49.820+0000 7f147e7fc700 1 -- 172.21.15.7:0/1466582853 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f148815ac30 con 0x7f1488154030 2024-01-11T07:07:50.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.071+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f148815ac30 con 0x7f1488154030 2024-01-11T07:07:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.674+0000 7f147e7fc700 1 -- 172.21.15.7:0/1466582853 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f148815ac30 con 0x7f1488154030 2024-01-11T07:07:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.680+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1468066f30 msgr2=0x7f14680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.680+0000 7f148cdaa700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1468066f30 0x7f14680693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1470006fd0 tx=0x7f1470008040 comp rx=0 tx=0).stop 2024-01-11T07:07:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.680+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 msgr2=0x7f148816a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.680+0000 7f148cdaa700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f148816a890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f147800c810 tx=0x7f1478007df0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.683+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 shutdown_connections 2024-01-11T07:07:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.683+0000 7f148cdaa700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1468066f30 0x7f14680693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.683+0000 7f148cdaa700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f148815daf0 0x7f148806ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.683+0000 7f148cdaa700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1488063230 0x7f14881523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.683+0000 7f148cdaa700 1 --2- 172.21.15.7:0/1466582853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1488154030 0x7f148816a890 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.683+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 >> 172.21.15.7:0/1466582853 conn(0x7f14880ca560 msgr2=0x7f1488153420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.683+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 shutdown_connections 2024-01-11T07:07:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.683+0000 7f148cdaa700 1 -- 172.21.15.7:0/1466582853 wait complete. 2024-01-11T07:07:50.684 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 4 and data pool 5 2024-01-11T07:07:50.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:50.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.833+0000 7f13ef356700 1 Processor -- start 2024-01-11T07:07:50.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.834+0000 7f13ef356700 1 -- start start 2024-01-11T07:07:50.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.834+0000 7f13ef356700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e80af660 0x7f13e80ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:50.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.834+0000 7f13ef356700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e80ae0e0 con 0x7f13e8157ab0 2024-01-11T07:07:50.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.834+0000 7f13ef356700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e80b0950 con 0x7f13e80adc80 2024-01-11T07:07:50.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.834+0000 7f13ef356700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e806bde0 con 0x7f13e80af660 2024-01-11T07:07:50.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13ed0f2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f13e8157ab0 0x7f13e80ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45034/0 (socket says 172.21.15.7:45034) 2024-01-11T07:07:50.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13ed0f2700 1 -- 172.21.15.7:0/1679741377 learned_addr learned my addr 172.21.15.7:0/1679741377 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:50.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13ec8f1700 1 --2- 172.21.15.7:0/1679741377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e80af660 0x7f13e80ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13dffff700 1 -- 172.21.15.7:0/1679741377 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128384866 0 0) 0x7f13e4001110 con 0x7f13e8157ab0 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13dffff700 1 -- 172.21.15.7:0/1679741377 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d4003570 con 0x7f13e8157ab0 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13dffff700 1 -- 172.21.15.7:0/1679741377 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123188663 0 0) 0x7f13e80b0950 con 0x7f13e80adc80 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13dffff700 1 -- 172.21.15.7:0/1679741377 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e4001110 con 0x7f13e80adc80 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13ec8f1700 1 -- 172.21.15.7:0/1679741377 >> v1:172.21.15.113:6789/0 conn(0x7f13e80adc80 legacy=0x7f13e80c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13ec8f1700 1 -- 172.21.15.7:0/1679741377 >> v1:172.21.15.7:6789/0 conn(0x7f13e8157ab0 legacy=0x7f13e80ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13ec8f1700 1 -- 172.21.15.7:0/1679741377 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80afad0 con 0x7f13e80af660 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.835+0000 7f13ec8f1700 1 --2- 172.21.15.7:0/1679741377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e80af660 0x7f13e80ad740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009910 tx=0x7f13d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38350078de90caee server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13dffff700 1 -- 172.21.15.7:0/1679741377 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8011630 con 0x7f13e80af660 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13dffff700 1 -- 172.21.15.7:0/1679741377 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8011790 con 0x7f13e80af660 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13dffff700 1 -- 172.21.15.7:0/1679741377 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8011b30 con 0x7f13e80af660 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13ef356700 1 -- 172.21.15.7:0/1679741377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e80af660 msgr2=0x7f13e80ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:50.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13ef356700 1 --2- 172.21.15.7:0/1679741377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e80af660 0x7f13e80ad740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009910 tx=0x7f13d80091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13ef356700 1 -- 172.21.15.7:0/1679741377 shutdown_connections 2024-01-11T07:07:50.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13ef356700 1 --2- 172.21.15.7:0/1679741377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e80af660 0x7f13e80ad740 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13ef356700 1 -- 172.21.15.7:0/1679741377 >> 172.21.15.7:0/1679741377 conn(0x7f13e8063230 msgr2=0x7f13e8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:50.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13ef356700 1 -- 172.21.15.7:0/1679741377 shutdown_connections 2024-01-11T07:07:50.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.836+0000 7f13ef356700 1 -- 172.21.15.7:0/1679741377 wait complete. 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.837+0000 7f13ef356700 1 Processor -- start 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.837+0000 7f13ef356700 1 -- start start 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.838+0000 7f13ef356700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e80adc80 0x7f13e80c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.838+0000 7f13ef356700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e80af660 0x7f13e81597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.838+0000 7f13ef356700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e8157ab0 0x7f13e814c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.838+0000 7f13ef356700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80c7720 con 0x7f13e80af660 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.838+0000 7f13ef356700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80c7490 con 0x7f13e80adc80 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.838+0000 7f13ef356700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80c5f40 con 0x7f13e8157ab0 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e80af660 0x7f13e81597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e80af660 0x7f13e81597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50510/0 (socket says 172.21.15.7:50510) 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 -- 172.21.15.7:0/178620222 learned_addr learned my addr 172.21.15.7:0/178620222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ed0f2700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e80adc80 0x7f13e80c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ed8f3700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e8157ab0 0x7f13e814c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 -- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e8157ab0 msgr2=0x7f13e814c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e8157ab0 0x7f13e814c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 -- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e80adc80 msgr2=0x7f13e80c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e80adc80 0x7f13e80c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 -- 172.21.15.7:0/178620222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8009420 con 0x7f13e80af660 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ed0f2700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e80adc80 0x7f13e80c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ec8f1700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e80af660 0x7f13e81597d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009f00 tx=0x7f13d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13dd7fa700 1 -- 172.21.15.7:0/178620222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d801f070 con 0x7f13e80af660 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e80c6220 con 0x7f13e80af660 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13dd7fa700 1 -- 172.21.15.7:0/178620222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d8000bc0 con 0x7f13e80af660 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13dd7fa700 1 -- 172.21.15.7:0/178620222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8025420 con 0x7f13e80af660 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.839+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e80c67e0 con 0x7f13e80af660 2024-01-11T07:07:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.840+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e80c8d30 con 0x7f13e80af660 2024-01-11T07:07:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.841+0000 7f13dd7fa700 1 -- 172.21.15.7:0/178620222 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f13d8023070 con 0x7f13e80af660 2024-01-11T07:07:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.841+0000 7f13dd7fa700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f13cc066f30 0x7f13cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.841+0000 7f13dd7fa700 1 -- 172.21.15.7:0/178620222 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f13d8085a70 con 0x7f13e80af660 2024-01-11T07:07:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.842+0000 7f13ed0f2700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f13cc066f30 0x7f13cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.842+0000 7f13ed0f2700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f13cc066f30 0x7f13cc0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f13e40058d0 tx=0x7f13e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:50.846+0000 7f13dd7fa700 1 -- 172.21.15.7:0/178620222 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13d8052c40 con 0x7f13e80af660 2024-01-11T07:07:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.056+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13e804c3c0 con 0x7f13e80af660 2024-01-11T07:07:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.057+0000 7f13dd7fa700 1 -- 172.21.15.7:0/178620222 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f13d802a020 con 0x7f13e80af660 2024-01-11T07:07:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:51.057 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:07:50.072328+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:07:47.856440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-11T07:07:49.053507+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:51.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.059+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f13cc066f30 msgr2=0x7f13cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.059+0000 7f13ef356700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f13cc066f30 0x7f13cc0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f13e40058d0 tx=0x7f13e4009040 comp rx=0 tx=0).stop 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.059+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e80af660 msgr2=0x7f13e81597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.059+0000 7f13ef356700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e80af660 0x7f13e81597d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009f00 tx=0x7f13d8017000 comp rx=0 tx=0).stop 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.060+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 shutdown_connections 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.060+0000 7f13ef356700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f13cc066f30 0x7f13cc0693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.060+0000 7f13ef356700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e8157ab0 0x7f13e814c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.060+0000 7f13ef356700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e80adc80 0x7f13e80c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.060+0000 7f13ef356700 1 --2- 172.21.15.7:0/178620222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e80af660 0x7f13e81597d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.060+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 >> 172.21.15.7:0/178620222 conn(0x7f13e8063230 msgr2=0x7f13e806b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.060+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 shutdown_connections 2024-01-11T07:07:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.060+0000 7f13ef356700 1 -- 172.21.15.7:0/178620222 wait complete. 2024-01-11T07:07:51.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:51.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.205+0000 7f9403f8b700 1 Processor -- start 2024-01-11T07:07:51.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.206+0000 7f9403f8b700 1 -- start start 2024-01-11T07:07:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.206+0000 7f9403f8b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc1579d0 0x7f93fc15c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.206+0000 7f9403f8b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93fc0ce3f0 0x7f93fc0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.206+0000 7f9403f8b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc06d270 con 0x7f93fc157da0 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.206+0000 7f9403f8b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc0cda40 con 0x7f93fc0ce3f0 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.206+0000 7f9403f8b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0cd5b0 con 0x7f93fc1579d0 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9402528700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93fc0ce3f0 0x7f93fc0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9401d27700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc1579d0 0x7f93fc15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9402528700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93fc0ce3f0 0x7f93fc0ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:42166/0 (socket says 172.21.15.7:42166) 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9401526700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f93fc157da0 0x7f93fc0cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:46282/0 (socket says 172.21.15.7:46282) 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9402528700 1 -- 172.21.15.7:0/3479692782 learned_addr learned my addr 172.21.15.7:0/3479692782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:51.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9401d27700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc1579d0 0x7f93fc15c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:33446/0 (socket says 172.21.15.7:33446) 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9401d27700 1 -- 172.21.15.7:0/3479692782 >> v2:172.21.15.113:3300/0 conn(0x7f93fc0ce3f0 msgr2=0x7f93fc0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9401d27700 1 --2- 172.21.15.7:0/3479692782 >> v2:172.21.15.113:3300/0 conn(0x7f93fc0ce3f0 0x7f93fc0ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9401d27700 1 -- 172.21.15.7:0/3479692782 >> v1:172.21.15.113:6789/0 conn(0x7f93fc157da0 legacy=0x7f93fc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9400d25700 1 -- 172.21.15.7:0/3479692782 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066431566 0 0) 0x7f93fc06d270 con 0x7f93fc157da0 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9401d27700 1 -- 172.21.15.7:0/3479692782 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0cc9a0 con 0x7f93fc1579d0 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9402528700 1 --2- 172.21.15.7:0/3479692782 >> v2:172.21.15.113:3300/0 conn(0x7f93fc0ce3f0 0x7f93fc0ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.207+0000 7f9401d27700 1 --2- 172.21.15.7:0/3479692782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc1579d0 0x7f93fc15c100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93f00098e0 tx=0x7f93f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633aacde356d4033 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9400d25700 1 -- 172.21.15.7:0/3479692782 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000b640 con 0x7f93fc1579d0 2024-01-11T07:07:51.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9400d25700 1 -- 172.21.15.7:0/3479692782 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f000b7a0 con 0x7f93fc1579d0 2024-01-11T07:07:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9400d25700 1 -- 172.21.15.7:0/3479692782 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000bb40 con 0x7f93fc1579d0 2024-01-11T07:07:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9403f8b700 1 -- 172.21.15.7:0/3479692782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc1579d0 msgr2=0x7f93fc15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9403f8b700 1 --2- 172.21.15.7:0/3479692782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc1579d0 0x7f93fc15c100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93f00098e0 tx=0x7f93f00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9403f8b700 1 -- 172.21.15.7:0/3479692782 shutdown_connections 2024-01-11T07:07:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9403f8b700 1 --2- 172.21.15.7:0/3479692782 >> v2:172.21.15.113:3300/0 conn(0x7f93fc0ce3f0 0x7f93fc0ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9403f8b700 1 --2- 172.21.15.7:0/3479692782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc1579d0 0x7f93fc15c100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9403f8b700 1 -- 172.21.15.7:0/3479692782 >> 172.21.15.7:0/3479692782 conn(0x7f93fc06ae00 msgr2=0x7f93fc06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:51.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9403f8b700 1 -- 172.21.15.7:0/3479692782 shutdown_connections 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.208+0000 7f9403f8b700 1 -- 172.21.15.7:0/3479692782 wait complete. 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.209+0000 7f9403f8b700 1 Processor -- start 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9403f8b700 1 -- start start 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9403f8b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93fc0ce3f0 0x7f93fc16aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9403f8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93fc1579d0 0x7f93fc0c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9403f8b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc157da0 0x7f93fc155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9403f8b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0c6e20 con 0x7f93fc1579d0 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9403f8b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc1595e0 con 0x7f93fc0ce3f0 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9403f8b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc16b4f0 con 0x7f93fc157da0 2024-01-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9401526700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93fc1579d0 0x7f93fc0c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9401526700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93fc1579d0 0x7f93fc0c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50520/0 (socket says 172.21.15.7:50520) 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.210+0000 7f9401526700 1 -- 172.21.15.7:0/797109374 learned_addr learned my addr 172.21.15.7:0/797109374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9401d27700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93fc0ce3f0 0x7f93fc16aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9402528700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc157da0 0x7f93fc155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9401526700 1 -- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc157da0 msgr2=0x7f93fc155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9401526700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc157da0 0x7f93fc155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9401526700 1 -- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93fc0ce3f0 msgr2=0x7f93fc16aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9401526700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93fc0ce3f0 0x7f93fc16aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9401526700 1 -- 172.21.15.7:0/797109374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009ae0 con 0x7f93fc1579d0 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9401526700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93fc1579d0 0x7f93fc0c68e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93f800c5b0 tx=0x7f93f8001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:51.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f93edffb700 1 -- 172.21.15.7:0/797109374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8005a70 con 0x7f93fc1579d0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f0009420 con 0x7f93fc1579d0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f93edffb700 1 -- 172.21.15.7:0/797109374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f8005bd0 con 0x7f93fc1579d0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f93edffb700 1 -- 172.21.15.7:0/797109374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f80070d0 con 0x7f93fc1579d0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.211+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc16bab0 con 0x7f93fc1579d0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.213+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc159910 con 0x7f93fc1579d0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.213+0000 7f93edffb700 1 -- 172.21.15.7:0/797109374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f93f8007480 con 0x7f93fc1579d0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.214+0000 7f93edffb700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f93e0066f30 0x7f93e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.214+0000 7f93edffb700 1 -- 172.21.15.7:0/797109374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93f8083aa0 con 0x7f93fc1579d0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.214+0000 7f9401d27700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f93e0066f30 0x7f93e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.214+0000 7f9401d27700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f93e0066f30 0x7f93e00693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f93f00091e0 tx=0x7f93f0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:51.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.218+0000 7f93edffb700 1 -- 172.21.15.7:0/797109374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93f8050d10 con 0x7f93fc1579d0 2024-01-11T07:07:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.424+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93fc04c3c0 con 0x7f93fc1579d0 2024-01-11T07:07:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.425+0000 7f93edffb700 1 -- 172.21.15.7:0/797109374 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f93f8017020 con 0x7f93fc1579d0 2024-01-11T07:07:51.424 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:51.425 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:07:50.072328+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:07:47.856440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-11T07:07:49.053507+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f93e0066f30 msgr2=0x7f93e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f93e0066f30 0x7f93e00693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f93f00091e0 tx=0x7f93f0017000 comp rx=0 tx=0).stop 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93fc1579d0 msgr2=0x7f93fc0c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93fc1579d0 0x7f93fc0c68e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93f800c5b0 tx=0x7f93f8001040 comp rx=0 tx=0).stop 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 shutdown_connections 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f93e0066f30 0x7f93e00693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93fc157da0 0x7f93fc155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93fc0ce3f0 0x7f93fc16aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 --2- 172.21.15.7:0/797109374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93fc1579d0 0x7f93fc0c68e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 >> 172.21.15.7:0/797109374 conn(0x7f93fc06ae00 msgr2=0x7f93fc153510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 shutdown_connections 2024-01-11T07:07:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.427+0000 7f9403f8b700 1 -- 172.21.15.7:0/797109374 wait complete. 2024-01-11T07:07:51.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:07:51.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.575+0000 7f9b60a1f700 1 Processor -- start 2024-01-11T07:07:51.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.576+0000 7f9b60a1f700 1 -- start start 2024-01-11T07:07:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b60a1f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 0x7f9b5c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b60a1f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c0ca080 con 0x7f9b5c157f90 2024-01-11T07:07:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b60a1f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c0ca360 con 0x7f9b5c157bc0 2024-01-11T07:07:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b60a1f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0ca640 con 0x7f9b5c0b30d0 2024-01-11T07:07:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b59d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9b5c157f90 0x7f9b5c0c8f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45050/0 (socket says 172.21.15.7:45050) 2024-01-11T07:07:51.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b59d9b700 1 -- 172.21.15.7:0/1874707360 learned_addr learned my addr 172.21.15.7:0/1874707360 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5ad9d700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 0x7f9b5c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524599182 0 0) 0x7f9b5c0ca080 con 0x7f9b5c157f90 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b44003570 con 0x7f9b5c157f90 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679243802 0 0) 0x7f9b44003570 con 0x7f9b5c157f90 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 msgr2=0x7f9b5c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 0x7f9b5c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 >> v1:172.21.15.113:6789/0 conn(0x7f9b5c157bc0 legacy=0x7f9b5c0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0b18c0 con 0x7f9b5c157f90 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f9b50001470 con 0x7f9b5c157f90 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5ad9d700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 0x7f9b5c0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 >> v1:172.21.15.7:6789/0 conn(0x7f9b5c157f90 legacy=0x7f9b5c0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 0x7f9b44003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44003fd0 0x7f9b44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44004d50 0x7f9b440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0ca640 con 0x7f9b44004d50 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4400bb40 con 0x7f9b44003fd0 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.577+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4400bd00 con 0x7f9b5c0b30d0 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5ad9d700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44004d50 0x7f9b440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b59d9b700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44003fd0 0x7f9b44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5a59c700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 0x7f9b44003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5ad9d700 1 -- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 msgr2=0x7f9b44003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5ad9d700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c0b30d0 0x7f9b44003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5ad9d700 1 -- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44003fd0 msgr2=0x7f9b44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5ad9d700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44003fd0 0x7f9b44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5ad9d700 1 -- 172.21.15.7:0/1874707360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b54009a20 con 0x7f9b44004d50 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5ad9d700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44004d50 0x7f9b440051c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400e3c0 tx=0x7f9b5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b5be7144ff99e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54009c50 con 0x7f9b44004d50 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5400f040 con 0x7f9b44004d50 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.578+0000 7f9b5959a700 1 -- 172.21.15.7:0/1874707360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400ee60 con 0x7f9b44004d50 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.579+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1874707360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44004d50 msgr2=0x7f9b440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.579+0000 7f9b60a1f700 1 --2- 172.21.15.7:0/1874707360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b44004d50 0x7f9b440051c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400e3c0 tx=0x7f9b5400b040 comp rx=0 tx=0).stop 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.579+0000 7f9b5ad9d700 1 -- 172.21.15.7:0/1874707360 reap_dead start 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.579+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1874707360 shutdown_connections 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.579+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1874707360 >> 172.21.15.7:0/1874707360 conn(0x7f9b5c097a50 msgr2=0x7f9b5c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.579+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1874707360 shutdown_connections 2024-01-11T07:07:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.579+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1874707360 wait complete. 2024-01-11T07:07:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.581+0000 7f9b60a1f700 1 Processor -- start 2024-01-11T07:07:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.581+0000 7f9b60a1f700 1 -- start start 2024-01-11T07:07:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b60a1f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44004d50 0x7f9b5c0c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b60a1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b5c0c4e80 0x7f9b5c154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b60a1f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c159d20 0x7f9b5c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b60a1f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0b1ca0 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b60a1f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c15a6d0 con 0x7f9b44004d50 2024-01-11T07:07:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b60a1f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0c3330 con 0x7f9b5c159d20 2024-01-11T07:07:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b5c0c4e80 0x7f9b5c154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b5c0c4e80 0x7f9b5c154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50534/0 (socket says 172.21.15.7:50534) 2024-01-11T07:07:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 -- 172.21.15.7:0/1089599235 learned_addr learned my addr 172.21.15.7:0/1089599235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b5a59c700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44004d50 0x7f9b5c0c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b5ad9d700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c159d20 0x7f9b5c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 -- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c159d20 msgr2=0x7f9b5c15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c159d20 0x7f9b5c15a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 -- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44004d50 msgr2=0x7f9b5c0c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44004d50 0x7f9b5c0c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 -- 172.21.15.7:0/1089599235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0c3610 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.582+0000 7f9b59d9b700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b5c0c4e80 0x7f9b5c154ec0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9b500073c0 tx=0x7f9b50001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.583+0000 7f9b4a7fc700 1 -- 172.21.15.7:0/1089599235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50018030 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.583+0000 7f9b4a7fc700 1 -- 172.21.15.7:0/1089599235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b50000f80 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.583+0000 7f9b4a7fc700 1 -- 172.21.15.7:0/1089599235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b500117f0 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.583+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c0c38f0 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.583+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c155400 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.584+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c04c3c0 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.584+0000 7f9b4a7fc700 1 -- 172.21.15.7:0/1089599235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9b5001b070 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.584+0000 7f9b4a7fc700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b3c066f30 0x7f9b3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.585+0000 7f9b4a7fc700 1 -- 172.21.15.7:0/1089599235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9b50019070 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.585+0000 7f9b5a59c700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b3c066f30 0x7f9b3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.585+0000 7f9b5a59c700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b3c066f30 0x7f9b3c0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c001130 tx=0x7f9b4c007d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.589+0000 7f9b4a7fc700 1 -- 172.21.15.7:0/1089599235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b500514c0 con 0x7f9b5c0c4e80 2024-01-11T07:07:51.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:51.844+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9b5c0c9670 con 0x7f9b5c0c4e80 2024-01-11T07:07:52.610 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-11T07:07:52.610+0000 7fe3c4ef5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:07:52.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.686+0000 7f9b4a7fc700 1 -- 172.21.15.7:0/1089599235 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9b50054ae0 con 0x7f9b5c0c4e80 2024-01-11T07:07:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b3c066f30 msgr2=0x7f9b3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b3c066f30 0x7f9b3c0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c001130 tx=0x7f9b4c007d80 comp rx=0 tx=0).stop 2024-01-11T07:07:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b5c0c4e80 msgr2=0x7f9b5c154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b5c0c4e80 0x7f9b5c154ec0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9b500073c0 tx=0x7f9b50001df0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 shutdown_connections 2024-01-11T07:07:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b3c066f30 0x7f9b3c0693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b5c159d20 0x7f9b5c15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b44004d50 0x7f9b5c0c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 --2- 172.21.15.7:0/1089599235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b5c0c4e80 0x7f9b5c154ec0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 >> 172.21.15.7:0/1089599235 conn(0x7f9b5c097a50 msgr2=0x7f9b5c0cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:52.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 shutdown_connections 2024-01-11T07:07:52.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.689+0000 7f9b60a1f700 1 -- 172.21.15.7:0/1089599235 wait complete. 2024-01-11T07:07:52.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:07:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.843+0000 7f96a6bd0700 1 Processor -- start 2024-01-11T07:07:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.843+0000 7f96a6bd0700 1 -- start start 2024-01-11T07:07:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f96a6bd0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 0x7f96a00c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f96a6bd0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96a006e0d0 0x7f96a0159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f96a6bd0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a015b770 con 0x7f96a0159ff0 2024-01-11T07:07:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f96a6bd0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a015abf0 con 0x7f96a006e0d0 2024-01-11T07:07:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f96a6bd0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a015aed0 con 0x7f96a006dd00 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f969ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96a006e0d0 0x7f96a0159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f969ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96a006e0d0 0x7f96a0159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50550/0 (socket says 172.21.15.7:50550) 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f969ffff700 1 -- 172.21.15.7:0/300917975 learned_addr learned my addr 172.21.15.7:0/300917975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f96a496c700 1 --2- 172.21.15.7:0/300917975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 0x7f96a00c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f969ffff700 1 -- 172.21.15.7:0/300917975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 msgr2=0x7f96a00c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f969ffff700 1 --2- 172.21.15.7:0/300917975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 0x7f96a00c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f969ffff700 1 -- 172.21.15.7:0/300917975 >> v1:172.21.15.113:6789/0 conn(0x7f96a0159ff0 legacy=0x7f96a00c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.845+0000 7f969ffff700 1 -- 172.21.15.7:0/300917975 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a015b1b0 con 0x7f96a006e0d0 2024-01-11T07:07:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f969ffff700 1 --2- 172.21.15.7:0/300917975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a006e0d0 0x7f96a0159ab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f96940099b0 tx=0x7f9694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db3923b39bd7a4a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f969f7fe700 1 -- 172.21.15.7:0/300917975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400b860 con 0x7f96a006e0d0 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f969f7fe700 1 -- 172.21.15.7:0/300917975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969400b9c0 con 0x7f96a006e0d0 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f969f7fe700 1 -- 172.21.15.7:0/300917975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400bd60 con 0x7f96a006e0d0 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f96a6bd0700 1 -- 172.21.15.7:0/300917975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a006e0d0 msgr2=0x7f96a0159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/300917975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a006e0d0 0x7f96a0159ab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f96940099b0 tx=0x7f9694009250 comp rx=0 tx=0).stop 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f96a6bd0700 1 -- 172.21.15.7:0/300917975 shutdown_connections 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/300917975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 0x7f96a00c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/300917975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a006e0d0 0x7f96a0159ab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f96a6bd0700 1 -- 172.21.15.7:0/300917975 >> 172.21.15.7:0/300917975 conn(0x7f96a00680e0 msgr2=0x7f96a00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f96a6bd0700 1 -- 172.21.15.7:0/300917975 shutdown_connections 2024-01-11T07:07:52.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.846+0000 7f96a6bd0700 1 -- 172.21.15.7:0/300917975 wait complete. 2024-01-11T07:07:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.847+0000 7f96a6bd0700 1 Processor -- start 2024-01-11T07:07:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.848+0000 7f96a6bd0700 1 -- start start 2024-01-11T07:07:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.848+0000 7f96a6bd0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 0x7f96a0157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.848+0000 7f96a6bd0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a006e0d0 0x7f96a0158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.848+0000 7f96a6bd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a0159ff0 0x7f96a0158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.848+0000 7f96a6bd0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00c58f0 con 0x7f96a0159ff0 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.848+0000 7f96a6bd0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a0169df0 con 0x7f96a006e0d0 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.848+0000 7f96a6bd0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a016a630 con 0x7f96a006dd00 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a0159ff0 0x7f96a0158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a0159ff0 0x7f96a0158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50558/0 (socket says 172.21.15.7:50558) 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 -- 172.21.15.7:0/1880502995 learned_addr learned my addr 172.21.15.7:0/1880502995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a496c700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 0x7f96a0157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f969ffff700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a006e0d0 0x7f96a0158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 -- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 msgr2=0x7f96a0157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 0x7f96a0157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 -- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a006e0d0 msgr2=0x7f96a0158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a006e0d0 0x7f96a0158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 -- 172.21.15.7:0/1880502995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96940094f0 con 0x7f96a0159ff0 2024-01-11T07:07:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a516d700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a0159ff0 0x7f96a0158da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f969800c840 tx=0x7f9698001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f969cff9700 1 -- 172.21.15.7:0/1880502995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698012630 con 0x7f96a0159ff0 2024-01-11T07:07:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f969cff9700 1 -- 172.21.15.7:0/1880502995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96980127b0 con 0x7f96a0159ff0 2024-01-11T07:07:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a016e120 con 0x7f96a0159ff0 2024-01-11T07:07:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f969cff9700 1 -- 172.21.15.7:0/1880502995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698012b70 con 0x7f96a0159ff0 2024-01-11T07:07:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.849+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a016a2c0 con 0x7f96a0159ff0 2024-01-11T07:07:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.851+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a004c3c0 con 0x7f96a0159ff0 2024-01-11T07:07:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.851+0000 7f969cff9700 1 -- 172.21.15.7:0/1880502995 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f969801a3a0 con 0x7f96a0159ff0 2024-01-11T07:07:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.851+0000 7f969cff9700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f968c066f30 0x7f968c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.851+0000 7f969cff9700 1 -- 172.21.15.7:0/1880502995 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9698084610 con 0x7f96a0159ff0 2024-01-11T07:07:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.852+0000 7f96a496c700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f968c066f30 0x7f968c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.852+0000 7f96a496c700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f968c066f30 0x7f968c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9690006fd0 tx=0x7f9690008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:52.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:52.856+0000 7f969cff9700 1 -- 172.21.15.7:0/1880502995 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9698051880 con 0x7f96a0159ff0 2024-01-11T07:07:53.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.106+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f96a016a770 con 0x7f96a0159ff0 2024-01-11T07:07:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.688+0000 7f969cff9700 1 -- 172.21.15.7:0/1880502995 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9698054ea0 con 0x7f96a0159ff0 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f968c066f30 msgr2=0x7f968c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f968c066f30 0x7f968c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9690006fd0 tx=0x7f9690008040 comp rx=0 tx=0).stop 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a0159ff0 msgr2=0x7f96a0158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a0159ff0 0x7f96a0158da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f969800c840 tx=0x7f9698001670 comp rx=0 tx=0).stop 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 shutdown_connections 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f968c066f30 0x7f968c0693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a0159ff0 0x7f96a0158da0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96a006dd00 0x7f96a0157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 --2- 172.21.15.7:0/1880502995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96a006e0d0 0x7f96a0158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 >> 172.21.15.7:0/1880502995 conn(0x7f96a00680e0 msgr2=0x7f96a00afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:53.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 shutdown_connections 2024-01-11T07:07:53.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.691+0000 7f96a6bd0700 1 -- 172.21.15.7:0/1880502995 wait complete. 2024-01-11T07:07:53.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:53.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.843+0000 7f8477522700 1 Processor -- start 2024-01-11T07:07:53.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.843+0000 7f8477522700 1 -- start start 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.844+0000 7f8477522700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84700cafb0 0x7f84700ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.844+0000 7f8477522700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700b0b40 0x7f8470161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.844+0000 7f8477522700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8470159300 con 0x7f8470158280 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.844+0000 7f8477522700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84700cf260 con 0x7f84700cafb0 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.844+0000 7f8477522700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700cf540 con 0x7f84700b0b40 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8474abd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8470158280 0x7f84700b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45058/0 (socket says 172.21.15.7:45058) 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8474abd700 1 -- 172.21.15.7:0/512500407 learned_addr learned my addr 172.21.15.7:0/512500407 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f84752be700 1 --2- 172.21.15.7:0/512500407 >> v2:172.21.15.113:3300/0 conn(0x7f84700cafb0 0x7f84700ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8475abf700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700b0b40 0x7f8470161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156669742 0 0) 0x7f8470159300 con 0x7f8470158280 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f845c003570 con 0x7f8470158280 2024-01-11T07:07:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2829506079 0 0) 0x7f845c003570 con 0x7f8470158280 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700b0b40 msgr2=0x7f8470161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700b0b40 0x7f8470161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 >> v2:172.21.15.113:3300/0 conn(0x7f84700cafb0 msgr2=0x7f84700ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 --2- 172.21.15.7:0/512500407 >> v2:172.21.15.113:3300/0 conn(0x7f84700cafb0 0x7f84700ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700cf820 con 0x7f8470158280 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f8464001470 con 0x7f8470158280 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f84752be700 1 --2- 172.21.15.7:0/512500407 >> v2:172.21.15.113:3300/0 conn(0x7f84700cafb0 0x7f84700ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 >> v1:172.21.15.7:6789/0 conn(0x7f8470158280 legacy=0x7f84700b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845c003a60 0x7f845c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700b0b40 0x7f845c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f845c004d50 0x7f845c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700cf260 con 0x7f845c004d50 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700cf540 con 0x7f845c003a60 2024-01-11T07:07:53.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c00ba60 con 0x7f84700b0b40 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f8475abf700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f845c004d50 0x7f845c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.845+0000 7f84752be700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845c003a60 0x7f845c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8474abd700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700b0b40 0x7f845c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8475abf700 1 -- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700b0b40 msgr2=0x7f845c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8475abf700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700b0b40 0x7f845c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8475abf700 1 -- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845c003a60 msgr2=0x7f845c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8475abf700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845c003a60 0x7f845c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8475abf700 1 -- 172.21.15.7:0/512500407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c003570 con 0x7f845c004d50 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f84752be700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845c003a60 0x7f845c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8475abf700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f845c004d50 0x7f845c0051c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f846800e210 tx=0x7f846800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a676c498e4f467b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846800ea60 con 0x7f845c004d50 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846800f040 con 0x7f845c004d50 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.846+0000 7f8463fff700 1 -- 172.21.15.7:0/512500407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846800ee10 con 0x7f845c004d50 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.847+0000 7f8477522700 1 -- 172.21.15.7:0/512500407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f845c004d50 msgr2=0x7f845c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.847+0000 7f8477522700 1 --2- 172.21.15.7:0/512500407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f845c004d50 0x7f845c0051c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f846800e210 tx=0x7f846800b040 comp rx=0 tx=0).stop 2024-01-11T07:07:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.847+0000 7f8475abf700 1 -- 172.21.15.7:0/512500407 reap_dead start 2024-01-11T07:07:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.847+0000 7f8477522700 1 -- 172.21.15.7:0/512500407 shutdown_connections 2024-01-11T07:07:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.847+0000 7f8477522700 1 -- 172.21.15.7:0/512500407 >> 172.21.15.7:0/512500407 conn(0x7f8470071100 msgr2=0x7f847006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.847+0000 7f8477522700 1 -- 172.21.15.7:0/512500407 shutdown_connections 2024-01-11T07:07:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.847+0000 7f8477522700 1 -- 172.21.15.7:0/512500407 wait complete. 2024-01-11T07:07:53.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.848+0000 7f8477522700 1 Processor -- start 2024-01-11T07:07:53.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.848+0000 7f8477522700 1 -- start start 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f8477522700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015e2d0 0x7f847015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f8477522700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8470155310 0x7f847016a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f8477522700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700cd660 0x7f84700cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f8477522700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847016ac00 con 0x7f847015e2d0 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f8477522700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847016d640 con 0x7f8470155310 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f8477522700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700cdff0 con 0x7f84700cd660 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f84752be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015e2d0 0x7f847015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f84752be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015e2d0 0x7f847015e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50574/0 (socket says 172.21.15.7:50574) 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f84752be700 1 -- 172.21.15.7:0/860529643 learned_addr learned my addr 172.21.15.7:0/860529643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f8474abd700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8470155310 0x7f847016a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.849+0000 7f8475abf700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700cd660 0x7f84700cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84752be700 1 -- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700cd660 msgr2=0x7f84700cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84752be700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700cd660 0x7f84700cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84752be700 1 -- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8470155310 msgr2=0x7f847016a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84752be700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8470155310 0x7f847016a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84752be700 1 -- 172.21.15.7:0/860529643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847016d340 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84752be700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015e2d0 0x7f847015e6a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f846c011dd0 tx=0x7f846c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84617fa700 1 -- 172.21.15.7:0/860529643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c00fc20 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84617fa700 1 -- 172.21.15.7:0/860529643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c00fd80 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8470155b70 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f84617fa700 1 -- 172.21.15.7:0/860529643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c01d690 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.850+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847015bd20 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.851+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847004c3c0 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.852+0000 7f84617fa700 1 -- 172.21.15.7:0/860529643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f846c01da80 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.853+0000 7f84617fa700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.853+0000 7f84617fa700 1 -- 172.21.15.7:0/860529643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f846c018070 con 0x7f847015e2d0 2024-01-11T07:07:53.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.853+0000 7f8474abd700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.854+0000 7f8474abd700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8464001d20 tx=0x7f8464001e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:53.857+0000 7f84617fa700 1 -- 172.21.15.7:0/860529643 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f846c052dc0 con 0x7f847015e2d0 2024-01-11T07:07:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.107+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8470159550 con 0x7f847015e2d0 2024-01-11T07:07:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.108+0000 7f84617fa700 1 -- 172.21.15.7:0/860529643 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f846c0563e0 con 0x7f847015e2d0 2024-01-11T07:07:54.208 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:54.209 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:07:50.072668+0000","modified":"2024-01-11T07:07:53.685303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/3458914988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3458914988},{"type":"v1","addr":"172.21.15.7:6834","nonce":3458914988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 msgr2=0x7f84540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8464001d20 tx=0x7f8464001e90 comp rx=0 tx=0).stop 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015e2d0 msgr2=0x7f847015e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015e2d0 0x7f847015e6a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f846c011dd0 tx=0x7f846c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 shutdown_connections 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84700cd660 0x7f84700cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015e2d0 0x7f847015e6a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 --2- 172.21.15.7:0/860529643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8470155310 0x7f847016a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 >> 172.21.15.7:0/860529643 conn(0x7f8470071100 msgr2=0x7f84700653b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 shutdown_connections 2024-01-11T07:07:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.110+0000 7f8477522700 1 -- 172.21.15.7:0/860529643 wait complete. 2024-01-11T07:07:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T07:07:54.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:07:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.349+0000 7fb51fe7d700 1 Processor -- start 2024-01-11T07:07:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.349+0000 7fb51fe7d700 1 -- start start 2024-01-11T07:07:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51fe7d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5180c0a70 0x7fb518161da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51fe7d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb518063940 0x7fb5180c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51fe7d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180c83f0 con 0x7fb5181622e0 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51fe7d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51815e740 con 0x7fb5180c0a70 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51fe7d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb518162780 con 0x7fb518063940 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51d418700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb5181622e0 0x7fb518063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45066/0 (socket says 172.21.15.7:45066) 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51d418700 1 -- 172.21.15.7:0/1434528653 learned_addr learned my addr 172.21.15.7:0/1434528653 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51dc19700 1 --2- 172.21.15.7:0/1434528653 >> v2:172.21.15.113:3300/0 conn(0x7fb5180c0a70 0x7fb518161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51e41a700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb518063940 0x7fb5180c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119568454 0 0) 0x7fb5180c83f0 con 0x7fb5181622e0 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.350+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb504003570 con 0x7fb5181622e0 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240546176 0 0) 0x7fb504003570 con 0x7fb5181622e0 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb518063940 msgr2=0x7fb5180c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb518063940 0x7fb5180c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 >> v2:172.21.15.113:3300/0 conn(0x7fb5180c0a70 msgr2=0x7fb518161da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 --2- 172.21.15.7:0/1434528653 >> v2:172.21.15.113:3300/0 conn(0x7fb5180c0a70 0x7fb518161da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518162a60 con 0x7fb5181622e0 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fb508001470 con 0x7fb5181622e0 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 >> v1:172.21.15.7:6789/0 conn(0x7fb5181622e0 legacy=0x7fb518063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb504003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb518063940 0x7fb504004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb504004d50 0x7fb5040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb518162780 con 0x7fb504003a60 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50400ba60 con 0x7fb504004d50 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50400bba0 con 0x7fb518063940 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51dc19700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb504003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51d418700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb518063940 0x7fb504004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51e41a700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb504004d50 0x7fb5040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51dc19700 1 -- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb518063940 msgr2=0x7fb504004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51dc19700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb518063940 0x7fb504004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51dc19700 1 -- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb504004d50 msgr2=0x7fb5040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51dc19700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb504004d50 0x7fb5040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51dc19700 1 -- 172.21.15.7:0/1434528653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504003570 con 0x7fb504003a60 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.351+0000 7fb51dc19700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb504003e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb514009c00 tx=0x7fb5140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed48132278a3f500 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400e6b0 con 0x7fb504003a60 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51400e810 con 0x7fb504003a60 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51cc17700 1 -- 172.21.15.7:0/1434528653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400ebd0 con 0x7fb504003a60 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51fe7d700 1 -- 172.21.15.7:0/1434528653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 msgr2=0x7fb504003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51fe7d700 1 --2- 172.21.15.7:0/1434528653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb504003e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb514009c00 tx=0x7fb5140091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51e41a700 1 -- 172.21.15.7:0/1434528653 reap_dead start 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51fe7d700 1 -- 172.21.15.7:0/1434528653 shutdown_connections 2024-01-11T07:07:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51fe7d700 1 -- 172.21.15.7:0/1434528653 >> 172.21.15.7:0/1434528653 conn(0x7fb5180680e0 msgr2=0x7fb5180cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51fe7d700 1 -- 172.21.15.7:0/1434528653 shutdown_connections 2024-01-11T07:07:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.352+0000 7fb51fe7d700 1 -- 172.21.15.7:0/1434528653 wait complete. 2024-01-11T07:07:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.354+0000 7fb51fe7d700 1 Processor -- start 2024-01-11T07:07:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.354+0000 7fb51fe7d700 1 -- start start 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51fe7d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb51816b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51fe7d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180afa90 0x7fb51816e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51fe7d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5180b2c20 0x7fb5180b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51fe7d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180af0d0 con 0x7fb504003a60 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51fe7d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180aec40 con 0x7fb5180b2c20 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51fe7d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5180b3600 con 0x7fb5180afa90 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51dc19700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb51816b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51dc19700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb51816b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50598/0 (socket says 172.21.15.7:50598) 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51dc19700 1 -- 172.21.15.7:0/3777068895 learned_addr learned my addr 172.21.15.7:0/3777068895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51e41a700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5180b2c20 0x7fb5180b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51d418700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180afa90 0x7fb51816e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51dc19700 1 -- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180afa90 msgr2=0x7fb51816e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.355+0000 7fb51dc19700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180afa90 0x7fb51816e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb51dc19700 1 -- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5180b2c20 msgr2=0x7fb5180b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb51dc19700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5180b2c20 0x7fb5180b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb51dc19700 1 -- 172.21.15.7:0/3777068895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180ae030 con 0x7fb504003a60 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb51d418700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180afa90 0x7fb51816e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb51dc19700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb51816b660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb504004730 tx=0x7fb51401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb50dffb700 1 -- 172.21.15.7:0/3777068895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514022070 con 0x7fb504003a60 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb50dffb700 1 -- 172.21.15.7:0/3777068895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51400e980 con 0x7fb504003a60 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb50dffb700 1 -- 172.21.15.7:0/3777068895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400e6b0 con 0x7fb504003a60 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5180ae310 con 0x7fb504003a60 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.356+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5180ae8d0 con 0x7fb504003a60 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.357+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51804c3c0 con 0x7fb504003a60 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.358+0000 7fb50dffb700 1 -- 172.21.15.7:0/3777068895 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb51401c450 con 0x7fb504003a60 2024-01-11T07:07:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.358+0000 7fb50dffb700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4fc066f00 0x7fb4fc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.358+0000 7fb50dffb700 1 -- 172.21.15.7:0/3777068895 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb514085fb0 con 0x7fb504003a60 2024-01-11T07:07:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.358+0000 7fb51d418700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4fc066f00 0x7fb4fc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.359+0000 7fb51d418700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4fc066f00 0x7fb4fc0693b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5080010e0 tx=0x7fb508009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:54.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.363+0000 7fb50dffb700 1 -- 172.21.15.7:0/3777068895 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5140b75a0 con 0x7fb504003a60 2024-01-11T07:07:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.567+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5180aef20 con 0x7fb504003a60 2024-01-11T07:07:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.568+0000 7fb50dffb700 1 -- 172.21.15.7:0/3777068895 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fb514017090 con 0x7fb504003a60 2024-01-11T07:07:54.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:54.567 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:07:51.655695+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:07:47.856440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-11T07:07:49.053507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:07:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4fc066f00 msgr2=0x7fb4fc0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4fc066f00 0x7fb4fc0693b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5080010e0 tx=0x7fb508009040 comp rx=0 tx=0).stop 2024-01-11T07:07:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 msgr2=0x7fb51816b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb51816b660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb504004730 tx=0x7fb51401a000 comp rx=0 tx=0).stop 2024-01-11T07:07:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 shutdown_connections 2024-01-11T07:07:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4fc066f00 0x7fb4fc0693b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5180b2c20 0x7fb5180b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb504003a60 0x7fb51816b660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 --2- 172.21.15.7:0/3777068895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5180afa90 0x7fb51816e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 >> 172.21.15.7:0/3777068895 conn(0x7fb5180680e0 msgr2=0x7fb5180cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 shutdown_connections 2024-01-11T07:07:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.570+0000 7fb51fe7d700 1 -- 172.21.15.7:0/3777068895 wait complete. 2024-01-11T07:07:54.583 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:07:54.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.716+0000 7fe96c0f1700 1 Processor -- start 2024-01-11T07:07:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.716+0000 7fe96c0f1700 1 -- start start 2024-01-11T07:07:54.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe96c0f1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9641579b0 0x7fe96415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe96c0f1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe964157d80 0x7fe964063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe96c0f1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe964063b90 con 0x7fe964063770 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe96c0f1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9640aeab0 con 0x7fe9641579b0 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe96c0f1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9640aed90 con 0x7fe964157d80 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe969e8d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9641579b0 0x7fe96415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe969e8d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9641579b0 0x7fe96415bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50600/0 (socket says 172.21.15.7:50600) 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe969e8d700 1 -- 172.21.15.7:0/2437604058 learned_addr learned my addr 172.21.15.7:0/2437604058 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.717+0000 7fe96968c700 1 --2- 172.21.15.7:0/2437604058 >> v2:172.21.15.113:3300/0 conn(0x7fe964157d80 0x7fe964063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe969e8d700 1 -- 172.21.15.7:0/2437604058 >> v2:172.21.15.113:3300/0 conn(0x7fe964157d80 msgr2=0x7fe964063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe969e8d700 1 --2- 172.21.15.7:0/2437604058 >> v2:172.21.15.113:3300/0 conn(0x7fe964157d80 0x7fe964063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe969e8d700 1 -- 172.21.15.7:0/2437604058 >> v1:172.21.15.113:6789/0 conn(0x7fe964063770 legacy=0x7fe96406dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe969e8d700 1 -- 172.21.15.7:0/2437604058 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640af070 con 0x7fe9641579b0 2024-01-11T07:07:54.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe96968c700 1 --2- 172.21.15.7:0/2437604058 >> v2:172.21.15.113:3300/0 conn(0x7fe964157d80 0x7fe964063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe969e8d700 1 --2- 172.21.15.7:0/2437604058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9641579b0 0x7fe96415bf50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe9640cd830 tx=0x7fe960009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9992bdc57b8954a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe968e8b700 1 -- 172.21.15.7:0/2437604058 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000b860 con 0x7fe9641579b0 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe968e8b700 1 -- 172.21.15.7:0/2437604058 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96000b9c0 con 0x7fe9641579b0 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.718+0000 7fe968e8b700 1 -- 172.21.15.7:0/2437604058 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000bd60 con 0x7fe9641579b0 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.719+0000 7fe96c0f1700 1 -- 172.21.15.7:0/2437604058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9641579b0 msgr2=0x7fe96415bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.719+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/2437604058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9641579b0 0x7fe96415bf50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe9640cd830 tx=0x7fe960009250 comp rx=0 tx=0).stop 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.719+0000 7fe96c0f1700 1 -- 172.21.15.7:0/2437604058 shutdown_connections 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.719+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/2437604058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9641579b0 0x7fe96415bf50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.719+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/2437604058 >> v2:172.21.15.113:3300/0 conn(0x7fe964157d80 0x7fe964063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.719+0000 7fe96c0f1700 1 -- 172.21.15.7:0/2437604058 >> 172.21.15.7:0/2437604058 conn(0x7fe9640680e0 msgr2=0x7fe9640cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:54.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.719+0000 7fe96c0f1700 1 -- 172.21.15.7:0/2437604058 shutdown_connections 2024-01-11T07:07:54.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.719+0000 7fe96c0f1700 1 -- 172.21.15.7:0/2437604058 wait complete. 2024-01-11T07:07:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.720+0000 7fe96c0f1700 1 Processor -- start 2024-01-11T07:07:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.720+0000 7fe96c0f1700 1 -- start start 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe96c0f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964063770 0x7fe96416b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe96c0f1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9641579b0 0x7fe964154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe96c0f1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe964157d80 0x7fe964155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe96c0f1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640c68b0 con 0x7fe964063770 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe96c0f1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640b21d0 con 0x7fe9641579b0 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe96c0f1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640c6260 con 0x7fe964157d80 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964063770 0x7fe96416b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964063770 0x7fe96416b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50608/0 (socket says 172.21.15.7:50608) 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 -- 172.21.15.7:0/681508067 learned_addr learned my addr 172.21.15.7:0/681508067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe96968c700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9641579b0 0x7fe964154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe96a68e700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe964157d80 0x7fe964155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 -- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe964157d80 msgr2=0x7fe964155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe964157d80 0x7fe964155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 -- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9641579b0 msgr2=0x7fe964154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9641579b0 0x7fe964154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 -- 172.21.15.7:0/681508067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600094f0 con 0x7fe964063770 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.721+0000 7fe969e8d700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964063770 0x7fe96416b7f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe960009290 tx=0x7fe960017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.722+0000 7fe959ffb700 1 -- 172.21.15.7:0/681508067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960026480 con 0x7fe964063770 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.722+0000 7fe959ffb700 1 -- 172.21.15.7:0/681508067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96001f070 con 0x7fe964063770 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.722+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9640c6540 con 0x7fe964063770 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.722+0000 7fe959ffb700 1 -- 172.21.15.7:0/681508067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960026850 con 0x7fe964063770 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.722+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe964169140 con 0x7fe964063770 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.723+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9640c66d0 con 0x7fe964063770 2024-01-11T07:07:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.723+0000 7fe959ffb700 1 -- 172.21.15.7:0/681508067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe96001a430 con 0x7fe964063770 2024-01-11T07:07:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.724+0000 7fe959ffb700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe948066f30 0x7fe9480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.724+0000 7fe959ffb700 1 -- 172.21.15.7:0/681508067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe960021070 con 0x7fe964063770 2024-01-11T07:07:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.724+0000 7fe96968c700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe948066f30 0x7fe9480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.725+0000 7fe96968c700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe948066f30 0x7fe9480693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe9640c0ea0 tx=0x7fe954008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.728+0000 7fe959ffb700 1 -- 172.21.15.7:0/681508067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe960053670 con 0x7fe964063770 2024-01-11T07:07:54.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.925+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9640c69f0 con 0x7fe948066f30 2024-01-11T07:07:54.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.935+0000 7fe959ffb700 1 -- 172.21.15.7:0/681508067 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284856 (secure 0 0 0) 0x7fe954011700 con 0x7fe948066f30 2024-01-11T07:07:54.938 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:54.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe948066f30 msgr2=0x7fe9480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe948066f30 0x7fe9480693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe9640c0ea0 tx=0x7fe954008040 comp rx=0 tx=0).stop 2024-01-11T07:07:54.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964063770 msgr2=0x7fe96416b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:54.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964063770 0x7fe96416b7f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe960009290 tx=0x7fe960017000 comp rx=0 tx=0).stop 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 shutdown_connections 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe948066f30 0x7fe9480693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe964157d80 0x7fe964155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964063770 0x7fe96416b7f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 --2- 172.21.15.7:0/681508067 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9641579b0 0x7fe964154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 >> 172.21.15.7:0/681508067 conn(0x7fe9640680e0 msgr2=0x7fe9640ca180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 shutdown_connections 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:54.938+0000 7fe96c0f1700 1 -- 172.21.15.7:0/681508067 wait complete. 2024-01-11T07:07:54.941 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:54.951 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-01-11T07:07:53.851501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560187,"statfs":{"total":1600000000,"available":1597632452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.215497"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666466+0000","last_change":"2024-01-11T07:07:50.666400+0000","last_active":"2024-01-11T07:07:50.666466+0000","last_peered":"2024-01-11T07:07:50.666466+0000","last_clean":"2024-01-11T07:07:50.666466+0000","last_became_active":"2024-01-11T07:07:50.666132+0000","last_became_peered":"2024-01-11T07:07:50.666132+0000","last_unstale":"2024-01-11T07:07:50.666466+0000","last_undegraded":"2024-01-11T07:07:50.666466+0000","last_fullsized":"2024-01-11T07:07:50.666466+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:08:05.071473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666434+0000","last_change":"2024-01-11T07:07:50.665922+0000","last_active":"2024-01-11T07:07:50.666434+0000","last_peered":"2024-01-11T07:07:50.666434+0000","last_clean":"2024-01-11T07:07:50.666434+0000","last_became_active":"2024-01-11T07:07:50.664730+0000","last_became_peered":"2024-01-11T07:07:50.664730+0000","last_unstale":"2024-01-11T07:07:50.666434+0000","last_undegraded":"2024-01-11T07:07:50.666434+0000","last_fullsized":"2024-01-11T07:07:50.666434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:27:47.108755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667569+0000","last_change":"2024-01-11T07:07:50.665647+0000","last_active":"2024-01-11T07:07:50.667569+0000","last_peered":"2024-01-11T07:07:50.667569+0000","last_clean":"2024-01-11T07:07:50.667569+0000","last_became_active":"2024-01-11T07:07:50.665414+0000","last_became_peered":"2024-01-11T07:07:50.665414+0000","last_unstale":"2024-01-11T07:07:50.667569+0000","last_undegraded":"2024-01-11T07:07:50.667569+0000","last_fullsized":"2024-01-11T07:07:50.667569+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:16:07.194652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674341+0000","last_change":"2024-01-11T07:07:50.674227+0000","last_active":"2024-01-11T07:07:50.674341+0000","last_peered":"2024-01-11T07:07:50.674341+0000","last_clean":"2024-01-11T07:07:50.674341+0000","last_became_active":"2024-01-11T07:07:50.673950+0000","last_became_peered":"2024-01-11T07:07:50.673950+0000","last_unstale":"2024-01-11T07:07:50.674341+0000","last_undegraded":"2024-01-11T07:07:50.674341+0000","last_fullsized":"2024-01-11T07:07:50.674341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:53.758682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669715+0000","last_change":"2024-01-11T07:07:50.669577+0000","last_active":"2024-01-11T07:07:50.669715+0000","last_peered":"2024-01-11T07:07:50.669715+0000","last_clean":"2024-01-11T07:07:50.669715+0000","last_became_active":"2024-01-11T07:07:50.669363+0000","last_became_peered":"2024-01-11T07:07:50.669363+0000","last_unstale":"2024-01-11T07:07:50.669715+0000","last_undegraded":"2024-01-11T07:07:50.669715+0000","last_fullsized":"2024-01-11T07:07:50.669715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:21:45.579684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670779+0000","last_change":"2024-01-11T07:07:50.670615+0000","last_active":"2024-01-11T07:07:50.670779+0000","last_peered":"2024-01-11T07:07:50.670779+0000","last_clean":"2024-01-11T07:07:50.670779+0000","last_became_active":"2024-01-11T07:07:50.670099+0000","last_became_peered":"2024-01-11T07:07:50.670099+0000","last_unstale":"2024-01-11T07:07:50.670779+0000","last_undegraded":"2024-01-11T07:07:50.670779+0000","last_fullsized":"2024-01-11T07:07:50.670779+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:35:14.001395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667833+0000","last_change":"2024-01-11T07:07:50.667413+0000","last_active":"2024-01-11T07:07:50.667833+0000","last_peered":"2024-01-11T07:07:50.667833+0000","last_clean":"2024-01-11T07:07:50.667833+0000","last_became_active":"2024-01-11T07:07:50.666819+0000","last_became_peered":"2024-01-11T07:07:50.666819+0000","last_unstale":"2024-01-11T07:07:50.667833+0000","last_undegraded":"2024-01-11T07:07:50.667833+0000","last_fullsized":"2024-01-11T07:07:50.667833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:56:45.059589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668755+0000","last_change":"2024-01-11T07:07:50.668587+0000","last_active":"2024-01-11T07:07:50.668755+0000","last_peered":"2024-01-11T07:07:50.668755+0000","last_clean":"2024-01-11T07:07:50.668755+0000","last_became_active":"2024-01-11T07:07:50.668340+0000","last_became_peered":"2024-01-11T07:07:50.668340+0000","last_unstale":"2024-01-11T07:07:50.668755+0000","last_undegraded":"2024-01-11T07:07:50.668755+0000","last_fullsized":"2024-01-11T07:07:50.668755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:23.978674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674125+0000","last_change":"2024-01-11T07:07:50.674056+0000","last_active":"2024-01-11T07:07:50.674125+0000","last_peered":"2024-01-11T07:07:50.674125+0000","last_clean":"2024-01-11T07:07:50.674125+0000","last_became_active":"2024-01-11T07:07:50.673207+0000","last_became_peered":"2024-01-11T07:07:50.673207+0000","last_unstale":"2024-01-11T07:07:50.674125+0000","last_undegraded":"2024-01-11T07:07:50.674125+0000","last_fullsized":"2024-01-11T07:07:50.674125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:25.740402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.671505+0000","last_change":"2024-01-11T07:07:50.671435+0000","last_active":"2024-01-11T07:07:50.671505+0000","last_peered":"2024-01-11T07:07:50.671505+0000","last_clean":"2024-01-11T07:07:50.671505+0000","last_became_active":"2024-01-11T07:07:50.671173+0000","last_became_peered":"2024-01-11T07:07:50.671173+0000","last_unstale":"2024-01-11T07:07:50.671505+0000","last_undegraded":"2024-01-11T07:07:50.671505+0000","last_fullsized":"2024-01-11T07:07:50.671505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:31:26.200394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667865+0000","last_change":"2024-01-11T07:07:50.667767+0000","last_active":"2024-01-11T07:07:50.667865+0000","last_peered":"2024-01-11T07:07:50.667865+0000","last_clean":"2024-01-11T07:07:50.667865+0000","last_became_active":"2024-01-11T07:07:50.667510+0000","last_became_peered":"2024-01-11T07:07:50.667510+0000","last_unstale":"2024-01-11T07:07:50.667865+0000","last_undegraded":"2024-01-11T07:07:50.667865+0000","last_fullsized":"2024-01-11T07:07:50.667865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:35:14.001395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669817+0000","last_change":"2024-01-11T07:07:50.669678+0000","last_active":"2024-01-11T07:07:50.669817+0000","last_peered":"2024-01-11T07:07:50.669817+0000","last_clean":"2024-01-11T07:07:50.669817+0000","last_became_active":"2024-01-11T07:07:50.668955+0000","last_became_peered":"2024-01-11T07:07:50.668955+0000","last_unstale":"2024-01-11T07:07:50.669817+0000","last_undegraded":"2024-01-11T07:07:50.669817+0000","last_fullsized":"2024-01-11T07:07:50.669817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:59:47.509012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667553+0000","last_change":"2024-01-11T07:07:50.667073+0000","last_active":"2024-01-11T07:07:50.667553+0000","last_peered":"2024-01-11T07:07:50.667553+0000","last_clean":"2024-01-11T07:07:50.667553+0000","last_became_active":"2024-01-11T07:07:50.666041+0000","last_became_peered":"2024-01-11T07:07:50.666041+0000","last_unstale":"2024-01-11T07:07:50.667553+0000","last_undegraded":"2024-01-11T07:07:50.667553+0000","last_fullsized":"2024-01-11T07:07:50.667553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:28:35.204062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666452+0000","last_change":"2024-01-11T07:07:50.665980+0000","last_active":"2024-01-11T07:07:50.666452+0000","last_peered":"2024-01-11T07:07:50.666452+0000","last_clean":"2024-01-11T07:07:50.666452+0000","last_became_active":"2024-01-11T07:07:50.665058+0000","last_became_peered":"2024-01-11T07:07:50.665058+0000","last_unstale":"2024-01-11T07:07:50.666452+0000","last_undegraded":"2024-01-11T07:07:50.666452+0000","last_fullsized":"2024-01-11T07:07:50.666452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:00.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667585+0000","last_change":"2024-01-11T07:07:50.666503+0000","last_active":"2024-01-11T07:07:50.667585+0000","last_peered":"2024-01-11T07:07:50.667585+0000","last_clean":"2024-01-11T07:07:50.667585+0000","last_became_active":"2024-01-11T07:07:50.666316+0000","last_became_peered":"2024-01-11T07:07:50.666316+0000","last_unstale":"2024-01-11T07:07:50.667585+0000","last_undegraded":"2024-01-11T07:07:50.667585+0000","last_fullsized":"2024-01-11T07:07:50.667585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:35:14.001395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.672196+0000","last_change":"2024-01-11T07:07:50.672095+0000","last_active":"2024-01-11T07:07:50.672196+0000","last_peered":"2024-01-11T07:07:50.672196+0000","last_clean":"2024-01-11T07:07:50.672196+0000","last_became_active":"2024-01-11T07:07:50.671857+0000","last_became_peered":"2024-01-11T07:07:50.671857+0000","last_unstale":"2024-01-11T07:07:50.672196+0000","last_undegraded":"2024-01-11T07:07:50.672196+0000","last_fullsized":"2024-01-11T07:07:50.672196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:17:30.628791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667851+0000","last_change":"2024-01-11T07:07:50.667578+0000","last_active":"2024-01-11T07:07:50.667851+0000","last_peered":"2024-01-11T07:07:50.667851+0000","last_clean":"2024-01-11T07:07:50.667851+0000","last_became_active":"2024-01-11T07:07:50.667172+0000","last_became_peered":"2024-01-11T07:07:50.667172+0000","last_unstale":"2024-01-11T07:07:50.667851+0000","last_undegraded":"2024-01-11T07:07:50.667851+0000","last_fullsized":"2024-01-11T07:07:50.667851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:39:23.786711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666405+0000","last_change":"2024-01-11T07:07:50.665435+0000","last_active":"2024-01-11T07:07:50.666405+0000","last_peered":"2024-01-11T07:07:50.666405+0000","last_clean":"2024-01-11T07:07:50.666405+0000","last_became_active":"2024-01-11T07:07:50.665186+0000","last_became_peered":"2024-01-11T07:07:50.665186+0000","last_unstale":"2024-01-11T07:07:50.666405+0000","last_undegraded":"2024-01-11T07:07:50.666405+0000","last_fullsized":"2024-01-11T07:07:50.666405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:17:30.628791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666484+0000","last_change":"2024-01-11T07:07:50.666295+0000","last_active":"2024-01-11T07:07:50.666484+0000","last_peered":"2024-01-11T07:07:50.666484+0000","last_clean":"2024-01-11T07:07:50.666484+0000","last_became_active":"2024-01-11T07:07:50.665721+0000","last_became_peered":"2024-01-11T07:07:50.665721+0000","last_unstale":"2024-01-11T07:07:50.666484+0000","last_undegraded":"2024-01-11T07:07:50.666484+0000","last_fullsized":"2024-01-11T07:07:50.666484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:41:18.095732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666328+0000","last_change":"2024-01-11T07:07:50.666248+0000","last_active":"2024-01-11T07:07:50.666328+0000","last_peered":"2024-01-11T07:07:50.666328+0000","last_clean":"2024-01-11T07:07:50.666328+0000","last_became_active":"2024-01-11T07:07:50.665977+0000","last_became_peered":"2024-01-11T07:07:50.665977+0000","last_unstale":"2024-01-11T07:07:50.666328+0000","last_undegraded":"2024-01-11T07:07:50.666328+0000","last_fullsized":"2024-01-11T07:07:50.666328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:33:19.948536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.672763+0000","last_change":"2024-01-11T07:07:50.670883+0000","last_active":"2024-01-11T07:07:50.672763+0000","last_peered":"2024-01-11T07:07:50.672763+0000","last_clean":"2024-01-11T07:07:50.672763+0000","last_became_active":"2024-01-11T07:07:50.670622+0000","last_became_peered":"2024-01-11T07:07:50.670622+0000","last_unstale":"2024-01-11T07:07:50.672763+0000","last_undegraded":"2024-01-11T07:07:50.672763+0000","last_fullsized":"2024-01-11T07:07:50.672763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:34.294015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673436+0000","last_change":"2024-01-11T07:07:50.673365+0000","last_active":"2024-01-11T07:07:50.673436+0000","last_peered":"2024-01-11T07:07:50.673436+0000","last_clean":"2024-01-11T07:07:50.673436+0000","last_became_active":"2024-01-11T07:07:50.672859+0000","last_became_peered":"2024-01-11T07:07:50.672859+0000","last_unstale":"2024-01-11T07:07:50.673436+0000","last_undegraded":"2024-01-11T07:07:50.673436+0000","last_fullsized":"2024-01-11T07:07:50.673436+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:53.758682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668420+0000","last_change":"2024-01-11T07:07:50.668343+0000","last_active":"2024-01-11T07:07:50.668420+0000","last_peered":"2024-01-11T07:07:50.668420+0000","last_clean":"2024-01-11T07:07:50.668420+0000","last_became_active":"2024-01-11T07:07:50.667900+0000","last_became_peered":"2024-01-11T07:07:50.667900+0000","last_unstale":"2024-01-11T07:07:50.668420+0000","last_undegraded":"2024-01-11T07:07:50.668420+0000","last_fullsized":"2024-01-11T07:07:50.668420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:50:36.586926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667549+0000","last_change":"2024-01-11T07:07:50.665338+0000","last_active":"2024-01-11T07:07:50.667549+0000","last_peered":"2024-01-11T07:07:50.667549+0000","last_clean":"2024-01-11T07:07:50.667549+0000","last_became_active":"2024-01-11T07:07:50.665076+0000","last_became_peered":"2024-01-11T07:07:50.665076+0000","last_unstale":"2024-01-11T07:07:50.667549+0000","last_undegraded":"2024-01-11T07:07:50.667549+0000","last_fullsized":"2024-01-11T07:07:50.667549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:23.978674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.663375+0000","last_change":"2024-01-11T07:07:50.662657+0000","last_active":"2024-01-11T07:07:50.663375+0000","last_peered":"2024-01-11T07:07:50.663375+0000","last_clean":"2024-01-11T07:07:50.663375+0000","last_became_active":"2024-01-11T07:07:50.662395+0000","last_became_peered":"2024-01-11T07:07:50.662395+0000","last_unstale":"2024-01-11T07:07:50.663375+0000","last_undegraded":"2024-01-11T07:07:50.663375+0000","last_fullsized":"2024-01-11T07:07:50.663375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:45:34.065121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670764+0000","last_change":"2024-01-11T07:07:50.670470+0000","last_active":"2024-01-11T07:07:50.670764+0000","last_peered":"2024-01-11T07:07:50.670764+0000","last_clean":"2024-01-11T07:07:50.670764+0000","last_became_active":"2024-01-11T07:07:50.669782+0000","last_became_peered":"2024-01-11T07:07:50.669782+0000","last_unstale":"2024-01-11T07:07:50.670764+0000","last_undegraded":"2024-01-11T07:07:50.670764+0000","last_fullsized":"2024-01-11T07:07:50.670764+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:23.978674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674011+0000","last_change":"2024-01-11T07:07:50.673943+0000","last_active":"2024-01-11T07:07:50.674011+0000","last_peered":"2024-01-11T07:07:50.674011+0000","last_clean":"2024-01-11T07:07:50.674011+0000","last_became_active":"2024-01-11T07:07:50.673662+0000","last_became_peered":"2024-01-11T07:07:50.673662+0000","last_unstale":"2024-01-11T07:07:50.674011+0000","last_undegraded":"2024-01-11T07:07:50.674011+0000","last_fullsized":"2024-01-11T07:07:50.674011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:39:23.786711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670274+0000","last_change":"2024-01-11T07:07:50.670193+0000","last_active":"2024-01-11T07:07:50.670274+0000","last_peered":"2024-01-11T07:07:50.670274+0000","last_clean":"2024-01-11T07:07:50.670274+0000","last_became_active":"2024-01-11T07:07:50.669745+0000","last_became_peered":"2024-01-11T07:07:50.669745+0000","last_unstale":"2024-01-11T07:07:50.670274+0000","last_undegraded":"2024-01-11T07:07:50.670274+0000","last_fullsized":"2024-01-11T07:07:50.670274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:05.623217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673820+0000","last_change":"2024-01-11T07:07:50.673724+0000","last_active":"2024-01-11T07:07:50.673820+0000","last_peered":"2024-01-11T07:07:50.673820+0000","last_clean":"2024-01-11T07:07:50.673820+0000","last_became_active":"2024-01-11T07:07:50.672892+0000","last_became_peered":"2024-01-11T07:07:50.672892+0000","last_unstale":"2024-01-11T07:07:50.673820+0000","last_undegraded":"2024-01-11T07:07:50.673820+0000","last_fullsized":"2024-01-11T07:07:50.673820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:59.039502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668919+0000","last_change":"2024-01-11T07:07:50.668824+0000","last_active":"2024-01-11T07:07:50.668919+0000","last_peered":"2024-01-11T07:07:50.668919+0000","last_clean":"2024-01-11T07:07:50.668919+0000","last_became_active":"2024-01-11T07:07:50.668231+0000","last_became_peered":"2024-01-11T07:07:50.668231+0000","last_unstale":"2024-01-11T07:07:50.668919+0000","last_undegraded":"2024-01-11T07:07:50.668919+0000","last_fullsized":"2024-01-11T07:07:50.668919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:45:34.065121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.671207+0000","last_change":"2024-01-11T07:07:50.671135+0000","last_active":"2024-01-11T07:07:50.671207+0000","last_peered":"2024-01-11T07:07:50.671207+0000","last_clean":"2024-01-11T07:07:50.671207+0000","last_became_active":"2024-01-11T07:07:50.670891+0000","last_became_peered":"2024-01-11T07:07:50.670891+0000","last_unstale":"2024-01-11T07:07:50.671207+0000","last_undegraded":"2024-01-11T07:07:50.671207+0000","last_fullsized":"2024-01-11T07:07:50.671207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:34.294015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.675761+0000","last_change":"2024-01-11T07:07:50.675645+0000","last_active":"2024-01-11T07:07:50.675761+0000","last_peered":"2024-01-11T07:07:50.675761+0000","last_clean":"2024-01-11T07:07:50.675761+0000","last_became_active":"2024-01-11T07:07:50.675395+0000","last_became_peered":"2024-01-11T07:07:50.675395+0000","last_unstale":"2024-01-11T07:07:50.675761+0000","last_undegraded":"2024-01-11T07:07:50.675761+0000","last_fullsized":"2024-01-11T07:07:50.675761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:00.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668315+0000","last_change":"2024-01-11T07:07:50.668009+0000","last_active":"2024-01-11T07:07:50.668315+0000","last_peered":"2024-01-11T07:07:50.668315+0000","last_clean":"2024-01-11T07:07:50.668315+0000","last_became_active":"2024-01-11T07:07:50.667755+0000","last_became_peered":"2024-01-11T07:07:50.667755+0000","last_unstale":"2024-01-11T07:07:50.668315+0000","last_undegraded":"2024-01-11T07:07:50.668315+0000","last_fullsized":"2024-01-11T07:07:50.668315+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:53.758682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681867+0000","last_change":"2024-01-11T07:07:49.649839+0000","last_active":"2024-01-11T07:07:50.681867+0000","last_peered":"2024-01-11T07:07:50.681867+0000","last_clean":"2024-01-11T07:07:50.681867+0000","last_became_active":"2024-01-11T07:07:49.649617+0000","last_became_peered":"2024-01-11T07:07:49.649617+0000","last_unstale":"2024-01-11T07:07:50.681867+0000","last_undegraded":"2024-01-11T07:07:50.681867+0000","last_fullsized":"2024-01-11T07:07:50.681867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:28:34.194763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.675863+0000","last_change":"2024-01-11T07:07:50.675803+0000","last_active":"2024-01-11T07:07:50.675863+0000","last_peered":"2024-01-11T07:07:50.675863+0000","last_clean":"2024-01-11T07:07:50.675863+0000","last_became_active":"2024-01-11T07:07:50.675613+0000","last_became_peered":"2024-01-11T07:07:50.675613+0000","last_unstale":"2024-01-11T07:07:50.675863+0000","last_undegraded":"2024-01-11T07:07:50.675863+0000","last_fullsized":"2024-01-11T07:07:50.675863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:02:46.281372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681419+0000","last_change":"2024-01-11T07:07:49.652320+0000","last_active":"2024-01-11T07:07:50.681419+0000","last_peered":"2024-01-11T07:07:50.681419+0000","last_clean":"2024-01-11T07:07:50.681419+0000","last_became_active":"2024-01-11T07:07:49.651665+0000","last_became_peered":"2024-01-11T07:07:49.651665+0000","last_unstale":"2024-01-11T07:07:50.681419+0000","last_undegraded":"2024-01-11T07:07:50.681419+0000","last_fullsized":"2024-01-11T07:07:50.681419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:36:15.175122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673272+0000","last_change":"2024-01-11T07:07:50.673201+0000","last_active":"2024-01-11T07:07:50.673272+0000","last_peered":"2024-01-11T07:07:50.673272+0000","last_clean":"2024-01-11T07:07:50.673272+0000","last_became_active":"2024-01-11T07:07:50.672874+0000","last_became_peered":"2024-01-11T07:07:50.672874+0000","last_unstale":"2024-01-11T07:07:50.673272+0000","last_undegraded":"2024-01-11T07:07:50.673272+0000","last_fullsized":"2024-01-11T07:07:50.673272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:34:56.873661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.683033+0000","last_change":"2024-01-11T07:07:49.648156+0000","last_active":"2024-01-11T07:07:50.683033+0000","last_peered":"2024-01-11T07:07:50.683033+0000","last_clean":"2024-01-11T07:07:50.683033+0000","last_became_active":"2024-01-11T07:07:49.647891+0000","last_became_peered":"2024-01-11T07:07:49.647891+0000","last_unstale":"2024-01-11T07:07:50.683033+0000","last_undegraded":"2024-01-11T07:07:50.683033+0000","last_fullsized":"2024-01-11T07:07:50.683033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:08:04.062174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674942+0000","last_change":"2024-01-11T07:07:50.674868+0000","last_active":"2024-01-11T07:07:50.674942+0000","last_peered":"2024-01-11T07:07:50.674942+0000","last_clean":"2024-01-11T07:07:50.674942+0000","last_became_active":"2024-01-11T07:07:50.674658+0000","last_became_peered":"2024-01-11T07:07:50.674658+0000","last_unstale":"2024-01-11T07:07:50.674942+0000","last_undegraded":"2024-01-11T07:07:50.674942+0000","last_fullsized":"2024-01-11T07:07:50.674942+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:23:47.530004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.682450+0000","last_change":"2024-01-11T07:07:49.650961+0000","last_active":"2024-01-11T07:07:50.682450+0000","last_peered":"2024-01-11T07:07:50.682450+0000","last_clean":"2024-01-11T07:07:50.682450+0000","last_became_active":"2024-01-11T07:07:49.650703+0000","last_became_peered":"2024-01-11T07:07:49.650703+0000","last_unstale":"2024-01-11T07:07:50.682450+0000","last_undegraded":"2024-01-11T07:07:50.682450+0000","last_fullsized":"2024-01-11T07:07:50.682450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:42:11.078177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.676424+0000","last_change":"2024-01-11T07:07:50.676352+0000","last_active":"2024-01-11T07:07:50.676424+0000","last_peered":"2024-01-11T07:07:50.676424+0000","last_clean":"2024-01-11T07:07:50.676424+0000","last_became_active":"2024-01-11T07:07:50.675634+0000","last_became_peered":"2024-01-11T07:07:50.675634+0000","last_unstale":"2024-01-11T07:07:50.676424+0000","last_undegraded":"2024-01-11T07:07:50.676424+0000","last_fullsized":"2024-01-11T07:07:50.676424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:05.623217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681828+0000","last_change":"2024-01-11T07:07:49.650518+0000","last_active":"2024-01-11T07:07:50.681828+0000","last_peered":"2024-01-11T07:07:50.681828+0000","last_clean":"2024-01-11T07:07:50.681828+0000","last_became_active":"2024-01-11T07:07:49.649330+0000","last_became_peered":"2024-01-11T07:07:49.649330+0000","last_unstale":"2024-01-11T07:07:50.681828+0000","last_undegraded":"2024-01-11T07:07:50.681828+0000","last_fullsized":"2024-01-11T07:07:50.681828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:21:44.570385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669907+0000","last_change":"2024-01-11T07:07:50.669848+0000","last_active":"2024-01-11T07:07:50.669907+0000","last_peered":"2024-01-11T07:07:50.669907+0000","last_clean":"2024-01-11T07:07:50.669907+0000","last_became_active":"2024-01-11T07:07:50.669293+0000","last_became_peered":"2024-01-11T07:07:50.669293+0000","last_unstale":"2024-01-11T07:07:50.669907+0000","last_undegraded":"2024-01-11T07:07:50.669907+0000","last_fullsized":"2024-01-11T07:07:50.669907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:07:01.176205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681995+0000","last_change":"2024-01-11T07:07:49.647983+0000","last_active":"2024-01-11T07:07:50.681995+0000","last_peered":"2024-01-11T07:07:50.681995+0000","last_clean":"2024-01-11T07:07:50.681995+0000","last_became_active":"2024-01-11T07:07:49.647278+0000","last_became_peered":"2024-01-11T07:07:49.647278+0000","last_unstale":"2024-01-11T07:07:50.681995+0000","last_undegraded":"2024-01-11T07:07:50.681995+0000","last_fullsized":"2024-01-11T07:07:50.681995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:12:14.981796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673985+0000","last_change":"2024-01-11T07:07:50.673911+0000","last_active":"2024-01-11T07:07:50.673985+0000","last_peered":"2024-01-11T07:07:50.673985+0000","last_clean":"2024-01-11T07:07:50.673985+0000","last_became_active":"2024-01-11T07:07:50.673444+0000","last_became_peered":"2024-01-11T07:07:50.673444+0000","last_unstale":"2024-01-11T07:07:50.673985+0000","last_undegraded":"2024-01-11T07:07:50.673985+0000","last_fullsized":"2024-01-11T07:07:50.673985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:38.222386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.662928+0000","last_change":"2024-01-11T07:07:49.652067+0000","last_active":"2024-01-11T07:07:50.662928+0000","last_peered":"2024-01-11T07:07:50.662928+0000","last_clean":"2024-01-11T07:07:50.662928+0000","last_became_active":"2024-01-11T07:07:49.649322+0000","last_became_peered":"2024-01-11T07:07:49.649322+0000","last_unstale":"2024-01-11T07:07:50.662928+0000","last_undegraded":"2024-01-11T07:07:50.662928+0000","last_fullsized":"2024-01-11T07:07:50.662928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:33.284716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.671426+0000","last_change":"2024-01-11T07:07:50.668807+0000","last_active":"2024-01-11T07:07:50.671426+0000","last_peered":"2024-01-11T07:07:50.671426+0000","last_clean":"2024-01-11T07:07:50.671426+0000","last_became_active":"2024-01-11T07:07:50.668488+0000","last_became_peered":"2024-01-11T07:07:50.668488+0000","last_unstale":"2024-01-11T07:07:50.671426+0000","last_undegraded":"2024-01-11T07:07:50.671426+0000","last_fullsized":"2024-01-11T07:07:50.671426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:39.267002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668558+0000","last_change":"2024-01-11T07:07:49.645975+0000","last_active":"2024-01-11T07:07:50.668558+0000","last_peered":"2024-01-11T07:07:50.668558+0000","last_clean":"2024-01-11T07:07:50.668558+0000","last_became_active":"2024-01-11T07:07:49.645460+0000","last_became_peered":"2024-01-11T07:07:49.645460+0000","last_unstale":"2024-01-11T07:07:50.668558+0000","last_undegraded":"2024-01-11T07:07:50.668558+0000","last_fullsized":"2024-01-11T07:07:50.668558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:25:58.602861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667602+0000","last_change":"2024-01-11T07:07:50.667147+0000","last_active":"2024-01-11T07:07:50.667602+0000","last_peered":"2024-01-11T07:07:50.667602+0000","last_clean":"2024-01-11T07:07:50.667602+0000","last_became_active":"2024-01-11T07:07:50.666901+0000","last_became_peered":"2024-01-11T07:07:50.666901+0000","last_unstale":"2024-01-11T07:07:50.667602+0000","last_undegraded":"2024-01-11T07:07:50.667602+0000","last_fullsized":"2024-01-11T07:07:50.667602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:56:45.059589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668335+0000","last_change":"2024-01-11T07:07:49.654073+0000","last_active":"2024-01-11T07:07:50.668335+0000","last_peered":"2024-01-11T07:07:50.668335+0000","last_clean":"2024-01-11T07:07:50.668335+0000","last_became_active":"2024-01-11T07:07:49.652864+0000","last_became_peered":"2024-01-11T07:07:49.652864+0000","last_unstale":"2024-01-11T07:07:50.668335+0000","last_undegraded":"2024-01-11T07:07:50.668335+0000","last_fullsized":"2024-01-11T07:07:50.668335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:50:35.577627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670747+0000","last_change":"2024-01-11T07:07:50.670324+0000","last_active":"2024-01-11T07:07:50.670747+0000","last_peered":"2024-01-11T07:07:50.670747+0000","last_clean":"2024-01-11T07:07:50.670747+0000","last_became_active":"2024-01-11T07:07:50.669495+0000","last_became_peered":"2024-01-11T07:07:50.669495+0000","last_unstale":"2024-01-11T07:07:50.670747+0000","last_undegraded":"2024-01-11T07:07:50.670747+0000","last_fullsized":"2024-01-11T07:07:50.670747+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:07:01.176205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.656700+0000","last_change":"2024-01-11T07:07:49.650880+0000","last_active":"2024-01-11T07:07:50.656700+0000","last_peered":"2024-01-11T07:07:50.656700+0000","last_clean":"2024-01-11T07:07:50.656700+0000","last_became_active":"2024-01-11T07:07:49.650581+0000","last_became_peered":"2024-01-11T07:07:49.650581+0000","last_unstale":"2024-01-11T07:07:50.656700+0000","last_undegraded":"2024-01-11T07:07:50.656700+0000","last_fullsized":"2024-01-11T07:07:50.656700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:55:42.742639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667620+0000","last_change":"2024-01-11T07:07:50.667504+0000","last_active":"2024-01-11T07:07:50.667620+0000","last_peered":"2024-01-11T07:07:50.667620+0000","last_clean":"2024-01-11T07:07:50.667620+0000","last_became_active":"2024-01-11T07:07:50.667268+0000","last_became_peered":"2024-01-11T07:07:50.667268+0000","last_unstale":"2024-01-11T07:07:50.667620+0000","last_undegraded":"2024-01-11T07:07:50.667620+0000","last_fullsized":"2024-01-11T07:07:50.667620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:11:27.491967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.680388+0000","last_change":"2024-01-11T07:07:49.649102+0000","last_active":"2024-01-11T07:07:50.680388+0000","last_peered":"2024-01-11T07:07:50.680388+0000","last_clean":"2024-01-11T07:07:50.680388+0000","last_became_active":"2024-01-11T07:07:49.648353+0000","last_became_peered":"2024-01-11T07:07:49.648353+0000","last_unstale":"2024-01-11T07:07:50.680388+0000","last_undegraded":"2024-01-11T07:07:50.680388+0000","last_fullsized":"2024-01-11T07:07:50.680388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:33.284716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670256+0000","last_change":"2024-01-11T07:07:50.670036+0000","last_active":"2024-01-11T07:07:50.670256+0000","last_peered":"2024-01-11T07:07:50.670256+0000","last_clean":"2024-01-11T07:07:50.670256+0000","last_became_active":"2024-01-11T07:07:50.669431+0000","last_became_peered":"2024-01-11T07:07:50.669431+0000","last_unstale":"2024-01-11T07:07:50.670256+0000","last_undegraded":"2024-01-11T07:07:50.670256+0000","last_fullsized":"2024-01-11T07:07:50.670256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:00.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.684447+0000","last_change":"2024-01-11T07:07:49.652256+0000","last_active":"2024-01-11T07:07:50.684447+0000","last_peered":"2024-01-11T07:07:50.684447+0000","last_clean":"2024-01-11T07:07:50.684447+0000","last_became_active":"2024-01-11T07:07:49.650792+0000","last_became_peered":"2024-01-11T07:07:49.650792+0000","last_unstale":"2024-01-11T07:07:50.684447+0000","last_undegraded":"2024-01-11T07:07:50.684447+0000","last_fullsized":"2024-01-11T07:07:50.684447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:36.267494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668299+0000","last_change":"2024-01-11T07:07:50.667824+0000","last_active":"2024-01-11T07:07:50.668299+0000","last_peered":"2024-01-11T07:07:50.668299+0000","last_clean":"2024-01-11T07:07:50.668299+0000","last_became_active":"2024-01-11T07:07:50.667197+0000","last_became_peered":"2024-01-11T07:07:50.667197+0000","last_unstale":"2024-01-11T07:07:50.668299+0000","last_undegraded":"2024-01-11T07:07:50.668299+0000","last_fullsized":"2024-01-11T07:07:50.668299+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:59.039502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.682136+0000","last_change":"2024-01-11T07:07:49.645809+0000","last_active":"2024-01-11T07:07:50.682136+0000","last_peered":"2024-01-11T07:07:50.682136+0000","last_clean":"2024-01-11T07:07:50.682136+0000","last_became_active":"2024-01-11T07:07:49.645185+0000","last_became_peered":"2024-01-11T07:07:49.645185+0000","last_unstale":"2024-01-11T07:07:50.682136+0000","last_undegraded":"2024-01-11T07:07:50.682136+0000","last_fullsized":"2024-01-11T07:07:50.682136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:28:34.194763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668275+0000","last_change":"2024-01-11T07:07:50.668176+0000","last_active":"2024-01-11T07:07:50.668275+0000","last_peered":"2024-01-11T07:07:50.668275+0000","last_clean":"2024-01-11T07:07:50.668275+0000","last_became_active":"2024-01-11T07:07:50.667850+0000","last_became_peered":"2024-01-11T07:07:50.667850+0000","last_unstale":"2024-01-11T07:07:50.668275+0000","last_undegraded":"2024-01-11T07:07:50.668275+0000","last_fullsized":"2024-01-11T07:07:50.668275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:39.267002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.659902+0000","last_change":"2024-01-11T07:07:49.647297+0000","last_active":"2024-01-11T07:07:50.659902+0000","last_peered":"2024-01-11T07:07:50.659902+0000","last_clean":"2024-01-11T07:07:50.659902+0000","last_became_active":"2024-01-11T07:07:49.646536+0000","last_became_peered":"2024-01-11T07:07:49.646536+0000","last_unstale":"2024-01-11T07:07:50.659902+0000","last_undegraded":"2024-01-11T07:07:50.659902+0000","last_fullsized":"2024-01-11T07:07:50.659902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:28.539684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.675959+0000","last_change":"2024-01-11T07:07:50.675880+0000","last_active":"2024-01-11T07:07:50.675959+0000","last_peered":"2024-01-11T07:07:50.675959+0000","last_clean":"2024-01-11T07:07:50.675959+0000","last_became_active":"2024-01-11T07:07:50.675308+0000","last_became_peered":"2024-01-11T07:07:50.675308+0000","last_unstale":"2024-01-11T07:07:50.675959+0000","last_undegraded":"2024-01-11T07:07:50.675959+0000","last_fullsized":"2024-01-11T07:07:50.675959+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:59:47.509012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.660405+0000","last_change":"2024-01-11T07:07:49.650024+0000","last_active":"2024-01-11T07:07:50.660405+0000","last_peered":"2024-01-11T07:07:50.660405+0000","last_clean":"2024-01-11T07:07:50.660405+0000","last_became_active":"2024-01-11T07:07:49.649691+0000","last_became_peered":"2024-01-11T07:07:49.649691+0000","last_unstale":"2024-01-11T07:07:50.660405+0000","last_undegraded":"2024-01-11T07:07:50.660405+0000","last_fullsized":"2024-01-11T07:07:50.660405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:28.539684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.677026+0000","last_change":"2024-01-11T07:07:50.676903+0000","last_active":"2024-01-11T07:07:50.677026+0000","last_peered":"2024-01-11T07:07:50.677026+0000","last_clean":"2024-01-11T07:07:50.677026+0000","last_became_active":"2024-01-11T07:07:50.676625+0000","last_became_peered":"2024-01-11T07:07:50.676625+0000","last_unstale":"2024-01-11T07:07:50.677026+0000","last_undegraded":"2024-01-11T07:07:50.677026+0000","last_fullsized":"2024-01-11T07:07:50.677026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:06:54.741409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.684656+0000","last_change":"2024-01-11T07:07:49.646993+0000","last_active":"2024-01-11T07:07:50.684656+0000","last_peered":"2024-01-11T07:07:50.684656+0000","last_clean":"2024-01-11T07:07:50.684656+0000","last_became_active":"2024-01-11T07:07:49.646720+0000","last_became_peered":"2024-01-11T07:07:49.646720+0000","last_unstale":"2024-01-11T07:07:50.684656+0000","last_undegraded":"2024-01-11T07:07:50.684656+0000","last_fullsized":"2024-01-11T07:07:50.684656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:24.731102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673855+0000","last_change":"2024-01-11T07:07:50.673760+0000","last_active":"2024-01-11T07:07:50.673855+0000","last_peered":"2024-01-11T07:07:50.673855+0000","last_clean":"2024-01-11T07:07:50.673855+0000","last_became_active":"2024-01-11T07:07:50.672827+0000","last_became_peered":"2024-01-11T07:07:50.672827+0000","last_unstale":"2024-01-11T07:07:50.673855+0000","last_undegraded":"2024-01-11T07:07:50.673855+0000","last_fullsized":"2024-01-11T07:07:50.673855+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:50:36.586926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.683493+0000","last_change":"2024-01-11T07:07:49.655412+0000","last_active":"2024-01-11T07:07:50.683493+0000","last_peered":"2024-01-11T07:07:50.683493+0000","last_clean":"2024-01-11T07:07:50.683493+0000","last_became_active":"2024-01-11T07:07:49.655226+0000","last_became_peered":"2024-01-11T07:07:49.655226+0000","last_unstale":"2024-01-11T07:07:50.683493+0000","last_undegraded":"2024-01-11T07:07:50.683493+0000","last_fullsized":"2024-01-11T07:07:50.683493+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:04:09.767719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674340+0000","last_change":"2024-01-11T07:07:50.674273+0000","last_active":"2024-01-11T07:07:50.674340+0000","last_peered":"2024-01-11T07:07:50.674340+0000","last_clean":"2024-01-11T07:07:50.674340+0000","last_became_active":"2024-01-11T07:07:50.674057+0000","last_became_peered":"2024-01-11T07:07:50.674057+0000","last_unstale":"2024-01-11T07:07:50.674340+0000","last_undegraded":"2024-01-11T07:07:50.674340+0000","last_fullsized":"2024-01-11T07:07:50.674340+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:39:23.786711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681917+0000","last_change":"2024-01-11T07:07:49.645682+0000","last_active":"2024-01-11T07:07:50.681917+0000","last_peered":"2024-01-11T07:07:50.681917+0000","last_clean":"2024-01-11T07:07:50.681917+0000","last_became_active":"2024-01-11T07:07:49.644880+0000","last_became_peered":"2024-01-11T07:07:49.644880+0000","last_unstale":"2024-01-11T07:07:50.681917+0000","last_undegraded":"2024-01-11T07:07:50.681917+0000","last_fullsized":"2024-01-11T07:07:50.681917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:53:57.003779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673332+0000","last_change":"2024-01-11T07:07:50.673259+0000","last_active":"2024-01-11T07:07:50.673332+0000","last_peered":"2024-01-11T07:07:50.673332+0000","last_clean":"2024-01-11T07:07:50.673332+0000","last_became_active":"2024-01-11T07:07:50.673030+0000","last_became_peered":"2024-01-11T07:07:50.673030+0000","last_unstale":"2024-01-11T07:07:50.673332+0000","last_undegraded":"2024-01-11T07:07:50.673332+0000","last_fullsized":"2024-01-11T07:07:50.673332+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:42:49.114805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.682903+0000","last_change":"2024-01-11T07:07:49.651771+0000","last_active":"2024-01-11T07:07:50.682903+0000","last_peered":"2024-01-11T07:07:50.682903+0000","last_clean":"2024-01-11T07:07:50.682903+0000","last_became_active":"2024-01-11T07:07:49.647635+0000","last_became_peered":"2024-01-11T07:07:49.647635+0000","last_unstale":"2024-01-11T07:07:50.682903+0000","last_undegraded":"2024-01-11T07:07:50.682903+0000","last_fullsized":"2024-01-11T07:07:50.682903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:40.362580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.664284+0000","last_change":"2024-01-11T07:07:50.664005+0000","last_active":"2024-01-11T07:07:50.664284+0000","last_peered":"2024-01-11T07:07:50.664284+0000","last_clean":"2024-01-11T07:07:50.664284+0000","last_became_active":"2024-01-11T07:07:50.663280+0000","last_became_peered":"2024-01-11T07:07:50.663280+0000","last_unstale":"2024-01-11T07:07:50.664284+0000","last_undegraded":"2024-01-11T07:07:50.664284+0000","last_fullsized":"2024-01-11T07:07:50.664284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:55.653328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.663873+0000","last_change":"2024-01-11T07:07:49.650692+0000","last_active":"2024-01-11T07:07:50.663873+0000","last_peered":"2024-01-11T07:07:50.663873+0000","last_clean":"2024-01-11T07:07:50.663873+0000","last_became_active":"2024-01-11T07:07:49.649691+0000","last_became_peered":"2024-01-11T07:07:49.649691+0000","last_unstale":"2024-01-11T07:07:50.663873+0000","last_undegraded":"2024-01-11T07:07:50.663873+0000","last_fullsized":"2024-01-11T07:07:50.663873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:01:38.347171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674494+0000","last_change":"2024-01-11T07:07:50.674411+0000","last_active":"2024-01-11T07:07:50.674494+0000","last_peered":"2024-01-11T07:07:50.674494+0000","last_clean":"2024-01-11T07:07:50.674494+0000","last_became_active":"2024-01-11T07:07:50.674129+0000","last_became_peered":"2024-01-11T07:07:50.674129+0000","last_unstale":"2024-01-11T07:07:50.674494+0000","last_undegraded":"2024-01-11T07:07:50.674494+0000","last_fullsized":"2024-01-11T07:07:50.674494+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:25:59.612160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.682508+0000","last_change":"2024-01-11T07:07:49.649183+0000","last_active":"2024-01-11T07:07:50.682508+0000","last_peered":"2024-01-11T07:07:50.682508+0000","last_clean":"2024-01-11T07:07:50.682508+0000","last_became_active":"2024-01-11T07:07:49.647457+0000","last_became_peered":"2024-01-11T07:07:49.647457+0000","last_unstale":"2024-01-11T07:07:50.682508+0000","last_undegraded":"2024-01-11T07:07:50.682508+0000","last_fullsized":"2024-01-11T07:07:50.682508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:04:09.767719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667593+0000","last_change":"2024-01-11T07:07:50.667481+0000","last_active":"2024-01-11T07:07:50.667593+0000","last_peered":"2024-01-11T07:07:50.667593+0000","last_clean":"2024-01-11T07:07:50.667593+0000","last_became_active":"2024-01-11T07:07:50.666780+0000","last_became_peered":"2024-01-11T07:07:50.666780+0000","last_unstale":"2024-01-11T07:07:50.667593+0000","last_undegraded":"2024-01-11T07:07:50.667593+0000","last_fullsized":"2024-01-11T07:07:50.667593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:53:58.013078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.660414+0000","last_change":"2024-01-11T07:07:49.651247+0000","last_active":"2024-01-11T07:07:50.660414+0000","last_peered":"2024-01-11T07:07:50.660414+0000","last_clean":"2024-01-11T07:07:50.660414+0000","last_became_active":"2024-01-11T07:07:49.651010+0000","last_became_peered":"2024-01-11T07:07:49.651010+0000","last_unstale":"2024-01-11T07:07:50.660414+0000","last_undegraded":"2024-01-11T07:07:50.660414+0000","last_fullsized":"2024-01-11T07:07:50.660414+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:25:58.602861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.663689+0000","last_change":"2024-01-11T07:07:50.663030+0000","last_active":"2024-01-11T07:07:50.663689+0000","last_peered":"2024-01-11T07:07:50.663689+0000","last_clean":"2024-01-11T07:07:50.663689+0000","last_became_active":"2024-01-11T07:07:50.662740+0000","last_became_peered":"2024-01-11T07:07:50.662740+0000","last_unstale":"2024-01-11T07:07:50.663689+0000","last_undegraded":"2024-01-11T07:07:50.663689+0000","last_fullsized":"2024-01-11T07:07:50.663689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:15:03.302073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668757+0000","last_change":"2024-01-11T07:07:49.652646+0000","last_active":"2024-01-11T07:07:50.668757+0000","last_peered":"2024-01-11T07:07:50.668757+0000","last_clean":"2024-01-11T07:07:50.668757+0000","last_became_active":"2024-01-11T07:07:49.652414+0000","last_became_peered":"2024-01-11T07:07:49.652414+0000","last_unstale":"2024-01-11T07:07:50.668757+0000","last_undegraded":"2024-01-11T07:07:50.668757+0000","last_fullsized":"2024-01-11T07:07:50.668757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:15:02.292774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669074+0000","last_change":"2024-01-11T07:07:50.668983+0000","last_active":"2024-01-11T07:07:50.669074+0000","last_peered":"2024-01-11T07:07:50.669074+0000","last_clean":"2024-01-11T07:07:50.669074+0000","last_became_active":"2024-01-11T07:07:50.668565+0000","last_became_peered":"2024-01-11T07:07:50.668565+0000","last_unstale":"2024-01-11T07:07:50.669074+0000","last_undegraded":"2024-01-11T07:07:50.669074+0000","last_fullsized":"2024-01-11T07:07:50.669074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:23:47.530004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.660897+0000","last_change":"2024-01-11T07:07:49.650617+0000","last_active":"2024-01-11T07:07:50.660897+0000","last_peered":"2024-01-11T07:07:50.660897+0000","last_clean":"2024-01-11T07:07:50.660897+0000","last_became_active":"2024-01-11T07:07:49.650368+0000","last_became_peered":"2024-01-11T07:07:49.650368+0000","last_unstale":"2024-01-11T07:07:50.660897+0000","last_undegraded":"2024-01-11T07:07:50.660897+0000","last_fullsized":"2024-01-11T07:07:50.660897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:08:04.062174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667574+0000","last_change":"2024-01-11T07:07:50.667315+0000","last_active":"2024-01-11T07:07:50.667574+0000","last_peered":"2024-01-11T07:07:50.667574+0000","last_clean":"2024-01-11T07:07:50.667574+0000","last_became_active":"2024-01-11T07:07:50.666407+0000","last_became_peered":"2024-01-11T07:07:50.666407+0000","last_unstale":"2024-01-11T07:07:50.667574+0000","last_undegraded":"2024-01-11T07:07:50.667574+0000","last_fullsized":"2024-01-11T07:07:50.667574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:55:43.751938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.683152+0000","last_change":"2024-01-11T07:07:49.649435+0000","last_active":"2024-01-11T07:07:50.683152+0000","last_peered":"2024-01-11T07:07:50.683152+0000","last_clean":"2024-01-11T07:07:50.683152+0000","last_became_active":"2024-01-11T07:07:49.649030+0000","last_became_peered":"2024-01-11T07:07:49.649030+0000","last_unstale":"2024-01-11T07:07:50.683152+0000","last_undegraded":"2024-01-11T07:07:50.683152+0000","last_fullsized":"2024-01-11T07:07:50.683152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:54.644029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.672577+0000","last_change":"2024-01-11T07:07:50.672455+0000","last_active":"2024-01-11T07:07:50.672577+0000","last_peered":"2024-01-11T07:07:50.672577+0000","last_clean":"2024-01-11T07:07:50.672577+0000","last_became_active":"2024-01-11T07:07:50.672220+0000","last_became_peered":"2024-01-11T07:07:50.672220+0000","last_unstale":"2024-01-11T07:07:50.672577+0000","last_undegraded":"2024-01-11T07:07:50.672577+0000","last_fullsized":"2024-01-11T07:07:50.672577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:17:30.628791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-11T07:07:51.658443+0000","last_change":"2024-01-11T07:07:49.648998+0000","last_active":"2024-01-11T07:07:51.658443+0000","last_peered":"2024-01-11T07:07:51.658443+0000","last_clean":"2024-01-11T07:07:51.658443+0000","last_became_active":"2024-01-11T07:07:49.647096+0000","last_became_peered":"2024-01-11T07:07:49.647096+0000","last_unstale":"2024-01-11T07:07:51.658443+0000","last_undegraded":"2024-01-11T07:07:51.658443+0000","last_fullsized":"2024-01-11T07:07:51.658443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:28.539684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.662017+0000","last_change":"2024-01-11T07:07:50.661214+0000","last_active":"2024-01-11T07:07:50.662017+0000","last_peered":"2024-01-11T07:07:50.662017+0000","last_clean":"2024-01-11T07:07:50.662017+0000","last_became_active":"2024-01-11T07:07:50.660970+0000","last_became_peered":"2024-01-11T07:07:50.660970+0000","last_unstale":"2024-01-11T07:07:50.662017+0000","last_undegraded":"2024-01-11T07:07:50.662017+0000","last_fullsized":"2024-01-11T07:07:50.662017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:37.276793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.659588+0000","last_change":"2024-01-11T07:07:49.647565+0000","last_active":"2024-01-11T07:07:50.659588+0000","last_peered":"2024-01-11T07:07:50.659588+0000","last_clean":"2024-01-11T07:07:50.659588+0000","last_became_active":"2024-01-11T07:07:49.647080+0000","last_became_peered":"2024-01-11T07:07:49.647080+0000","last_unstale":"2024-01-11T07:07:50.659588+0000","last_undegraded":"2024-01-11T07:07:50.659588+0000","last_fullsized":"2024-01-11T07:07:50.659588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:36:15.175122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669487+0000","last_change":"2024-01-11T07:07:50.669391+0000","last_active":"2024-01-11T07:07:50.669487+0000","last_peered":"2024-01-11T07:07:50.669487+0000","last_clean":"2024-01-11T07:07:50.669487+0000","last_became_active":"2024-01-11T07:07:50.668266+0000","last_became_peered":"2024-01-11T07:07:50.668266+0000","last_unstale":"2024-01-11T07:07:50.669487+0000","last_undegraded":"2024-01-11T07:07:50.669487+0000","last_fullsized":"2024-01-11T07:07:50.669487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:45:46.131204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.657140+0000","last_change":"2024-01-11T07:07:49.648524+0000","last_active":"2024-01-11T07:07:50.657140+0000","last_peered":"2024-01-11T07:07:50.657140+0000","last_clean":"2024-01-11T07:07:50.657140+0000","last_became_active":"2024-01-11T07:07:49.648282+0000","last_became_peered":"2024-01-11T07:07:49.648282+0000","last_unstale":"2024-01-11T07:07:50.657140+0000","last_undegraded":"2024-01-11T07:07:50.657140+0000","last_fullsized":"2024-01-11T07:07:50.657140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:54.644029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668635+0000","last_change":"2024-01-11T07:07:50.668528+0000","last_active":"2024-01-11T07:07:50.668635+0000","last_peered":"2024-01-11T07:07:50.668635+0000","last_clean":"2024-01-11T07:07:50.668635+0000","last_became_active":"2024-01-11T07:07:50.668291+0000","last_became_peered":"2024-01-11T07:07:50.668291+0000","last_unstale":"2024-01-11T07:07:50.668635+0000","last_undegraded":"2024-01-11T07:07:50.668635+0000","last_fullsized":"2024-01-11T07:07:50.668635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:38.222386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.659021+0000","last_change":"2024-01-11T07:07:49.649273+0000","last_active":"2024-01-11T07:07:50.659021+0000","last_peered":"2024-01-11T07:07:50.659021+0000","last_clean":"2024-01-11T07:07:50.659021+0000","last_became_active":"2024-01-11T07:07:49.648633+0000","last_became_peered":"2024-01-11T07:07:49.648633+0000","last_unstale":"2024-01-11T07:07:50.659021+0000","last_undegraded":"2024-01-11T07:07:50.659021+0000","last_fullsized":"2024-01-11T07:07:50.659021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:55:42.742639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666468+0000","last_change":"2024-01-11T07:07:50.666135+0000","last_active":"2024-01-11T07:07:50.666468+0000","last_peered":"2024-01-11T07:07:50.666468+0000","last_clean":"2024-01-11T07:07:50.666468+0000","last_became_active":"2024-01-11T07:07:50.665386+0000","last_became_peered":"2024-01-11T07:07:50.665386+0000","last_unstale":"2024-01-11T07:07:50.666468+0000","last_undegraded":"2024-01-11T07:07:50.666468+0000","last_fullsized":"2024-01-11T07:07:50.666468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:16:07.194652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670514+0000","last_change":"2024-01-11T07:07:49.649920+0000","last_active":"2024-01-11T07:07:50.670514+0000","last_peered":"2024-01-11T07:07:50.670514+0000","last_clean":"2024-01-11T07:07:50.670514+0000","last_became_active":"2024-01-11T07:07:49.649674+0000","last_became_peered":"2024-01-11T07:07:49.649674+0000","last_unstale":"2024-01-11T07:07:50.670514+0000","last_undegraded":"2024-01-11T07:07:50.670514+0000","last_fullsized":"2024-01-11T07:07:50.670514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:18:25.817021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673457+0000","last_change":"2024-01-11T07:07:50.673375+0000","last_active":"2024-01-11T07:07:50.673457+0000","last_peered":"2024-01-11T07:07:50.673457+0000","last_clean":"2024-01-11T07:07:50.673457+0000","last_became_active":"2024-01-11T07:07:50.672959+0000","last_became_peered":"2024-01-11T07:07:50.672959+0000","last_unstale":"2024-01-11T07:07:50.673457+0000","last_undegraded":"2024-01-11T07:07:50.673457+0000","last_fullsized":"2024-01-11T07:07:50.673457+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:37.276793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.662817+0000","last_change":"2024-01-11T07:07:49.652438+0000","last_active":"2024-01-11T07:07:50.662817+0000","last_peered":"2024-01-11T07:07:50.662817+0000","last_clean":"2024-01-11T07:07:50.662817+0000","last_became_active":"2024-01-11T07:07:49.651815+0000","last_became_peered":"2024-01-11T07:07:49.651815+0000","last_unstale":"2024-01-11T07:07:50.662817+0000","last_undegraded":"2024-01-11T07:07:50.662817+0000","last_fullsized":"2024-01-11T07:07:50.662817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:37.213087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666283+0000","last_change":"2024-01-11T07:07:50.666204+0000","last_active":"2024-01-11T07:07:50.666283+0000","last_peered":"2024-01-11T07:07:50.666283+0000","last_clean":"2024-01-11T07:07:50.666283+0000","last_became_active":"2024-01-11T07:07:50.665960+0000","last_became_peered":"2024-01-11T07:07:50.665960+0000","last_unstale":"2024-01-11T07:07:50.666283+0000","last_undegraded":"2024-01-11T07:07:50.666283+0000","last_fullsized":"2024-01-11T07:07:50.666283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:07:01.176205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.660528+0000","last_change":"2024-01-11T07:07:49.649057+0000","last_active":"2024-01-11T07:07:50.660528+0000","last_peered":"2024-01-11T07:07:50.660528+0000","last_clean":"2024-01-11T07:07:50.660528+0000","last_became_active":"2024-01-11T07:07:49.648785+0000","last_became_peered":"2024-01-11T07:07:49.648785+0000","last_unstale":"2024-01-11T07:07:50.660528+0000","last_undegraded":"2024-01-11T07:07:50.660528+0000","last_fullsized":"2024-01-11T07:07:50.660528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:15:02.292774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.662544+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:07:50.662544+0000","last_peered":"2024-01-11T07:07:50.662544+0000","last_clean":"2024-01-11T07:07:50.662544+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:07:50.662544+0000","last_undegraded":"2024-01-11T07:07:50.662544+0000","last_fullsized":"2024-01-11T07:07:50.662544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199260981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46700000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.71599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":145,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195166,"statfs":{"total":200000000,"available":199850851,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.49399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":146,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195165,"statfs":{"total":200000000,"available":199849881,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.438}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.503}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199261071,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.7960000000000003,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.62}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":8.0730000000000004,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":8.0399999999999991,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.819,"5min":2.819,"15min":2.819},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.87},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":8.0030000000000001,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.89500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":145,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195167,"statfs":{"total":200000000,"available":199851369,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63300000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":146,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195165,"statfs":{"total":200000000,"available":199849881,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":11.481,"5min":11.481,"15min":11.481},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":17.516999999999999,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.44},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":17.553000000000001,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.474,"5min":17.474,"15min":17.474},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":17.393000000000001,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":11.436999999999999,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":17.434000000000001,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":145,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195167,"statfs":{"total":200000000,"available":199851335,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":8.484,"5min":8.484,"15min":8.484},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.2599999999999998,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":8.5340000000000007,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":9.2279999999999998,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.751}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":9.5640000000000001,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.8639999999999999,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":8.9779999999999998,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.7309999999999999,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.388999999999999,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":8.6509999999999998,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1449999999999996,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.629}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.4640000000000004,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.61099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":139,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195172,"statfs":{"total":200000000,"available":199857083,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.78500000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83499999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.87}]},{"osd":4,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.63}]},{"osd":7,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.66400000000000003}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:54.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:07:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.126+0000 7f22c3f41700 1 Processor -- start 2024-01-11T07:07:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.127+0000 7f22c3f41700 1 -- start start 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.127+0000 7f22c3f41700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc0ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.127+0000 7f22c3f41700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22bc069d60 0x7f22bc0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.127+0000 7f22c3f41700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0ad300 con 0x7f22bc063db0 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.127+0000 7f22c3f41700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22bc063920 con 0x7f22bc069d60 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.127+0000 7f22c3f41700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0ad020 con 0x7f22bc155650 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c14dc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22bc069d60 0x7f22bc0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c14dc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22bc069d60 0x7f22bc0b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50622/0 (socket says 172.21.15.7:50622) 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c24de700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f22bc063db0 0x7f22bc06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45072/0 (socket says 172.21.15.7:45072) 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c14dc700 1 -- 172.21.15.7:0/2784116501 learned_addr learned my addr 172.21.15.7:0/2784116501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c1cdd700 1 --2- 172.21.15.7:0/2784116501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc0ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c14dc700 1 -- 172.21.15.7:0/2784116501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 msgr2=0x7f22bc0ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c14dc700 1 --2- 172.21.15.7:0/2784116501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc0ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c14dc700 1 -- 172.21.15.7:0/2784116501 >> v1:172.21.15.7:6789/0 conn(0x7f22bc063db0 legacy=0x7f22bc06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c14dc700 1 -- 172.21.15.7:0/2784116501 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0632d0 con 0x7f22bc069d60 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c1cdd700 1 --2- 172.21.15.7:0/2784116501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc0ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c0cdb700 1 -- 172.21.15.7:0/2784116501 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365578415 0 0) 0x7f22bc0ad300 con 0x7f22bc063db0 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c14dc700 1 --2- 172.21.15.7:0/2784116501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc069d60 0x7f22bc0b2b70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f22b80099b0 tx=0x7f22b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb1cee45c4729e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c0cdb700 1 -- 172.21.15.7:0/2784116501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b800b860 con 0x7f22bc069d60 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c0cdb700 1 -- 172.21.15.7:0/2784116501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b800b9c0 con 0x7f22bc069d60 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.128+0000 7f22c0cdb700 1 -- 172.21.15.7:0/2784116501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b800bd60 con 0x7f22bc069d60 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.129+0000 7f22c3f41700 1 -- 172.21.15.7:0/2784116501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc069d60 msgr2=0x7f22bc0b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.129+0000 7f22c3f41700 1 --2- 172.21.15.7:0/2784116501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc069d60 0x7f22bc0b2b70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f22b80099b0 tx=0x7f22b8009250 comp rx=0 tx=0).stop 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.129+0000 7f22c3f41700 1 -- 172.21.15.7:0/2784116501 shutdown_connections 2024-01-11T07:07:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.129+0000 7f22c3f41700 1 --2- 172.21.15.7:0/2784116501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc0ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.129+0000 7f22c3f41700 1 --2- 172.21.15.7:0/2784116501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc069d60 0x7f22bc0b2b70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.129+0000 7f22c3f41700 1 -- 172.21.15.7:0/2784116501 >> 172.21.15.7:0/2784116501 conn(0x7f22bc0cd910 msgr2=0x7f22bc0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.129+0000 7f22c3f41700 1 -- 172.21.15.7:0/2784116501 shutdown_connections 2024-01-11T07:07:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.129+0000 7f22c3f41700 1 -- 172.21.15.7:0/2784116501 wait complete. 2024-01-11T07:07:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.130+0000 7f22c3f41700 1 Processor -- start 2024-01-11T07:07:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.130+0000 7f22c3f41700 1 -- start start 2024-01-11T07:07:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c3f41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc063db0 0x7f22bc1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c3f41700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc069d60 0x7f22bc150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c3f41700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c3f41700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0ae8e0 con 0x7f22bc063db0 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c3f41700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0aebc0 con 0x7f22bc069d60 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c3f41700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0aeea0 con 0x7f22bc155650 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c1cdd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc063db0 0x7f22bc1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c1cdd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc063db0 0x7f22bc1492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50624/0 (socket says 172.21.15.7:50624) 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c1cdd700 1 -- 172.21.15.7:0/1449125062 learned_addr learned my addr 172.21.15.7:0/1449125062 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c14dc700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc069d60 0x7f22bc150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.131+0000 7f22c24de700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c1cdd700 1 -- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 msgr2=0x7f22bc15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c1cdd700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc15a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c1cdd700 1 -- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc069d60 msgr2=0x7f22bc150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c1cdd700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc069d60 0x7f22bc150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c1cdd700 1 -- 172.21.15.7:0/1449125062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b80094f0 con 0x7f22bc063db0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c14dc700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc069d60 0x7f22bc150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c1cdd700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc063db0 0x7f22bc1492c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00d860 tx=0x7f22ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22b1ffb700 1 -- 172.21.15.7:0/1449125062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0125f0 con 0x7f22bc063db0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22b1ffb700 1 -- 172.21.15.7:0/1449125062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac012750 con 0x7f22bc063db0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22b1ffb700 1 -- 172.21.15.7:0/1449125062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac012ad0 con 0x7f22bc063db0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc0af180 con 0x7f22bc063db0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.132+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc0ae500 con 0x7f22bc063db0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.133+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22bc14ed70 con 0x7f22bc063db0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.134+0000 7f22b1ffb700 1 -- 172.21.15.7:0/1449125062 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f22ac014430 con 0x7f22bc063db0 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.134+0000 7f22b1ffb700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f22a0066f30 0x7f22a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.134+0000 7f22b1ffb700 1 -- 172.21.15.7:0/1449125062 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22ac084980 con 0x7f22bc063db0 2024-01-11T07:07:55.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.134+0000 7f22c14dc700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f22a0066f30 0x7f22a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.135+0000 7f22c14dc700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f22a0066f30 0x7f22a00693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22b80099b0 tx=0x7f22b8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:55.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.139+0000 7f22b1ffb700 1 -- 172.21.15.7:0/1449125062 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22ac055680 con 0x7f22bc063db0 2024-01-11T07:07:55.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.336+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22bc0aed30 con 0x7f22a0066f30 2024-01-11T07:07:55.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.345+0000 7f22b1ffb700 1 -- 172.21.15.7:0/1449125062 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284856 (secure 0 0 0) 0x7f22bc0aed30 con 0x7f22a0066f30 2024-01-11T07:07:55.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f22a0066f30 msgr2=0x7f22a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f22a0066f30 0x7f22a00693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22b80099b0 tx=0x7f22b8017000 comp rx=0 tx=0).stop 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc063db0 msgr2=0x7f22bc1492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc063db0 0x7f22bc1492c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00d860 tx=0x7f22ac008040 comp rx=0 tx=0).stop 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 shutdown_connections 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f22a0066f30 0x7f22a00693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22bc155650 0x7f22bc15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc063db0 0x7f22bc1492c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 --2- 172.21.15.7:0/1449125062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc069d60 0x7f22bc150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 >> 172.21.15.7:0/1449125062 conn(0x7f22bc0cd910 msgr2=0x7f22bc06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:55.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 shutdown_connections 2024-01-11T07:07:55.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.348+0000 7f22c3f41700 1 -- 172.21.15.7:0/1449125062 wait complete. 2024-01-11T07:07:55.351 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:07:55.361 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-01-11T07:07:53.851501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2312,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560187,"statfs":{"total":1600000000,"available":1597632452,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.215497"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666466+0000","last_change":"2024-01-11T07:07:50.666400+0000","last_active":"2024-01-11T07:07:50.666466+0000","last_peered":"2024-01-11T07:07:50.666466+0000","last_clean":"2024-01-11T07:07:50.666466+0000","last_became_active":"2024-01-11T07:07:50.666132+0000","last_became_peered":"2024-01-11T07:07:50.666132+0000","last_unstale":"2024-01-11T07:07:50.666466+0000","last_undegraded":"2024-01-11T07:07:50.666466+0000","last_fullsized":"2024-01-11T07:07:50.666466+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:08:05.071473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666434+0000","last_change":"2024-01-11T07:07:50.665922+0000","last_active":"2024-01-11T07:07:50.666434+0000","last_peered":"2024-01-11T07:07:50.666434+0000","last_clean":"2024-01-11T07:07:50.666434+0000","last_became_active":"2024-01-11T07:07:50.664730+0000","last_became_peered":"2024-01-11T07:07:50.664730+0000","last_unstale":"2024-01-11T07:07:50.666434+0000","last_undegraded":"2024-01-11T07:07:50.666434+0000","last_fullsized":"2024-01-11T07:07:50.666434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:27:47.108755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667569+0000","last_change":"2024-01-11T07:07:50.665647+0000","last_active":"2024-01-11T07:07:50.667569+0000","last_peered":"2024-01-11T07:07:50.667569+0000","last_clean":"2024-01-11T07:07:50.667569+0000","last_became_active":"2024-01-11T07:07:50.665414+0000","last_became_peered":"2024-01-11T07:07:50.665414+0000","last_unstale":"2024-01-11T07:07:50.667569+0000","last_undegraded":"2024-01-11T07:07:50.667569+0000","last_fullsized":"2024-01-11T07:07:50.667569+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:16:07.194652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674341+0000","last_change":"2024-01-11T07:07:50.674227+0000","last_active":"2024-01-11T07:07:50.674341+0000","last_peered":"2024-01-11T07:07:50.674341+0000","last_clean":"2024-01-11T07:07:50.674341+0000","last_became_active":"2024-01-11T07:07:50.673950+0000","last_became_peered":"2024-01-11T07:07:50.673950+0000","last_unstale":"2024-01-11T07:07:50.674341+0000","last_undegraded":"2024-01-11T07:07:50.674341+0000","last_fullsized":"2024-01-11T07:07:50.674341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:53.758682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669715+0000","last_change":"2024-01-11T07:07:50.669577+0000","last_active":"2024-01-11T07:07:50.669715+0000","last_peered":"2024-01-11T07:07:50.669715+0000","last_clean":"2024-01-11T07:07:50.669715+0000","last_became_active":"2024-01-11T07:07:50.669363+0000","last_became_peered":"2024-01-11T07:07:50.669363+0000","last_unstale":"2024-01-11T07:07:50.669715+0000","last_undegraded":"2024-01-11T07:07:50.669715+0000","last_fullsized":"2024-01-11T07:07:50.669715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:21:45.579684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670779+0000","last_change":"2024-01-11T07:07:50.670615+0000","last_active":"2024-01-11T07:07:50.670779+0000","last_peered":"2024-01-11T07:07:50.670779+0000","last_clean":"2024-01-11T07:07:50.670779+0000","last_became_active":"2024-01-11T07:07:50.670099+0000","last_became_peered":"2024-01-11T07:07:50.670099+0000","last_unstale":"2024-01-11T07:07:50.670779+0000","last_undegraded":"2024-01-11T07:07:50.670779+0000","last_fullsized":"2024-01-11T07:07:50.670779+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:35:14.001395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667833+0000","last_change":"2024-01-11T07:07:50.667413+0000","last_active":"2024-01-11T07:07:50.667833+0000","last_peered":"2024-01-11T07:07:50.667833+0000","last_clean":"2024-01-11T07:07:50.667833+0000","last_became_active":"2024-01-11T07:07:50.666819+0000","last_became_peered":"2024-01-11T07:07:50.666819+0000","last_unstale":"2024-01-11T07:07:50.667833+0000","last_undegraded":"2024-01-11T07:07:50.667833+0000","last_fullsized":"2024-01-11T07:07:50.667833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:56:45.059589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668755+0000","last_change":"2024-01-11T07:07:50.668587+0000","last_active":"2024-01-11T07:07:50.668755+0000","last_peered":"2024-01-11T07:07:50.668755+0000","last_clean":"2024-01-11T07:07:50.668755+0000","last_became_active":"2024-01-11T07:07:50.668340+0000","last_became_peered":"2024-01-11T07:07:50.668340+0000","last_unstale":"2024-01-11T07:07:50.668755+0000","last_undegraded":"2024-01-11T07:07:50.668755+0000","last_fullsized":"2024-01-11T07:07:50.668755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:23.978674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674125+0000","last_change":"2024-01-11T07:07:50.674056+0000","last_active":"2024-01-11T07:07:50.674125+0000","last_peered":"2024-01-11T07:07:50.674125+0000","last_clean":"2024-01-11T07:07:50.674125+0000","last_became_active":"2024-01-11T07:07:50.673207+0000","last_became_peered":"2024-01-11T07:07:50.673207+0000","last_unstale":"2024-01-11T07:07:50.674125+0000","last_undegraded":"2024-01-11T07:07:50.674125+0000","last_fullsized":"2024-01-11T07:07:50.674125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:25.740402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.671505+0000","last_change":"2024-01-11T07:07:50.671435+0000","last_active":"2024-01-11T07:07:50.671505+0000","last_peered":"2024-01-11T07:07:50.671505+0000","last_clean":"2024-01-11T07:07:50.671505+0000","last_became_active":"2024-01-11T07:07:50.671173+0000","last_became_peered":"2024-01-11T07:07:50.671173+0000","last_unstale":"2024-01-11T07:07:50.671505+0000","last_undegraded":"2024-01-11T07:07:50.671505+0000","last_fullsized":"2024-01-11T07:07:50.671505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:31:26.200394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667865+0000","last_change":"2024-01-11T07:07:50.667767+0000","last_active":"2024-01-11T07:07:50.667865+0000","last_peered":"2024-01-11T07:07:50.667865+0000","last_clean":"2024-01-11T07:07:50.667865+0000","last_became_active":"2024-01-11T07:07:50.667510+0000","last_became_peered":"2024-01-11T07:07:50.667510+0000","last_unstale":"2024-01-11T07:07:50.667865+0000","last_undegraded":"2024-01-11T07:07:50.667865+0000","last_fullsized":"2024-01-11T07:07:50.667865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:35:14.001395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669817+0000","last_change":"2024-01-11T07:07:50.669678+0000","last_active":"2024-01-11T07:07:50.669817+0000","last_peered":"2024-01-11T07:07:50.669817+0000","last_clean":"2024-01-11T07:07:50.669817+0000","last_became_active":"2024-01-11T07:07:50.668955+0000","last_became_peered":"2024-01-11T07:07:50.668955+0000","last_unstale":"2024-01-11T07:07:50.669817+0000","last_undegraded":"2024-01-11T07:07:50.669817+0000","last_fullsized":"2024-01-11T07:07:50.669817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:59:47.509012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667553+0000","last_change":"2024-01-11T07:07:50.667073+0000","last_active":"2024-01-11T07:07:50.667553+0000","last_peered":"2024-01-11T07:07:50.667553+0000","last_clean":"2024-01-11T07:07:50.667553+0000","last_became_active":"2024-01-11T07:07:50.666041+0000","last_became_peered":"2024-01-11T07:07:50.666041+0000","last_unstale":"2024-01-11T07:07:50.667553+0000","last_undegraded":"2024-01-11T07:07:50.667553+0000","last_fullsized":"2024-01-11T07:07:50.667553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:28:35.204062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666452+0000","last_change":"2024-01-11T07:07:50.665980+0000","last_active":"2024-01-11T07:07:50.666452+0000","last_peered":"2024-01-11T07:07:50.666452+0000","last_clean":"2024-01-11T07:07:50.666452+0000","last_became_active":"2024-01-11T07:07:50.665058+0000","last_became_peered":"2024-01-11T07:07:50.665058+0000","last_unstale":"2024-01-11T07:07:50.666452+0000","last_undegraded":"2024-01-11T07:07:50.666452+0000","last_fullsized":"2024-01-11T07:07:50.666452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:00.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667585+0000","last_change":"2024-01-11T07:07:50.666503+0000","last_active":"2024-01-11T07:07:50.667585+0000","last_peered":"2024-01-11T07:07:50.667585+0000","last_clean":"2024-01-11T07:07:50.667585+0000","last_became_active":"2024-01-11T07:07:50.666316+0000","last_became_peered":"2024-01-11T07:07:50.666316+0000","last_unstale":"2024-01-11T07:07:50.667585+0000","last_undegraded":"2024-01-11T07:07:50.667585+0000","last_fullsized":"2024-01-11T07:07:50.667585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:35:14.001395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.672196+0000","last_change":"2024-01-11T07:07:50.672095+0000","last_active":"2024-01-11T07:07:50.672196+0000","last_peered":"2024-01-11T07:07:50.672196+0000","last_clean":"2024-01-11T07:07:50.672196+0000","last_became_active":"2024-01-11T07:07:50.671857+0000","last_became_peered":"2024-01-11T07:07:50.671857+0000","last_unstale":"2024-01-11T07:07:50.672196+0000","last_undegraded":"2024-01-11T07:07:50.672196+0000","last_fullsized":"2024-01-11T07:07:50.672196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:17:30.628791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667851+0000","last_change":"2024-01-11T07:07:50.667578+0000","last_active":"2024-01-11T07:07:50.667851+0000","last_peered":"2024-01-11T07:07:50.667851+0000","last_clean":"2024-01-11T07:07:50.667851+0000","last_became_active":"2024-01-11T07:07:50.667172+0000","last_became_peered":"2024-01-11T07:07:50.667172+0000","last_unstale":"2024-01-11T07:07:50.667851+0000","last_undegraded":"2024-01-11T07:07:50.667851+0000","last_fullsized":"2024-01-11T07:07:50.667851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:39:23.786711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666405+0000","last_change":"2024-01-11T07:07:50.665435+0000","last_active":"2024-01-11T07:07:50.666405+0000","last_peered":"2024-01-11T07:07:50.666405+0000","last_clean":"2024-01-11T07:07:50.666405+0000","last_became_active":"2024-01-11T07:07:50.665186+0000","last_became_peered":"2024-01-11T07:07:50.665186+0000","last_unstale":"2024-01-11T07:07:50.666405+0000","last_undegraded":"2024-01-11T07:07:50.666405+0000","last_fullsized":"2024-01-11T07:07:50.666405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:17:30.628791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666484+0000","last_change":"2024-01-11T07:07:50.666295+0000","last_active":"2024-01-11T07:07:50.666484+0000","last_peered":"2024-01-11T07:07:50.666484+0000","last_clean":"2024-01-11T07:07:50.666484+0000","last_became_active":"2024-01-11T07:07:50.665721+0000","last_became_peered":"2024-01-11T07:07:50.665721+0000","last_unstale":"2024-01-11T07:07:50.666484+0000","last_undegraded":"2024-01-11T07:07:50.666484+0000","last_fullsized":"2024-01-11T07:07:50.666484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:41:18.095732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666328+0000","last_change":"2024-01-11T07:07:50.666248+0000","last_active":"2024-01-11T07:07:50.666328+0000","last_peered":"2024-01-11T07:07:50.666328+0000","last_clean":"2024-01-11T07:07:50.666328+0000","last_became_active":"2024-01-11T07:07:50.665977+0000","last_became_peered":"2024-01-11T07:07:50.665977+0000","last_unstale":"2024-01-11T07:07:50.666328+0000","last_undegraded":"2024-01-11T07:07:50.666328+0000","last_fullsized":"2024-01-11T07:07:50.666328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:33:19.948536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.672763+0000","last_change":"2024-01-11T07:07:50.670883+0000","last_active":"2024-01-11T07:07:50.672763+0000","last_peered":"2024-01-11T07:07:50.672763+0000","last_clean":"2024-01-11T07:07:50.672763+0000","last_became_active":"2024-01-11T07:07:50.670622+0000","last_became_peered":"2024-01-11T07:07:50.670622+0000","last_unstale":"2024-01-11T07:07:50.672763+0000","last_undegraded":"2024-01-11T07:07:50.672763+0000","last_fullsized":"2024-01-11T07:07:50.672763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:34.294015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673436+0000","last_change":"2024-01-11T07:07:50.673365+0000","last_active":"2024-01-11T07:07:50.673436+0000","last_peered":"2024-01-11T07:07:50.673436+0000","last_clean":"2024-01-11T07:07:50.673436+0000","last_became_active":"2024-01-11T07:07:50.672859+0000","last_became_peered":"2024-01-11T07:07:50.672859+0000","last_unstale":"2024-01-11T07:07:50.673436+0000","last_undegraded":"2024-01-11T07:07:50.673436+0000","last_fullsized":"2024-01-11T07:07:50.673436+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:53.758682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668420+0000","last_change":"2024-01-11T07:07:50.668343+0000","last_active":"2024-01-11T07:07:50.668420+0000","last_peered":"2024-01-11T07:07:50.668420+0000","last_clean":"2024-01-11T07:07:50.668420+0000","last_became_active":"2024-01-11T07:07:50.667900+0000","last_became_peered":"2024-01-11T07:07:50.667900+0000","last_unstale":"2024-01-11T07:07:50.668420+0000","last_undegraded":"2024-01-11T07:07:50.668420+0000","last_fullsized":"2024-01-11T07:07:50.668420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:50:36.586926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667549+0000","last_change":"2024-01-11T07:07:50.665338+0000","last_active":"2024-01-11T07:07:50.667549+0000","last_peered":"2024-01-11T07:07:50.667549+0000","last_clean":"2024-01-11T07:07:50.667549+0000","last_became_active":"2024-01-11T07:07:50.665076+0000","last_became_peered":"2024-01-11T07:07:50.665076+0000","last_unstale":"2024-01-11T07:07:50.667549+0000","last_undegraded":"2024-01-11T07:07:50.667549+0000","last_fullsized":"2024-01-11T07:07:50.667549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:23.978674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.663375+0000","last_change":"2024-01-11T07:07:50.662657+0000","last_active":"2024-01-11T07:07:50.663375+0000","last_peered":"2024-01-11T07:07:50.663375+0000","last_clean":"2024-01-11T07:07:50.663375+0000","last_became_active":"2024-01-11T07:07:50.662395+0000","last_became_peered":"2024-01-11T07:07:50.662395+0000","last_unstale":"2024-01-11T07:07:50.663375+0000","last_undegraded":"2024-01-11T07:07:50.663375+0000","last_fullsized":"2024-01-11T07:07:50.663375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:45:34.065121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670764+0000","last_change":"2024-01-11T07:07:50.670470+0000","last_active":"2024-01-11T07:07:50.670764+0000","last_peered":"2024-01-11T07:07:50.670764+0000","last_clean":"2024-01-11T07:07:50.670764+0000","last_became_active":"2024-01-11T07:07:50.669782+0000","last_became_peered":"2024-01-11T07:07:50.669782+0000","last_unstale":"2024-01-11T07:07:50.670764+0000","last_undegraded":"2024-01-11T07:07:50.670764+0000","last_fullsized":"2024-01-11T07:07:50.670764+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:23.978674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674011+0000","last_change":"2024-01-11T07:07:50.673943+0000","last_active":"2024-01-11T07:07:50.674011+0000","last_peered":"2024-01-11T07:07:50.674011+0000","last_clean":"2024-01-11T07:07:50.674011+0000","last_became_active":"2024-01-11T07:07:50.673662+0000","last_became_peered":"2024-01-11T07:07:50.673662+0000","last_unstale":"2024-01-11T07:07:50.674011+0000","last_undegraded":"2024-01-11T07:07:50.674011+0000","last_fullsized":"2024-01-11T07:07:50.674011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:39:23.786711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670274+0000","last_change":"2024-01-11T07:07:50.670193+0000","last_active":"2024-01-11T07:07:50.670274+0000","last_peered":"2024-01-11T07:07:50.670274+0000","last_clean":"2024-01-11T07:07:50.670274+0000","last_became_active":"2024-01-11T07:07:50.669745+0000","last_became_peered":"2024-01-11T07:07:50.669745+0000","last_unstale":"2024-01-11T07:07:50.670274+0000","last_undegraded":"2024-01-11T07:07:50.670274+0000","last_fullsized":"2024-01-11T07:07:50.670274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:05.623217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673820+0000","last_change":"2024-01-11T07:07:50.673724+0000","last_active":"2024-01-11T07:07:50.673820+0000","last_peered":"2024-01-11T07:07:50.673820+0000","last_clean":"2024-01-11T07:07:50.673820+0000","last_became_active":"2024-01-11T07:07:50.672892+0000","last_became_peered":"2024-01-11T07:07:50.672892+0000","last_unstale":"2024-01-11T07:07:50.673820+0000","last_undegraded":"2024-01-11T07:07:50.673820+0000","last_fullsized":"2024-01-11T07:07:50.673820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:59.039502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668919+0000","last_change":"2024-01-11T07:07:50.668824+0000","last_active":"2024-01-11T07:07:50.668919+0000","last_peered":"2024-01-11T07:07:50.668919+0000","last_clean":"2024-01-11T07:07:50.668919+0000","last_became_active":"2024-01-11T07:07:50.668231+0000","last_became_peered":"2024-01-11T07:07:50.668231+0000","last_unstale":"2024-01-11T07:07:50.668919+0000","last_undegraded":"2024-01-11T07:07:50.668919+0000","last_fullsized":"2024-01-11T07:07:50.668919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:45:34.065121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.671207+0000","last_change":"2024-01-11T07:07:50.671135+0000","last_active":"2024-01-11T07:07:50.671207+0000","last_peered":"2024-01-11T07:07:50.671207+0000","last_clean":"2024-01-11T07:07:50.671207+0000","last_became_active":"2024-01-11T07:07:50.670891+0000","last_became_peered":"2024-01-11T07:07:50.670891+0000","last_unstale":"2024-01-11T07:07:50.671207+0000","last_undegraded":"2024-01-11T07:07:50.671207+0000","last_fullsized":"2024-01-11T07:07:50.671207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:34.294015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.675761+0000","last_change":"2024-01-11T07:07:50.675645+0000","last_active":"2024-01-11T07:07:50.675761+0000","last_peered":"2024-01-11T07:07:50.675761+0000","last_clean":"2024-01-11T07:07:50.675761+0000","last_became_active":"2024-01-11T07:07:50.675395+0000","last_became_peered":"2024-01-11T07:07:50.675395+0000","last_unstale":"2024-01-11T07:07:50.675761+0000","last_undegraded":"2024-01-11T07:07:50.675761+0000","last_fullsized":"2024-01-11T07:07:50.675761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:00.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668315+0000","last_change":"2024-01-11T07:07:50.668009+0000","last_active":"2024-01-11T07:07:50.668315+0000","last_peered":"2024-01-11T07:07:50.668315+0000","last_clean":"2024-01-11T07:07:50.668315+0000","last_became_active":"2024-01-11T07:07:50.667755+0000","last_became_peered":"2024-01-11T07:07:50.667755+0000","last_unstale":"2024-01-11T07:07:50.668315+0000","last_undegraded":"2024-01-11T07:07:50.668315+0000","last_fullsized":"2024-01-11T07:07:50.668315+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:37:53.758682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681867+0000","last_change":"2024-01-11T07:07:49.649839+0000","last_active":"2024-01-11T07:07:50.681867+0000","last_peered":"2024-01-11T07:07:50.681867+0000","last_clean":"2024-01-11T07:07:50.681867+0000","last_became_active":"2024-01-11T07:07:49.649617+0000","last_became_peered":"2024-01-11T07:07:49.649617+0000","last_unstale":"2024-01-11T07:07:50.681867+0000","last_undegraded":"2024-01-11T07:07:50.681867+0000","last_fullsized":"2024-01-11T07:07:50.681867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:28:34.194763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.675863+0000","last_change":"2024-01-11T07:07:50.675803+0000","last_active":"2024-01-11T07:07:50.675863+0000","last_peered":"2024-01-11T07:07:50.675863+0000","last_clean":"2024-01-11T07:07:50.675863+0000","last_became_active":"2024-01-11T07:07:50.675613+0000","last_became_peered":"2024-01-11T07:07:50.675613+0000","last_unstale":"2024-01-11T07:07:50.675863+0000","last_undegraded":"2024-01-11T07:07:50.675863+0000","last_fullsized":"2024-01-11T07:07:50.675863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:02:46.281372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681419+0000","last_change":"2024-01-11T07:07:49.652320+0000","last_active":"2024-01-11T07:07:50.681419+0000","last_peered":"2024-01-11T07:07:50.681419+0000","last_clean":"2024-01-11T07:07:50.681419+0000","last_became_active":"2024-01-11T07:07:49.651665+0000","last_became_peered":"2024-01-11T07:07:49.651665+0000","last_unstale":"2024-01-11T07:07:50.681419+0000","last_undegraded":"2024-01-11T07:07:50.681419+0000","last_fullsized":"2024-01-11T07:07:50.681419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:36:15.175122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673272+0000","last_change":"2024-01-11T07:07:50.673201+0000","last_active":"2024-01-11T07:07:50.673272+0000","last_peered":"2024-01-11T07:07:50.673272+0000","last_clean":"2024-01-11T07:07:50.673272+0000","last_became_active":"2024-01-11T07:07:50.672874+0000","last_became_peered":"2024-01-11T07:07:50.672874+0000","last_unstale":"2024-01-11T07:07:50.673272+0000","last_undegraded":"2024-01-11T07:07:50.673272+0000","last_fullsized":"2024-01-11T07:07:50.673272+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:34:56.873661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.683033+0000","last_change":"2024-01-11T07:07:49.648156+0000","last_active":"2024-01-11T07:07:50.683033+0000","last_peered":"2024-01-11T07:07:50.683033+0000","last_clean":"2024-01-11T07:07:50.683033+0000","last_became_active":"2024-01-11T07:07:49.647891+0000","last_became_peered":"2024-01-11T07:07:49.647891+0000","last_unstale":"2024-01-11T07:07:50.683033+0000","last_undegraded":"2024-01-11T07:07:50.683033+0000","last_fullsized":"2024-01-11T07:07:50.683033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:08:04.062174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674942+0000","last_change":"2024-01-11T07:07:50.674868+0000","last_active":"2024-01-11T07:07:50.674942+0000","last_peered":"2024-01-11T07:07:50.674942+0000","last_clean":"2024-01-11T07:07:50.674942+0000","last_became_active":"2024-01-11T07:07:50.674658+0000","last_became_peered":"2024-01-11T07:07:50.674658+0000","last_unstale":"2024-01-11T07:07:50.674942+0000","last_undegraded":"2024-01-11T07:07:50.674942+0000","last_fullsized":"2024-01-11T07:07:50.674942+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:23:47.530004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.682450+0000","last_change":"2024-01-11T07:07:49.650961+0000","last_active":"2024-01-11T07:07:50.682450+0000","last_peered":"2024-01-11T07:07:50.682450+0000","last_clean":"2024-01-11T07:07:50.682450+0000","last_became_active":"2024-01-11T07:07:49.650703+0000","last_became_peered":"2024-01-11T07:07:49.650703+0000","last_unstale":"2024-01-11T07:07:50.682450+0000","last_undegraded":"2024-01-11T07:07:50.682450+0000","last_fullsized":"2024-01-11T07:07:50.682450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:42:11.078177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.676424+0000","last_change":"2024-01-11T07:07:50.676352+0000","last_active":"2024-01-11T07:07:50.676424+0000","last_peered":"2024-01-11T07:07:50.676424+0000","last_clean":"2024-01-11T07:07:50.676424+0000","last_became_active":"2024-01-11T07:07:50.675634+0000","last_became_peered":"2024-01-11T07:07:50.675634+0000","last_unstale":"2024-01-11T07:07:50.676424+0000","last_undegraded":"2024-01-11T07:07:50.676424+0000","last_fullsized":"2024-01-11T07:07:50.676424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:05.623217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681828+0000","last_change":"2024-01-11T07:07:49.650518+0000","last_active":"2024-01-11T07:07:50.681828+0000","last_peered":"2024-01-11T07:07:50.681828+0000","last_clean":"2024-01-11T07:07:50.681828+0000","last_became_active":"2024-01-11T07:07:49.649330+0000","last_became_peered":"2024-01-11T07:07:49.649330+0000","last_unstale":"2024-01-11T07:07:50.681828+0000","last_undegraded":"2024-01-11T07:07:50.681828+0000","last_fullsized":"2024-01-11T07:07:50.681828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:21:44.570385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669907+0000","last_change":"2024-01-11T07:07:50.669848+0000","last_active":"2024-01-11T07:07:50.669907+0000","last_peered":"2024-01-11T07:07:50.669907+0000","last_clean":"2024-01-11T07:07:50.669907+0000","last_became_active":"2024-01-11T07:07:50.669293+0000","last_became_peered":"2024-01-11T07:07:50.669293+0000","last_unstale":"2024-01-11T07:07:50.669907+0000","last_undegraded":"2024-01-11T07:07:50.669907+0000","last_fullsized":"2024-01-11T07:07:50.669907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:07:01.176205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681995+0000","last_change":"2024-01-11T07:07:49.647983+0000","last_active":"2024-01-11T07:07:50.681995+0000","last_peered":"2024-01-11T07:07:50.681995+0000","last_clean":"2024-01-11T07:07:50.681995+0000","last_became_active":"2024-01-11T07:07:49.647278+0000","last_became_peered":"2024-01-11T07:07:49.647278+0000","last_unstale":"2024-01-11T07:07:50.681995+0000","last_undegraded":"2024-01-11T07:07:50.681995+0000","last_fullsized":"2024-01-11T07:07:50.681995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:12:14.981796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673985+0000","last_change":"2024-01-11T07:07:50.673911+0000","last_active":"2024-01-11T07:07:50.673985+0000","last_peered":"2024-01-11T07:07:50.673985+0000","last_clean":"2024-01-11T07:07:50.673985+0000","last_became_active":"2024-01-11T07:07:50.673444+0000","last_became_peered":"2024-01-11T07:07:50.673444+0000","last_unstale":"2024-01-11T07:07:50.673985+0000","last_undegraded":"2024-01-11T07:07:50.673985+0000","last_fullsized":"2024-01-11T07:07:50.673985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:38.222386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.662928+0000","last_change":"2024-01-11T07:07:49.652067+0000","last_active":"2024-01-11T07:07:50.662928+0000","last_peered":"2024-01-11T07:07:50.662928+0000","last_clean":"2024-01-11T07:07:50.662928+0000","last_became_active":"2024-01-11T07:07:49.649322+0000","last_became_peered":"2024-01-11T07:07:49.649322+0000","last_unstale":"2024-01-11T07:07:50.662928+0000","last_undegraded":"2024-01-11T07:07:50.662928+0000","last_fullsized":"2024-01-11T07:07:50.662928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:33.284716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.671426+0000","last_change":"2024-01-11T07:07:50.668807+0000","last_active":"2024-01-11T07:07:50.671426+0000","last_peered":"2024-01-11T07:07:50.671426+0000","last_clean":"2024-01-11T07:07:50.671426+0000","last_became_active":"2024-01-11T07:07:50.668488+0000","last_became_peered":"2024-01-11T07:07:50.668488+0000","last_unstale":"2024-01-11T07:07:50.671426+0000","last_undegraded":"2024-01-11T07:07:50.671426+0000","last_fullsized":"2024-01-11T07:07:50.671426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:39.267002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668558+0000","last_change":"2024-01-11T07:07:49.645975+0000","last_active":"2024-01-11T07:07:50.668558+0000","last_peered":"2024-01-11T07:07:50.668558+0000","last_clean":"2024-01-11T07:07:50.668558+0000","last_became_active":"2024-01-11T07:07:49.645460+0000","last_became_peered":"2024-01-11T07:07:49.645460+0000","last_unstale":"2024-01-11T07:07:50.668558+0000","last_undegraded":"2024-01-11T07:07:50.668558+0000","last_fullsized":"2024-01-11T07:07:50.668558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:25:58.602861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667602+0000","last_change":"2024-01-11T07:07:50.667147+0000","last_active":"2024-01-11T07:07:50.667602+0000","last_peered":"2024-01-11T07:07:50.667602+0000","last_clean":"2024-01-11T07:07:50.667602+0000","last_became_active":"2024-01-11T07:07:50.666901+0000","last_became_peered":"2024-01-11T07:07:50.666901+0000","last_unstale":"2024-01-11T07:07:50.667602+0000","last_undegraded":"2024-01-11T07:07:50.667602+0000","last_fullsized":"2024-01-11T07:07:50.667602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:56:45.059589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668335+0000","last_change":"2024-01-11T07:07:49.654073+0000","last_active":"2024-01-11T07:07:50.668335+0000","last_peered":"2024-01-11T07:07:50.668335+0000","last_clean":"2024-01-11T07:07:50.668335+0000","last_became_active":"2024-01-11T07:07:49.652864+0000","last_became_peered":"2024-01-11T07:07:49.652864+0000","last_unstale":"2024-01-11T07:07:50.668335+0000","last_undegraded":"2024-01-11T07:07:50.668335+0000","last_fullsized":"2024-01-11T07:07:50.668335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:50:35.577627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670747+0000","last_change":"2024-01-11T07:07:50.670324+0000","last_active":"2024-01-11T07:07:50.670747+0000","last_peered":"2024-01-11T07:07:50.670747+0000","last_clean":"2024-01-11T07:07:50.670747+0000","last_became_active":"2024-01-11T07:07:50.669495+0000","last_became_peered":"2024-01-11T07:07:50.669495+0000","last_unstale":"2024-01-11T07:07:50.670747+0000","last_undegraded":"2024-01-11T07:07:50.670747+0000","last_fullsized":"2024-01-11T07:07:50.670747+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:07:01.176205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.656700+0000","last_change":"2024-01-11T07:07:49.650880+0000","last_active":"2024-01-11T07:07:50.656700+0000","last_peered":"2024-01-11T07:07:50.656700+0000","last_clean":"2024-01-11T07:07:50.656700+0000","last_became_active":"2024-01-11T07:07:49.650581+0000","last_became_peered":"2024-01-11T07:07:49.650581+0000","last_unstale":"2024-01-11T07:07:50.656700+0000","last_undegraded":"2024-01-11T07:07:50.656700+0000","last_fullsized":"2024-01-11T07:07:50.656700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:55:42.742639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667620+0000","last_change":"2024-01-11T07:07:50.667504+0000","last_active":"2024-01-11T07:07:50.667620+0000","last_peered":"2024-01-11T07:07:50.667620+0000","last_clean":"2024-01-11T07:07:50.667620+0000","last_became_active":"2024-01-11T07:07:50.667268+0000","last_became_peered":"2024-01-11T07:07:50.667268+0000","last_unstale":"2024-01-11T07:07:50.667620+0000","last_undegraded":"2024-01-11T07:07:50.667620+0000","last_fullsized":"2024-01-11T07:07:50.667620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:11:27.491967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.680388+0000","last_change":"2024-01-11T07:07:49.649102+0000","last_active":"2024-01-11T07:07:50.680388+0000","last_peered":"2024-01-11T07:07:50.680388+0000","last_clean":"2024-01-11T07:07:50.680388+0000","last_became_active":"2024-01-11T07:07:49.648353+0000","last_became_peered":"2024-01-11T07:07:49.648353+0000","last_unstale":"2024-01-11T07:07:50.680388+0000","last_undegraded":"2024-01-11T07:07:50.680388+0000","last_fullsized":"2024-01-11T07:07:50.680388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:46:33.284716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670256+0000","last_change":"2024-01-11T07:07:50.670036+0000","last_active":"2024-01-11T07:07:50.670256+0000","last_peered":"2024-01-11T07:07:50.670256+0000","last_clean":"2024-01-11T07:07:50.670256+0000","last_became_active":"2024-01-11T07:07:50.669431+0000","last_became_peered":"2024-01-11T07:07:50.669431+0000","last_unstale":"2024-01-11T07:07:50.670256+0000","last_undegraded":"2024-01-11T07:07:50.670256+0000","last_fullsized":"2024-01-11T07:07:50.670256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:00.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.684447+0000","last_change":"2024-01-11T07:07:49.652256+0000","last_active":"2024-01-11T07:07:50.684447+0000","last_peered":"2024-01-11T07:07:50.684447+0000","last_clean":"2024-01-11T07:07:50.684447+0000","last_became_active":"2024-01-11T07:07:49.650792+0000","last_became_peered":"2024-01-11T07:07:49.650792+0000","last_unstale":"2024-01-11T07:07:50.684447+0000","last_undegraded":"2024-01-11T07:07:50.684447+0000","last_fullsized":"2024-01-11T07:07:50.684447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:36.267494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668299+0000","last_change":"2024-01-11T07:07:50.667824+0000","last_active":"2024-01-11T07:07:50.668299+0000","last_peered":"2024-01-11T07:07:50.668299+0000","last_clean":"2024-01-11T07:07:50.668299+0000","last_became_active":"2024-01-11T07:07:50.667197+0000","last_became_peered":"2024-01-11T07:07:50.667197+0000","last_unstale":"2024-01-11T07:07:50.668299+0000","last_undegraded":"2024-01-11T07:07:50.668299+0000","last_fullsized":"2024-01-11T07:07:50.668299+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:59.039502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.682136+0000","last_change":"2024-01-11T07:07:49.645809+0000","last_active":"2024-01-11T07:07:50.682136+0000","last_peered":"2024-01-11T07:07:50.682136+0000","last_clean":"2024-01-11T07:07:50.682136+0000","last_became_active":"2024-01-11T07:07:49.645185+0000","last_became_peered":"2024-01-11T07:07:49.645185+0000","last_unstale":"2024-01-11T07:07:50.682136+0000","last_undegraded":"2024-01-11T07:07:50.682136+0000","last_fullsized":"2024-01-11T07:07:50.682136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:28:34.194763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668275+0000","last_change":"2024-01-11T07:07:50.668176+0000","last_active":"2024-01-11T07:07:50.668275+0000","last_peered":"2024-01-11T07:07:50.668275+0000","last_clean":"2024-01-11T07:07:50.668275+0000","last_became_active":"2024-01-11T07:07:50.667850+0000","last_became_peered":"2024-01-11T07:07:50.667850+0000","last_unstale":"2024-01-11T07:07:50.668275+0000","last_undegraded":"2024-01-11T07:07:50.668275+0000","last_fullsized":"2024-01-11T07:07:50.668275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:39.267002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.659902+0000","last_change":"2024-01-11T07:07:49.647297+0000","last_active":"2024-01-11T07:07:50.659902+0000","last_peered":"2024-01-11T07:07:50.659902+0000","last_clean":"2024-01-11T07:07:50.659902+0000","last_became_active":"2024-01-11T07:07:49.646536+0000","last_became_peered":"2024-01-11T07:07:49.646536+0000","last_unstale":"2024-01-11T07:07:50.659902+0000","last_undegraded":"2024-01-11T07:07:50.659902+0000","last_fullsized":"2024-01-11T07:07:50.659902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:28.539684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.675959+0000","last_change":"2024-01-11T07:07:50.675880+0000","last_active":"2024-01-11T07:07:50.675959+0000","last_peered":"2024-01-11T07:07:50.675959+0000","last_clean":"2024-01-11T07:07:50.675959+0000","last_became_active":"2024-01-11T07:07:50.675308+0000","last_became_peered":"2024-01-11T07:07:50.675308+0000","last_unstale":"2024-01-11T07:07:50.675959+0000","last_undegraded":"2024-01-11T07:07:50.675959+0000","last_fullsized":"2024-01-11T07:07:50.675959+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:59:47.509012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.660405+0000","last_change":"2024-01-11T07:07:49.650024+0000","last_active":"2024-01-11T07:07:50.660405+0000","last_peered":"2024-01-11T07:07:50.660405+0000","last_clean":"2024-01-11T07:07:50.660405+0000","last_became_active":"2024-01-11T07:07:49.649691+0000","last_became_peered":"2024-01-11T07:07:49.649691+0000","last_unstale":"2024-01-11T07:07:50.660405+0000","last_undegraded":"2024-01-11T07:07:50.660405+0000","last_fullsized":"2024-01-11T07:07:50.660405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:28.539684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.677026+0000","last_change":"2024-01-11T07:07:50.676903+0000","last_active":"2024-01-11T07:07:50.677026+0000","last_peered":"2024-01-11T07:07:50.677026+0000","last_clean":"2024-01-11T07:07:50.677026+0000","last_became_active":"2024-01-11T07:07:50.676625+0000","last_became_peered":"2024-01-11T07:07:50.676625+0000","last_unstale":"2024-01-11T07:07:50.677026+0000","last_undegraded":"2024-01-11T07:07:50.677026+0000","last_fullsized":"2024-01-11T07:07:50.677026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:06:54.741409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.684656+0000","last_change":"2024-01-11T07:07:49.646993+0000","last_active":"2024-01-11T07:07:50.684656+0000","last_peered":"2024-01-11T07:07:50.684656+0000","last_clean":"2024-01-11T07:07:50.684656+0000","last_became_active":"2024-01-11T07:07:49.646720+0000","last_became_peered":"2024-01-11T07:07:49.646720+0000","last_unstale":"2024-01-11T07:07:50.684656+0000","last_undegraded":"2024-01-11T07:07:50.684656+0000","last_fullsized":"2024-01-11T07:07:50.684656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:24.731102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673855+0000","last_change":"2024-01-11T07:07:50.673760+0000","last_active":"2024-01-11T07:07:50.673855+0000","last_peered":"2024-01-11T07:07:50.673855+0000","last_clean":"2024-01-11T07:07:50.673855+0000","last_became_active":"2024-01-11T07:07:50.672827+0000","last_became_peered":"2024-01-11T07:07:50.672827+0000","last_unstale":"2024-01-11T07:07:50.673855+0000","last_undegraded":"2024-01-11T07:07:50.673855+0000","last_fullsized":"2024-01-11T07:07:50.673855+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:50:36.586926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.683493+0000","last_change":"2024-01-11T07:07:49.655412+0000","last_active":"2024-01-11T07:07:50.683493+0000","last_peered":"2024-01-11T07:07:50.683493+0000","last_clean":"2024-01-11T07:07:50.683493+0000","last_became_active":"2024-01-11T07:07:49.655226+0000","last_became_peered":"2024-01-11T07:07:49.655226+0000","last_unstale":"2024-01-11T07:07:50.683493+0000","last_undegraded":"2024-01-11T07:07:50.683493+0000","last_fullsized":"2024-01-11T07:07:50.683493+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:04:09.767719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674340+0000","last_change":"2024-01-11T07:07:50.674273+0000","last_active":"2024-01-11T07:07:50.674340+0000","last_peered":"2024-01-11T07:07:50.674340+0000","last_clean":"2024-01-11T07:07:50.674340+0000","last_became_active":"2024-01-11T07:07:50.674057+0000","last_became_peered":"2024-01-11T07:07:50.674057+0000","last_unstale":"2024-01-11T07:07:50.674340+0000","last_undegraded":"2024-01-11T07:07:50.674340+0000","last_fullsized":"2024-01-11T07:07:50.674340+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:39:23.786711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.681917+0000","last_change":"2024-01-11T07:07:49.645682+0000","last_active":"2024-01-11T07:07:50.681917+0000","last_peered":"2024-01-11T07:07:50.681917+0000","last_clean":"2024-01-11T07:07:50.681917+0000","last_became_active":"2024-01-11T07:07:49.644880+0000","last_became_peered":"2024-01-11T07:07:49.644880+0000","last_unstale":"2024-01-11T07:07:50.681917+0000","last_undegraded":"2024-01-11T07:07:50.681917+0000","last_fullsized":"2024-01-11T07:07:50.681917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:53:57.003779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673332+0000","last_change":"2024-01-11T07:07:50.673259+0000","last_active":"2024-01-11T07:07:50.673332+0000","last_peered":"2024-01-11T07:07:50.673332+0000","last_clean":"2024-01-11T07:07:50.673332+0000","last_became_active":"2024-01-11T07:07:50.673030+0000","last_became_peered":"2024-01-11T07:07:50.673030+0000","last_unstale":"2024-01-11T07:07:50.673332+0000","last_undegraded":"2024-01-11T07:07:50.673332+0000","last_fullsized":"2024-01-11T07:07:50.673332+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:42:49.114805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.682903+0000","last_change":"2024-01-11T07:07:49.651771+0000","last_active":"2024-01-11T07:07:50.682903+0000","last_peered":"2024-01-11T07:07:50.682903+0000","last_clean":"2024-01-11T07:07:50.682903+0000","last_became_active":"2024-01-11T07:07:49.647635+0000","last_became_peered":"2024-01-11T07:07:49.647635+0000","last_unstale":"2024-01-11T07:07:50.682903+0000","last_undegraded":"2024-01-11T07:07:50.682903+0000","last_fullsized":"2024-01-11T07:07:50.682903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:40.362580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.664284+0000","last_change":"2024-01-11T07:07:50.664005+0000","last_active":"2024-01-11T07:07:50.664284+0000","last_peered":"2024-01-11T07:07:50.664284+0000","last_clean":"2024-01-11T07:07:50.664284+0000","last_became_active":"2024-01-11T07:07:50.663280+0000","last_became_peered":"2024-01-11T07:07:50.663280+0000","last_unstale":"2024-01-11T07:07:50.664284+0000","last_undegraded":"2024-01-11T07:07:50.664284+0000","last_fullsized":"2024-01-11T07:07:50.664284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:55.653328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.663873+0000","last_change":"2024-01-11T07:07:49.650692+0000","last_active":"2024-01-11T07:07:50.663873+0000","last_peered":"2024-01-11T07:07:50.663873+0000","last_clean":"2024-01-11T07:07:50.663873+0000","last_became_active":"2024-01-11T07:07:49.649691+0000","last_became_peered":"2024-01-11T07:07:49.649691+0000","last_unstale":"2024-01-11T07:07:50.663873+0000","last_undegraded":"2024-01-11T07:07:50.663873+0000","last_fullsized":"2024-01-11T07:07:50.663873+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:01:38.347171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.674494+0000","last_change":"2024-01-11T07:07:50.674411+0000","last_active":"2024-01-11T07:07:50.674494+0000","last_peered":"2024-01-11T07:07:50.674494+0000","last_clean":"2024-01-11T07:07:50.674494+0000","last_became_active":"2024-01-11T07:07:50.674129+0000","last_became_peered":"2024-01-11T07:07:50.674129+0000","last_unstale":"2024-01-11T07:07:50.674494+0000","last_undegraded":"2024-01-11T07:07:50.674494+0000","last_fullsized":"2024-01-11T07:07:50.674494+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:25:59.612160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.682508+0000","last_change":"2024-01-11T07:07:49.649183+0000","last_active":"2024-01-11T07:07:50.682508+0000","last_peered":"2024-01-11T07:07:50.682508+0000","last_clean":"2024-01-11T07:07:50.682508+0000","last_became_active":"2024-01-11T07:07:49.647457+0000","last_became_peered":"2024-01-11T07:07:49.647457+0000","last_unstale":"2024-01-11T07:07:50.682508+0000","last_undegraded":"2024-01-11T07:07:50.682508+0000","last_fullsized":"2024-01-11T07:07:50.682508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:04:09.767719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667593+0000","last_change":"2024-01-11T07:07:50.667481+0000","last_active":"2024-01-11T07:07:50.667593+0000","last_peered":"2024-01-11T07:07:50.667593+0000","last_clean":"2024-01-11T07:07:50.667593+0000","last_became_active":"2024-01-11T07:07:50.666780+0000","last_became_peered":"2024-01-11T07:07:50.666780+0000","last_unstale":"2024-01-11T07:07:50.667593+0000","last_undegraded":"2024-01-11T07:07:50.667593+0000","last_fullsized":"2024-01-11T07:07:50.667593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:53:58.013078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.660414+0000","last_change":"2024-01-11T07:07:49.651247+0000","last_active":"2024-01-11T07:07:50.660414+0000","last_peered":"2024-01-11T07:07:50.660414+0000","last_clean":"2024-01-11T07:07:50.660414+0000","last_became_active":"2024-01-11T07:07:49.651010+0000","last_became_peered":"2024-01-11T07:07:49.651010+0000","last_unstale":"2024-01-11T07:07:50.660414+0000","last_undegraded":"2024-01-11T07:07:50.660414+0000","last_fullsized":"2024-01-11T07:07:50.660414+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:25:58.602861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.663689+0000","last_change":"2024-01-11T07:07:50.663030+0000","last_active":"2024-01-11T07:07:50.663689+0000","last_peered":"2024-01-11T07:07:50.663689+0000","last_clean":"2024-01-11T07:07:50.663689+0000","last_became_active":"2024-01-11T07:07:50.662740+0000","last_became_peered":"2024-01-11T07:07:50.662740+0000","last_unstale":"2024-01-11T07:07:50.663689+0000","last_undegraded":"2024-01-11T07:07:50.663689+0000","last_fullsized":"2024-01-11T07:07:50.663689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:15:03.302073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668757+0000","last_change":"2024-01-11T07:07:49.652646+0000","last_active":"2024-01-11T07:07:50.668757+0000","last_peered":"2024-01-11T07:07:50.668757+0000","last_clean":"2024-01-11T07:07:50.668757+0000","last_became_active":"2024-01-11T07:07:49.652414+0000","last_became_peered":"2024-01-11T07:07:49.652414+0000","last_unstale":"2024-01-11T07:07:50.668757+0000","last_undegraded":"2024-01-11T07:07:50.668757+0000","last_fullsized":"2024-01-11T07:07:50.668757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:15:02.292774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669074+0000","last_change":"2024-01-11T07:07:50.668983+0000","last_active":"2024-01-11T07:07:50.669074+0000","last_peered":"2024-01-11T07:07:50.669074+0000","last_clean":"2024-01-11T07:07:50.669074+0000","last_became_active":"2024-01-11T07:07:50.668565+0000","last_became_peered":"2024-01-11T07:07:50.668565+0000","last_unstale":"2024-01-11T07:07:50.669074+0000","last_undegraded":"2024-01-11T07:07:50.669074+0000","last_fullsized":"2024-01-11T07:07:50.669074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:23:47.530004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.660897+0000","last_change":"2024-01-11T07:07:49.650617+0000","last_active":"2024-01-11T07:07:50.660897+0000","last_peered":"2024-01-11T07:07:50.660897+0000","last_clean":"2024-01-11T07:07:50.660897+0000","last_became_active":"2024-01-11T07:07:49.650368+0000","last_became_peered":"2024-01-11T07:07:49.650368+0000","last_unstale":"2024-01-11T07:07:50.660897+0000","last_undegraded":"2024-01-11T07:07:50.660897+0000","last_fullsized":"2024-01-11T07:07:50.660897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:08:04.062174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.667574+0000","last_change":"2024-01-11T07:07:50.667315+0000","last_active":"2024-01-11T07:07:50.667574+0000","last_peered":"2024-01-11T07:07:50.667574+0000","last_clean":"2024-01-11T07:07:50.667574+0000","last_became_active":"2024-01-11T07:07:50.666407+0000","last_became_peered":"2024-01-11T07:07:50.666407+0000","last_unstale":"2024-01-11T07:07:50.667574+0000","last_undegraded":"2024-01-11T07:07:50.667574+0000","last_fullsized":"2024-01-11T07:07:50.667574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:55:43.751938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.683152+0000","last_change":"2024-01-11T07:07:49.649435+0000","last_active":"2024-01-11T07:07:50.683152+0000","last_peered":"2024-01-11T07:07:50.683152+0000","last_clean":"2024-01-11T07:07:50.683152+0000","last_became_active":"2024-01-11T07:07:49.649030+0000","last_became_peered":"2024-01-11T07:07:49.649030+0000","last_unstale":"2024-01-11T07:07:50.683152+0000","last_undegraded":"2024-01-11T07:07:50.683152+0000","last_fullsized":"2024-01-11T07:07:50.683152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:54.644029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.672577+0000","last_change":"2024-01-11T07:07:50.672455+0000","last_active":"2024-01-11T07:07:50.672577+0000","last_peered":"2024-01-11T07:07:50.672577+0000","last_clean":"2024-01-11T07:07:50.672577+0000","last_became_active":"2024-01-11T07:07:50.672220+0000","last_became_peered":"2024-01-11T07:07:50.672220+0000","last_unstale":"2024-01-11T07:07:50.672577+0000","last_undegraded":"2024-01-11T07:07:50.672577+0000","last_fullsized":"2024-01-11T07:07:50.672577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:17:30.628791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-11T07:07:51.658443+0000","last_change":"2024-01-11T07:07:49.648998+0000","last_active":"2024-01-11T07:07:51.658443+0000","last_peered":"2024-01-11T07:07:51.658443+0000","last_clean":"2024-01-11T07:07:51.658443+0000","last_became_active":"2024-01-11T07:07:49.647096+0000","last_became_peered":"2024-01-11T07:07:49.647096+0000","last_unstale":"2024-01-11T07:07:51.658443+0000","last_undegraded":"2024-01-11T07:07:51.658443+0000","last_fullsized":"2024-01-11T07:07:51.658443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:28.539684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.662017+0000","last_change":"2024-01-11T07:07:50.661214+0000","last_active":"2024-01-11T07:07:50.662017+0000","last_peered":"2024-01-11T07:07:50.662017+0000","last_clean":"2024-01-11T07:07:50.662017+0000","last_became_active":"2024-01-11T07:07:50.660970+0000","last_became_peered":"2024-01-11T07:07:50.660970+0000","last_unstale":"2024-01-11T07:07:50.662017+0000","last_undegraded":"2024-01-11T07:07:50.662017+0000","last_fullsized":"2024-01-11T07:07:50.662017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:37.276793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.659588+0000","last_change":"2024-01-11T07:07:49.647565+0000","last_active":"2024-01-11T07:07:50.659588+0000","last_peered":"2024-01-11T07:07:50.659588+0000","last_clean":"2024-01-11T07:07:50.659588+0000","last_became_active":"2024-01-11T07:07:49.647080+0000","last_became_peered":"2024-01-11T07:07:49.647080+0000","last_unstale":"2024-01-11T07:07:50.659588+0000","last_undegraded":"2024-01-11T07:07:50.659588+0000","last_fullsized":"2024-01-11T07:07:50.659588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:36:15.175122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.669487+0000","last_change":"2024-01-11T07:07:50.669391+0000","last_active":"2024-01-11T07:07:50.669487+0000","last_peered":"2024-01-11T07:07:50.669487+0000","last_clean":"2024-01-11T07:07:50.669487+0000","last_became_active":"2024-01-11T07:07:50.668266+0000","last_became_peered":"2024-01-11T07:07:50.668266+0000","last_unstale":"2024-01-11T07:07:50.669487+0000","last_undegraded":"2024-01-11T07:07:50.669487+0000","last_fullsized":"2024-01-11T07:07:50.669487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:45:46.131204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.657140+0000","last_change":"2024-01-11T07:07:49.648524+0000","last_active":"2024-01-11T07:07:50.657140+0000","last_peered":"2024-01-11T07:07:50.657140+0000","last_clean":"2024-01-11T07:07:50.657140+0000","last_became_active":"2024-01-11T07:07:49.648282+0000","last_became_peered":"2024-01-11T07:07:49.648282+0000","last_unstale":"2024-01-11T07:07:50.657140+0000","last_undegraded":"2024-01-11T07:07:50.657140+0000","last_fullsized":"2024-01-11T07:07:50.657140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:54.644029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.668635+0000","last_change":"2024-01-11T07:07:50.668528+0000","last_active":"2024-01-11T07:07:50.668635+0000","last_peered":"2024-01-11T07:07:50.668635+0000","last_clean":"2024-01-11T07:07:50.668635+0000","last_became_active":"2024-01-11T07:07:50.668291+0000","last_became_peered":"2024-01-11T07:07:50.668291+0000","last_unstale":"2024-01-11T07:07:50.668635+0000","last_undegraded":"2024-01-11T07:07:50.668635+0000","last_fullsized":"2024-01-11T07:07:50.668635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:38.222386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.659021+0000","last_change":"2024-01-11T07:07:49.649273+0000","last_active":"2024-01-11T07:07:50.659021+0000","last_peered":"2024-01-11T07:07:50.659021+0000","last_clean":"2024-01-11T07:07:50.659021+0000","last_became_active":"2024-01-11T07:07:49.648633+0000","last_became_peered":"2024-01-11T07:07:49.648633+0000","last_unstale":"2024-01-11T07:07:50.659021+0000","last_undegraded":"2024-01-11T07:07:50.659021+0000","last_fullsized":"2024-01-11T07:07:50.659021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:55:42.742639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666468+0000","last_change":"2024-01-11T07:07:50.666135+0000","last_active":"2024-01-11T07:07:50.666468+0000","last_peered":"2024-01-11T07:07:50.666468+0000","last_clean":"2024-01-11T07:07:50.666468+0000","last_became_active":"2024-01-11T07:07:50.665386+0000","last_became_peered":"2024-01-11T07:07:50.665386+0000","last_unstale":"2024-01-11T07:07:50.666468+0000","last_undegraded":"2024-01-11T07:07:50.666468+0000","last_fullsized":"2024-01-11T07:07:50.666468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:16:07.194652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.670514+0000","last_change":"2024-01-11T07:07:49.649920+0000","last_active":"2024-01-11T07:07:50.670514+0000","last_peered":"2024-01-11T07:07:50.670514+0000","last_clean":"2024-01-11T07:07:50.670514+0000","last_became_active":"2024-01-11T07:07:49.649674+0000","last_became_peered":"2024-01-11T07:07:49.649674+0000","last_unstale":"2024-01-11T07:07:50.670514+0000","last_undegraded":"2024-01-11T07:07:50.670514+0000","last_fullsized":"2024-01-11T07:07:50.670514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:18:25.817021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.673457+0000","last_change":"2024-01-11T07:07:50.673375+0000","last_active":"2024-01-11T07:07:50.673457+0000","last_peered":"2024-01-11T07:07:50.673457+0000","last_clean":"2024-01-11T07:07:50.673457+0000","last_became_active":"2024-01-11T07:07:50.672959+0000","last_became_peered":"2024-01-11T07:07:50.672959+0000","last_unstale":"2024-01-11T07:07:50.673457+0000","last_undegraded":"2024-01-11T07:07:50.673457+0000","last_fullsized":"2024-01-11T07:07:50.673457+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:37.276793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.662817+0000","last_change":"2024-01-11T07:07:49.652438+0000","last_active":"2024-01-11T07:07:50.662817+0000","last_peered":"2024-01-11T07:07:50.662817+0000","last_clean":"2024-01-11T07:07:50.662817+0000","last_became_active":"2024-01-11T07:07:49.651815+0000","last_became_peered":"2024-01-11T07:07:49.651815+0000","last_unstale":"2024-01-11T07:07:50.662817+0000","last_undegraded":"2024-01-11T07:07:50.662817+0000","last_fullsized":"2024-01-11T07:07:50.662817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:37.213087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.666283+0000","last_change":"2024-01-11T07:07:50.666204+0000","last_active":"2024-01-11T07:07:50.666283+0000","last_peered":"2024-01-11T07:07:50.666283+0000","last_clean":"2024-01-11T07:07:50.666283+0000","last_became_active":"2024-01-11T07:07:50.665960+0000","last_became_peered":"2024-01-11T07:07:50.665960+0000","last_unstale":"2024-01-11T07:07:50.666283+0000","last_undegraded":"2024-01-11T07:07:50.666283+0000","last_fullsized":"2024-01-11T07:07:50.666283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:49.634880+0000","last_clean_scrub_stamp":"2024-01-11T07:07:49.634880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:07:01.176205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.660528+0000","last_change":"2024-01-11T07:07:49.649057+0000","last_active":"2024-01-11T07:07:50.660528+0000","last_peered":"2024-01-11T07:07:50.660528+0000","last_clean":"2024-01-11T07:07:50.660528+0000","last_became_active":"2024-01-11T07:07:49.648785+0000","last_became_peered":"2024-01-11T07:07:49.648785+0000","last_unstale":"2024-01-11T07:07:50.660528+0000","last_undegraded":"2024-01-11T07:07:50.660528+0000","last_fullsized":"2024-01-11T07:07:50.660528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:07:48.625581+0000","last_clean_scrub_stamp":"2024-01-11T07:07:48.625581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:15:02.292774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-11T07:07:50.662544+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:07:50.662544+0000","last_peered":"2024-01-11T07:07:50.662544+0000","last_clean":"2024-01-11T07:07:50.662544+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:07:50.662544+0000","last_undegraded":"2024-01-11T07:07:50.662544+0000","last_fullsized":"2024-01-11T07:07:50.662544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199260981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46700000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.71599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":145,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195166,"statfs":{"total":200000000,"available":199850851,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.49399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":146,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195165,"statfs":{"total":200000000,"available":199849881,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.438}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.503}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199261071,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.7960000000000003,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.62}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":8.0730000000000004,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":8.0399999999999991,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.819,"5min":2.819,"15min":2.819},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.87},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":8.0030000000000001,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.89500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":145,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195167,"statfs":{"total":200000000,"available":199851369,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63300000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":146,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195165,"statfs":{"total":200000000,"available":199849881,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":11.481,"5min":11.481,"15min":11.481},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":17.516999999999999,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.44},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":17.553000000000001,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.474,"5min":17.474,"15min":17.474},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":17.393000000000001,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":11.436999999999999,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":17.434000000000001,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.66400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":145,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195167,"statfs":{"total":200000000,"available":199851335,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":8.484,"5min":8.484,"15min":8.484},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.2599999999999998,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":8.5340000000000007,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":9.2279999999999998,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.751}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":9.5640000000000001,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.8639999999999999,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":8.9779999999999998,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.7309999999999999,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.388999999999999,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":8.6509999999999998,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1449999999999996,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.629}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.4640000000000004,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.61099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":139,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195172,"statfs":{"total":200000000,"available":199857083,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.78500000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.83499999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.87}]},{"osd":4,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.63}]},{"osd":7,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.66400000000000003}]}]}],"pool_statfs":[]}} 2024-01-11T07:07:55.366 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:07:55.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:07:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.546+0000 7f468feaa700 1 Processor -- start 2024-01-11T07:07:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.546+0000 7f468feaa700 1 -- start start 2024-01-11T07:07:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468feaa700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46880cafb0 0x7f46880ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468feaa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4688158280 0x7f46880b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468feaa700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4688159300 con 0x7f46880b0970 2024-01-11T07:07:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468feaa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880b0dc0 con 0x7f4688158280 2024-01-11T07:07:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468feaa700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880cf260 con 0x7f46880cafb0 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468d445700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4688158280 0x7f46880b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468d445700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4688158280 0x7f46880b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50636/0 (socket says 172.21.15.7:50636) 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468d445700 1 -- 172.21.15.7:0/927817783 learned_addr learned my addr 172.21.15.7:0/927817783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468dc46700 1 --2- 172.21.15.7:0/927817783 >> v2:172.21.15.113:3300/0 conn(0x7f46880cafb0 0x7f46880ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468d445700 1 -- 172.21.15.7:0/927817783 >> v2:172.21.15.113:3300/0 conn(0x7f46880cafb0 msgr2=0x7f46880ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468d445700 1 --2- 172.21.15.7:0/927817783 >> v2:172.21.15.113:3300/0 conn(0x7f46880cafb0 0x7f46880ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468d445700 1 -- 172.21.15.7:0/927817783 >> v1:172.21.15.113:6789/0 conn(0x7f46880b0970 legacy=0x7f4688159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468d445700 1 -- 172.21.15.7:0/927817783 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880cf540 con 0x7f4688158280 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.547+0000 7f468d445700 1 --2- 172.21.15.7:0/927817783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880b0430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f46880cfb70 tx=0x7f4684009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afccc65f9ab6f79f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468cc44700 1 -- 172.21.15.7:0/927817783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684009e70 con 0x7f4688158280 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468cc44700 1 -- 172.21.15.7:0/927817783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468400b860 con 0x7f4688158280 2024-01-11T07:07:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468cc44700 1 -- 172.21.15.7:0/927817783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468400bc00 con 0x7f4688158280 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468feaa700 1 -- 172.21.15.7:0/927817783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 msgr2=0x7f46880b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468feaa700 1 --2- 172.21.15.7:0/927817783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880b0430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f46880cfb70 tx=0x7f4684009250 comp rx=0 tx=0).stop 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468feaa700 1 -- 172.21.15.7:0/927817783 shutdown_connections 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468feaa700 1 --2- 172.21.15.7:0/927817783 >> v2:172.21.15.113:3300/0 conn(0x7f46880cafb0 0x7f46880ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468feaa700 1 --2- 172.21.15.7:0/927817783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880b0430 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468feaa700 1 -- 172.21.15.7:0/927817783 >> 172.21.15.7:0/927817783 conn(0x7f4688071100 msgr2=0x7f468806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468feaa700 1 -- 172.21.15.7:0/927817783 shutdown_connections 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.548+0000 7f468feaa700 1 -- 172.21.15.7:0/927817783 wait complete. 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.550+0000 7f468feaa700 1 Processor -- start 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.550+0000 7f468feaa700 1 -- start start 2024-01-11T07:07:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.550+0000 7f468feaa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46880b0970 0x7f46880ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.550+0000 7f468feaa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46880cafb0 0x7f46880b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.550+0000 7f468feaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.550+0000 7f468feaa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880c46f0 con 0x7f4688158280 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.550+0000 7f468feaa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880ca7c0 con 0x7f46880cafb0 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.550+0000 7f468feaa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46880af570 con 0x7f46880b0970 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50646/0 (socket says 172.21.15.7:50646) 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 -- 172.21.15.7:0/1155402617 learned_addr learned my addr 172.21.15.7:0/1155402617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468d445700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46880cafb0 0x7f46880b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468dc46700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46880b0970 0x7f46880ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 -- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46880b0970 msgr2=0x7f46880ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46880b0970 0x7f46880ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 -- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46880cafb0 msgr2=0x7f46880b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46880cafb0 0x7f46880b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 -- 172.21.15.7:0/1155402617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840094f0 con 0x7f4688158280 2024-01-11T07:07:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468e447700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880c4b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f468000c8e0 tx=0x7f4680001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f467dffb700 1 -- 172.21.15.7:0/1155402617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000ce80 con 0x7f4688158280 2024-01-11T07:07:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f467dffb700 1 -- 172.21.15.7:0/1155402617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000a7e0 con 0x7f4688158280 2024-01-11T07:07:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f467dffb700 1 -- 172.21.15.7:0/1155402617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000ab80 con 0x7f4688158280 2024-01-11T07:07:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.551+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46880c3800 con 0x7f4688158280 2024-01-11T07:07:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.552+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46880c3dc0 con 0x7f4688158280 2024-01-11T07:07:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.553+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46880c4860 con 0x7f4688158280 2024-01-11T07:07:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.553+0000 7f467dffb700 1 -- 172.21.15.7:0/1155402617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4680012420 con 0x7f4688158280 2024-01-11T07:07:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.553+0000 7f467dffb700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f466c066f60 0x7f466c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.553+0000 7f467dffb700 1 -- 172.21.15.7:0/1155402617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4680084820 con 0x7f4688158280 2024-01-11T07:07:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.554+0000 7f468dc46700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f466c066f60 0x7f466c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.554+0000 7f468dc46700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f466c066f60 0x7f466c069410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4678009d40 tx=0x7f4678008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.558+0000 7f467dffb700 1 -- 172.21.15.7:0/1155402617 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4680051a90 con 0x7f4688158280 2024-01-11T07:07:55.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.849+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f468804c3c0 con 0x7f4688158280 2024-01-11T07:07:55.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.853+0000 7f467dffb700 1 -- 172.21.15.7:0/1155402617 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f46800550b0 con 0x7f4688158280 2024-01-11T07:07:55.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f466c066f60 msgr2=0x7f466c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f466c066f60 0x7f466c069410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4678009d40 tx=0x7f4678008040 comp rx=0 tx=0).stop 2024-01-11T07:07:55.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 msgr2=0x7f46880c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:55.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880c4b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f468000c8e0 tx=0x7f4680001710 comp rx=0 tx=0).stop 2024-01-11T07:07:55.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 shutdown_connections 2024-01-11T07:07:55.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f466c066f60 0x7f466c069410 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4688158280 0x7f46880c4b70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46880b0970 0x7f46880ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 --2- 172.21.15.7:0/1155402617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46880cafb0 0x7f46880b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:55.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 >> 172.21.15.7:0/1155402617 conn(0x7f4688071100 msgr2=0x7f4688154160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:55.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 shutdown_connections 2024-01-11T07:07:55.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:55.855+0000 7f468feaa700 1 -- 172.21.15.7:0/1155402617 wait complete. 2024-01-11T07:07:55.856 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:07:55.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.002+0000 7fe9eadcf700 1 Processor -- start 2024-01-11T07:07:56.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.002+0000 7fe9eadcf700 1 -- start start 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9eadcf700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9e406e0d0 0x7fe9e4159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9eadcf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e415b770 con 0x7fe9e406dd00 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9eadcf700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e415abf0 con 0x7fe9e4159ff0 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9eadcf700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e415aed0 con 0x7fe9e406e0d0 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e8b6b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe9e406dd00 0x7fe9e40c98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45080/0 (socket says 172.21.15.7:45080) 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e8b6b700 1 -- 172.21.15.7:0/4055619237 learned_addr learned my addr 172.21.15.7:0/4055619237 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e3fff700 1 --2- 172.21.15.7:0/4055619237 >> v2:172.21.15.113:3300/0 conn(0x7fe9e406e0d0 0x7fe9e4159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e37fe700 1 -- 172.21.15.7:0/4055619237 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355959735 0 0) 0x7fe9e415b770 con 0x7fe9e406dd00 2024-01-11T07:07:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e37fe700 1 -- 172.21.15.7:0/4055619237 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9cc003570 con 0x7fe9e406dd00 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e3fff700 1 -- 172.21.15.7:0/4055619237 >> v1:172.21.15.113:6789/0 conn(0x7fe9e4159ff0 legacy=0x7fe9e4069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e37fe700 1 -- 172.21.15.7:0/4055619237 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647265885 0 0) 0x7fe9e415abf0 con 0x7fe9e4159ff0 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e3fff700 1 -- 172.21.15.7:0/4055619237 >> v1:172.21.15.7:6789/0 conn(0x7fe9e406dd00 legacy=0x7fe9e40c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.003+0000 7fe9e3fff700 1 -- 172.21.15.7:0/4055619237 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e415b1b0 con 0x7fe9e406e0d0 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9e3fff700 1 --2- 172.21.15.7:0/4055619237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406e0d0 0x7fe9e4159ab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80098e0 tx=0x7fe9d80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e60787205a95e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9e37fe700 1 -- 172.21.15.7:0/4055619237 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d800b640 con 0x7fe9e406e0d0 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9e37fe700 1 -- 172.21.15.7:0/4055619237 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d800b7a0 con 0x7fe9e406e0d0 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9e37fe700 1 -- 172.21.15.7:0/4055619237 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d800bb40 con 0x7fe9e406e0d0 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9eadcf700 1 -- 172.21.15.7:0/4055619237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406e0d0 msgr2=0x7fe9e4159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9eadcf700 1 --2- 172.21.15.7:0/4055619237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406e0d0 0x7fe9e4159ab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80098e0 tx=0x7fe9d80091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9eadcf700 1 -- 172.21.15.7:0/4055619237 shutdown_connections 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9eadcf700 1 --2- 172.21.15.7:0/4055619237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406e0d0 0x7fe9e4159ab0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9eadcf700 1 -- 172.21.15.7:0/4055619237 >> 172.21.15.7:0/4055619237 conn(0x7fe9e40680e0 msgr2=0x7fe9e40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9eadcf700 1 -- 172.21.15.7:0/4055619237 shutdown_connections 2024-01-11T07:07:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.004+0000 7fe9eadcf700 1 -- 172.21.15.7:0/4055619237 wait complete. 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.006+0000 7fe9eadcf700 1 Processor -- start 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.006+0000 7fe9eadcf700 1 -- start start 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9eadcf700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406dd00 0x7fe9e415de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9eadcf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e406e0d0 0x7fe9e4156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9eadcf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9e4159ff0 0x7fe9e415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9eadcf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e415d990 con 0x7fe9e4159ff0 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9eadcf700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e41570a0 con 0x7fe9e406dd00 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9eadcf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e415cd80 con 0x7fe9e406e0d0 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9e4159ff0 0x7fe9e415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9e4159ff0 0x7fe9e415c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50656/0 (socket says 172.21.15.7:50656) 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 -- 172.21.15.7:0/2853090065 learned_addr learned my addr 172.21.15.7:0/2853090065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e8b6b700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406dd00 0x7fe9e415de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e3fff700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e406e0d0 0x7fe9e4156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 -- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e406e0d0 msgr2=0x7fe9e4156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e406e0d0 0x7fe9e4156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 -- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406dd00 msgr2=0x7fe9e415de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406dd00 0x7fe9e415de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 -- 172.21.15.7:0/2853090065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d8009420 con 0x7fe9e4159ff0 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e8b6b700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406dd00 0x7fe9e415de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e936c700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9e4159ff0 0x7fe9e415c450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00c890 tx=0x7fe9dc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.007+0000 7fe9e0ff9700 1 -- 172.21.15.7:0/2853090065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc017070 con 0x7fe9e4159ff0 2024-01-11T07:07:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.008+0000 7fe9e0ff9700 1 -- 172.21.15.7:0/2853090065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc006ba0 con 0x7fe9e4159ff0 2024-01-11T07:07:56.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.008+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e415d060 con 0x7fe9e4159ff0 2024-01-11T07:07:56.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.008+0000 7fe9e0ff9700 1 -- 172.21.15.7:0/2853090065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00a6c0 con 0x7fe9e4159ff0 2024-01-11T07:07:56.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.008+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e415d620 con 0x7fe9e4159ff0 2024-01-11T07:07:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.009+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e415db00 con 0x7fe9e4159ff0 2024-01-11T07:07:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.009+0000 7fe9e0ff9700 1 -- 172.21.15.7:0/2853090065 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe9dc00aaf0 con 0x7fe9e4159ff0 2024-01-11T07:07:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.009+0000 7fe9e0ff9700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe9d0066f30 0x7fe9d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.010+0000 7fe9e0ff9700 1 -- 172.21.15.7:0/2853090065 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe9dc084430 con 0x7fe9e4159ff0 2024-01-11T07:07:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.010+0000 7fe9e8b6b700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe9d0066f30 0x7fe9d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.010+0000 7fe9e8b6b700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe9d0066f30 0x7fe9d00693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40058d0 tx=0x7fe9d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.014+0000 7fe9e0ff9700 1 -- 172.21.15.7:0/2853090065 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9dc0516a0 con 0x7fe9e4159ff0 2024-01-11T07:07:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.263+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9e404c3c0 con 0x7fe9e4159ff0 2024-01-11T07:07:56.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.263+0000 7fe9e0ff9700 1 -- 172.21.15.7:0/2853090065 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fe9dc054cc0 con 0x7fe9e4159ff0 2024-01-11T07:07:56.263 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:56.263 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:07:50.072668+0000","modified":"2024-01-11T07:07:53.685303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/3458914988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3458914988},{"type":"v1","addr":"172.21.15.7:6834","nonce":3458914988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe9d0066f30 msgr2=0x7fe9d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe9d0066f30 0x7fe9d00693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40058d0 tx=0x7fe9d4009040 comp rx=0 tx=0).stop 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9e4159ff0 msgr2=0x7fe9e415c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9e4159ff0 0x7fe9e415c450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00c890 tx=0x7fe9dc001670 comp rx=0 tx=0).stop 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 shutdown_connections 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe9d0066f30 0x7fe9d00693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9e4159ff0 0x7fe9e415c450 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e406dd00 0x7fe9e415de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 --2- 172.21.15.7:0/2853090065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e406e0d0 0x7fe9e4156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.265+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 >> 172.21.15.7:0/2853090065 conn(0x7fe9e40680e0 msgr2=0x7fe9e40cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:56.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.266+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 shutdown_connections 2024-01-11T07:07:56.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.266+0000 7fe9eadcf700 1 -- 172.21.15.7:0/2853090065 wait complete. 2024-01-11T07:07:56.267 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T07:07:56.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:07:50.072668+0000', 'modified': '2024-01-11T07:07:53.685303+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4917}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4917': {'gid': 4917, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.7:6834/3458914988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6832', 'nonce': 3458914988}, {'type': 'v1', 'addr': '172.21.15.7:6834', 'nonce': 3458914988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:07:56.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:07:56.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-11T07:07:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.825+0000 7fa84d483700 1 Processor -- start 2024-01-11T07:07:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.825+0000 7fa84d483700 1 -- start start 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.825+0000 7fa84d483700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 0x7fa84806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa84d483700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84815dbb0 con 0x7fa848159d10 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa84d483700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84815de90 con 0x7fa84806f1d0 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa84d483700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815d9a0 con 0x7fa8480693a0 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa846ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa848159d10 0x7fa848068c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45082/0 (socket says 172.21.15.7:45082) 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa846ffd700 1 -- 172.21.15.7:0/1856414483 learned_addr learned my addr 172.21.15.7:0/1856414483 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa8467fc700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 0x7fa84806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541917114 0 0) 0x7fa84815dbb0 con 0x7fa848159d10 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa830003570 con 0x7fa848159d10 2024-01-11T07:07:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382383210 0 0) 0x7fa84815de90 con 0x7fa84806f1d0 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa84815dbb0 con 0x7fa84806f1d0 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1963369026 0 0) 0x7fa830003570 con 0x7fa848159d10 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 msgr2=0x7fa84806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 0x7fa84806ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 >> v1:172.21.15.113:6789/0 conn(0x7fa84806f1d0 legacy=0x7fa8480b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84815e170 con 0x7fa848159d10 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa838001470 con 0x7fa848159d10 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa8467fc700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 0x7fa84806ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 >> v1:172.21.15.7:6789/0 conn(0x7fa848159d10 legacy=0x7fa848068c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa830003c20 0x7fa830004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830004810 0x7fa830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 0x7fa8300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815d9a0 con 0x7fa830003c20 2024-01-11T07:07:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.826+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83000bb40 con 0x7fa830004810 2024-01-11T07:07:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83000bd00 con 0x7fa8480693a0 2024-01-11T07:07:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa846ffd700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa830003c20 0x7fa830004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa8467fc700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830004810 0x7fa830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa8477fe700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 0x7fa8300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa846ffd700 1 -- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 msgr2=0x7fa8300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa846ffd700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480693a0 0x7fa8300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa846ffd700 1 -- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830004810 msgr2=0x7fa830004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa846ffd700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830004810 0x7fa830004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa846ffd700 1 -- 172.21.15.7:0/1856414483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c009420 con 0x7fa830003c20 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa846ffd700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa830003c20 0x7fa830004010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c2d0 tx=0x7fa838001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4f4155c95a5f19 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83800cba0 con 0x7fa830003c20 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83800cd00 con 0x7fa830003c20 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.827+0000 7fa845ffb700 1 -- 172.21.15.7:0/1856414483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838005b70 con 0x7fa830003c20 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.828+0000 7fa84d483700 1 -- 172.21.15.7:0/1856414483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa830003c20 msgr2=0x7fa830004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.828+0000 7fa84d483700 1 --2- 172.21.15.7:0/1856414483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa830003c20 0x7fa830004010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c2d0 tx=0x7fa838001e10 comp rx=0 tx=0).stop 2024-01-11T07:07:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.828+0000 7fa8477fe700 1 -- 172.21.15.7:0/1856414483 reap_dead start 2024-01-11T07:07:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.828+0000 7fa84d483700 1 -- 172.21.15.7:0/1856414483 shutdown_connections 2024-01-11T07:07:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.828+0000 7fa84d483700 1 -- 172.21.15.7:0/1856414483 >> 172.21.15.7:0/1856414483 conn(0x7fa8480ce9f0 msgr2=0x7fa8480cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.828+0000 7fa84d483700 1 -- 172.21.15.7:0/1856414483 shutdown_connections 2024-01-11T07:07:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.828+0000 7fa84d483700 1 -- 172.21.15.7:0/1856414483 wait complete. 2024-01-11T07:07:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.829+0000 7fa84d483700 1 Processor -- start 2024-01-11T07:07:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.829+0000 7fa84d483700 1 -- start start 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa84d483700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830003c20 0x7fa848146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa84d483700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84815a0e0 0x7fa84815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa84d483700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480cedf0 0x7fa8480cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa84d483700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480cf7d0 con 0x7fa84815a0e0 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa84d483700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815b7b0 con 0x7fa830003c20 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa84d483700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815aba0 con 0x7fa8480cedf0 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8467fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84815a0e0 0x7fa84815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8467fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84815a0e0 0x7fa84815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50684/0 (socket says 172.21.15.7:50684) 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8467fc700 1 -- 172.21.15.7:0/2144665759 learned_addr learned my addr 172.21.15.7:0/2144665759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa846ffd700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830003c20 0x7fa848146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8477fe700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480cedf0 0x7fa8480cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8467fc700 1 -- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480cedf0 msgr2=0x7fa8480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8467fc700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480cedf0 0x7fa8480cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8467fc700 1 -- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830003c20 msgr2=0x7fa848146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8467fc700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830003c20 0x7fa848146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8467fc700 1 -- 172.21.15.7:0/2144665759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84815ae80 con 0x7fa84815a0e0 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.830+0000 7fa8477fe700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480cedf0 0x7fa8480cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.831+0000 7fa8467fc700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84815a0e0 0x7fa84815a550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa83c011d70 tx=0x7fa83c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.831+0000 7fa82effd700 1 -- 172.21.15.7:0/2144665759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00fcd0 con 0x7fa84815a0e0 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.831+0000 7fa82effd700 1 -- 172.21.15.7:0/2144665759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c00fe30 con 0x7fa84815a0e0 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.831+0000 7fa84d483700 1 -- 172.21.15.7:0/2144665759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84815b160 con 0x7fa84815a0e0 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.831+0000 7fa82effd700 1 -- 172.21.15.7:0/2144665759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c014670 con 0x7fa84815a0e0 2024-01-11T07:07:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.831+0000 7fa84d483700 1 -- 172.21.15.7:0/2144665759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8480c8510 con 0x7fa84815a0e0 2024-01-11T07:07:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.832+0000 7fa82effd700 1 -- 172.21.15.7:0/2144665759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa83c014a60 con 0x7fa84815a0e0 2024-01-11T07:07:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.833+0000 7fa82effd700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa834066f30 0x7fa8340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.833+0000 7fa82effd700 1 -- 172.21.15.7:0/2144665759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa83c018030 con 0x7fa84815a0e0 2024-01-11T07:07:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.833+0000 7fa846ffd700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa834066f30 0x7fa8340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.834+0000 7fa846ffd700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa834066f30 0x7fa8340693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa838007c90 tx=0x7fa838015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.835+0000 7fa856783b80 1 Processor -- start 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.835+0000 7fa856783b80 1 -- start start 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa856783b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 0x55e9c59ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa856783b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa856783b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa856783b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c58db3e0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa856783b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c58d8be0 con 0x55e9c59acfa0 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa856783b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9c58d49d0 con 0x55e9c59a9170 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa8467fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa8467fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59a8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50700/0 (socket says 172.21.15.7:50700) 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa8467fc700 1 -- 172.21.15.7:0/361393841 learned_addr learned my addr 172.21.15.7:0/361393841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa846ffd700 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 0x55e9c59ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.836+0000 7fa8477fe700 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa8467fc700 1 -- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 msgr2=0x55e9c59c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa8467fc700 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa8467fc700 1 -- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 msgr2=0x55e9c59ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa8467fc700 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 0x55e9c59ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa8467fc700 1 -- 172.21.15.7:0/361393841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9c5768360 con 0x55e9c59ad8b0 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa8477fe700 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa8467fc700 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59a8c30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009a30 tx=0x7fa83c083620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fcbb7abb019ce7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa827fff700 1 -- 172.21.15.7:0/361393841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c056150 con 0x55e9c59ad8b0 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa827fff700 1 -- 172.21.15.7:0/361393841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c0593a0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.837+0000 7fa827fff700 1 -- 172.21.15.7:0/361393841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c05d0f0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 -- 172.21.15.7:0/361393841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 msgr2=0x55e9c59a8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59a8c30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009a30 tx=0x7fa83c083620 comp rx=0 tx=0).stop 2024-01-11T07:07:56.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 -- 172.21.15.7:0/361393841 shutdown_connections 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 0x55e9c59ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 --2- 172.21.15.7:0/361393841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59a8c30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 -- 172.21.15.7:0/361393841 >> 172.21.15.7:0/361393841 conn(0x55e9c5983440 msgr2=0x55e9c59cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 -- 172.21.15.7:0/361393841 shutdown_connections 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 -- 172.21.15.7:0/361393841 wait complete. 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.838+0000 7fa856783b80 1 -- start start 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.839+0000 7fa856783b80 1 Processor -- start 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa856783b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa856783b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 0x55e9c59c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa856783b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa856783b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c58811b0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa856783b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c5880e50 con 0x55e9c59acfa0 2024-01-11T07:07:56.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa856783b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9c57cde40 con 0x55e9c59a9170 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa856783b80 10 client.0 _ll_register_callbacks cb 0x55e9c57d07b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50712/0 (socket says 172.21.15.7:50712) 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa8477fe700 1 -- 172.21.15.7:0/288780022 learned_addr learned my addr 172.21.15.7:0/288780022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa846ffd700 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa8467fc700 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 0x55e9c59c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa8477fe700 1 -- 172.21.15.7:0/288780022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 msgr2=0x55e9c59ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.840+0000 7fa8477fe700 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8477fe700 1 -- 172.21.15.7:0/288780022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 msgr2=0x55e9c59c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8477fe700 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 0x55e9c59c4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8477fe700 1 -- 172.21.15.7:0/288780022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8480c8510 con 0x55e9c59ad8b0 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8477fe700 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59abfa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa84806be70 tx=0x7fa840009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9442447ec59df0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8257fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:07:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8257fa700 1 -- 172.21.15.7:0/288780022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840002030 con 0x55e9c59ad8b0 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8257fa700 1 -- 172.21.15.7:0/288780022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa840002190 con 0x55e9c59ad8b0 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 1 -- 172.21.15.7:0/288780022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9c5955eb0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8257fa700 1 -- 172.21.15.7:0/288780022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8400025a0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 10 client.5094 fetch_fsmap learned FSMap version 16 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 1 -- 172.21.15.7:0/288780022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55e9c5714b10 con 0x55e9c59ad8b0 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8257fa700 1 -- 172.21.15.7:0/288780022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e9c5955eb0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa8257fa700 1 -- 172.21.15.7:0/288780022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa8400016e0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 10 client.5094 fetch_fsmap finished waiting for FSMap version 16 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 10 client.5094 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 20 client.5094 populate_metadata read hostname 'smithi007' 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] conn(0x55e9c59ac4e0 0x55e9c59aae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 4 client.5094 mds_command: new command op to 4917 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:07:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.841+0000 7fa856783b80 1 -- 172.21.15.7:0/288780022 --> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9c58ab880 con 0x55e9c59ac4e0 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.842+0000 7fa846ffd700 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] conn(0x55e9c59ac4e0 0x55e9c59aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.842+0000 7fa8257fa700 10 client.5094 ms_handle_connect on v2:172.21.15.7:6832/3458914988 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.842+0000 7fa846ffd700 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] conn(0x55e9c59ac4e0 0x55e9c59aae70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c0db294fe70471 server_cookie=d1c18ad6ac014d26 in_seq=0 out_seq=0 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa8257fa700 1 -- 172.21.15.7:0/288780022 <== mds.0 v2:172.21.15.7:6832/3458914988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e9c58ab880 con 0x55e9c59ac4e0 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa8257fa700 10 client.5094 handle_command_reply: tid=0 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 client.5094 shutdown 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 -- 172.21.15.7:0/288780022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 msgr2=0x55e9c59abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59abfa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa84806be70 tx=0x7fa840009040 comp rx=0 tx=0).stop 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 -- 172.21.15.7:0/288780022 shutdown_connections 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] conn(0x55e9c59ac4e0 0x55e9c59aae70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c59abfa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59a9170 0x55e9c59ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 --2- 172.21.15.7:0/288780022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59acfa0 0x55e9c59c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa8257fa700 0 client.5094 ms_handle_reset on v2:172.21.15.7:6832/3458914988 2024-01-11T07:07:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 -- 172.21.15.7:0/288780022 >> 172.21.15.7:0/288780022 conn(0x55e9c5983440 msgr2=0x55e9c59b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 -- 172.21.15.7:0/288780022 shutdown_connections 2024-01-11T07:07:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 1 -- 172.21.15.7:0/288780022 wait complete. 2024-01-11T07:07:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.843+0000 7fa856783b80 20 client.5094 trim_cache size 0 max 16384 2024-01-11T07:07:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa856783b80 1 Processor -- start 2024-01-11T07:07:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa856783b80 1 -- start start 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa856783b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 0x55e9c59a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa856783b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 0x55e9c597b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa856783b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa856783b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c59ca3f0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa856783b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c5982dc0 con 0x55e9c59a9170 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa856783b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9c5987c20 con 0x55e9c59acfa0 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.862+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50718/0 (socket says 172.21.15.7:50718) 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8477fe700 1 -- 172.21.15.7:0/2266858277 learned_addr learned my addr 172.21.15.7:0/2266858277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa846ffd700 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 0x55e9c59a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8467fc700 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 0x55e9c597b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8477fe700 1 -- 172.21.15.7:0/2266858277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 msgr2=0x55e9c597b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8477fe700 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 0x55e9c597b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8477fe700 1 -- 172.21.15.7:0/2266858277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 msgr2=0x55e9c59a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8477fe700 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 0x55e9c59a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8477fe700 1 -- 172.21.15.7:0/2266858277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9c58a3290 con 0x55e9c59ad8b0 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa8477fe700 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa840007d80 tx=0x7fa84001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d85ce01e13cc8f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa82cff9700 1 -- 172.21.15.7:0/2266858277 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840001ba0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa82cff9700 1 -- 172.21.15.7:0/2266858277 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa840000fb0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.863+0000 7fa82cff9700 1 -- 172.21.15.7:0/2266858277 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8400025a0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 -- 172.21.15.7:0/2266858277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 msgr2=0x55e9c597b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa840007d80 tx=0x7fa84001d000 comp rx=0 tx=0).stop 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 -- 172.21.15.7:0/2266858277 shutdown_connections 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 0x55e9c59a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 --2- 172.21.15.7:0/2266858277 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 0x55e9c597b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 -- 172.21.15.7:0/2266858277 >> 172.21.15.7:0/2266858277 conn(0x55e9c5983440 msgr2=0x55e9c59c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:56.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 -- 172.21.15.7:0/2266858277 shutdown_connections 2024-01-11T07:07:56.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 -- 172.21.15.7:0/2266858277 wait complete. 2024-01-11T07:07:56.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.864+0000 7fa856783b80 1 -- start start 2024-01-11T07:07:56.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.865+0000 7fa856783b80 1 Processor -- start 2024-01-11T07:07:56.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa856783b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 0x55e9c5a19480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa856783b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 0x55e9c597b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa856783b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa856783b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c5a33ec0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa856783b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9c59c68a0 con 0x55e9c59a9170 2024-01-11T07:07:56.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa856783b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9c59c6af0 con 0x55e9c59acfa0 2024-01-11T07:07:56.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa856783b80 10 client.0 _ll_register_callbacks cb 0x7fa840002cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50720/0 (socket says 172.21.15.7:50720) 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 -- 172.21.15.7:0/2592033504 learned_addr learned my addr 172.21.15.7:0/2592033504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8467fc700 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 0x55e9c597b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa846ffd700 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 0x55e9c5a19480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 -- 172.21.15.7:0/2592033504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 msgr2=0x55e9c597b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 0x55e9c597b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 -- 172.21.15.7:0/2592033504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 msgr2=0x55e9c5a19480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 0x55e9c5a19480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 -- 172.21.15.7:0/2592033504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9c5714b10 con 0x55e9c59ad8b0 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8477fe700 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa840007bb0 tx=0x7fa840025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48dbe3cb954fb07 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8257fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8257fa700 1 -- 172.21.15.7:0/2592033504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840014c90 con 0x55e9c59ad8b0 2024-01-11T07:07:56.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa8257fa700 1 -- 172.21.15.7:0/2592033504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa840014df0 con 0x55e9c59ad8b0 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.866+0000 7fa856783b80 1 -- 172.21.15.7:0/2592033504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9c5979490 con 0x55e9c59ad8b0 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa8257fa700 1 -- 172.21.15.7:0/2592033504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa840014640 con 0x55e9c59ad8b0 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa856783b80 10 client.5100 fetch_fsmap learned FSMap version 16 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa856783b80 1 -- 172.21.15.7:0/2592033504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55e9c576cc80 con 0x55e9c59ad8b0 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa8257fa700 1 -- 172.21.15.7:0/2592033504 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e9c5979490 con 0x55e9c59ad8b0 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa8257fa700 1 -- 172.21.15.7:0/2592033504 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa84002a040 con 0x55e9c59ad8b0 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa856783b80 10 client.5100 fetch_fsmap finished waiting for FSMap version 16 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa856783b80 10 client.5100 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa856783b80 20 client.5100 populate_metadata read hostname 'smithi007' 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa856783b80 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] conn(0x55e9c59c95e0 0x55e9c59c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa856783b80 4 client.5100 mds_command: new command op to 4917 tid=0[{"prefix": "status"}] 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa856783b80 1 -- 172.21.15.7:0/2592033504 --> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e9c5877910 con 0x55e9c59c95e0 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa846ffd700 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] conn(0x55e9c59c95e0 0x55e9c59c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:56.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa8257fa700 10 client.5100 ms_handle_connect on v2:172.21.15.7:6832/3458914988 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.867+0000 7fa846ffd700 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] conn(0x55e9c59c95e0 0x55e9c59c99b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648492fa658a199d server_cookie=304f3689ffcf16be in_seq=0 out_seq=0 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa8257fa700 1 -- 172.21.15.7:0/2592033504 <== mds.0 v2:172.21.15.7:6832/3458914988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e9c5877910 con 0x55e9c59c95e0 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa8257fa700 10 client.5100 handle_command_reply: tid=0 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 client.5100 shutdown 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 -- 172.21.15.7:0/2592033504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 msgr2=0x55e9c597b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa840007bb0 tx=0x7fa840025040 comp rx=0 tx=0).stop 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 -- 172.21.15.7:0/2592033504 shutdown_connections 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.7:6832/3458914988,v1:172.21.15.7:6834/3458914988] conn(0x55e9c59c95e0 0x55e9c59c99b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9c59ad8b0 0x55e9c597b5c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e9c59a9170 0x55e9c5a19480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 --2- 172.21.15.7:0/2592033504 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e9c59acfa0 0x55e9c597b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa8257fa700 0 client.5100 ms_handle_reset on v2:172.21.15.7:6832/3458914988 2024-01-11T07:07:56.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 -- 172.21.15.7:0/2592033504 >> 172.21.15.7:0/2592033504 conn(0x55e9c5983440 msgr2=0x55e9c59b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:56.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 -- 172.21.15.7:0/2592033504 shutdown_connections 2024-01-11T07:07:56.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 1 -- 172.21.15.7:0/2592033504 wait complete. 2024-01-11T07:07:56.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa856783b80 20 client.5100 trim_cache size 0 max 16384 2024-01-11T07:07:56.876 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:07:56.876 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:07:56.876 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:07:56.876 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4917, 2024-01-11T07:07:56.876 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:07:56.877 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:07:56.877 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:07:56.877 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1966303619999996, 2024-01-11T07:07:56.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 16, 2024-01-11T07:07:56.877 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 34, 2024-01-11T07:07:56.877 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 34, 2024-01-11T07:07:56.877 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 15.089248679000001 2024-01-11T07:07:56.877 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa84d483700 1 -- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa834066f30 msgr2=0x7fa8340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa84d483700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa834066f30 0x7fa8340693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa838007c90 tx=0x7fa838015000 comp rx=0 tx=0).stop 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa84d483700 1 -- 172.21.15.7:0/2144665759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84815a0e0 msgr2=0x7fa84815a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.868+0000 7fa84d483700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84815a0e0 0x7fa84815a550 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa83c011d70 tx=0x7fa83c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.869+0000 7fa84d483700 1 -- 172.21.15.7:0/2144665759 shutdown_connections 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.869+0000 7fa84d483700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa834066f30 0x7fa8340693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.869+0000 7fa84d483700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8480cedf0 0x7fa8480cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.869+0000 7fa84d483700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa830003c20 0x7fa848146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.869+0000 7fa84d483700 1 --2- 172.21.15.7:0/2144665759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84815a0e0 0x7fa84815a550 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:56.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.869+0000 7fa84d483700 1 -- 172.21.15.7:0/2144665759 >> 172.21.15.7:0/2144665759 conn(0x7fa8480ce9f0 msgr2=0x7fa848158ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:56.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.869+0000 7fa84d483700 1 -- 172.21.15.7:0/2144665759 shutdown_connections 2024-01-11T07:07:56.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:56.869+0000 7fa84d483700 1 -- 172.21.15.7:0/2144665759 wait complete. 2024-01-11T07:07:56.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:07:57.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.013+0000 7f336159f700 1 Processor -- start 2024-01-11T07:07:57.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.013+0000 7f336159f700 1 -- start start 2024-01-11T07:07:57.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f336159f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f335c0cd6d0 0x7f335c0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:57.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f336159f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f335c15d8a0 0x7f335c15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:57.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f336159f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c15e700 con 0x7f335c15e280 2024-01-11T07:07:57.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f336159f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335c15acf0 con 0x7f335c15d8a0 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f336159f700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335c15afd0 con 0x7f335c0cd6d0 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f335a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f335c15d8a0 0x7f335c15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f335a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f335c15d8a0 0x7f335c15dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50734/0 (socket says 172.21.15.7:50734) 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f335b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f335c15e280 0x7f335c0cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45090/0 (socket says 172.21.15.7:45090) 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f335a7fc700 1 -- 172.21.15.7:0/2667111171 learned_addr learned my addr 172.21.15.7:0/2667111171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.014+0000 7f335affd700 1 --2- 172.21.15.7:0/2667111171 >> v2:172.21.15.113:3300/0 conn(0x7f335c0cd6d0 0x7f335c0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f335a7fc700 1 -- 172.21.15.7:0/2667111171 >> v2:172.21.15.113:3300/0 conn(0x7f335c0cd6d0 msgr2=0x7f335c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f335a7fc700 1 --2- 172.21.15.7:0/2667111171 >> v2:172.21.15.113:3300/0 conn(0x7f335c0cd6d0 0x7f335c0cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f335a7fc700 1 -- 172.21.15.7:0/2667111171 >> v1:172.21.15.7:6789/0 conn(0x7f335c15e280 legacy=0x7f335c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f335a7fc700 1 -- 172.21.15.7:0/2667111171 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0ce980 con 0x7f335c15d8a0 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f3359ffb700 1 -- 172.21.15.7:0/2667111171 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688313948 0 0) 0x7f335c15e700 con 0x7f335c15e280 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f335a7fc700 1 --2- 172.21.15.7:0/2667111171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c15dd40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f335c0677c0 tx=0x7f3344009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdee3c3a4fb2286c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f3359ffb700 1 -- 172.21.15.7:0/2667111171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400b860 con 0x7f335c15d8a0 2024-01-11T07:07:57.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f3359ffb700 1 -- 172.21.15.7:0/2667111171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334400b9c0 con 0x7f335c15d8a0 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.015+0000 7f3359ffb700 1 -- 172.21.15.7:0/2667111171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400bd60 con 0x7f335c15d8a0 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.016+0000 7f336159f700 1 -- 172.21.15.7:0/2667111171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 msgr2=0x7f335c15dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.016+0000 7f336159f700 1 --2- 172.21.15.7:0/2667111171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c15dd40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f335c0677c0 tx=0x7f3344009250 comp rx=0 tx=0).stop 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.016+0000 7f336159f700 1 -- 172.21.15.7:0/2667111171 shutdown_connections 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.016+0000 7f336159f700 1 --2- 172.21.15.7:0/2667111171 >> v2:172.21.15.113:3300/0 conn(0x7f335c0cd6d0 0x7f335c0cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.016+0000 7f336159f700 1 --2- 172.21.15.7:0/2667111171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c15dd40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.016+0000 7f336159f700 1 -- 172.21.15.7:0/2667111171 >> 172.21.15.7:0/2667111171 conn(0x7f335c071120 msgr2=0x7f335c06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.016+0000 7f336159f700 1 -- 172.21.15.7:0/2667111171 shutdown_connections 2024-01-11T07:07:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.016+0000 7f336159f700 1 -- 172.21.15.7:0/2667111171 wait complete. 2024-01-11T07:07:57.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.017+0000 7f336159f700 1 Processor -- start 2024-01-11T07:07:57.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.017+0000 7f336159f700 1 -- start start 2024-01-11T07:07:57.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f336159f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0cd6d0 0x7f335c156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:57.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f336159f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:57.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f336159f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c15e280 0x7f335c0c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f336159f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0b0440 con 0x7f335c15d8a0 2024-01-11T07:07:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f336159f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0c9180 con 0x7f335c15e280 2024-01-11T07:07:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f336159f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0b67d0 con 0x7f335c0cd6d0 2024-01-11T07:07:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c0aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50740/0 (socket says 172.21.15.7:50740) 2024-01-11T07:07:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335a7fc700 1 -- 172.21.15.7:0/2399025468 learned_addr learned my addr 172.21.15.7:0/2399025468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:57.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335b7fe700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c15e280 0x7f335c0c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335affd700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0cd6d0 0x7f335c156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335a7fc700 1 -- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0cd6d0 msgr2=0x7f335c156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335a7fc700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0cd6d0 0x7f335c156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335a7fc700 1 -- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c15e280 msgr2=0x7f335c0c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335a7fc700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c15e280 0x7f335c0c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.018+0000 7f335a7fc700 1 -- 172.21.15.7:0/2399025468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440094f0 con 0x7f335c15d8a0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.019+0000 7f335a7fc700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c0aff00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3344006f50 tx=0x7f3344017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.019+0000 7f3352ffd700 1 -- 172.21.15.7:0/2399025468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344011e40 con 0x7f335c15d8a0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.019+0000 7f3352ffd700 1 -- 172.21.15.7:0/2399025468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3344024440 con 0x7f335c15d8a0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.019+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c0c8850 con 0x7f335c15d8a0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.019+0000 7f3352ffd700 1 -- 172.21.15.7:0/2399025468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440247e0 con 0x7f335c15d8a0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.019+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c0c8e10 con 0x7f335c15d8a0 2024-01-11T07:07:57.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.020+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c0c92f0 con 0x7f335c15d8a0 2024-01-11T07:07:57.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.020+0000 7f3352ffd700 1 -- 172.21.15.7:0/2399025468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f334402d440 con 0x7f335c15d8a0 2024-01-11T07:07:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.021+0000 7f3352ffd700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f333c066f30 0x7f333c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.021+0000 7f3352ffd700 1 -- 172.21.15.7:0/2399025468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3344085f30 con 0x7f335c15d8a0 2024-01-11T07:07:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.021+0000 7f335affd700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f333c066f30 0x7f333c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:57.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.021+0000 7f335affd700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f333c066f30 0x7f333c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f334c006fd0 tx=0x7f334c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.025+0000 7f3352ffd700 1 -- 172.21.15.7:0/2399025468 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33440531a0 con 0x7f335c15d8a0 2024-01-11T07:07:57.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.282+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f335c04c3c0 con 0x7f335c15d8a0 2024-01-11T07:07:57.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.282+0000 7f3352ffd700 1 -- 172.21.15.7:0/2399025468 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f33440567c0 con 0x7f335c15d8a0 2024-01-11T07:07:57.282 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:07:57.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f333c066f30 msgr2=0x7f333c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:57.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f333c066f30 0x7f333c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f334c006fd0 tx=0x7f334c008040 comp rx=0 tx=0).stop 2024-01-11T07:07:57.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 msgr2=0x7f335c0aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:57.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c0aff00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3344006f50 tx=0x7f3344017000 comp rx=0 tx=0).stop 2024-01-11T07:07:57.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 shutdown_connections 2024-01-11T07:07:57.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f333c066f30 0x7f333c0693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f335c15e280 0x7f335c0c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f335c0cd6d0 0x7f335c156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 --2- 172.21.15.7:0/2399025468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f335c15d8a0 0x7f335c0aff00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:57.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 >> 172.21.15.7:0/2399025468 conn(0x7f335c071120 msgr2=0x7f335c157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:57.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 shutdown_connections 2024-01-11T07:07:57.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:57.284+0000 7f336159f700 1 -- 172.21.15.7:0/2399025468 wait complete. 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:07:57.298 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:07:57.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:57.299 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42340sec preferred_lft 42340sec 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:07:57.320 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:07:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:07:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:07:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:57.321 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:07:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:07:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:07:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:57.322 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:07:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:07:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:07:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:57.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:57.322 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:07:57.392 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:07:57.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:57.393 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:07:57.457 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:07:57.457 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:07:57.457 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:07:57.457 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:07:57.457 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:07:57.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:57.458 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:07:57.458 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:07:57.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:07:57.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:07:57.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:07:57.458 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:07:57.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:57.582 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:07:57.602 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:07:57.603 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:07:57.603 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:07:57.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:57.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:07:57.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:57.679 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:07:57.741 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:07:57.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:57.742 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:07:57.802 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:07:57.802 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:07:57.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:07:57.896 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:07:57.897 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:07:57.897 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:57.906 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:07:57.906 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:07:57.906 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:57.906 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:57.906 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:57.923 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:07:57.924 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:07:57.924 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:07:57.924 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:57.924 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:57.924 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:07:57.924 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:07:57.924 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:07:57.926 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:07:57.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:57.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:07:57.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:07:58.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.139+0000 7fcef59f2700 1 Processor -- start 2024-01-11T07:07:58.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.139+0000 7fcef59f2700 1 -- start start 2024-01-11T07:07:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.140+0000 7fcef59f2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef00ac3a0 0x7fcef00cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.140+0000 7fcef59f2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcef015d920 0x7fcef015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.140+0000 7fcef59f2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef015e2d0 con 0x7fcef015a2c0 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.140+0000 7fcef59f2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef006ac30 con 0x7fcef015d920 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.140+0000 7fcef59f2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00ac770 con 0x7fcef00ac3a0 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceee7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcef015d920 0x7fcef015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceee7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcef015d920 0x7fcef015dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42402/0 (socket says 172.21.15.7:42402) 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceee7fc700 1 -- 172.21.15.7:0/415643618 learned_addr learned my addr 172.21.15.7:0/415643618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceeeffd700 1 --2- 172.21.15.7:0/415643618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef00ac3a0 0x7fcef00cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceee7fc700 1 -- 172.21.15.7:0/415643618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef00ac3a0 msgr2=0x7fcef00cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceee7fc700 1 --2- 172.21.15.7:0/415643618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef00ac3a0 0x7fcef00cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceee7fc700 1 -- 172.21.15.7:0/415643618 >> v1:172.21.15.113:6789/0 conn(0x7fcef015a2c0 legacy=0x7fcef015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceee7fc700 1 -- 172.21.15.7:0/415643618 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef006a300 con 0x7fcef015d920 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceee7fc700 1 --2- 172.21.15.7:0/415643618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef015dd90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcee40099b0 tx=0x7fcee4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb60d2161581bfef server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceedffb700 1 -- 172.21.15.7:0/415643618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4009ea0 con 0x7fcef015d920 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceedffb700 1 -- 172.21.15.7:0/415643618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee400b860 con 0x7fcef015d920 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.141+0000 7fceedffb700 1 -- 172.21.15.7:0/415643618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee400bc00 con 0x7fcef015d920 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.142+0000 7fcef59f2700 1 -- 172.21.15.7:0/415643618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 msgr2=0x7fcef015dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.142+0000 7fcef59f2700 1 --2- 172.21.15.7:0/415643618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef015dd90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcee40099b0 tx=0x7fcee4009250 comp rx=0 tx=0).stop 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.142+0000 7fcef59f2700 1 -- 172.21.15.7:0/415643618 shutdown_connections 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.142+0000 7fcef59f2700 1 --2- 172.21.15.7:0/415643618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef00ac3a0 0x7fcef00cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.142+0000 7fcef59f2700 1 --2- 172.21.15.7:0/415643618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef015dd90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.142+0000 7fcef59f2700 1 -- 172.21.15.7:0/415643618 >> 172.21.15.7:0/415643618 conn(0x7fcef00cd090 msgr2=0x7fcef00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.142+0000 7fcef59f2700 1 -- 172.21.15.7:0/415643618 shutdown_connections 2024-01-11T07:07:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.142+0000 7fcef59f2700 1 -- 172.21.15.7:0/415643618 wait complete. 2024-01-11T07:07:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.144+0000 7fcef59f2700 1 Processor -- start 2024-01-11T07:07:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.144+0000 7fcef59f2700 1 -- start start 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fcef59f2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef00ac3a0 0x7fcef006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fcef59f2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef015a2c0 0x7fcef0142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fcef59f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef016d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fcef59f2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef006edf0 con 0x7fcef015d920 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fcef59f2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef006e960 con 0x7fcef00ac3a0 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fcef59f2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef006dd50 con 0x7fcef015a2c0 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef016d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef016d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42408/0 (socket says 172.21.15.7:42408) 2024-01-11T07:07:58.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceef7fe700 1 -- 172.21.15.7:0/2880089301 learned_addr learned my addr 172.21.15.7:0/2880089301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceeeffd700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef00ac3a0 0x7fcef006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceee7fc700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef015a2c0 0x7fcef0142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceef7fe700 1 -- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef015a2c0 msgr2=0x7fcef0142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceef7fe700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef015a2c0 0x7fcef0142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceef7fe700 1 -- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef00ac3a0 msgr2=0x7fcef006d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.145+0000 7fceef7fe700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef00ac3a0 0x7fcef006d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.146+0000 7fceef7fe700 1 -- 172.21.15.7:0/2880089301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40094f0 con 0x7fcef015d920 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.146+0000 7fceef7fe700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef016d790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcee800c840 tx=0x7fcee8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.146+0000 7fced6ffd700 1 -- 172.21.15.7:0/2880089301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8012630 con 0x7fcef015d920 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.146+0000 7fced6ffd700 1 -- 172.21.15.7:0/2880089301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee80127b0 con 0x7fcef015d920 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.146+0000 7fced6ffd700 1 -- 172.21.15.7:0/2880089301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8012b70 con 0x7fcef015d920 2024-01-11T07:07:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.146+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef006e030 con 0x7fcef015d920 2024-01-11T07:07:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.146+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef006e5f0 con 0x7fcef015d920 2024-01-11T07:07:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.147+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef004c3c0 con 0x7fcef015d920 2024-01-11T07:07:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.148+0000 7fced6ffd700 1 -- 172.21.15.7:0/2880089301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcee8024420 con 0x7fcef015d920 2024-01-11T07:07:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.148+0000 7fced6ffd700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcedc066f30 0x7fcedc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.148+0000 7fced6ffd700 1 -- 172.21.15.7:0/2880089301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcee8084de0 con 0x7fcef015d920 2024-01-11T07:07:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.149+0000 7fceeeffd700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcedc066f30 0x7fcedc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.149+0000 7fceeeffd700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcedc066f30 0x7fcedc0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009d10 tx=0x7fcee00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.152+0000 7fced6ffd700 1 -- 172.21.15.7:0/2880089301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcee8052030 con 0x7fcef015d920 2024-01-11T07:07:58.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.406+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcef006a5e0 con 0x7fcef015d920 2024-01-11T07:07:58.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.406+0000 7fced6ffd700 1 -- 172.21.15.7:0/2880089301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcee8055650 con 0x7fcef015d920 2024-01-11T07:07:58.406 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcedc066f30 msgr2=0x7fcedc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcedc066f30 0x7fcedc0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009d10 tx=0x7fcee00091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 msgr2=0x7fcef016d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef016d790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcee800c840 tx=0x7fcee8001670 comp rx=0 tx=0).stop 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 shutdown_connections 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcedc066f30 0x7fcedc0693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcef015d920 0x7fcef016d790 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef00ac3a0 0x7fcef006d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 --2- 172.21.15.7:0/2880089301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef015a2c0 0x7fcef0142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 >> 172.21.15.7:0/2880089301 conn(0x7fcef00cd090 msgr2=0x7fcef0157f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 shutdown_connections 2024-01-11T07:07:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:58.409+0000 7fcef59f2700 1 -- 172.21.15.7:0/2880089301 wait complete. 2024-01-11T07:07:58.421 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:07:58.422 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:07:58.422 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:07:58.422 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:07:58.422 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:07:58.422 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:07:58.422 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:07:58.422 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:07:58.422 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:07:58.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:58.422 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:07:58.445 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:07:58.445 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:07:58.445 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:07:58.445 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:58.445 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:07:58.445 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42382sec preferred_lft 42382sec 2024-01-11T07:07:58.446 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:07:58.447 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:07:58.448 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:07:58.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:58.448 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:07:58.516 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:07:58.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:58.518 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:07:58.580 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:07:58.580 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:07:58.580 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:07:58.580 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:07:58.580 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:07:58.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:58.580 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:07:58.581 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:07:58.581 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:07:58.581 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:07:58.581 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:07:58.581 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:07:58.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:58.694 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:07:58.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:07:58.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:07:58.712 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:07:58.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:58.712 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:07:58.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:58.780 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:07:58.837 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:07:58.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:07:58.838 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:07:58.894 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:07:58.894 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:07:58.894 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:07:58.979 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:07:58.979 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:07:58.979 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:58.985 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:07:58.985 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:07:58.985 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:58.985 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:58.985 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:59.002 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:07:59.003 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:07:59.003 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:07:59.003 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:07:59.003 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:07:59.003 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:07:59.003 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:07:59.003 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:07:59.005 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:07:59.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:07:59.005 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:07:59.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:07:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.215+0000 7f3e93f28700 1 Processor -- start 2024-01-11T07:07:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.216+0000 7f3e93f28700 1 -- start start 2024-01-11T07:07:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.216+0000 7f3e93f28700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e8c159c20 0x7f3e8c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e93f28700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c067de0 con 0x7f3e8c159ff0 2024-01-11T07:07:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e93f28700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c0b20e0 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e93f28700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8c069570 con 0x7f3e8c159c20 2024-01-11T07:07:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e914c3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3e8c159ff0 0x7f3e8c068270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51294/0 (socket says 172.21.15.7:51294) 2024-01-11T07:07:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e914c3700 1 -- 172.21.15.7:0/1047009925 learned_addr learned my addr 172.21.15.7:0/1047009925 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:07:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e91cc4700 1 --2- 172.21.15.7:0/1047009925 >> v2:172.21.15.113:3300/0 conn(0x7f3e8c159c20 0x7f3e8c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e90cc2700 1 -- 172.21.15.7:0/1047009925 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951707405 0 0) 0x7f3e8c067de0 con 0x7f3e8c159ff0 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e90cc2700 1 -- 172.21.15.7:0/1047009925 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e78003570 con 0x7f3e8c159ff0 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e90cc2700 1 -- 172.21.15.7:0/1047009925 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498973172 0 0) 0x7f3e8c0b20e0 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e90cc2700 1 -- 172.21.15.7:0/1047009925 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8c067de0 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e91cc4700 1 -- 172.21.15.7:0/1047009925 >> v1:172.21.15.113:6789/0 conn(0x7f3e8c06f4d0 legacy=0x7f3e8c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e91cc4700 1 -- 172.21.15.7:0/1047009925 >> v1:172.21.15.7:6789/0 conn(0x7f3e8c159ff0 legacy=0x7f3e8c068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e90cc2700 1 -- 172.21.15.7:0/1047009925 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2086803690 0 0) 0x7f3e78003570 con 0x7f3e8c159ff0 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.217+0000 7f3e91cc4700 1 -- 172.21.15.7:0/1047009925 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c06f950 con 0x7f3e8c159c20 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e91cc4700 1 --2- 172.21.15.7:0/1047009925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 0x7f3e8c0b1820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e880098e0 tx=0x7f3e880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=762827de34042343 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e90cc2700 1 -- 172.21.15.7:0/1047009925 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88011630 con 0x7f3e8c159c20 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e90cc2700 1 -- 172.21.15.7:0/1047009925 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e88011790 con 0x7f3e8c159c20 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e90cc2700 1 -- 172.21.15.7:0/1047009925 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88011b30 con 0x7f3e8c159c20 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e93f28700 1 -- 172.21.15.7:0/1047009925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 msgr2=0x7f3e8c0b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e93f28700 1 --2- 172.21.15.7:0/1047009925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 0x7f3e8c0b1820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e880098e0 tx=0x7f3e880091a0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e93f28700 1 -- 172.21.15.7:0/1047009925 shutdown_connections 2024-01-11T07:07:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e93f28700 1 --2- 172.21.15.7:0/1047009925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 0x7f3e8c0b1820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e93f28700 1 -- 172.21.15.7:0/1047009925 >> 172.21.15.7:0/1047009925 conn(0x7f3e8c071160 msgr2=0x7f3e8c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e93f28700 1 -- 172.21.15.7:0/1047009925 shutdown_connections 2024-01-11T07:07:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.218+0000 7f3e93f28700 1 -- 172.21.15.7:0/1047009925 wait complete. 2024-01-11T07:07:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.220+0000 7f3e93f28700 1 Processor -- start 2024-01-11T07:07:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.220+0000 7f3e93f28700 1 -- start start 2024-01-11T07:07:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.220+0000 7f3e93f28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8c06f4d0 0x7f3e8c156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.220+0000 7f3e93f28700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 0x7f3e8c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.220+0000 7f3e93f28700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e8c159ff0 0x7f3e8c0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.220+0000 7f3e93f28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c156d70 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.220+0000 7f3e93f28700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0c89c0 con 0x7f3e8c159c20 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.220+0000 7f3e93f28700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0c7db0 con 0x7f3e8c159ff0 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8c06f4d0 0x7f3e8c156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8c06f4d0 0x7f3e8c156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42410/0 (socket says 172.21.15.7:42410) 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 -- 172.21.15.7:0/306119637 learned_addr learned my addr 172.21.15.7:0/306119637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e914c3700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 0x7f3e8c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e924c5700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e8c159ff0 0x7f3e8c0c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 -- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e8c159ff0 msgr2=0x7f3e8c0c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e8c159ff0 0x7f3e8c0c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 -- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 msgr2=0x7f3e8c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 0x7f3e8c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 -- 172.21.15.7:0/306119637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009420 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e914c3700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 0x7f3e8c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:59.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e924c5700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e8c159ff0 0x7f3e8c0c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:07:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e91cc4700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8c06f4d0 0x7f3e8c156830 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c06a1d0 tx=0x7f3e88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e81ffb700 1 -- 172.21.15.7:0/306119637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8801f070 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e81ffb700 1 -- 172.21.15.7:0/306119637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e88011b10 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c0c8090 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.221+0000 7f3e81ffb700 1 -- 172.21.15.7:0/306119637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88023640 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.222+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c0c8650 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.222+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8c04c3c0 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.223+0000 7f3e81ffb700 1 -- 172.21.15.7:0/306119637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3e8802c3b0 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.223+0000 7f3e81ffb700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e70066f30 0x7f3e700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.224+0000 7f3e81ffb700 1 -- 172.21.15.7:0/306119637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e88085950 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.224+0000 7f3e914c3700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e70066f30 0x7f3e700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.224+0000 7f3e914c3700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e70066f30 0x7f3e700693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0cc600 tx=0x7f3e7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:59.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.228+0000 7f3e81ffb700 1 -- 172.21.15.7:0/306119637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e88052b20 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.479+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e8c0c8ca0 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.480+0000 7f3e81ffb700 1 -- 172.21.15.7:0/306119637 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f3e88056140 con 0x7f3e8c06f4d0 2024-01-11T07:07:59.480 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:07:59.480 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:07:50.072668+0000","modified":"2024-01-11T07:07:53.685303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/3458914988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3458914988},{"type":"v1","addr":"172.21.15.7:6834","nonce":3458914988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e70066f30 msgr2=0x7f3e700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e70066f30 0x7f3e700693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0cc600 tx=0x7f3e7c009040 comp rx=0 tx=0).stop 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8c06f4d0 msgr2=0x7f3e8c156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8c06f4d0 0x7f3e8c156830 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c06a1d0 tx=0x7f3e88017000 comp rx=0 tx=0).stop 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 shutdown_connections 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3e70066f30 0x7f3e700693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e8c159ff0 0x7f3e8c0c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8c06f4d0 0x7f3e8c156830 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 --2- 172.21.15.7:0/306119637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e8c159c20 0x7f3e8c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 >> 172.21.15.7:0/306119637 conn(0x7f3e8c071160 msgr2=0x7f3e8c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 shutdown_connections 2024-01-11T07:07:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.482+0000 7f3e93f28700 1 -- 172.21.15.7:0/306119637 wait complete. 2024-01-11T07:07:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T07:07:59.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:07:59.646 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":4917,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.9646917150000007,"mdsmap_epoch":16,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":17.857314511999999} 2024-01-11T07:07:59.647 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 4917, "mdsmap_epoch": 16, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 8.964691715, "state": "up:active", "uptime": 17.857314512, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:07:59.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-01-11T07:07:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.778+0000 7fb4cbce7700 1 Processor -- start 2024-01-11T07:07:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.778+0000 7fb4cbce7700 1 -- start start 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4cbce7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4c4063230 0x7fb4c4063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4cbce7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4c415a280 0x7fb4c40aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4cbce7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c40ac5b0 0x7fb4c40aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4cbce7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c415d880 con 0x7fb4c415a280 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4cbce7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c406c5b0 con 0x7fb4c4063230 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4cbce7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4063b40 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4c9282700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4c415a280 0x7fb4c40aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4c9282700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4c415a280 0x7fb4c40aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42424/0 (socket says 172.21.15.7:42424) 2024-01-11T07:07:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4c9282700 1 -- 172.21.15.7:0/3148920346 learned_addr learned my addr 172.21.15.7:0/3148920346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4c9a83700 1 --2- 172.21.15.7:0/3148920346 >> v2:172.21.15.113:3300/0 conn(0x7fb4c4063230 0x7fb4c4063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.779+0000 7fb4ca284700 1 --2- 172.21.15.7:0/3148920346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c40ac5b0 0x7fb4c40aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c9282700 1 -- 172.21.15.7:0/3148920346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c40ac5b0 msgr2=0x7fb4c40aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c9282700 1 --2- 172.21.15.7:0/3148920346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c40ac5b0 0x7fb4c40aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c9282700 1 -- 172.21.15.7:0/3148920346 >> v2:172.21.15.113:3300/0 conn(0x7fb4c4063230 msgr2=0x7fb4c4063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c9282700 1 --2- 172.21.15.7:0/3148920346 >> v2:172.21.15.113:3300/0 conn(0x7fb4c4063230 0x7fb4c4063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c9282700 1 -- 172.21.15.7:0/3148920346 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c415a7f0 con 0x7fb4c415a280 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c9a83700 1 --2- 172.21.15.7:0/3148920346 >> v2:172.21.15.113:3300/0 conn(0x7fb4c4063230 0x7fb4c4063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c9282700 1 --2- 172.21.15.7:0/3148920346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 0x7fb4c40aa9f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40ad1f0 tx=0x7fb4b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f2f25d3e414857 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c8a81700 1 -- 172.21.15.7:0/3148920346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400b640 con 0x7fb4c415a280 2024-01-11T07:07:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c8a81700 1 -- 172.21.15.7:0/3148920346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400b7a0 con 0x7fb4c415a280 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.780+0000 7fb4c8a81700 1 -- 172.21.15.7:0/3148920346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400bb60 con 0x7fb4c415a280 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 -- 172.21.15.7:0/3148920346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 msgr2=0x7fb4c40aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/3148920346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 0x7fb4c40aa9f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40ad1f0 tx=0x7fb4b4009250 comp rx=0 tx=0).stop 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 -- 172.21.15.7:0/3148920346 shutdown_connections 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/3148920346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c40ac5b0 0x7fb4c40aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/3148920346 >> v2:172.21.15.113:3300/0 conn(0x7fb4c4063230 0x7fb4c4063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/3148920346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 0x7fb4c40aa9f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 -- 172.21.15.7:0/3148920346 >> 172.21.15.7:0/3148920346 conn(0x7fb4c40680e0 msgr2=0x7fb4c40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 -- 172.21.15.7:0/3148920346 shutdown_connections 2024-01-11T07:07:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.781+0000 7fb4cbce7700 1 -- 172.21.15.7:0/3148920346 wait complete. 2024-01-11T07:07:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.782+0000 7fb4cbce7700 1 Processor -- start 2024-01-11T07:07:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.782+0000 7fb4cbce7700 1 -- start start 2024-01-11T07:07:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4cbce7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c4063230 0x7fb4c40c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4cbce7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c40ac5b0 0x7fb4c40ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4cbce7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 0x7fb4c40c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4cbce7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b1690 con 0x7fb4c415a280 2024-01-11T07:07:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4cbce7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b1f30 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4cbce7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40c36f0 con 0x7fb4c4063230 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9282700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c40ac5b0 0x7fb4c40ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9a83700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c4063230 0x7fb4c40c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4ca284700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 0x7fb4c40c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9282700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c40ac5b0 0x7fb4c40ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:34302/0 (socket says 172.21.15.7:34302) 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9a83700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c4063230 0x7fb4c40c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:44838/0 (socket says 172.21.15.7:44838) 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9282700 1 -- 172.21.15.7:0/2875501111 learned_addr learned my addr 172.21.15.7:0/2875501111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9282700 1 -- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c4063230 msgr2=0x7fb4c40c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9282700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c4063230 0x7fb4c40c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9282700 1 -- 172.21.15.7:0/2875501111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 msgr2=0x7fb4c40c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9282700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 0x7fb4c40c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4c9282700 1 -- 172.21.15.7:0/2875501111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c0009420 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.783+0000 7fb4ca284700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 0x7fb4c40c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.784+0000 7fb4c9282700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c40ac5b0 0x7fb4c40ad220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4006fd0 tx=0x7fb4b4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.784+0000 7fb4b9ffb700 1 -- 172.21.15.7:0/2875501111 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400b910 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.784+0000 7fb4b9ffb700 1 -- 172.21.15.7:0/2875501111 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400ba70 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.784+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b40094f0 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.784+0000 7fb4b9ffb700 1 -- 172.21.15.7:0/2875501111 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b401a680 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.784+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c40b18e0 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.785+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c404c3c0 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.785+0000 7fb4b9ffb700 1 -- 172.21.15.7:0/2875501111 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb4b401aa30 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.786+0000 7fb4b9ffb700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4a8066f30 0x7fb4a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.786+0000 7fb4b9ffb700 1 -- 172.21.15.7:0/2875501111 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb4b4085cd0 con 0x7fb4c40ac5b0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.786+0000 7fb4c9a83700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4a8066f30 0x7fb4a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:07:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.787+0000 7fb4c9a83700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4a8066f30 0x7fb4a80693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009ea0 tx=0x7fb4c00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:07:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:07:59.791+0000 7fb4b9ffb700 1 -- 172.21.15.7:0/2875501111 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4b4052f40 con 0x7fb4c40ac5b0 2024-01-11T07:08:00.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.080+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fb4c40b2070 con 0x7fb4c40ac5b0 2024-01-11T07:08:00.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.081+0000 7fb4b9ffb700 1 -- 172.21.15.7:0/2875501111 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1618 (secure 0 0 0) 0x7fb4b4056560 con 0x7fb4c40ac5b0 2024-01-11T07:08:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:00.081 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "stats": { 2024-01-11T07:08:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "total_bytes": 1600000000, 2024-01-11T07:08:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "total_avail_bytes": 1597632452, 2024-01-11T07:08:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_bytes": 2367548, 2024-01-11T07:08:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_raw_bytes": 2367548, 2024-01-11T07:08:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_raw_ratio": 0.0014797175535932183, 2024-01-11T07:08:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "num_osds": 8, 2024-01-11T07:08:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "num_per_pool_osds": 0, 2024-01-11T07:08:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "num_per_pool_omap_osds": 0 2024-01-11T07:08:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "stats_by_class": { 2024-01-11T07:08:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "hdd": { 2024-01-11T07:08:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "total_bytes": 1600000000, 2024-01-11T07:08:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "total_avail_bytes": 1597632452, 2024-01-11T07:08:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_bytes": 2367548, 2024-01-11T07:08:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_raw_bytes": 2367548, 2024-01-11T07:08:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "total_used_raw_ratio": 0.0014797175535932183 2024-01-11T07:08:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:08:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": ".mgr", 2024-01-11T07:08:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1, 2024-01-11T07:08:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "stats": { 2024-01-11T07:08:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "stored": 590368, 2024-01-11T07:08:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "objects": 2, 2024-01-11T07:08:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "kb_used": 577, 2024-01-11T07:08:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "bytes_used": 590368, 2024-01-11T07:08:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "percent_used": 0.000529633485712111, 2024-01-11T07:08:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "max_avail": 557041216 2024-01-11T07:08:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephfs_metadata", 2024-01-11T07:08:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4, 2024-01-11T07:08:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "stats": { 2024-01-11T07:08:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "stored": 2306, 2024-01-11T07:08:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "objects": 22, 2024-01-11T07:08:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "kb_used": 3, 2024-01-11T07:08:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "bytes_used": 2306, 2024-01-11T07:08:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "percent_used": 2.0698605567304185e-06, 2024-01-11T07:08:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max_avail": 557041216 2024-01-11T07:08:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephfs_data", 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5, 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "stats": { 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "stored": 0, 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "objects": 0, 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "kb_used": 0, 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "bytes_used": 0, 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "percent_used": 0, 2024-01-11T07:08:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max_avail": 557041216 2024-01-11T07:08:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:00.088 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:00.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4a8066f30 msgr2=0x7fb4a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:00.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4a8066f30 0x7fb4a80693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009ea0 tx=0x7fb4c00091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c40ac5b0 msgr2=0x7fb4c40ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:00.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c40ac5b0 0x7fb4c40ad220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4006fd0 tx=0x7fb4b4017000 comp rx=0 tx=0).stop 2024-01-11T07:08:00.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 shutdown_connections 2024-01-11T07:08:00.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb4a8066f30 0x7fb4a80693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c415a280 0x7fb4c40c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4c4063230 0x7fb4c40c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 --2- 172.21.15.7:0/2875501111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4c40ac5b0 0x7fb4c40ad220 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 >> 172.21.15.7:0/2875501111 conn(0x7fb4c40680e0 msgr2=0x7fb4c406be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:00.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 shutdown_connections 2024-01-11T07:08:00.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.083+0000 7fb4cbce7700 1 -- 172.21.15.7:0/2875501111 wait complete. 2024-01-11T07:08:00.096 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:08:00.096 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:08:00.096 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-11T07:08:00.096 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-11T07:08:00.096 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-11T07:08:00.096 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-11T07:08:00.096 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:08:00.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:08:00.336 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:08:00.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:00.360 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T07:08:00.478 INFO:teuthology.orchestra.run.smithi007.stdout:client5112 2024-01-11T07:08:00.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:00.479 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c0c5e15-bc38-4016-89e4-891c3586efbb.client5112/osdmap') 2024-01-11T07:08:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 34 barrier 34 flags 0x588000 2024-01-11T07:08:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-11T07:08:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-11T07:08:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-11T07:08:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-11T07:08:00.572 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.113:6801 100% (exists, up) 100% -1 2024-01-11T07:08:00.572 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.113:6817 100% (exists, up) 100% -1 2024-01-11T07:08:00.572 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.113:6809 100% (exists, up) 100% -1 2024-01-11T07:08:00.572 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.113:6825 100% (exists, up) 100% -1 2024-01-11T07:08:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-11T07:08:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-11T07:08:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:791 bytes copied, 5.3751e-05 s, 14.7 MB/s 2024-01-11T07:08:00.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-01-11T07:08:00.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.712+0000 7f1090978700 1 Processor -- start 2024-01-11T07:08:00.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.712+0000 7f1090978700 1 -- start start 2024-01-11T07:08:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.713+0000 7f1090978700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f108c069320 0x7f108c1649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.713+0000 7f1090978700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0ce9b0 con 0x7f108c159c90 2024-01-11T07:08:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f1090978700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0cf530 con 0x7f108c15a060 2024-01-11T07:08:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f1090978700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0b1f30 con 0x7f108c069320 2024-01-11T07:08:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f108c159c90 0x7f108c0b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51310/0 (socket says 172.21.15.7:51310) 2024-01-11T07:08:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108a59c700 1 -- 172.21.15.7:0/2122731011 learned_addr learned my addr 172.21.15.7:0/2122731011 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108ad9d700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f108c069320 0x7f108c1649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611542518 0 0) 0x7f108c0ce9b0 con 0x7f108c159c90 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1074003570 con 0x7f108c159c90 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624700529 0 0) 0x7f108c0cf530 con 0x7f108c15a060 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108c0ce9b0 con 0x7f108c15a060 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1182085448 0 0) 0x7f1074003570 con 0x7f108c159c90 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f108c069320 msgr2=0x7f108c1649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f108c069320 0x7f108c1649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 >> v1:172.21.15.113:6789/0 conn(0x7f108c15a060 legacy=0x7f108c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c0cf9c0 con 0x7f108c159c90 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f107c001470 con 0x7f108c159c90 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 >> v1:172.21.15.7:6789/0 conn(0x7f108c159c90 legacy=0x7f108c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f108c069320 0x7f1074003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1074004180 0x7f1074004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1074004d50 0x7f10740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107400bb70 con 0x7f1074004180 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107400bce0 con 0x7f1074004d50 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.714+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107400bea0 con 0x7f108c069320 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f1089d9b700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1074004180 0x7f1074004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f108a59c700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f108c069320 0x7f1074003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f108ad9d700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1074004d50 0x7f10740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f1089d9b700 1 -- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f108c069320 msgr2=0x7f1074003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f1089d9b700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f108c069320 0x7f1074003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f1089d9b700 1 -- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1074004d50 msgr2=0x7f10740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f1089d9b700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1074004d50 0x7f10740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f1089d9b700 1 -- 172.21.15.7:0/2122731011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1074003570 con 0x7f1074004180 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f108ad9d700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1074004d50 0x7f10740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f1089d9b700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1074004180 0x7f1074004810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1080001a10 tx=0x7f1080007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bab3fa541138b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000c5d0 con 0x7f1074004180 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108000c730 con 0x7f1074004180 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.715+0000 7f108959a700 1 -- 172.21.15.7:0/2122731011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000cad0 con 0x7f1074004180 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.716+0000 7f1090978700 1 -- 172.21.15.7:0/2122731011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1074004180 msgr2=0x7f1074004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.716+0000 7f1090978700 1 --2- 172.21.15.7:0/2122731011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1074004180 0x7f1074004810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1080001a10 tx=0x7f1080007d30 comp rx=0 tx=0).stop 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.716+0000 7f108ad9d700 1 -- 172.21.15.7:0/2122731011 reap_dead start 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.716+0000 7f1090978700 1 -- 172.21.15.7:0/2122731011 shutdown_connections 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.716+0000 7f1090978700 1 -- 172.21.15.7:0/2122731011 >> 172.21.15.7:0/2122731011 conn(0x7f108c071100 msgr2=0x7f108c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.716+0000 7f1090978700 1 -- 172.21.15.7:0/2122731011 shutdown_connections 2024-01-11T07:08:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.716+0000 7f1090978700 1 -- 172.21.15.7:0/2122731011 wait complete. 2024-01-11T07:08:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.717+0000 7f1090978700 1 Processor -- start 2024-01-11T07:08:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1090978700 1 -- start start 2024-01-11T07:08:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1090978700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1074004180 0x7f108c14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1090978700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f108c16adb0 0x7f108c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1090978700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c15c790 0x7f108c16d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1090978700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c15cc30 con 0x7f108c16adb0 2024-01-11T07:08:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1090978700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c16a030 con 0x7f108c15c790 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1090978700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c16a2d0 con 0x7f1074004180 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1089d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f108c16adb0 0x7f108c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1089d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f108c16adb0 0x7f108c15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42460/0 (socket says 172.21.15.7:42460) 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.718+0000 7f1089d9b700 1 -- 172.21.15.7:0/4126538245 learned_addr learned my addr 172.21.15.7:0/4126538245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f108ad9d700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c15c790 0x7f108c16d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f108a59c700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1074004180 0x7f108c14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f1089d9b700 1 -- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1074004180 msgr2=0x7f108c14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f1089d9b700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1074004180 0x7f108c14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f1089d9b700 1 -- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c15c790 msgr2=0x7f108c16d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f1089d9b700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c15c790 0x7f108c16d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f1089d9b700 1 -- 172.21.15.7:0/4126538245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c16a5b0 con 0x7f108c16adb0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f1089d9b700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f108c16adb0 0x7f108c15c250 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f108000c3c0 tx=0x7f1080015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f107a7fc700 1 -- 172.21.15.7:0/4126538245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080023070 con 0x7f108c16adb0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f107a7fc700 1 -- 172.21.15.7:0/4126538245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1080000fe0 con 0x7f108c16adb0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c16a890 con 0x7f108c16adb0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f107a7fc700 1 -- 172.21.15.7:0/4126538245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000c5d0 con 0x7f108c16adb0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.719+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c16db20 con 0x7f108c16adb0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.720+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c04c3c0 con 0x7f108c16adb0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.721+0000 7f107a7fc700 1 -- 172.21.15.7:0/4126538245 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1080019070 con 0x7f108c16adb0 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.721+0000 7f107a7fc700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f106c066f30 0x7f106c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:00.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.721+0000 7f107a7fc700 1 -- 172.21.15.7:0/4126538245 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1080085390 con 0x7f108c16adb0 2024-01-11T07:08:00.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.721+0000 7f108a59c700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f106c066f30 0x7f106c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:00.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.722+0000 7f108a59c700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f106c066f30 0x7f106c0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f107c007e90 tx=0x7f107c001df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:00.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.726+0000 7f107a7fc700 1 -- 172.21.15.7:0/4126538245 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1080052600 con 0x7f108c16adb0 2024-01-11T07:08:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:00.940+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f108c16aa20 con 0x7f108c16adb0 2024-01-11T07:08:01.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.182+0000 7f107a7fc700 1 -- 172.21.15.7:0/4126538245 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v35) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1080017020 con 0x7f108c16adb0 2024-01-11T07:08:01.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f106c066f30 msgr2=0x7f106c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:01.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f106c066f30 0x7f106c0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f107c007e90 tx=0x7f107c001df0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f108c16adb0 msgr2=0x7f108c15c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f108c16adb0 0x7f108c15c250 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f108000c3c0 tx=0x7f1080015000 comp rx=0 tx=0).stop 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 shutdown_connections 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f106c066f30 0x7f106c0693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c15c790 0x7f108c16d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1074004180 0x7f108c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 --2- 172.21.15.7:0/4126538245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f108c16adb0 0x7f108c15c250 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 >> 172.21.15.7:0/4126538245 conn(0x7f108c071100 msgr2=0x7f108c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 shutdown_connections 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.186+0000 7f1090978700 1 -- 172.21.15.7:0/4126538245 wait complete. 2024-01-11T07:08:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:pauserd,pausewr is set 2024-01-11T07:08:01.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-01-11T07:08:01.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.338+0000 7f861e165700 1 Processor -- start 2024-01-11T07:08:01.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.338+0000 7f861e165700 1 -- start start 2024-01-11T07:08:01.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f861e165700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86180cafb0 0x7f86180ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:01.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f861e165700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8618158280 0x7f86180b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:01.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f861e165700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86180b0970 0x7f8618159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f861e165700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86180b0df0 con 0x7f8618158280 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f861e165700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8618159300 con 0x7f86180b0970 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f861e165700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86180cf260 con 0x7f86180cafb0 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8616ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8618158280 0x7f86180b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8616ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8618158280 0x7f86180b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42464/0 (socket says 172.21.15.7:42464) 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8616ffd700 1 -- 172.21.15.7:0/3693585599 learned_addr learned my addr 172.21.15.7:0/3693585599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f86177fe700 1 --2- 172.21.15.7:0/3693585599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86180cafb0 0x7f86180ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8617fff700 1 --2- 172.21.15.7:0/3693585599 >> v2:172.21.15.113:3300/0 conn(0x7f86180b0970 0x7f8618159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8616ffd700 1 -- 172.21.15.7:0/3693585599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86180cafb0 msgr2=0x7f86180ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8616ffd700 1 --2- 172.21.15.7:0/3693585599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86180cafb0 0x7f86180ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8616ffd700 1 -- 172.21.15.7:0/3693585599 >> v2:172.21.15.113:3300/0 conn(0x7f86180b0970 msgr2=0x7f8618159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8616ffd700 1 --2- 172.21.15.7:0/3693585599 >> v2:172.21.15.113:3300/0 conn(0x7f86180b0970 0x7f8618159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.339+0000 7f8616ffd700 1 -- 172.21.15.7:0/3693585599 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180cf540 con 0x7f8618158280 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f8616ffd700 1 --2- 172.21.15.7:0/3693585599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618158280 0x7f86180b0430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8618063d90 tx=0x7f860c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb08a048fa8783bc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f86167fc700 1 -- 172.21.15.7:0/3693585599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c00b860 con 0x7f8618158280 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f86167fc700 1 -- 172.21.15.7:0/3693585599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860c00b9c0 con 0x7f8618158280 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f86167fc700 1 -- 172.21.15.7:0/3693585599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c00bd60 con 0x7f8618158280 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f861e165700 1 -- 172.21.15.7:0/3693585599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618158280 msgr2=0x7f86180b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f861e165700 1 --2- 172.21.15.7:0/3693585599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618158280 0x7f86180b0430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8618063d90 tx=0x7f860c009250 comp rx=0 tx=0).stop 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f861e165700 1 -- 172.21.15.7:0/3693585599 shutdown_connections 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f861e165700 1 --2- 172.21.15.7:0/3693585599 >> v2:172.21.15.113:3300/0 conn(0x7f86180b0970 0x7f8618159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f861e165700 1 --2- 172.21.15.7:0/3693585599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f86180cafb0 0x7f86180ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f861e165700 1 --2- 172.21.15.7:0/3693585599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8618158280 0x7f86180b0430 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.340+0000 7f861e165700 1 -- 172.21.15.7:0/3693585599 >> 172.21.15.7:0/3693585599 conn(0x7f8618071100 msgr2=0x7f861806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.341+0000 7f861e165700 1 -- 172.21.15.7:0/3693585599 shutdown_connections 2024-01-11T07:08:01.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.341+0000 7f861e165700 1 -- 172.21.15.7:0/3693585599 wait complete. 2024-01-11T07:08:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.342+0000 7f861e165700 1 Processor -- start 2024-01-11T07:08:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.342+0000 7f861e165700 1 -- start start 2024-01-11T07:08:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f861e165700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86180b0970 0x7f861814f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f861e165700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86180cafb0 0x7f861815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f861e165700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8618158280 0x7f861815e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f861e165700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180681a0 con 0x7f86180b0970 2024-01-11T07:08:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f861e165700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8618150ff0 con 0x7f86180cafb0 2024-01-11T07:08:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f861e165700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8618067b50 con 0x7f8618158280 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86180b0970 0x7f861814f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86180b0970 0x7f861814f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42472/0 (socket says 172.21.15.7:42472) 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 -- 172.21.15.7:0/543850163 learned_addr learned my addr 172.21.15.7:0/543850163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f8616ffd700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86180cafb0 0x7f861815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f8617fff700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8618158280 0x7f861815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 -- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8618158280 msgr2=0x7f861815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8618158280 0x7f861815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 -- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86180cafb0 msgr2=0x7f861815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86180cafb0 0x7f861815dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 -- 172.21.15.7:0/543850163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c0094f0 con 0x7f86180b0970 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f8616ffd700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86180cafb0 0x7f861815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f8617fff700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8618158280 0x7f861815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f86177fe700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86180b0970 0x7f861814f9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f860800d890 tx=0x7f8608008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860800b600 con 0x7f86180b0970 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.343+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860800b760 con 0x7f86180b0970 2024-01-11T07:08:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.344+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8618067e30 con 0x7f86180b0970 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.344+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860800bae0 con 0x7f86180b0970 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.344+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86181512d0 con 0x7f86180b0970 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.344+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861804c3c0 con 0x7f86180b0970 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.345+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f860801e460 con 0x7f86180b0970 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.345+0000 7f85ff7fe700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8604066f30 0x7f86040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.346+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8608084be0 con 0x7f86180b0970 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.346+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=36+}) v3 -- 0x7f860406ea50 con 0x7f86180b0970 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.346+0000 7f8616ffd700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8604066f30 0x7f86040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.346+0000 7f8616ffd700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8604066f30 0x7f86040693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8618063d90 tx=0x7f860c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:01.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.350+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8608051e50 con 0x7f86180b0970 2024-01-11T07:08:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:01.566+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f86180cf820 con 0x7f86180b0970 2024-01-11T07:08:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.189+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(36..36 src has 1..36) v4 ==== 280+0+0 (secure 0 0 0) 0x7f8608056210 con 0x7f86180b0970 2024-01-11T07:08:02.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.189+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=37}) v3 -- 0x7f860406f340 con 0x7f86180b0970 2024-01-11T07:08:02.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.192+0000 7f85ff7fe700 1 -- 172.21.15.7:0/543850163 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8608055470 con 0x7f86180b0970 2024-01-11T07:08:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8604066f30 msgr2=0x7f86040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8604066f30 0x7f86040693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8618063d90 tx=0x7f860c017000 comp rx=0 tx=0).stop 2024-01-11T07:08:02.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86180b0970 msgr2=0x7f861814f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86180b0970 0x7f861814f9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f860800d890 tx=0x7f8608008040 comp rx=0 tx=0).stop 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 shutdown_connections 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8604066f30 0x7f86040693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8618158280 0x7f861815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86180b0970 0x7f861814f9f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 --2- 172.21.15.7:0/543850163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86180cafb0 0x7f861815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 >> 172.21.15.7:0/543850163 conn(0x7f8618071100 msgr2=0x7f86180ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.195+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 shutdown_connections 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.196+0000 7f861e165700 1 -- 172.21.15.7:0/543850163 wait complete. 2024-01-11T07:08:02.196 INFO:teuthology.orchestra.run.smithi007.stderr:pauserd,pausewr is unset 2024-01-11T07:08:02.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.346+0000 7f8217bd8700 1 Processor -- start 2024-01-11T07:08:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.346+0000 7f8217bd8700 1 -- start start 2024-01-11T07:08:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8217bd8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f821015d920 0x7f821015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8217bd8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82100aa8c0 0x7f82100aad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8217bd8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821015e2d0 con 0x7f82100ac3a0 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8217bd8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821006ac30 con 0x7f82100aa8c0 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8217bd8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82100cb370 con 0x7f821015d920 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8216175700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82100aa8c0 0x7f82100aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8216175700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82100aa8c0 0x7f82100aad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42488/0 (socket says 172.21.15.7:42488) 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8216175700 1 -- 172.21.15.7:0/297388129 learned_addr learned my addr 172.21.15.7:0/297388129 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.347+0000 7f8215173700 1 --2- 172.21.15.7:0/297388129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f821015d920 0x7f821015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8216175700 1 -- 172.21.15.7:0/297388129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f821015d920 msgr2=0x7f821015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8216175700 1 --2- 172.21.15.7:0/297388129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f821015d920 0x7f821015dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8216175700 1 -- 172.21.15.7:0/297388129 >> v1:172.21.15.113:6789/0 conn(0x7f82100ac3a0 legacy=0x7f82100cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8216175700 1 -- 172.21.15.7:0/297388129 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82100ac770 con 0x7f82100aa8c0 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8216175700 1 --2- 172.21.15.7:0/297388129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82100aa8c0 0x7f82100aad40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8208009f20 tx=0x7f8208009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1091aa2f54630f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8214972700 1 -- 172.21.15.7:0/297388129 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820800e070 con 0x7f82100aa8c0 2024-01-11T07:08:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8214972700 1 -- 172.21.15.7:0/297388129 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820800ba80 con 0x7f82100aa8c0 2024-01-11T07:08:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8214972700 1 -- 172.21.15.7:0/297388129 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82080092e0 con 0x7f82100aa8c0 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8217bd8700 1 -- 172.21.15.7:0/297388129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82100aa8c0 msgr2=0x7f82100aad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.348+0000 7f8217bd8700 1 --2- 172.21.15.7:0/297388129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82100aa8c0 0x7f82100aad40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8208009f20 tx=0x7f8208009840 comp rx=0 tx=0).stop 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.349+0000 7f8217bd8700 1 -- 172.21.15.7:0/297388129 shutdown_connections 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.349+0000 7f8217bd8700 1 --2- 172.21.15.7:0/297388129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82100aa8c0 0x7f82100aad40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.349+0000 7f8217bd8700 1 --2- 172.21.15.7:0/297388129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f821015d920 0x7f821015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.349+0000 7f8217bd8700 1 -- 172.21.15.7:0/297388129 >> 172.21.15.7:0/297388129 conn(0x7f82100cd090 msgr2=0x7f82100cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.349+0000 7f8217bd8700 1 -- 172.21.15.7:0/297388129 shutdown_connections 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.349+0000 7f8217bd8700 1 -- 172.21.15.7:0/297388129 wait complete. 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.350+0000 7f8217bd8700 1 Processor -- start 2024-01-11T07:08:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.350+0000 7f8217bd8700 1 -- start start 2024-01-11T07:08:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8217bd8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82100aa8c0 0x7f82100c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8217bd8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82100ac3a0 0x7f82100c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8217bd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821015d920 0x7f82100c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8217bd8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82100c89e0 con 0x7f821015d920 2024-01-11T07:08:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8217bd8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821013d080 con 0x7f82100ac3a0 2024-01-11T07:08:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8217bd8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82100c8140 con 0x7f82100aa8c0 2024-01-11T07:08:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821015d920 0x7f82100c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821015d920 0x7f82100c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42492/0 (socket says 172.21.15.7:42492) 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 -- 172.21.15.7:0/3165326230 learned_addr learned my addr 172.21.15.7:0/3165326230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8215173700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82100ac3a0 0x7f82100c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8215974700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82100aa8c0 0x7f82100c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 -- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82100aa8c0 msgr2=0x7f82100c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82100aa8c0 0x7f82100c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 -- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82100ac3a0 msgr2=0x7f82100c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82100ac3a0 0x7f82100c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 -- 172.21.15.7:0/3165326230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8208009ae0 con 0x7f821015d920 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8216175700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821015d920 0x7f82100c6280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f820800b6c0 tx=0x7f8208018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.351+0000 7f8201ffb700 1 -- 172.21.15.7:0/3165326230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82080092e0 con 0x7f821015d920 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.352+0000 7f8201ffb700 1 -- 172.21.15.7:0/3165326230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208008d10 con 0x7f821015d920 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.352+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82100c8420 con 0x7f821015d920 2024-01-11T07:08:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.352+0000 7f8201ffb700 1 -- 172.21.15.7:0/3165326230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82080266b0 con 0x7f821015d920 2024-01-11T07:08:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.352+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82100c1fd0 con 0x7f821015d920 2024-01-11T07:08:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.353+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82100c1e30 con 0x7f821015d920 2024-01-11T07:08:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.353+0000 7f8201ffb700 1 -- 172.21.15.7:0/3165326230 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8208003040 con 0x7f821015d920 2024-01-11T07:08:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.354+0000 7f8201ffb700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f81f4066f30 0x7f81f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.354+0000 7f8201ffb700 1 -- 172.21.15.7:0/3165326230 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8208086200 con 0x7f821015d920 2024-01-11T07:08:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.354+0000 7f8215974700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f81f4066f30 0x7f81f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.354+0000 7f8215974700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f81f4066f30 0x7f81f40693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8204003a80 tx=0x7f8204007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.358+0000 7f8201ffb700 1 -- 172.21.15.7:0/3165326230 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82100c1e30 con 0x7f821015d920 2024-01-11T07:08:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.563+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82100c1e30 con 0x7f821015d920 2024-01-11T07:08:02.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.564+0000 7f8201ffb700 1 -- 172.21.15.7:0/3165326230 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f82100c1e30 con 0x7f821015d920 2024-01-11T07:08:02.564 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:02.564 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:02.185556+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:07:47.856440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-11T07:07:49.053507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f81f4066f30 msgr2=0x7f81f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f81f4066f30 0x7f81f40693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8204003a80 tx=0x7f8204007ec0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821015d920 msgr2=0x7f82100c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821015d920 0x7f82100c6280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f820800b6c0 tx=0x7f8208018000 comp rx=0 tx=0).stop 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 shutdown_connections 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f81f4066f30 0x7f81f40693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821015d920 0x7f82100c6280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82100aa8c0 0x7f82100c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 --2- 172.21.15.7:0/3165326230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82100ac3a0 0x7f82100c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 >> 172.21.15.7:0/3165326230 conn(0x7f82100cd090 msgr2=0x7f821015cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 shutdown_connections 2024-01-11T07:08:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:02.566+0000 7f8217bd8700 1 -- 172.21.15.7:0/3165326230 wait complete. 2024-01-11T07:08:02.580 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:08:02.580 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:08:02.580 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-11T07:08:02.580 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-11T07:08:02.580 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-11T07:08:02.580 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-11T07:08:02.580 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:08:02.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:02.844 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T07:08:02.926 INFO:teuthology.orchestra.run.smithi007.stdout:client5112 2024-01-11T07:08:02.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:02.927 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c0c5e15-bc38-4016-89e4-891c3586efbb.client5112/osdmap') 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 34 barrier 34 flags 0x588000 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-11T07:08:03.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.113:6801 100% (exists, up) 100% -1 2024-01-11T07:08:03.022 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.113:6817 100% (exists, up) 100% -1 2024-01-11T07:08:03.022 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.113:6809 100% (exists, up) 100% -1 2024-01-11T07:08:03.022 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.113:6825 100% (exists, up) 100% -1 2024-01-11T07:08:03.022 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-11T07:08:03.023 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-11T07:08:03.023 INFO:teuthology.orchestra.run.smithi007.stderr:791 bytes copied, 4.705e-05 s, 16.8 MB/s 2024-01-11T07:08:03.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:03.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.162+0000 7fe856c74700 1 Processor -- start 2024-01-11T07:08:03.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.162+0000 7fe856c74700 1 -- start start 2024-01-11T07:08:03.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe856c74700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe85006afc0 0x7fe85015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe856c74700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85015b770 con 0x7fe85006dd00 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe856c74700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85015abf0 con 0x7fe85006e0d0 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe856c74700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85015aed0 con 0x7fe85006afc0 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe854a10700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe85006dd00 0x7fe8500c98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51318/0 (socket says 172.21.15.7:51318) 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe854a10700 1 -- 172.21.15.7:0/412102008 learned_addr learned my addr 172.21.15.7:0/412102008 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe855211700 1 --2- 172.21.15.7:0/412102008 >> v2:172.21.15.113:3300/0 conn(0x7fe85006afc0 0x7fe85015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735943361 0 0) 0x7fe85015b770 con 0x7fe85006dd00 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.163+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe840003570 con 0x7fe85006dd00 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041950285 0 0) 0x7fe85015abf0 con 0x7fe85006e0d0 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85015b770 con 0x7fe85006e0d0 2024-01-11T07:08:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434828877 0 0) 0x7fe840003570 con 0x7fe85006dd00 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 >> v2:172.21.15.113:3300/0 conn(0x7fe85006afc0 msgr2=0x7fe85015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 --2- 172.21.15.7:0/412102008 >> v2:172.21.15.113:3300/0 conn(0x7fe85006afc0 0x7fe85015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 >> v1:172.21.15.113:6789/0 conn(0x7fe85006e0d0 legacy=0x7fe850159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85015b1b0 con 0x7fe85006dd00 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fe844001470 con 0x7fe85006dd00 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 >> v1:172.21.15.7:6789/0 conn(0x7fe85006dd00 legacy=0x7fe8500c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe840003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe840004810 0x7fe840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8400051c0 0x7fe84000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85015aed0 con 0x7fe840003bf0 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84000c160 con 0x7fe840004810 2024-01-11T07:08:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84000c320 con 0x7fe8400051c0 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe855211700 1 --2- 172.21.15.7:0/412102008 >> v2:172.21.15.113:3300/0 conn(0x7fe85006afc0 0x7fe85015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe854a10700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe840003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe84ffff700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe840004810 0x7fe840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe855211700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8400051c0 0x7fe84000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe854a10700 1 -- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8400051c0 msgr2=0x7fe84000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe854a10700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8400051c0 0x7fe84000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe854a10700 1 -- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe840004810 msgr2=0x7fe840004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe854a10700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe840004810 0x7fe840004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe854a10700 1 -- 172.21.15.7:0/412102008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe840003570 con 0x7fe840003bf0 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe855211700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8400051c0 0x7fe84000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.164+0000 7fe855211700 1 -- 172.21.15.7:0/412102008 reap_dead start 2024-01-11T07:08:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe854a10700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe840003fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe84400c2a0 tx=0x7fe844001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a33fea20e8f44d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400cb70 con 0x7fe840003bf0 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84400ccd0 con 0x7fe840003bf0 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe84f7fe700 1 -- 172.21.15.7:0/412102008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844005b70 con 0x7fe840003bf0 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe856c74700 1 -- 172.21.15.7:0/412102008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 msgr2=0x7fe840003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe856c74700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe840003fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe84400c2a0 tx=0x7fe844001e10 comp rx=0 tx=0).stop 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe856c74700 1 -- 172.21.15.7:0/412102008 shutdown_connections 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe856c74700 1 --2- 172.21.15.7:0/412102008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe840003fe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe856c74700 1 -- 172.21.15.7:0/412102008 >> 172.21.15.7:0/412102008 conn(0x7fe8500680e0 msgr2=0x7fe8500cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe856c74700 1 -- 172.21.15.7:0/412102008 shutdown_connections 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.165+0000 7fe856c74700 1 -- 172.21.15.7:0/412102008 wait complete. 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.167+0000 7fe856c74700 1 Processor -- start 2024-01-11T07:08:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.167+0000 7fe856c74700 1 -- start start 2024-01-11T07:08:03.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe856c74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe850147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:03.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe856c74700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8500af6d0 0x7fe850168090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:03.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe856c74700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8500b28d0 0x7fe8500b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe856c74700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500b3290 con 0x7fe840003bf0 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe856c74700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500c48e0 con 0x7fe8500b28d0 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe856c74700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500c4bc0 con 0x7fe8500af6d0 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe854a10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe850147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe854a10700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe850147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42502/0 (socket says 172.21.15.7:42502) 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.168+0000 7fe854a10700 1 -- 172.21.15.7:0/2047809600 learned_addr learned my addr 172.21.15.7:0/2047809600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe855211700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8500b28d0 0x7fe8500b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe84ffff700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8500af6d0 0x7fe850168090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe854a10700 1 -- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8500af6d0 msgr2=0x7fe850168090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe854a10700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8500af6d0 0x7fe850168090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe854a10700 1 -- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8500b28d0 msgr2=0x7fe8500b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe854a10700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8500b28d0 0x7fe8500b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe854a10700 1 -- 172.21.15.7:0/2047809600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe850168910 con 0x7fe840003bf0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe855211700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8500b28d0 0x7fe8500b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe84ffff700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8500af6d0 0x7fe850168090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe854a10700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe850147c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe84000c130 tx=0x7fe844015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe84cff9700 1 -- 172.21.15.7:0/2047809600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844004ac0 con 0x7fe840003bf0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe84cff9700 1 -- 172.21.15.7:0/2047809600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe844004c20 con 0x7fe840003bf0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe850168bf0 con 0x7fe840003bf0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe84cff9700 1 -- 172.21.15.7:0/2047809600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8440070d0 con 0x7fe840003bf0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.169+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8501481c0 con 0x7fe840003bf0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.170+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8500c9120 con 0x7fe840003bf0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.171+0000 7fe84cff9700 1 -- 172.21.15.7:0/2047809600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe844007440 con 0x7fe840003bf0 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.171+0000 7fe84cff9700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe834066f30 0x7fe8340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.171+0000 7fe84cff9700 1 -- 172.21.15.7:0/2047809600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe844085520 con 0x7fe840003bf0 2024-01-11T07:08:03.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.171+0000 7fe84ffff700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe834066f30 0x7fe8340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:03.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.172+0000 7fe84ffff700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe834066f30 0x7fe8340693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe83c000c00 tx=0x7fe83c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:03.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.175+0000 7fe84cff9700 1 -- 172.21.15.7:0/2047809600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe844052790 con 0x7fe840003bf0 2024-01-11T07:08:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.428+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe85004c3c0 con 0x7fe840003bf0 2024-01-11T07:08:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.428+0000 7fe84cff9700 1 -- 172.21.15.7:0/2047809600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fe844017090 con 0x7fe840003bf0 2024-01-11T07:08:03.428 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:03.428 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:07:50.072668+0000","modified":"2024-01-11T07:07:53.685303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/3458914988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3458914988},{"type":"v1","addr":"172.21.15.7:6834","nonce":3458914988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-11T07:08:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.430+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe834066f30 msgr2=0x7fe8340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.430+0000 7fe856c74700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe834066f30 0x7fe8340693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe83c000c00 tx=0x7fe83c009040 comp rx=0 tx=0).stop 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.430+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 msgr2=0x7fe850147c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe850147c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe84000c130 tx=0x7fe844015000 comp rx=0 tx=0).stop 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 shutdown_connections 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe834066f30 0x7fe8340693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8500b28d0 0x7fe8500b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840003bf0 0x7fe850147c80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 --2- 172.21.15.7:0/2047809600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe8500af6d0 0x7fe850168090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 >> 172.21.15.7:0/2047809600 conn(0x7fe8500680e0 msgr2=0x7fe8500cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 shutdown_connections 2024-01-11T07:08:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:03.431+0000 7fe856c74700 1 -- 172.21.15.7:0/2047809600 wait complete. 2024-01-11T07:08:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T07:08:03.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 36 2024-01-11T07:08:03.594 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-01-11T07:08:03.594 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:08:03.594 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:08:03.594 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-11T07:08:03.594 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-11T07:08:03.594 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-11T07:08:03.594 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-11T07:08:03.595 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:08:03.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:03.858 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T07:08:03.941 INFO:teuthology.orchestra.run.smithi007.stdout:client5112 2024-01-11T07:08:03.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:03.941 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c0c5e15-bc38-4016-89e4-891c3586efbb.client5112/osdmap') 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 36 barrier 36 flags 0x588000 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.113:6801 100% (exists, up) 100% -1 2024-01-11T07:08:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.113:6817 100% (exists, up) 100% -1 2024-01-11T07:08:04.036 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.113:6809 100% (exists, up) 100% -1 2024-01-11T07:08:04.036 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.113:6825 100% (exists, up) 100% -1 2024-01-11T07:08:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-11T07:08:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-11T07:08:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:791 bytes copied, 4.5246e-05 s, 17.5 MB/s 2024-01-11T07:08:04.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:04.037 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T07:08:04.077 INFO:teuthology.orchestra.run.smithi007.stdout:client5112 2024-01-11T07:08:04.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:04.078 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c0c5e15-bc38-4016-89e4-891c3586efbb.client5112/osdmap') 2024-01-11T07:08:04.170 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 36 barrier 36 flags 0x588000 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.113:6801 100% (exists, up) 100% -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.113:6817 100% (exists, up) 100% -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.113:6809 100% (exists, up) 100% -1 2024-01-11T07:08:04.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.113:6825 100% (exists, up) 100% -1 2024-01-11T07:08:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-11T07:08:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-11T07:08:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:791 bytes copied, 4.9928e-05 s, 15.8 MB/s 2024-01-11T07:08:04.173 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:08:04.173 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:08:04.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:08:04.220 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:08:04.265 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:08:04.265 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:08:04.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:04.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:08:04.350 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:08:04.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:04.351 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:08:04.372 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:08:04.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:04.372 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:08:04.420 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:08:04.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:04.420 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:08:04.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.577+0000 7f720dbd4700 1 Processor -- start 2024-01-11T07:08:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.577+0000 7f720dbd4700 1 -- start start 2024-01-11T07:08:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.578+0000 7f720dbd4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7208063230 0x7f7208063680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.578+0000 7f720dbd4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7208063bc0 con 0x7f72080aff20 2024-01-11T07:08:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.578+0000 7f720dbd4700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720815d910 con 0x7f72080c8020 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.578+0000 7f720dbd4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72080b02f0 con 0x7f7208063230 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.578+0000 7f72077fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72080aff20 0x7f72080aab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51322/0 (socket says 172.21.15.7:51322) 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.578+0000 7f72077fe700 1 -- 172.21.15.7:0/1958483643 learned_addr learned my addr 172.21.15.7:0/1958483643 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.578+0000 7f7206ffd700 1 --2- 172.21.15.7:0/1958483643 >> v2:172.21.15.113:3300/0 conn(0x7f7208063230 0x7f7208063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f72067fc700 1 -- 172.21.15.7:0/1958483643 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905120548 0 0) 0x7f7208063bc0 con 0x7f72080aff20 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f72067fc700 1 -- 172.21.15.7:0/1958483643 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f0003570 con 0x7f72080aff20 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f7206ffd700 1 -- 172.21.15.7:0/1958483643 >> v1:172.21.15.113:6789/0 conn(0x7f72080c8020 legacy=0x7f72080c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f7206ffd700 1 -- 172.21.15.7:0/1958483643 >> v1:172.21.15.7:6789/0 conn(0x7f72080aff20 legacy=0x7f72080aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f7206ffd700 1 -- 172.21.15.7:0/1958483643 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720815dbf0 con 0x7f7208063230 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f7206ffd700 1 --2- 172.21.15.7:0/1958483643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7208063230 0x7f7208063680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0098e0 tx=0x7f71fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced3020e430f26af server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f72067fc700 1 -- 172.21.15.7:0/1958483643 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00b640 con 0x7f7208063230 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f72067fc700 1 -- 172.21.15.7:0/1958483643 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc00b7a0 con 0x7f7208063230 2024-01-11T07:08:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.579+0000 7f72067fc700 1 -- 172.21.15.7:0/1958483643 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00bb20 con 0x7f7208063230 2024-01-11T07:08:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.580+0000 7f720dbd4700 1 -- 172.21.15.7:0/1958483643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7208063230 msgr2=0x7f7208063680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.580+0000 7f720dbd4700 1 --2- 172.21.15.7:0/1958483643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7208063230 0x7f7208063680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0098e0 tx=0x7f71fc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.580+0000 7f720dbd4700 1 -- 172.21.15.7:0/1958483643 shutdown_connections 2024-01-11T07:08:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.580+0000 7f720dbd4700 1 --2- 172.21.15.7:0/1958483643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7208063230 0x7f7208063680 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.580+0000 7f720dbd4700 1 -- 172.21.15.7:0/1958483643 >> 172.21.15.7:0/1958483643 conn(0x7f7208068090 msgr2=0x7f72080cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.580+0000 7f720dbd4700 1 -- 172.21.15.7:0/1958483643 shutdown_connections 2024-01-11T07:08:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.580+0000 7f720dbd4700 1 -- 172.21.15.7:0/1958483643 wait complete. 2024-01-11T07:08:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.581+0000 7f720dbd4700 1 Processor -- start 2024-01-11T07:08:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.581+0000 7f720dbd4700 1 -- start start 2024-01-11T07:08:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f720dbd4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7208063230 0x7f72080b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f720dbd4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72080aff20 0x7f7208152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f720dbd4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72080c8020 0x7f72081564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f720dbd4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080c56d0 con 0x7f7208063230 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f720dbd4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080b1550 con 0x7f72080aff20 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f720dbd4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72080b17d0 con 0x7f72080c8020 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f72077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7208063230 0x7f72080b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f72077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7208063230 0x7f72080b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42508/0 (socket says 172.21.15.7:42508) 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f72077fe700 1 -- 172.21.15.7:0/14006985 learned_addr learned my addr 172.21.15.7:0/14006985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f7206ffd700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72080aff20 0x7f7208152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.582+0000 7f7207fff700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72080c8020 0x7f72081564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f72077fe700 1 -- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72080c8020 msgr2=0x7f72081564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f72077fe700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72080c8020 0x7f72081564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f72077fe700 1 -- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72080aff20 msgr2=0x7f7208152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f72077fe700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72080aff20 0x7f7208152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f72077fe700 1 -- 172.21.15.7:0/14006985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009420 con 0x7f7208063230 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f72077fe700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7208063230 0x7f72080b1010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c590 tx=0x7f71f8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f71ef7fe700 1 -- 172.21.15.7:0/14006985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8005cd0 con 0x7f7208063230 2024-01-11T07:08:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f71ef7fe700 1 -- 172.21.15.7:0/14006985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8005e50 con 0x7f7208063230 2024-01-11T07:08:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f71ef7fe700 1 -- 172.21.15.7:0/14006985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8007320 con 0x7f7208063230 2024-01-11T07:08:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72080b1ab0 con 0x7f7208063230 2024-01-11T07:08:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.583+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72080b2070 con 0x7f7208063230 2024-01-11T07:08:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.584+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72080c54a0 con 0x7f7208063230 2024-01-11T07:08:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.584+0000 7f71ef7fe700 1 -- 172.21.15.7:0/14006985 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f71f801b3b0 con 0x7f7208063230 2024-01-11T07:08:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.585+0000 7f71ef7fe700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f71f4066f30 0x7f71f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.585+0000 7f71ef7fe700 1 -- 172.21.15.7:0/14006985 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f71f8083c20 con 0x7f7208063230 2024-01-11T07:08:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.585+0000 7f7206ffd700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f71f4066f30 0x7f71f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.586+0000 7f7206ffd700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f71f4066f30 0x7f71f40693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0093f0 tx=0x7f71fc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.589+0000 7f71ef7fe700 1 -- 172.21.15.7:0/14006985 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72080c54a0 con 0x7f7208063230 2024-01-11T07:08:04.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.841+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72080c54a0 con 0x7f7208063230 2024-01-11T07:08:04.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.841+0000 7f71ef7fe700 1 -- 172.21.15.7:0/14006985 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f72080c54a0 con 0x7f7208063230 2024-01-11T07:08:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:04.841 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:07:50.072668+0000","modified":"2024-01-11T07:07:53.685303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/3458914988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3458914988},{"type":"v1","addr":"172.21.15.7:6834","nonce":3458914988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-11T07:08:04.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f71f4066f30 msgr2=0x7f71f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f71f4066f30 0x7f71f40693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0093f0 tx=0x7f71fc017000 comp rx=0 tx=0).stop 2024-01-11T07:08:04.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7208063230 msgr2=0x7f72080b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7208063230 0x7f72080b1010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c590 tx=0x7f71f8007e00 comp rx=0 tx=0).stop 2024-01-11T07:08:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 shutdown_connections 2024-01-11T07:08:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f71f4066f30 0x7f71f40693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72080c8020 0x7f72081564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7208063230 0x7f72080b1010 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 --2- 172.21.15.7:0/14006985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72080aff20 0x7f7208152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 >> 172.21.15.7:0/14006985 conn(0x7f7208068090 msgr2=0x7f7208153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 shutdown_connections 2024-01-11T07:08:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.843+0000 7f720dbd4700 1 -- 172.21.15.7:0/14006985 wait complete. 2024-01-11T07:08:04.845 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T07:08:04.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.988+0000 7f79a44cf700 1 Processor -- start 2024-01-11T07:08:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.988+0000 7f79a44cf700 1 -- start start 2024-01-11T07:08:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.989+0000 7f79a44cf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c152400 0x7f799c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.989+0000 7f79a44cf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799c159130 con 0x7f799c158150 2024-01-11T07:08:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a44cf700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799c158890 con 0x7f799c152030 2024-01-11T07:08:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a44cf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c070a50 con 0x7f799c152400 2024-01-11T07:08:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a2a6c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f799c158150 0x7f799c15bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51336/0 (socket says 172.21.15.7:51336) 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a2a6c700 1 -- 172.21.15.7:0/2092246402 learned_addr learned my addr 172.21.15.7:0/2092246402 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1a6a700 1 --2- 172.21.15.7:0/2092246402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c152400 0x7f799c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1269700 1 -- 172.21.15.7:0/2092246402 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825603344 0 0) 0x7f7994001110 con 0x7f799c158150 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1269700 1 -- 172.21.15.7:0/2092246402 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7984003570 con 0x7f799c158150 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1269700 1 -- 172.21.15.7:0/2092246402 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371660133 0 0) 0x7f799c158890 con 0x7f799c152030 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1269700 1 -- 172.21.15.7:0/2092246402 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7994001110 con 0x7f799c152030 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1a6a700 1 -- 172.21.15.7:0/2092246402 >> v1:172.21.15.113:6789/0 conn(0x7f799c152030 legacy=0x7f799c069610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1a6a700 1 -- 172.21.15.7:0/2092246402 >> v1:172.21.15.7:6789/0 conn(0x7f799c158150 legacy=0x7f799c15bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1269700 1 -- 172.21.15.7:0/2092246402 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070983724 0 0) 0x7f7984003570 con 0x7f799c158150 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.990+0000 7f79a1a6a700 1 -- 172.21.15.7:0/2092246402 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c158b70 con 0x7f799c152400 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a1a6a700 1 --2- 172.21.15.7:0/2092246402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c152400 0x7f799c06d400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7990009910 tx=0x7f79900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8231885efb516ed server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a1269700 1 -- 172.21.15.7:0/2092246402 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994001600 con 0x7f799c152400 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a1269700 1 -- 172.21.15.7:0/2092246402 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7990011420 con 0x7f799c152400 2024-01-11T07:08:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a1269700 1 -- 172.21.15.7:0/2092246402 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79900117c0 con 0x7f799c152400 2024-01-11T07:08:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a44cf700 1 -- 172.21.15.7:0/2092246402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c152400 msgr2=0x7f799c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a44cf700 1 --2- 172.21.15.7:0/2092246402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c152400 0x7f799c06d400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7990009910 tx=0x7f79900091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a44cf700 1 -- 172.21.15.7:0/2092246402 shutdown_connections 2024-01-11T07:08:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a44cf700 1 --2- 172.21.15.7:0/2092246402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c152400 0x7f799c06d400 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a44cf700 1 -- 172.21.15.7:0/2092246402 >> 172.21.15.7:0/2092246402 conn(0x7f799c0cd090 msgr2=0x7f799c0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a44cf700 1 -- 172.21.15.7:0/2092246402 shutdown_connections 2024-01-11T07:08:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.991+0000 7f79a44cf700 1 -- 172.21.15.7:0/2092246402 wait complete. 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.993+0000 7f79a44cf700 1 Processor -- start 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.993+0000 7f79a44cf700 1 -- start start 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.993+0000 7f79a44cf700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c152030 0x7f799c14dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.993+0000 7f79a44cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f799c152400 0x7f799c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.993+0000 7f79a44cf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c158150 0x7f799c159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.993+0000 7f79a44cf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c150e90 con 0x7f799c152400 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.993+0000 7f79a44cf700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c0c4bf0 con 0x7f799c152030 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.993+0000 7f79a44cf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c168d90 con 0x7f799c158150 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f799c152400 0x7f799c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f799c152400 0x7f799c1592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42520/0 (socket says 172.21.15.7:42520) 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 -- 172.21.15.7:0/2921714177 learned_addr learned my addr 172.21.15.7:0/2921714177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a226b700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c152030 0x7f799c14dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a2a6c700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c158150 0x7f799c159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 -- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c158150 msgr2=0x7f799c159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c158150 0x7f799c159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 -- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c152030 msgr2=0x7f799c14dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c152030 0x7f799c14dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 -- 172.21.15.7:0/2921714177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7990009420 con 0x7f799c152400 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a226b700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c152030 0x7f799c14dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a1a6a700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f799c152400 0x7f799c1592d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f79900091e0 tx=0x7f7990016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f799a7fc700 1 -- 172.21.15.7:0/2921714177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799001cc90 con 0x7f799c152400 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f799a7fc700 1 -- 172.21.15.7:0/2921714177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799001cdf0 con 0x7f799c152400 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c0c42c0 con 0x7f799c152400 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f799a7fc700 1 -- 172.21.15.7:0/2921714177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799001a6a0 con 0x7f799c152400 2024-01-11T07:08:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.994+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c0c4880 con 0x7f799c152400 2024-01-11T07:08:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.995+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c0c4d80 con 0x7f799c152400 2024-01-11T07:08:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.996+0000 7f799a7fc700 1 -- 172.21.15.7:0/2921714177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f799001aa90 con 0x7f799c152400 2024-01-11T07:08:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.996+0000 7f799a7fc700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7988066f30 0x7f79880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.996+0000 7f799a7fc700 1 -- 172.21.15.7:0/2921714177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f79900864f0 con 0x7f799c152400 2024-01-11T07:08:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.997+0000 7f79a226b700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7988066f30 0x7f79880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:04.997+0000 7f79a226b700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7988066f30 0x7f79880693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f798c004040 tx=0x7f798c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:05.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.001+0000 7f799a7fc700 1 -- 172.21.15.7:0/2921714177 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79900536c0 con 0x7f799c152400 2024-01-11T07:08:05.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.207+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f799c04c3c0 con 0x7f799c152400 2024-01-11T07:08:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.208+0000 7f799a7fc700 1 -- 172.21.15.7:0/2921714177 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f7990056ce0 con 0x7f799c152400 2024-01-11T07:08:05.207 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:02.185556+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:07:47.856440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-11T07:07:49.053507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7988066f30 msgr2=0x7f79880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7988066f30 0x7f79880693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f798c004040 tx=0x7f798c009040 comp rx=0 tx=0).stop 2024-01-11T07:08:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f799c152400 msgr2=0x7f799c1592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f799c152400 0x7f799c1592d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f79900091e0 tx=0x7f7990016000 comp rx=0 tx=0).stop 2024-01-11T07:08:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 shutdown_connections 2024-01-11T07:08:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7988066f30 0x7f79880693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f799c158150 0x7f799c159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c152030 0x7f799c14dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 --2- 172.21.15.7:0/2921714177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f799c152400 0x7f799c1592d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 >> 172.21.15.7:0/2921714177 conn(0x7f799c0cd090 msgr2=0x7f799c0c2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.210+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 shutdown_connections 2024-01-11T07:08:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.211+0000 7f79a44cf700 1 -- 172.21.15.7:0/2921714177 wait complete. 2024-01-11T07:08:05.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:07:47.856440+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:07:49.053507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:08:05.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:08:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.361+0000 7fab6fda8700 1 Processor -- start 2024-01-11T07:08:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.361+0000 7fab6fda8700 1 -- start start 2024-01-11T07:08:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6fda8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6fda8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab6806e0d0 0x7fab68159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6fda8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab68159ff0 0x7fab680c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6fda8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6815b770 con 0x7fab6806e0d0 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6fda8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6815abf0 con 0x7fab68159ff0 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6fda8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6815aed0 con 0x7fab6806dd00 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6d343700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab6806e0d0 0x7fab68159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6d343700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab6806e0d0 0x7fab68159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42522/0 (socket says 172.21.15.7:42522) 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6d343700 1 -- 172.21.15.7:0/25583335 learned_addr learned my addr 172.21.15.7:0/25583335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6e345700 1 --2- 172.21.15.7:0/25583335 >> v2:172.21.15.113:3300/0 conn(0x7fab68159ff0 0x7fab680c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.362+0000 7fab6db44700 1 --2- 172.21.15.7:0/25583335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6d343700 1 -- 172.21.15.7:0/25583335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 msgr2=0x7fab680c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6d343700 1 --2- 172.21.15.7:0/25583335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6d343700 1 -- 172.21.15.7:0/25583335 >> v2:172.21.15.113:3300/0 conn(0x7fab68159ff0 msgr2=0x7fab680c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6d343700 1 --2- 172.21.15.7:0/25583335 >> v2:172.21.15.113:3300/0 conn(0x7fab68159ff0 0x7fab680c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6d343700 1 -- 172.21.15.7:0/25583335 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6815b1b0 con 0x7fab6806e0d0 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6db44700 1 --2- 172.21.15.7:0/25583335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6d343700 1 --2- 172.21.15.7:0/25583335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab68159ab0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fab6806c5b0 tx=0x7fab5800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb21e256cea173d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6cb42700 1 -- 172.21.15.7:0/25583335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5800dba0 con 0x7fab6806e0d0 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6cb42700 1 -- 172.21.15.7:0/25583335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5800dd00 con 0x7fab6806e0d0 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.363+0000 7fab6cb42700 1 -- 172.21.15.7:0/25583335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab580196d0 con 0x7fab6806e0d0 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 -- 172.21.15.7:0/25583335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 msgr2=0x7fab68159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 --2- 172.21.15.7:0/25583335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab68159ab0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fab6806c5b0 tx=0x7fab5800b4f0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 -- 172.21.15.7:0/25583335 shutdown_connections 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 --2- 172.21.15.7:0/25583335 >> v2:172.21.15.113:3300/0 conn(0x7fab68159ff0 0x7fab680c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 --2- 172.21.15.7:0/25583335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 --2- 172.21.15.7:0/25583335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab68159ab0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 -- 172.21.15.7:0/25583335 >> 172.21.15.7:0/25583335 conn(0x7fab680680e0 msgr2=0x7fab680cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 -- 172.21.15.7:0/25583335 shutdown_connections 2024-01-11T07:08:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.364+0000 7fab6fda8700 1 -- 172.21.15.7:0/25583335 wait complete. 2024-01-11T07:08:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.365+0000 7fab6fda8700 1 Processor -- start 2024-01-11T07:08:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.365+0000 7fab6fda8700 1 -- start start 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6fda8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6fda8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab681651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6fda8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab68159ff0 0x7fab680b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6fda8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681656f0 con 0x7fab6806e0d0 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6fda8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680b34f0 con 0x7fab68159ff0 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6fda8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab680b2610 con 0x7fab6806dd00 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6d343700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab681651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6db44700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6d343700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab681651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42538/0 (socket says 172.21.15.7:42538) 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6e345700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab68159ff0 0x7fab680b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6d343700 1 -- 172.21.15.7:0/660290805 learned_addr learned my addr 172.21.15.7:0/660290805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6d343700 1 -- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 msgr2=0x7fab680c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6d343700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6d343700 1 -- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab68159ff0 msgr2=0x7fab680b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6d343700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab68159ff0 0x7fab680b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6d343700 1 -- 172.21.15.7:0/660290805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5800b790 con 0x7fab6806e0d0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.366+0000 7fab6e345700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab68159ff0 0x7fab680b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.367+0000 7fab6d343700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab681651b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab58006fd0 tx=0x7fab5801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.367+0000 7fab5dffb700 1 -- 172.21.15.7:0/660290805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58010680 con 0x7fab6806e0d0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.367+0000 7fab5dffb700 1 -- 172.21.15.7:0/660290805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab58010800 con 0x7fab6806e0d0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.367+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab680b28e0 con 0x7fab6806e0d0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.367+0000 7fab5dffb700 1 -- 172.21.15.7:0/660290805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58010bc0 con 0x7fab6806e0d0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.367+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab680b2ea0 con 0x7fab6806e0d0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.368+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab68147530 con 0x7fab6806e0d0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.369+0000 7fab5dffb700 1 -- 172.21.15.7:0/660290805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fab58023070 con 0x7fab6806e0d0 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.369+0000 7fab5dffb700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab4c066f30 0x7fab4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.369+0000 7fab5dffb700 1 -- 172.21.15.7:0/660290805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fab58014070 con 0x7fab6806e0d0 2024-01-11T07:08:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.370+0000 7fab6db44700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab4c066f30 0x7fab4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.370+0000 7fab6db44700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab4c066f30 0x7fab4c0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab64006f00 tx=0x7fab64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.374+0000 7fab5dffb700 1 -- 172.21.15.7:0/660290805 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab58055b00 con 0x7fab6806e0d0 2024-01-11T07:08:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.576+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab6815b490 con 0x7fab6806e0d0 2024-01-11T07:08:05.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.577+0000 7fab5dffb700 1 -- 172.21.15.7:0/660290805 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab58059120 con 0x7fab6806e0d0 2024-01-11T07:08:05.577 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:08:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab4c066f30 msgr2=0x7fab4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab4c066f30 0x7fab4c0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab64006f00 tx=0x7fab64008040 comp rx=0 tx=0).stop 2024-01-11T07:08:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 msgr2=0x7fab681651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab681651b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab58006fd0 tx=0x7fab5801b000 comp rx=0 tx=0).stop 2024-01-11T07:08:05.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 shutdown_connections 2024-01-11T07:08:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab4c066f30 0x7fab4c0693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab68159ff0 0x7fab680b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6806dd00 0x7fab680c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 --2- 172.21.15.7:0/660290805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab6806e0d0 0x7fab681651b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 >> 172.21.15.7:0/660290805 conn(0x7fab680680e0 msgr2=0x7fab680c9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 shutdown_connections 2024-01-11T07:08:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.579+0000 7fab6fda8700 1 -- 172.21.15.7:0/660290805 wait complete. 2024-01-11T07:08:05.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:08:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.728+0000 7feb5b556700 1 Processor -- start 2024-01-11T07:08:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.728+0000 7feb5b556700 1 -- start start 2024-01-11T07:08:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb5b556700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb54157a90 0x7feb54157e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb5b556700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb540cf1b0 0x7feb540c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb5b556700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb540cf5c0 0x7feb540c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb5b556700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb540ca3d0 con 0x7feb540cf1b0 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb5b556700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb540b1790 con 0x7feb540cf5c0 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb5b556700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb540cd9a0 con 0x7feb54157a90 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb58af1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb540cf1b0 0x7feb540c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb58af1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb540cf1b0 0x7feb540c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42540/0 (socket says 172.21.15.7:42540) 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb58af1700 1 -- 172.21.15.7:0/3782576621 learned_addr learned my addr 172.21.15.7:0/3782576621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb592f2700 1 --2- 172.21.15.7:0/3782576621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb54157a90 0x7feb54157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb59af3700 1 --2- 172.21.15.7:0/3782576621 >> v2:172.21.15.113:3300/0 conn(0x7feb540cf5c0 0x7feb540c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb58af1700 1 -- 172.21.15.7:0/3782576621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb54157a90 msgr2=0x7feb54157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb58af1700 1 --2- 172.21.15.7:0/3782576621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb54157a90 0x7feb54157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb58af1700 1 -- 172.21.15.7:0/3782576621 >> v2:172.21.15.113:3300/0 conn(0x7feb540cf5c0 msgr2=0x7feb540c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb58af1700 1 --2- 172.21.15.7:0/3782576621 >> v2:172.21.15.113:3300/0 conn(0x7feb540cf5c0 0x7feb540c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.729+0000 7feb58af1700 1 -- 172.21.15.7:0/3782576621 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb540cdc80 con 0x7feb540cf1b0 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb58af1700 1 --2- 172.21.15.7:0/3782576621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb540cf1b0 0x7feb540c8f60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feb440099b0 tx=0x7feb44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865434376b11d3e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb4bfff700 1 -- 172.21.15.7:0/3782576621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400b820 con 0x7feb540cf1b0 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb4bfff700 1 -- 172.21.15.7:0/3782576621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4400b980 con 0x7feb540cf1b0 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb4bfff700 1 -- 172.21.15.7:0/3782576621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400bd20 con 0x7feb540cf1b0 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb5b556700 1 -- 172.21.15.7:0/3782576621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb540cf1b0 msgr2=0x7feb540c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb5b556700 1 --2- 172.21.15.7:0/3782576621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb540cf1b0 0x7feb540c8f60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feb440099b0 tx=0x7feb44009250 comp rx=0 tx=0).stop 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb5b556700 1 -- 172.21.15.7:0/3782576621 shutdown_connections 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb5b556700 1 --2- 172.21.15.7:0/3782576621 >> v2:172.21.15.113:3300/0 conn(0x7feb540cf5c0 0x7feb540c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb5b556700 1 --2- 172.21.15.7:0/3782576621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb54157a90 0x7feb54157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb5b556700 1 --2- 172.21.15.7:0/3782576621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb540cf1b0 0x7feb540c8f60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.730+0000 7feb5b556700 1 -- 172.21.15.7:0/3782576621 >> 172.21.15.7:0/3782576621 conn(0x7feb54097a50 msgr2=0x7feb540ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.731+0000 7feb5b556700 1 -- 172.21.15.7:0/3782576621 shutdown_connections 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.731+0000 7feb5b556700 1 -- 172.21.15.7:0/3782576621 wait complete. 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.732+0000 7feb5b556700 1 Processor -- start 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.732+0000 7feb5b556700 1 -- start start 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb5b556700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb540cf1b0 0x7feb5414ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb5b556700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb540cf5c0 0x7feb5415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb5b556700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54157a90 0x7feb5415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb5b556700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5415cf80 con 0x7feb54157a90 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb5b556700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540c2780 con 0x7feb540cf1b0 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb5b556700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb54155380 con 0x7feb540cf5c0 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54157a90 0x7feb5415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54157a90 0x7feb5415c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42544/0 (socket says 172.21.15.7:42544) 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 -- 172.21.15.7:0/3304889047 learned_addr learned my addr 172.21.15.7:0/3304889047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb592f2700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb540cf1b0 0x7feb5414ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb58af1700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb540cf5c0 0x7feb5415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 -- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb540cf5c0 msgr2=0x7feb5415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb540cf5c0 0x7feb5415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 -- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb540cf1b0 msgr2=0x7feb5414ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb540cf1b0 0x7feb5414ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 -- 172.21.15.7:0/3304889047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440094f0 con 0x7feb54157a90 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.733+0000 7feb59af3700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54157a90 0x7feb5415c360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00e0e0 tx=0x7feb4c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.734+0000 7feb497fa700 1 -- 172.21.15.7:0/3304889047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00ed80 con 0x7feb54157a90 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.734+0000 7feb497fa700 1 -- 172.21.15.7:0/3304889047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c014420 con 0x7feb54157a90 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.734+0000 7feb497fa700 1 -- 172.21.15.7:0/3304889047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00eee0 con 0x7feb54157a90 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.734+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5415c9c0 con 0x7feb54157a90 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.734+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb540c2410 con 0x7feb54157a90 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.735+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb540c2270 con 0x7feb54157a90 2024-01-11T07:08:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.735+0000 7feb497fa700 1 -- 172.21.15.7:0/3304889047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feb4c0149f0 con 0x7feb54157a90 2024-01-11T07:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.736+0000 7feb497fa700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.736+0000 7feb497fa700 1 -- 172.21.15.7:0/3304889047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feb4c084d00 con 0x7feb54157a90 2024-01-11T07:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.736+0000 7feb592f2700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.737+0000 7feb592f2700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feb38066f30 0x7feb380693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feb50006520 tx=0x7feb50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.740+0000 7feb497fa700 1 -- 172.21.15.7:0/3304889047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb540c2270 con 0x7feb54157a90 2024-01-11T07:08:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.944+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb540c2270 con 0x7feb54157a90 2024-01-11T07:08:05.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.944+0000 7feb497fa700 1 -- 172.21.15.7:0/3304889047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb540c2270 con 0x7feb54157a90 2024-01-11T07:08:05.944 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:08:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feb38066f30 msgr2=0x7feb380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feb38066f30 0x7feb380693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feb50006520 tx=0x7feb50008040 comp rx=0 tx=0).stop 2024-01-11T07:08:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54157a90 msgr2=0x7feb5415c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54157a90 0x7feb5415c360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00e0e0 tx=0x7feb4c009900 comp rx=0 tx=0).stop 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 shutdown_connections 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feb38066f30 0x7feb380693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb54157a90 0x7feb5415c360 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb540cf1b0 0x7feb5414ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 --2- 172.21.15.7:0/3304889047 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb540cf5c0 0x7feb5415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.946+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 >> 172.21.15.7:0/3304889047 conn(0x7feb54097a50 msgr2=0x7feb5406c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.947+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 shutdown_connections 2024-01-11T07:08:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:05.947+0000 7feb5b556700 1 -- 172.21.15.7:0/3304889047 wait complete. 2024-01-11T07:08:05.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:08:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.094+0000 7fd068da3700 1 Processor -- start 2024-01-11T07:08:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.094+0000 7fd068da3700 1 -- start start 2024-01-11T07:08:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd068da3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 0x7fd0640b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd068da3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd0640b0160 0x7fd0640c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd068da3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0640c91c0 0x7fd06406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd068da3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064069c70 con 0x7fd0640c91c0 2024-01-11T07:08:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd068da3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064069f50 con 0x7fd0640b0160 2024-01-11T07:08:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd068da3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06406a230 con 0x7fd0640afd90 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd062d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0640c91c0 0x7fd06406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd062d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0640c91c0 0x7fd06406dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42552/0 (socket says 172.21.15.7:42552) 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd062d9d700 1 -- 172.21.15.7:0/4061313472 learned_addr learned my addr 172.21.15.7:0/4061313472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd061d9b700 1 --2- 172.21.15.7:0/4061313472 >> v2:172.21.15.113:3300/0 conn(0x7fd0640b0160 0x7fd0640c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.095+0000 7fd06259c700 1 --2- 172.21.15.7:0/4061313472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 0x7fd0640b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd062d9d700 1 -- 172.21.15.7:0/4061313472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 msgr2=0x7fd0640b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd062d9d700 1 --2- 172.21.15.7:0/4061313472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 0x7fd0640b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd062d9d700 1 -- 172.21.15.7:0/4061313472 >> v2:172.21.15.113:3300/0 conn(0x7fd0640b0160 msgr2=0x7fd0640c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd062d9d700 1 --2- 172.21.15.7:0/4061313472 >> v2:172.21.15.113:3300/0 conn(0x7fd0640b0160 0x7fd0640c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd062d9d700 1 -- 172.21.15.7:0/4061313472 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06406a510 con 0x7fd0640c91c0 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd061d9b700 1 --2- 172.21.15.7:0/4061313472 >> v2:172.21.15.113:3300/0 conn(0x7fd0640b0160 0x7fd0640c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd062d9d700 1 --2- 172.21.15.7:0/4061313472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd06406dbe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd06406d5f0 tx=0x7fd05c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c9f9ede74846a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd06159a700 1 -- 172.21.15.7:0/4061313472 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00e070 con 0x7fd0640c91c0 2024-01-11T07:08:06.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd06159a700 1 -- 172.21.15.7:0/4061313472 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c00bae0 con 0x7fd0640c91c0 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd06159a700 1 -- 172.21.15.7:0/4061313472 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0092e0 con 0x7fd0640c91c0 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd068da3700 1 -- 172.21.15.7:0/4061313472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 msgr2=0x7fd06406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.096+0000 7fd068da3700 1 --2- 172.21.15.7:0/4061313472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd06406dbe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd06406d5f0 tx=0x7fd05c009840 comp rx=0 tx=0).stop 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.097+0000 7fd068da3700 1 -- 172.21.15.7:0/4061313472 shutdown_connections 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.097+0000 7fd068da3700 1 --2- 172.21.15.7:0/4061313472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd06406dbe0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.097+0000 7fd068da3700 1 --2- 172.21.15.7:0/4061313472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 0x7fd0640b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.097+0000 7fd068da3700 1 --2- 172.21.15.7:0/4061313472 >> v2:172.21.15.113:3300/0 conn(0x7fd0640b0160 0x7fd0640c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.097+0000 7fd068da3700 1 -- 172.21.15.7:0/4061313472 >> 172.21.15.7:0/4061313472 conn(0x7fd0640680e0 msgr2=0x7fd0640cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.097+0000 7fd068da3700 1 -- 172.21.15.7:0/4061313472 shutdown_connections 2024-01-11T07:08:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.097+0000 7fd068da3700 1 -- 172.21.15.7:0/4061313472 wait complete. 2024-01-11T07:08:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.098+0000 7fd068da3700 1 Processor -- start 2024-01-11T07:08:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.098+0000 7fd068da3700 1 -- start start 2024-01-11T07:08:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd068da3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 0x7fd064169470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd068da3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0640b0160 0x7fd06414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd068da3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd0640adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd068da3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640a64c0 con 0x7fd0640c91c0 2024-01-11T07:08:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd068da3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640ae9c0 con 0x7fd0640b0160 2024-01-11T07:08:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd068da3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640a6710 con 0x7fd0640afd90 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd0640adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd06259c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 0x7fd064169470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd0640adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42566/0 (socket says 172.21.15.7:42566) 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 -- 172.21.15.7:0/2897080905 learned_addr learned my addr 172.21.15.7:0/2897080905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd061d9b700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0640b0160 0x7fd06414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 -- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 msgr2=0x7fd064169470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 0x7fd064169470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 -- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0640b0160 msgr2=0x7fd06414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0640b0160 0x7fd06414dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 -- 172.21.15.7:0/2897080905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c009ae0 con 0x7fd0640c91c0 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.099+0000 7fd062d9d700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd0640adeb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd0640696d0 tx=0x7fd05c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.100+0000 7fd0527fc700 1 -- 172.21.15.7:0/2897080905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0032f0 con 0x7fd0640c91c0 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.100+0000 7fd0527fc700 1 -- 172.21.15.7:0/2897080905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c008ce0 con 0x7fd0640c91c0 2024-01-11T07:08:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.100+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0640a69f0 con 0x7fd0640c91c0 2024-01-11T07:08:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.100+0000 7fd0527fc700 1 -- 172.21.15.7:0/2897080905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c01d670 con 0x7fd0640c91c0 2024-01-11T07:08:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.100+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06414e220 con 0x7fd0640c91c0 2024-01-11T07:08:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.101+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd064064a40 con 0x7fd0640c91c0 2024-01-11T07:08:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.101+0000 7fd0527fc700 1 -- 172.21.15.7:0/2897080905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd05c003040 con 0x7fd0640c91c0 2024-01-11T07:08:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.102+0000 7fd0527fc700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd044066f30 0x7fd0440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.102+0000 7fd0527fc700 1 -- 172.21.15.7:0/2897080905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd05c086040 con 0x7fd0640c91c0 2024-01-11T07:08:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.102+0000 7fd06259c700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd044066f30 0x7fd0440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.102+0000 7fd06259c700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd044066f30 0x7fd0440693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd054006fd0 tx=0x7fd054008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.106+0000 7fd0527fc700 1 -- 172.21.15.7:0/2897080905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd05c0532b0 con 0x7fd0640c91c0 2024-01-11T07:08:06.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.310+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0640a6f10 con 0x7fd0640c91c0 2024-01-11T07:08:06.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.310+0000 7fd0527fc700 1 -- 172.21.15.7:0/2897080905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0640a6f10 con 0x7fd0640c91c0 2024-01-11T07:08:06.310 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd044066f30 msgr2=0x7fd0440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd044066f30 0x7fd0440693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd054006fd0 tx=0x7fd054008040 comp rx=0 tx=0).stop 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 msgr2=0x7fd0640adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd0640adeb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd0640696d0 tx=0x7fd05c018000 comp rx=0 tx=0).stop 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 shutdown_connections 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd044066f30 0x7fd0440693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0640c91c0 0x7fd0640adeb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd0640afd90 0x7fd064169470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 --2- 172.21.15.7:0/2897080905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0640b0160 0x7fd06414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 >> 172.21.15.7:0/2897080905 conn(0x7fd0640680e0 msgr2=0x7fd0641599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 shutdown_connections 2024-01-11T07:08:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.313+0000 7fd068da3700 1 -- 172.21.15.7:0/2897080905 wait complete. 2024-01-11T07:08:06.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.459+0000 7f184ec62700 1 Processor -- start 2024-01-11T07:08:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.459+0000 7f184ec62700 1 -- start start 2024-01-11T07:08:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184ec62700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f1848063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184ec62700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18480ac5b0 0x7f18480aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184ec62700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18480ab100 con 0x7f184815a280 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184ec62700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184815d880 con 0x7f18480ac5b0 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184ec62700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184806c5b0 con 0x7f1848063230 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f1847fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f184815a280 0x7f18480aa9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51340/0 (socket says 172.21.15.7:51340) 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f1847fff700 1 -- 172.21.15.7:0/898771599 learned_addr learned my addr 172.21.15.7:0/898771599 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184d1ff700 1 --2- 172.21.15.7:0/898771599 >> v2:172.21.15.113:3300/0 conn(0x7f18480ac5b0 0x7f18480aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184c9fe700 1 --2- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f1848063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f18477fe700 1 -- 172.21.15.7:0/898771599 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078294862 0 0) 0x7f18480ab100 con 0x7f184815a280 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f18477fe700 1 -- 172.21.15.7:0/898771599 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1830003570 con 0x7f184815a280 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184d1ff700 1 -- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 msgr2=0x7f1848063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f18477fe700 1 -- 172.21.15.7:0/898771599 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1423128511 0 0) 0x7f1830003570 con 0x7f184815a280 2024-01-11T07:08:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184d1ff700 1 --2- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f1848063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184d1ff700 1 -- 172.21.15.7:0/898771599 >> v1:172.21.15.7:6789/0 conn(0x7f184815a280 legacy=0x7f18480aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184d1ff700 1 -- 172.21.15.7:0/898771599 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848063b40 con 0x7f18480ac5b0 2024-01-11T07:08:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.460+0000 7f184c9fe700 1 --2- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f1848063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.461+0000 7f184d1ff700 1 --2- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 0x7f18480aca30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1840009e60 tx=0x7f1840009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5631a201d0413997 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.461+0000 7f18477fe700 1 -- 172.21.15.7:0/898771599 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184000e070 con 0x7f18480ac5b0 2024-01-11T07:08:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.461+0000 7f18477fe700 1 -- 172.21.15.7:0/898771599 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184000b7a0 con 0x7f18480ac5b0 2024-01-11T07:08:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.461+0000 7f18477fe700 1 -- 172.21.15.7:0/898771599 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c001470 con 0x7f18480ac5b0 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.461+0000 7f184ec62700 1 -- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 msgr2=0x7f18480aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.461+0000 7f184ec62700 1 --2- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 0x7f18480aca30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1840009e60 tx=0x7f1840009780 comp rx=0 tx=0).stop 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.462+0000 7f184ec62700 1 -- 172.21.15.7:0/898771599 shutdown_connections 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.462+0000 7f184ec62700 1 --2- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 0x7f18480aca30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.462+0000 7f184ec62700 1 --2- 172.21.15.7:0/898771599 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f1848063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.462+0000 7f184ec62700 1 -- 172.21.15.7:0/898771599 >> 172.21.15.7:0/898771599 conn(0x7f18480680e0 msgr2=0x7f18480cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.462+0000 7f184ec62700 1 -- 172.21.15.7:0/898771599 shutdown_connections 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.462+0000 7f184ec62700 1 -- 172.21.15.7:0/898771599 wait complete. 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.463+0000 7f184ec62700 1 Processor -- start 2024-01-11T07:08:06.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.463+0000 7f184ec62700 1 -- start start 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184ec62700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f184816a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184ec62700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 0x7f18480acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184ec62700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184815a280 0x7f18480c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184ec62700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184816ab80 con 0x7f184815a280 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184ec62700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18480ad0d0 con 0x7f18480ac5b0 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184ec62700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18480ad300 con 0x7f1848063230 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184d1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184815a280 0x7f18480c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184d1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184815a280 0x7f18480c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42582/0 (socket says 172.21.15.7:42582) 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184d1ff700 1 -- 172.21.15.7:0/2332354038 learned_addr learned my addr 172.21.15.7:0/2332354038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f1847fff700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 0x7f18480acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184c9fe700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f184816a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184d1ff700 1 -- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 msgr2=0x7f184816a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184d1ff700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f184816a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184d1ff700 1 -- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 msgr2=0x7f18480acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184d1ff700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 0x7f18480acb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f184d1ff700 1 -- 172.21.15.7:0/2332354038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1838009420 con 0x7f184815a280 2024-01-11T07:08:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.464+0000 7f1847fff700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 0x7f18480acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.465+0000 7f184d1ff700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184815a280 0x7f18480c3970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f184000b610 tx=0x7f1840018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.465+0000 7f1844ff9700 1 -- 172.21.15.7:0/2332354038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840012d50 con 0x7f184815a280 2024-01-11T07:08:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.465+0000 7f1844ff9700 1 -- 172.21.15.7:0/2332354038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184001e430 con 0x7f184815a280 2024-01-11T07:08:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.465+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1840009a20 con 0x7f184815a280 2024-01-11T07:08:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.465+0000 7f1844ff9700 1 -- 172.21.15.7:0/2332354038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184001e7f0 con 0x7f184815a280 2024-01-11T07:08:06.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.465+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18480ad8c0 con 0x7f184815a280 2024-01-11T07:08:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.466+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184816ad90 con 0x7f184815a280 2024-01-11T07:08:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.466+0000 7f1844ff9700 1 -- 172.21.15.7:0/2332354038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f184001eba0 con 0x7f184815a280 2024-01-11T07:08:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.467+0000 7f1844ff9700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1834066f30 0x7f18340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.467+0000 7f1844ff9700 1 -- 172.21.15.7:0/2332354038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1840086790 con 0x7f184815a280 2024-01-11T07:08:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.467+0000 7f184c9fe700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1834066f30 0x7f18340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.467+0000 7f184c9fe700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1834066f30 0x7f18340693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f18380093f0 tx=0x7f1838008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.471+0000 7f1844ff9700 1 -- 172.21.15.7:0/2332354038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1840053a00 con 0x7f184815a280 2024-01-11T07:08:06.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.729+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f184804c3c0 con 0x7f184815a280 2024-01-11T07:08:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.729+0000 7f1844ff9700 1 -- 172.21.15.7:0/2332354038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f1840057020 con 0x7f184815a280 2024-01-11T07:08:06.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:07:50.072668+0000","modified":"2024-01-11T07:07:53.685303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/3458914988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3458914988},{"type":"v1","addr":"172.21.15.7:6834","nonce":3458914988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-11T07:08:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1834066f30 msgr2=0x7f18340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1834066f30 0x7f18340693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f18380093f0 tx=0x7f1838008040 comp rx=0 tx=0).stop 2024-01-11T07:08:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184815a280 msgr2=0x7f18480c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184815a280 0x7f18480c3970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f184000b610 tx=0x7f1840018000 comp rx=0 tx=0).stop 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 shutdown_connections 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1834066f30 0x7f18340693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f184815a280 0x7f18480c3970 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1848063230 0x7f184816a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.731+0000 7f184ec62700 1 --2- 172.21.15.7:0/2332354038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18480ac5b0 0x7f18480acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.732+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 >> 172.21.15.7:0/2332354038 conn(0x7f18480680e0 msgr2=0x7f18480cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.732+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 shutdown_connections 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.732+0000 7f184ec62700 1 -- 172.21.15.7:0/2332354038 wait complete. 2024-01-11T07:08:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T07:08:06.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.877+0000 7f4b2a66d700 1 Processor -- start 2024-01-11T07:08:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.877+0000 7f4b2a66d700 1 -- start start 2024-01-11T07:08:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b2a66d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b24157da0 0x7f4b240cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b2a66d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b240ce220 0x7f4b240ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b2a66d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2406d270 con 0x7f4b241579d0 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b2a66d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b240cda40 con 0x7f4b24157da0 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b2a66d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240cd5b0 con 0x7f4b240ce220 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b237fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b24157da0 0x7f4b240cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b23fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4b241579d0 0x7f4b2415c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51352/0 (socket says 172.21.15.7:51352) 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b23fff700 1 -- 172.21.15.7:0/1054405721 learned_addr learned my addr 172.21.15.7:0/1054405721 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.878+0000 7f4b28c0a700 1 --2- 172.21.15.7:0/1054405721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b240ce220 0x7f4b240ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b237fe700 1 -- 172.21.15.7:0/1054405721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b240ce220 msgr2=0x7f4b240ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b237fe700 1 --2- 172.21.15.7:0/1054405721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b240ce220 0x7f4b240ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b237fe700 1 -- 172.21.15.7:0/1054405721 >> v1:172.21.15.7:6789/0 conn(0x7f4b241579d0 legacy=0x7f4b2415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b237fe700 1 -- 172.21.15.7:0/1054405721 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240ce660 con 0x7f4b24157da0 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b237fe700 1 --2- 172.21.15.7:0/1054405721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b24157da0 0x7f4b240cdce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b240ce7f0 tx=0x7f4b180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1157a45cdfe4dd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1054405721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800b6b0 con 0x7f4b24157da0 2024-01-11T07:08:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1054405721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1800b810 con 0x7f4b24157da0 2024-01-11T07:08:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b22ffd700 1 -- 172.21.15.7:0/1054405721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800bbd0 con 0x7f4b24157da0 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1054405721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b24157da0 msgr2=0x7f4b240cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.879+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1054405721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b24157da0 0x7f4b240cdce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b240ce7f0 tx=0x7f4b180091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.880+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1054405721 shutdown_connections 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.880+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1054405721 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b240ce220 0x7f4b240ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.880+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1054405721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b24157da0 0x7f4b240cdce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.880+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1054405721 >> 172.21.15.7:0/1054405721 conn(0x7f4b2406ae00 msgr2=0x7f4b2406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.880+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1054405721 shutdown_connections 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.880+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1054405721 wait complete. 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.881+0000 7f4b2a66d700 1 Processor -- start 2024-01-11T07:08:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.881+0000 7f4b2a66d700 1 -- start start 2024-01-11T07:08:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b2a66d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b240ce220 0x7f4b2415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b2a66d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b241579d0 0x7f4b240cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b2a66d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b24157da0 0x7f4b240cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b2a66d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b24067c70 con 0x7f4b241579d0 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b2a66d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240c8910 con 0x7f4b240ce220 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b2a66d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240c5aa0 con 0x7f4b24157da0 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b237fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b241579d0 0x7f4b240cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b237fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b241579d0 0x7f4b240cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42606/0 (socket says 172.21.15.7:42606) 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b237fe700 1 -- 172.21.15.7:0/1053243566 learned_addr learned my addr 172.21.15.7:0/1053243566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b23fff700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b240ce220 0x7f4b2415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b28c0a700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b24157da0 0x7f4b240cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b237fe700 1 -- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b24157da0 msgr2=0x7f4b240cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b237fe700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b24157da0 0x7f4b240cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b237fe700 1 -- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b240ce220 msgr2=0x7f4b2415aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b237fe700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b240ce220 0x7f4b2415aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b237fe700 1 -- 172.21.15.7:0/1053243566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18009420 con 0x7f4b241579d0 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.882+0000 7f4b23fff700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b240ce220 0x7f4b2415aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.883+0000 7f4b237fe700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b241579d0 0x7f4b240cae40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406fda0 tx=0x7f4b18016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.883+0000 7f4b0bfff700 1 -- 172.21.15.7:0/1053243566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18011bc0 con 0x7f4b241579d0 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.883+0000 7f4b0bfff700 1 -- 172.21.15.7:0/1053243566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1801f070 con 0x7f4b241579d0 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.883+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b240c5d80 con 0x7f4b241579d0 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.883+0000 7f4b0bfff700 1 -- 172.21.15.7:0/1053243566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1801a470 con 0x7f4b241579d0 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.883+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b24067900 con 0x7f4b241579d0 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.884+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b24067e20 con 0x7f4b241579d0 2024-01-11T07:08:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.884+0000 7f4b0bfff700 1 -- 172.21.15.7:0/1053243566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4b18021030 con 0x7f4b241579d0 2024-01-11T07:08:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.885+0000 7f4b0bfff700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b10066f30 0x7f4b100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.885+0000 7f4b0bfff700 1 -- 172.21.15.7:0/1053243566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4b18085e30 con 0x7f4b241579d0 2024-01-11T07:08:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.885+0000 7f4b23fff700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b10066f30 0x7f4b100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.885+0000 7f4b23fff700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b10066f30 0x7f4b100693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406be30 tx=0x7f4b14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:06.889+0000 7f4b0bfff700 1 -- 172.21.15.7:0/1053243566 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b24067e20 con 0x7f4b241579d0 2024-01-11T07:08:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.094+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b2404c3c0 con 0x7f4b241579d0 2024-01-11T07:08:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.095+0000 7f4b0bfff700 1 -- 172.21.15.7:0/1053243566 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f4b18053080 con 0x7f4b241579d0 2024-01-11T07:08:07.094 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:07.095 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:02.185556+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:07:47.856440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-11T07:07:49.053507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:07.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b10066f30 msgr2=0x7f4b100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b10066f30 0x7f4b100693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406be30 tx=0x7f4b14009040 comp rx=0 tx=0).stop 2024-01-11T07:08:07.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b241579d0 msgr2=0x7f4b240cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b241579d0 0x7f4b240cae40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4b2406fda0 tx=0x7f4b18016000 comp rx=0 tx=0).stop 2024-01-11T07:08:07.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 shutdown_connections 2024-01-11T07:08:07.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4b10066f30 0x7f4b100693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b24157da0 0x7f4b240cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b240ce220 0x7f4b2415aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 --2- 172.21.15.7:0/1053243566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b241579d0 0x7f4b240cae40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 >> 172.21.15.7:0/1053243566 conn(0x7f4b2406ae00 msgr2=0x7f4b241535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:07.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 shutdown_connections 2024-01-11T07:08:07.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.097+0000 7f4b2a66d700 1 -- 172.21.15.7:0/1053243566 wait complete. 2024-01-11T07:08:07.110 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:08:07.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.244+0000 7fac96b02700 1 Processor -- start 2024-01-11T07:08:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.244+0000 7fac96b02700 1 -- start start 2024-01-11T07:08:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac96b02700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fac90157ab0 0x7fac900af660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac96b02700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fac900ceeb0 0x7fac900ad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac96b02700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac900b0950 con 0x7fac900adc80 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac96b02700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9006bde0 con 0x7fac900ceeb0 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac96b02700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac900ae070 con 0x7fac90157ab0 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac8ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fac900ceeb0 0x7fac900ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac9509f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fac900adc80 0x7fac9015b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51356/0 (socket says 172.21.15.7:51356) 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac8ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fac900ceeb0 0x7fac900ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42614/0 (socket says 172.21.15.7:42614) 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac9509f700 1 -- 172.21.15.7:0/3449131969 learned_addr learned my addr 172.21.15.7:0/3449131969 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.245+0000 7fac9489e700 1 --2- 172.21.15.7:0/3449131969 >> v2:172.21.15.113:3300/0 conn(0x7fac90157ab0 0x7fac900af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8ffff700 1 -- 172.21.15.7:0/3449131969 >> v2:172.21.15.113:3300/0 conn(0x7fac90157ab0 msgr2=0x7fac900af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8ffff700 1 --2- 172.21.15.7:0/3449131969 >> v2:172.21.15.113:3300/0 conn(0x7fac90157ab0 0x7fac900af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8ffff700 1 -- 172.21.15.7:0/3449131969 >> v1:172.21.15.7:6789/0 conn(0x7fac900adc80 legacy=0x7fac9015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8f7fe700 1 -- 172.21.15.7:0/3449131969 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878983222 0 0) 0x7fac900b0950 con 0x7fac900adc80 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8ffff700 1 -- 172.21.15.7:0/3449131969 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac900afd40 con 0x7fac900ceeb0 2024-01-11T07:08:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8ffff700 1 --2- 172.21.15.7:0/3449131969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac900ceeb0 0x7fac900ad740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac900b1530 tx=0x7fac840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67981eaec9d974b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8f7fe700 1 -- 172.21.15.7:0/3449131969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84011630 con 0x7fac900ceeb0 2024-01-11T07:08:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8f7fe700 1 -- 172.21.15.7:0/3449131969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac84011790 con 0x7fac900ceeb0 2024-01-11T07:08:07.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac8f7fe700 1 -- 172.21.15.7:0/3449131969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84011b30 con 0x7fac900ceeb0 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac96b02700 1 -- 172.21.15.7:0/3449131969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac900ceeb0 msgr2=0x7fac900ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.246+0000 7fac96b02700 1 --2- 172.21.15.7:0/3449131969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac900ceeb0 0x7fac900ad740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac900b1530 tx=0x7fac840091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.247+0000 7fac96b02700 1 -- 172.21.15.7:0/3449131969 shutdown_connections 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.247+0000 7fac96b02700 1 --2- 172.21.15.7:0/3449131969 >> v2:172.21.15.113:3300/0 conn(0x7fac90157ab0 0x7fac900af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.247+0000 7fac96b02700 1 --2- 172.21.15.7:0/3449131969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac900ceeb0 0x7fac900ad740 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.247+0000 7fac96b02700 1 -- 172.21.15.7:0/3449131969 >> 172.21.15.7:0/3449131969 conn(0x7fac90063230 msgr2=0x7fac90063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.247+0000 7fac96b02700 1 -- 172.21.15.7:0/3449131969 shutdown_connections 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.247+0000 7fac96b02700 1 -- 172.21.15.7:0/3449131969 wait complete. 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.248+0000 7fac96b02700 1 Processor -- start 2024-01-11T07:08:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.248+0000 7fac96b02700 1 -- start start 2024-01-11T07:08:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac96b02700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac900adc80 0x7fac90168db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac96b02700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac900ceeb0 0x7fac9016a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac96b02700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac90157ab0 0x7fac900c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac96b02700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90167f80 con 0x7fac90157ab0 2024-01-11T07:08:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac96b02700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90168260 con 0x7fac900adc80 2024-01-11T07:08:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac96b02700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac90168540 con 0x7fac900ceeb0 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9509f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac90157ab0 0x7fac900c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9509f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac90157ab0 0x7fac900c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42620/0 (socket says 172.21.15.7:42620) 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9509f700 1 -- 172.21.15.7:0/3044650387 learned_addr learned my addr 172.21.15.7:0/3044650387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9489e700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac900adc80 0x7fac90168db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac8ffff700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac900ceeb0 0x7fac9016a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9509f700 1 -- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac900ceeb0 msgr2=0x7fac9016a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9509f700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac900ceeb0 0x7fac9016a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9509f700 1 -- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac900adc80 msgr2=0x7fac90168db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9509f700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac900adc80 0x7fac90168db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.249+0000 7fac9509f700 1 -- 172.21.15.7:0/3044650387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84009420 con 0x7fac90157ab0 2024-01-11T07:08:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.250+0000 7fac9509f700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac90157ab0 0x7fac900c9880 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fac8800cc90 tx=0x7fac88001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.250+0000 7fac8cff9700 1 -- 172.21.15.7:0/3044650387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800a8b0 con 0x7fac90157ab0 2024-01-11T07:08:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.250+0000 7fac8cff9700 1 -- 172.21.15.7:0/3044650387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8800aa10 con 0x7fac90157ab0 2024-01-11T07:08:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.250+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac90168820 con 0x7fac90157ab0 2024-01-11T07:08:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.250+0000 7fac8cff9700 1 -- 172.21.15.7:0/3044650387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800add0 con 0x7fac90157ab0 2024-01-11T07:08:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.250+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac90167c80 con 0x7fac90157ab0 2024-01-11T07:08:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.251+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9004c3c0 con 0x7fac90157ab0 2024-01-11T07:08:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.251+0000 7fac8cff9700 1 -- 172.21.15.7:0/3044650387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fac88007300 con 0x7fac90157ab0 2024-01-11T07:08:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.252+0000 7fac8cff9700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fac7c066f30 0x7fac7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.252+0000 7fac8cff9700 1 -- 172.21.15.7:0/3044650387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac88084850 con 0x7fac90157ab0 2024-01-11T07:08:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.252+0000 7fac9489e700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fac7c066f30 0x7fac7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.253+0000 7fac9489e700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fac7c066f30 0x7fac7c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fac80009d10 tx=0x7fac800091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.257+0000 7fac8cff9700 1 -- 172.21.15.7:0/3044650387 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac88051b40 con 0x7fac90157ab0 2024-01-11T07:08:07.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.508+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac900b0020 con 0x7fac90157ab0 2024-01-11T07:08:07.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.509+0000 7fac8cff9700 1 -- 172.21.15.7:0/3044650387 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fac88055160 con 0x7fac90157ab0 2024-01-11T07:08:07.509 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:07.509 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:07:50.072668+0000","modified":"2024-01-11T07:07:53.685303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/3458914988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3458914988},{"type":"v1","addr":"172.21.15.7:6834","nonce":3458914988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-11T07:08:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fac7c066f30 msgr2=0x7fac7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fac7c066f30 0x7fac7c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fac80009d10 tx=0x7fac800091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac90157ab0 msgr2=0x7fac900c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac90157ab0 0x7fac900c9880 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fac8800cc90 tx=0x7fac88001a80 comp rx=0 tx=0).stop 2024-01-11T07:08:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 shutdown_connections 2024-01-11T07:08:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fac7c066f30 0x7fac7c0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac90157ab0 0x7fac900c9880 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac900adc80 0x7fac90168db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 --2- 172.21.15.7:0/3044650387 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac900ceeb0 0x7fac9016a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 >> 172.21.15.7:0/3044650387 conn(0x7fac90063230 msgr2=0x7fac900cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:07.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 shutdown_connections 2024-01-11T07:08:07.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.511+0000 7fac96b02700 1 -- 172.21.15.7:0/3044650387 wait complete. 2024-01-11T07:08:07.513 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T07:08:07.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.657+0000 7f6de70c3700 1 Processor -- start 2024-01-11T07:08:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.657+0000 7f6de70c3700 1 -- start start 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.658+0000 7f6de70c3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6de006dd00 0x7f6de00c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.658+0000 7f6de70c3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 0x7f6de0159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.658+0000 7f6de70c3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de015b770 con 0x7f6de0159ff0 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.658+0000 7f6de70c3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de015abf0 con 0x7f6de006dd00 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.658+0000 7f6de70c3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de015aed0 con 0x7f6de006e0d0 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.658+0000 7f6de4e5f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6de006dd00 0x7f6de00c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.658+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 0x7f6de0159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.658+0000 7f6de5660700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6de0159ff0 0x7f6de00c7e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:38768/0 (socket says 172.21.15.7:38768) 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6de5660700 1 -- 172.21.15.7:0/2473028085 learned_addr learned my addr 172.21.15.7:0/2473028085 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6de4e5f700 1 -- 172.21.15.7:0/2473028085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 msgr2=0x7f6de0159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6de4e5f700 1 --2- 172.21.15.7:0/2473028085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 0x7f6de0159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6de4e5f700 1 -- 172.21.15.7:0/2473028085 >> v1:172.21.15.113:6789/0 conn(0x7f6de0159ff0 legacy=0x7f6de00c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6de4e5f700 1 -- 172.21.15.7:0/2473028085 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de015b1b0 con 0x7f6de006dd00 2024-01-11T07:08:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6ddf7fe700 1 -- 172.21.15.7:0/2473028085 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717413254 0 0) 0x7f6de015b770 con 0x7f6de0159ff0 2024-01-11T07:08:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6de4e5f700 1 --2- 172.21.15.7:0/2473028085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 0x7f6de00c98d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00098e0 tx=0x7f6dd00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cfa4dc7162801f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6ddf7fe700 1 -- 172.21.15.7:0/2473028085 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000b640 con 0x7f6de006dd00 2024-01-11T07:08:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6ddf7fe700 1 -- 172.21.15.7:0/2473028085 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd000b7a0 con 0x7f6de006dd00 2024-01-11T07:08:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.659+0000 7f6ddf7fe700 1 -- 172.21.15.7:0/2473028085 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000bb40 con 0x7f6de006dd00 2024-01-11T07:08:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.660+0000 7f6de70c3700 1 -- 172.21.15.7:0/2473028085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 msgr2=0x7f6de00c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.660+0000 7f6de70c3700 1 --2- 172.21.15.7:0/2473028085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 0x7f6de00c98d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00098e0 tx=0x7f6dd00091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.660+0000 7f6de70c3700 1 -- 172.21.15.7:0/2473028085 shutdown_connections 2024-01-11T07:08:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.660+0000 7f6de70c3700 1 --2- 172.21.15.7:0/2473028085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 0x7f6de00c98d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.660+0000 7f6de70c3700 1 --2- 172.21.15.7:0/2473028085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 0x7f6de0159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.660+0000 7f6de70c3700 1 -- 172.21.15.7:0/2473028085 >> 172.21.15.7:0/2473028085 conn(0x7f6de00680e0 msgr2=0x7f6de00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.660+0000 7f6de70c3700 1 -- 172.21.15.7:0/2473028085 shutdown_connections 2024-01-11T07:08:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.660+0000 7f6de70c3700 1 -- 172.21.15.7:0/2473028085 wait complete. 2024-01-11T07:08:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.662+0000 7f6de70c3700 1 Processor -- start 2024-01-11T07:08:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.662+0000 7f6de70c3700 1 -- start start 2024-01-11T07:08:07.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de70c3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 0x7f6de0158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de70c3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 0x7f6de0149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de70c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6de0159ff0 0x7f6de01512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de70c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de014e070 con 0x7f6de0159ff0 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de70c3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de014dd20 con 0x7f6de006dd00 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de70c3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00c7050 con 0x7f6de006e0d0 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de5660700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6de0159ff0 0x7f6de01512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de5660700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6de0159ff0 0x7f6de01512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42256/0 (socket says 172.21.15.7:42256) 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de5660700 1 -- 172.21.15.7:0/3596076828 learned_addr learned my addr 172.21.15.7:0/3596076828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de4e5f700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 0x7f6de0158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6ddffff700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 0x7f6de0149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de5660700 1 -- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 msgr2=0x7f6de0149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de5660700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 0x7f6de0149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de5660700 1 -- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 msgr2=0x7f6de0158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de5660700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 0x7f6de0158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.663+0000 7f6de5660700 1 -- 172.21.15.7:0/3596076828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0009420 con 0x7f6de0159ff0 2024-01-11T07:08:07.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.664+0000 7f6de5660700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6de0159ff0 0x7f6de01512b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800cbf0 tx=0x7f6dd80019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:07.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.664+0000 7f6ddcff9700 1 -- 172.21.15.7:0/3596076828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800a8b0 con 0x7f6de0159ff0 2024-01-11T07:08:07.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.664+0000 7f6ddcff9700 1 -- 172.21.15.7:0/3596076828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800aa10 con 0x7f6de0159ff0 2024-01-11T07:08:07.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.664+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de00c7330 con 0x7f6de0159ff0 2024-01-11T07:08:07.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.664+0000 7f6ddcff9700 1 -- 172.21.15.7:0/3596076828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800add0 con 0x7f6de0159ff0 2024-01-11T07:08:07.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.664+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de0159190 con 0x7f6de0159ff0 2024-01-11T07:08:07.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.665+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de014e220 con 0x7f6de0159ff0 2024-01-11T07:08:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.666+0000 7f6ddcff9700 1 -- 172.21.15.7:0/3596076828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6dd8007300 con 0x7f6de0159ff0 2024-01-11T07:08:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.666+0000 7f6ddcff9700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6dc0066f30 0x7f6dc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.666+0000 7f6ddcff9700 1 -- 172.21.15.7:0/3596076828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6dd8085080 con 0x7f6de0159ff0 2024-01-11T07:08:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.666+0000 7f6de4e5f700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6dc0066f30 0x7f6dc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.667+0000 7f6de4e5f700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6dc0066f30 0x7f6dc00693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00093f0 tx=0x7f6dd0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:07.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.670+0000 7f6ddcff9700 1 -- 172.21.15.7:0/3596076828 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dd8052450 con 0x7f6de0159ff0 2024-01-11T07:08:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.923+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6de004c3c0 con 0x7f6de0159ff0 2024-01-11T07:08:07.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.924+0000 7f6ddcff9700 1 -- 172.21.15.7:0/3596076828 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f6dd8055a70 con 0x7f6de0159ff0 2024-01-11T07:08:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:07.924 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:07:50.072668+0000","modified":"2024-01-11T07:07:53.685303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4917},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4917":{"gid":4917,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6834/3458914988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3458914988},{"type":"v1","addr":"172.21.15.7:6834","nonce":3458914988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-01-11T07:08:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6dc0066f30 msgr2=0x7f6dc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6dc0066f30 0x7f6dc00693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00093f0 tx=0x7f6dd0017000 comp rx=0 tx=0).stop 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6de0159ff0 msgr2=0x7f6de01512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6de0159ff0 0x7f6de01512b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800cbf0 tx=0x7f6dd80019b0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 shutdown_connections 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6dc0066f30 0x7f6dc00693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6de0159ff0 0x7f6de01512b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6de006dd00 0x7f6de0158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 --2- 172.21.15.7:0/3596076828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6de006e0d0 0x7f6de0149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 >> 172.21.15.7:0/3596076828 conn(0x7f6de00680e0 msgr2=0x7f6de00ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 shutdown_connections 2024-01-11T07:08:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:07.926+0000 7f6de70c3700 1 -- 172.21.15.7:0/3596076828 wait complete. 2024-01-11T07:08:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-01-11T07:08:07.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.072+0000 7f84a0f60700 1 Processor -- start 2024-01-11T07:08:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.072+0000 7f84a0f60700 1 -- start start 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.073+0000 7f84a0f60700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f849c0cafb0 0x7f849c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.073+0000 7f84a0f60700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849c159300 con 0x7f849c158280 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.073+0000 7f84a0f60700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849c0cf260 con 0x7f849c0b0b40 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.073+0000 7f84a0f60700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849c0cf540 con 0x7f849c0cafb0 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f849c0cafb0 0x7f849c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f8499d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f849c158280 0x7f849c0b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34082/0 (socket says 172.21.15.7:34082) 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f849c0cafb0 0x7f849c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42266/0 (socket says 172.21.15.7:42266) 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f8499d9b700 1 -- 172.21.15.7:0/1468912660 learned_addr learned my addr 172.21.15.7:0/1468912660 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849a59c700 1 -- 172.21.15.7:0/1468912660 >> v1:172.21.15.113:6789/0 conn(0x7f849c0b0b40 legacy=0x7f849c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849a59c700 1 -- 172.21.15.7:0/1468912660 >> v1:172.21.15.7:6789/0 conn(0x7f849c158280 legacy=0x7f849c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849a59c700 1 -- 172.21.15.7:0/1468912660 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c0cf820 con 0x7f849c0cafb0 2024-01-11T07:08:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849959a700 1 -- 172.21.15.7:0/1468912660 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314745524 0 0) 0x7f849c159300 con 0x7f849c158280 2024-01-11T07:08:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849a59c700 1 --2- 172.21.15.7:0/1468912660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c0ce8f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f84840098e0 tx=0x7f84840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf55083bd1cde6cb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849959a700 1 -- 172.21.15.7:0/1468912660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484009e80 con 0x7f849c0cafb0 2024-01-11T07:08:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849959a700 1 -- 172.21.15.7:0/1468912660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848400b860 con 0x7f849c0cafb0 2024-01-11T07:08:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.074+0000 7f849959a700 1 -- 172.21.15.7:0/1468912660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400bbe0 con 0x7f849c0cafb0 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.075+0000 7f84a0f60700 1 -- 172.21.15.7:0/1468912660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 msgr2=0x7f849c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.075+0000 7f84a0f60700 1 --2- 172.21.15.7:0/1468912660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c0ce8f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f84840098e0 tx=0x7f84840091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.075+0000 7f84a0f60700 1 -- 172.21.15.7:0/1468912660 shutdown_connections 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.075+0000 7f84a0f60700 1 --2- 172.21.15.7:0/1468912660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c0ce8f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.075+0000 7f84a0f60700 1 -- 172.21.15.7:0/1468912660 >> 172.21.15.7:0/1468912660 conn(0x7f849c071100 msgr2=0x7f849c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.075+0000 7f84a0f60700 1 -- 172.21.15.7:0/1468912660 shutdown_connections 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.075+0000 7f84a0f60700 1 -- 172.21.15.7:0/1468912660 wait complete. 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.076+0000 7f84a0f60700 1 Processor -- start 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.076+0000 7f84a0f60700 1 -- start start 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f84a0f60700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f849c0b0b40 0x7f849c1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f84a0f60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c068080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f84a0f60700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849c158280 0x7f849c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f84a0f60700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c149ae0 con 0x7f849c0cafb0 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f84a0f60700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c069600 con 0x7f849c158280 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f84a0f60700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849c0689f0 con 0x7f849c0b0b40 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f8499d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f8499d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c068080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42274/0 (socket says 172.21.15.7:42274) 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f8499d9b700 1 -- 172.21.15.7:0/3106176281 learned_addr learned my addr 172.21.15.7:0/3106176281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f849a59c700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f849c0b0b40 0x7f849c1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f849ad9d700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849c158280 0x7f849c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f8499d9b700 1 -- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f849c0b0b40 msgr2=0x7f849c1495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f8499d9b700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f849c0b0b40 0x7f849c1495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f8499d9b700 1 -- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849c158280 msgr2=0x7f849c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f8499d9b700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849c158280 0x7f849c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.077+0000 7f8499d9b700 1 -- 172.21.15.7:0/3106176281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8484009420 con 0x7f849c0cafb0 2024-01-11T07:08:08.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.078+0000 7f8499d9b700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c068080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f848c00c530 tx=0x7f848c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.078+0000 7f84927fc700 1 -- 172.21.15.7:0/3106176281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c005b30 con 0x7f849c0cafb0 2024-01-11T07:08:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.078+0000 7f84927fc700 1 -- 172.21.15.7:0/3106176281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c005c90 con 0x7f849c0cafb0 2024-01-11T07:08:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.078+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849c068cd0 con 0x7f849c0cafb0 2024-01-11T07:08:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.078+0000 7f84927fc700 1 -- 172.21.15.7:0/3106176281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c007320 con 0x7f849c0cafb0 2024-01-11T07:08:08.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.078+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c069290 con 0x7f849c0cafb0 2024-01-11T07:08:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.079+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849c0697b0 con 0x7f849c0cafb0 2024-01-11T07:08:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.080+0000 7f84927fc700 1 -- 172.21.15.7:0/3106176281 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f848c01b3b0 con 0x7f849c0cafb0 2024-01-11T07:08:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.080+0000 7f84927fc700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f847c066f30 0x7f847c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.080+0000 7f84927fc700 1 -- 172.21.15.7:0/3106176281 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f848c083a00 con 0x7f849c0cafb0 2024-01-11T07:08:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.080+0000 7f849a59c700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f847c066f30 0x7f847c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.081+0000 7f849a59c700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f847c066f30 0x7f847c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f84840093f0 tx=0x7f8484017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.085+0000 7f84927fc700 1 -- 172.21.15.7:0/3106176281 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f848c050c70 con 0x7f849c0cafb0 2024-01-11T07:08:08.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.292+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f849c04c3c0 con 0x7f849c0cafb0 2024-01-11T07:08:08.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.293+0000 7f84927fc700 1 -- 172.21.15.7:0/3106176281 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f848c017020 con 0x7f849c0cafb0 2024-01-11T07:08:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:08.293 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:02.185556+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:07:47.856440+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-11T07:07:49.053507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:08.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f847c066f30 msgr2=0x7f847c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f847c066f30 0x7f847c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f84840093f0 tx=0x7f8484017000 comp rx=0 tx=0).stop 2024-01-11T07:08:08.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 msgr2=0x7f849c068080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c068080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f848c00c530 tx=0x7f848c007f90 comp rx=0 tx=0).stop 2024-01-11T07:08:08.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 shutdown_connections 2024-01-11T07:08:08.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f847c066f30 0x7f847c0693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f849c158280 0x7f849c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f849c0b0b40 0x7f849c1495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 --2- 172.21.15.7:0/3106176281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f849c0cafb0 0x7f849c068080 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 >> 172.21.15.7:0/3106176281 conn(0x7f849c071100 msgr2=0x7f849c1573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 shutdown_connections 2024-01-11T07:08:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.295+0000 7f84a0f60700 1 -- 172.21.15.7:0/3106176281 wait complete. 2024-01-11T07:08:08.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:08:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.441+0000 7f57c4a09700 1 Processor -- start 2024-01-11T07:08:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.441+0000 7f57c4a09700 1 -- start start 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57c4a09700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57c0063230 0x7f57c0063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57c4a09700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 0x7f57c00aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57c4a09700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f57c00ac5b0 0x7f57c00aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57c4a09700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c015d880 con 0x7f57c0063230 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57c4a09700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c006c5b0 con 0x7f57c00ac5b0 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57c4a09700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0063b40 con 0x7f57c015a280 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57be59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57c0063230 0x7f57c0063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57be59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57c0063230 0x7f57c0063600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42276/0 (socket says 172.21.15.7:42276) 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.442+0000 7f57be59c700 1 -- 172.21.15.7:0/1724563469 learned_addr learned my addr 172.21.15.7:0/1724563469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57bdd9b700 1 --2- 172.21.15.7:0/1724563469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 0x7f57c00aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57bed9d700 1 --2- 172.21.15.7:0/1724563469 >> v2:172.21.15.113:3300/0 conn(0x7f57c00ac5b0 0x7f57c00aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57be59c700 1 -- 172.21.15.7:0/1724563469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 msgr2=0x7f57c00aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57be59c700 1 --2- 172.21.15.7:0/1724563469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 0x7f57c00aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57be59c700 1 -- 172.21.15.7:0/1724563469 >> v2:172.21.15.113:3300/0 conn(0x7f57c00ac5b0 msgr2=0x7f57c00aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57be59c700 1 --2- 172.21.15.7:0/1724563469 >> v2:172.21.15.113:3300/0 conn(0x7f57c00ac5b0 0x7f57c00aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57be59c700 1 -- 172.21.15.7:0/1724563469 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c015a7f0 con 0x7f57c0063230 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57be59c700 1 --2- 172.21.15.7:0/1724563469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c0063600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f57b00099b0 tx=0x7f57b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c863413cb7156a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57bd59a700 1 -- 172.21.15.7:0/1724563469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000b820 con 0x7f57c0063230 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57bd59a700 1 -- 172.21.15.7:0/1724563469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b000b980 con 0x7f57c0063230 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.443+0000 7f57bd59a700 1 -- 172.21.15.7:0/1724563469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000bd20 con 0x7f57c0063230 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 -- 172.21.15.7:0/1724563469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 msgr2=0x7f57c0063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 --2- 172.21.15.7:0/1724563469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c0063600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f57b00099b0 tx=0x7f57b0009250 comp rx=0 tx=0).stop 2024-01-11T07:08:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 -- 172.21.15.7:0/1724563469 shutdown_connections 2024-01-11T07:08:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 --2- 172.21.15.7:0/1724563469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c0063600 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 --2- 172.21.15.7:0/1724563469 >> v2:172.21.15.113:3300/0 conn(0x7f57c00ac5b0 0x7f57c00aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 --2- 172.21.15.7:0/1724563469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 0x7f57c00aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 -- 172.21.15.7:0/1724563469 >> 172.21.15.7:0/1724563469 conn(0x7f57c00680e0 msgr2=0x7f57c00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 -- 172.21.15.7:0/1724563469 shutdown_connections 2024-01-11T07:08:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.444+0000 7f57c4a09700 1 -- 172.21.15.7:0/1724563469 wait complete. 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.445+0000 7f57c4a09700 1 Processor -- start 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.445+0000 7f57c4a09700 1 -- start start 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57c4a09700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c00ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57c4a09700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57c00ac5b0 0x7f57c00cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57c4a09700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 0x7f57c00afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57c4a09700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00ccd70 con 0x7f57c0063230 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57c4a09700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c015dd10 con 0x7f57c00ac5b0 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57c4a09700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c015dff0 con 0x7f57c015a280 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c00ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c00ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42284/0 (socket says 172.21.15.7:42284) 2024-01-11T07:08:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 -- 172.21.15.7:0/2851580516 learned_addr learned my addr 172.21.15.7:0/2851580516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57bdd9b700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57c00ac5b0 0x7f57c00cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57bed9d700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 0x7f57c00afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 -- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 msgr2=0x7f57c00afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 0x7f57c00afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 -- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57c00ac5b0 msgr2=0x7f57c00cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57c00ac5b0 0x7f57c00cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 -- 172.21.15.7:0/2851580516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00094f0 con 0x7f57c0063230 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.446+0000 7f57be59c700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c00ad290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f57b00094c0 tx=0x7f57b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.447+0000 7f57ae7fc700 1 -- 172.21.15.7:0/2851580516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0011d50 con 0x7f57c0063230 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.447+0000 7f57ae7fc700 1 -- 172.21.15.7:0/2851580516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0024440 con 0x7f57c0063230 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.447+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c00b02a0 con 0x7f57c0063230 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.447+0000 7f57ae7fc700 1 -- 172.21.15.7:0/2851580516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0024800 con 0x7f57c0063230 2024-01-11T07:08:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.447+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c00adb50 con 0x7f57c0063230 2024-01-11T07:08:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.448+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c006c3f0 con 0x7f57c0063230 2024-01-11T07:08:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.448+0000 7f57ae7fc700 1 -- 172.21.15.7:0/2851580516 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f57b002d440 con 0x7f57c0063230 2024-01-11T07:08:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.448+0000 7f57ae7fc700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f57a0066f30 0x7f57a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.449+0000 7f57ae7fc700 1 -- 172.21.15.7:0/2851580516 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f57b0085cd0 con 0x7f57c0063230 2024-01-11T07:08:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.449+0000 7f57bdd9b700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f57a0066f30 0x7f57a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.449+0000 7f57bdd9b700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f57a0066f30 0x7f57a00693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f57c0153bb0 tx=0x7f57b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.453+0000 7f57ae7fc700 1 -- 172.21.15.7:0/2851580516 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57b0052fc0 con 0x7f57c0063230 2024-01-11T07:08:08.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.707+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f57c004c3c0 con 0x7f57c0063230 2024-01-11T07:08:08.707 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:08:08.707+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:08:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.711+0000 7f57ae7fc700 1 -- 172.21.15.7:0/2851580516 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f57b00565e0 con 0x7f57c0063230 2024-01-11T07:08:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.714+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f57a0066f30 msgr2=0x7f57a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.714+0000 7f57c4a09700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f57a0066f30 0x7f57a00693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f57c0153bb0 tx=0x7f57b4008040 comp rx=0 tx=0).stop 2024-01-11T07:08:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.714+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 msgr2=0x7f57c00ad290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.714+0000 7f57c4a09700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c00ad290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f57b00094c0 tx=0x7f57b0017000 comp rx=0 tx=0).stop 2024-01-11T07:08:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.715+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 shutdown_connections 2024-01-11T07:08:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.715+0000 7f57c4a09700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f57a0066f30 0x7f57a00693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.715+0000 7f57c4a09700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f57c015a280 0x7f57c00afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.715+0000 7f57c4a09700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57c0063230 0x7f57c00ad290 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.715+0000 7f57c4a09700 1 --2- 172.21.15.7:0/2851580516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57c00ac5b0 0x7f57c00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.715+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 >> 172.21.15.7:0/2851580516 conn(0x7f57c00680e0 msgr2=0x7f57c015b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:08.716 INFO:tasks.ceph.mds.c.smithi007.stderr: -3466> 2024-01-11T07:07:52.610+0000 7fe3c4ef5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:08:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.715+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 shutdown_connections 2024-01-11T07:08:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.715+0000 7f57c4a09700 1 -- 172.21.15.7:0/2851580516 wait complete. 2024-01-11T07:08:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:08:08.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:08:08.741 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:08:08.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.864+0000 7fb3c29bd700 1 Processor -- start 2024-01-11T07:08:08.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.864+0000 7fb3c29bd700 1 -- start start 2024-01-11T07:08:08.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3c29bd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3bc15a280 0x7fb3bc0aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3c29bd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3bc15d880 con 0x7fb3bc0ac5b0 2024-01-11T07:08:08.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3c29bd700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3bc06c5b0 con 0x7fb3bc063230 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3c29bd700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3bc15a7f0 con 0x7fb3bc15a280 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3c0f5a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb3bc0ac5b0 0x7fb3bc0ac9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34098/0 (socket says 172.21.15.7:34098) 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3c0f5a700 1 -- 172.21.15.7:0/1226948895 learned_addr learned my addr 172.21.15.7:0/1226948895 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3bb7fe700 1 --2- 172.21.15.7:0/1226948895 >> v2:172.21.15.113:3300/0 conn(0x7fb3bc15a280 0x7fb3bc0aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3baffd700 1 -- 172.21.15.7:0/1226948895 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315983950 0 0) 0x7fb3bc15d880 con 0x7fb3bc0ac5b0 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3baffd700 1 -- 172.21.15.7:0/1226948895 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3a8003570 con 0x7fb3bc0ac5b0 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.865+0000 7fb3baffd700 1 -- 172.21.15.7:0/1226948895 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757418069 0 0) 0x7fb3bc06c5b0 con 0x7fb3bc063230 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.866+0000 7fb3baffd700 1 -- 172.21.15.7:0/1226948895 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3bc15d880 con 0x7fb3bc063230 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.866+0000 7fb3bb7fe700 1 -- 172.21.15.7:0/1226948895 >> v1:172.21.15.113:6789/0 conn(0x7fb3bc063230 legacy=0x7fb3bc063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.866+0000 7fb3bb7fe700 1 -- 172.21.15.7:0/1226948895 >> v1:172.21.15.7:6789/0 conn(0x7fb3bc0ac5b0 legacy=0x7fb3bc0ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.866+0000 7fb3bb7fe700 1 -- 172.21.15.7:0/1226948895 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc15aad0 con 0x7fb3bc15a280 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.866+0000 7fb3bb7fe700 1 --2- 172.21.15.7:0/1226948895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0aa9f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40098e0 tx=0x7fb3a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a44486a88ccd179 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.866+0000 7fb3baffd700 1 -- 172.21.15.7:0/1226948895 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4011630 con 0x7fb3bc15a280 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.866+0000 7fb3baffd700 1 -- 172.21.15.7:0/1226948895 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a4011790 con 0x7fb3bc15a280 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.866+0000 7fb3baffd700 1 -- 172.21.15.7:0/1226948895 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4011b30 con 0x7fb3bc15a280 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.867+0000 7fb3c29bd700 1 -- 172.21.15.7:0/1226948895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc15a280 msgr2=0x7fb3bc0aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.867+0000 7fb3c29bd700 1 --2- 172.21.15.7:0/1226948895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0aa9f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40098e0 tx=0x7fb3a40091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.867+0000 7fb3c29bd700 1 -- 172.21.15.7:0/1226948895 shutdown_connections 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.867+0000 7fb3c29bd700 1 --2- 172.21.15.7:0/1226948895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0aa9f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.867+0000 7fb3c29bd700 1 -- 172.21.15.7:0/1226948895 >> 172.21.15.7:0/1226948895 conn(0x7fb3bc0680e0 msgr2=0x7fb3bc0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.867+0000 7fb3c29bd700 1 -- 172.21.15.7:0/1226948895 shutdown_connections 2024-01-11T07:08:08.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.867+0000 7fb3c29bd700 1 -- 172.21.15.7:0/1226948895 wait complete. 2024-01-11T07:08:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.868+0000 7fb3c29bd700 1 Processor -- start 2024-01-11T07:08:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.868+0000 7fb3c29bd700 1 -- start start 2024-01-11T07:08:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c29bd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc063230 0x7fb3bc145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c29bd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3bc0ac5b0 0x7fb3bc0ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c29bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c29bd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0b3080 con 0x7fb3bc15a280 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c29bd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0b2470 con 0x7fb3bc063230 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c29bd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0b2750 con 0x7fb3bc0ac5b0 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c0f5a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c0f5a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42310/0 (socket says 172.21.15.7:42310) 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c0f5a700 1 -- 172.21.15.7:0/3317519675 learned_addr learned my addr 172.21.15.7:0/3317519675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3bbfff700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc063230 0x7fb3bc145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3bb7fe700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3bc0ac5b0 0x7fb3bc0ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c0f5a700 1 -- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3bc0ac5b0 msgr2=0x7fb3bc0ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c0f5a700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3bc0ac5b0 0x7fb3bc0ad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c0f5a700 1 -- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc063230 msgr2=0x7fb3bc145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c0f5a700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc063230 0x7fb3bc145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.869+0000 7fb3c0f5a700 1 -- 172.21.15.7:0/3317519675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a4009420 con 0x7fb3bc15a280 2024-01-11T07:08:08.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.870+0000 7fb3c0f5a700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0ad6f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400cc60 tx=0x7fb3b4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.870+0000 7fb3b3fff700 1 -- 172.21.15.7:0/3317519675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400a8b0 con 0x7fb3bc15a280 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.870+0000 7fb3b3fff700 1 -- 172.21.15.7:0/3317519675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b400aa30 con 0x7fb3bc15a280 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.870+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3bc0b2a30 con 0x7fb3bc15a280 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.870+0000 7fb3b3fff700 1 -- 172.21.15.7:0/3317519675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400adf0 con 0x7fb3bc15a280 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.870+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3bc146110 con 0x7fb3bc15a280 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.871+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3bc04c3c0 con 0x7fb3bc15a280 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.871+0000 7fb3b3fff700 1 -- 172.21.15.7:0/3317519675 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb3b4007300 con 0x7fb3bc15a280 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.872+0000 7fb3b3fff700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb39c066f30 0x7fb39c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.872+0000 7fb3b3fff700 1 -- 172.21.15.7:0/3317519675 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fb3b40849c0 con 0x7fb3bc15a280 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.872+0000 7fb3bbfff700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb39c066f30 0x7fb39c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:08.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.872+0000 7fb3bbfff700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb39c066f30 0x7fb39c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac007fd0 tx=0x7fb3ac007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:08.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:08.876+0000 7fb3b3fff700 1 -- 172.21.15.7:0/3317519675 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3b4055670 con 0x7fb3bc15a280 2024-01-11T07:08:09.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.125+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb3bc15adb0 con 0x7fb3bc15a280 2024-01-11T07:08:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.713+0000 7fb3b3fff700 1 -- 172.21.15.7:0/3317519675 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3b4055200 con 0x7fb3bc15a280 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb39c066f30 msgr2=0x7fb39c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb39c066f30 0x7fb39c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac007fd0 tx=0x7fb3ac007d50 comp rx=0 tx=0).stop 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3bc15a280 msgr2=0x7fb3bc0ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0ad6f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400cc60 tx=0x7fb3b4001ae0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 shutdown_connections 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb39c066f30 0x7fb39c0693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3bc15a280 0x7fb3bc0ad6f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3bc063230 0x7fb3bc145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 --2- 172.21.15.7:0/3317519675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3bc0ac5b0 0x7fb3bc0ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 >> 172.21.15.7:0/3317519675 conn(0x7fb3bc0680e0 msgr2=0x7fb3bc153d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 shutdown_connections 2024-01-11T07:08:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.716+0000 7fb3c29bd700 1 -- 172.21.15.7:0/3317519675 wait complete. 2024-01-11T07:08:09.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:08:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.913+0000 7fbbe8a4b700 1 Processor -- start 2024-01-11T07:08:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.913+0000 7fbbe8a4b700 1 -- start start 2024-01-11T07:08:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.914+0000 7fbbe8a4b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbe4069ea0 0x7fbbe406db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.914+0000 7fbbe8a4b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe406a270 0x7fbbe406e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.914+0000 7fbbe8a4b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe415b0a0 con 0x7fbbe4159ab0 2024-01-11T07:08:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.914+0000 7fbbe8a4b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe40c9860 con 0x7fbbe4069ea0 2024-01-11T07:08:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.914+0000 7fbbe8a4b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe40c8e80 con 0x7fbbe406a270 2024-01-11T07:08:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe1d9b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe406a270 0x7fbbe406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:09.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe259c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbe4069ea0 0x7fbbe406db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe2d9d700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbbe4159ab0 0x7fbbe4159ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:38816/0 (socket says 172.21.15.7:38816) 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe2d9d700 1 -- 172.21.15.7:0/3420821752 learned_addr learned my addr 172.21.15.7:0/3420821752 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe1d9b700 1 -- 172.21.15.7:0/3420821752 >> v2:172.21.15.113:3300/0 conn(0x7fbbe4069ea0 msgr2=0x7fbbe406db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe1d9b700 1 --2- 172.21.15.7:0/3420821752 >> v2:172.21.15.113:3300/0 conn(0x7fbbe4069ea0 0x7fbbe406db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe1d9b700 1 -- 172.21.15.7:0/3420821752 >> v1:172.21.15.113:6789/0 conn(0x7fbbe4159ab0 legacy=0x7fbbe4159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe1d9b700 1 -- 172.21.15.7:0/3420821752 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe406e6f0 con 0x7fbbe406a270 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe159a700 1 -- 172.21.15.7:0/3420821752 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819941436 0 0) 0x7fbbe415b0a0 con 0x7fbbe4159ab0 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe259c700 1 --2- 172.21.15.7:0/3420821752 >> v2:172.21.15.113:3300/0 conn(0x7fbbe4069ea0 0x7fbbe406db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe1d9b700 1 --2- 172.21.15.7:0/3420821752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe406a270 0x7fbbe406e090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800bb90 tx=0x7fbbd800b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2a9012ae076c001 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.915+0000 7fbbe159a700 1 -- 172.21.15.7:0/3420821752 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800db40 con 0x7fbbe406a270 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe159a700 1 -- 172.21.15.7:0/3420821752 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800dca0 con 0x7fbbe406a270 2024-01-11T07:08:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe159a700 1 -- 172.21.15.7:0/3420821752 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80196b0 con 0x7fbbe406a270 2024-01-11T07:08:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3420821752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe406a270 msgr2=0x7fbbe406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3420821752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe406a270 0x7fbbe406e090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800bb90 tx=0x7fbbd800b430 comp rx=0 tx=0).stop 2024-01-11T07:08:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3420821752 shutdown_connections 2024-01-11T07:08:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3420821752 >> v2:172.21.15.113:3300/0 conn(0x7fbbe4069ea0 0x7fbbe406db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3420821752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe406a270 0x7fbbe406e090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3420821752 >> 172.21.15.7:0/3420821752 conn(0x7fbbe40680a0 msgr2=0x7fbbe40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3420821752 shutdown_connections 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.916+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3420821752 wait complete. 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.917+0000 7fbbe8a4b700 1 Processor -- start 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.917+0000 7fbbe8a4b700 1 -- start start 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe8a4b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbe4069ea0 0x7fbbe4149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe8a4b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe406a270 0x7fbbe40c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe8a4b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe4159ab0 0x7fbbe4166780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe8a4b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe414a2e0 con 0x7fbbe406a270 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe8a4b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe41662f0 con 0x7fbbe4069ea0 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe8a4b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe40cab30 con 0x7fbbe4159ab0 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe406a270 0x7fbbe40c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe406a270 0x7fbbe40c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42312/0 (socket says 172.21.15.7:42312) 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe1d9b700 1 -- 172.21.15.7:0/3394156761 learned_addr learned my addr 172.21.15.7:0/3394156761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe259c700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbe4069ea0 0x7fbbe4149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.918+0000 7fbbe2d9d700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe4159ab0 0x7fbbe4166780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbe1d9b700 1 -- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe4159ab0 msgr2=0x7fbbe4166780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbe1d9b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe4159ab0 0x7fbbe4166780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbe1d9b700 1 -- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbe4069ea0 msgr2=0x7fbbe4149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbe1d9b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbe4069ea0 0x7fbbe4149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbe1d9b700 1 -- 172.21.15.7:0/3394156761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd4009420 con 0x7fbbe406a270 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbe1d9b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe406a270 0x7fbbe40c75c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800b2c0 tx=0x7fbbd801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbd27fc700 1 -- 172.21.15.7:0/3394156761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8010680 con 0x7fbbe406a270 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbd27fc700 1 -- 172.21.15.7:0/3394156761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd80107e0 con 0x7fbbe406a270 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd800b6d0 con 0x7fbbe406a270 2024-01-11T07:08:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbd27fc700 1 -- 172.21.15.7:0/3394156761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8010ba0 con 0x7fbbe406a270 2024-01-11T07:08:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.919+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe40cb0f0 con 0x7fbbe406a270 2024-01-11T07:08:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.920+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe404c3c0 con 0x7fbbe406a270 2024-01-11T07:08:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.921+0000 7fbbd27fc700 1 -- 172.21.15.7:0/3394156761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbbd8023070 con 0x7fbbe406a270 2024-01-11T07:08:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.921+0000 7fbbd27fc700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbc4066f30 0x7fbbc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.921+0000 7fbbd27fc700 1 -- 172.21.15.7:0/3394156761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fbbd8014070 con 0x7fbbe406a270 2024-01-11T07:08:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.921+0000 7fbbe259c700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbc4066f30 0x7fbbc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.922+0000 7fbbe259c700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbc4066f30 0x7fbbc40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009f40 tx=0x7fbbd40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:09.926+0000 7fbbd27fc700 1 -- 172.21.15.7:0/3394156761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbd8055760 con 0x7fbbe406a270 2024-01-11T07:08:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.130+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbbe415a770 con 0x7fbbe406a270 2024-01-11T07:08:10.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.719+0000 7fbbd27fc700 1 -- 172.21.15.7:0/3394156761 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbbd8058d80 con 0x7fbbe406a270 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbc4066f30 msgr2=0x7fbbc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbc4066f30 0x7fbbc40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009f40 tx=0x7fbbd40091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe406a270 msgr2=0x7fbbe40c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe406a270 0x7fbbe40c75c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800b2c0 tx=0x7fbbd801b000 comp rx=0 tx=0).stop 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 shutdown_connections 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbc4066f30 0x7fbbc40693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbbe4159ab0 0x7fbbe4166780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbe4069ea0 0x7fbbe4149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 --2- 172.21.15.7:0/3394156761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbe406a270 0x7fbbe40c75c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 >> 172.21.15.7:0/3394156761 conn(0x7fbbe40680a0 msgr2=0x7fbbe4063230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 shutdown_connections 2024-01-11T07:08:10.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.722+0000 7fbbe8a4b700 1 -- 172.21.15.7:0/3394156761 wait complete. 2024-01-11T07:08:10.723 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:08:10.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:08:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.877+0000 7fc840ba8700 1 Processor -- start 2024-01-11T07:08:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.877+0000 7fc840ba8700 1 -- start start 2024-01-11T07:08:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc840ba8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc83c0c8fd0 0x7fc83c0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc840ba8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc83c0c93a0 0x7fc83c0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc840ba8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83c06a820 con 0x7fc83c0cac60 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc840ba8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83c0c86e0 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc840ba8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83c0ccd50 con 0x7fc83c0c93a0 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc83a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc83c0c8fd0 0x7fc83c0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc83a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc83c0c8fd0 0x7fc83c0cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42318/0 (socket says 172.21.15.7:42318) 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc83a59c700 1 -- 172.21.15.7:0/121298012 learned_addr learned my addr 172.21.15.7:0/121298012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.878+0000 7fc839d9b700 1 --2- 172.21.15.7:0/121298012 >> v2:172.21.15.113:3300/0 conn(0x7fc83c0c93a0 0x7fc83c0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc83a59c700 1 -- 172.21.15.7:0/121298012 >> v2:172.21.15.113:3300/0 conn(0x7fc83c0c93a0 msgr2=0x7fc83c0aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc83a59c700 1 --2- 172.21.15.7:0/121298012 >> v2:172.21.15.113:3300/0 conn(0x7fc83c0c93a0 0x7fc83c0aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc83a59c700 1 -- 172.21.15.7:0/121298012 >> v1:172.21.15.113:6789/0 conn(0x7fc83c0cac60 legacy=0x7fc83c1593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc83a59c700 1 -- 172.21.15.7:0/121298012 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c06a1d0 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc839d9b700 1 --2- 172.21.15.7:0/121298012 >> v2:172.21.15.113:3300/0 conn(0x7fc83c0c93a0 0x7fc83c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc83a59c700 1 --2- 172.21.15.7:0/121298012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0cc750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0685a0 tx=0x7fc824009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93d40abe3d08783 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc83959a700 1 -- 172.21.15.7:0/121298012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82400b820 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc83959a700 1 -- 172.21.15.7:0/121298012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82400b980 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc83959a700 1 -- 172.21.15.7:0/121298012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82400bd20 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc840ba8700 1 -- 172.21.15.7:0/121298012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 msgr2=0x7fc83c0cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.879+0000 7fc840ba8700 1 --2- 172.21.15.7:0/121298012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0cc750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0685a0 tx=0x7fc824009250 comp rx=0 tx=0).stop 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.880+0000 7fc840ba8700 1 -- 172.21.15.7:0/121298012 shutdown_connections 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.880+0000 7fc840ba8700 1 --2- 172.21.15.7:0/121298012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0cc750 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.880+0000 7fc840ba8700 1 --2- 172.21.15.7:0/121298012 >> v2:172.21.15.113:3300/0 conn(0x7fc83c0c93a0 0x7fc83c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.880+0000 7fc840ba8700 1 -- 172.21.15.7:0/121298012 >> 172.21.15.7:0/121298012 conn(0x7fc83c0680e0 msgr2=0x7fc83c0cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.880+0000 7fc840ba8700 1 -- 172.21.15.7:0/121298012 shutdown_connections 2024-01-11T07:08:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.880+0000 7fc840ba8700 1 -- 172.21.15.7:0/121298012 wait complete. 2024-01-11T07:08:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.881+0000 7fc840ba8700 1 Processor -- start 2024-01-11T07:08:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.881+0000 7fc840ba8700 1 -- start start 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc840ba8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc840ba8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc83c0c93a0 0x7fc83c0b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc840ba8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc83c0cac60 0x7fc83c0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc840ba8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c16d590 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc840ba8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c16e110 con 0x7fc83c0c93a0 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc840ba8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c16ae90 con 0x7fc83c0cac60 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42324/0 (socket says 172.21.15.7:42324) 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 -- 172.21.15.7:0/1943999002 learned_addr learned my addr 172.21.15.7:0/1943999002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc839d9b700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc83c0c93a0 0x7fc83c0b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83ad9d700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc83c0cac60 0x7fc83c0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 -- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc83c0cac60 msgr2=0x7fc83c0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc83c0cac60 0x7fc83c0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 -- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc83c0c93a0 msgr2=0x7fc83c0b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc83c0c93a0 0x7fc83c0b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 -- 172.21.15.7:0/1943999002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240094f0 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.882+0000 7fc83a59c700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0c5c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc8240030c0 tx=0x7fc824016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.883+0000 7fc8327fc700 1 -- 172.21.15.7:0/1943999002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824011d00 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.883+0000 7fc8327fc700 1 -- 172.21.15.7:0/1943999002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824020070 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.883+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83c16d7e0 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.883+0000 7fc8327fc700 1 -- 172.21.15.7:0/1943999002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82401b6a0 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.883+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83c16dda0 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.884+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83c04c3c0 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.884+0000 7fc8327fc700 1 -- 172.21.15.7:0/1943999002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc824022030 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.885+0000 7fc8327fc700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc81c066f30 0x7fc81c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:10.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.885+0000 7fc8327fc700 1 -- 172.21.15.7:0/1943999002 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fc824086210 con 0x7fc83c0c8fd0 2024-01-11T07:08:10.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.885+0000 7fc839d9b700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc81c066f30 0x7fc81c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:10.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.886+0000 7fc839d9b700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc81c066f30 0x7fc81c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc83c167ae0 tx=0x7fc82c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:10.889+0000 7fc8327fc700 1 -- 172.21.15.7:0/1943999002 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8240535e0 con 0x7fc83c0c8fd0 2024-01-11T07:08:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.093+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc83c04c3c0 con 0x7fc83c0c8fd0 2024-01-11T07:08:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.723+0000 7fc8327fc700 1 -- 172.21.15.7:0/1943999002 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc824011e60 con 0x7fc83c0c8fd0 2024-01-11T07:08:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc81c066f30 msgr2=0x7fc81c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc81c066f30 0x7fc81c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc83c167ae0 tx=0x7fc82c008040 comp rx=0 tx=0).stop 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 msgr2=0x7fc83c0c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0c5c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc8240030c0 tx=0x7fc824016000 comp rx=0 tx=0).stop 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 shutdown_connections 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc81c066f30 0x7fc81c0693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc83c0cac60 0x7fc83c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc83c0c8fd0 0x7fc83c0c5c90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 --2- 172.21.15.7:0/1943999002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc83c0c93a0 0x7fc83c0b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 >> 172.21.15.7:0/1943999002 conn(0x7fc83c0680e0 msgr2=0x7fc83c15b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 shutdown_connections 2024-01-11T07:08:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.728+0000 7fc840ba8700 1 -- 172.21.15.7:0/1943999002 wait complete. 2024-01-11T07:08:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:08:11.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:08:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.887+0000 7f0b0d990700 1 Processor -- start 2024-01-11T07:08:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.887+0000 7f0b0d990700 1 -- start start 2024-01-11T07:08:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b0d990700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b08063480 0x7f0b080c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b0d990700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b08140540 0x7f0b080c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b0d990700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b080c8b50 con 0x7f0b08063850 2024-01-11T07:08:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b0d990700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b080cd320 con 0x7f0b08140540 2024-01-11T07:08:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b0d990700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b080cb550 con 0x7f0b08063480 2024-01-11T07:08:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b08140540 0x7f0b080c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b067fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0b08063850 0x7f0b0813fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34114/0 (socket says 172.21.15.7:34114) 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b067fc700 1 -- 172.21.15.7:0/2402586688 learned_addr learned my addr 172.21.15.7:0/2402586688 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b06ffd700 1 --2- 172.21.15.7:0/2402586688 >> v2:172.21.15.113:3300/0 conn(0x7f0b08063480 0x7f0b080c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b077fe700 1 -- 172.21.15.7:0/2402586688 >> v2:172.21.15.113:3300/0 conn(0x7f0b08063480 msgr2=0x7f0b080c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b077fe700 1 --2- 172.21.15.7:0/2402586688 >> v2:172.21.15.113:3300/0 conn(0x7f0b08063480 0x7f0b080c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b077fe700 1 -- 172.21.15.7:0/2402586688 >> v1:172.21.15.7:6789/0 conn(0x7f0b08063850 legacy=0x7f0b0813fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b077fe700 1 -- 172.21.15.7:0/2402586688 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b080cb830 con 0x7f0b08140540 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.888+0000 7f0b05ffb700 1 -- 172.21.15.7:0/2402586688 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210686604 0 0) 0x7f0b080c8b50 con 0x7f0b08063850 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b077fe700 1 --2- 172.21.15.7:0/2402586688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08140540 0x7f0b080c8460 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0b0806d5d0 tx=0x7f0b00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f5a98f0ccaf16e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b05ffb700 1 -- 172.21.15.7:0/2402586688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000e070 con 0x7f0b08140540 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b05ffb700 1 -- 172.21.15.7:0/2402586688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0000bb60 con 0x7f0b08140540 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b05ffb700 1 -- 172.21.15.7:0/2402586688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00012430 con 0x7f0b08140540 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b0d990700 1 -- 172.21.15.7:0/2402586688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08140540 msgr2=0x7f0b080c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2402586688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08140540 0x7f0b080c8460 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0b0806d5d0 tx=0x7f0b00009780 comp rx=0 tx=0).stop 2024-01-11T07:08:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b0d990700 1 -- 172.21.15.7:0/2402586688 shutdown_connections 2024-01-11T07:08:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2402586688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08140540 0x7f0b080c8460 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2402586688 >> v2:172.21.15.113:3300/0 conn(0x7f0b08063480 0x7f0b080c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b0d990700 1 -- 172.21.15.7:0/2402586688 >> 172.21.15.7:0/2402586688 conn(0x7f0b080cd8e0 msgr2=0x7f0b080cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b0d990700 1 -- 172.21.15.7:0/2402586688 shutdown_connections 2024-01-11T07:08:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.889+0000 7f0b0d990700 1 -- 172.21.15.7:0/2402586688 wait complete. 2024-01-11T07:08:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.891+0000 7f0b0d990700 1 Processor -- start 2024-01-11T07:08:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.891+0000 7f0b0d990700 1 -- start start 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.891+0000 7f0b0d990700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b08063480 0x7f0b080aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b0d990700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08063850 0x7f0b080ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b0d990700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08140540 0x7f0b0813eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b0d990700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080acbf0 con 0x7f0b08063850 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b0d990700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08169c40 con 0x7f0b08063480 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b0d990700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0816aa90 con 0x7f0b08140540 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08063850 0x7f0b080ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08063850 0x7f0b080ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42352/0 (socket says 172.21.15.7:42352) 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 -- 172.21.15.7:0/2477461560 learned_addr learned my addr 172.21.15.7:0/2477461560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b06ffd700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b08063480 0x7f0b080aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b077fe700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08140540 0x7f0b0813eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 -- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08140540 msgr2=0x7f0b0813eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08140540 0x7f0b0813eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 -- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b08063480 msgr2=0x7f0b080aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b08063480 0x7f0b080aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 -- 172.21.15.7:0/2477461560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00009a20 con 0x7f0b08063850 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b067fc700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08063850 0x7f0b080ac6b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c810 tx=0x7f0af8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0afeffd700 1 -- 172.21.15.7:0/2477461560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8005b30 con 0x7f0b08063850 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0afeffd700 1 -- 172.21.15.7:0/2477461560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af8005c90 con 0x7f0b08063850 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b08169e80 con 0x7f0b08063850 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.892+0000 7f0afeffd700 1 -- 172.21.15.7:0/2477461560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8007300 con 0x7f0b08063850 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.893+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0816a440 con 0x7f0b08063850 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.894+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b08064a40 con 0x7f0b08063850 2024-01-11T07:08:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.894+0000 7f0afeffd700 1 -- 172.21.15.7:0/2477461560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0af801b3b0 con 0x7f0b08063850 2024-01-11T07:08:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.894+0000 7f0afeffd700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0ae8066f30 0x7f0ae80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.895+0000 7f0afeffd700 1 -- 172.21.15.7:0/2477461560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0af8083740 con 0x7f0b08063850 2024-01-11T07:08:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.895+0000 7f0b06ffd700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0ae8066f30 0x7f0ae80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.895+0000 7f0b06ffd700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0ae8066f30 0x7f0ae80693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009d10 tx=0x7f0af00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:11.899+0000 7f0afeffd700 1 -- 172.21.15.7:0/2477461560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0af8050cb0 con 0x7f0b08063850 2024-01-11T07:08:12.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.103+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0b0804c3c0 con 0x7f0b08063850 2024-01-11T07:08:12.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.107+0000 7f0afeffd700 1 -- 172.21.15.7:0/2477461560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0af80542d0 con 0x7f0b08063850 2024-01-11T07:08:12.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0ae8066f30 msgr2=0x7f0ae80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0ae8066f30 0x7f0ae80693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009d10 tx=0x7f0af00091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08063850 msgr2=0x7f0b080ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08063850 0x7f0b080ac6b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c810 tx=0x7f0af8007df0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 shutdown_connections 2024-01-11T07:08:12.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0ae8066f30 0x7f0ae80693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b08140540 0x7f0b0813eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b08063480 0x7f0b080aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 --2- 172.21.15.7:0/2477461560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b08063850 0x7f0b080ac6b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 >> 172.21.15.7:0/2477461560 conn(0x7f0b080cd8e0 msgr2=0x7f0b080c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:12.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 shutdown_connections 2024-01-11T07:08:12.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.109+0000 7f0b0d990700 1 -- 172.21.15.7:0/2477461560 wait complete. 2024-01-11T07:08:12.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-01-11T07:08:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.254+0000 7f1d9bb07700 1 Processor -- start 2024-01-11T07:08:12.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.255+0000 7f1d9bb07700 1 -- start start 2024-01-11T07:08:12.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.255+0000 7f1d9bb07700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d94063230 0x7f1d94063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d9bb07700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d940ab100 con 0x7f1d940ac5b0 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d9bb07700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9415d880 con 0x7f1d9415a280 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d9bb07700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9406c5b0 con 0x7f1d94063230 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d998a3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d94063230 0x7f1d94063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d9a0a4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1d940ac5b0 0x7f1d940ac9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34122/0 (socket says 172.21.15.7:34122) 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d9a0a4700 1 -- 172.21.15.7:0/2037322138 learned_addr learned my addr 172.21.15.7:0/2037322138 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d998a3700 1 -- 172.21.15.7:0/2037322138 >> v1:172.21.15.113:6789/0 conn(0x7f1d9415a280 legacy=0x7f1d940aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d998a3700 1 -- 172.21.15.7:0/2037322138 >> v1:172.21.15.7:6789/0 conn(0x7f1d940ac5b0 legacy=0x7f1d940ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d998a3700 1 -- 172.21.15.7:0/2037322138 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94063b40 con 0x7f1d94063230 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d988a1700 1 -- 172.21.15.7:0/2037322138 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000529029 0 0) 0x7f1d940ab100 con 0x7f1d940ac5b0 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d998a3700 1 --2- 172.21.15.7:0/2037322138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d94063230 0x7f1d94063600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1d900098e0 tx=0x7f1d900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adf32b11b75e970 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d988a1700 1 -- 172.21.15.7:0/2037322138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90009e80 con 0x7f1d94063230 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.256+0000 7f1d988a1700 1 -- 172.21.15.7:0/2037322138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9000b860 con 0x7f1d94063230 2024-01-11T07:08:12.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.257+0000 7f1d988a1700 1 -- 172.21.15.7:0/2037322138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9000bbe0 con 0x7f1d94063230 2024-01-11T07:08:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.257+0000 7f1d9bb07700 1 -- 172.21.15.7:0/2037322138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d94063230 msgr2=0x7f1d94063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.257+0000 7f1d9bb07700 1 --2- 172.21.15.7:0/2037322138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d94063230 0x7f1d94063600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1d900098e0 tx=0x7f1d900091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.257+0000 7f1d9bb07700 1 -- 172.21.15.7:0/2037322138 shutdown_connections 2024-01-11T07:08:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.257+0000 7f1d9bb07700 1 --2- 172.21.15.7:0/2037322138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d94063230 0x7f1d94063600 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.257+0000 7f1d9bb07700 1 -- 172.21.15.7:0/2037322138 >> 172.21.15.7:0/2037322138 conn(0x7f1d940680e0 msgr2=0x7f1d940cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.257+0000 7f1d9bb07700 1 -- 172.21.15.7:0/2037322138 shutdown_connections 2024-01-11T07:08:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.257+0000 7f1d9bb07700 1 -- 172.21.15.7:0/2037322138 wait complete. 2024-01-11T07:08:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.259+0000 7f1d9bb07700 1 Processor -- start 2024-01-11T07:08:12.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.259+0000 7f1d9bb07700 1 -- start start 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9bb07700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94063230 0x7f1d940adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9bb07700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d940ac5b0 0x7f1d940ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9bb07700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415a280 0x7f1d940cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9bb07700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415e9a0 con 0x7f1d9415a280 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9bb07700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415ec80 con 0x7f1d94063230 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9bb07700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415ef60 con 0x7f1d940ac5b0 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9a0a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415a280 0x7f1d940cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9a0a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415a280 0x7f1d940cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42380/0 (socket says 172.21.15.7:42380) 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9a0a4700 1 -- 172.21.15.7:0/247776043 learned_addr learned my addr 172.21.15.7:0/247776043 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d998a3700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94063230 0x7f1d940adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d990a2700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d940ac5b0 0x7f1d940ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9a0a4700 1 -- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d940ac5b0 msgr2=0x7f1d940ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9a0a4700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d940ac5b0 0x7f1d940ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9a0a4700 1 -- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94063230 msgr2=0x7f1d940adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9a0a4700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94063230 0x7f1d940adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.260+0000 7f1d9a0a4700 1 -- 172.21.15.7:0/247776043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90009420 con 0x7f1d9415a280 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.261+0000 7f1d9a0a4700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415a280 0x7f1d940cd6c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00cc90 tx=0x7f1d8c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.261+0000 7f1d89ffb700 1 -- 172.21.15.7:0/247776043 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00cea0 con 0x7f1d9415a280 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.261+0000 7f1d89ffb700 1 -- 172.21.15.7:0/247776043 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00ac00 con 0x7f1d9415a280 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.261+0000 7f1d89ffb700 1 -- 172.21.15.7:0/247776043 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c0070f0 con 0x7f1d9415a280 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.261+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9415f240 con 0x7f1d9415a280 2024-01-11T07:08:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.261+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d940cdc00 con 0x7f1d9415a280 2024-01-11T07:08:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.262+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9415f6e0 con 0x7f1d9415a280 2024-01-11T07:08:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.262+0000 7f1d89ffb700 1 -- 172.21.15.7:0/247776043 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1d8c007460 con 0x7f1d9415a280 2024-01-11T07:08:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.263+0000 7f1d89ffb700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d78066f30 0x7f1d780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.263+0000 7f1d89ffb700 1 -- 172.21.15.7:0/247776043 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1d8c0847e0 con 0x7f1d9415a280 2024-01-11T07:08:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.263+0000 7f1d998a3700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d78066f30 0x7f1d780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.263+0000 7f1d998a3700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d78066f30 0x7f1d780693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1d900093f0 tx=0x7f1d90017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.267+0000 7f1d89ffb700 1 -- 172.21.15.7:0/247776043 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d8c051d30 con 0x7f1d9415a280 2024-01-11T07:08:12.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.560+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f1d9404c3c0 con 0x7f1d9415a280 2024-01-11T07:08:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.725+0000 7f1d89ffb700 1 -- 172.21.15.7:0/247776043 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v86) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1d8c055350 con 0x7f1d9415a280 2024-01-11T07:08:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d78066f30 msgr2=0x7f1d780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d78066f30 0x7f1d780693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1d900093f0 tx=0x7f1d90017000 comp rx=0 tx=0).stop 2024-01-11T07:08:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415a280 msgr2=0x7f1d940cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415a280 0x7f1d940cd6c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00cc90 tx=0x7f1d8c001a80 comp rx=0 tx=0).stop 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 shutdown_connections 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d78066f30 0x7f1d780693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415a280 0x7f1d940cd6c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94063230 0x7f1d940adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.727+0000 7f1d9bb07700 1 --2- 172.21.15.7:0/247776043 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d940ac5b0 0x7f1d940ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.728+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 >> 172.21.15.7:0/247776043 conn(0x7f1d940680e0 msgr2=0x7f1d940cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.728+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 shutdown_connections 2024-01-11T07:08:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.728+0000 7f1d9bb07700 1 -- 172.21.15.7:0/247776043 wait complete. 2024-01-11T07:08:12.742 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-01-11T07:08:12.742 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2024-01-11T07:08:12.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-01-11T07:08:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.876+0000 7f124f74a700 1 Processor -- start 2024-01-11T07:08:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.876+0000 7f124f74a700 1 -- start start 2024-01-11T07:08:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124f74a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1248157da0 0x7f12480cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124f74a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12480ce220 0x7f12480ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124f74a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124806d270 con 0x7f12481579d0 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124f74a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12480ce6a0 con 0x7f12480ce220 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124f74a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12480cda40 con 0x7f1248157da0 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124dce7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12480ce220 0x7f12480ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124d4e6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f12481579d0 0x7f124815c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34138/0 (socket says 172.21.15.7:34138) 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124d4e6700 1 -- 172.21.15.7:0/3823442260 learned_addr learned my addr 172.21.15.7:0/3823442260 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.877+0000 7f124cce5700 1 --2- 172.21.15.7:0/3823442260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1248157da0 0x7f12480cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f124dce7700 1 -- 172.21.15.7:0/3823442260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1248157da0 msgr2=0x7f12480cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f124dce7700 1 --2- 172.21.15.7:0/3823442260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1248157da0 0x7f12480cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f124dce7700 1 -- 172.21.15.7:0/3823442260 >> v1:172.21.15.7:6789/0 conn(0x7f12481579d0 legacy=0x7f124815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f124dce7700 1 -- 172.21.15.7:0/3823442260 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480cd5b0 con 0x7f12480ce220 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f123bfff700 1 -- 172.21.15.7:0/3823442260 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121284336 0 0) 0x7f124806d270 con 0x7f12481579d0 2024-01-11T07:08:12.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f124dce7700 1 --2- 172.21.15.7:0/3823442260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12480ce220 0x7f12480ca500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1240009e60 tx=0x7f1240009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b9dad7e0b77e1b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f123bfff700 1 -- 172.21.15.7:0/3823442260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124000e070 con 0x7f12480ce220 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f123bfff700 1 -- 172.21.15.7:0/3823442260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124000bb60 con 0x7f12480ce220 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f123bfff700 1 -- 172.21.15.7:0/3823442260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240012430 con 0x7f12480ce220 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f124f74a700 1 -- 172.21.15.7:0/3823442260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12480ce220 msgr2=0x7f12480ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.878+0000 7f124f74a700 1 --2- 172.21.15.7:0/3823442260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12480ce220 0x7f12480ca500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1240009e60 tx=0x7f1240009780 comp rx=0 tx=0).stop 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.879+0000 7f124f74a700 1 -- 172.21.15.7:0/3823442260 shutdown_connections 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.879+0000 7f124f74a700 1 --2- 172.21.15.7:0/3823442260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12480ce220 0x7f12480ca500 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.879+0000 7f124f74a700 1 --2- 172.21.15.7:0/3823442260 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1248157da0 0x7f12480cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.879+0000 7f124f74a700 1 -- 172.21.15.7:0/3823442260 >> 172.21.15.7:0/3823442260 conn(0x7f124806ae00 msgr2=0x7f124806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.879+0000 7f124f74a700 1 -- 172.21.15.7:0/3823442260 shutdown_connections 2024-01-11T07:08:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.879+0000 7f124f74a700 1 -- 172.21.15.7:0/3823442260 wait complete. 2024-01-11T07:08:12.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.880+0000 7f124f74a700 1 Processor -- start 2024-01-11T07:08:12.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.880+0000 7f124f74a700 1 -- start start 2024-01-11T07:08:12.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124f74a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480ce220 0x7f1248158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124f74a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12481579d0 0x7f12480b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124f74a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1248157da0 0x7f1248167430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124f74a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12480b2ae0 con 0x7f1248157da0 2024-01-11T07:08:12.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124f74a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12481591e0 con 0x7f12480ce220 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124f74a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12480b4250 con 0x7f12481579d0 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1248157da0 0x7f1248167430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1248157da0 0x7f1248167430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42412/0 (socket says 172.21.15.7:42412) 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 -- 172.21.15.7:0/780804759 learned_addr learned my addr 172.21.15.7:0/780804759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124d4e6700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480ce220 0x7f1248158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124cce5700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12481579d0 0x7f12480b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 -- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12481579d0 msgr2=0x7f12480b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12481579d0 0x7f12480b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 -- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480ce220 msgr2=0x7f1248158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480ce220 0x7f1248158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 -- 172.21.15.7:0/780804759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1240009a20 con 0x7f1248157da0 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.881+0000 7f124dce7700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1248157da0 0x7f1248167430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f124806e9f0 tx=0x7f1240019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:12.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.882+0000 7f12397fa700 1 -- 172.21.15.7:0/780804759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240027470 con 0x7f1248157da0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.882+0000 7f12397fa700 1 -- 172.21.15.7:0/780804759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1240027610 con 0x7f1248157da0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.882+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1248167970 con 0x7f1248157da0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.882+0000 7f12397fa700 1 -- 172.21.15.7:0/780804759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12400279d0 con 0x7f1248157da0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.882+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12480b3920 con 0x7f1248157da0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.883+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124804c3c0 con 0x7f1248157da0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.883+0000 7f12397fa700 1 -- 172.21.15.7:0/780804759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1240024070 con 0x7f1248157da0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.884+0000 7f12397fa700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f122c066f30 0x7f122c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.884+0000 7f12397fa700 1 -- 172.21.15.7:0/780804759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1240086cb0 con 0x7f1248157da0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.884+0000 7f124d4e6700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f122c066f30 0x7f122c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:12.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.884+0000 7f124d4e6700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f122c066f30 0x7f122c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f12440011e0 tx=0x7f1244009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:12.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:12.888+0000 7f12397fa700 1 -- 172.21.15.7:0/780804759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1240054220 con 0x7f1248157da0 2024-01-11T07:08:13.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.176+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f12480cc9a0 con 0x7f1248157da0 2024-01-11T07:08:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.727+0000 7f12397fa700 1 -- 172.21.15.7:0/780804759 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v87) v1 ==== 145+0+0 (secure 0 0 0) 0x7f124002c0d0 con 0x7f1248157da0 2024-01-11T07:08:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f122c066f30 msgr2=0x7f122c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f122c066f30 0x7f122c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f12440011e0 tx=0x7f1244009040 comp rx=0 tx=0).stop 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1248157da0 msgr2=0x7f1248167430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1248157da0 0x7f1248167430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f124806e9f0 tx=0x7f1240019000 comp rx=0 tx=0).stop 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 shutdown_connections 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f122c066f30 0x7f122c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1248157da0 0x7f1248167430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12480ce220 0x7f1248158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 --2- 172.21.15.7:0/780804759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12481579d0 0x7f12480b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 >> 172.21.15.7:0/780804759 conn(0x7f124806ae00 msgr2=0x7f12480ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 shutdown_connections 2024-01-11T07:08:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:13.730+0000 7f124f74a700 1 -- 172.21.15.7:0/780804759 wait complete. 2024-01-11T07:08:13.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-11T07:08:13.901 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-11T07:08:13.901 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-11T07:08:13.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:08:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.037+0000 7fa8467f0700 1 Processor -- start 2024-01-11T07:08:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.037+0000 7fa8467f0700 1 -- start start 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa8467f0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa84006a9c0 0x7fa84015af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa8467f0700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa840063230 0x7fa840063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa8467f0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84015b4b0 con 0x7fa8401579a0 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa8467f0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84006bb60 con 0x7fa84006a9c0 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa8467f0700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84006e8b0 con 0x7fa840063230 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa83f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa84006a9c0 0x7fa84015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa83ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa8401579a0 0x7fa84006bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34148/0 (socket says 172.21.15.7:34148) 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa83ffff700 1 -- 172.21.15.7:0/913799544 learned_addr learned my addr 172.21.15.7:0/913799544 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa844d8d700 1 --2- 172.21.15.7:0/913799544 >> v2:172.21.15.113:3300/0 conn(0x7fa840063230 0x7fa840063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa83f7fe700 1 -- 172.21.15.7:0/913799544 >> v2:172.21.15.113:3300/0 conn(0x7fa840063230 msgr2=0x7fa840063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa83f7fe700 1 --2- 172.21.15.7:0/913799544 >> v2:172.21.15.113:3300/0 conn(0x7fa840063230 0x7fa840063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa83f7fe700 1 -- 172.21.15.7:0/913799544 >> v1:172.21.15.7:6789/0 conn(0x7fa8401579a0 legacy=0x7fa84006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa83f7fe700 1 -- 172.21.15.7:0/913799544 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840063b40 con 0x7fa84006a9c0 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.038+0000 7fa83f7fe700 1 --2- 172.21.15.7:0/913799544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84006a9c0 0x7fa84015af70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa8400b05f0 tx=0x7fa8340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1523f71783d6cab server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa83effd700 1 -- 172.21.15.7:0/913799544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400b6b0 con 0x7fa84006a9c0 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa83effd700 1 -- 172.21.15.7:0/913799544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400b810 con 0x7fa84006a9c0 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa83effd700 1 -- 172.21.15.7:0/913799544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400bbb0 con 0x7fa84006a9c0 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa8467f0700 1 -- 172.21.15.7:0/913799544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84006a9c0 msgr2=0x7fa84015af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa8467f0700 1 --2- 172.21.15.7:0/913799544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84006a9c0 0x7fa84015af70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa8400b05f0 tx=0x7fa8340091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa8467f0700 1 -- 172.21.15.7:0/913799544 shutdown_connections 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa8467f0700 1 --2- 172.21.15.7:0/913799544 >> v2:172.21.15.113:3300/0 conn(0x7fa840063230 0x7fa840063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa8467f0700 1 --2- 172.21.15.7:0/913799544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa84006a9c0 0x7fa84015af70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa8467f0700 1 -- 172.21.15.7:0/913799544 >> 172.21.15.7:0/913799544 conn(0x7fa8400cd8f0 msgr2=0x7fa8400cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa8467f0700 1 -- 172.21.15.7:0/913799544 shutdown_connections 2024-01-11T07:08:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.039+0000 7fa8467f0700 1 -- 172.21.15.7:0/913799544 wait complete. 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.040+0000 7fa8467f0700 1 Processor -- start 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.041+0000 7fa8467f0700 1 -- start start 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.041+0000 7fa8467f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa840063230 0x7fa84015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.041+0000 7fa8467f0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa84006a9c0 0x7fa8400b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa8467f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8401579a0 0x7fa8400b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa8467f0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400b1760 con 0x7fa840063230 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa8467f0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa840153d40 con 0x7fa84006a9c0 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa8467f0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400aeda0 con 0x7fa8401579a0 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa840063230 0x7fa84015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa840063230 0x7fa84015a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42442/0 (socket says 172.21.15.7:42442) 2024-01-11T07:08:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 -- 172.21.15.7:0/2812647858 learned_addr learned my addr 172.21.15.7:0/2812647858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa844d8d700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8401579a0 0x7fa8400b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83f7fe700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa84006a9c0 0x7fa8400b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 -- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8401579a0 msgr2=0x7fa8400b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8401579a0 0x7fa8400b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 -- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa84006a9c0 msgr2=0x7fa8400b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa84006a9c0 0x7fa8400b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 -- 172.21.15.7:0/2812647858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834009420 con 0x7fa840063230 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83f7fe700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa84006a9c0 0x7fa8400b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa83ffff700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa840063230 0x7fa84015a420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c590 tx=0x7fa830001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa827fff700 1 -- 172.21.15.7:0/2812647858 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830005b30 con 0x7fa840063230 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa827fff700 1 -- 172.21.15.7:0/2812647858 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa830005cb0 con 0x7fa840063230 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa827fff700 1 -- 172.21.15.7:0/2812647858 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830007320 con 0x7fa840063230 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.042+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8400afbd0 con 0x7fa840063230 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.043+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8400ae470 con 0x7fa840063230 2024-01-11T07:08:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.043+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa840064a40 con 0x7fa840063230 2024-01-11T07:08:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.044+0000 7fa827fff700 1 -- 172.21.15.7:0/2812647858 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa830007690 con 0x7fa840063230 2024-01-11T07:08:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.044+0000 7fa827fff700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa82c066f30 0x7fa82c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.044+0000 7fa827fff700 1 -- 172.21.15.7:0/2812647858 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa830083480 con 0x7fa840063230 2024-01-11T07:08:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.045+0000 7fa83f7fe700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa82c066f30 0x7fa82c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.045+0000 7fa83f7fe700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa82c066f30 0x7fa82c0693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8340093f0 tx=0x7fa834017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.049+0000 7fa827fff700 1 -- 172.21.15.7:0/2812647858 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8300509f0 con 0x7fa840063230 2024-01-11T07:08:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.253+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa84015b4b0 con 0x7fa840063230 2024-01-11T07:08:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.256+0000 7fa827fff700 1 -- 172.21.15.7:0/2812647858 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa84015b4b0 con 0x7fa840063230 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa82c066f30 msgr2=0x7fa82c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa82c066f30 0x7fa82c0693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8340093f0 tx=0x7fa834017000 comp rx=0 tx=0).stop 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa840063230 msgr2=0x7fa84015a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa840063230 0x7fa84015a420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c590 tx=0x7fa830001020 comp rx=0 tx=0).stop 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 shutdown_connections 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa82c066f30 0x7fa82c0693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8401579a0 0x7fa8400b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa840063230 0x7fa84015a420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 --2- 172.21.15.7:0/2812647858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa84006a9c0 0x7fa8400b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 >> 172.21.15.7:0/2812647858 conn(0x7fa8400cd8f0 msgr2=0x7fa8400cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 shutdown_connections 2024-01-11T07:08:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.259+0000 7fa8467f0700 1 -- 172.21.15.7:0/2812647858 wait complete. 2024-01-11T07:08:14.272 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:08:14.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:08:14.311 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:08:14.362 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:08:14.362 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:08:14.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.500+0000 7f6d64d08700 1 Processor -- start 2024-01-11T07:08:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.500+0000 7f6d64d08700 1 -- start start 2024-01-11T07:08:14.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d64d08700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d601579b0 0x7f6d6015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d64d08700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6d60063940 0x7f6d60069c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d64d08700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d600aeab0 con 0x7f6d60157d80 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d64d08700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d600aed90 con 0x7f6d60063940 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d64d08700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600af070 con 0x7f6d601579b0 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d5dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6d60157d80 0x7f6d60063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34160/0 (socket says 172.21.15.7:34160) 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d5dd9b700 1 -- 172.21.15.7:0/2207407728 learned_addr learned my addr 172.21.15.7:0/2207407728 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d5ed9d700 1 --2- 172.21.15.7:0/2207407728 >> v2:172.21.15.113:3300/0 conn(0x7f6d60063940 0x7f6d60069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d5e59c700 1 --2- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d601579b0 0x7f6d6015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d5d59a700 1 -- 172.21.15.7:0/2207407728 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076581563 0 0) 0x7f6d600aeab0 con 0x7f6d60157d80 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d5d59a700 1 -- 172.21.15.7:0/2207407728 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d48003570 con 0x7f6d60157d80 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.501+0000 7f6d5d59a700 1 -- 172.21.15.7:0/2207407728 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 490959729 0 0) 0x7f6d48003570 con 0x7f6d60157d80 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5ed9d700 1 -- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d601579b0 msgr2=0x7f6d6015bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5ed9d700 1 --2- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d601579b0 0x7f6d6015bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5ed9d700 1 -- 172.21.15.7:0/2207407728 >> v1:172.21.15.7:6789/0 conn(0x7f6d60157d80 legacy=0x7f6d60063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5ed9d700 1 -- 172.21.15.7:0/2207407728 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d600af350 con 0x7f6d60063940 2024-01-11T07:08:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5e59c700 1 --2- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d601579b0 0x7f6d6015bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5ed9d700 1 --2- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d60063940 0x7f6d60069c40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009e60 tx=0x7f6d58009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e43763d050997a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5d59a700 1 -- 172.21.15.7:0/2207407728 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5800e070 con 0x7f6d60063940 2024-01-11T07:08:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5d59a700 1 -- 172.21.15.7:0/2207407728 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5800b7a0 con 0x7f6d60063940 2024-01-11T07:08:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.502+0000 7f6d5d59a700 1 -- 172.21.15.7:0/2207407728 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54001470 con 0x7f6d60063940 2024-01-11T07:08:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.503+0000 7f6d64d08700 1 -- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d60063940 msgr2=0x7f6d60069c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.503+0000 7f6d64d08700 1 --2- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d60063940 0x7f6d60069c40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009e60 tx=0x7f6d58009780 comp rx=0 tx=0).stop 2024-01-11T07:08:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.503+0000 7f6d64d08700 1 -- 172.21.15.7:0/2207407728 shutdown_connections 2024-01-11T07:08:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.503+0000 7f6d64d08700 1 --2- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d60063940 0x7f6d60069c40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.503+0000 7f6d64d08700 1 --2- 172.21.15.7:0/2207407728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d601579b0 0x7f6d6015bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.503+0000 7f6d64d08700 1 -- 172.21.15.7:0/2207407728 >> 172.21.15.7:0/2207407728 conn(0x7f6d600680e0 msgr2=0x7f6d600cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.503+0000 7f6d64d08700 1 -- 172.21.15.7:0/2207407728 shutdown_connections 2024-01-11T07:08:14.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.503+0000 7f6d64d08700 1 -- 172.21.15.7:0/2207407728 wait complete. 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.504+0000 7f6d64d08700 1 Processor -- start 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.504+0000 7f6d64d08700 1 -- start start 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d64d08700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d60063940 0x7f6d600c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d64d08700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d601579b0 0x7f6d6015a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d64d08700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d60157d80 0x7f6d60154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d64d08700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600c31b0 con 0x7f6d60063940 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d64d08700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600c1bc0 con 0x7f6d601579b0 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d64d08700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d600c1290 con 0x7f6d60157d80 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d60063940 0x7f6d600c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d60063940 0x7f6d600c33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42450/0 (socket says 172.21.15.7:42450) 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 -- 172.21.15.7:0/471561061 learned_addr learned my addr 172.21.15.7:0/471561061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5dd9b700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d601579b0 0x7f6d6015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5ed9d700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d60157d80 0x7f6d60154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 -- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d60157d80 msgr2=0x7f6d60154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d60157d80 0x7f6d60154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 -- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d601579b0 msgr2=0x7f6d6015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d601579b0 0x7f6d6015a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 -- 172.21.15.7:0/471561061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50009420 con 0x7f6d60063940 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5dd9b700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d601579b0 0x7f6d6015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.505+0000 7f6d5e59c700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d60063940 0x7f6d600c33e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000ec80 tx=0x7f6d500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.506+0000 7f6d4e7fc700 1 -- 172.21.15.7:0/471561061 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000c920 con 0x7f6d60063940 2024-01-11T07:08:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.506+0000 7f6d4e7fc700 1 -- 172.21.15.7:0/471561061 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5000ca80 con 0x7f6d60063940 2024-01-11T07:08:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.506+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d58009a20 con 0x7f6d60063940 2024-01-11T07:08:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.506+0000 7f6d4e7fc700 1 -- 172.21.15.7:0/471561061 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000ce00 con 0x7f6d60063940 2024-01-11T07:08:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.506+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d600c1850 con 0x7f6d60063940 2024-01-11T07:08:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.507+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6004c3c0 con 0x7f6d60063940 2024-01-11T07:08:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.507+0000 7f6d4e7fc700 1 -- 172.21.15.7:0/471561061 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6d50013680 con 0x7f6d60063940 2024-01-11T07:08:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.507+0000 7f6d4e7fc700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d40066f30 0x7f6d400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.508+0000 7f6d4e7fc700 1 -- 172.21.15.7:0/471561061 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6d50084610 con 0x7f6d60063940 2024-01-11T07:08:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.508+0000 7f6d5dd9b700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d40066f30 0x7f6d400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.508+0000 7f6d5dd9b700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d40066f30 0x7f6d400693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6d6016b020 tx=0x7f6d54001930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.512+0000 7f6d4e7fc700 1 -- 172.21.15.7:0/471561061 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d50051b80 con 0x7f6d60063940 2024-01-11T07:08:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.766+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d600af630 con 0x7f6d60063940 2024-01-11T07:08:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.766+0000 7f6d4e7fc700 1 -- 172.21.15.7:0/471561061 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f6d50017de0 con 0x7f6d60063940 2024-01-11T07:08:14.766 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:14.766 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1969624458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1969624458},{"type":"v1","addr":"172.21.15.7:6835","nonce":1969624458}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2712382722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2712382722},{"type":"v1","addr":"172.21.15.113:6836","nonce":2712382722}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3601043111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3601043111},{"type":"v1","addr":"172.21.15.113:6837","nonce":3601043111}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5232,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/1406022767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1406022767},{"type":"v1","addr":"172.21.15.7:6834","nonce":1406022767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2024-01-11T07:08:14.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d40066f30 msgr2=0x7f6d400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d40066f30 0x7f6d400693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6d6016b020 tx=0x7f6d54001930 comp rx=0 tx=0).stop 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d60063940 msgr2=0x7f6d600c33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d60063940 0x7f6d600c33e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000ec80 tx=0x7f6d500091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 shutdown_connections 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d40066f30 0x7f6d400693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d60157d80 0x7f6d60154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d60063940 0x7f6d600c33e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 --2- 172.21.15.7:0/471561061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d601579b0 0x7f6d6015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 >> 172.21.15.7:0/471561061 conn(0x7f6d600680e0 msgr2=0x7f6d60069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 shutdown_connections 2024-01-11T07:08:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.769+0000 7f6d64d08700 1 -- 172.21.15.7:0/471561061 wait complete. 2024-01-11T07:08:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-01-11T07:08:14.782 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:08:14.782 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:08:14.782 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:08:14.782 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:08:14.782 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:08:14.782 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:08:14.783 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:08:14.783 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:08:14.783 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:08:14.783 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:08:14.783 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:08:14.783 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:08:14.783 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:08:14.783 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:08:14.784 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:08:14.784 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:08:14.791 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:08:14.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:08:14.792 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:08:14.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:08:14.797 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:08:14.797 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:08:14.797 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:08:14.798 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:08:14.834 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:08:14.834 DEBUG:teuthology.parallel:result is None 2024-01-11T07:08:14.835 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:08:14.835 DEBUG:teuthology.parallel:result is None 2024-01-11T07:08:14.840 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:08:14.840 DEBUG:teuthology.parallel:result is None 2024-01-11T07:08:14.840 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:08:14.841 DEBUG:teuthology.parallel:result is None 2024-01-11T07:08:14.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:08:14.924 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:08:14.924 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:08:14.925 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:08:14.925 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:08:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.986+0000 7f519d556700 1 Processor -- start 2024-01-11T07:08:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.986+0000 7f519d556700 1 -- start start 2024-01-11T07:08:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.987+0000 7f519d556700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51980cd6d0 0x7f51980cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.987+0000 7f519d556700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f519815e450 0x7f5198159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.987+0000 7f519d556700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519815acf0 con 0x7f519815d8a0 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.987+0000 7f519d556700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519815afd0 con 0x7f519815e450 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.987+0000 7f519d556700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980ce980 con 0x7f51980cd6d0 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.987+0000 7f51977fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f519815e450 0x7f5198159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f51967fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f519815d8a0 0x7f519815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34196/0 (socket says 172.21.15.7:34196) 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f51977fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f519815e450 0x7f5198159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42490/0 (socket says 172.21.15.7:42490) 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f51967fc700 1 -- 172.21.15.7:0/133272306 learned_addr learned my addr 172.21.15.7:0/133272306 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f5196ffd700 1 --2- 172.21.15.7:0/133272306 >> v2:172.21.15.113:3300/0 conn(0x7f51980cd6d0 0x7f51980cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f51977fe700 1 -- 172.21.15.7:0/133272306 >> v2:172.21.15.113:3300/0 conn(0x7f51980cd6d0 msgr2=0x7f51980cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f51977fe700 1 --2- 172.21.15.7:0/133272306 >> v2:172.21.15.113:3300/0 conn(0x7f51980cd6d0 0x7f51980cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f51977fe700 1 -- 172.21.15.7:0/133272306 >> v1:172.21.15.7:6789/0 conn(0x7f519815d8a0 legacy=0x7f519815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f51977fe700 1 -- 172.21.15.7:0/133272306 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980cec60 con 0x7f519815e450 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f5195ffb700 1 -- 172.21.15.7:0/133272306 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775363296 0 0) 0x7f519815acf0 con 0x7f519815d8a0 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f51977fe700 1 --2- 172.21.15.7:0/133272306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519815e450 0x7f5198159600 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f51980677c0 tx=0x7f5190009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a78694aa4b8938 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f5195ffb700 1 -- 172.21.15.7:0/133272306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000e070 con 0x7f519815e450 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f5195ffb700 1 -- 172.21.15.7:0/133272306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519000bb60 con 0x7f519815e450 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.988+0000 7f5195ffb700 1 -- 172.21.15.7:0/133272306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190012430 con 0x7f519815e450 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.989+0000 7f519d556700 1 -- 172.21.15.7:0/133272306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519815e450 msgr2=0x7f5198159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.989+0000 7f519d556700 1 --2- 172.21.15.7:0/133272306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519815e450 0x7f5198159600 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f51980677c0 tx=0x7f5190009780 comp rx=0 tx=0).stop 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.989+0000 7f519d556700 1 -- 172.21.15.7:0/133272306 shutdown_connections 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.989+0000 7f519d556700 1 --2- 172.21.15.7:0/133272306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f519815e450 0x7f5198159600 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.989+0000 7f519d556700 1 --2- 172.21.15.7:0/133272306 >> v2:172.21.15.113:3300/0 conn(0x7f51980cd6d0 0x7f51980cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.989+0000 7f519d556700 1 -- 172.21.15.7:0/133272306 >> 172.21.15.7:0/133272306 conn(0x7f5198071120 msgr2=0x7f519806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.989+0000 7f519d556700 1 -- 172.21.15.7:0/133272306 shutdown_connections 2024-01-11T07:08:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.989+0000 7f519d556700 1 -- 172.21.15.7:0/133272306 wait complete. 2024-01-11T07:08:14.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.991+0000 7f519d556700 1 Processor -- start 2024-01-11T07:08:14.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.991+0000 7f519d556700 1 -- start start 2024-01-11T07:08:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f519d556700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51980cd6d0 0x7f519816da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f519d556700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f519815d8a0 0x7f5198146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f519d556700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f519815e450 0x7f5198146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f519d556700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981474c0 con 0x7f51980cd6d0 2024-01-11T07:08:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f519d556700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5198148310 con 0x7f519815e450 2024-01-11T07:08:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f519d556700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5198147700 con 0x7f519815d8a0 2024-01-11T07:08:14.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f5196ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51980cd6d0 0x7f519816da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f5196ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51980cd6d0 0x7f519816da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42496/0 (socket says 172.21.15.7:42496) 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f5196ffd700 1 -- 172.21.15.7:0/437215687 learned_addr learned my addr 172.21.15.7:0/437215687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.992+0000 7f51967fc700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f519815d8a0 0x7f5198146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f51977fe700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f519815e450 0x7f5198146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f5196ffd700 1 -- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f519815d8a0 msgr2=0x7f5198146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f5196ffd700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f519815d8a0 0x7f5198146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f5196ffd700 1 -- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f519815e450 msgr2=0x7f5198146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f5196ffd700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f519815e450 0x7f5198146f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f5196ffd700 1 -- 172.21.15.7:0/437215687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5190009a20 con 0x7f51980cd6d0 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f51967fc700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f519815d8a0 0x7f5198146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f51977fe700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f519815e450 0x7f5198146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f5196ffd700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51980cd6d0 0x7f519816da30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f518800ec80 tx=0x7f51880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f518effd700 1 -- 172.21.15.7:0/437215687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518800c9d0 con 0x7f51980cd6d0 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f518effd700 1 -- 172.21.15.7:0/437215687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518800cb30 con 0x7f51980cd6d0 2024-01-11T07:08:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51981479e0 con 0x7f51980cd6d0 2024-01-11T07:08:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f518effd700 1 -- 172.21.15.7:0/437215687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188013470 con 0x7f51980cd6d0 2024-01-11T07:08:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.993+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5198147fa0 con 0x7f51980cd6d0 2024-01-11T07:08:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.994+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519804c3c0 con 0x7f51980cd6d0 2024-01-11T07:08:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.995+0000 7f518effd700 1 -- 172.21.15.7:0/437215687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5188013820 con 0x7f51980cd6d0 2024-01-11T07:08:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.995+0000 7f518effd700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5178066f30 0x7f51780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.995+0000 7f518effd700 1 -- 172.21.15.7:0/437215687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f51880845a0 con 0x7f51980cd6d0 2024-01-11T07:08:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.995+0000 7f51967fc700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5178066f30 0x7f51780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.996+0000 7f51967fc700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5178066f30 0x7f51780693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f519816e8f0 tx=0x7f5180009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:14.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:14.999+0000 7f518effd700 1 -- 172.21.15.7:0/437215687 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5188051b10 con 0x7f51980cd6d0 2024-01-11T07:08:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.219+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f51980d0420 con 0x7f51980cd6d0 2024-01-11T07:08:15.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.750+0000 7f518effd700 1 -- 172.21.15.7:0/437215687 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5188055130 con 0x7f51980cd6d0 2024-01-11T07:08:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.753+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5178066f30 msgr2=0x7f51780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.753+0000 7f519d556700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5178066f30 0x7f51780693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f519816e8f0 tx=0x7f5180009040 comp rx=0 tx=0).stop 2024-01-11T07:08:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.753+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51980cd6d0 msgr2=0x7f519816da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.753+0000 7f519d556700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51980cd6d0 0x7f519816da30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f518800ec80 tx=0x7f51880091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.754+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 shutdown_connections 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.754+0000 7f519d556700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5178066f30 0x7f51780693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.754+0000 7f519d556700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f519815e450 0x7f5198146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.754+0000 7f519d556700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51980cd6d0 0x7f519816da30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.754+0000 7f519d556700 1 --2- 172.21.15.7:0/437215687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f519815d8a0 0x7f5198146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.754+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 >> 172.21.15.7:0/437215687 conn(0x7f5198071120 msgr2=0x7f5198158770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.754+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 shutdown_connections 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.754+0000 7f519d556700 1 -- 172.21.15.7:0/437215687 wait complete. 2024-01-11T07:08:15.754 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:08:15.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:08:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.906+0000 7eff0f2b5700 1 Processor -- start 2024-01-11T07:08:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.907+0000 7eff0f2b5700 1 -- start start 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.907+0000 7eff0f2b5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff0815a060 0x7eff080b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.907+0000 7eff0f2b5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff08069320 0x7eff080b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.907+0000 7eff0f2b5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff080ce9b0 con 0x7eff08159c90 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.907+0000 7eff0f2b5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff080b1f30 con 0x7eff08069320 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.907+0000 7eff0f2b5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff080cf530 con 0x7eff0815a060 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0d852700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff08069320 0x7eff080b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0d852700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff08069320 0x7eff080b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42504/0 (socket says 172.21.15.7:42504) 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0d852700 1 -- 172.21.15.7:0/2464514693 learned_addr learned my addr 172.21.15.7:0/2464514693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0c850700 1 --2- 172.21.15.7:0/2464514693 >> v2:172.21.15.113:3300/0 conn(0x7eff0815a060 0x7eff080b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0d852700 1 -- 172.21.15.7:0/2464514693 >> v2:172.21.15.113:3300/0 conn(0x7eff0815a060 msgr2=0x7eff080b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0d852700 1 --2- 172.21.15.7:0/2464514693 >> v2:172.21.15.113:3300/0 conn(0x7eff0815a060 0x7eff080b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0d852700 1 -- 172.21.15.7:0/2464514693 >> v1:172.21.15.113:6789/0 conn(0x7eff08159c90 legacy=0x7eff080b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0d852700 1 -- 172.21.15.7:0/2464514693 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080cf9c0 con 0x7eff08069320 2024-01-11T07:08:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7eff0d852700 1 --2- 172.21.15.7:0/2464514693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff08069320 0x7eff080b0970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7eff080699e0 tx=0x7eff00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb983e73c1ff689 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7efefffff700 1 -- 172.21.15.7:0/2464514693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000e070 con 0x7eff08069320 2024-01-11T07:08:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7efefffff700 1 -- 172.21.15.7:0/2464514693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0000ba20 con 0x7eff08069320 2024-01-11T07:08:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.908+0000 7efefffff700 1 -- 172.21.15.7:0/2464514693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff000092e0 con 0x7eff08069320 2024-01-11T07:08:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.909+0000 7eff0f2b5700 1 -- 172.21.15.7:0/2464514693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff08069320 msgr2=0x7eff080b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.909+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/2464514693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff08069320 0x7eff080b0970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7eff080699e0 tx=0x7eff00009840 comp rx=0 tx=0).stop 2024-01-11T07:08:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.909+0000 7eff0f2b5700 1 -- 172.21.15.7:0/2464514693 shutdown_connections 2024-01-11T07:08:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.909+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/2464514693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff08069320 0x7eff080b0970 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.909+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/2464514693 >> v2:172.21.15.113:3300/0 conn(0x7eff0815a060 0x7eff080b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.909+0000 7eff0f2b5700 1 -- 172.21.15.7:0/2464514693 >> 172.21.15.7:0/2464514693 conn(0x7eff08071100 msgr2=0x7eff0806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.909+0000 7eff0f2b5700 1 -- 172.21.15.7:0/2464514693 shutdown_connections 2024-01-11T07:08:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.909+0000 7eff0f2b5700 1 -- 172.21.15.7:0/2464514693 wait complete. 2024-01-11T07:08:15.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.910+0000 7eff0f2b5700 1 Processor -- start 2024-01-11T07:08:15.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.910+0000 7eff0f2b5700 1 -- start start 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0f2b5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff08069320 0x7eff081536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0f2b5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff08159c90 0x7eff080b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0f2b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0815a060 0x7eff080b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0f2b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0815d690 con 0x7eff0815a060 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0f2b5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0815dc50 con 0x7eff08159c90 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0f2b5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff080aed50 con 0x7eff08069320 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0d852700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0815a060 0x7eff080b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0d852700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0815a060 0x7eff080b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42524/0 (socket says 172.21.15.7:42524) 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.911+0000 7eff0d852700 1 -- 172.21.15.7:0/841475219 learned_addr learned my addr 172.21.15.7:0/841475219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0c850700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff08159c90 0x7eff080b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0d051700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff08069320 0x7eff081536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:15.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0d852700 1 -- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff08069320 msgr2=0x7eff081536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0d852700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff08069320 0x7eff081536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0d852700 1 -- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff08159c90 msgr2=0x7eff080b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0d852700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff08159c90 0x7eff080b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0d852700 1 -- 172.21.15.7:0/841475219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff00009ae0 con 0x7eff0815a060 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0d852700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0815a060 0x7eff080b0650 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7eff0806a1d0 tx=0x7eff00018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7efefd7fa700 1 -- 172.21.15.7:0/841475219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff000089b0 con 0x7eff0815a060 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7efefd7fa700 1 -- 172.21.15.7:0/841475219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff00008b10 con 0x7eff0815a060 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff080af030 con 0x7eff0815a060 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7efefd7fa700 1 -- 172.21.15.7:0/841475219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00025460 con 0x7eff0815a060 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.912+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff08153c30 con 0x7eff0815a060 2024-01-11T07:08:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.913+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0815dde0 con 0x7eff0815a060 2024-01-11T07:08:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.914+0000 7efefd7fa700 1 -- 172.21.15.7:0/841475219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7eff00003040 con 0x7eff0815a060 2024-01-11T07:08:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.914+0000 7efefd7fa700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efeec066f30 0x7efeec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.914+0000 7efefd7fa700 1 -- 172.21.15.7:0/841475219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7eff00085c90 con 0x7eff0815a060 2024-01-11T07:08:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.915+0000 7eff0d051700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efeec066f30 0x7efeec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:15.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.915+0000 7eff0d051700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efeec066f30 0x7efeec0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff04003a80 tx=0x7eff04007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:15.919+0000 7efefd7fa700 1 -- 172.21.15.7:0/841475219 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff00053250 con 0x7eff0815a060 2024-01-11T07:08:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.199+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7eff0804c3c0 con 0x7eff0815a060 2024-01-11T07:08:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.199+0000 7efefd7fa700 1 -- 172.21.15.7:0/841475219 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7eff00056870 con 0x7eff0815a060 2024-01-11T07:08:16.199 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:16.199 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:16.199 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:08:16.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.199 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:08:16.199 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:08:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:08:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:08:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:08:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:16.212 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.201+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efeec066f30 msgr2=0x7efeec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efeec066f30 0x7efeec0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff04003a80 tx=0x7eff04007ec0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0815a060 msgr2=0x7eff080b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0815a060 0x7eff080b0650 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7eff0806a1d0 tx=0x7eff00018000 comp rx=0 tx=0).stop 2024-01-11T07:08:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 shutdown_connections 2024-01-11T07:08:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efeec066f30 0x7efeec0693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff0815a060 0x7eff080b0650 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff08069320 0x7eff081536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 --2- 172.21.15.7:0/841475219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff08159c90 0x7eff080b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 >> 172.21.15.7:0/841475219 conn(0x7eff08071100 msgr2=0x7eff08156650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 shutdown_connections 2024-01-11T07:08:16.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.202+0000 7eff0f2b5700 1 -- 172.21.15.7:0/841475219 wait complete. 2024-01-11T07:08:16.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.347+0000 7ff62bdf0700 1 Processor -- start 2024-01-11T07:08:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.347+0000 7ff62bdf0700 1 -- start start 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.348+0000 7ff62bdf0700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff6240cc8c0 0x7ff62406d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.348+0000 7ff62bdf0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6240c9860 con 0x7ff6240c9110 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.348+0000 7ff62bdf0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff624069d90 con 0x7ff62415e120 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.348+0000 7ff62bdf0700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6240c9580 con 0x7ff6240cc8c0 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.348+0000 7ff62938b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff6240c9110 0x7ff6240cc1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34204/0 (socket says 172.21.15.7:34204) 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.348+0000 7ff62938b700 1 -- 172.21.15.7:0/1810190184 learned_addr learned my addr 172.21.15.7:0/1810190184 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff62a38d700 1 --2- 172.21.15.7:0/1810190184 >> v2:172.21.15.113:3300/0 conn(0x7ff6240cc8c0 0x7ff62406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff628b8a700 1 -- 172.21.15.7:0/1810190184 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785634188 0 0) 0x7ff6240c9860 con 0x7ff6240c9110 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff628b8a700 1 -- 172.21.15.7:0/1810190184 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60c003570 con 0x7ff6240c9110 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff628b8a700 1 -- 172.21.15.7:0/1810190184 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1019631192 0 0) 0x7ff60c003570 con 0x7ff6240c9110 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff62a38d700 1 -- 172.21.15.7:0/1810190184 >> v1:172.21.15.113:6789/0 conn(0x7ff62415e120 legacy=0x7ff6241626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff62a38d700 1 -- 172.21.15.7:0/1810190184 >> v1:172.21.15.7:6789/0 conn(0x7ff6240c9110 legacy=0x7ff6240cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff62a38d700 1 -- 172.21.15.7:0/1810190184 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff624069740 con 0x7ff6240cc8c0 2024-01-11T07:08:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff62a38d700 1 --2- 172.21.15.7:0/1810190184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240cc8c0 0x7ff62406d400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0098e0 tx=0x7ff61c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1205c4d22962aed7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff628b8a700 1 -- 172.21.15.7:0/1810190184 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618001470 con 0x7ff6240cc8c0 2024-01-11T07:08:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff628b8a700 1 -- 172.21.15.7:0/1810190184 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c011420 con 0x7ff6240cc8c0 2024-01-11T07:08:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.349+0000 7ff628b8a700 1 -- 172.21.15.7:0/1810190184 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c0117c0 con 0x7ff6240cc8c0 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.350+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1810190184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240cc8c0 msgr2=0x7ff62406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.350+0000 7ff62bdf0700 1 --2- 172.21.15.7:0/1810190184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240cc8c0 0x7ff62406d400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0098e0 tx=0x7ff61c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.350+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1810190184 shutdown_connections 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.350+0000 7ff62bdf0700 1 --2- 172.21.15.7:0/1810190184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240cc8c0 0x7ff62406d400 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.350+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1810190184 >> 172.21.15.7:0/1810190184 conn(0x7ff6240cd090 msgr2=0x7ff6240cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.350+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1810190184 shutdown_connections 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.350+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1810190184 wait complete. 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.351+0000 7ff62bdf0700 1 Processor -- start 2024-01-11T07:08:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.351+0000 7ff62bdf0700 1 -- start start 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62bdf0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240c9110 0x7ff62415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62bdf0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6240cc8c0 0x7ff62415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62bdf0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62415e120 0x7ff6240b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62bdf0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62415c570 con 0x7ff62415e120 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62bdf0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6240b27b0 con 0x7ff6240c9110 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62bdf0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6240c4f70 con 0x7ff6240cc8c0 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62a38d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62415e120 0x7ff6240b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62a38d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62415e120 0x7ff6240b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42528/0 (socket says 172.21.15.7:42528) 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff62a38d700 1 -- 172.21.15.7:0/1368230495 learned_addr learned my addr 172.21.15.7:0/1368230495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.352+0000 7ff629b8c700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240c9110 0x7ff62415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62938b700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6240cc8c0 0x7ff62415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62a38d700 1 -- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6240cc8c0 msgr2=0x7ff62415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62a38d700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6240cc8c0 0x7ff62415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62a38d700 1 -- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240c9110 msgr2=0x7ff62415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62a38d700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240c9110 0x7ff62415baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62a38d700 1 -- 172.21.15.7:0/1368230495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c009420 con 0x7ff62415e120 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62a38d700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62415e120 0x7ff6240b0e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff62406fdf0 tx=0x7ff61c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff621ffb700 1 -- 172.21.15.7:0/1368230495 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c010c20 con 0x7ff62415e120 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff621ffb700 1 -- 172.21.15.7:0/1368230495 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c010d80 con 0x7ff62415e120 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6240b1f10 con 0x7ff62415e120 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff621ffb700 1 -- 172.21.15.7:0/1368230495 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c01d640 con 0x7ff62415e120 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.353+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6240b24d0 con 0x7ff62415e120 2024-01-11T07:08:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.354+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62404c3c0 con 0x7ff62415e120 2024-01-11T07:08:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.355+0000 7ff621ffb700 1 -- 172.21.15.7:0/1368230495 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff61c01da30 con 0x7ff62415e120 2024-01-11T07:08:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.355+0000 7ff621ffb700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff610066f30 0x7ff6100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.355+0000 7ff621ffb700 1 -- 172.21.15.7:0/1368230495 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff61c086140 con 0x7ff62415e120 2024-01-11T07:08:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.356+0000 7ff629b8c700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff610066f30 0x7ff6100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.356+0000 7ff629b8c700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff610066f30 0x7ff6100693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff614007fd0 tx=0x7ff614009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.360+0000 7ff621ffb700 1 -- 172.21.15.7:0/1368230495 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff61c053660 con 0x7ff62415e120 2024-01-11T07:08:16.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.563+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6240c50b0 con 0x7ff62415e120 2024-01-11T07:08:16.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.564+0000 7ff621ffb700 1 -- 172.21.15.7:0/1368230495 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12086 (secure 0 0 0) 0x7ff61c056c80 con 0x7ff62415e120 2024-01-11T07:08:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:16.564 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:15.755091+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:16.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff610066f30 msgr2=0x7ff6100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff610066f30 0x7ff6100693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff614007fd0 tx=0x7ff614009040 comp rx=0 tx=0).stop 2024-01-11T07:08:16.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62415e120 msgr2=0x7ff6240b0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62415e120 0x7ff6240b0e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff62406fdf0 tx=0x7ff61c017000 comp rx=0 tx=0).stop 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 shutdown_connections 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff610066f30 0x7ff6100693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62415e120 0x7ff6240b0e60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6240c9110 0x7ff62415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 --2- 172.21.15.7:0/1368230495 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6240cc8c0 0x7ff62415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.566+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 >> 172.21.15.7:0/1368230495 conn(0x7ff6240cd090 msgr2=0x7ff62406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.567+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 shutdown_connections 2024-01-11T07:08:16.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.567+0000 7ff62bdf0700 1 -- 172.21.15.7:0/1368230495 wait complete. 2024-01-11T07:08:16.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:08:16.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:08:16.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.715+0000 7f3340218700 1 Processor -- start 2024-01-11T07:08:16.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.716+0000 7f3340218700 1 -- start start 2024-01-11T07:08:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.716+0000 7f3340218700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f33380b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.716+0000 7f3340218700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333815b940 con 0x7f333806dbe0 2024-01-11T07:08:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.716+0000 7f3340218700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3338069c70 con 0x7f33380b0160 2024-01-11T07:08:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f3340218700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3338069f50 con 0x7f33380afd90 2024-01-11T07:08:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333e7b5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f333806dbe0 0x7f333806ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34210/0 (socket says 172.21.15.7:34210) 2024-01-11T07:08:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333e7b5700 1 -- 172.21.15.7:0/1052869513 learned_addr learned my addr 172.21.15.7:0/1052869513 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333dfb4700 1 --2- 172.21.15.7:0/1052869513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f33380b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333cfb2700 1 -- 172.21.15.7:0/1052869513 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312941067 0 0) 0x7f333815b940 con 0x7f333806dbe0 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333cfb2700 1 -- 172.21.15.7:0/1052869513 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3324003570 con 0x7f333806dbe0 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333cfb2700 1 -- 172.21.15.7:0/1052869513 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837854758 0 0) 0x7f3338069c70 con 0x7f33380b0160 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333cfb2700 1 -- 172.21.15.7:0/1052869513 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f333815b940 con 0x7f33380b0160 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333dfb4700 1 -- 172.21.15.7:0/1052869513 >> v1:172.21.15.113:6789/0 conn(0x7f33380b0160 legacy=0x7f33380c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333dfb4700 1 -- 172.21.15.7:0/1052869513 >> v1:172.21.15.7:6789/0 conn(0x7f333806dbe0 legacy=0x7f333806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333dfb4700 1 -- 172.21.15.7:0/1052869513 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333806a230 con 0x7f33380afd90 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.717+0000 7f333dfb4700 1 --2- 172.21.15.7:0/1052869513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f33380b1070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3334009910 tx=0x7f33340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48033093f2c9173f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f333cfb2700 1 -- 172.21.15.7:0/1052869513 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333400b690 con 0x7f33380afd90 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f333cfb2700 1 -- 172.21.15.7:0/1052869513 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333400b7f0 con 0x7f33380afd90 2024-01-11T07:08:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f333cfb2700 1 -- 172.21.15.7:0/1052869513 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333400bb90 con 0x7f33380afd90 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f3340218700 1 -- 172.21.15.7:0/1052869513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 msgr2=0x7f33380b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f3340218700 1 --2- 172.21.15.7:0/1052869513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f33380b1070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3334009910 tx=0x7f33340091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f3340218700 1 -- 172.21.15.7:0/1052869513 shutdown_connections 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f3340218700 1 --2- 172.21.15.7:0/1052869513 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f33380b1070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f3340218700 1 -- 172.21.15.7:0/1052869513 >> 172.21.15.7:0/1052869513 conn(0x7f33380680e0 msgr2=0x7f33380cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f3340218700 1 -- 172.21.15.7:0/1052869513 shutdown_connections 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.718+0000 7f3340218700 1 -- 172.21.15.7:0/1052869513 wait complete. 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.719+0000 7f3340218700 1 Processor -- start 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f3340218700 1 -- start start 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f3340218700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333806dbe0 0x7f3338141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f3340218700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f333815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f3340218700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33380b0160 0x7f333815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f3340218700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338151310 con 0x7f333806dbe0 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f3340218700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3338151b50 con 0x7f33380b0160 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f3340218700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33380c6bc0 con 0x7f33380afd90 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f333dfb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333806dbe0 0x7f3338141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f333dfb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333806dbe0 0x7f3338141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42534/0 (socket says 172.21.15.7:42534) 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.720+0000 7f333dfb4700 1 -- 172.21.15.7:0/2717883000 learned_addr learned my addr 172.21.15.7:0/2717883000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333d7b3700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f333815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333e7b5700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33380b0160 0x7f333815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333dfb4700 1 -- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 msgr2=0x7f333815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333dfb4700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f333815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333dfb4700 1 -- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33380b0160 msgr2=0x7f333815b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333dfb4700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33380b0160 0x7f333815b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333dfb4700 1 -- 172.21.15.7:0/2717883000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334009420 con 0x7f333806dbe0 2024-01-11T07:08:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333e7b5700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33380b0160 0x7f333815b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f333dfb4700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333806dbe0 0x7f3338141b00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3334009f00 tx=0x7f3334017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f332dffb700 1 -- 172.21.15.7:0/2717883000 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333401f070 con 0x7f333806dbe0 2024-01-11T07:08:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f332dffb700 1 -- 172.21.15.7:0/2717883000 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334000bc0 con 0x7f333806dbe0 2024-01-11T07:08:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3338151540 con 0x7f333806dbe0 2024-01-11T07:08:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f332dffb700 1 -- 172.21.15.7:0/2717883000 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334025420 con 0x7f333806dbe0 2024-01-11T07:08:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.721+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33381517e0 con 0x7f333806dbe0 2024-01-11T07:08:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.723+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3338151e30 con 0x7f333806dbe0 2024-01-11T07:08:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.723+0000 7f332dffb700 1 -- 172.21.15.7:0/2717883000 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3334023070 con 0x7f333806dbe0 2024-01-11T07:08:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.724+0000 7f332dffb700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f331c066f30 0x7f331c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.724+0000 7f332dffb700 1 -- 172.21.15.7:0/2717883000 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3334085740 con 0x7f333806dbe0 2024-01-11T07:08:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.724+0000 7f333d7b3700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f331c066f30 0x7f331c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.724+0000 7f333d7b3700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f331c066f30 0x7f331c0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f333815b740 tx=0x7f3328009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.728+0000 7f332dffb700 1 -- 172.21.15.7:0/2717883000 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3334052d00 con 0x7f333806dbe0 2024-01-11T07:08:16.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.932+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33380685c0 con 0x7f333806dbe0 2024-01-11T07:08:16.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.932+0000 7f332dffb700 1 -- 172.21.15.7:0/2717883000 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f333402a020 con 0x7f333806dbe0 2024-01-11T07:08:16.932 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:08:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f331c066f30 msgr2=0x7f331c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f331c066f30 0x7f331c0693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f333815b740 tx=0x7f3328009040 comp rx=0 tx=0).stop 2024-01-11T07:08:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333806dbe0 msgr2=0x7f3338141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333806dbe0 0x7f3338141b00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3334009f00 tx=0x7f3334017000 comp rx=0 tx=0).stop 2024-01-11T07:08:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 shutdown_connections 2024-01-11T07:08:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f331c066f30 0x7f331c0693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33380b0160 0x7f333815b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f333806dbe0 0x7f3338141b00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 --2- 172.21.15.7:0/2717883000 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f33380afd90 0x7f333815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 >> 172.21.15.7:0/2717883000 conn(0x7f33380680e0 msgr2=0x7f33380c8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 shutdown_connections 2024-01-11T07:08:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:16.934+0000 7f3340218700 1 -- 172.21.15.7:0/2717883000 wait complete. 2024-01-11T07:08:16.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:08:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.080+0000 7f1b3f720700 1 Processor -- start 2024-01-11T07:08:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.081+0000 7f1b3f720700 1 -- start start 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.081+0000 7f1b3f720700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b3806f3c0 0x7f1b380ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.081+0000 7f1b3f720700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b380c8f80 con 0x7f1b38157e80 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.081+0000 7f1b3f720700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b380c9de0 con 0x7f1b38157ab0 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.081+0000 7f1b3f720700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b380c91d0 con 0x7f1b3806f3c0 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b3ccbb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1b38157e80 0x7f1b380b0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34214/0 (socket says 172.21.15.7:34214) 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b3ccbb700 1 -- 172.21.15.7:0/1316097570 learned_addr learned my addr 172.21.15.7:0/1316097570 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b3dcbd700 1 --2- 172.21.15.7:0/1316097570 >> v2:172.21.15.113:3300/0 conn(0x7f1b3806f3c0 0x7f1b380ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821432104 0 0) 0x7f1b380c8f80 con 0x7f1b38157e80 2024-01-11T07:08:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b24003570 con 0x7f1b38157e80 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3094018627 0 0) 0x7f1b24003570 con 0x7f1b38157e80 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 >> v2:172.21.15.113:3300/0 conn(0x7f1b3806f3c0 msgr2=0x7f1b380ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 --2- 172.21.15.7:0/1316097570 >> v2:172.21.15.113:3300/0 conn(0x7f1b3806f3c0 0x7f1b380ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 >> v1:172.21.15.113:6789/0 conn(0x7f1b38157ab0 legacy=0x7f1b3806ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380c94b0 con 0x7f1b38157e80 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f1b28001470 con 0x7f1b38157e80 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 >> v1:172.21.15.7:6789/0 conn(0x7f1b38157e80 legacy=0x7f1b380b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24003ab0 0x7f1b24003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b24004810 0x7f1b24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b24005210 0x7f1b2400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380c91d0 con 0x7f1b24004810 2024-01-11T07:08:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2400c130 con 0x7f1b24003ab0 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.082+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2400c2f0 con 0x7f1b24005210 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3ccbb700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b24004810 0x7f1b24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3d4bc700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24003ab0 0x7f1b24003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3dcbd700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b24005210 0x7f1b2400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3ccbb700 1 -- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b24005210 msgr2=0x7f1b2400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3ccbb700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b24005210 0x7f1b2400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3ccbb700 1 -- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24003ab0 msgr2=0x7f1b24003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3ccbb700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24003ab0 0x7f1b24003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3ccbb700 1 -- 172.21.15.7:0/1316097570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24003570 con 0x7f1b24004810 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3dcbd700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b24005210 0x7f1b2400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3dcbd700 1 -- 172.21.15.7:0/1316097570 reap_dead start 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b3ccbb700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b24004810 0x7f1b24004c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800c2a0 tx=0x7f1b380aed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d200e02c26fab89e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800cb70 con 0x7f1b24004810 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2800ccd0 con 0x7f1b24004810 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.083+0000 7f1b2ffff700 1 -- 172.21.15.7:0/1316097570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28005b70 con 0x7f1b24004810 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.084+0000 7f1b3f720700 1 -- 172.21.15.7:0/1316097570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b24004810 msgr2=0x7f1b24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.084+0000 7f1b3f720700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b24004810 0x7f1b24004c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800c2a0 tx=0x7f1b380aed40 comp rx=0 tx=0).stop 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.084+0000 7f1b3f720700 1 -- 172.21.15.7:0/1316097570 shutdown_connections 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.084+0000 7f1b3f720700 1 --2- 172.21.15.7:0/1316097570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b24004810 0x7f1b24004c80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.084+0000 7f1b3f720700 1 -- 172.21.15.7:0/1316097570 >> 172.21.15.7:0/1316097570 conn(0x7f1b38070690 msgr2=0x7f1b38070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.084+0000 7f1b3f720700 1 -- 172.21.15.7:0/1316097570 shutdown_connections 2024-01-11T07:08:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.084+0000 7f1b3f720700 1 -- 172.21.15.7:0/1316097570 wait complete. 2024-01-11T07:08:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.085+0000 7f1b3f720700 1 Processor -- start 2024-01-11T07:08:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.085+0000 7f1b3f720700 1 -- start start 2024-01-11T07:08:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3f720700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24004810 0x7f1b38148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3f720700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381687c0 0x7f1b3815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3f720700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b3815aa50 0x7f1b380cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3f720700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3815aef0 con 0x7f1b381687c0 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3f720700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b381493e0 con 0x7f1b24004810 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3f720700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380cdf40 con 0x7f1b3815aa50 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3ccbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381687c0 0x7f1b3815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3ccbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381687c0 0x7f1b3815a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42560/0 (socket says 172.21.15.7:42560) 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3ccbb700 1 -- 172.21.15.7:0/1903354205 learned_addr learned my addr 172.21.15.7:0/1903354205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3d4bc700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24004810 0x7f1b38148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3dcbd700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b3815aa50 0x7f1b380cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3ccbb700 1 -- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b3815aa50 msgr2=0x7f1b380cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3ccbb700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b3815aa50 0x7f1b380cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3ccbb700 1 -- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24004810 msgr2=0x7f1b38148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3ccbb700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24004810 0x7f1b38148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3ccbb700 1 -- 172.21.15.7:0/1903354205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380c24e0 con 0x7f1b381687c0 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.086+0000 7f1b3d4bc700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24004810 0x7f1b38148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.087+0000 7f1b3ccbb700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381687c0 0x7f1b3815a510 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1b240057d0 tx=0x7f1b28015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.087+0000 7f1b2d7fa700 1 -- 172.21.15.7:0/1903354205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28022070 con 0x7f1b381687c0 2024-01-11T07:08:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.087+0000 7f1b2d7fa700 1 -- 172.21.15.7:0/1903354205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28004e40 con 0x7f1b381687c0 2024-01-11T07:08:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.087+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b380c27c0 con 0x7f1b381687c0 2024-01-11T07:08:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.087+0000 7f1b2d7fa700 1 -- 172.21.15.7:0/1903354205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800cb70 con 0x7f1b381687c0 2024-01-11T07:08:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.087+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b380ce220 con 0x7f1b381687c0 2024-01-11T07:08:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.088+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3804c3c0 con 0x7f1b381687c0 2024-01-11T07:08:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.088+0000 7f1b2d7fa700 1 -- 172.21.15.7:0/1903354205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1b2801e430 con 0x7f1b381687c0 2024-01-11T07:08:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.089+0000 7f1b2d7fa700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b1c066f30 0x7f1b1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.089+0000 7f1b2d7fa700 1 -- 172.21.15.7:0/1903354205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1b28085320 con 0x7f1b381687c0 2024-01-11T07:08:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.089+0000 7f1b3d4bc700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b1c066f30 0x7f1b1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.090+0000 7f1b3d4bc700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b1c066f30 0x7f1b1c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1b34007bc0 tx=0x7f1b34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:17.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.093+0000 7f1b2d7fa700 1 -- 172.21.15.7:0/1903354205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b280528e0 con 0x7f1b381687c0 2024-01-11T07:08:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.298+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1b3806b040 con 0x7f1b381687c0 2024-01-11T07:08:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.299+0000 7f1b2d7fa700 1 -- 172.21.15.7:0/1903354205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1b28017090 con 0x7f1b381687c0 2024-01-11T07:08:17.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:17.298 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:08:17.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:17.299 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:08:17.299 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:08:17.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:17.299 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b1c066f30 msgr2=0x7f1b1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b1c066f30 0x7f1b1c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1b34007bc0 tx=0x7f1b34009040 comp rx=0 tx=0).stop 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381687c0 msgr2=0x7f1b3815a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381687c0 0x7f1b3815a510 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1b240057d0 tx=0x7f1b28015000 comp rx=0 tx=0).stop 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 shutdown_connections 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b1c066f30 0x7f1b1c0693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b3815aa50 0x7f1b380cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b24004810 0x7f1b38148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 --2- 172.21.15.7:0/1903354205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381687c0 0x7f1b3815a510 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 >> 172.21.15.7:0/1903354205 conn(0x7f1b38070690 msgr2=0x7f1b38070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 shutdown_connections 2024-01-11T07:08:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.301+0000 7f1b3f720700 1 -- 172.21.15.7:0/1903354205 wait complete. 2024-01-11T07:08:17.314 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:08:17.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:08:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.450+0000 7ff64659b700 1 Processor -- start 2024-01-11T07:08:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.450+0000 7ff64659b700 1 -- start start 2024-01-11T07:08:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff64659b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff640063850 0x7ff64006a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff64659b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64006ac80 con 0x7ff640154540 2024-01-11T07:08:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff64659b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64006e630 con 0x7ff640063480 2024-01-11T07:08:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff64659b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64006a9a0 con 0x7ff640063850 2024-01-11T07:08:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff63f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff640154540 0x7ff640069cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40914/0 (socket says 172.21.15.7:40914) 2024-01-11T07:08:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff63f7fe700 1 -- 172.21.15.7:0/3667079806 learned_addr learned my addr 172.21.15.7:0/3667079806 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff644b38700 1 --2- 172.21.15.7:0/3667079806 >> v2:172.21.15.113:3300/0 conn(0x7ff640063850 0x7ff64006a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845648651 0 0) 0x7ff64006ac80 con 0x7ff640154540 2024-01-11T07:08:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.451+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62c003570 con 0x7ff640154540 2024-01-11T07:08:17.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3547751723 0 0) 0x7ff62c003570 con 0x7ff640154540 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 >> v2:172.21.15.113:3300/0 conn(0x7ff640063850 msgr2=0x7ff64006a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 --2- 172.21.15.7:0/3667079806 >> v2:172.21.15.113:3300/0 conn(0x7ff640063850 0x7ff64006a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 >> v1:172.21.15.113:6789/0 conn(0x7ff640063480 legacy=0x7ff640153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64006dfe0 con 0x7ff640154540 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7ff628001470 con 0x7ff640154540 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 >> v1:172.21.15.7:6789/0 conn(0x7ff640154540 legacy=0x7ff640069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff62c003a60 0x7ff62c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c004810 0x7ff62c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c0051c0 0x7ff62c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64006a9a0 con 0x7ff62c0051c0 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c00c0d0 con 0x7ff62c004810 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c00c290 con 0x7ff62c003a60 2024-01-11T07:08:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63f7fe700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c004810 0x7ff62c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff644b38700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c0051c0 0x7ff62c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63ffff700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff62c003a60 0x7ff62c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff644b38700 1 -- 172.21.15.7:0/3667079806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff62c003a60 msgr2=0x7ff62c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff644b38700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff62c003a60 0x7ff62c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff644b38700 1 -- 172.21.15.7:0/3667079806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c004810 msgr2=0x7ff62c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff644b38700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c004810 0x7ff62c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff644b38700 1 -- 172.21.15.7:0/3667079806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c003570 con 0x7ff62c0051c0 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff644b38700 1 -- 172.21.15.7:0/3667079806 reap_dead start 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff644b38700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c0051c0 0x7ff62c00ba60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e210 tx=0x7ff63800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd71c36879f11af server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.452+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800eac0 con 0x7ff62c0051c0 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63800f040 con 0x7ff62c0051c0 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff63effd700 1 -- 172.21.15.7:0/3667079806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800ee70 con 0x7ff62c0051c0 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff64659b700 1 -- 172.21.15.7:0/3667079806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c0051c0 msgr2=0x7ff62c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff64659b700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c0051c0 0x7ff62c00ba60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e210 tx=0x7ff63800b040 comp rx=0 tx=0).stop 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff64659b700 1 -- 172.21.15.7:0/3667079806 shutdown_connections 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff64659b700 1 --2- 172.21.15.7:0/3667079806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c0051c0 0x7ff62c00ba60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff64659b700 1 -- 172.21.15.7:0/3667079806 >> 172.21.15.7:0/3667079806 conn(0x7ff64006bce0 msgr2=0x7ff64006c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff64659b700 1 -- 172.21.15.7:0/3667079806 shutdown_connections 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.453+0000 7ff64659b700 1 -- 172.21.15.7:0/3667079806 wait complete. 2024-01-11T07:08:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.454+0000 7ff64659b700 1 Processor -- start 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff64659b700 1 -- start start 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff64659b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c0051c0 0x7ff64016af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff64659b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff64016d960 0x7ff64014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff64659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff640159190 0x7ff640159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff64659b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640159b70 con 0x7ff640159190 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff64659b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64016b530 con 0x7ff62c0051c0 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff64659b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff640150ec0 con 0x7ff64016d960 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff644b38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff640159190 0x7ff640159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff644b38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff640159190 0x7ff640159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47040/0 (socket says 172.21.15.7:47040) 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.455+0000 7ff644b38700 1 -- 172.21.15.7:0/3756067948 learned_addr learned my addr 172.21.15.7:0/3756067948 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff63f7fe700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff64016d960 0x7ff64014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff63ffff700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c0051c0 0x7ff64016af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff644b38700 1 -- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff64016d960 msgr2=0x7ff64014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff644b38700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff64016d960 0x7ff64014e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff644b38700 1 -- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c0051c0 msgr2=0x7ff64016af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff644b38700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c0051c0 0x7ff64016af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff644b38700 1 -- 172.21.15.7:0/3756067948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64016e140 con 0x7ff640159190 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff63f7fe700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff64016d960 0x7ff64014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff63ffff700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c0051c0 0x7ff64016af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff644b38700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff640159190 0x7ff640159630 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff64006d590 tx=0x7ff638019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff637fff700 1 -- 172.21.15.7:0/3756067948 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63801d660 con 0x7ff640159190 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff637fff700 1 -- 172.21.15.7:0/3756067948 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638022070 con 0x7ff640159190 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64016e420 con 0x7ff640159190 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff637fff700 1 -- 172.21.15.7:0/3756067948 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63801da30 con 0x7ff640159190 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.456+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64014ef20 con 0x7ff640159190 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.457+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64004c3c0 con 0x7ff640159190 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.457+0000 7ff637fff700 1 -- 172.21.15.7:0/3756067948 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff638003020 con 0x7ff640159190 2024-01-11T07:08:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.458+0000 7ff637fff700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.458+0000 7ff637fff700 1 -- 172.21.15.7:0/3756067948 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff638024070 con 0x7ff640159190 2024-01-11T07:08:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.458+0000 7ff63ffff700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.459+0000 7ff63ffff700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f60 tx=0x7ff630009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.462+0000 7ff637fff700 1 -- 172.21.15.7:0/3756067948 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff638053dd0 con 0x7ff640159190 2024-01-11T07:08:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:17.666+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff640151030 con 0x7ff640159190 2024-01-11T07:08:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.176+0000 7ff637fff700 1 -- 172.21.15.7:0/3756067948 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff63801ceb0 con 0x7ff640159190 2024-01-11T07:08:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.180+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 msgr2=0x7ff6200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.180+0000 7ff64659b700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f60 tx=0x7ff630009040 comp rx=0 tx=0).stop 2024-01-11T07:08:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.180+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff640159190 msgr2=0x7ff640159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.180+0000 7ff64659b700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff640159190 0x7ff640159630 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff64006d590 tx=0x7ff638019000 comp rx=0 tx=0).stop 2024-01-11T07:08:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.181+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 shutdown_connections 2024-01-11T07:08:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.181+0000 7ff64659b700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.181+0000 7ff64659b700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff640159190 0x7ff640159630 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.181+0000 7ff64659b700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff62c0051c0 0x7ff64016af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.181+0000 7ff64659b700 1 --2- 172.21.15.7:0/3756067948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff64016d960 0x7ff64014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.181+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 >> 172.21.15.7:0/3756067948 conn(0x7ff64006bce0 msgr2=0x7ff640157f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.181+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 shutdown_connections 2024-01-11T07:08:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.182+0000 7ff64659b700 1 -- 172.21.15.7:0/3756067948 wait complete. 2024-01-11T07:08:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:08:18.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:08:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.340+0000 7fdbf0e28700 1 Processor -- start 2024-01-11T07:08:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.340+0000 7fdbf0e28700 1 -- start start 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.341+0000 7fdbf0e28700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec157e80 0x7fdbec0b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.341+0000 7fdbf0e28700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0c8f80 con 0x7fdbec06f3c0 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.341+0000 7fdbf0e28700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0c9de0 con 0x7fdbec157ab0 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.341+0000 7fdbf0e28700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0c91d0 con 0x7fdbec157e80 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbead9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdbec06f3c0 0x7fdbec0ad530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40916/0 (socket says 172.21.15.7:40916) 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbead9d700 1 -- 172.21.15.7:0/415226639 learned_addr learned my addr 172.21.15.7:0/415226639 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe9d9b700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec157e80 0x7fdbec0b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115812054 0 0) 0x7fdbec0c8f80 con 0x7fdbec06f3c0 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd4003570 con 0x7fdbec06f3c0 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438630053 0 0) 0x7fdbd4003570 con 0x7fdbec06f3c0 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec157e80 msgr2=0x7fdbec0b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec157e80 0x7fdbec0b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 >> v1:172.21.15.113:6789/0 conn(0x7fdbec157ab0 legacy=0x7fdbec06ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec0c94b0 con 0x7fdbec06f3c0 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fdbe40017f0 con 0x7fdbec06f3c0 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 >> v1:172.21.15.7:6789/0 conn(0x7fdbec06f3c0 legacy=0x7fdbec0ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4003a90 0x7fdbd4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4004810 0x7fdbd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec157e80 0x7fdbd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0c91d0 con 0x7fdbd4004810 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd400bb40 con 0x7fdbd4003a90 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd400bd00 con 0x7fdbec157e80 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbe9d9b700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4004810 0x7fdbd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.342+0000 7fdbea59c700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4003a90 0x7fdbd4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbead9d700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec157e80 0x7fdbd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe9d9b700 1 -- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec157e80 msgr2=0x7fdbd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe9d9b700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec157e80 0x7fdbd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe9d9b700 1 -- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4003a90 msgr2=0x7fdbd4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe9d9b700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4003a90 0x7fdbd4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe9d9b700 1 -- 172.21.15.7:0/415226639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4003570 con 0x7fdbd4004810 2024-01-11T07:08:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe9d9b700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4004810 0x7fdbd4004c80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009c00 tx=0x7fdbe00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9a2a4f7c3ac0c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000e710 con 0x7fdbd4004810 2024-01-11T07:08:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe000e870 con 0x7fdbd4004810 2024-01-11T07:08:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.343+0000 7fdbe959a700 1 -- 172.21.15.7:0/415226639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000ec30 con 0x7fdbd4004810 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.344+0000 7fdbf0e28700 1 -- 172.21.15.7:0/415226639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4004810 msgr2=0x7fdbd4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.344+0000 7fdbf0e28700 1 --2- 172.21.15.7:0/415226639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbd4004810 0x7fdbd4004c80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009c00 tx=0x7fdbe00091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.344+0000 7fdbead9d700 1 -- 172.21.15.7:0/415226639 reap_dead start 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.344+0000 7fdbf0e28700 1 -- 172.21.15.7:0/415226639 shutdown_connections 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.344+0000 7fdbf0e28700 1 -- 172.21.15.7:0/415226639 >> 172.21.15.7:0/415226639 conn(0x7fdbec070690 msgr2=0x7fdbec070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.344+0000 7fdbf0e28700 1 -- 172.21.15.7:0/415226639 shutdown_connections 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.344+0000 7fdbf0e28700 1 -- 172.21.15.7:0/415226639 wait complete. 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.345+0000 7fdbf0e28700 1 Processor -- start 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.345+0000 7fdbf0e28700 1 -- start start 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbf0e28700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4004810 0x7fdbec0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbf0e28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec0c59f0 0x7fdbec0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbf0e28700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec0cdf60 0x7fdbec154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbf0e28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec156ef0 con 0x7fdbec0c59f0 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbf0e28700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0c89c0 con 0x7fdbd4004810 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbf0e28700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec151e40 con 0x7fdbec0cdf60 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbe9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec0c59f0 0x7fdbec0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbe9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec0c59f0 0x7fdbec0cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47056/0 (socket says 172.21.15.7:47056) 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbe9d9b700 1 -- 172.21.15.7:0/350827234 learned_addr learned my addr 172.21.15.7:0/350827234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbead9d700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec0cdf60 0x7fdbec154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbea59c700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4004810 0x7fdbec0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbe9d9b700 1 -- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec0cdf60 msgr2=0x7fdbec154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbe9d9b700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec0cdf60 0x7fdbec154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbe9d9b700 1 -- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4004810 msgr2=0x7fdbec0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbe9d9b700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4004810 0x7fdbec0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbe9d9b700 1 -- 172.21.15.7:0/350827234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec0c8e50 con 0x7fdbec0c59f0 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbead9d700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec0cdf60 0x7fdbec154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.346+0000 7fdbea59c700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4004810 0x7fdbec0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.347+0000 7fdbe9d9b700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec0c59f0 0x7fdbec0cda20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400bb10 tx=0x7fdbe001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.347+0000 7fdbda7fc700 1 -- 172.21.15.7:0/350827234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0022070 con 0x7fdbec0c59f0 2024-01-11T07:08:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.347+0000 7fdbda7fc700 1 -- 172.21.15.7:0/350827234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe000ebf0 con 0x7fdbec0c59f0 2024-01-11T07:08:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.347+0000 7fdbda7fc700 1 -- 172.21.15.7:0/350827234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe001c6a0 con 0x7fdbec0c59f0 2024-01-11T07:08:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.347+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec155db0 con 0x7fdbec0c59f0 2024-01-11T07:08:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.347+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec0c8370 con 0x7fdbec0c59f0 2024-01-11T07:08:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.348+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec0c8b70 con 0x7fdbec0c59f0 2024-01-11T07:08:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.349+0000 7fdbda7fc700 1 -- 172.21.15.7:0/350827234 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdbe001ca50 con 0x7fdbec0c59f0 2024-01-11T07:08:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.349+0000 7fdbda7fc700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdbcc066f30 0x7fdbcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.349+0000 7fdbda7fc700 1 -- 172.21.15.7:0/350827234 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fdbe0085e40 con 0x7fdbec0c59f0 2024-01-11T07:08:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.350+0000 7fdbea59c700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdbcc066f30 0x7fdbcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.350+0000 7fdbea59c700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdbcc066f30 0x7fdbcc0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc007ba0 tx=0x7fdbdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.354+0000 7fdbda7fc700 1 -- 172.21.15.7:0/350827234 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbe0053390 con 0x7fdbec0c59f0 2024-01-11T07:08:18.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:18.558+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdbec0c4ea0 con 0x7fdbec0c59f0 2024-01-11T07:08:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.183+0000 7fdbda7fc700 1 -- 172.21.15.7:0/350827234 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdbe00170d0 con 0x7fdbec0c59f0 2024-01-11T07:08:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdbcc066f30 msgr2=0x7fdbcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdbcc066f30 0x7fdbcc0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc007ba0 tx=0x7fdbdc009040 comp rx=0 tx=0).stop 2024-01-11T07:08:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec0c59f0 msgr2=0x7fdbec0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec0c59f0 0x7fdbec0cda20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400bb10 tx=0x7fdbe001a000 comp rx=0 tx=0).stop 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 shutdown_connections 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdbcc066f30 0x7fdbcc0693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdbec0cdf60 0x7fdbec154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbd4004810 0x7fdbec0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 --2- 172.21.15.7:0/350827234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbec0c59f0 0x7fdbec0cda20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.196+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 >> 172.21.15.7:0/350827234 conn(0x7fdbec070690 msgr2=0x7fdbec0b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.197+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 shutdown_connections 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.197+0000 7fdbf0e28700 1 -- 172.21.15.7:0/350827234 wait complete. 2024-01-11T07:08:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:08:19.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:08:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.351+0000 7f2ced021700 1 Processor -- start 2024-01-11T07:08:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.352+0000 7f2ced021700 1 -- start start 2024-01-11T07:08:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.352+0000 7f2ced021700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ce815a1a0 0x7f2ce80cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.352+0000 7f2ced021700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.352+0000 7f2ced021700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce80ced10 con 0x7f2ce80cbca0 2024-01-11T07:08:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.352+0000 7f2ced021700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce80714b0 con 0x7f2ce815a1a0 2024-01-11T07:08:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.352+0000 7f2ced021700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80cc140 con 0x7f2ce80b0030 2024-01-11T07:08:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2ce80cbca0 0x7f2ce80af920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40924/0 (socket says 172.21.15.7:40924) 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce659c700 1 -- 172.21.15.7:0/3449138358 learned_addr learned my addr 172.21.15.7:0/3449138358 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce759e700 1 --2- 172.21.15.7:0/3449138358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce6d9d700 1 --2- 172.21.15.7:0/3449138358 >> v2:172.21.15.113:3300/0 conn(0x7f2ce815a1a0 0x7f2ce80cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce5d9b700 1 -- 172.21.15.7:0/3449138358 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567166311 0 0) 0x7f2ce80ced10 con 0x7f2ce80cbca0 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce5d9b700 1 -- 172.21.15.7:0/3449138358 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd4003570 con 0x7f2ce80cbca0 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce759e700 1 -- 172.21.15.7:0/3449138358 >> v2:172.21.15.113:3300/0 conn(0x7f2ce815a1a0 msgr2=0x7f2ce80cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce759e700 1 --2- 172.21.15.7:0/3449138358 >> v2:172.21.15.113:3300/0 conn(0x7f2ce815a1a0 0x7f2ce80cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce759e700 1 -- 172.21.15.7:0/3449138358 >> v1:172.21.15.7:6789/0 conn(0x7f2ce80cbca0 legacy=0x7f2ce80af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce759e700 1 -- 172.21.15.7:0/3449138358 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce80cc960 con 0x7f2ce80b0030 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce6d9d700 1 --2- 172.21.15.7:0/3449138358 >> v2:172.21.15.113:3300/0 conn(0x7f2ce815a1a0 0x7f2ce80cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.353+0000 7f2ce759e700 1 --2- 172.21.15.7:0/3449138358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8163d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009e60 tx=0x7f2ce0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8032c85b8b370bfe server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ce5d9b700 1 -- 172.21.15.7:0/3449138358 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000e070 con 0x7f2ce80b0030 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ce5d9b700 1 -- 172.21.15.7:0/3449138358 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce000bab0 con 0x7f2ce80b0030 2024-01-11T07:08:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ce5d9b700 1 -- 172.21.15.7:0/3449138358 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000be70 con 0x7f2ce80b0030 2024-01-11T07:08:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ced021700 1 -- 172.21.15.7:0/3449138358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 msgr2=0x7f2ce8163d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ced021700 1 --2- 172.21.15.7:0/3449138358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8163d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009e60 tx=0x7f2ce0009780 comp rx=0 tx=0).stop 2024-01-11T07:08:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ced021700 1 -- 172.21.15.7:0/3449138358 shutdown_connections 2024-01-11T07:08:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ced021700 1 --2- 172.21.15.7:0/3449138358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8163d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ced021700 1 --2- 172.21.15.7:0/3449138358 >> v2:172.21.15.113:3300/0 conn(0x7f2ce815a1a0 0x7f2ce80cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ced021700 1 -- 172.21.15.7:0/3449138358 >> 172.21.15.7:0/3449138358 conn(0x7f2ce806e6c0 msgr2=0x7f2ce8070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ced021700 1 -- 172.21.15.7:0/3449138358 shutdown_connections 2024-01-11T07:08:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.354+0000 7f2ced021700 1 -- 172.21.15.7:0/3449138358 wait complete. 2024-01-11T07:08:19.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.356+0000 7f2ced021700 1 Processor -- start 2024-01-11T07:08:19.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.356+0000 7f2ced021700 1 -- start start 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.356+0000 7f2ced021700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.356+0000 7f2ced021700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80cbca0 0x7f2ce815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.356+0000 7f2ced021700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ce815a1a0 0x7f2ce814b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.356+0000 7f2ced021700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8155ae0 con 0x7f2ce80cbca0 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.356+0000 7f2ced021700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80cae30 con 0x7f2ce815a1a0 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.356+0000 7f2ced021700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8157380 con 0x7f2ce80b0030 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80cbca0 0x7f2ce815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80cbca0 0x7f2ce815cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47072/0 (socket says 172.21.15.7:47072) 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 -- 172.21.15.7:0/2581603288 learned_addr learned my addr 172.21.15.7:0/2581603288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce6d9d700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce759e700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ce815a1a0 0x7f2ce814b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 -- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 msgr2=0x7f2ce8156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 -- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ce815a1a0 msgr2=0x7f2ce814b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ce815a1a0 0x7f2ce814b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 -- 172.21.15.7:0/2581603288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8009420 con 0x7f2ce80cbca0 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce6d9d700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ce659c700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80cbca0 0x7f2ce815cb30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c7c0 tx=0x7f2cd0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2cdeffd700 1 -- 172.21.15.7:0/2581603288 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0005cd0 con 0x7f2ce80cbca0 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2cdeffd700 1 -- 172.21.15.7:0/2581603288 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd0005e30 con 0x7f2ce80cbca0 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce0009a20 con 0x7f2ce80cbca0 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.357+0000 7f2cdeffd700 1 -- 172.21.15.7:0/2581603288 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0007320 con 0x7f2ce80cbca0 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.358+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce81565b0 con 0x7f2ce80cbca0 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.359+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce8089350 con 0x7f2ce80cbca0 2024-01-11T07:08:19.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.359+0000 7f2cdeffd700 1 -- 172.21.15.7:0/2581603288 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2cd001b3b0 con 0x7f2ce80cbca0 2024-01-11T07:08:19.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.359+0000 7f2cdeffd700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2cc8066f30 0x7f2cc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:19.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.360+0000 7f2cdeffd700 1 -- 172.21.15.7:0/2581603288 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f2cd0083b70 con 0x7f2ce80cbca0 2024-01-11T07:08:19.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.360+0000 7f2ce6d9d700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2cc8066f30 0x7f2cc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:19.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.360+0000 7f2ce6d9d700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2cc8066f30 0x7f2cc80693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8011560 tx=0x7f2cd80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:19.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.364+0000 7f2cdeffd700 1 -- 172.21.15.7:0/2581603288 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cd0050e40 con 0x7f2ce80cbca0 2024-01-11T07:08:19.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:19.619+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2ce80890f0 con 0x7f2ce80cbca0 2024-01-11T07:08:20.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.187+0000 7f2cdeffd700 1 -- 172.21.15.7:0/2581603288 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2cd0017020 con 0x7f2ce80cbca0 2024-01-11T07:08:20.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.194+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2cc8066f30 msgr2=0x7f2cc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.194+0000 7f2ced021700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2cc8066f30 0x7f2cc80693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8011560 tx=0x7f2cd80091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.194+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80cbca0 msgr2=0x7f2ce815cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.194+0000 7f2ced021700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80cbca0 0x7f2ce815cb30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c7c0 tx=0x7f2cd0007e00 comp rx=0 tx=0).stop 2024-01-11T07:08:20.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.195+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 shutdown_connections 2024-01-11T07:08:20.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.195+0000 7f2ced021700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2cc8066f30 0x7f2cc80693e0 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8011560 tx=0x7f2cd80091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.195+0000 7f2ced021700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ce815a1a0 0x7f2ce814b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.195+0000 7f2ced021700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ce80b0030 0x7f2ce8156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.195+0000 7f2ced021700 1 --2- 172.21.15.7:0/2581603288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80cbca0 0x7f2ce815cb30 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000c7c0 tx=0x7f2cd0007e00 comp rx=0 tx=0).stop 2024-01-11T07:08:20.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.195+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 >> 172.21.15.7:0/2581603288 conn(0x7f2ce806e6c0 msgr2=0x7f2ce80b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:20.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.197+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 shutdown_connections 2024-01-11T07:08:20.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.197+0000 7f2ced021700 1 -- 172.21.15.7:0/2581603288 wait complete. 2024-01-11T07:08:20.208 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 6 and data pool 7 2024-01-11T07:08:20.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.355+0000 7ff644982700 1 Processor -- start 2024-01-11T07:08:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.356+0000 7ff644982700 1 -- start start 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.356+0000 7ff644982700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff6400cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.356+0000 7ff644982700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6400ab8d0 0x7ff64015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.356+0000 7ff644982700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64015e2d0 con 0x7ff64015d920 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.356+0000 7ff644982700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64006ac30 con 0x7ff6400ab8d0 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.356+0000 7ff644982700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400ac770 con 0x7ff6400ac3a0 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6400ab8d0 0x7ff64015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff635d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff64015d920 0x7ff64015a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40928/0 (socket says 172.21.15.7:40928) 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff635d9b700 1 -- 172.21.15.7:0/2987846894 learned_addr learned my addr 172.21.15.7:0/2987846894 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63e59c700 1 --2- 172.21.15.7:0/2987846894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff6400cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63ed9d700 1 -- 172.21.15.7:0/2987846894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 msgr2=0x7ff6400cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63ed9d700 1 --2- 172.21.15.7:0/2987846894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff6400cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63ed9d700 1 -- 172.21.15.7:0/2987846894 >> v1:172.21.15.7:6789/0 conn(0x7ff64015d920 legacy=0x7ff64015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63ed9d700 1 -- 172.21.15.7:0/2987846894 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64006a300 con 0x7ff6400ab8d0 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63dd9b700 1 -- 172.21.15.7:0/2987846894 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820784969 0 0) 0x7ff64015e2d0 con 0x7ff64015d920 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63ed9d700 1 --2- 172.21.15.7:0/2987846894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64015dd90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff638009e60 tx=0x7ff638009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de2778606a07f0a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63dd9b700 1 -- 172.21.15.7:0/2987846894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800e070 con 0x7ff6400ab8d0 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63dd9b700 1 -- 172.21.15.7:0/2987846894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63800bc30 con 0x7ff6400ab8d0 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.357+0000 7ff63dd9b700 1 -- 172.21.15.7:0/2987846894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638012430 con 0x7ff6400ab8d0 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.358+0000 7ff644982700 1 -- 172.21.15.7:0/2987846894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 msgr2=0x7ff64015dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.358+0000 7ff644982700 1 --2- 172.21.15.7:0/2987846894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64015dd90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff638009e60 tx=0x7ff638009780 comp rx=0 tx=0).stop 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.358+0000 7ff644982700 1 -- 172.21.15.7:0/2987846894 shutdown_connections 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.358+0000 7ff644982700 1 --2- 172.21.15.7:0/2987846894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64015dd90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.358+0000 7ff644982700 1 --2- 172.21.15.7:0/2987846894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff6400cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.358+0000 7ff644982700 1 -- 172.21.15.7:0/2987846894 >> 172.21.15.7:0/2987846894 conn(0x7ff6400cd090 msgr2=0x7ff6400cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.358+0000 7ff644982700 1 -- 172.21.15.7:0/2987846894 shutdown_connections 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.358+0000 7ff644982700 1 -- 172.21.15.7:0/2987846894 wait complete. 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.360+0000 7ff644982700 1 Processor -- start 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.360+0000 7ff644982700 1 -- start start 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff644982700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64016d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff644982700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff640140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff644982700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff64015d920 0x7ff6400ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff644982700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640164620 con 0x7ff6400ab8d0 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff644982700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64016dd00 con 0x7ff64015d920 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff644982700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64016dfd0 con 0x7ff6400ac3a0 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64016d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64016d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47084/0 (socket says 172.21.15.7:47084) 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 -- 172.21.15.7:0/3647803227 learned_addr learned my addr 172.21.15.7:0/3647803227 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff635d9b700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff640140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63ed9d700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff64015d920 0x7ff6400ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 -- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 msgr2=0x7ff640140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff640140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 -- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff64015d920 msgr2=0x7ff6400ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff64015d920 0x7ff6400ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 -- 172.21.15.7:0/3647803227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638009a20 con 0x7ff6400ab8d0 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff635d9b700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff640140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.361+0000 7ff63e59c700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64016d7c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff63000d860 tx=0x7ff630008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.362+0000 7ff636ffd700 1 -- 172.21.15.7:0/3647803227 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630009610 con 0x7ff6400ab8d0 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.362+0000 7ff636ffd700 1 -- 172.21.15.7:0/3647803227 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff630009770 con 0x7ff6400ab8d0 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.362+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64016e2b0 con 0x7ff6400ab8d0 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.362+0000 7ff636ffd700 1 -- 172.21.15.7:0/3647803227 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630009b30 con 0x7ff6400ab8d0 2024-01-11T07:08:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.362+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff640140600 con 0x7ff6400ab8d0 2024-01-11T07:08:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.363+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6400abe00 con 0x7ff6400ab8d0 2024-01-11T07:08:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.363+0000 7ff636ffd700 1 -- 172.21.15.7:0/3647803227 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff63001e460 con 0x7ff6400ab8d0 2024-01-11T07:08:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.364+0000 7ff636ffd700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.364+0000 7ff636ffd700 1 -- 172.21.15.7:0/3647803227 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff630084b50 con 0x7ff6400ab8d0 2024-01-11T07:08:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.364+0000 7ff635d9b700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.364+0000 7ff635d9b700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff640140550 tx=0x7ff628009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.368+0000 7ff636ffd700 1 -- 172.21.15.7:0/3647803227 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff630051dc0 con 0x7ff6400ab8d0 2024-01-11T07:08:20.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.571+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff64004c3c0 con 0x7ff6400ab8d0 2024-01-11T07:08:20.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.572+0000 7ff636ffd700 1 -- 172.21.15.7:0/3647803227 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15315 (secure 0 0 0) 0x7ff6300553e0 con 0x7ff6400ab8d0 2024-01-11T07:08:20.572 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:20.572 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:19.620503+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:08:17.698588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-11T07:08:18.590636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 msgr2=0x7ff6200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff640140550 tx=0x7ff628009040 comp rx=0 tx=0).stop 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 msgr2=0x7ff64016d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64016d7c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff63000d860 tx=0x7ff630008040 comp rx=0 tx=0).stop 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 shutdown_connections 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff620066f30 0x7ff6200693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff64015d920 0x7ff6400ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6400ab8d0 0x7ff64016d7c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 --2- 172.21.15.7:0/3647803227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6400ac3a0 0x7ff640140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 >> 172.21.15.7:0/3647803227 conn(0x7ff6400cd090 msgr2=0x7ff64006ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 shutdown_connections 2024-01-11T07:08:20.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.575+0000 7ff644982700 1 -- 172.21.15.7:0/3647803227 wait complete. 2024-01-11T07:08:20.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:20.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.721+0000 7f1c69bb9700 1 Processor -- start 2024-01-11T07:08:20.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.721+0000 7f1c69bb9700 1 -- start start 2024-01-11T07:08:20.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c69bb9700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c64155650 0x7f1c640ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c69bb9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 0x7f1c640b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c69bb9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c64063db0 0x7f1c640b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c69bb9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c640ad300 con 0x7f1c64063db0 2024-01-11T07:08:20.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c69bb9700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c64063920 con 0x7f1c64155650 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c69bb9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640ad020 con 0x7f1c64069d60 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c63fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c64063db0 0x7f1c640b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c63fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c64063db0 0x7f1c640b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47094/0 (socket says 172.21.15.7:47094) 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c63fff700 1 -- 172.21.15.7:0/957825224 learned_addr learned my addr 172.21.15.7:0/957825224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c62ffd700 1 --2- 172.21.15.7:0/957825224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 0x7f1c640b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c637fe700 1 --2- 172.21.15.7:0/957825224 >> v2:172.21.15.113:3300/0 conn(0x7f1c64155650 0x7f1c640ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c63fff700 1 -- 172.21.15.7:0/957825224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 msgr2=0x7f1c640b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.722+0000 7f1c63fff700 1 --2- 172.21.15.7:0/957825224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 0x7f1c640b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c63fff700 1 -- 172.21.15.7:0/957825224 >> v2:172.21.15.113:3300/0 conn(0x7f1c64155650 msgr2=0x7f1c640ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c63fff700 1 --2- 172.21.15.7:0/957825224 >> v2:172.21.15.113:3300/0 conn(0x7f1c64155650 0x7f1c640ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c63fff700 1 -- 172.21.15.7:0/957825224 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c640632d0 con 0x7f1c64063db0 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c63fff700 1 --2- 172.21.15.7:0/957825224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c640b30b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1c64068fa0 tx=0x7f1c5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e071b874124665e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c627fc700 1 -- 172.21.15.7:0/957825224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00e070 con 0x7f1c64063db0 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c627fc700 1 -- 172.21.15.7:0/957825224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c00b970 con 0x7f1c64063db0 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c627fc700 1 -- 172.21.15.7:0/957825224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0092e0 con 0x7f1c64063db0 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c69bb9700 1 -- 172.21.15.7:0/957825224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 msgr2=0x7f1c640b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/957825224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c640b30b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1c64068fa0 tx=0x7f1c5c009840 comp rx=0 tx=0).stop 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.723+0000 7f1c69bb9700 1 -- 172.21.15.7:0/957825224 shutdown_connections 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.724+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/957825224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c640b30b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.724+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/957825224 >> v2:172.21.15.113:3300/0 conn(0x7f1c64155650 0x7f1c640ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.724+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/957825224 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 0x7f1c640b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.724+0000 7f1c69bb9700 1 -- 172.21.15.7:0/957825224 >> 172.21.15.7:0/957825224 conn(0x7f1c640cd910 msgr2=0x7f1c640cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.724+0000 7f1c69bb9700 1 -- 172.21.15.7:0/957825224 shutdown_connections 2024-01-11T07:08:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.724+0000 7f1c69bb9700 1 -- 172.21.15.7:0/957825224 wait complete. 2024-01-11T07:08:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.725+0000 7f1c69bb9700 1 Processor -- start 2024-01-11T07:08:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.725+0000 7f1c69bb9700 1 -- start start 2024-01-11T07:08:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c69bb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c64160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c69bb9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 0x7f1c6415cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c69bb9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c64155650 0x7f1c6415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c69bb9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640a6ca0 con 0x7f1c64063db0 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c69bb9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c64154810 con 0x7f1c64155650 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c69bb9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6415fc20 con 0x7f1c64069d60 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c64160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c64160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47110/0 (socket says 172.21.15.7:47110) 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 -- 172.21.15.7:0/2596357689 learned_addr learned my addr 172.21.15.7:0/2596357689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c63fff700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c64155650 0x7f1c6415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c62ffd700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 0x7f1c6415cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 -- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 msgr2=0x7f1c6415cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 0x7f1c6415cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 -- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c64155650 msgr2=0x7f1c6415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c64155650 0x7f1c6415d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 -- 172.21.15.7:0/2596357689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c009ae0 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c637fe700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c64160140 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400e930 tx=0x7f1c540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.726+0000 7f1c4b7fe700 1 -- 172.21.15.7:0/2596357689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54013680 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.727+0000 7f1c4b7fe700 1 -- 172.21.15.7:0/2596357689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c540137e0 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.727+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c640a6650 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.727+0000 7f1c4b7fe700 1 -- 172.21.15.7:0/2596357689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54013b80 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.727+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6415f7b0 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.728+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c640a64b0 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.728+0000 7f1c4b7fe700 1 -- 172.21.15.7:0/2596357689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1c5400c3e0 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.728+0000 7f1c4b7fe700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c50066f30 0x7f1c500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.728+0000 7f1c4b7fe700 1 -- 172.21.15.7:0/2596357689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c540845d0 con 0x7f1c64063db0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.729+0000 7f1c62ffd700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c50066f30 0x7f1c500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:20.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.729+0000 7f1c62ffd700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c50066f30 0x7f1c500693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c64065280 tx=0x7f1c580091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:20.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.733+0000 7f1c4b7fe700 1 -- 172.21.15.7:0/2596357689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c640a64b0 con 0x7f1c64063db0 2024-01-11T07:08:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.938+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c640a64b0 con 0x7f1c64063db0 2024-01-11T07:08:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.939+0000 7f1c4b7fe700 1 -- 172.21.15.7:0/2596357689 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f1c640a64b0 con 0x7f1c64063db0 2024-01-11T07:08:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:20.939 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:19.620503+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:08:17.698588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-11T07:08:18.590636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c50066f30 msgr2=0x7f1c500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c50066f30 0x7f1c500693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c64065280 tx=0x7f1c580091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 msgr2=0x7f1c64160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c64160140 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400e930 tx=0x7f1c540091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 shutdown_connections 2024-01-11T07:08:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c50066f30 0x7f1c500693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c64155650 0x7f1c6415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c64063db0 0x7f1c64160140 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 --2- 172.21.15.7:0/2596357689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c64069d60 0x7f1c6415cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 >> 172.21.15.7:0/2596357689 conn(0x7f1c640cd910 msgr2=0x7f1c640cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 shutdown_connections 2024-01-11T07:08:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:20.941+0000 7f1c69bb9700 1 -- 172.21.15.7:0/2596357689 wait complete. 2024-01-11T07:08:20.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:08:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.090+0000 7fd26418d700 1 Processor -- start 2024-01-11T07:08:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.090+0000 7fd26418d700 1 -- start start 2024-01-11T07:08:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd26418d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd25c156320 0x7fd25c0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd26418d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c159400 0x7fd25c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd26418d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c157f60 con 0x7fd25c0b0350 2024-01-11T07:08:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd26418d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c1578b0 con 0x7fd25c156320 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd26418d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0b0980 con 0x7fd25c159400 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd261728700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd25c0b0350 0x7fd25c1566f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40942/0 (socket says 172.21.15.7:40942) 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd261728700 1 -- 172.21.15.7:0/518510711 learned_addr learned my addr 172.21.15.7:0/518510711 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd26272a700 1 --2- 172.21.15.7:0/518510711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c159400 0x7fd25c156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd261f29700 1 --2- 172.21.15.7:0/518510711 >> v2:172.21.15.113:3300/0 conn(0x7fd25c156320 0x7fd25c0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd260f27700 1 -- 172.21.15.7:0/518510711 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130450636 0 0) 0x7fd25c157f60 con 0x7fd25c0b0350 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd260f27700 1 -- 172.21.15.7:0/518510711 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd248003570 con 0x7fd25c0b0350 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd261f29700 1 -- 172.21.15.7:0/518510711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c159400 msgr2=0x7fd25c156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd261f29700 1 --2- 172.21.15.7:0/518510711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c159400 0x7fd25c156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd261f29700 1 -- 172.21.15.7:0/518510711 >> v1:172.21.15.7:6789/0 conn(0x7fd25c0b0350 legacy=0x7fd25c1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.091+0000 7fd261f29700 1 -- 172.21.15.7:0/518510711 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0b0c60 con 0x7fd25c156320 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.092+0000 7fd261f29700 1 --2- 172.21.15.7:0/518510711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c156320 0x7fd25c0afe10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2500098e0 tx=0x7fd2500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44e3862f577023d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.092+0000 7fd260f27700 1 -- 172.21.15.7:0/518510711 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250010600 con 0x7fd25c156320 2024-01-11T07:08:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.092+0000 7fd260f27700 1 -- 172.21.15.7:0/518510711 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250010760 con 0x7fd25c156320 2024-01-11T07:08:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.092+0000 7fd260f27700 1 -- 172.21.15.7:0/518510711 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250010b00 con 0x7fd25c156320 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.092+0000 7fd26418d700 1 -- 172.21.15.7:0/518510711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c156320 msgr2=0x7fd25c0afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.092+0000 7fd26418d700 1 --2- 172.21.15.7:0/518510711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c156320 0x7fd25c0afe10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2500098e0 tx=0x7fd2500091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.093+0000 7fd26418d700 1 -- 172.21.15.7:0/518510711 shutdown_connections 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.093+0000 7fd26418d700 1 --2- 172.21.15.7:0/518510711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c159400 0x7fd25c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.093+0000 7fd26418d700 1 --2- 172.21.15.7:0/518510711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c156320 0x7fd25c0afe10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.093+0000 7fd26418d700 1 -- 172.21.15.7:0/518510711 >> 172.21.15.7:0/518510711 conn(0x7fd25c0680a0 msgr2=0x7fd25c0cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.093+0000 7fd26418d700 1 -- 172.21.15.7:0/518510711 shutdown_connections 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.093+0000 7fd26418d700 1 -- 172.21.15.7:0/518510711 wait complete. 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.094+0000 7fd26418d700 1 Processor -- start 2024-01-11T07:08:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.094+0000 7fd26418d700 1 -- start start 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26418d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd25c0b0350 0x7fd25c0a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26418d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c156320 0x7fd25c140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26418d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c159400 0x7fd25c15d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26418d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0a84b0 con 0x7fd25c0b0350 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26418d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0a6a20 con 0x7fd25c159400 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26418d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0a6d00 con 0x7fd25c156320 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd25c0b0350 0x7fd25c0a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd25c0b0350 0x7fd25c0a7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47114/0 (socket says 172.21.15.7:47114) 2024-01-11T07:08:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 -- 172.21.15.7:0/2343174048 learned_addr learned my addr 172.21.15.7:0/2343174048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26272a700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c159400 0x7fd25c15d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261728700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c156320 0x7fd25c140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 -- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c156320 msgr2=0x7fd25c140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c156320 0x7fd25c140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 -- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c159400 msgr2=0x7fd25c15d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c159400 0x7fd25c15d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 -- 172.21.15.7:0/2343174048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250009420 con 0x7fd25c0b0350 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26272a700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c159400 0x7fd25c15d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261728700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c156320 0x7fd25c140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd261f29700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd25c0b0350 0x7fd25c0a7b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd25c064360 tx=0x7fd250018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd24dffb700 1 -- 172.21.15.7:0/2343174048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25001d070 con 0x7fd25c0b0350 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd24dffb700 1 -- 172.21.15.7:0/2343174048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250000bc0 con 0x7fd25c0b0350 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c0a6fe0 con 0x7fd25c0b0350 2024-01-11T07:08:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.095+0000 7fd24dffb700 1 -- 172.21.15.7:0/2343174048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250010b00 con 0x7fd25c0b0350 2024-01-11T07:08:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.096+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0a80d0 con 0x7fd25c0b0350 2024-01-11T07:08:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.097+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c04c3c0 con 0x7fd25c0b0350 2024-01-11T07:08:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.097+0000 7fd24dffb700 1 -- 172.21.15.7:0/2343174048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd250021070 con 0x7fd25c0b0350 2024-01-11T07:08:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.097+0000 7fd24dffb700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd240066f30 0x7fd2400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.098+0000 7fd24dffb700 1 -- 172.21.15.7:0/2343174048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2500856c0 con 0x7fd25c0b0350 2024-01-11T07:08:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.098+0000 7fd261728700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd240066f30 0x7fd2400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.098+0000 7fd261728700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd240066f30 0x7fd2400693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0aa1b0 tx=0x7fd258009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.102+0000 7fd24dffb700 1 -- 172.21.15.7:0/2343174048 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd250052930 con 0x7fd25c0b0350 2024-01-11T07:08:21.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:21.361+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd25c0b0f40 con 0x7fd25c0b0350 2024-01-11T07:08:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.210+0000 7fd24dffb700 1 -- 172.21.15.7:0/2343174048 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd250055f50 con 0x7fd25c0b0350 2024-01-11T07:08:22.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.212+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd240066f30 msgr2=0x7fd2400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:22.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd240066f30 0x7fd2400693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0aa1b0 tx=0x7fd258009040 comp rx=0 tx=0).stop 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd25c0b0350 msgr2=0x7fd25c0a7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd25c0b0350 0x7fd25c0a7b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd25c064360 tx=0x7fd250018000 comp rx=0 tx=0).stop 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 shutdown_connections 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd240066f30 0x7fd2400693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd25c159400 0x7fd25c15d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd25c0b0350 0x7fd25c0a7b90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 --2- 172.21.15.7:0/2343174048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd25c156320 0x7fd25c140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 >> 172.21.15.7:0/2343174048 conn(0x7fd25c0680a0 msgr2=0x7fd25c0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 shutdown_connections 2024-01-11T07:08:22.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.213+0000 7fd26418d700 1 -- 172.21.15.7:0/2343174048 wait complete. 2024-01-11T07:08:22.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:08:22.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.367+0000 7f99a0332700 1 Processor -- start 2024-01-11T07:08:22.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.368+0000 7f99a0332700 1 -- start start 2024-01-11T07:08:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f99a0332700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99980cdf10 0x7f99980cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f99a0332700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cbf00 0x7f99980cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f99a0332700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99980cee70 con 0x7f9998159ea0 2024-01-11T07:08:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f99a0332700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99980afee0 con 0x7f99980cdf10 2024-01-11T07:08:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f99a0332700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99980cc380 con 0x7f99980cbf00 2024-01-11T07:08:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999d8cd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99980cdf10 0x7f99980cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999e8cf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cbf00 0x7f99980cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999e0ce700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9998159ea0 0x7f99980af7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:46022/0 (socket says 172.21.15.7:46022) 2024-01-11T07:08:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999e0ce700 1 -- 172.21.15.7:0/946735796 learned_addr learned my addr 172.21.15.7:0/946735796 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999d0cc700 1 -- 172.21.15.7:0/946735796 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131951356 0 0) 0x7f99980cee70 con 0x7f9998159ea0 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999d0cc700 1 -- 172.21.15.7:0/946735796 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9984003570 con 0x7f9998159ea0 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999e8cf700 1 -- 172.21.15.7:0/946735796 >> v2:172.21.15.113:3300/0 conn(0x7f99980cdf10 msgr2=0x7f99980cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999e8cf700 1 --2- 172.21.15.7:0/946735796 >> v2:172.21.15.113:3300/0 conn(0x7f99980cdf10 0x7f99980cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999e8cf700 1 -- 172.21.15.7:0/946735796 >> v1:172.21.15.113:6789/0 conn(0x7f9998159ea0 legacy=0x7f99980af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999e8cf700 1 -- 172.21.15.7:0/946735796 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980ce860 con 0x7f99980cbf00 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.369+0000 7f999d8cd700 1 --2- 172.21.15.7:0/946735796 >> v2:172.21.15.113:3300/0 conn(0x7f99980cdf10 0x7f99980cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.370+0000 7f999e8cf700 1 --2- 172.21.15.7:0/946735796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cbf00 0x7f99980cf300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9990009e60 tx=0x7f9990009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de29698968fd529e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.370+0000 7f999d0cc700 1 -- 172.21.15.7:0/946735796 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999000e070 con 0x7f99980cbf00 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.370+0000 7f999d0cc700 1 -- 172.21.15.7:0/946735796 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999000bab0 con 0x7f99980cbf00 2024-01-11T07:08:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.370+0000 7f999d0cc700 1 -- 172.21.15.7:0/946735796 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999000be70 con 0x7f99980cbf00 2024-01-11T07:08:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.370+0000 7f99a0332700 1 -- 172.21.15.7:0/946735796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cbf00 msgr2=0x7f99980cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.370+0000 7f99a0332700 1 --2- 172.21.15.7:0/946735796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cbf00 0x7f99980cf300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9990009e60 tx=0x7f9990009780 comp rx=0 tx=0).stop 2024-01-11T07:08:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.371+0000 7f99a0332700 1 -- 172.21.15.7:0/946735796 shutdown_connections 2024-01-11T07:08:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.371+0000 7f99a0332700 1 --2- 172.21.15.7:0/946735796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cbf00 0x7f99980cf300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.371+0000 7f99a0332700 1 --2- 172.21.15.7:0/946735796 >> v2:172.21.15.113:3300/0 conn(0x7f99980cdf10 0x7f99980cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.371+0000 7f99a0332700 1 -- 172.21.15.7:0/946735796 >> 172.21.15.7:0/946735796 conn(0x7f9998071120 msgr2=0x7f999806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.371+0000 7f99a0332700 1 -- 172.21.15.7:0/946735796 shutdown_connections 2024-01-11T07:08:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.371+0000 7f99a0332700 1 -- 172.21.15.7:0/946735796 wait complete. 2024-01-11T07:08:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.372+0000 7f99a0332700 1 Processor -- start 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.372+0000 7f99a0332700 1 -- start start 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f99a0332700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99980cbf00 0x7f99980ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f99a0332700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cdf10 0x7f99981584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f99a0332700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159ea0 0x7f999815da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f99a0332700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998169ad0 con 0x7f9998159ea0 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f99a0332700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998169db0 con 0x7f99980cbf00 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f99a0332700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999816a090 con 0x7f99980cdf10 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159ea0 0x7f999815da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159ea0 0x7f999815da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47126/0 (socket says 172.21.15.7:47126) 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 -- 172.21.15.7:0/1318765486 learned_addr learned my addr 172.21.15.7:0/1318765486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e0ce700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99980cbf00 0x7f99980ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999d8cd700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cdf10 0x7f99981584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 -- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cdf10 msgr2=0x7f99981584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cdf10 0x7f99981584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 -- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99980cbf00 msgr2=0x7f99980ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99980cbf00 0x7f99980ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 -- 172.21.15.7:0/1318765486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994009420 con 0x7f9998159ea0 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.373+0000 7f999e8cf700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159ea0 0x7f999815da10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f999000b610 tx=0x7f9990019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.374+0000 7f998a7fc700 1 -- 172.21.15.7:0/1318765486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990012e10 con 0x7f9998159ea0 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.374+0000 7f998a7fc700 1 -- 172.21.15.7:0/1318765486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990027470 con 0x7f9998159ea0 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.374+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9990009a20 con 0x7f9998159ea0 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.374+0000 7f998a7fc700 1 -- 172.21.15.7:0/1318765486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990027830 con 0x7f9998159ea0 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.374+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99980ca930 con 0x7f9998159ea0 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.375+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999816a5b0 con 0x7f9998159ea0 2024-01-11T07:08:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.375+0000 7f998a7fc700 1 -- 172.21.15.7:0/1318765486 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9990024070 con 0x7f9998159ea0 2024-01-11T07:08:22.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.376+0000 7f998a7fc700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f997c066f60 0x7f997c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:22.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.376+0000 7f998a7fc700 1 -- 172.21.15.7:0/1318765486 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9990086bd0 con 0x7f9998159ea0 2024-01-11T07:08:22.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.376+0000 7f999e0ce700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f997c066f60 0x7f997c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:22.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.377+0000 7f999e0ce700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f997c066f60 0x7f997c069410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f998c007fd0 tx=0x7f998c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.380+0000 7f998a7fc700 1 -- 172.21.15.7:0/1318765486 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9990053e40 con 0x7f9998159ea0 2024-01-11T07:08:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:22.634+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f99980695e0 con 0x7f9998159ea0 2024-01-11T07:08:23.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.210+0000 7f998a7fc700 1 -- 172.21.15.7:0/1318765486 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f999002c0c0 con 0x7f9998159ea0 2024-01-11T07:08:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f997c066f60 msgr2=0x7f997c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f997c066f60 0x7f997c069410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f998c007fd0 tx=0x7f998c009040 comp rx=0 tx=0).stop 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159ea0 msgr2=0x7f999815da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159ea0 0x7f999815da10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f999000b610 tx=0x7f9990019000 comp rx=0 tx=0).stop 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 shutdown_connections 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f997c066f60 0x7f997c069410 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9998159ea0 0x7f999815da10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99980cbf00 0x7f99980ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 --2- 172.21.15.7:0/1318765486 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99980cdf10 0x7f99981584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.212+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 >> 172.21.15.7:0/1318765486 conn(0x7f9998071120 msgr2=0x7f999815be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.213+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 shutdown_connections 2024-01-11T07:08:23.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.213+0000 7f99a0332700 1 -- 172.21.15.7:0/1318765486 wait complete. 2024-01-11T07:08:23.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:23.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.364+0000 7f164ebf8700 1 Processor -- start 2024-01-11T07:08:23.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.364+0000 7f164ebf8700 1 -- start start 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.364+0000 7f164ebf8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1648159c90 0x7f16480b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.364+0000 7f164ebf8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16480ce9b0 con 0x7f164815a060 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.364+0000 7f164ebf8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16480cf530 con 0x7f1648069320 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.364+0000 7f164ebf8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16480cf9c0 con 0x7f1648159c90 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f164c994700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1648159c90 0x7f16480b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f1647fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f164815a060 0x7f16480b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40944/0 (socket says 172.21.15.7:40944) 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f1647fff700 1 -- 172.21.15.7:0/1084086304 learned_addr learned my addr 172.21.15.7:0/1084086304 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f164c994700 1 -- 172.21.15.7:0/1084086304 >> v1:172.21.15.113:6789/0 conn(0x7f1648069320 legacy=0x7f16480b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f164c994700 1 -- 172.21.15.7:0/1084086304 >> v1:172.21.15.7:6789/0 conn(0x7f164815a060 legacy=0x7f16480b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f16477fe700 1 -- 172.21.15.7:0/1084086304 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570163195 0 0) 0x7f16480ce9b0 con 0x7f164815a060 2024-01-11T07:08:23.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f164c994700 1 -- 172.21.15.7:0/1084086304 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16480cec00 con 0x7f1648159c90 2024-01-11T07:08:23.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f164c994700 1 --2- 172.21.15.7:0/1084086304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1648159c90 0x7f16480b1820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f163c0098e0 tx=0x7f163c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad89c3b2a8f9a84b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:23.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f16477fe700 1 -- 172.21.15.7:0/1084086304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c009e80 con 0x7f1648159c90 2024-01-11T07:08:23.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f16477fe700 1 -- 172.21.15.7:0/1084086304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c00b860 con 0x7f1648159c90 2024-01-11T07:08:23.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.365+0000 7f16477fe700 1 -- 172.21.15.7:0/1084086304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c00bbe0 con 0x7f1648159c90 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.366+0000 7f164ebf8700 1 -- 172.21.15.7:0/1084086304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1648159c90 msgr2=0x7f16480b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.366+0000 7f164ebf8700 1 --2- 172.21.15.7:0/1084086304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1648159c90 0x7f16480b1820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f163c0098e0 tx=0x7f163c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.366+0000 7f164ebf8700 1 -- 172.21.15.7:0/1084086304 shutdown_connections 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.366+0000 7f164ebf8700 1 --2- 172.21.15.7:0/1084086304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1648159c90 0x7f16480b1820 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.366+0000 7f164ebf8700 1 -- 172.21.15.7:0/1084086304 >> 172.21.15.7:0/1084086304 conn(0x7f1648071100 msgr2=0x7f164806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.366+0000 7f164ebf8700 1 -- 172.21.15.7:0/1084086304 shutdown_connections 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.366+0000 7f164ebf8700 1 -- 172.21.15.7:0/1084086304 wait complete. 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.367+0000 7f164ebf8700 1 Processor -- start 2024-01-11T07:08:23.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.368+0000 7f164ebf8700 1 -- start start 2024-01-11T07:08:23.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.368+0000 7f164ebf8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1648069320 0x7f164815e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.368+0000 7f164ebf8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1648159c90 0x7f16480b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.368+0000 7f164ebf8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164815a060 0x7f1648155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.368+0000 7f164ebf8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16480c9480 con 0x7f164815a060 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.368+0000 7f164ebf8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164816ada0 con 0x7f1648159c90 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.368+0000 7f164ebf8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1648169ed0 con 0x7f1648069320 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164815a060 0x7f1648155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164815a060 0x7f1648155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47152/0 (socket says 172.21.15.7:47152) 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 -- 172.21.15.7:0/998772145 learned_addr learned my addr 172.21.15.7:0/998772145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164c994700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1648069320 0x7f164815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f1647fff700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1648159c90 0x7f16480b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 -- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1648069320 msgr2=0x7f164815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1648069320 0x7f164815e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 -- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1648159c90 msgr2=0x7f16480b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1648159c90 0x7f16480b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 -- 172.21.15.7:0/998772145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c009420 con 0x7f164815a060 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164d195700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164815a060 0x7f1648155170 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f164000c840 tx=0x7f1640001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:23.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f1644ff9700 1 -- 172.21.15.7:0/998772145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640012630 con 0x7f164815a060 2024-01-11T07:08:23.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f1644ff9700 1 -- 172.21.15.7:0/998772145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16400127b0 con 0x7f164815a060 2024-01-11T07:08:23.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164816a190 con 0x7f164815a060 2024-01-11T07:08:23.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f1644ff9700 1 -- 172.21.15.7:0/998772145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640012b70 con 0x7f164815a060 2024-01-11T07:08:23.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.369+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164816aa30 con 0x7f164815a060 2024-01-11T07:08:23.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.370+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164816a590 con 0x7f164815a060 2024-01-11T07:08:23.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.371+0000 7f1644ff9700 1 -- 172.21.15.7:0/998772145 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f164001a3a0 con 0x7f164815a060 2024-01-11T07:08:23.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.371+0000 7f1644ff9700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f162c066f30 0x7f162c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.371+0000 7f1644ff9700 1 -- 172.21.15.7:0/998772145 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1640084610 con 0x7f164815a060 2024-01-11T07:08:23.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.372+0000 7f164c994700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f162c066f30 0x7f162c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.372+0000 7f164c994700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f162c066f30 0x7f162c0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f163c0093f0 tx=0x7f163c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.376+0000 7f1644ff9700 1 -- 172.21.15.7:0/998772145 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1640051880 con 0x7f164815a060 2024-01-11T07:08:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.625+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f164804c3c0 con 0x7f164815a060 2024-01-11T07:08:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.626+0000 7f1644ff9700 1 -- 172.21.15.7:0/998772145 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f1640054ea0 con 0x7f164815a060 2024-01-11T07:08:23.626 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:23.626 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f162c066f30 msgr2=0x7f162c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f162c066f30 0x7f162c0693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f163c0093f0 tx=0x7f163c017000 comp rx=0 tx=0).stop 2024-01-11T07:08:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164815a060 msgr2=0x7f1648155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164815a060 0x7f1648155170 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f164000c840 tx=0x7f1640001670 comp rx=0 tx=0).stop 2024-01-11T07:08:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 shutdown_connections 2024-01-11T07:08:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f162c066f30 0x7f162c0693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164815a060 0x7f1648155170 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1648069320 0x7f164815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 --2- 172.21.15.7:0/998772145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1648159c90 0x7f16480b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 >> 172.21.15.7:0/998772145 conn(0x7f1648071100 msgr2=0x7f16480b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 shutdown_connections 2024-01-11T07:08:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.628+0000 7f164ebf8700 1 -- 172.21.15.7:0/998772145 wait complete. 2024-01-11T07:08:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:23.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.773+0000 7f1d9a89d700 1 Processor -- start 2024-01-11T07:08:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.773+0000 7f1d9a89d700 1 -- start start 2024-01-11T07:08:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d9a89d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9406e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d9a89d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d94069c80 0x7f1d9406a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d9a89d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9406e3f0 con 0x7f1d9415b260 2024-01-11T07:08:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d9a89d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940b1d20 con 0x7f1d94069c80 2024-01-11T07:08:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d9a89d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9406a640 con 0x7f1d94156760 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d98e3a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d94069c80 0x7f1d9406a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d937fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1d9415b260 0x7f1d940b1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40954/0 (socket says 172.21.15.7:40954) 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d937fe700 1 -- 172.21.15.7:0/712311498 learned_addr learned my addr 172.21.15.7:0/712311498 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d93fff700 1 --2- 172.21.15.7:0/712311498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9406e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d98e3a700 1 -- 172.21.15.7:0/712311498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 msgr2=0x7f1d9406e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d98e3a700 1 --2- 172.21.15.7:0/712311498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9406e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d98e3a700 1 -- 172.21.15.7:0/712311498 >> v1:172.21.15.7:6789/0 conn(0x7f1d9415b260 legacy=0x7f1d940b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d98e3a700 1 -- 172.21.15.7:0/712311498 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94157b40 con 0x7f1d94069c80 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d93fff700 1 --2- 172.21.15.7:0/712311498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9406e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.774+0000 7f1d92ffd700 1 -- 172.21.15.7:0/712311498 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185668188 0 0) 0x7f1d9406e3f0 con 0x7f1d9415b260 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d98e3a700 1 --2- 172.21.15.7:0/712311498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d94069c80 0x7f1d9406a100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009e90 tx=0x7f1d8c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f455e442967fb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d92ffd700 1 -- 172.21.15.7:0/712311498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00e070 con 0x7f1d94069c80 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d92ffd700 1 -- 172.21.15.7:0/712311498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00bbf0 con 0x7f1d94069c80 2024-01-11T07:08:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d92ffd700 1 -- 172.21.15.7:0/712311498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c012430 con 0x7f1d94069c80 2024-01-11T07:08:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d9a89d700 1 -- 172.21.15.7:0/712311498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d94069c80 msgr2=0x7f1d9406a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/712311498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d94069c80 0x7f1d9406a100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009e90 tx=0x7f1d8c009780 comp rx=0 tx=0).stop 2024-01-11T07:08:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d9a89d700 1 -- 172.21.15.7:0/712311498 shutdown_connections 2024-01-11T07:08:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/712311498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d94069c80 0x7f1d9406a100 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/712311498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9406e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d9a89d700 1 -- 172.21.15.7:0/712311498 >> 172.21.15.7:0/712311498 conn(0x7f1d940cd910 msgr2=0x7f1d940cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d9a89d700 1 -- 172.21.15.7:0/712311498 shutdown_connections 2024-01-11T07:08:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.775+0000 7f1d9a89d700 1 -- 172.21.15.7:0/712311498 wait complete. 2024-01-11T07:08:23.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.777+0000 7f1d9a89d700 1 Processor -- start 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.777+0000 7f1d9a89d700 1 -- start start 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d9a89d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94069c80 0x7f1d940cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d9a89d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d9a89d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415b260 0x7f1d9414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d9a89d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9414e8c0 con 0x7f1d9415b260 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d9a89d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415cc50 con 0x7f1d94069c80 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d9a89d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9415ce90 con 0x7f1d94156760 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415b260 0x7f1d9414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415b260 0x7f1d9414d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47168/0 (socket says 172.21.15.7:47168) 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 -- 172.21.15.7:0/2944290830 learned_addr learned my addr 172.21.15.7:0/2944290830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d937fe700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d93fff700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94069c80 0x7f1d940cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 -- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 msgr2=0x7f1d9415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 -- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94069c80 msgr2=0x7f1d940cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94069c80 0x7f1d940cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 -- 172.21.15.7:0/2944290830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c009a20 con 0x7f1d9415b260 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.778+0000 7f1d98e3a700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415b260 0x7f1d9414d7d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009de0 tx=0x7f1d8c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.779+0000 7f1d8bfff700 1 -- 172.21.15.7:0/2944290830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c012e20 con 0x7f1d9415b260 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.779+0000 7f1d8bfff700 1 -- 172.21.15.7:0/2944290830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c01d630 con 0x7f1d9415b260 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.779+0000 7f1d8bfff700 1 -- 172.21.15.7:0/2944290830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c01d9f0 con 0x7f1d9415b260 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.779+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d940ca330 con 0x7f1d9415b260 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.779+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d940ca8f0 con 0x7f1d9415b260 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.780+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9404c3c0 con 0x7f1d9415b260 2024-01-11T07:08:23.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.780+0000 7f1d8bfff700 1 -- 172.21.15.7:0/2944290830 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1d8c024070 con 0x7f1d9415b260 2024-01-11T07:08:23.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.781+0000 7f1d8bfff700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d74066f30 0x7f1d740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:23.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.781+0000 7f1d8bfff700 1 -- 172.21.15.7:0/2944290830 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d8c086d80 con 0x7f1d9415b260 2024-01-11T07:08:23.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.781+0000 7f1d93fff700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d74066f30 0x7f1d740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:23.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.781+0000 7f1d93fff700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d74066f30 0x7f1d740693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1d84006f00 tx=0x7f1d84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:23.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.785+0000 7f1d8bfff700 1 -- 172.21.15.7:0/2944290830 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d8c053ff0 con 0x7f1d9415b260 2024-01-11T07:08:23.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.990+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d940caae0 con 0x7f1d9415b260 2024-01-11T07:08:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.991+0000 7f1d8bfff700 1 -- 172.21.15.7:0/2944290830 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15302 (secure 0 0 0) 0x7f1d8c057610 con 0x7f1d9415b260 2024-01-11T07:08:23.991 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:23.991 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":47,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:21.182993+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:08:17.698588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-11T07:08:18.590636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d74066f30 msgr2=0x7f1d740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d74066f30 0x7f1d740693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1d84006f00 tx=0x7f1d84008040 comp rx=0 tx=0).stop 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415b260 msgr2=0x7f1d9414d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415b260 0x7f1d9414d7d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009de0 tx=0x7f1d8c019000 comp rx=0 tx=0).stop 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 shutdown_connections 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d74066f30 0x7f1d740693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d9415b260 0x7f1d9414d7d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d94069c80 0x7f1d940cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 --2- 172.21.15.7:0/2944290830 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d94156760 0x7f1d9415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 >> 172.21.15.7:0/2944290830 conn(0x7f1d940cd910 msgr2=0x7f1d9415a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 shutdown_connections 2024-01-11T07:08:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:23.993+0000 7f1d9a89d700 1 -- 172.21.15.7:0/2944290830 wait complete. 2024-01-11T07:08:24.007 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:08:24.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:08:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.138+0000 7f699fbcb700 1 Processor -- start 2024-01-11T07:08:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.138+0000 7f699fbcb700 1 -- start start 2024-01-11T07:08:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.139+0000 7f699fbcb700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69980c0a70 0x7f6998161da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.139+0000 7f699fbcb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6998063940 0x7f69980c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.139+0000 7f699fbcb700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69980c83f0 con 0x7f69981622e0 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.139+0000 7f699fbcb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699815e740 con 0x7f6998063940 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.139+0000 7f699fbcb700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6998162780 con 0x7f69980c0a70 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.139+0000 7f699e168700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6998063940 0x7f69980c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699e168700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6998063940 0x7f69980c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47170/0 (socket says 172.21.15.7:47170) 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699e168700 1 -- 172.21.15.7:0/317234625 learned_addr learned my addr 172.21.15.7:0/317234625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699d967700 1 --2- 172.21.15.7:0/317234625 >> v2:172.21.15.113:3300/0 conn(0x7f69980c0a70 0x7f6998161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699e168700 1 -- 172.21.15.7:0/317234625 >> v2:172.21.15.113:3300/0 conn(0x7f69980c0a70 msgr2=0x7f6998161da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699e168700 1 --2- 172.21.15.7:0/317234625 >> v2:172.21.15.113:3300/0 conn(0x7f69980c0a70 0x7f6998161da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699e168700 1 -- 172.21.15.7:0/317234625 >> v1:172.21.15.113:6789/0 conn(0x7f69981622e0 legacy=0x7f6998063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699e168700 1 -- 172.21.15.7:0/317234625 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6998162a60 con 0x7f6998063940 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699e168700 1 --2- 172.21.15.7:0/317234625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6998063940 0x7f69980c8880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f699806d2e0 tx=0x7f6990009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abb910c9572a65e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699c965700 1 -- 172.21.15.7:0/317234625 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699000e070 con 0x7f6998063940 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699c965700 1 -- 172.21.15.7:0/317234625 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699000ba20 con 0x7f6998063940 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.140+0000 7f699c965700 1 -- 172.21.15.7:0/317234625 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69900092e0 con 0x7f6998063940 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.141+0000 7f699fbcb700 1 -- 172.21.15.7:0/317234625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6998063940 msgr2=0x7f69980c8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.141+0000 7f699fbcb700 1 --2- 172.21.15.7:0/317234625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6998063940 0x7f69980c8880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f699806d2e0 tx=0x7f6990009840 comp rx=0 tx=0).stop 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.141+0000 7f699fbcb700 1 -- 172.21.15.7:0/317234625 shutdown_connections 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.141+0000 7f699fbcb700 1 --2- 172.21.15.7:0/317234625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6998063940 0x7f69980c8880 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.141+0000 7f699fbcb700 1 --2- 172.21.15.7:0/317234625 >> v2:172.21.15.113:3300/0 conn(0x7f69980c0a70 0x7f6998161da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.141+0000 7f699fbcb700 1 -- 172.21.15.7:0/317234625 >> 172.21.15.7:0/317234625 conn(0x7f69980680e0 msgr2=0x7f69980cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.141+0000 7f699fbcb700 1 -- 172.21.15.7:0/317234625 shutdown_connections 2024-01-11T07:08:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.141+0000 7f699fbcb700 1 -- 172.21.15.7:0/317234625 wait complete. 2024-01-11T07:08:24.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.142+0000 7f699fbcb700 1 Processor -- start 2024-01-11T07:08:24.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.142+0000 7f699fbcb700 1 -- start start 2024-01-11T07:08:24.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699fbcb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6998063940 0x7f69980b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699fbcb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69980c0a70 0x7f699815d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699fbcb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69981622e0 0x7f69980acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699fbcb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699815d740 con 0x7f69980c0a70 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699fbcb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69980ad420 con 0x7f69981622e0 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699fbcb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980ad6f0 con 0x7f6998063940 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d166700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69980c0a70 0x7f699815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d166700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69980c0a70 0x7f699815d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47176/0 (socket says 172.21.15.7:47176) 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d166700 1 -- 172.21.15.7:0/1237234156 learned_addr learned my addr 172.21.15.7:0/1237234156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d967700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6998063940 0x7f69980b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699e168700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69981622e0 0x7f69980acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d166700 1 -- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6998063940 msgr2=0x7f69980b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d166700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6998063940 0x7f69980b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d166700 1 -- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69981622e0 msgr2=0x7f69980acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d166700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69981622e0 0x7f69980acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.143+0000 7f699d166700 1 -- 172.21.15.7:0/1237234156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6990009ae0 con 0x7f69980c0a70 2024-01-11T07:08:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.144+0000 7f699d166700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69980c0a70 0x7f699815d200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f698800c4e0 tx=0x7f698800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.144+0000 7f698dffb700 1 -- 172.21.15.7:0/1237234156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988011590 con 0x7f69980c0a70 2024-01-11T07:08:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.144+0000 7f698dffb700 1 -- 172.21.15.7:0/1237234156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6988011710 con 0x7f69980c0a70 2024-01-11T07:08:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.144+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69980ad9d0 con 0x7f69980c0a70 2024-01-11T07:08:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.144+0000 7f698dffb700 1 -- 172.21.15.7:0/1237234156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988011ad0 con 0x7f69980c0a70 2024-01-11T07:08:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.144+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69980adf90 con 0x7f69980c0a70 2024-01-11T07:08:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.145+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699804c3c0 con 0x7f69980c0a70 2024-01-11T07:08:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.146+0000 7f698dffb700 1 -- 172.21.15.7:0/1237234156 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f698801b3b0 con 0x7f69980c0a70 2024-01-11T07:08:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.146+0000 7f698dffb700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f697c066f30 0x7f697c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.147+0000 7f698dffb700 1 -- 172.21.15.7:0/1237234156 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f69880838c0 con 0x7f69980c0a70 2024-01-11T07:08:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.147+0000 7f699d967700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f697c066f30 0x7f697c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.147+0000 7f699d967700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f697c066f30 0x7f697c0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6994009db0 tx=0x7f69940091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.151+0000 7f698dffb700 1 -- 172.21.15.7:0/1237234156 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6988050b30 con 0x7f69980c0a70 2024-01-11T07:08:24.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.351+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69980c8080 con 0x7f697c066f30 2024-01-11T07:08:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.360+0000 7f698dffb700 1 -- 172.21.15.7:0/1237234156 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284757 (secure 0 0 0) 0x7f69980c8080 con 0x7f697c066f30 2024-01-11T07:08:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f697c066f30 msgr2=0x7f697c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f697c066f30 0x7f697c0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6994009db0 tx=0x7f69940091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69980c0a70 msgr2=0x7f699815d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69980c0a70 0x7f699815d200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f698800c4e0 tx=0x7f698800c7f0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 shutdown_connections 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f697c066f30 0x7f697c0693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69981622e0 0x7f69980acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6998063940 0x7f69980b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 --2- 172.21.15.7:0/1237234156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69980c0a70 0x7f699815d200 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 >> 172.21.15.7:0/1237234156 conn(0x7f69980680e0 msgr2=0x7f699815e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 shutdown_connections 2024-01-11T07:08:24.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.363+0000 7f699fbcb700 1 -- 172.21.15.7:0/1237234156 wait complete. 2024-01-11T07:08:24.368 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:08:24.377 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-01-11T07:08:23.858376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559597,"statfs":{"total":1600000000,"available":1597027332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.682400"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194645+0000","last_change":"2024-01-11T07:08:20.194581+0000","last_active":"2024-01-11T07:08:20.194645+0000","last_peered":"2024-01-11T07:08:20.194645+0000","last_clean":"2024-01-11T07:08:20.194645+0000","last_became_active":"2024-01-11T07:08:20.194200+0000","last_became_peered":"2024-01-11T07:08:20.194200+0000","last_unstale":"2024-01-11T07:08:20.194645+0000","last_undegraded":"2024-01-11T07:08:20.194645+0000","last_fullsized":"2024-01-11T07:08:20.194645+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:09:31.114790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.190369+0000","last_change":"2024-01-11T07:08:20.190291+0000","last_active":"2024-01-11T07:08:20.190369+0000","last_peered":"2024-01-11T07:08:20.190369+0000","last_clean":"2024-01-11T07:08:20.190369+0000","last_became_active":"2024-01-11T07:08:20.189763+0000","last_became_peered":"2024-01-11T07:08:20.189763+0000","last_unstale":"2024-01-11T07:08:20.190369+0000","last_undegraded":"2024-01-11T07:08:20.190369+0000","last_fullsized":"2024-01-11T07:08:20.190369+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:30:54.201577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197148+0000","last_change":"2024-01-11T07:08:20.197066+0000","last_active":"2024-01-11T07:08:20.197148+0000","last_peered":"2024-01-11T07:08:20.197148+0000","last_clean":"2024-01-11T07:08:20.197148+0000","last_became_active":"2024-01-11T07:08:20.196788+0000","last_became_peered":"2024-01-11T07:08:20.196788+0000","last_unstale":"2024-01-11T07:08:20.197148+0000","last_undegraded":"2024-01-11T07:08:20.197148+0000","last_fullsized":"2024-01-11T07:08:20.197148+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:34.002120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.193402+0000","last_change":"2024-01-11T07:08:20.193273+0000","last_active":"2024-01-11T07:08:20.193402+0000","last_peered":"2024-01-11T07:08:20.193402+0000","last_clean":"2024-01-11T07:08:20.193402+0000","last_became_active":"2024-01-11T07:08:20.192972+0000","last_became_peered":"2024-01-11T07:08:20.192972+0000","last_unstale":"2024-01-11T07:08:20.193402+0000","last_undegraded":"2024-01-11T07:08:20.193402+0000","last_fullsized":"2024-01-11T07:08:20.193402+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:05:37.421591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194894+0000","last_change":"2024-01-11T07:08:20.194801+0000","last_active":"2024-01-11T07:08:20.194894+0000","last_peered":"2024-01-11T07:08:20.194894+0000","last_clean":"2024-01-11T07:08:20.194894+0000","last_became_active":"2024-01-11T07:08:20.194195+0000","last_became_peered":"2024-01-11T07:08:20.194195+0000","last_unstale":"2024-01-11T07:08:20.194894+0000","last_undegraded":"2024-01-11T07:08:20.194894+0000","last_fullsized":"2024-01-11T07:08:20.194894+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:05:37.421591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207326+0000","last_change":"2024-01-11T07:08:20.207120+0000","last_active":"2024-01-11T07:08:20.207326+0000","last_peered":"2024-01-11T07:08:20.207326+0000","last_clean":"2024-01-11T07:08:20.207326+0000","last_became_active":"2024-01-11T07:08:20.205327+0000","last_became_peered":"2024-01-11T07:08:20.205327+0000","last_unstale":"2024-01-11T07:08:20.207326+0000","last_undegraded":"2024-01-11T07:08:20.207326+0000","last_fullsized":"2024-01-11T07:08:20.207326+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:53:51.094526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.200615+0000","last_change":"2024-01-11T07:08:20.200548+0000","last_active":"2024-01-11T07:08:20.200615+0000","last_peered":"2024-01-11T07:08:20.200615+0000","last_clean":"2024-01-11T07:08:20.200615+0000","last_became_active":"2024-01-11T07:08:20.200343+0000","last_became_peered":"2024-01-11T07:08:20.200343+0000","last_unstale":"2024-01-11T07:08:20.200615+0000","last_undegraded":"2024-01-11T07:08:20.200615+0000","last_fullsized":"2024-01-11T07:08:20.200615+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:30:54.201577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192903+0000","last_change":"2024-01-11T07:08:20.192773+0000","last_active":"2024-01-11T07:08:20.192903+0000","last_peered":"2024-01-11T07:08:20.192903+0000","last_clean":"2024-01-11T07:08:20.192903+0000","last_became_active":"2024-01-11T07:08:20.192516+0000","last_became_peered":"2024-01-11T07:08:20.192516+0000","last_unstale":"2024-01-11T07:08:20.192903+0000","last_undegraded":"2024-01-11T07:08:20.192903+0000","last_fullsized":"2024-01-11T07:08:20.192903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:09:31.114790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205794+0000","last_change":"2024-01-11T07:08:20.205692+0000","last_active":"2024-01-11T07:08:20.205794+0000","last_peered":"2024-01-11T07:08:20.205794+0000","last_clean":"2024-01-11T07:08:20.205794+0000","last_became_active":"2024-01-11T07:08:20.205243+0000","last_became_peered":"2024-01-11T07:08:20.205243+0000","last_unstale":"2024-01-11T07:08:20.205794+0000","last_undegraded":"2024-01-11T07:08:20.205794+0000","last_fullsized":"2024-01-11T07:08:20.205794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:48.277623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192864+0000","last_change":"2024-01-11T07:08:20.192498+0000","last_active":"2024-01-11T07:08:20.192864+0000","last_peered":"2024-01-11T07:08:20.192864+0000","last_clean":"2024-01-11T07:08:20.192864+0000","last_became_active":"2024-01-11T07:08:20.191901+0000","last_became_peered":"2024-01-11T07:08:20.191901+0000","last_unstale":"2024-01-11T07:08:20.192864+0000","last_undegraded":"2024-01-11T07:08:20.192864+0000","last_fullsized":"2024-01-11T07:08:20.192864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:30:54.201577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201634+0000","last_change":"2024-01-11T07:08:20.201535+0000","last_active":"2024-01-11T07:08:20.201634+0000","last_peered":"2024-01-11T07:08:20.201634+0000","last_clean":"2024-01-11T07:08:20.201634+0000","last_became_active":"2024-01-11T07:08:20.201262+0000","last_became_peered":"2024-01-11T07:08:20.201262+0000","last_unstale":"2024-01-11T07:08:20.201634+0000","last_undegraded":"2024-01-11T07:08:20.201634+0000","last_fullsized":"2024-01-11T07:08:20.201634+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:34.002120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196233+0000","last_change":"2024-01-11T07:08:20.196164+0000","last_active":"2024-01-11T07:08:20.196233+0000","last_peered":"2024-01-11T07:08:20.196233+0000","last_clean":"2024-01-11T07:08:20.196233+0000","last_became_active":"2024-01-11T07:08:20.195903+0000","last_became_peered":"2024-01-11T07:08:20.195903+0000","last_unstale":"2024-01-11T07:08:20.196233+0000","last_undegraded":"2024-01-11T07:08:20.196233+0000","last_fullsized":"2024-01-11T07:08:20.196233+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:03.517036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197772+0000","last_change":"2024-01-11T07:08:20.197103+0000","last_active":"2024-01-11T07:08:20.197772+0000","last_peered":"2024-01-11T07:08:20.197772+0000","last_clean":"2024-01-11T07:08:20.197772+0000","last_became_active":"2024-01-11T07:08:20.196823+0000","last_became_peered":"2024-01-11T07:08:20.196823+0000","last_unstale":"2024-01-11T07:08:20.197772+0000","last_undegraded":"2024-01-11T07:08:20.197772+0000","last_fullsized":"2024-01-11T07:08:20.197772+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:42:18.426535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205772+0000","last_change":"2024-01-11T07:08:20.205508+0000","last_active":"2024-01-11T07:08:20.205772+0000","last_peered":"2024-01-11T07:08:20.205772+0000","last_clean":"2024-01-11T07:08:20.205772+0000","last_became_active":"2024-01-11T07:08:20.204921+0000","last_became_peered":"2024-01-11T07:08:20.204921+0000","last_unstale":"2024-01-11T07:08:20.205772+0000","last_undegraded":"2024-01-11T07:08:20.205772+0000","last_fullsized":"2024-01-11T07:08:20.205772+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:48:10.992014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.195123+0000","last_change":"2024-01-11T07:08:20.194376+0000","last_active":"2024-01-11T07:08:20.195123+0000","last_peered":"2024-01-11T07:08:20.195123+0000","last_clean":"2024-01-11T07:08:20.195123+0000","last_became_active":"2024-01-11T07:08:20.194135+0000","last_became_peered":"2024-01-11T07:08:20.194135+0000","last_unstale":"2024-01-11T07:08:20.195123+0000","last_undegraded":"2024-01-11T07:08:20.195123+0000","last_fullsized":"2024-01-11T07:08:20.195123+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:39:22.741253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197937+0000","last_change":"2024-01-11T07:08:20.197876+0000","last_active":"2024-01-11T07:08:20.197937+0000","last_peered":"2024-01-11T07:08:20.197937+0000","last_clean":"2024-01-11T07:08:20.197937+0000","last_became_active":"2024-01-11T07:08:20.197494+0000","last_became_peered":"2024-01-11T07:08:20.197494+0000","last_unstale":"2024-01-11T07:08:20.197937+0000","last_undegraded":"2024-01-11T07:08:20.197937+0000","last_fullsized":"2024-01-11T07:08:20.197937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:22.022631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189025+0000","last_change":"2024-01-11T07:08:20.188895+0000","last_active":"2024-01-11T07:08:20.189025+0000","last_peered":"2024-01-11T07:08:20.189025+0000","last_clean":"2024-01-11T07:08:20.189025+0000","last_became_active":"2024-01-11T07:08:20.188507+0000","last_became_peered":"2024-01-11T07:08:20.188507+0000","last_unstale":"2024-01-11T07:08:20.189025+0000","last_undegraded":"2024-01-11T07:08:20.189025+0000","last_fullsized":"2024-01-11T07:08:20.189025+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:50:49.026546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198775+0000","last_change":"2024-01-11T07:08:20.198582+0000","last_active":"2024-01-11T07:08:20.198775+0000","last_peered":"2024-01-11T07:08:20.198775+0000","last_clean":"2024-01-11T07:08:20.198775+0000","last_became_active":"2024-01-11T07:08:20.198342+0000","last_became_peered":"2024-01-11T07:08:20.198342+0000","last_unstale":"2024-01-11T07:08:20.198775+0000","last_undegraded":"2024-01-11T07:08:20.198775+0000","last_fullsized":"2024-01-11T07:08:20.198775+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:03:35.170409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.195874+0000","last_change":"2024-01-11T07:08:20.195724+0000","last_active":"2024-01-11T07:08:20.195874+0000","last_peered":"2024-01-11T07:08:20.195874+0000","last_clean":"2024-01-11T07:08:20.195874+0000","last_became_active":"2024-01-11T07:08:20.195137+0000","last_became_peered":"2024-01-11T07:08:20.195137+0000","last_unstale":"2024-01-11T07:08:20.195874+0000","last_undegraded":"2024-01-11T07:08:20.195874+0000","last_fullsized":"2024-01-11T07:08:20.195874+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:01:20.083688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198699+0000","last_change":"2024-01-11T07:08:20.198637+0000","last_active":"2024-01-11T07:08:20.198699+0000","last_peered":"2024-01-11T07:08:20.198699+0000","last_clean":"2024-01-11T07:08:20.198699+0000","last_became_active":"2024-01-11T07:08:20.198393+0000","last_became_peered":"2024-01-11T07:08:20.198393+0000","last_unstale":"2024-01-11T07:08:20.198699+0000","last_undegraded":"2024-01-11T07:08:20.198699+0000","last_fullsized":"2024-01-11T07:08:20.198699+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:21:03.453181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207309+0000","last_change":"2024-01-11T07:08:20.206298+0000","last_active":"2024-01-11T07:08:20.207309+0000","last_peered":"2024-01-11T07:08:20.207309+0000","last_clean":"2024-01-11T07:08:20.207309+0000","last_became_active":"2024-01-11T07:08:20.201918+0000","last_became_peered":"2024-01-11T07:08:20.201918+0000","last_unstale":"2024-01-11T07:08:20.207309+0000","last_undegraded":"2024-01-11T07:08:20.207309+0000","last_fullsized":"2024-01-11T07:08:20.207309+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:43:35.757837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194507+0000","last_change":"2024-01-11T07:08:20.194403+0000","last_active":"2024-01-11T07:08:20.194507+0000","last_peered":"2024-01-11T07:08:20.194507+0000","last_clean":"2024-01-11T07:08:20.194507+0000","last_became_active":"2024-01-11T07:08:20.193864+0000","last_became_peered":"2024-01-11T07:08:20.193864+0000","last_unstale":"2024-01-11T07:08:20.194507+0000","last_undegraded":"2024-01-11T07:08:20.194507+0000","last_fullsized":"2024-01-11T07:08:20.194507+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:03.517036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194722+0000","last_change":"2024-01-11T07:08:20.194666+0000","last_active":"2024-01-11T07:08:20.194722+0000","last_peered":"2024-01-11T07:08:20.194722+0000","last_clean":"2024-01-11T07:08:20.194722+0000","last_became_active":"2024-01-11T07:08:20.194470+0000","last_became_peered":"2024-01-11T07:08:20.194470+0000","last_unstale":"2024-01-11T07:08:20.194722+0000","last_undegraded":"2024-01-11T07:08:20.194722+0000","last_fullsized":"2024-01-11T07:08:20.194722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:48.860647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.195930+0000","last_change":"2024-01-11T07:08:20.195875+0000","last_active":"2024-01-11T07:08:20.195930+0000","last_peered":"2024-01-11T07:08:20.195930+0000","last_clean":"2024-01-11T07:08:20.195930+0000","last_became_active":"2024-01-11T07:08:20.195527+0000","last_became_peered":"2024-01-11T07:08:20.195527+0000","last_unstale":"2024-01-11T07:08:20.195930+0000","last_undegraded":"2024-01-11T07:08:20.195930+0000","last_fullsized":"2024-01-11T07:08:20.195930+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:01.586031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201300+0000","last_change":"2024-01-11T07:08:20.201146+0000","last_active":"2024-01-11T07:08:20.201300+0000","last_peered":"2024-01-11T07:08:20.201300+0000","last_clean":"2024-01-11T07:08:20.201300+0000","last_became_active":"2024-01-11T07:08:20.200921+0000","last_became_peered":"2024-01-11T07:08:20.200921+0000","last_unstale":"2024-01-11T07:08:20.201300+0000","last_undegraded":"2024-01-11T07:08:20.201300+0000","last_fullsized":"2024-01-11T07:08:20.201300+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:59:26.717733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.208920+0000","last_change":"2024-01-11T07:08:20.208547+0000","last_active":"2024-01-11T07:08:20.208920+0000","last_peered":"2024-01-11T07:08:20.208920+0000","last_clean":"2024-01-11T07:08:20.208920+0000","last_became_active":"2024-01-11T07:08:20.208162+0000","last_became_peered":"2024-01-11T07:08:20.208162+0000","last_unstale":"2024-01-11T07:08:20.208920+0000","last_undegraded":"2024-01-11T07:08:20.208920+0000","last_fullsized":"2024-01-11T07:08:20.208920+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:34.414872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189007+0000","last_change":"2024-01-11T07:08:20.188720+0000","last_active":"2024-01-11T07:08:20.189007+0000","last_peered":"2024-01-11T07:08:20.189007+0000","last_clean":"2024-01-11T07:08:20.189007+0000","last_became_active":"2024-01-11T07:08:20.188228+0000","last_became_peered":"2024-01-11T07:08:20.188228+0000","last_unstale":"2024-01-11T07:08:20.189007+0000","last_undegraded":"2024-01-11T07:08:20.189007+0000","last_fullsized":"2024-01-11T07:08:20.189007+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:07:00.866101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194876+0000","last_change":"2024-01-11T07:08:20.194725+0000","last_active":"2024-01-11T07:08:20.194876+0000","last_peered":"2024-01-11T07:08:20.194876+0000","last_clean":"2024-01-11T07:08:20.194876+0000","last_became_active":"2024-01-11T07:08:20.194486+0000","last_became_peered":"2024-01-11T07:08:20.194486+0000","last_unstale":"2024-01-11T07:08:20.194876+0000","last_undegraded":"2024-01-11T07:08:20.194876+0000","last_fullsized":"2024-01-11T07:08:20.194876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:18:35.823955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203163+0000","last_change":"2024-01-11T07:08:20.203093+0000","last_active":"2024-01-11T07:08:20.203163+0000","last_peered":"2024-01-11T07:08:20.203163+0000","last_clean":"2024-01-11T07:08:20.203163+0000","last_became_active":"2024-01-11T07:08:20.202833+0000","last_became_peered":"2024-01-11T07:08:20.202833+0000","last_unstale":"2024-01-11T07:08:20.203163+0000","last_undegraded":"2024-01-11T07:08:20.203163+0000","last_fullsized":"2024-01-11T07:08:20.203163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:39:51.189000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194126+0000","last_change":"2024-01-11T07:08:20.193537+0000","last_active":"2024-01-11T07:08:20.194126+0000","last_peered":"2024-01-11T07:08:20.194126+0000","last_clean":"2024-01-11T07:08:20.194126+0000","last_became_active":"2024-01-11T07:08:20.193316+0000","last_became_peered":"2024-01-11T07:08:20.193316+0000","last_unstale":"2024-01-11T07:08:20.194126+0000","last_undegraded":"2024-01-11T07:08:20.194126+0000","last_fullsized":"2024-01-11T07:08:20.194126+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:31.579570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207290+0000","last_change":"2024-01-11T07:08:20.205602+0000","last_active":"2024-01-11T07:08:20.207290+0000","last_peered":"2024-01-11T07:08:20.207290+0000","last_clean":"2024-01-11T07:08:20.207290+0000","last_became_active":"2024-01-11T07:08:20.201609+0000","last_became_peered":"2024-01-11T07:08:20.201609+0000","last_unstale":"2024-01-11T07:08:20.207290+0000","last_undegraded":"2024-01-11T07:08:20.207290+0000","last_fullsized":"2024-01-11T07:08:20.207290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:24:38.219798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.204605+0000","last_change":"2024-01-11T07:08:20.204488+0000","last_active":"2024-01-11T07:08:20.204605+0000","last_peered":"2024-01-11T07:08:20.204605+0000","last_clean":"2024-01-11T07:08:20.204605+0000","last_became_active":"2024-01-11T07:08:20.204253+0000","last_became_peered":"2024-01-11T07:08:20.204253+0000","last_unstale":"2024-01-11T07:08:20.204605+0000","last_undegraded":"2024-01-11T07:08:20.204605+0000","last_fullsized":"2024-01-11T07:08:20.204605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:47:01.654557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201752+0000","last_change":"2024-01-11T07:08:20.201661+0000","last_active":"2024-01-11T07:08:20.201752+0000","last_peered":"2024-01-11T07:08:20.201752+0000","last_clean":"2024-01-11T07:08:20.201752+0000","last_became_active":"2024-01-11T07:08:20.200942+0000","last_became_peered":"2024-01-11T07:08:20.200942+0000","last_unstale":"2024-01-11T07:08:20.201752+0000","last_undegraded":"2024-01-11T07:08:20.201752+0000","last_fullsized":"2024-01-11T07:08:20.201752+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:31.579570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207444+0000","last_change":"2024-01-11T07:08:19.190727+0000","last_active":"2024-01-11T07:08:20.207444+0000","last_peered":"2024-01-11T07:08:20.207444+0000","last_clean":"2024-01-11T07:08:20.207444+0000","last_became_active":"2024-01-11T07:08:19.190254+0000","last_became_peered":"2024-01-11T07:08:19.190254+0000","last_unstale":"2024-01-11T07:08:20.207444+0000","last_undegraded":"2024-01-11T07:08:20.207444+0000","last_fullsized":"2024-01-11T07:08:20.207444+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:06:59.864072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194340+0000","last_change":"2024-01-11T07:08:20.194287+0000","last_active":"2024-01-11T07:08:20.194340+0000","last_peered":"2024-01-11T07:08:20.194340+0000","last_clean":"2024-01-11T07:08:20.194340+0000","last_became_active":"2024-01-11T07:08:20.193701+0000","last_became_peered":"2024-01-11T07:08:20.193701+0000","last_unstale":"2024-01-11T07:08:20.194340+0000","last_undegraded":"2024-01-11T07:08:20.194340+0000","last_fullsized":"2024-01-11T07:08:20.194340+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:27:16.713517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205515+0000","last_change":"2024-01-11T07:08:19.193499+0000","last_active":"2024-01-11T07:08:20.205515+0000","last_peered":"2024-01-11T07:08:20.205515+0000","last_clean":"2024-01-11T07:08:20.205515+0000","last_became_active":"2024-01-11T07:08:19.193251+0000","last_became_peered":"2024-01-11T07:08:19.193251+0000","last_unstale":"2024-01-11T07:08:20.205515+0000","last_undegraded":"2024-01-11T07:08:20.205515+0000","last_fullsized":"2024-01-11T07:08:20.205515+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:36.551051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.202722+0000","last_change":"2024-01-11T07:08:20.202656+0000","last_active":"2024-01-11T07:08:20.202722+0000","last_peered":"2024-01-11T07:08:20.202722+0000","last_clean":"2024-01-11T07:08:20.202722+0000","last_became_active":"2024-01-11T07:08:20.202296+0000","last_became_peered":"2024-01-11T07:08:20.202296+0000","last_unstale":"2024-01-11T07:08:20.202722+0000","last_undegraded":"2024-01-11T07:08:20.202722+0000","last_fullsized":"2024-01-11T07:08:20.202722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:55.269737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.206864+0000","last_change":"2024-01-11T07:08:19.195656+0000","last_active":"2024-01-11T07:08:20.206864+0000","last_peered":"2024-01-11T07:08:20.206864+0000","last_clean":"2024-01-11T07:08:20.206864+0000","last_became_active":"2024-01-11T07:08:19.195322+0000","last_became_peered":"2024-01-11T07:08:19.195322+0000","last_unstale":"2024-01-11T07:08:20.206864+0000","last_undegraded":"2024-01-11T07:08:20.206864+0000","last_fullsized":"2024-01-11T07:08:20.206864+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:59:25.715704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201243+0000","last_change":"2024-01-11T07:08:20.201115+0000","last_active":"2024-01-11T07:08:20.201243+0000","last_peered":"2024-01-11T07:08:20.201243+0000","last_clean":"2024-01-11T07:08:20.201243+0000","last_became_active":"2024-01-11T07:08:20.200690+0000","last_became_peered":"2024-01-11T07:08:20.200690+0000","last_unstale":"2024-01-11T07:08:20.201243+0000","last_undegraded":"2024-01-11T07:08:20.201243+0000","last_fullsized":"2024-01-11T07:08:20.201243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:01:20.083688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203421+0000","last_change":"2024-01-11T07:08:19.194071+0000","last_active":"2024-01-11T07:08:20.203421+0000","last_peered":"2024-01-11T07:08:20.203421+0000","last_clean":"2024-01-11T07:08:20.203421+0000","last_became_active":"2024-01-11T07:08:19.193826+0000","last_became_peered":"2024-01-11T07:08:19.193826+0000","last_unstale":"2024-01-11T07:08:20.203421+0000","last_undegraded":"2024-01-11T07:08:20.203421+0000","last_fullsized":"2024-01-11T07:08:20.203421+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:06:59.864072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.193411+0000","last_change":"2024-01-11T07:08:20.192206+0000","last_active":"2024-01-11T07:08:20.193411+0000","last_peered":"2024-01-11T07:08:20.193411+0000","last_clean":"2024-01-11T07:08:20.193411+0000","last_became_active":"2024-01-11T07:08:20.191989+0000","last_became_peered":"2024-01-11T07:08:20.191989+0000","last_unstale":"2024-01-11T07:08:20.193411+0000","last_undegraded":"2024-01-11T07:08:20.193411+0000","last_fullsized":"2024-01-11T07:08:20.193411+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:02:16.481219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207420+0000","last_change":"2024-01-11T07:08:19.190068+0000","last_active":"2024-01-11T07:08:20.207420+0000","last_peered":"2024-01-11T07:08:20.207420+0000","last_clean":"2024-01-11T07:08:20.207420+0000","last_became_active":"2024-01-11T07:08:19.189360+0000","last_became_peered":"2024-01-11T07:08:19.189360+0000","last_unstale":"2024-01-11T07:08:20.207420+0000","last_undegraded":"2024-01-11T07:08:20.207420+0000","last_fullsized":"2024-01-11T07:08:20.207420+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:15:57.438193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197093+0000","last_change":"2024-01-11T07:08:20.196667+0000","last_active":"2024-01-11T07:08:20.197093+0000","last_peered":"2024-01-11T07:08:20.197093+0000","last_clean":"2024-01-11T07:08:20.197093+0000","last_became_active":"2024-01-11T07:08:20.196419+0000","last_became_peered":"2024-01-11T07:08:20.196419+0000","last_unstale":"2024-01-11T07:08:20.197093+0000","last_undegraded":"2024-01-11T07:08:20.197093+0000","last_fullsized":"2024-01-11T07:08:20.197093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:02:16.481219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205639+0000","last_change":"2024-01-11T07:08:19.196131+0000","last_active":"2024-01-11T07:08:20.205639+0000","last_peered":"2024-01-11T07:08:20.205639+0000","last_clean":"2024-01-11T07:08:20.205639+0000","last_became_active":"2024-01-11T07:08:19.195665+0000","last_became_peered":"2024-01-11T07:08:19.195665+0000","last_unstale":"2024-01-11T07:08:20.205639+0000","last_undegraded":"2024-01-11T07:08:20.205639+0000","last_fullsized":"2024-01-11T07:08:20.205639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:12:11.866608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192884+0000","last_change":"2024-01-11T07:08:20.192652+0000","last_active":"2024-01-11T07:08:20.192884+0000","last_peered":"2024-01-11T07:08:20.192884+0000","last_clean":"2024-01-11T07:08:20.192884+0000","last_became_active":"2024-01-11T07:08:20.192244+0000","last_became_peered":"2024-01-11T07:08:20.192244+0000","last_unstale":"2024-01-11T07:08:20.192884+0000","last_undegraded":"2024-01-11T07:08:20.192884+0000","last_fullsized":"2024-01-11T07:08:20.192884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:22:43.122964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.185973+0000","last_change":"2024-01-11T07:08:19.193865+0000","last_active":"2024-01-11T07:08:20.185973+0000","last_peered":"2024-01-11T07:08:20.185973+0000","last_clean":"2024-01-11T07:08:20.185973+0000","last_became_active":"2024-01-11T07:08:19.193487+0000","last_became_peered":"2024-01-11T07:08:19.193487+0000","last_unstale":"2024-01-11T07:08:20.185973+0000","last_undegraded":"2024-01-11T07:08:20.185973+0000","last_fullsized":"2024-01-11T07:08:20.185973+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:08:27.086505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196455+0000","last_change":"2024-01-11T07:08:20.196334+0000","last_active":"2024-01-11T07:08:20.196455+0000","last_peered":"2024-01-11T07:08:20.196455+0000","last_clean":"2024-01-11T07:08:20.196455+0000","last_became_active":"2024-01-11T07:08:20.196103+0000","last_became_peered":"2024-01-11T07:08:20.196103+0000","last_unstale":"2024-01-11T07:08:20.196455+0000","last_undegraded":"2024-01-11T07:08:20.196455+0000","last_fullsized":"2024-01-11T07:08:20.196455+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:20:30.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.185008+0000","last_change":"2024-01-11T07:08:19.192371+0000","last_active":"2024-01-11T07:08:20.185008+0000","last_peered":"2024-01-11T07:08:20.185008+0000","last_clean":"2024-01-11T07:08:20.185008+0000","last_became_active":"2024-01-11T07:08:19.192074+0000","last_became_peered":"2024-01-11T07:08:19.192074+0000","last_unstale":"2024-01-11T07:08:20.185008+0000","last_undegraded":"2024-01-11T07:08:20.185008+0000","last_fullsized":"2024-01-11T07:08:20.185008+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:47.858617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189855+0000","last_change":"2024-01-11T07:08:20.189726+0000","last_active":"2024-01-11T07:08:20.189855+0000","last_peered":"2024-01-11T07:08:20.189855+0000","last_clean":"2024-01-11T07:08:20.189855+0000","last_became_active":"2024-01-11T07:08:20.189484+0000","last_became_peered":"2024-01-11T07:08:20.189484+0000","last_unstale":"2024-01-11T07:08:20.189855+0000","last_undegraded":"2024-01-11T07:08:20.189855+0000","last_fullsized":"2024-01-11T07:08:20.189855+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:39.407590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.185421+0000","last_change":"2024-01-11T07:08:19.194714+0000","last_active":"2024-01-11T07:08:20.185421+0000","last_peered":"2024-01-11T07:08:20.185421+0000","last_clean":"2024-01-11T07:08:20.185421+0000","last_became_active":"2024-01-11T07:08:19.194450+0000","last_became_peered":"2024-01-11T07:08:19.194450+0000","last_unstale":"2024-01-11T07:08:20.185421+0000","last_undegraded":"2024-01-11T07:08:20.185421+0000","last_fullsized":"2024-01-11T07:08:20.185421+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:47.858617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194490+0000","last_change":"2024-01-11T07:08:20.194425+0000","last_active":"2024-01-11T07:08:20.194490+0000","last_peered":"2024-01-11T07:08:20.194490+0000","last_clean":"2024-01-11T07:08:20.194490+0000","last_became_active":"2024-01-11T07:08:20.193860+0000","last_became_peered":"2024-01-11T07:08:20.193860+0000","last_unstale":"2024-01-11T07:08:20.194490+0000","last_undegraded":"2024-01-11T07:08:20.194490+0000","last_fullsized":"2024-01-11T07:08:20.194490+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:21:42.367721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.184425+0000","last_change":"2024-01-11T07:08:19.193264+0000","last_active":"2024-01-11T07:08:20.184425+0000","last_peered":"2024-01-11T07:08:20.184425+0000","last_clean":"2024-01-11T07:08:20.184425+0000","last_became_active":"2024-01-11T07:08:19.192455+0000","last_became_peered":"2024-01-11T07:08:19.192455+0000","last_unstale":"2024-01-11T07:08:20.184425+0000","last_undegraded":"2024-01-11T07:08:20.184425+0000","last_fullsized":"2024-01-11T07:08:20.184425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:57.779357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196107+0000","last_change":"2024-01-11T07:08:20.195266+0000","last_active":"2024-01-11T07:08:20.196107+0000","last_peered":"2024-01-11T07:08:20.196107+0000","last_clean":"2024-01-11T07:08:20.196107+0000","last_became_active":"2024-01-11T07:08:20.194866+0000","last_became_peered":"2024-01-11T07:08:20.194866+0000","last_unstale":"2024-01-11T07:08:20.196107+0000","last_undegraded":"2024-01-11T07:08:20.196107+0000","last_fullsized":"2024-01-11T07:08:20.196107+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:43:06.345810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.204215+0000","last_change":"2024-01-11T07:08:19.191266+0000","last_active":"2024-01-11T07:08:20.204215+0000","last_peered":"2024-01-11T07:08:20.204215+0000","last_clean":"2024-01-11T07:08:20.204215+0000","last_became_active":"2024-01-11T07:08:19.190885+0000","last_became_peered":"2024-01-11T07:08:19.190885+0000","last_unstale":"2024-01-11T07:08:20.204215+0000","last_undegraded":"2024-01-11T07:08:20.204215+0000","last_fullsized":"2024-01-11T07:08:20.204215+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:33.000091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203070+0000","last_change":"2024-01-11T07:08:20.202282+0000","last_active":"2024-01-11T07:08:20.203070+0000","last_peered":"2024-01-11T07:08:20.203070+0000","last_clean":"2024-01-11T07:08:20.203070+0000","last_became_active":"2024-01-11T07:08:20.202063+0000","last_became_peered":"2024-01-11T07:08:20.202063+0000","last_unstale":"2024-01-11T07:08:20.203070+0000","last_undegraded":"2024-01-11T07:08:20.203070+0000","last_fullsized":"2024-01-11T07:08:20.203070+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:57.072685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.211563+0000","last_change":"2024-01-11T07:08:19.195225+0000","last_active":"2024-01-11T07:08:20.211563+0000","last_peered":"2024-01-11T07:08:20.211563+0000","last_clean":"2024-01-11T07:08:20.211563+0000","last_became_active":"2024-01-11T07:08:19.194975+0000","last_became_peered":"2024-01-11T07:08:19.194975+0000","last_unstale":"2024-01-11T07:08:20.211563+0000","last_undegraded":"2024-01-11T07:08:20.211563+0000","last_fullsized":"2024-01-11T07:08:20.211563+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:27.730368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197759+0000","last_change":"2024-01-11T07:08:20.197679+0000","last_active":"2024-01-11T07:08:20.197759+0000","last_peered":"2024-01-11T07:08:20.197759+0000","last_clean":"2024-01-11T07:08:20.197759+0000","last_became_active":"2024-01-11T07:08:20.197235+0000","last_became_peered":"2024-01-11T07:08:20.197235+0000","last_unstale":"2024-01-11T07:08:20.197759+0000","last_undegraded":"2024-01-11T07:08:20.197759+0000","last_fullsized":"2024-01-11T07:08:20.197759+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:41:01.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205746+0000","last_change":"2024-01-11T07:08:19.192936+0000","last_active":"2024-01-11T07:08:20.205746+0000","last_peered":"2024-01-11T07:08:20.205746+0000","last_clean":"2024-01-11T07:08:20.205746+0000","last_became_active":"2024-01-11T07:08:19.191106+0000","last_became_peered":"2024-01-11T07:08:19.191106+0000","last_unstale":"2024-01-11T07:08:20.205746+0000","last_undegraded":"2024-01-11T07:08:20.205746+0000","last_fullsized":"2024-01-11T07:08:20.205746+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:33.000091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197056+0000","last_change":"2024-01-11T07:08:20.196029+0000","last_active":"2024-01-11T07:08:20.197056+0000","last_peered":"2024-01-11T07:08:20.197056+0000","last_clean":"2024-01-11T07:08:20.197056+0000","last_became_active":"2024-01-11T07:08:20.195439+0000","last_became_peered":"2024-01-11T07:08:20.195439+0000","last_unstale":"2024-01-11T07:08:20.197056+0000","last_undegraded":"2024-01-11T07:08:20.197056+0000","last_fullsized":"2024-01-11T07:08:20.197056+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:50:49.026546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192011+0000","last_change":"2024-01-11T07:08:19.193437+0000","last_active":"2024-01-11T07:08:20.192011+0000","last_peered":"2024-01-11T07:08:20.192011+0000","last_clean":"2024-01-11T07:08:20.192011+0000","last_became_active":"2024-01-11T07:08:19.192841+0000","last_became_peered":"2024-01-11T07:08:19.192841+0000","last_unstale":"2024-01-11T07:08:20.192011+0000","last_undegraded":"2024-01-11T07:08:20.192011+0000","last_fullsized":"2024-01-11T07:08:20.192011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:02:15.479190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.202956+0000","last_change":"2024-01-11T07:08:20.202642+0000","last_active":"2024-01-11T07:08:20.202956+0000","last_peered":"2024-01-11T07:08:20.202956+0000","last_clean":"2024-01-11T07:08:20.202956+0000","last_became_active":"2024-01-11T07:08:20.202155+0000","last_became_peered":"2024-01-11T07:08:20.202155+0000","last_unstale":"2024-01-11T07:08:20.202956+0000","last_undegraded":"2024-01-11T07:08:20.202956+0000","last_fullsized":"2024-01-11T07:08:20.202956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:40:53.941699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189704+0000","last_change":"2024-01-11T07:08:19.189618+0000","last_active":"2024-01-11T07:08:20.189704+0000","last_peered":"2024-01-11T07:08:20.189704+0000","last_clean":"2024-01-11T07:08:20.189704+0000","last_became_active":"2024-01-11T07:08:19.189004+0000","last_became_peered":"2024-01-11T07:08:19.189004+0000","last_unstale":"2024-01-11T07:08:20.189704+0000","last_undegraded":"2024-01-11T07:08:20.189704+0000","last_fullsized":"2024-01-11T07:08:20.189704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:54.267708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197088+0000","last_change":"2024-01-11T07:08:20.196988+0000","last_active":"2024-01-11T07:08:20.197088+0000","last_peered":"2024-01-11T07:08:20.197088+0000","last_clean":"2024-01-11T07:08:20.197088+0000","last_became_active":"2024-01-11T07:08:20.196752+0000","last_became_peered":"2024-01-11T07:08:20.196752+0000","last_unstale":"2024-01-11T07:08:20.197088+0000","last_undegraded":"2024-01-11T07:08:20.197088+0000","last_fullsized":"2024-01-11T07:08:20.197088+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:22:43.122964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.210422+0000","last_change":"2024-01-11T07:08:19.191492+0000","last_active":"2024-01-11T07:08:20.210422+0000","last_peered":"2024-01-11T07:08:20.210422+0000","last_clean":"2024-01-11T07:08:20.210422+0000","last_became_active":"2024-01-11T07:08:19.191199+0000","last_became_peered":"2024-01-11T07:08:19.191199+0000","last_unstale":"2024-01-11T07:08:20.210422+0000","last_undegraded":"2024-01-11T07:08:20.210422+0000","last_fullsized":"2024-01-11T07:08:20.210422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:36:32.123150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197103+0000","last_change":"2024-01-11T07:08:20.196101+0000","last_active":"2024-01-11T07:08:20.197103+0000","last_peered":"2024-01-11T07:08:20.197103+0000","last_clean":"2024-01-11T07:08:20.197103+0000","last_became_active":"2024-01-11T07:08:20.195843+0000","last_became_peered":"2024-01-11T07:08:20.195843+0000","last_unstale":"2024-01-11T07:08:20.197103+0000","last_undegraded":"2024-01-11T07:08:20.197103+0000","last_fullsized":"2024-01-11T07:08:20.197103+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:01.134482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.209375+0000","last_change":"2024-01-11T07:08:19.189720+0000","last_active":"2024-01-11T07:08:20.209375+0000","last_peered":"2024-01-11T07:08:20.209375+0000","last_clean":"2024-01-11T07:08:20.209375+0000","last_became_active":"2024-01-11T07:08:19.189485+0000","last_became_peered":"2024-01-11T07:08:19.189485+0000","last_unstale":"2024-01-11T07:08:20.209375+0000","last_undegraded":"2024-01-11T07:08:20.209375+0000","last_fullsized":"2024-01-11T07:08:20.209375+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:48.988448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.199963+0000","last_change":"2024-01-11T07:08:20.199890+0000","last_active":"2024-01-11T07:08:20.199963+0000","last_peered":"2024-01-11T07:08:20.199963+0000","last_clean":"2024-01-11T07:08:20.199963+0000","last_became_active":"2024-01-11T07:08:20.199629+0000","last_became_peered":"2024-01-11T07:08:20.199629+0000","last_unstale":"2024-01-11T07:08:20.199963+0000","last_undegraded":"2024-01-11T07:08:20.199963+0000","last_fullsized":"2024-01-11T07:08:20.199963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:15:58.440223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.206869+0000","last_change":"2024-01-11T07:08:19.192733+0000","last_active":"2024-01-11T07:08:20.206869+0000","last_peered":"2024-01-11T07:08:20.206869+0000","last_clean":"2024-01-11T07:08:20.206869+0000","last_became_active":"2024-01-11T07:08:19.192509+0000","last_became_peered":"2024-01-11T07:08:19.192509+0000","last_unstale":"2024-01-11T07:08:20.206869+0000","last_undegraded":"2024-01-11T07:08:20.206869+0000","last_fullsized":"2024-01-11T07:08:20.206869+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:36:32.123150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198065+0000","last_change":"2024-01-11T07:08:20.197998+0000","last_active":"2024-01-11T07:08:20.198065+0000","last_peered":"2024-01-11T07:08:20.198065+0000","last_clean":"2024-01-11T07:08:20.198065+0000","last_became_active":"2024-01-11T07:08:20.197352+0000","last_became_peered":"2024-01-11T07:08:20.197352+0000","last_unstale":"2024-01-11T07:08:20.198065+0000","last_undegraded":"2024-01-11T07:08:20.198065+0000","last_fullsized":"2024-01-11T07:08:20.198065+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:08:45.898498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.204145+0000","last_change":"2024-01-11T07:08:19.191891+0000","last_active":"2024-01-11T07:08:20.204145+0000","last_peered":"2024-01-11T07:08:20.204145+0000","last_clean":"2024-01-11T07:08:20.204145+0000","last_became_active":"2024-01-11T07:08:19.191660+0000","last_became_peered":"2024-01-11T07:08:19.191660+0000","last_unstale":"2024-01-11T07:08:20.204145+0000","last_undegraded":"2024-01-11T07:08:20.204145+0000","last_fullsized":"2024-01-11T07:08:20.204145+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:03:34.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.199159+0000","last_change":"2024-01-11T07:08:20.199093+0000","last_active":"2024-01-11T07:08:20.199159+0000","last_peered":"2024-01-11T07:08:20.199159+0000","last_clean":"2024-01-11T07:08:20.199159+0000","last_became_active":"2024-01-11T07:08:20.198880+0000","last_became_peered":"2024-01-11T07:08:20.198880+0000","last_unstale":"2024-01-11T07:08:20.199159+0000","last_undegraded":"2024-01-11T07:08:20.199159+0000","last_fullsized":"2024-01-11T07:08:20.199159+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:40:27.902284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189820+0000","last_change":"2024-01-11T07:08:19.194355+0000","last_active":"2024-01-11T07:08:20.189820+0000","last_peered":"2024-01-11T07:08:20.189820+0000","last_clean":"2024-01-11T07:08:20.189820+0000","last_became_active":"2024-01-11T07:08:19.194093+0000","last_became_peered":"2024-01-11T07:08:19.194093+0000","last_unstale":"2024-01-11T07:08:20.189820+0000","last_undegraded":"2024-01-11T07:08:20.189820+0000","last_fullsized":"2024-01-11T07:08:20.189820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:47.275593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196086+0000","last_change":"2024-01-11T07:08:20.195107+0000","last_active":"2024-01-11T07:08:20.196086+0000","last_peered":"2024-01-11T07:08:20.196086+0000","last_clean":"2024-01-11T07:08:20.196086+0000","last_became_active":"2024-01-11T07:08:20.194547+0000","last_became_peered":"2024-01-11T07:08:20.194547+0000","last_unstale":"2024-01-11T07:08:20.196086+0000","last_undegraded":"2024-01-11T07:08:20.196086+0000","last_fullsized":"2024-01-11T07:08:20.196086+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:35:14.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207218+0000","last_change":"2024-01-11T07:08:19.190574+0000","last_active":"2024-01-11T07:08:20.207218+0000","last_peered":"2024-01-11T07:08:20.207218+0000","last_clean":"2024-01-11T07:08:20.207218+0000","last_became_active":"2024-01-11T07:08:19.189479+0000","last_became_peered":"2024-01-11T07:08:19.189479+0000","last_unstale":"2024-01-11T07:08:20.207218+0000","last_undegraded":"2024-01-11T07:08:20.207218+0000","last_fullsized":"2024-01-11T07:08:20.207218+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:38.405561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201083+0000","last_change":"2024-01-11T07:08:20.200551+0000","last_active":"2024-01-11T07:08:20.201083+0000","last_peered":"2024-01-11T07:08:20.201083+0000","last_clean":"2024-01-11T07:08:20.201083+0000","last_became_active":"2024-01-11T07:08:20.200244+0000","last_became_peered":"2024-01-11T07:08:20.200244+0000","last_unstale":"2024-01-11T07:08:20.201083+0000","last_undegraded":"2024-01-11T07:08:20.201083+0000","last_fullsized":"2024-01-11T07:08:20.201083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:08:45.898498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.186106+0000","last_change":"2024-01-11T07:08:19.193728+0000","last_active":"2024-01-11T07:08:20.186106+0000","last_peered":"2024-01-11T07:08:20.186106+0000","last_clean":"2024-01-11T07:08:20.186106+0000","last_became_active":"2024-01-11T07:08:19.193174+0000","last_became_peered":"2024-01-11T07:08:19.193174+0000","last_unstale":"2024-01-11T07:08:20.186106+0000","last_undegraded":"2024-01-11T07:08:20.186106+0000","last_fullsized":"2024-01-11T07:08:20.186106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:12:26.681729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198019+0000","last_change":"2024-01-11T07:08:20.197949+0000","last_active":"2024-01-11T07:08:20.198019+0000","last_peered":"2024-01-11T07:08:20.198019+0000","last_clean":"2024-01-11T07:08:20.198019+0000","last_became_active":"2024-01-11T07:08:20.197700+0000","last_became_peered":"2024-01-11T07:08:20.197700+0000","last_unstale":"2024-01-11T07:08:20.198019+0000","last_undegraded":"2024-01-11T07:08:20.198019+0000","last_fullsized":"2024-01-11T07:08:20.198019+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:49:29.301371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.184902+0000","last_change":"2024-01-11T07:08:19.190534+0000","last_active":"2024-01-11T07:08:20.184902+0000","last_peered":"2024-01-11T07:08:20.184902+0000","last_clean":"2024-01-11T07:08:20.184902+0000","last_became_active":"2024-01-11T07:08:19.190208+0000","last_became_peered":"2024-01-11T07:08:19.190208+0000","last_unstale":"2024-01-11T07:08:20.184902+0000","last_undegraded":"2024-01-11T07:08:20.184902+0000","last_fullsized":"2024-01-11T07:08:20.184902+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:12:11.866608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.202900+0000","last_change":"2024-01-11T07:08:20.202798+0000","last_active":"2024-01-11T07:08:20.202900+0000","last_peered":"2024-01-11T07:08:20.202900+0000","last_clean":"2024-01-11T07:08:20.202900+0000","last_became_active":"2024-01-11T07:08:20.202567+0000","last_became_peered":"2024-01-11T07:08:20.202567+0000","last_unstale":"2024-01-11T07:08:20.202900+0000","last_undegraded":"2024-01-11T07:08:20.202900+0000","last_fullsized":"2024-01-11T07:08:20.202900+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:55.269737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192104+0000","last_change":"2024-01-11T07:08:19.193553+0000","last_active":"2024-01-11T07:08:20.192104+0000","last_peered":"2024-01-11T07:08:20.192104+0000","last_clean":"2024-01-11T07:08:20.192104+0000","last_became_active":"2024-01-11T07:08:19.193113+0000","last_became_peered":"2024-01-11T07:08:19.193113+0000","last_unstale":"2024-01-11T07:08:20.192104+0000","last_undegraded":"2024-01-11T07:08:20.192104+0000","last_fullsized":"2024-01-11T07:08:20.192104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:50:48.024516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198232+0000","last_change":"2024-01-11T07:08:20.198157+0000","last_active":"2024-01-11T07:08:20.198232+0000","last_peered":"2024-01-11T07:08:20.198232+0000","last_clean":"2024-01-11T07:08:20.198232+0000","last_became_active":"2024-01-11T07:08:20.197590+0000","last_became_peered":"2024-01-11T07:08:20.197590+0000","last_unstale":"2024-01-11T07:08:20.198232+0000","last_undegraded":"2024-01-11T07:08:20.198232+0000","last_fullsized":"2024-01-11T07:08:20.198232+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:31.579570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.193256+0000","last_change":"2024-01-11T07:08:19.196552+0000","last_active":"2024-01-11T07:08:20.193256+0000","last_peered":"2024-01-11T07:08:20.193256+0000","last_clean":"2024-01-11T07:08:20.193256+0000","last_became_active":"2024-01-11T07:08:19.196286+0000","last_became_peered":"2024-01-11T07:08:19.196286+0000","last_unstale":"2024-01-11T07:08:20.193256+0000","last_undegraded":"2024-01-11T07:08:20.193256+0000","last_fullsized":"2024-01-11T07:08:20.193256+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:03:34.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197074+0000","last_change":"2024-01-11T07:08:20.196206+0000","last_active":"2024-01-11T07:08:20.197074+0000","last_peered":"2024-01-11T07:08:20.197074+0000","last_clean":"2024-01-11T07:08:20.197074+0000","last_became_active":"2024-01-11T07:08:20.195784+0000","last_became_peered":"2024-01-11T07:08:20.195784+0000","last_unstale":"2024-01-11T07:08:20.197074+0000","last_undegraded":"2024-01-11T07:08:20.197074+0000","last_fullsized":"2024-01-11T07:08:20.197074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:18:35.823955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.191296+0000","last_change":"2024-01-11T07:08:19.192165+0000","last_active":"2024-01-11T07:08:20.191296+0000","last_peered":"2024-01-11T07:08:20.191296+0000","last_clean":"2024-01-11T07:08:20.191296+0000","last_became_active":"2024-01-11T07:08:19.191920+0000","last_became_peered":"2024-01-11T07:08:19.191920+0000","last_unstale":"2024-01-11T07:08:20.191296+0000","last_undegraded":"2024-01-11T07:08:20.191296+0000","last_fullsized":"2024-01-11T07:08:20.191296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:38.405561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.199895+0000","last_change":"2024-01-11T07:08:20.199837+0000","last_active":"2024-01-11T07:08:20.199895+0000","last_peered":"2024-01-11T07:08:20.199895+0000","last_clean":"2024-01-11T07:08:20.199895+0000","last_became_active":"2024-01-11T07:08:20.199690+0000","last_became_peered":"2024-01-11T07:08:20.199690+0000","last_unstale":"2024-01-11T07:08:20.199895+0000","last_undegraded":"2024-01-11T07:08:20.199895+0000","last_fullsized":"2024-01-11T07:08:20.199895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:18:23.886765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194451+0000","last_change":"2024-01-11T07:08:19.191884+0000","last_active":"2024-01-11T07:08:20.194451+0000","last_peered":"2024-01-11T07:08:20.194451+0000","last_clean":"2024-01-11T07:08:20.194451+0000","last_became_active":"2024-01-11T07:08:19.191631+0000","last_became_peered":"2024-01-11T07:08:19.191631+0000","last_unstale":"2024-01-11T07:08:20.194451+0000","last_undegraded":"2024-01-11T07:08:20.194451+0000","last_fullsized":"2024-01-11T07:08:20.194451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:00.584001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203005+0000","last_change":"2024-01-11T07:08:20.202826+0000","last_active":"2024-01-11T07:08:20.203005+0000","last_peered":"2024-01-11T07:08:20.203005+0000","last_clean":"2024-01-11T07:08:20.203005+0000","last_became_active":"2024-01-11T07:08:20.202236+0000","last_became_peered":"2024-01-11T07:08:20.202236+0000","last_unstale":"2024-01-11T07:08:20.203005+0000","last_undegraded":"2024-01-11T07:08:20.203005+0000","last_fullsized":"2024-01-11T07:08:20.203005+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:24:11.496779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192494+0000","last_change":"2024-01-11T07:08:19.193308+0000","last_active":"2024-01-11T07:08:20.192494+0000","last_peered":"2024-01-11T07:08:20.192494+0000","last_clean":"2024-01-11T07:08:20.192494+0000","last_became_active":"2024-01-11T07:08:19.192509+0000","last_became_peered":"2024-01-11T07:08:19.192509+0000","last_unstale":"2024-01-11T07:08:20.192494+0000","last_undegraded":"2024-01-11T07:08:20.192494+0000","last_fullsized":"2024-01-11T07:08:20.192494+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:03:34.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207476+0000","last_change":"2024-01-11T07:08:20.207302+0000","last_active":"2024-01-11T07:08:20.207476+0000","last_peered":"2024-01-11T07:08:20.207476+0000","last_clean":"2024-01-11T07:08:20.207476+0000","last_became_active":"2024-01-11T07:08:20.206541+0000","last_became_peered":"2024-01-11T07:08:20.206541+0000","last_unstale":"2024-01-11T07:08:20.207476+0000","last_undegraded":"2024-01-11T07:08:20.207476+0000","last_fullsized":"2024-01-11T07:08:20.207476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:42:13.324724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.186796+0000","last_change":"2024-01-11T07:08:19.196814+0000","last_active":"2024-01-11T07:08:20.186796+0000","last_peered":"2024-01-11T07:08:20.186796+0000","last_clean":"2024-01-11T07:08:20.186796+0000","last_became_active":"2024-01-11T07:08:19.196578+0000","last_became_peered":"2024-01-11T07:08:19.196578+0000","last_unstale":"2024-01-11T07:08:20.186796+0000","last_undegraded":"2024-01-11T07:08:20.186796+0000","last_fullsized":"2024-01-11T07:08:20.186796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:20:29.426380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.202158+0000","last_change":"2024-01-11T07:08:20.202063+0000","last_active":"2024-01-11T07:08:20.202158+0000","last_peered":"2024-01-11T07:08:20.202158+0000","last_clean":"2024-01-11T07:08:20.202158+0000","last_became_active":"2024-01-11T07:08:20.201801+0000","last_became_peered":"2024-01-11T07:08:20.201801+0000","last_unstale":"2024-01-11T07:08:20.202158+0000","last_undegraded":"2024-01-11T07:08:20.202158+0000","last_fullsized":"2024-01-11T07:08:20.202158+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:22:43.122964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197075+0000","last_change":"2024-01-11T07:08:19.190951+0000","last_active":"2024-01-11T07:08:20.197075+0000","last_peered":"2024-01-11T07:08:20.197075+0000","last_clean":"2024-01-11T07:08:20.197075+0000","last_became_active":"2024-01-11T07:08:19.190698+0000","last_became_peered":"2024-01-11T07:08:19.190698+0000","last_unstale":"2024-01-11T07:08:20.197075+0000","last_undegraded":"2024-01-11T07:08:20.197075+0000","last_fullsized":"2024-01-11T07:08:20.197075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:20:29.426380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196104+0000","last_change":"2024-01-11T07:08:20.195862+0000","last_active":"2024-01-11T07:08:20.196104+0000","last_peered":"2024-01-11T07:08:20.196104+0000","last_clean":"2024-01-11T07:08:20.196104+0000","last_became_active":"2024-01-11T07:08:20.195449+0000","last_became_peered":"2024-01-11T07:08:20.195449+0000","last_unstale":"2024-01-11T07:08:20.196104+0000","last_undegraded":"2024-01-11T07:08:20.196104+0000","last_fullsized":"2024-01-11T07:08:20.196104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:08:28.088534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.184456+0000","last_change":"2024-01-11T07:08:19.193254+0000","last_active":"2024-01-11T07:08:20.184456+0000","last_peered":"2024-01-11T07:08:20.184456+0000","last_clean":"2024-01-11T07:08:20.184456+0000","last_became_active":"2024-01-11T07:08:19.192022+0000","last_became_peered":"2024-01-11T07:08:19.192022+0000","last_unstale":"2024-01-11T07:08:20.184456+0000","last_undegraded":"2024-01-11T07:08:20.184456+0000","last_fullsized":"2024-01-11T07:08:20.184456+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:36.551051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.195230+0000","last_change":"2024-01-11T07:08:20.195146+0000","last_active":"2024-01-11T07:08:20.195230+0000","last_peered":"2024-01-11T07:08:20.195230+0000","last_clean":"2024-01-11T07:08:20.195230+0000","last_became_active":"2024-01-11T07:08:20.194423+0000","last_became_peered":"2024-01-11T07:08:20.194423+0000","last_unstale":"2024-01-11T07:08:20.195230+0000","last_undegraded":"2024-01-11T07:08:20.195230+0000","last_fullsized":"2024-01-11T07:08:20.195230+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:39.407590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203447+0000","last_change":"2024-01-11T07:08:19.195471+0000","last_active":"2024-01-11T07:08:20.203447+0000","last_peered":"2024-01-11T07:08:20.203447+0000","last_clean":"2024-01-11T07:08:20.203447+0000","last_became_active":"2024-01-11T07:08:19.195204+0000","last_became_peered":"2024-01-11T07:08:19.195204+0000","last_unstale":"2024-01-11T07:08:20.203447+0000","last_undegraded":"2024-01-11T07:08:20.203447+0000","last_fullsized":"2024-01-11T07:08:20.203447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:38.405561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.191403+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:08:20.191403+0000","last_peered":"2024-01-11T07:08:20.191403+0000","last_clean":"2024-01-11T07:08:20.191403+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:08:20.191403+0000","last_undegraded":"2024-01-11T07:08:20.191403+0000","last_fullsized":"2024-01-11T07:08:20.191403+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":794,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194517,"statfs":{"total":200000000,"available":199186127,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.497}]},{"osd":3,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.439}]},{"osd":5,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.629},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.63500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195093,"statfs":{"total":200000000,"available":199775941,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.749},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.65900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195094,"statfs":{"total":200000000,"available":199776515,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.38},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":794,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194517,"statfs":{"total":200000000,"available":199186183,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.312},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.7960000000000003,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":8.0730000000000004,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":0.38}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":8.0399999999999991,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.432}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.819,"5min":2.819,"15min":2.819},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.503}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":8.0030000000000001,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.52400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.85299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195093,"statfs":{"total":200000000,"available":199775975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":11.481,"5min":11.481,"15min":11.481},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.379}]},{"osd":1,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":17.516999999999999,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":17.553000000000001,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.474,"5min":17.474,"15min":17.474},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":17.393000000000001,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":11.436999999999999,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":17.434000000000001,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.65700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775049,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":8.484,"5min":8.484,"15min":8.484},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.2599999999999998,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.504}]},{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":8.5340000000000007,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":9.2279999999999998,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":9.5640000000000001,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.8639999999999999,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":8.9779999999999998,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.7309999999999999,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.441}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.388999999999999,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":8.6509999999999998,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1449999999999996,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.4640000000000004,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.56499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195094,"statfs":{"total":200000000,"available":199776515,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.5}]},{"osd":3,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.439},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47799999999999998}]}]}],"pool_statfs":[]}} 2024-01-11T07:08:24.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:08:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.553+0000 7ff602e8c700 1 Processor -- start 2024-01-11T07:08:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.553+0000 7ff602e8c700 1 -- start start 2024-01-11T07:08:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.554+0000 7ff602e8c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5fc069320 0x7ff5fc1649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.554+0000 7ff602e8c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5fc0ce9b0 con 0x7ff5fc15a060 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.554+0000 7ff602e8c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5fc0cf530 con 0x7ff5fc159c90 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.554+0000 7ff602e8c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5fc0b1f30 con 0x7ff5fc069320 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.554+0000 7ff601429700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5fc069320 0x7ff5fc1649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.554+0000 7ff5fbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff5fc15a060 0x7ff5fc0b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40958/0 (socket says 172.21.15.7:40958) 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.554+0000 7ff5fbfff700 1 -- 172.21.15.7:0/812109856 learned_addr learned my addr 172.21.15.7:0/812109856 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.555+0000 7ff601429700 1 -- 172.21.15.7:0/812109856 >> v1:172.21.15.113:6789/0 conn(0x7ff5fc159c90 legacy=0x7ff5fc0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.555+0000 7ff601429700 1 -- 172.21.15.7:0/812109856 >> v1:172.21.15.7:6789/0 conn(0x7ff5fc15a060 legacy=0x7ff5fc0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.555+0000 7ff601429700 1 -- 172.21.15.7:0/812109856 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0cf9c0 con 0x7ff5fc069320 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.555+0000 7ff601429700 1 --2- 172.21.15.7:0/812109856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc1649a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009e60 tx=0x7ff5f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d569b71fc947baa6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.555+0000 7ff5fb7fe700 1 -- 172.21.15.7:0/812109856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400e070 con 0x7ff5fc069320 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.555+0000 7ff5fb7fe700 1 -- 172.21.15.7:0/812109856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f400bc30 con 0x7ff5fc069320 2024-01-11T07:08:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.555+0000 7ff5fb7fe700 1 -- 172.21.15.7:0/812109856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4012430 con 0x7ff5fc069320 2024-01-11T07:08:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.556+0000 7ff602e8c700 1 -- 172.21.15.7:0/812109856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 msgr2=0x7ff5fc1649a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.556+0000 7ff602e8c700 1 --2- 172.21.15.7:0/812109856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc1649a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009e60 tx=0x7ff5f4009780 comp rx=0 tx=0).stop 2024-01-11T07:08:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.556+0000 7ff602e8c700 1 -- 172.21.15.7:0/812109856 shutdown_connections 2024-01-11T07:08:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.556+0000 7ff602e8c700 1 --2- 172.21.15.7:0/812109856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc1649a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.556+0000 7ff602e8c700 1 -- 172.21.15.7:0/812109856 >> 172.21.15.7:0/812109856 conn(0x7ff5fc071100 msgr2=0x7ff5fc06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.556+0000 7ff602e8c700 1 -- 172.21.15.7:0/812109856 shutdown_connections 2024-01-11T07:08:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.556+0000 7ff602e8c700 1 -- 172.21.15.7:0/812109856 wait complete. 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.557+0000 7ff602e8c700 1 Processor -- start 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.558+0000 7ff602e8c700 1 -- start start 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff602e8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc16a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff602e8c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc159c90 0x7ff5fc155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff602e8c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5fc15a060 0x7ff5fc0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff602e8c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc16ac10 con 0x7ff5fc069320 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff602e8c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc155ff0 con 0x7ff5fc159c90 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff602e8c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0ccbe0 con 0x7ff5fc15a060 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc16a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc16a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47206/0 (socket says 172.21.15.7:47206) 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 -- 172.21.15.7:0/1813740678 learned_addr learned my addr 172.21.15.7:0/1813740678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff5fbfff700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc159c90 0x7ff5fc155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff601429700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5fc15a060 0x7ff5fc0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 -- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5fc15a060 msgr2=0x7ff5fc0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5fc15a060 0x7ff5fc0cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 -- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc159c90 msgr2=0x7ff5fc155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc159c90 0x7ff5fc155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 -- 172.21.15.7:0/1813740678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4009a20 con 0x7ff5fc069320 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff600c28700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc16a6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c790 tx=0x7ff5f0007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff5f8ff9700 1 -- 172.21.15.7:0/1813740678 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0011590 con 0x7ff5fc069320 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff5f8ff9700 1 -- 172.21.15.7:0/1813740678 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f0011710 con 0x7ff5fc069320 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.559+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc0ccec0 con 0x7ff5fc069320 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.560+0000 7ff5f8ff9700 1 -- 172.21.15.7:0/1813740678 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0011ad0 con 0x7ff5fc069320 2024-01-11T07:08:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.560+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc0cd480 con 0x7ff5fc069320 2024-01-11T07:08:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.561+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc04c3c0 con 0x7ff5fc069320 2024-01-11T07:08:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.561+0000 7ff5f8ff9700 1 -- 172.21.15.7:0/1813740678 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff5f001b3b0 con 0x7ff5fc069320 2024-01-11T07:08:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.561+0000 7ff5f8ff9700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.562+0000 7ff5f8ff9700 1 -- 172.21.15.7:0/1813740678 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff5f00838f0 con 0x7ff5fc069320 2024-01-11T07:08:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.562+0000 7ff5fbfff700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.562+0000 7ff5fbfff700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff5e0066f30 0x7ff5e00693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8007130 tx=0x7ff5e8007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.566+0000 7ff5f8ff9700 1 -- 172.21.15.7:0/1813740678 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5f0050b60 con 0x7ff5fc069320 2024-01-11T07:08:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.766+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5fc15e5d0 con 0x7ff5e0066f30 2024-01-11T07:08:24.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.775+0000 7ff5f8ff9700 1 -- 172.21.15.7:0/1813740678 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284757 (secure 0 0 0) 0x7ff5fc15e5d0 con 0x7ff5e0066f30 2024-01-11T07:08:24.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff5e0066f30 msgr2=0x7ff5e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff5e0066f30 0x7ff5e00693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8007130 tx=0x7ff5e8007eb0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 msgr2=0x7ff5fc16a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc16a6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c790 tx=0x7ff5f0007f40 comp rx=0 tx=0).stop 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 shutdown_connections 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5fc15a060 0x7ff5fc0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5fc069320 0x7ff5fc16a6d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 --2- 172.21.15.7:0/1813740678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc159c90 0x7ff5fc155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 >> 172.21.15.7:0/1813740678 conn(0x7ff5fc071100 msgr2=0x7ff5fc15d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 shutdown_connections 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.778+0000 7ff602e8c700 1 -- 172.21.15.7:0/1813740678 wait complete. 2024-01-11T07:08:24.783 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:08:24.792 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2024-01-11T07:08:23.858376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559597,"statfs":{"total":1600000000,"available":1597027332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.682400"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194645+0000","last_change":"2024-01-11T07:08:20.194581+0000","last_active":"2024-01-11T07:08:20.194645+0000","last_peered":"2024-01-11T07:08:20.194645+0000","last_clean":"2024-01-11T07:08:20.194645+0000","last_became_active":"2024-01-11T07:08:20.194200+0000","last_became_peered":"2024-01-11T07:08:20.194200+0000","last_unstale":"2024-01-11T07:08:20.194645+0000","last_undegraded":"2024-01-11T07:08:20.194645+0000","last_fullsized":"2024-01-11T07:08:20.194645+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:09:31.114790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.190369+0000","last_change":"2024-01-11T07:08:20.190291+0000","last_active":"2024-01-11T07:08:20.190369+0000","last_peered":"2024-01-11T07:08:20.190369+0000","last_clean":"2024-01-11T07:08:20.190369+0000","last_became_active":"2024-01-11T07:08:20.189763+0000","last_became_peered":"2024-01-11T07:08:20.189763+0000","last_unstale":"2024-01-11T07:08:20.190369+0000","last_undegraded":"2024-01-11T07:08:20.190369+0000","last_fullsized":"2024-01-11T07:08:20.190369+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:30:54.201577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197148+0000","last_change":"2024-01-11T07:08:20.197066+0000","last_active":"2024-01-11T07:08:20.197148+0000","last_peered":"2024-01-11T07:08:20.197148+0000","last_clean":"2024-01-11T07:08:20.197148+0000","last_became_active":"2024-01-11T07:08:20.196788+0000","last_became_peered":"2024-01-11T07:08:20.196788+0000","last_unstale":"2024-01-11T07:08:20.197148+0000","last_undegraded":"2024-01-11T07:08:20.197148+0000","last_fullsized":"2024-01-11T07:08:20.197148+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:34.002120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.193402+0000","last_change":"2024-01-11T07:08:20.193273+0000","last_active":"2024-01-11T07:08:20.193402+0000","last_peered":"2024-01-11T07:08:20.193402+0000","last_clean":"2024-01-11T07:08:20.193402+0000","last_became_active":"2024-01-11T07:08:20.192972+0000","last_became_peered":"2024-01-11T07:08:20.192972+0000","last_unstale":"2024-01-11T07:08:20.193402+0000","last_undegraded":"2024-01-11T07:08:20.193402+0000","last_fullsized":"2024-01-11T07:08:20.193402+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:05:37.421591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194894+0000","last_change":"2024-01-11T07:08:20.194801+0000","last_active":"2024-01-11T07:08:20.194894+0000","last_peered":"2024-01-11T07:08:20.194894+0000","last_clean":"2024-01-11T07:08:20.194894+0000","last_became_active":"2024-01-11T07:08:20.194195+0000","last_became_peered":"2024-01-11T07:08:20.194195+0000","last_unstale":"2024-01-11T07:08:20.194894+0000","last_undegraded":"2024-01-11T07:08:20.194894+0000","last_fullsized":"2024-01-11T07:08:20.194894+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:05:37.421591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207326+0000","last_change":"2024-01-11T07:08:20.207120+0000","last_active":"2024-01-11T07:08:20.207326+0000","last_peered":"2024-01-11T07:08:20.207326+0000","last_clean":"2024-01-11T07:08:20.207326+0000","last_became_active":"2024-01-11T07:08:20.205327+0000","last_became_peered":"2024-01-11T07:08:20.205327+0000","last_unstale":"2024-01-11T07:08:20.207326+0000","last_undegraded":"2024-01-11T07:08:20.207326+0000","last_fullsized":"2024-01-11T07:08:20.207326+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:53:51.094526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.200615+0000","last_change":"2024-01-11T07:08:20.200548+0000","last_active":"2024-01-11T07:08:20.200615+0000","last_peered":"2024-01-11T07:08:20.200615+0000","last_clean":"2024-01-11T07:08:20.200615+0000","last_became_active":"2024-01-11T07:08:20.200343+0000","last_became_peered":"2024-01-11T07:08:20.200343+0000","last_unstale":"2024-01-11T07:08:20.200615+0000","last_undegraded":"2024-01-11T07:08:20.200615+0000","last_fullsized":"2024-01-11T07:08:20.200615+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:30:54.201577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192903+0000","last_change":"2024-01-11T07:08:20.192773+0000","last_active":"2024-01-11T07:08:20.192903+0000","last_peered":"2024-01-11T07:08:20.192903+0000","last_clean":"2024-01-11T07:08:20.192903+0000","last_became_active":"2024-01-11T07:08:20.192516+0000","last_became_peered":"2024-01-11T07:08:20.192516+0000","last_unstale":"2024-01-11T07:08:20.192903+0000","last_undegraded":"2024-01-11T07:08:20.192903+0000","last_fullsized":"2024-01-11T07:08:20.192903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:09:31.114790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205794+0000","last_change":"2024-01-11T07:08:20.205692+0000","last_active":"2024-01-11T07:08:20.205794+0000","last_peered":"2024-01-11T07:08:20.205794+0000","last_clean":"2024-01-11T07:08:20.205794+0000","last_became_active":"2024-01-11T07:08:20.205243+0000","last_became_peered":"2024-01-11T07:08:20.205243+0000","last_unstale":"2024-01-11T07:08:20.205794+0000","last_undegraded":"2024-01-11T07:08:20.205794+0000","last_fullsized":"2024-01-11T07:08:20.205794+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:48.277623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192864+0000","last_change":"2024-01-11T07:08:20.192498+0000","last_active":"2024-01-11T07:08:20.192864+0000","last_peered":"2024-01-11T07:08:20.192864+0000","last_clean":"2024-01-11T07:08:20.192864+0000","last_became_active":"2024-01-11T07:08:20.191901+0000","last_became_peered":"2024-01-11T07:08:20.191901+0000","last_unstale":"2024-01-11T07:08:20.192864+0000","last_undegraded":"2024-01-11T07:08:20.192864+0000","last_fullsized":"2024-01-11T07:08:20.192864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:30:54.201577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201634+0000","last_change":"2024-01-11T07:08:20.201535+0000","last_active":"2024-01-11T07:08:20.201634+0000","last_peered":"2024-01-11T07:08:20.201634+0000","last_clean":"2024-01-11T07:08:20.201634+0000","last_became_active":"2024-01-11T07:08:20.201262+0000","last_became_peered":"2024-01-11T07:08:20.201262+0000","last_unstale":"2024-01-11T07:08:20.201634+0000","last_undegraded":"2024-01-11T07:08:20.201634+0000","last_fullsized":"2024-01-11T07:08:20.201634+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:34.002120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196233+0000","last_change":"2024-01-11T07:08:20.196164+0000","last_active":"2024-01-11T07:08:20.196233+0000","last_peered":"2024-01-11T07:08:20.196233+0000","last_clean":"2024-01-11T07:08:20.196233+0000","last_became_active":"2024-01-11T07:08:20.195903+0000","last_became_peered":"2024-01-11T07:08:20.195903+0000","last_unstale":"2024-01-11T07:08:20.196233+0000","last_undegraded":"2024-01-11T07:08:20.196233+0000","last_fullsized":"2024-01-11T07:08:20.196233+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:03.517036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197772+0000","last_change":"2024-01-11T07:08:20.197103+0000","last_active":"2024-01-11T07:08:20.197772+0000","last_peered":"2024-01-11T07:08:20.197772+0000","last_clean":"2024-01-11T07:08:20.197772+0000","last_became_active":"2024-01-11T07:08:20.196823+0000","last_became_peered":"2024-01-11T07:08:20.196823+0000","last_unstale":"2024-01-11T07:08:20.197772+0000","last_undegraded":"2024-01-11T07:08:20.197772+0000","last_fullsized":"2024-01-11T07:08:20.197772+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:42:18.426535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205772+0000","last_change":"2024-01-11T07:08:20.205508+0000","last_active":"2024-01-11T07:08:20.205772+0000","last_peered":"2024-01-11T07:08:20.205772+0000","last_clean":"2024-01-11T07:08:20.205772+0000","last_became_active":"2024-01-11T07:08:20.204921+0000","last_became_peered":"2024-01-11T07:08:20.204921+0000","last_unstale":"2024-01-11T07:08:20.205772+0000","last_undegraded":"2024-01-11T07:08:20.205772+0000","last_fullsized":"2024-01-11T07:08:20.205772+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:48:10.992014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.195123+0000","last_change":"2024-01-11T07:08:20.194376+0000","last_active":"2024-01-11T07:08:20.195123+0000","last_peered":"2024-01-11T07:08:20.195123+0000","last_clean":"2024-01-11T07:08:20.195123+0000","last_became_active":"2024-01-11T07:08:20.194135+0000","last_became_peered":"2024-01-11T07:08:20.194135+0000","last_unstale":"2024-01-11T07:08:20.195123+0000","last_undegraded":"2024-01-11T07:08:20.195123+0000","last_fullsized":"2024-01-11T07:08:20.195123+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:39:22.741253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197937+0000","last_change":"2024-01-11T07:08:20.197876+0000","last_active":"2024-01-11T07:08:20.197937+0000","last_peered":"2024-01-11T07:08:20.197937+0000","last_clean":"2024-01-11T07:08:20.197937+0000","last_became_active":"2024-01-11T07:08:20.197494+0000","last_became_peered":"2024-01-11T07:08:20.197494+0000","last_unstale":"2024-01-11T07:08:20.197937+0000","last_undegraded":"2024-01-11T07:08:20.197937+0000","last_fullsized":"2024-01-11T07:08:20.197937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:22.022631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189025+0000","last_change":"2024-01-11T07:08:20.188895+0000","last_active":"2024-01-11T07:08:20.189025+0000","last_peered":"2024-01-11T07:08:20.189025+0000","last_clean":"2024-01-11T07:08:20.189025+0000","last_became_active":"2024-01-11T07:08:20.188507+0000","last_became_peered":"2024-01-11T07:08:20.188507+0000","last_unstale":"2024-01-11T07:08:20.189025+0000","last_undegraded":"2024-01-11T07:08:20.189025+0000","last_fullsized":"2024-01-11T07:08:20.189025+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:50:49.026546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198775+0000","last_change":"2024-01-11T07:08:20.198582+0000","last_active":"2024-01-11T07:08:20.198775+0000","last_peered":"2024-01-11T07:08:20.198775+0000","last_clean":"2024-01-11T07:08:20.198775+0000","last_became_active":"2024-01-11T07:08:20.198342+0000","last_became_peered":"2024-01-11T07:08:20.198342+0000","last_unstale":"2024-01-11T07:08:20.198775+0000","last_undegraded":"2024-01-11T07:08:20.198775+0000","last_fullsized":"2024-01-11T07:08:20.198775+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:03:35.170409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.195874+0000","last_change":"2024-01-11T07:08:20.195724+0000","last_active":"2024-01-11T07:08:20.195874+0000","last_peered":"2024-01-11T07:08:20.195874+0000","last_clean":"2024-01-11T07:08:20.195874+0000","last_became_active":"2024-01-11T07:08:20.195137+0000","last_became_peered":"2024-01-11T07:08:20.195137+0000","last_unstale":"2024-01-11T07:08:20.195874+0000","last_undegraded":"2024-01-11T07:08:20.195874+0000","last_fullsized":"2024-01-11T07:08:20.195874+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:01:20.083688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198699+0000","last_change":"2024-01-11T07:08:20.198637+0000","last_active":"2024-01-11T07:08:20.198699+0000","last_peered":"2024-01-11T07:08:20.198699+0000","last_clean":"2024-01-11T07:08:20.198699+0000","last_became_active":"2024-01-11T07:08:20.198393+0000","last_became_peered":"2024-01-11T07:08:20.198393+0000","last_unstale":"2024-01-11T07:08:20.198699+0000","last_undegraded":"2024-01-11T07:08:20.198699+0000","last_fullsized":"2024-01-11T07:08:20.198699+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:21:03.453181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207309+0000","last_change":"2024-01-11T07:08:20.206298+0000","last_active":"2024-01-11T07:08:20.207309+0000","last_peered":"2024-01-11T07:08:20.207309+0000","last_clean":"2024-01-11T07:08:20.207309+0000","last_became_active":"2024-01-11T07:08:20.201918+0000","last_became_peered":"2024-01-11T07:08:20.201918+0000","last_unstale":"2024-01-11T07:08:20.207309+0000","last_undegraded":"2024-01-11T07:08:20.207309+0000","last_fullsized":"2024-01-11T07:08:20.207309+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:43:35.757837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194507+0000","last_change":"2024-01-11T07:08:20.194403+0000","last_active":"2024-01-11T07:08:20.194507+0000","last_peered":"2024-01-11T07:08:20.194507+0000","last_clean":"2024-01-11T07:08:20.194507+0000","last_became_active":"2024-01-11T07:08:20.193864+0000","last_became_peered":"2024-01-11T07:08:20.193864+0000","last_unstale":"2024-01-11T07:08:20.194507+0000","last_undegraded":"2024-01-11T07:08:20.194507+0000","last_fullsized":"2024-01-11T07:08:20.194507+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:03.517036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194722+0000","last_change":"2024-01-11T07:08:20.194666+0000","last_active":"2024-01-11T07:08:20.194722+0000","last_peered":"2024-01-11T07:08:20.194722+0000","last_clean":"2024-01-11T07:08:20.194722+0000","last_became_active":"2024-01-11T07:08:20.194470+0000","last_became_peered":"2024-01-11T07:08:20.194470+0000","last_unstale":"2024-01-11T07:08:20.194722+0000","last_undegraded":"2024-01-11T07:08:20.194722+0000","last_fullsized":"2024-01-11T07:08:20.194722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:48.860647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.195930+0000","last_change":"2024-01-11T07:08:20.195875+0000","last_active":"2024-01-11T07:08:20.195930+0000","last_peered":"2024-01-11T07:08:20.195930+0000","last_clean":"2024-01-11T07:08:20.195930+0000","last_became_active":"2024-01-11T07:08:20.195527+0000","last_became_peered":"2024-01-11T07:08:20.195527+0000","last_unstale":"2024-01-11T07:08:20.195930+0000","last_undegraded":"2024-01-11T07:08:20.195930+0000","last_fullsized":"2024-01-11T07:08:20.195930+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:01.586031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201300+0000","last_change":"2024-01-11T07:08:20.201146+0000","last_active":"2024-01-11T07:08:20.201300+0000","last_peered":"2024-01-11T07:08:20.201300+0000","last_clean":"2024-01-11T07:08:20.201300+0000","last_became_active":"2024-01-11T07:08:20.200921+0000","last_became_peered":"2024-01-11T07:08:20.200921+0000","last_unstale":"2024-01-11T07:08:20.201300+0000","last_undegraded":"2024-01-11T07:08:20.201300+0000","last_fullsized":"2024-01-11T07:08:20.201300+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:59:26.717733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.208920+0000","last_change":"2024-01-11T07:08:20.208547+0000","last_active":"2024-01-11T07:08:20.208920+0000","last_peered":"2024-01-11T07:08:20.208920+0000","last_clean":"2024-01-11T07:08:20.208920+0000","last_became_active":"2024-01-11T07:08:20.208162+0000","last_became_peered":"2024-01-11T07:08:20.208162+0000","last_unstale":"2024-01-11T07:08:20.208920+0000","last_undegraded":"2024-01-11T07:08:20.208920+0000","last_fullsized":"2024-01-11T07:08:20.208920+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:34.414872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189007+0000","last_change":"2024-01-11T07:08:20.188720+0000","last_active":"2024-01-11T07:08:20.189007+0000","last_peered":"2024-01-11T07:08:20.189007+0000","last_clean":"2024-01-11T07:08:20.189007+0000","last_became_active":"2024-01-11T07:08:20.188228+0000","last_became_peered":"2024-01-11T07:08:20.188228+0000","last_unstale":"2024-01-11T07:08:20.189007+0000","last_undegraded":"2024-01-11T07:08:20.189007+0000","last_fullsized":"2024-01-11T07:08:20.189007+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:07:00.866101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194876+0000","last_change":"2024-01-11T07:08:20.194725+0000","last_active":"2024-01-11T07:08:20.194876+0000","last_peered":"2024-01-11T07:08:20.194876+0000","last_clean":"2024-01-11T07:08:20.194876+0000","last_became_active":"2024-01-11T07:08:20.194486+0000","last_became_peered":"2024-01-11T07:08:20.194486+0000","last_unstale":"2024-01-11T07:08:20.194876+0000","last_undegraded":"2024-01-11T07:08:20.194876+0000","last_fullsized":"2024-01-11T07:08:20.194876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:18:35.823955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203163+0000","last_change":"2024-01-11T07:08:20.203093+0000","last_active":"2024-01-11T07:08:20.203163+0000","last_peered":"2024-01-11T07:08:20.203163+0000","last_clean":"2024-01-11T07:08:20.203163+0000","last_became_active":"2024-01-11T07:08:20.202833+0000","last_became_peered":"2024-01-11T07:08:20.202833+0000","last_unstale":"2024-01-11T07:08:20.203163+0000","last_undegraded":"2024-01-11T07:08:20.203163+0000","last_fullsized":"2024-01-11T07:08:20.203163+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:39:51.189000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194126+0000","last_change":"2024-01-11T07:08:20.193537+0000","last_active":"2024-01-11T07:08:20.194126+0000","last_peered":"2024-01-11T07:08:20.194126+0000","last_clean":"2024-01-11T07:08:20.194126+0000","last_became_active":"2024-01-11T07:08:20.193316+0000","last_became_peered":"2024-01-11T07:08:20.193316+0000","last_unstale":"2024-01-11T07:08:20.194126+0000","last_undegraded":"2024-01-11T07:08:20.194126+0000","last_fullsized":"2024-01-11T07:08:20.194126+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:31.579570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207290+0000","last_change":"2024-01-11T07:08:20.205602+0000","last_active":"2024-01-11T07:08:20.207290+0000","last_peered":"2024-01-11T07:08:20.207290+0000","last_clean":"2024-01-11T07:08:20.207290+0000","last_became_active":"2024-01-11T07:08:20.201609+0000","last_became_peered":"2024-01-11T07:08:20.201609+0000","last_unstale":"2024-01-11T07:08:20.207290+0000","last_undegraded":"2024-01-11T07:08:20.207290+0000","last_fullsized":"2024-01-11T07:08:20.207290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:24:38.219798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.204605+0000","last_change":"2024-01-11T07:08:20.204488+0000","last_active":"2024-01-11T07:08:20.204605+0000","last_peered":"2024-01-11T07:08:20.204605+0000","last_clean":"2024-01-11T07:08:20.204605+0000","last_became_active":"2024-01-11T07:08:20.204253+0000","last_became_peered":"2024-01-11T07:08:20.204253+0000","last_unstale":"2024-01-11T07:08:20.204605+0000","last_undegraded":"2024-01-11T07:08:20.204605+0000","last_fullsized":"2024-01-11T07:08:20.204605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:47:01.654557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201752+0000","last_change":"2024-01-11T07:08:20.201661+0000","last_active":"2024-01-11T07:08:20.201752+0000","last_peered":"2024-01-11T07:08:20.201752+0000","last_clean":"2024-01-11T07:08:20.201752+0000","last_became_active":"2024-01-11T07:08:20.200942+0000","last_became_peered":"2024-01-11T07:08:20.200942+0000","last_unstale":"2024-01-11T07:08:20.201752+0000","last_undegraded":"2024-01-11T07:08:20.201752+0000","last_fullsized":"2024-01-11T07:08:20.201752+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:31.579570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207444+0000","last_change":"2024-01-11T07:08:19.190727+0000","last_active":"2024-01-11T07:08:20.207444+0000","last_peered":"2024-01-11T07:08:20.207444+0000","last_clean":"2024-01-11T07:08:20.207444+0000","last_became_active":"2024-01-11T07:08:19.190254+0000","last_became_peered":"2024-01-11T07:08:19.190254+0000","last_unstale":"2024-01-11T07:08:20.207444+0000","last_undegraded":"2024-01-11T07:08:20.207444+0000","last_fullsized":"2024-01-11T07:08:20.207444+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:06:59.864072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194340+0000","last_change":"2024-01-11T07:08:20.194287+0000","last_active":"2024-01-11T07:08:20.194340+0000","last_peered":"2024-01-11T07:08:20.194340+0000","last_clean":"2024-01-11T07:08:20.194340+0000","last_became_active":"2024-01-11T07:08:20.193701+0000","last_became_peered":"2024-01-11T07:08:20.193701+0000","last_unstale":"2024-01-11T07:08:20.194340+0000","last_undegraded":"2024-01-11T07:08:20.194340+0000","last_fullsized":"2024-01-11T07:08:20.194340+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:27:16.713517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205515+0000","last_change":"2024-01-11T07:08:19.193499+0000","last_active":"2024-01-11T07:08:20.205515+0000","last_peered":"2024-01-11T07:08:20.205515+0000","last_clean":"2024-01-11T07:08:20.205515+0000","last_became_active":"2024-01-11T07:08:19.193251+0000","last_became_peered":"2024-01-11T07:08:19.193251+0000","last_unstale":"2024-01-11T07:08:20.205515+0000","last_undegraded":"2024-01-11T07:08:20.205515+0000","last_fullsized":"2024-01-11T07:08:20.205515+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:36.551051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.202722+0000","last_change":"2024-01-11T07:08:20.202656+0000","last_active":"2024-01-11T07:08:20.202722+0000","last_peered":"2024-01-11T07:08:20.202722+0000","last_clean":"2024-01-11T07:08:20.202722+0000","last_became_active":"2024-01-11T07:08:20.202296+0000","last_became_peered":"2024-01-11T07:08:20.202296+0000","last_unstale":"2024-01-11T07:08:20.202722+0000","last_undegraded":"2024-01-11T07:08:20.202722+0000","last_fullsized":"2024-01-11T07:08:20.202722+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:55.269737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.206864+0000","last_change":"2024-01-11T07:08:19.195656+0000","last_active":"2024-01-11T07:08:20.206864+0000","last_peered":"2024-01-11T07:08:20.206864+0000","last_clean":"2024-01-11T07:08:20.206864+0000","last_became_active":"2024-01-11T07:08:19.195322+0000","last_became_peered":"2024-01-11T07:08:19.195322+0000","last_unstale":"2024-01-11T07:08:20.206864+0000","last_undegraded":"2024-01-11T07:08:20.206864+0000","last_fullsized":"2024-01-11T07:08:20.206864+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:59:25.715704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201243+0000","last_change":"2024-01-11T07:08:20.201115+0000","last_active":"2024-01-11T07:08:20.201243+0000","last_peered":"2024-01-11T07:08:20.201243+0000","last_clean":"2024-01-11T07:08:20.201243+0000","last_became_active":"2024-01-11T07:08:20.200690+0000","last_became_peered":"2024-01-11T07:08:20.200690+0000","last_unstale":"2024-01-11T07:08:20.201243+0000","last_undegraded":"2024-01-11T07:08:20.201243+0000","last_fullsized":"2024-01-11T07:08:20.201243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:01:20.083688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203421+0000","last_change":"2024-01-11T07:08:19.194071+0000","last_active":"2024-01-11T07:08:20.203421+0000","last_peered":"2024-01-11T07:08:20.203421+0000","last_clean":"2024-01-11T07:08:20.203421+0000","last_became_active":"2024-01-11T07:08:19.193826+0000","last_became_peered":"2024-01-11T07:08:19.193826+0000","last_unstale":"2024-01-11T07:08:20.203421+0000","last_undegraded":"2024-01-11T07:08:20.203421+0000","last_fullsized":"2024-01-11T07:08:20.203421+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:06:59.864072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.193411+0000","last_change":"2024-01-11T07:08:20.192206+0000","last_active":"2024-01-11T07:08:20.193411+0000","last_peered":"2024-01-11T07:08:20.193411+0000","last_clean":"2024-01-11T07:08:20.193411+0000","last_became_active":"2024-01-11T07:08:20.191989+0000","last_became_peered":"2024-01-11T07:08:20.191989+0000","last_unstale":"2024-01-11T07:08:20.193411+0000","last_undegraded":"2024-01-11T07:08:20.193411+0000","last_fullsized":"2024-01-11T07:08:20.193411+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:02:16.481219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207420+0000","last_change":"2024-01-11T07:08:19.190068+0000","last_active":"2024-01-11T07:08:20.207420+0000","last_peered":"2024-01-11T07:08:20.207420+0000","last_clean":"2024-01-11T07:08:20.207420+0000","last_became_active":"2024-01-11T07:08:19.189360+0000","last_became_peered":"2024-01-11T07:08:19.189360+0000","last_unstale":"2024-01-11T07:08:20.207420+0000","last_undegraded":"2024-01-11T07:08:20.207420+0000","last_fullsized":"2024-01-11T07:08:20.207420+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:15:57.438193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197093+0000","last_change":"2024-01-11T07:08:20.196667+0000","last_active":"2024-01-11T07:08:20.197093+0000","last_peered":"2024-01-11T07:08:20.197093+0000","last_clean":"2024-01-11T07:08:20.197093+0000","last_became_active":"2024-01-11T07:08:20.196419+0000","last_became_peered":"2024-01-11T07:08:20.196419+0000","last_unstale":"2024-01-11T07:08:20.197093+0000","last_undegraded":"2024-01-11T07:08:20.197093+0000","last_fullsized":"2024-01-11T07:08:20.197093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:02:16.481219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205639+0000","last_change":"2024-01-11T07:08:19.196131+0000","last_active":"2024-01-11T07:08:20.205639+0000","last_peered":"2024-01-11T07:08:20.205639+0000","last_clean":"2024-01-11T07:08:20.205639+0000","last_became_active":"2024-01-11T07:08:19.195665+0000","last_became_peered":"2024-01-11T07:08:19.195665+0000","last_unstale":"2024-01-11T07:08:20.205639+0000","last_undegraded":"2024-01-11T07:08:20.205639+0000","last_fullsized":"2024-01-11T07:08:20.205639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:12:11.866608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192884+0000","last_change":"2024-01-11T07:08:20.192652+0000","last_active":"2024-01-11T07:08:20.192884+0000","last_peered":"2024-01-11T07:08:20.192884+0000","last_clean":"2024-01-11T07:08:20.192884+0000","last_became_active":"2024-01-11T07:08:20.192244+0000","last_became_peered":"2024-01-11T07:08:20.192244+0000","last_unstale":"2024-01-11T07:08:20.192884+0000","last_undegraded":"2024-01-11T07:08:20.192884+0000","last_fullsized":"2024-01-11T07:08:20.192884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:22:43.122964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.185973+0000","last_change":"2024-01-11T07:08:19.193865+0000","last_active":"2024-01-11T07:08:20.185973+0000","last_peered":"2024-01-11T07:08:20.185973+0000","last_clean":"2024-01-11T07:08:20.185973+0000","last_became_active":"2024-01-11T07:08:19.193487+0000","last_became_peered":"2024-01-11T07:08:19.193487+0000","last_unstale":"2024-01-11T07:08:20.185973+0000","last_undegraded":"2024-01-11T07:08:20.185973+0000","last_fullsized":"2024-01-11T07:08:20.185973+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:08:27.086505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196455+0000","last_change":"2024-01-11T07:08:20.196334+0000","last_active":"2024-01-11T07:08:20.196455+0000","last_peered":"2024-01-11T07:08:20.196455+0000","last_clean":"2024-01-11T07:08:20.196455+0000","last_became_active":"2024-01-11T07:08:20.196103+0000","last_became_peered":"2024-01-11T07:08:20.196103+0000","last_unstale":"2024-01-11T07:08:20.196455+0000","last_undegraded":"2024-01-11T07:08:20.196455+0000","last_fullsized":"2024-01-11T07:08:20.196455+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:20:30.428409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.185008+0000","last_change":"2024-01-11T07:08:19.192371+0000","last_active":"2024-01-11T07:08:20.185008+0000","last_peered":"2024-01-11T07:08:20.185008+0000","last_clean":"2024-01-11T07:08:20.185008+0000","last_became_active":"2024-01-11T07:08:19.192074+0000","last_became_peered":"2024-01-11T07:08:19.192074+0000","last_unstale":"2024-01-11T07:08:20.185008+0000","last_undegraded":"2024-01-11T07:08:20.185008+0000","last_fullsized":"2024-01-11T07:08:20.185008+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:47.858617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189855+0000","last_change":"2024-01-11T07:08:20.189726+0000","last_active":"2024-01-11T07:08:20.189855+0000","last_peered":"2024-01-11T07:08:20.189855+0000","last_clean":"2024-01-11T07:08:20.189855+0000","last_became_active":"2024-01-11T07:08:20.189484+0000","last_became_peered":"2024-01-11T07:08:20.189484+0000","last_unstale":"2024-01-11T07:08:20.189855+0000","last_undegraded":"2024-01-11T07:08:20.189855+0000","last_fullsized":"2024-01-11T07:08:20.189855+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:39.407590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.185421+0000","last_change":"2024-01-11T07:08:19.194714+0000","last_active":"2024-01-11T07:08:20.185421+0000","last_peered":"2024-01-11T07:08:20.185421+0000","last_clean":"2024-01-11T07:08:20.185421+0000","last_became_active":"2024-01-11T07:08:19.194450+0000","last_became_peered":"2024-01-11T07:08:19.194450+0000","last_unstale":"2024-01-11T07:08:20.185421+0000","last_undegraded":"2024-01-11T07:08:20.185421+0000","last_fullsized":"2024-01-11T07:08:20.185421+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:47.858617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194490+0000","last_change":"2024-01-11T07:08:20.194425+0000","last_active":"2024-01-11T07:08:20.194490+0000","last_peered":"2024-01-11T07:08:20.194490+0000","last_clean":"2024-01-11T07:08:20.194490+0000","last_became_active":"2024-01-11T07:08:20.193860+0000","last_became_peered":"2024-01-11T07:08:20.193860+0000","last_unstale":"2024-01-11T07:08:20.194490+0000","last_undegraded":"2024-01-11T07:08:20.194490+0000","last_fullsized":"2024-01-11T07:08:20.194490+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:21:42.367721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.184425+0000","last_change":"2024-01-11T07:08:19.193264+0000","last_active":"2024-01-11T07:08:20.184425+0000","last_peered":"2024-01-11T07:08:20.184425+0000","last_clean":"2024-01-11T07:08:20.184425+0000","last_became_active":"2024-01-11T07:08:19.192455+0000","last_became_peered":"2024-01-11T07:08:19.192455+0000","last_unstale":"2024-01-11T07:08:20.184425+0000","last_undegraded":"2024-01-11T07:08:20.184425+0000","last_fullsized":"2024-01-11T07:08:20.184425+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:57.779357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196107+0000","last_change":"2024-01-11T07:08:20.195266+0000","last_active":"2024-01-11T07:08:20.196107+0000","last_peered":"2024-01-11T07:08:20.196107+0000","last_clean":"2024-01-11T07:08:20.196107+0000","last_became_active":"2024-01-11T07:08:20.194866+0000","last_became_peered":"2024-01-11T07:08:20.194866+0000","last_unstale":"2024-01-11T07:08:20.196107+0000","last_undegraded":"2024-01-11T07:08:20.196107+0000","last_fullsized":"2024-01-11T07:08:20.196107+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:43:06.345810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.204215+0000","last_change":"2024-01-11T07:08:19.191266+0000","last_active":"2024-01-11T07:08:20.204215+0000","last_peered":"2024-01-11T07:08:20.204215+0000","last_clean":"2024-01-11T07:08:20.204215+0000","last_became_active":"2024-01-11T07:08:19.190885+0000","last_became_peered":"2024-01-11T07:08:19.190885+0000","last_unstale":"2024-01-11T07:08:20.204215+0000","last_undegraded":"2024-01-11T07:08:20.204215+0000","last_fullsized":"2024-01-11T07:08:20.204215+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:33.000091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203070+0000","last_change":"2024-01-11T07:08:20.202282+0000","last_active":"2024-01-11T07:08:20.203070+0000","last_peered":"2024-01-11T07:08:20.203070+0000","last_clean":"2024-01-11T07:08:20.203070+0000","last_became_active":"2024-01-11T07:08:20.202063+0000","last_became_peered":"2024-01-11T07:08:20.202063+0000","last_unstale":"2024-01-11T07:08:20.203070+0000","last_undegraded":"2024-01-11T07:08:20.203070+0000","last_fullsized":"2024-01-11T07:08:20.203070+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:57.072685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.211563+0000","last_change":"2024-01-11T07:08:19.195225+0000","last_active":"2024-01-11T07:08:20.211563+0000","last_peered":"2024-01-11T07:08:20.211563+0000","last_clean":"2024-01-11T07:08:20.211563+0000","last_became_active":"2024-01-11T07:08:19.194975+0000","last_became_peered":"2024-01-11T07:08:19.194975+0000","last_unstale":"2024-01-11T07:08:20.211563+0000","last_undegraded":"2024-01-11T07:08:20.211563+0000","last_fullsized":"2024-01-11T07:08:20.211563+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:27.730368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197759+0000","last_change":"2024-01-11T07:08:20.197679+0000","last_active":"2024-01-11T07:08:20.197759+0000","last_peered":"2024-01-11T07:08:20.197759+0000","last_clean":"2024-01-11T07:08:20.197759+0000","last_became_active":"2024-01-11T07:08:20.197235+0000","last_became_peered":"2024-01-11T07:08:20.197235+0000","last_unstale":"2024-01-11T07:08:20.197759+0000","last_undegraded":"2024-01-11T07:08:20.197759+0000","last_fullsized":"2024-01-11T07:08:20.197759+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:41:01.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.205746+0000","last_change":"2024-01-11T07:08:19.192936+0000","last_active":"2024-01-11T07:08:20.205746+0000","last_peered":"2024-01-11T07:08:20.205746+0000","last_clean":"2024-01-11T07:08:20.205746+0000","last_became_active":"2024-01-11T07:08:19.191106+0000","last_became_peered":"2024-01-11T07:08:19.191106+0000","last_unstale":"2024-01-11T07:08:20.205746+0000","last_undegraded":"2024-01-11T07:08:20.205746+0000","last_fullsized":"2024-01-11T07:08:20.205746+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:33.000091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197056+0000","last_change":"2024-01-11T07:08:20.196029+0000","last_active":"2024-01-11T07:08:20.197056+0000","last_peered":"2024-01-11T07:08:20.197056+0000","last_clean":"2024-01-11T07:08:20.197056+0000","last_became_active":"2024-01-11T07:08:20.195439+0000","last_became_peered":"2024-01-11T07:08:20.195439+0000","last_unstale":"2024-01-11T07:08:20.197056+0000","last_undegraded":"2024-01-11T07:08:20.197056+0000","last_fullsized":"2024-01-11T07:08:20.197056+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:50:49.026546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192011+0000","last_change":"2024-01-11T07:08:19.193437+0000","last_active":"2024-01-11T07:08:20.192011+0000","last_peered":"2024-01-11T07:08:20.192011+0000","last_clean":"2024-01-11T07:08:20.192011+0000","last_became_active":"2024-01-11T07:08:19.192841+0000","last_became_peered":"2024-01-11T07:08:19.192841+0000","last_unstale":"2024-01-11T07:08:20.192011+0000","last_undegraded":"2024-01-11T07:08:20.192011+0000","last_fullsized":"2024-01-11T07:08:20.192011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:02:15.479190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.202956+0000","last_change":"2024-01-11T07:08:20.202642+0000","last_active":"2024-01-11T07:08:20.202956+0000","last_peered":"2024-01-11T07:08:20.202956+0000","last_clean":"2024-01-11T07:08:20.202956+0000","last_became_active":"2024-01-11T07:08:20.202155+0000","last_became_peered":"2024-01-11T07:08:20.202155+0000","last_unstale":"2024-01-11T07:08:20.202956+0000","last_undegraded":"2024-01-11T07:08:20.202956+0000","last_fullsized":"2024-01-11T07:08:20.202956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:40:53.941699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189704+0000","last_change":"2024-01-11T07:08:19.189618+0000","last_active":"2024-01-11T07:08:20.189704+0000","last_peered":"2024-01-11T07:08:20.189704+0000","last_clean":"2024-01-11T07:08:20.189704+0000","last_became_active":"2024-01-11T07:08:19.189004+0000","last_became_peered":"2024-01-11T07:08:19.189004+0000","last_unstale":"2024-01-11T07:08:20.189704+0000","last_undegraded":"2024-01-11T07:08:20.189704+0000","last_fullsized":"2024-01-11T07:08:20.189704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:54.267708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197088+0000","last_change":"2024-01-11T07:08:20.196988+0000","last_active":"2024-01-11T07:08:20.197088+0000","last_peered":"2024-01-11T07:08:20.197088+0000","last_clean":"2024-01-11T07:08:20.197088+0000","last_became_active":"2024-01-11T07:08:20.196752+0000","last_became_peered":"2024-01-11T07:08:20.196752+0000","last_unstale":"2024-01-11T07:08:20.197088+0000","last_undegraded":"2024-01-11T07:08:20.197088+0000","last_fullsized":"2024-01-11T07:08:20.197088+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:22:43.122964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.210422+0000","last_change":"2024-01-11T07:08:19.191492+0000","last_active":"2024-01-11T07:08:20.210422+0000","last_peered":"2024-01-11T07:08:20.210422+0000","last_clean":"2024-01-11T07:08:20.210422+0000","last_became_active":"2024-01-11T07:08:19.191199+0000","last_became_peered":"2024-01-11T07:08:19.191199+0000","last_unstale":"2024-01-11T07:08:20.210422+0000","last_undegraded":"2024-01-11T07:08:20.210422+0000","last_fullsized":"2024-01-11T07:08:20.210422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:36:32.123150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197103+0000","last_change":"2024-01-11T07:08:20.196101+0000","last_active":"2024-01-11T07:08:20.197103+0000","last_peered":"2024-01-11T07:08:20.197103+0000","last_clean":"2024-01-11T07:08:20.197103+0000","last_became_active":"2024-01-11T07:08:20.195843+0000","last_became_peered":"2024-01-11T07:08:20.195843+0000","last_unstale":"2024-01-11T07:08:20.197103+0000","last_undegraded":"2024-01-11T07:08:20.197103+0000","last_fullsized":"2024-01-11T07:08:20.197103+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:01.134482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.209375+0000","last_change":"2024-01-11T07:08:19.189720+0000","last_active":"2024-01-11T07:08:20.209375+0000","last_peered":"2024-01-11T07:08:20.209375+0000","last_clean":"2024-01-11T07:08:20.209375+0000","last_became_active":"2024-01-11T07:08:19.189485+0000","last_became_peered":"2024-01-11T07:08:19.189485+0000","last_unstale":"2024-01-11T07:08:20.209375+0000","last_undegraded":"2024-01-11T07:08:20.209375+0000","last_fullsized":"2024-01-11T07:08:20.209375+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:48.988448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.199963+0000","last_change":"2024-01-11T07:08:20.199890+0000","last_active":"2024-01-11T07:08:20.199963+0000","last_peered":"2024-01-11T07:08:20.199963+0000","last_clean":"2024-01-11T07:08:20.199963+0000","last_became_active":"2024-01-11T07:08:20.199629+0000","last_became_peered":"2024-01-11T07:08:20.199629+0000","last_unstale":"2024-01-11T07:08:20.199963+0000","last_undegraded":"2024-01-11T07:08:20.199963+0000","last_fullsized":"2024-01-11T07:08:20.199963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:15:58.440223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.206869+0000","last_change":"2024-01-11T07:08:19.192733+0000","last_active":"2024-01-11T07:08:20.206869+0000","last_peered":"2024-01-11T07:08:20.206869+0000","last_clean":"2024-01-11T07:08:20.206869+0000","last_became_active":"2024-01-11T07:08:19.192509+0000","last_became_peered":"2024-01-11T07:08:19.192509+0000","last_unstale":"2024-01-11T07:08:20.206869+0000","last_undegraded":"2024-01-11T07:08:20.206869+0000","last_fullsized":"2024-01-11T07:08:20.206869+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:36:32.123150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198065+0000","last_change":"2024-01-11T07:08:20.197998+0000","last_active":"2024-01-11T07:08:20.198065+0000","last_peered":"2024-01-11T07:08:20.198065+0000","last_clean":"2024-01-11T07:08:20.198065+0000","last_became_active":"2024-01-11T07:08:20.197352+0000","last_became_peered":"2024-01-11T07:08:20.197352+0000","last_unstale":"2024-01-11T07:08:20.198065+0000","last_undegraded":"2024-01-11T07:08:20.198065+0000","last_fullsized":"2024-01-11T07:08:20.198065+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:08:45.898498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.204145+0000","last_change":"2024-01-11T07:08:19.191891+0000","last_active":"2024-01-11T07:08:20.204145+0000","last_peered":"2024-01-11T07:08:20.204145+0000","last_clean":"2024-01-11T07:08:20.204145+0000","last_became_active":"2024-01-11T07:08:19.191660+0000","last_became_peered":"2024-01-11T07:08:19.191660+0000","last_unstale":"2024-01-11T07:08:20.204145+0000","last_undegraded":"2024-01-11T07:08:20.204145+0000","last_fullsized":"2024-01-11T07:08:20.204145+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:03:34.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.199159+0000","last_change":"2024-01-11T07:08:20.199093+0000","last_active":"2024-01-11T07:08:20.199159+0000","last_peered":"2024-01-11T07:08:20.199159+0000","last_clean":"2024-01-11T07:08:20.199159+0000","last_became_active":"2024-01-11T07:08:20.198880+0000","last_became_peered":"2024-01-11T07:08:20.198880+0000","last_unstale":"2024-01-11T07:08:20.199159+0000","last_undegraded":"2024-01-11T07:08:20.199159+0000","last_fullsized":"2024-01-11T07:08:20.199159+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:40:27.902284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.189820+0000","last_change":"2024-01-11T07:08:19.194355+0000","last_active":"2024-01-11T07:08:20.189820+0000","last_peered":"2024-01-11T07:08:20.189820+0000","last_clean":"2024-01-11T07:08:20.189820+0000","last_became_active":"2024-01-11T07:08:19.194093+0000","last_became_peered":"2024-01-11T07:08:19.194093+0000","last_unstale":"2024-01-11T07:08:20.189820+0000","last_undegraded":"2024-01-11T07:08:20.189820+0000","last_fullsized":"2024-01-11T07:08:20.189820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:47.275593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196086+0000","last_change":"2024-01-11T07:08:20.195107+0000","last_active":"2024-01-11T07:08:20.196086+0000","last_peered":"2024-01-11T07:08:20.196086+0000","last_clean":"2024-01-11T07:08:20.196086+0000","last_became_active":"2024-01-11T07:08:20.194547+0000","last_became_peered":"2024-01-11T07:08:20.194547+0000","last_unstale":"2024-01-11T07:08:20.196086+0000","last_undegraded":"2024-01-11T07:08:20.196086+0000","last_fullsized":"2024-01-11T07:08:20.196086+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:35:14.232973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207218+0000","last_change":"2024-01-11T07:08:19.190574+0000","last_active":"2024-01-11T07:08:20.207218+0000","last_peered":"2024-01-11T07:08:20.207218+0000","last_clean":"2024-01-11T07:08:20.207218+0000","last_became_active":"2024-01-11T07:08:19.189479+0000","last_became_peered":"2024-01-11T07:08:19.189479+0000","last_unstale":"2024-01-11T07:08:20.207218+0000","last_undegraded":"2024-01-11T07:08:20.207218+0000","last_fullsized":"2024-01-11T07:08:20.207218+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:38.405561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.201083+0000","last_change":"2024-01-11T07:08:20.200551+0000","last_active":"2024-01-11T07:08:20.201083+0000","last_peered":"2024-01-11T07:08:20.201083+0000","last_clean":"2024-01-11T07:08:20.201083+0000","last_became_active":"2024-01-11T07:08:20.200244+0000","last_became_peered":"2024-01-11T07:08:20.200244+0000","last_unstale":"2024-01-11T07:08:20.201083+0000","last_undegraded":"2024-01-11T07:08:20.201083+0000","last_fullsized":"2024-01-11T07:08:20.201083+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:08:45.898498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.186106+0000","last_change":"2024-01-11T07:08:19.193728+0000","last_active":"2024-01-11T07:08:20.186106+0000","last_peered":"2024-01-11T07:08:20.186106+0000","last_clean":"2024-01-11T07:08:20.186106+0000","last_became_active":"2024-01-11T07:08:19.193174+0000","last_became_peered":"2024-01-11T07:08:19.193174+0000","last_unstale":"2024-01-11T07:08:20.186106+0000","last_undegraded":"2024-01-11T07:08:20.186106+0000","last_fullsized":"2024-01-11T07:08:20.186106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:12:26.681729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198019+0000","last_change":"2024-01-11T07:08:20.197949+0000","last_active":"2024-01-11T07:08:20.198019+0000","last_peered":"2024-01-11T07:08:20.198019+0000","last_clean":"2024-01-11T07:08:20.198019+0000","last_became_active":"2024-01-11T07:08:20.197700+0000","last_became_peered":"2024-01-11T07:08:20.197700+0000","last_unstale":"2024-01-11T07:08:20.198019+0000","last_undegraded":"2024-01-11T07:08:20.198019+0000","last_fullsized":"2024-01-11T07:08:20.198019+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:49:29.301371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.184902+0000","last_change":"2024-01-11T07:08:19.190534+0000","last_active":"2024-01-11T07:08:20.184902+0000","last_peered":"2024-01-11T07:08:20.184902+0000","last_clean":"2024-01-11T07:08:20.184902+0000","last_became_active":"2024-01-11T07:08:19.190208+0000","last_became_peered":"2024-01-11T07:08:19.190208+0000","last_unstale":"2024-01-11T07:08:20.184902+0000","last_undegraded":"2024-01-11T07:08:20.184902+0000","last_fullsized":"2024-01-11T07:08:20.184902+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:12:11.866608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.202900+0000","last_change":"2024-01-11T07:08:20.202798+0000","last_active":"2024-01-11T07:08:20.202900+0000","last_peered":"2024-01-11T07:08:20.202900+0000","last_clean":"2024-01-11T07:08:20.202900+0000","last_became_active":"2024-01-11T07:08:20.202567+0000","last_became_peered":"2024-01-11T07:08:20.202567+0000","last_unstale":"2024-01-11T07:08:20.202900+0000","last_undegraded":"2024-01-11T07:08:20.202900+0000","last_fullsized":"2024-01-11T07:08:20.202900+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:55.269737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192104+0000","last_change":"2024-01-11T07:08:19.193553+0000","last_active":"2024-01-11T07:08:20.192104+0000","last_peered":"2024-01-11T07:08:20.192104+0000","last_clean":"2024-01-11T07:08:20.192104+0000","last_became_active":"2024-01-11T07:08:19.193113+0000","last_became_peered":"2024-01-11T07:08:19.193113+0000","last_unstale":"2024-01-11T07:08:20.192104+0000","last_undegraded":"2024-01-11T07:08:20.192104+0000","last_fullsized":"2024-01-11T07:08:20.192104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:50:48.024516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.198232+0000","last_change":"2024-01-11T07:08:20.198157+0000","last_active":"2024-01-11T07:08:20.198232+0000","last_peered":"2024-01-11T07:08:20.198232+0000","last_clean":"2024-01-11T07:08:20.198232+0000","last_became_active":"2024-01-11T07:08:20.197590+0000","last_became_peered":"2024-01-11T07:08:20.197590+0000","last_unstale":"2024-01-11T07:08:20.198232+0000","last_undegraded":"2024-01-11T07:08:20.198232+0000","last_fullsized":"2024-01-11T07:08:20.198232+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:31.579570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.193256+0000","last_change":"2024-01-11T07:08:19.196552+0000","last_active":"2024-01-11T07:08:20.193256+0000","last_peered":"2024-01-11T07:08:20.193256+0000","last_clean":"2024-01-11T07:08:20.193256+0000","last_became_active":"2024-01-11T07:08:19.196286+0000","last_became_peered":"2024-01-11T07:08:19.196286+0000","last_unstale":"2024-01-11T07:08:20.193256+0000","last_undegraded":"2024-01-11T07:08:20.193256+0000","last_fullsized":"2024-01-11T07:08:20.193256+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:03:34.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197074+0000","last_change":"2024-01-11T07:08:20.196206+0000","last_active":"2024-01-11T07:08:20.197074+0000","last_peered":"2024-01-11T07:08:20.197074+0000","last_clean":"2024-01-11T07:08:20.197074+0000","last_became_active":"2024-01-11T07:08:20.195784+0000","last_became_peered":"2024-01-11T07:08:20.195784+0000","last_unstale":"2024-01-11T07:08:20.197074+0000","last_undegraded":"2024-01-11T07:08:20.197074+0000","last_fullsized":"2024-01-11T07:08:20.197074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:18:35.823955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.191296+0000","last_change":"2024-01-11T07:08:19.192165+0000","last_active":"2024-01-11T07:08:20.191296+0000","last_peered":"2024-01-11T07:08:20.191296+0000","last_clean":"2024-01-11T07:08:20.191296+0000","last_became_active":"2024-01-11T07:08:19.191920+0000","last_became_peered":"2024-01-11T07:08:19.191920+0000","last_unstale":"2024-01-11T07:08:20.191296+0000","last_undegraded":"2024-01-11T07:08:20.191296+0000","last_fullsized":"2024-01-11T07:08:20.191296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:38.405561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.199895+0000","last_change":"2024-01-11T07:08:20.199837+0000","last_active":"2024-01-11T07:08:20.199895+0000","last_peered":"2024-01-11T07:08:20.199895+0000","last_clean":"2024-01-11T07:08:20.199895+0000","last_became_active":"2024-01-11T07:08:20.199690+0000","last_became_peered":"2024-01-11T07:08:20.199690+0000","last_unstale":"2024-01-11T07:08:20.199895+0000","last_undegraded":"2024-01-11T07:08:20.199895+0000","last_fullsized":"2024-01-11T07:08:20.199895+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:18:23.886765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.194451+0000","last_change":"2024-01-11T07:08:19.191884+0000","last_active":"2024-01-11T07:08:20.194451+0000","last_peered":"2024-01-11T07:08:20.194451+0000","last_clean":"2024-01-11T07:08:20.194451+0000","last_became_active":"2024-01-11T07:08:19.191631+0000","last_became_peered":"2024-01-11T07:08:19.191631+0000","last_unstale":"2024-01-11T07:08:20.194451+0000","last_undegraded":"2024-01-11T07:08:20.194451+0000","last_fullsized":"2024-01-11T07:08:20.194451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:00.584001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203005+0000","last_change":"2024-01-11T07:08:20.202826+0000","last_active":"2024-01-11T07:08:20.203005+0000","last_peered":"2024-01-11T07:08:20.203005+0000","last_clean":"2024-01-11T07:08:20.203005+0000","last_became_active":"2024-01-11T07:08:20.202236+0000","last_became_peered":"2024-01-11T07:08:20.202236+0000","last_unstale":"2024-01-11T07:08:20.203005+0000","last_undegraded":"2024-01-11T07:08:20.203005+0000","last_fullsized":"2024-01-11T07:08:20.203005+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:24:11.496779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.192494+0000","last_change":"2024-01-11T07:08:19.193308+0000","last_active":"2024-01-11T07:08:20.192494+0000","last_peered":"2024-01-11T07:08:20.192494+0000","last_clean":"2024-01-11T07:08:20.192494+0000","last_became_active":"2024-01-11T07:08:19.192509+0000","last_became_peered":"2024-01-11T07:08:19.192509+0000","last_unstale":"2024-01-11T07:08:20.192494+0000","last_undegraded":"2024-01-11T07:08:20.192494+0000","last_fullsized":"2024-01-11T07:08:20.192494+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:03:34.168379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.207476+0000","last_change":"2024-01-11T07:08:20.207302+0000","last_active":"2024-01-11T07:08:20.207476+0000","last_peered":"2024-01-11T07:08:20.207476+0000","last_clean":"2024-01-11T07:08:20.207476+0000","last_became_active":"2024-01-11T07:08:20.206541+0000","last_became_peered":"2024-01-11T07:08:20.206541+0000","last_unstale":"2024-01-11T07:08:20.207476+0000","last_undegraded":"2024-01-11T07:08:20.207476+0000","last_fullsized":"2024-01-11T07:08:20.207476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:42:13.324724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.186796+0000","last_change":"2024-01-11T07:08:19.196814+0000","last_active":"2024-01-11T07:08:20.186796+0000","last_peered":"2024-01-11T07:08:20.186796+0000","last_clean":"2024-01-11T07:08:20.186796+0000","last_became_active":"2024-01-11T07:08:19.196578+0000","last_became_peered":"2024-01-11T07:08:19.196578+0000","last_unstale":"2024-01-11T07:08:20.186796+0000","last_undegraded":"2024-01-11T07:08:20.186796+0000","last_fullsized":"2024-01-11T07:08:20.186796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:20:29.426380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.202158+0000","last_change":"2024-01-11T07:08:20.202063+0000","last_active":"2024-01-11T07:08:20.202158+0000","last_peered":"2024-01-11T07:08:20.202158+0000","last_clean":"2024-01-11T07:08:20.202158+0000","last_became_active":"2024-01-11T07:08:20.201801+0000","last_became_peered":"2024-01-11T07:08:20.201801+0000","last_unstale":"2024-01-11T07:08:20.202158+0000","last_undegraded":"2024-01-11T07:08:20.202158+0000","last_fullsized":"2024-01-11T07:08:20.202158+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:22:43.122964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.197075+0000","last_change":"2024-01-11T07:08:19.190951+0000","last_active":"2024-01-11T07:08:20.197075+0000","last_peered":"2024-01-11T07:08:20.197075+0000","last_clean":"2024-01-11T07:08:20.197075+0000","last_became_active":"2024-01-11T07:08:19.190698+0000","last_became_peered":"2024-01-11T07:08:19.190698+0000","last_unstale":"2024-01-11T07:08:20.197075+0000","last_undegraded":"2024-01-11T07:08:20.197075+0000","last_fullsized":"2024-01-11T07:08:20.197075+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:20:29.426380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.196104+0000","last_change":"2024-01-11T07:08:20.195862+0000","last_active":"2024-01-11T07:08:20.196104+0000","last_peered":"2024-01-11T07:08:20.196104+0000","last_clean":"2024-01-11T07:08:20.196104+0000","last_became_active":"2024-01-11T07:08:20.195449+0000","last_became_peered":"2024-01-11T07:08:20.195449+0000","last_unstale":"2024-01-11T07:08:20.196104+0000","last_undegraded":"2024-01-11T07:08:20.196104+0000","last_fullsized":"2024-01-11T07:08:20.196104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:08:28.088534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.184456+0000","last_change":"2024-01-11T07:08:19.193254+0000","last_active":"2024-01-11T07:08:20.184456+0000","last_peered":"2024-01-11T07:08:20.184456+0000","last_clean":"2024-01-11T07:08:20.184456+0000","last_became_active":"2024-01-11T07:08:19.192022+0000","last_became_peered":"2024-01-11T07:08:19.192022+0000","last_unstale":"2024-01-11T07:08:20.184456+0000","last_undegraded":"2024-01-11T07:08:20.184456+0000","last_fullsized":"2024-01-11T07:08:20.184456+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:36.551051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.195230+0000","last_change":"2024-01-11T07:08:20.195146+0000","last_active":"2024-01-11T07:08:20.195230+0000","last_peered":"2024-01-11T07:08:20.195230+0000","last_clean":"2024-01-11T07:08:20.195230+0000","last_became_active":"2024-01-11T07:08:20.194423+0000","last_became_peered":"2024-01-11T07:08:20.194423+0000","last_unstale":"2024-01-11T07:08:20.195230+0000","last_undegraded":"2024-01-11T07:08:20.195230+0000","last_fullsized":"2024-01-11T07:08:20.195230+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:19.175458+0000","last_clean_scrub_stamp":"2024-01-11T07:08:19.175458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:39.407590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.203447+0000","last_change":"2024-01-11T07:08:19.195471+0000","last_active":"2024-01-11T07:08:20.203447+0000","last_peered":"2024-01-11T07:08:20.203447+0000","last_clean":"2024-01-11T07:08:20.203447+0000","last_became_active":"2024-01-11T07:08:19.195204+0000","last_became_peered":"2024-01-11T07:08:19.195204+0000","last_unstale":"2024-01-11T07:08:20.203447+0000","last_undegraded":"2024-01-11T07:08:20.203447+0000","last_fullsized":"2024-01-11T07:08:20.203447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:08:18.173429+0000","last_clean_scrub_stamp":"2024-01-11T07:08:18.173429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:38.405561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-11T07:08:20.191403+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:08:20.191403+0000","last_peered":"2024-01-11T07:08:20.191403+0000","last_clean":"2024-01-11T07:08:20.191403+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:08:20.191403+0000","last_undegraded":"2024-01-11T07:08:20.191403+0000","last_fullsized":"2024-01-11T07:08:20.191403+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":794,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194517,"statfs":{"total":200000000,"available":199186127,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.497}]},{"osd":3,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.439}]},{"osd":5,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.629},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.63500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195093,"statfs":{"total":200000000,"available":199775941,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.629},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.749},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.65900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195094,"statfs":{"total":200000000,"available":199776515,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.38},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":794,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194517,"statfs":{"total":200000000,"available":199186183,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.312},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":7.7960000000000003,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":8.0730000000000004,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":0.38}]},{"osd":4,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":8.0399999999999991,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.432}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.819,"5min":2.819,"15min":2.819},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.503}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":7.9539999999999997,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":8.0030000000000001,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.52400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775027,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:07:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.85299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195093,"statfs":{"total":200000000,"available":199775975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":11.481,"5min":11.481,"15min":11.481},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.379}]},{"osd":1,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":17.516999999999999,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":17.553000000000001,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":17.474,"5min":17.474,"15min":17.474},"last":0.74099999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.407,"15min":1.407},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":17.393000000000001,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":11.436999999999999,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:07:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":17.434000000000001,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.65700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775049,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":8.484,"5min":8.484,"15min":8.484},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.2599999999999998,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.504}]},{"osd":1,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":8.5340000000000007,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":9.2279999999999998,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":9.5640000000000001,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.8639999999999999,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":8.9779999999999998,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.7309999999999999,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.441}]},{"osd":5,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.388999999999999,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":8.6509999999999998,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1449999999999996,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:07:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":8.6050000000000004,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":9.4640000000000004,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.56499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195094,"statfs":{"total":200000000,"available":199776515,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.5}]},{"osd":3,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.439},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:07:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47799999999999998}]}]}],"pool_statfs":[]}} 2024-01-11T07:08:24.797 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:08:24.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:08:24.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.974+0000 7f9c513dd700 1 Processor -- start 2024-01-11T07:08:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.975+0000 7f9c513dd700 1 -- start start 2024-01-11T07:08:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.975+0000 7f9c513dd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c4c15d920 0x7f9c4c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.975+0000 7f9c513dd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c15e2d0 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.975+0000 7f9c513dd700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c06ac30 con 0x7f9c4c0aa8c0 2024-01-11T07:08:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.975+0000 7f9c513dd700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c0cb370 con 0x7f9c4c15d920 2024-01-11T07:08:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c4affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9c4c0ac3a0 0x7f9c4c0cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40968/0 (socket says 172.21.15.7:40968) 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c4affd700 1 -- 172.21.15.7:0/2258883765 learned_addr learned my addr 172.21.15.7:0/2258883765 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c4a7fc700 1 --2- 172.21.15.7:0/2258883765 >> v2:172.21.15.113:3300/0 conn(0x7f9c4c15d920 0x7f9c4c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c49ffb700 1 -- 172.21.15.7:0/2258883765 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371737378 0 0) 0x7f9c4c15e2d0 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c49ffb700 1 -- 172.21.15.7:0/2258883765 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c38003570 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/2258883765 >> v1:172.21.15.113:6789/0 conn(0x7f9c4c0aa8c0 legacy=0x7f9c4c0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c49ffb700 1 -- 172.21.15.7:0/2258883765 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201292242 0 0) 0x7f9c4c06ac30 con 0x7f9c4c0aa8c0 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/2258883765 >> v1:172.21.15.7:6789/0 conn(0x7f9c4c0ac3a0 legacy=0x7f9c4c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/2258883765 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0ac770 con 0x7f9c4c15d920 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.976+0000 7f9c4a7fc700 1 --2- 172.21.15.7:0/2258883765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 0x7f9c4c15dd90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9c340098e0 tx=0x7f9c340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce56ccfe10009f90 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c49ffb700 1 -- 172.21.15.7:0/2258883765 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400b640 con 0x7f9c4c15d920 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c49ffb700 1 -- 172.21.15.7:0/2258883765 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3400b7a0 con 0x7f9c4c15d920 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c49ffb700 1 -- 172.21.15.7:0/2258883765 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400bb20 con 0x7f9c4c15d920 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c513dd700 1 -- 172.21.15.7:0/2258883765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 msgr2=0x7f9c4c15dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c513dd700 1 --2- 172.21.15.7:0/2258883765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 0x7f9c4c15dd90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9c340098e0 tx=0x7f9c340091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c513dd700 1 -- 172.21.15.7:0/2258883765 shutdown_connections 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c513dd700 1 --2- 172.21.15.7:0/2258883765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 0x7f9c4c15dd90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c513dd700 1 -- 172.21.15.7:0/2258883765 >> 172.21.15.7:0/2258883765 conn(0x7f9c4c0cd090 msgr2=0x7f9c4c0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c513dd700 1 -- 172.21.15.7:0/2258883765 shutdown_connections 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.977+0000 7f9c513dd700 1 -- 172.21.15.7:0/2258883765 wait complete. 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.979+0000 7f9c513dd700 1 Processor -- start 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.979+0000 7f9c513dd700 1 -- start start 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.979+0000 7f9c513dd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c4c0aa8c0 0x7f9c4c0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.979+0000 7f9c513dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0ac3a0 0x7f9c4c0adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.979+0000 7f9c513dd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 0x7f9c4c06e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.979+0000 7f9c513dd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0b1710 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.979+0000 7f9c513dd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0b2570 con 0x7f9c4c15d920 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.979+0000 7f9c513dd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0b1960 con 0x7f9c4c0aa8c0 2024-01-11T07:08:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0ac3a0 0x7f9c4c0adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0ac3a0 0x7f9c4c0adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47216/0 (socket says 172.21.15.7:47216) 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/1117384679 learned_addr learned my addr 172.21.15.7:0/1117384679 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4affd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c4c0aa8c0 0x7f9c4c0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4b7fe700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 0x7f9c4c06e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c4c0aa8c0 msgr2=0x7f9c4c0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c4c0aa8c0 0x7f9c4c0ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 msgr2=0x7f9c4c06e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 0x7f9c4c06e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/1117384679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c34009420 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c4a7fc700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0ac3a0 0x7f9c4c0adfd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009f80 tx=0x7f9c34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c42ffd700 1 -- 172.21.15.7:0/1117384679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400bb20 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c42ffd700 1 -- 172.21.15.7:0/1117384679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c34000c90 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c0b1c40 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c42ffd700 1 -- 172.21.15.7:0/1117384679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3401a680 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.980+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c0b2200 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.981+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c0b2700 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.982+0000 7f9c42ffd700 1 -- 172.21.15.7:0/1117384679 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9c3401aa30 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.982+0000 7f9c42ffd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c2c066f30 0x7f9c2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.982+0000 7f9c42ffd700 1 -- 172.21.15.7:0/1117384679 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c34086120 con 0x7f9c4c0ac3a0 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.982+0000 7f9c4affd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c2c066f30 0x7f9c2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.983+0000 7f9c4affd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c2c066f30 0x7f9c2c0693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c007fd0 tx=0x7f9c3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:24.987+0000 7f9c42ffd700 1 -- 172.21.15.7:0/1117384679 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c34053390 con 0x7f9c4c0ac3a0 2024-01-11T07:08:25.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.278+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9c4c04c3c0 con 0x7f9c4c0ac3a0 2024-01-11T07:08:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.282+0000 7f9c42ffd700 1 -- 172.21.15.7:0/1117384679 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9c340569b0 con 0x7f9c4c0ac3a0 2024-01-11T07:08:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c2c066f30 msgr2=0x7f9c2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c2c066f30 0x7f9c2c0693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c007fd0 tx=0x7f9c3c009040 comp rx=0 tx=0).stop 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0ac3a0 msgr2=0x7f9c4c0adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0ac3a0 0x7f9c4c0adfd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009f80 tx=0x7f9c34017000 comp rx=0 tx=0).stop 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 shutdown_connections 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c2c066f30 0x7f9c2c0693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c4c15d920 0x7f9c4c06e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c4c0aa8c0 0x7f9c4c0ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 --2- 172.21.15.7:0/1117384679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0ac3a0 0x7f9c4c0adfd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 >> 172.21.15.7:0/1117384679 conn(0x7f9c4c0cd090 msgr2=0x7f9c4c15c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 shutdown_connections 2024-01-11T07:08:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.285+0000 7f9c513dd700 1 -- 172.21.15.7:0/1117384679 wait complete. 2024-01-11T07:08:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:08:25.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.430+0000 7fd2c7db5700 1 Processor -- start 2024-01-11T07:08:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.430+0000 7fd2c7db5700 1 -- start start 2024-01-11T07:08:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c7db5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c00b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c7db5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2c00b0430 0x7fd2c00b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c7db5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c0161b90 con 0x7fd2c01617c0 2024-01-11T07:08:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c7db5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c00b1d60 con 0x7fd2c00b0430 2024-01-11T07:08:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c7db5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00b0df0 con 0x7fd2c00cf530 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c6352700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2c00b0430 0x7fd2c00b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c5b51700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd2c01617c0 0x7fd2c00b4410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40980/0 (socket says 172.21.15.7:40980) 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c5b51700 1 -- 172.21.15.7:0/3402851523 learned_addr learned my addr 172.21.15.7:0/3402851523 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.431+0000 7fd2c5350700 1 --2- 172.21.15.7:0/3402851523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c00b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c6352700 1 -- 172.21.15.7:0/3402851523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 msgr2=0x7fd2c00b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c6352700 1 --2- 172.21.15.7:0/3402851523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c00b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c6352700 1 -- 172.21.15.7:0/3402851523 >> v1:172.21.15.7:6789/0 conn(0x7fd2c01617c0 legacy=0x7fd2c00b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c6352700 1 -- 172.21.15.7:0/3402851523 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c00b36c0 con 0x7fd2c00b0430 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c5350700 1 --2- 172.21.15.7:0/3402851523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c00b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c6352700 1 --2- 172.21.15.7:0/3402851523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c00b0430 0x7fd2c00b08b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009ec0 tx=0x7fd2b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520413fccb6998dc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c4b4f700 1 -- 172.21.15.7:0/3402851523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800e070 con 0x7fd2c00b0430 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c4b4f700 1 -- 172.21.15.7:0/3402851523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b800bbb0 con 0x7fd2c00b0430 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c4b4f700 1 -- 172.21.15.7:0/3402851523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8012430 con 0x7fd2c00b0430 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c7db5700 1 -- 172.21.15.7:0/3402851523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c00b0430 msgr2=0x7fd2c00b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.432+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/3402851523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c00b0430 0x7fd2c00b08b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009ec0 tx=0x7fd2b8009780 comp rx=0 tx=0).stop 2024-01-11T07:08:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.433+0000 7fd2c7db5700 1 -- 172.21.15.7:0/3402851523 shutdown_connections 2024-01-11T07:08:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.433+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/3402851523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c00b0430 0x7fd2c00b08b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.433+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/3402851523 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c00b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.433+0000 7fd2c7db5700 1 -- 172.21.15.7:0/3402851523 >> 172.21.15.7:0/3402851523 conn(0x7fd2c0071100 msgr2=0x7fd2c006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.433+0000 7fd2c7db5700 1 -- 172.21.15.7:0/3402851523 shutdown_connections 2024-01-11T07:08:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.433+0000 7fd2c7db5700 1 -- 172.21.15.7:0/3402851523 wait complete. 2024-01-11T07:08:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.434+0000 7fd2c7db5700 1 Processor -- start 2024-01-11T07:08:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.434+0000 7fd2c7db5700 1 -- start start 2024-01-11T07:08:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.435+0000 7fd2c7db5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c00b0430 0x7fd2c00a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.435+0000 7fd2c7db5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c015ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.435+0000 7fd2c7db5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c01617c0 0x7fd2c0163ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.435+0000 7fd2c7db5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00aa7d0 con 0x7fd2c01617c0 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.435+0000 7fd2c7db5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00aaab0 con 0x7fd2c00b0430 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.435+0000 7fd2c7db5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00aad90 con 0x7fd2c00cf530 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c01617c0 0x7fd2c0163ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c01617c0 0x7fd2c0163ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47238/0 (socket says 172.21.15.7:47238) 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 -- 172.21.15.7:0/180806719 learned_addr learned my addr 172.21.15.7:0/180806719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c5350700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c015ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c5b51700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c00b0430 0x7fd2c00a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 -- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 msgr2=0x7fd2c015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:25.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c015ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 -- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c00b0430 msgr2=0x7fd2c00a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c00b0430 0x7fd2c00a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 -- 172.21.15.7:0/180806719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b8009a20 con 0x7fd2c01617c0 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c6352700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c01617c0 0x7fd2c0163ee0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2c006a1d0 tx=0x7fd2b8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2b5ffb700 1 -- 172.21.15.7:0/180806719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8012e90 con 0x7fd2c01617c0 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2b5ffb700 1 -- 172.21.15.7:0/180806719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b8027470 con 0x7fd2c01617c0 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2b5ffb700 1 -- 172.21.15.7:0/180806719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8027810 con 0x7fd2c01617c0 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.436+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c00ab070 con 0x7fd2c01617c0 2024-01-11T07:08:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.437+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c00ac730 con 0x7fd2c01617c0 2024-01-11T07:08:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.438+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c004c3c0 con 0x7fd2c01617c0 2024-01-11T07:08:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.438+0000 7fd2b5ffb700 1 -- 172.21.15.7:0/180806719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd2b8024070 con 0x7fd2c01617c0 2024-01-11T07:08:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.439+0000 7fd2b5ffb700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2a4066f00 0x7fd2a40693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.439+0000 7fd2b5ffb700 1 -- 172.21.15.7:0/180806719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2b8086cd0 con 0x7fd2c01617c0 2024-01-11T07:08:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.439+0000 7fd2c5b51700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2a4066f00 0x7fd2a40693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.440+0000 7fd2c5b51700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2a4066f00 0x7fd2a40693b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc001950 tx=0x7fd2bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.443+0000 7fd2b5ffb700 1 -- 172.21.15.7:0/180806719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2b8053ea0 con 0x7fd2c01617c0 2024-01-11T07:08:25.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.693+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c00ae100 con 0x7fd2c01617c0 2024-01-11T07:08:25.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.693+0000 7fd2b5ffb700 1 -- 172.21.15.7:0/180806719 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fd2b802c090 con 0x7fd2c01617c0 2024-01-11T07:08:25.694 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:25.694 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.695+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2a4066f00 msgr2=0x7fd2a40693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.695+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2a4066f00 0x7fd2a40693b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc001950 tx=0x7fd2bc009040 comp rx=0 tx=0).stop 2024-01-11T07:08:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.695+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c01617c0 msgr2=0x7fd2c0163ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.695+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c01617c0 0x7fd2c0163ee0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2c006a1d0 tx=0x7fd2b8019000 comp rx=0 tx=0).stop 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.696+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 shutdown_connections 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.696+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2a4066f00 0x7fd2a40693b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.696+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2c01617c0 0x7fd2c0163ee0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.696+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c00b0430 0x7fd2c00a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.696+0000 7fd2c7db5700 1 --2- 172.21.15.7:0/180806719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2c00cf530 0x7fd2c015ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.696+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 >> 172.21.15.7:0/180806719 conn(0x7fd2c0071100 msgr2=0x7fd2c00ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.696+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 shutdown_connections 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:25.696+0000 7fd2c7db5700 1 -- 172.21.15.7:0/180806719 wait complete. 2024-01-11T07:08:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:25.748 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-11T07:08:25.748+0000 7f46b3937700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:08:26.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:08:19.620845+0000', 'modified': '2024-01-11T07:08:23.209392+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5310}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5310': {'gid': 5310, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/2351618262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 2351618262}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 2351618262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:08:26.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:08:26.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-11T07:08:26.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.255+0000 7f70c2511700 1 Processor -- start 2024-01-11T07:08:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.255+0000 7f70c2511700 1 -- start start 2024-01-11T07:08:26.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70c2511700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc0cafb0 0x7f70bc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70c2511700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70bc0b0b40 0x7f70bc161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70c2511700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc159300 con 0x7f70bc158280 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70c2511700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70bc0cf260 con 0x7f70bc0b0b40 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70c2511700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0cf540 con 0x7f70bc0cafb0 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70bb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f70bc158280 0x7f70bc0b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40988/0 (socket says 172.21.15.7:40988) 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70bb7fe700 1 -- 172.21.15.7:0/4131255483 learned_addr learned my addr 172.21.15.7:0/4131255483 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70bbfff700 1 --2- 172.21.15.7:0/4131255483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc0cafb0 0x7f70bc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70c0aae700 1 --2- 172.21.15.7:0/4131255483 >> v2:172.21.15.113:3300/0 conn(0x7f70bc0b0b40 0x7f70bc161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70baffd700 1 -- 172.21.15.7:0/4131255483 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029280658 0 0) 0x7f70bc159300 con 0x7f70bc158280 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.256+0000 7f70baffd700 1 -- 172.21.15.7:0/4131255483 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a8003570 con 0x7f70bc158280 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70bbfff700 1 -- 172.21.15.7:0/4131255483 >> v2:172.21.15.113:3300/0 conn(0x7f70bc0b0b40 msgr2=0x7f70bc161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70bbfff700 1 --2- 172.21.15.7:0/4131255483 >> v2:172.21.15.113:3300/0 conn(0x7f70bc0b0b40 0x7f70bc161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70baffd700 1 -- 172.21.15.7:0/4131255483 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2812829683 0 0) 0x7f70a8003570 con 0x7f70bc158280 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70bbfff700 1 -- 172.21.15.7:0/4131255483 >> v1:172.21.15.7:6789/0 conn(0x7f70bc158280 legacy=0x7f70bc0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70bbfff700 1 -- 172.21.15.7:0/4131255483 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc0cf820 con 0x7f70bc0cafb0 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70c0aae700 1 --2- 172.21.15.7:0/4131255483 >> v2:172.21.15.113:3300/0 conn(0x7f70bc0b0b40 0x7f70bc161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70bbfff700 1 --2- 172.21.15.7:0/4131255483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc0cafb0 0x7f70bc0ce8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0098e0 tx=0x7f70ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682ed486144b724 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70baffd700 1 -- 172.21.15.7:0/4131255483 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4001470 con 0x7f70bc0cafb0 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70baffd700 1 -- 172.21.15.7:0/4131255483 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac011420 con 0x7f70bc0cafb0 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.257+0000 7f70baffd700 1 -- 172.21.15.7:0/4131255483 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0117c0 con 0x7f70bc0cafb0 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.258+0000 7f70c2511700 1 -- 172.21.15.7:0/4131255483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc0cafb0 msgr2=0x7f70bc0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.258+0000 7f70c2511700 1 --2- 172.21.15.7:0/4131255483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc0cafb0 0x7f70bc0ce8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0098e0 tx=0x7f70ac0091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.258+0000 7f70c2511700 1 -- 172.21.15.7:0/4131255483 shutdown_connections 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.258+0000 7f70c2511700 1 --2- 172.21.15.7:0/4131255483 >> v2:172.21.15.113:3300/0 conn(0x7f70bc0b0b40 0x7f70bc161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.258+0000 7f70c2511700 1 --2- 172.21.15.7:0/4131255483 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc0cafb0 0x7f70bc0ce8f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.258+0000 7f70c2511700 1 -- 172.21.15.7:0/4131255483 >> 172.21.15.7:0/4131255483 conn(0x7f70bc071100 msgr2=0x7f70bc06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.258+0000 7f70c2511700 1 -- 172.21.15.7:0/4131255483 shutdown_connections 2024-01-11T07:08:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.258+0000 7f70c2511700 1 -- 172.21.15.7:0/4131255483 wait complete. 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.259+0000 7f70c2511700 1 Processor -- start 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.260+0000 7f70c2511700 1 -- start start 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70c2511700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70bc0b0b40 0x7f70bc0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70c2511700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70bc0cafb0 0x7f70bc155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70c2511700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc158280 0x7f70bc0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70c2511700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc167e90 con 0x7f70bc0b0b40 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70c2511700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0b3190 con 0x7f70bc0cafb0 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70c2511700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0b2660 con 0x7f70bc158280 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70bc0b0b40 0x7f70bc0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70bc0b0b40 0x7f70bc0b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47250/0 (socket says 172.21.15.7:47250) 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 -- 172.21.15.7:0/4226532972 learned_addr learned my addr 172.21.15.7:0/4226532972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:26.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70c0aae700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc158280 0x7f70bc0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bb7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70bc0cafb0 0x7f70bc155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 -- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc158280 msgr2=0x7f70bc0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc158280 0x7f70bc0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 -- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70bc0cafb0 msgr2=0x7f70bc155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70bc0cafb0 0x7f70bc155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 -- 172.21.15.7:0/4226532972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4009a20 con 0x7f70bc0b0b40 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70bc0cafb0 0x7f70bc155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.261+0000 7f70bbfff700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70bc0b0b40 0x7f70bc0b2c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0cf680 tx=0x7f70ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.262+0000 7f70b3fff700 1 -- 172.21.15.7:0/4226532972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac01cba0 con 0x7f70bc0b0b40 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.262+0000 7f70b3fff700 1 -- 172.21.15.7:0/4226532972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac01cd00 con 0x7f70bc0b0b40 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.262+0000 7f70c2511700 1 -- 172.21.15.7:0/4226532972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ac009420 con 0x7f70bc0b0b40 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.262+0000 7f70b3fff700 1 -- 172.21.15.7:0/4226532972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac01a6a0 con 0x7f70bc0b0b40 2024-01-11T07:08:26.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.262+0000 7f70c2511700 1 -- 172.21.15.7:0/4226532972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc0b20a0 con 0x7f70bc0b0b40 2024-01-11T07:08:26.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.265+0000 7f70cb811b80 1 Processor -- start 2024-01-11T07:08:26.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.265+0000 7f70cb811b80 1 -- start start 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.266+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c398370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 0x56242c393c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 0x56242c3b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70cb811b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242c2c63e0 con 0x56242c397fa0 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70cb811b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242c2c3be0 con 0x56242c3988b0 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70cb811b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56242c2bf9d0 con 0x56242c394170 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c398370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c398370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47260/0 (socket says 172.21.15.7:47260) 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 -- 172.21.15.7:0/1960425300 learned_addr learned my addr 172.21.15.7:0/1960425300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/1960425300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 0x56242c3b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70b3fff700 1 -- 172.21.15.7:0/4226532972 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f70ac01aad0 con 0x7f70bc0b0b40 2024-01-11T07:08:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 -- 172.21.15.7:0/1960425300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 msgr2=0x56242c3b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 --2- 172.21.15.7:0/1960425300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 0x56242c3b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 -- 172.21.15.7:0/1960425300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 msgr2=0x56242c393c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 --2- 172.21.15.7:0/1960425300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 0x56242c393c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 -- 172.21.15.7:0/1960425300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56242c153360 con 0x56242c397fa0 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70c0aae700 1 --2- 172.21.15.7:0/1960425300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c398370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f70b400f670 tx=0x7f70b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0e6fe02b700b07 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.267+0000 7f70b3fff700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f709c066f30 0x7f709c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70b3fff700 1 -- 172.21.15.7:0/4226532972 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70ac085d40 con 0x7f70bc0b0b40 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70b17fa700 1 -- 172.21.15.7:0/1960425300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b400e040 con 0x56242c397fa0 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70bbfff700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f709c066f30 0x7f709c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70b17fa700 1 -- 172.21.15.7:0/1960425300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b400f040 con 0x56242c397fa0 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70b17fa700 1 -- 172.21.15.7:0/1960425300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4015870 con 0x56242c397fa0 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70bbfff700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f709c066f30 0x7f709c0693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0842f0 tx=0x7f70ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70cb811b80 1 -- 172.21.15.7:0/1960425300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 msgr2=0x56242c398370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70cb811b80 1 --2- 172.21.15.7:0/1960425300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c398370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f70b400f670 tx=0x7f70b400b040 comp rx=0 tx=0).stop 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70cb811b80 1 -- 172.21.15.7:0/1960425300 shutdown_connections 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70cb811b80 1 --2- 172.21.15.7:0/1960425300 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 0x56242c3b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70cb811b80 1 --2- 172.21.15.7:0/1960425300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c398370 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70cb811b80 1 --2- 172.21.15.7:0/1960425300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 0x56242c393c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70cb811b80 1 -- 172.21.15.7:0/1960425300 >> 172.21.15.7:0/1960425300 conn(0x56242c36e440 msgr2=0x56242c3b9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.268+0000 7f70cb811b80 1 -- 172.21.15.7:0/1960425300 shutdown_connections 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.269+0000 7f70cb811b80 1 -- 172.21.15.7:0/1960425300 wait complete. 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.269+0000 7f70cb811b80 1 -- start start 2024-01-11T07:08:26.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.269+0000 7f70cb811b80 1 Processor -- start 2024-01-11T07:08:26.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c394170 0x56242c3c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 0x56242c386ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70cb811b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242c26c1b0 con 0x56242c397fa0 2024-01-11T07:08:26.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70cb811b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242c26be50 con 0x56242c394170 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70cb811b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56242c1b8e40 con 0x56242c3988b0 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70cb811b80 10 client.0 _ll_register_callbacks cb 0x56242c1bb7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47266/0 (socket says 172.21.15.7:47266) 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70bb7fe700 1 -- 172.21.15.7:0/3378853201 learned_addr learned my addr 172.21.15.7:0/3378853201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70c0aae700 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c394170 0x56242c3c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.270+0000 7f70c0aae700 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 0x56242c386ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70bb7fe700 1 -- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 msgr2=0x56242c386ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 0x56242c386ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70bb7fe700 1 -- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c394170 msgr2=0x56242c3c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c394170 0x56242c3c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70bb7fe700 1 -- 172.21.15.7:0/3378853201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b400f340 con 0x56242c397fa0 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70c0aae700 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c394170 0x56242c3c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:26.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3c97d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f70a4007470 tx=0x7f70a40011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d27ebbcdb55015 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f709a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f709a7fc700 1 -- 172.21.15.7:0/3378853201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4001c90 con 0x56242c397fa0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f709a7fc700 1 -- 172.21.15.7:0/3378853201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70a4012a30 con 0x56242c397fa0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 1 -- 172.21.15.7:0/3378853201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56242c340eb0 con 0x56242c397fa0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f709a7fc700 1 -- 172.21.15.7:0/3378853201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70a4012e00 con 0x56242c397fa0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 10 client.5448 fetch_fsmap learned FSMap version 25 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 1 -- 172.21.15.7:0/3378853201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x56242c0ffb10 con 0x56242c397fa0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f709a7fc700 1 -- 172.21.15.7:0/3378853201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56242c340eb0 con 0x56242c397fa0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f709a7fc700 1 -- 172.21.15.7:0/3378853201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f70a401e440 con 0x56242c397fa0 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 10 client.5448 fetch_fsmap finished waiting for FSMap version 25 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 10 client.5448 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 20 client.5448 populate_metadata read hostname 'smithi007' 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] conn(0x56242c3c5910 0x56242c3afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 4 client.5448 mds_command: new command op to 5310 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:08:26.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.271+0000 7f70cb811b80 1 -- 172.21.15.7:0/3378853201 --> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56242c296880 con 0x56242c3c5910 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.272+0000 7f70c0aae700 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] conn(0x56242c3c5910 0x56242c3afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.272+0000 7f70c0aae700 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] conn(0x56242c3c5910 0x56242c3afc20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9591083ff3278776 server_cookie=8537cdb39066a494 in_seq=0 out_seq=0 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.272+0000 7f709a7fc700 10 client.5448 ms_handle_connect on v2:172.21.15.7:6833/2351618262 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.272+0000 7f709a7fc700 1 -- 172.21.15.7:0/3378853201 <== mds.0 v2:172.21.15.7:6833/2351618262 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56242c296880 con 0x56242c3c5910 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f709a7fc700 10 client.5448 handle_command_reply: tid=0 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 client.5448 shutdown 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 -- 172.21.15.7:0/3378853201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 msgr2=0x56242c3c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3c97d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f70a4007470 tx=0x7f70a40011a0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 -- 172.21.15.7:0/3378853201 shutdown_connections 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] conn(0x56242c3c5910 0x56242c3afc20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 0x56242c386ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c394170 0x56242c3c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 --2- 172.21.15.7:0/3378853201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3c97d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f709a7fc700 0 client.5448 ms_handle_reset on v2:172.21.15.7:6833/2351618262 2024-01-11T07:08:26.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 -- 172.21.15.7:0/3378853201 >> 172.21.15.7:0/3378853201 conn(0x56242c36e440 msgr2=0x56242c3938d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:26.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 -- 172.21.15.7:0/3378853201 shutdown_connections 2024-01-11T07:08:26.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 1 -- 172.21.15.7:0/3378853201 wait complete. 2024-01-11T07:08:26.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.273+0000 7f70cb811b80 20 client.5448 trim_cache size 0 max 16384 2024-01-11T07:08:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.293+0000 7f70cb811b80 1 Processor -- start 2024-01-11T07:08:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.293+0000 7f70cb811b80 1 -- start start 2024-01-11T07:08:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 0x56242c3b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 0x56242c3b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70cb811b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242c370450 con 0x56242c397fa0 2024-01-11T07:08:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70cb811b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242c395f30 con 0x56242c3988b0 2024-01-11T07:08:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70cb811b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56242c39f390 con 0x56242c394170 2024-01-11T07:08:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47274/0 (socket says 172.21.15.7:47274) 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 -- 172.21.15.7:0/2844360143 learned_addr learned my addr 172.21.15.7:0/2844360143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70c0aae700 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 0x56242c3b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70c0aae700 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 0x56242c3b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 -- 172.21.15.7:0/2844360143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 msgr2=0x56242c3b9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 0x56242c3b9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 -- 172.21.15.7:0/2844360143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 msgr2=0x56242c3b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 0x56242c3b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 -- 172.21.15.7:0/2844360143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56242c28e290 con 0x56242c397fa0 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.294+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3afc20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f70a40011b0 tx=0x7f70a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0e3c70250edeb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.295+0000 7f70b1ffb700 1 -- 172.21.15.7:0/2844360143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4012a30 con 0x56242c397fa0 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.295+0000 7f70b1ffb700 1 -- 172.21.15.7:0/2844360143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4001c90 con 0x56242c397fa0 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.295+0000 7f70b1ffb700 1 -- 172.21.15.7:0/2844360143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a401e9a0 con 0x56242c397fa0 2024-01-11T07:08:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.295+0000 7f70cb811b80 1 -- 172.21.15.7:0/2844360143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 msgr2=0x56242c3afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.295+0000 7f70cb811b80 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3afc20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f70a40011b0 tx=0x7f70a4019000 comp rx=0 tx=0).stop 2024-01-11T07:08:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 -- 172.21.15.7:0/2844360143 shutdown_connections 2024-01-11T07:08:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c3988b0 0x56242c3b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c394170 0x56242c3b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 --2- 172.21.15.7:0/2844360143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c397fa0 0x56242c3afc20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 -- 172.21.15.7:0/2844360143 >> 172.21.15.7:0/2844360143 conn(0x56242c36e440 msgr2=0x56242c372c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 -- 172.21.15.7:0/2844360143 shutdown_connections 2024-01-11T07:08:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 -- 172.21.15.7:0/2844360143 wait complete. 2024-01-11T07:08:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 -- start start 2024-01-11T07:08:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.296+0000 7f70cb811b80 1 Processor -- start 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c394170 0x56242c396fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c397fa0 0x56242c41ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70cb811b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 0x56242c366530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70cb811b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242c385f00 con 0x56242c394170 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70cb811b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242c3975b0 con 0x56242c397fa0 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70cb811b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56242c3d77c0 con 0x56242c3988b0 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70cb811b80 10 client.0 _ll_register_callbacks cb 0x7f70a4006c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70c0aae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c394170 0x56242c396fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70c0aae700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c394170 0x56242c396fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47276/0 (socket says 172.21.15.7:47276) 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70c0aae700 1 -- 172.21.15.7:0/4200791453 learned_addr learned my addr 172.21.15.7:0/4200791453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.297+0000 7f70c0aae700 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 0x56242c366530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c397fa0 0x56242c41ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70c0aae700 1 -- 172.21.15.7:0/4200791453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 msgr2=0x56242c366530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70c0aae700 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 0x56242c366530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70c0aae700 1 -- 172.21.15.7:0/4200791453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c397fa0 msgr2=0x56242c41ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70c0aae700 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c397fa0 0x56242c41ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70c0aae700 1 -- 172.21.15.7:0/4200791453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56242c0ffb10 con 0x56242c394170 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70c0aae700 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c394170 0x56242c396fb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f70b40011d0 tx=0x7f70b4020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25537189fd39197a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f709a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f709a7fc700 1 -- 172.21.15.7:0/4200791453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4032070 con 0x56242c394170 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f709a7fc700 1 -- 172.21.15.7:0/4200791453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70b402e700 con 0x56242c394170 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 1 -- 172.21.15.7:0/4200791453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56242c364490 con 0x56242c394170 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f709a7fc700 1 -- 172.21.15.7:0/4200791453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70b400e070 con 0x56242c394170 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 10 client.5454 fetch_fsmap learned FSMap version 25 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 1 -- 172.21.15.7:0/4200791453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x56242c157c80 con 0x56242c394170 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f709a7fc700 1 -- 172.21.15.7:0/4200791453 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b403d020 con 0x56242c394170 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f709a7fc700 1 -- 172.21.15.7:0/4200791453 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f70b402e020 con 0x56242c394170 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 10 client.5454 fetch_fsmap finished waiting for FSMap version 25 2024-01-11T07:08:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 10 client.5454 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 20 client.5454 populate_metadata read hostname 'smithi007' 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] conn(0x56242c3afc20 0x56242c3afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 4 client.5454 mds_command: new command op to 5310 tid=0[{"prefix": "status"}] 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.298+0000 7f70cb811b80 1 -- 172.21.15.7:0/4200791453 --> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56242c262910 con 0x56242c3afc20 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] conn(0x56242c3afc20 0x56242c3afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f709a7fc700 10 client.5454 ms_handle_connect on v2:172.21.15.7:6833/2351618262 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70bb7fe700 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] conn(0x56242c3afc20 0x56242c3afff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7ee615aa6d3d53 server_cookie=3cfe58144898b54a in_seq=0 out_seq=0 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f709a7fc700 1 -- 172.21.15.7:0/4200791453 <== mds.0 v2:172.21.15.7:6833/2351618262 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56242c262910 con 0x56242c3afc20 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f709a7fc700 10 client.5454 handle_command_reply: tid=0 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 client.5454 shutdown 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 -- 172.21.15.7:0/4200791453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c394170 msgr2=0x56242c396fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c394170 0x56242c396fb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f70b40011d0 tx=0x7f70b4020040 comp rx=0 tx=0).stop 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 -- 172.21.15.7:0/4200791453 shutdown_connections 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.7:6833/2351618262,v1:172.21.15.7:6835/2351618262] conn(0x56242c3afc20 0x56242c3afff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56242c3988b0 0x56242c366530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56242c394170 0x56242c396fb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f709a7fc700 0 client.5454 ms_handle_reset on v2:172.21.15.7:6833/2351618262 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 --2- 172.21.15.7:0/4200791453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56242c397fa0 0x56242c41ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 -- 172.21.15.7:0/4200791453 >> 172.21.15.7:0/4200791453 conn(0x56242c36e440 msgr2=0x56242c3ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 -- 172.21.15.7:0/4200791453 shutdown_connections 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.299+0000 7f70cb811b80 1 -- 172.21.15.7:0/4200791453 wait complete. 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70cb811b80 20 client.5454 trim_cache size 0 max 16384 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5310, 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:08:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:08:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:08:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1177461769999999, 2024-01-11T07:08:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 25, 2024-01-11T07:08:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 47, 2024-01-11T07:08:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 47, 2024-01-11T07:08:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.373213402999999 2024-01-11T07:08:26.307 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 -- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f709c066f30 msgr2=0x7f709c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f709c066f30 0x7f709c0693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0842f0 tx=0x7f70ac008040 comp rx=0 tx=0).stop 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 -- 172.21.15.7:0/4226532972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70bc0b0b40 msgr2=0x7f70bc0b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70bc0b0b40 0x7f70bc0b2c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0cf680 tx=0x7f70ac016000 comp rx=0 tx=0).stop 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 -- 172.21.15.7:0/4226532972 shutdown_connections 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f709c066f30 0x7f709c0693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70bc158280 0x7f70bc0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70bc0b0b40 0x7f70bc0b2c50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 --2- 172.21.15.7:0/4226532972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70bc0cafb0 0x7f70bc155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 -- 172.21.15.7:0/4226532972 >> 172.21.15.7:0/4226532972 conn(0x7f70bc071100 msgr2=0x7f70bc0ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 -- 172.21.15.7:0/4226532972 shutdown_connections 2024-01-11T07:08:26.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.300+0000 7f70c2511700 1 -- 172.21.15.7:0/4226532972 wait complete. 2024-01-11T07:08:26.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:08:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.445+0000 7f7321e4a700 1 Processor -- start 2024-01-11T07:08:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.445+0000 7f7321e4a700 1 -- start start 2024-01-11T07:08:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f7321e4a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f731c159c90 0x7f731c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f7321e4a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f731c069320 0x7f731c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f7321e4a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c0ce9b0 con 0x7f731c15a060 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f7321e4a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0cf530 con 0x7f731c069320 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f7321e4a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0cf9c0 con 0x7f731c159c90 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f731bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f731c069320 0x7f731c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f731bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f731c069320 0x7f731c0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47286/0 (socket says 172.21.15.7:47286) 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f731bfff700 1 -- 172.21.15.7:0/4028664072 learned_addr learned my addr 172.21.15.7:0/4028664072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.446+0000 7f731b7fe700 1 --2- 172.21.15.7:0/4028664072 >> v2:172.21.15.113:3300/0 conn(0x7f731c159c90 0x7f731c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.447+0000 7f731bfff700 1 -- 172.21.15.7:0/4028664072 >> v2:172.21.15.113:3300/0 conn(0x7f731c159c90 msgr2=0x7f731c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.447+0000 7f731bfff700 1 --2- 172.21.15.7:0/4028664072 >> v2:172.21.15.113:3300/0 conn(0x7f731c159c90 0x7f731c0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.447+0000 7f731bfff700 1 -- 172.21.15.7:0/4028664072 >> v1:172.21.15.113:6789/0 conn(0x7f731c15a060 legacy=0x7f731c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.447+0000 7f731bfff700 1 -- 172.21.15.7:0/4028664072 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c0cec00 con 0x7f731c069320 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.447+0000 7f731bfff700 1 --2- 172.21.15.7:0/4028664072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c069320 0x7f731c0b3520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f731c05f2e0 tx=0x7f7314009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5e78d8a926be07 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.447+0000 7f731a7fc700 1 -- 172.21.15.7:0/4028664072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731400e070 con 0x7f731c069320 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.447+0000 7f731a7fc700 1 -- 172.21.15.7:0/4028664072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731400ba10 con 0x7f731c069320 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.447+0000 7f731a7fc700 1 -- 172.21.15.7:0/4028664072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140092e0 con 0x7f731c069320 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.448+0000 7f7321e4a700 1 -- 172.21.15.7:0/4028664072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c069320 msgr2=0x7f731c0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.448+0000 7f7321e4a700 1 --2- 172.21.15.7:0/4028664072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c069320 0x7f731c0b3520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f731c05f2e0 tx=0x7f7314009840 comp rx=0 tx=0).stop 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.448+0000 7f7321e4a700 1 -- 172.21.15.7:0/4028664072 shutdown_connections 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.448+0000 7f7321e4a700 1 --2- 172.21.15.7:0/4028664072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c069320 0x7f731c0b3520 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.448+0000 7f7321e4a700 1 --2- 172.21.15.7:0/4028664072 >> v2:172.21.15.113:3300/0 conn(0x7f731c159c90 0x7f731c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.448+0000 7f7321e4a700 1 -- 172.21.15.7:0/4028664072 >> 172.21.15.7:0/4028664072 conn(0x7f731c071100 msgr2=0x7f731c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.448+0000 7f7321e4a700 1 -- 172.21.15.7:0/4028664072 shutdown_connections 2024-01-11T07:08:26.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.448+0000 7f7321e4a700 1 -- 172.21.15.7:0/4028664072 wait complete. 2024-01-11T07:08:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.449+0000 7f7321e4a700 1 Processor -- start 2024-01-11T07:08:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.449+0000 7f7321e4a700 1 -- start start 2024-01-11T07:08:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f7321e4a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c069320 0x7f731c16d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f7321e4a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c159c90 0x7f731c16d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f7321e4a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c15a060 0x7f731c0ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f7321e4a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0aeb90 con 0x7f731c15a060 2024-01-11T07:08:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f7321e4a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0aee70 con 0x7f731c069320 2024-01-11T07:08:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f7321e4a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0af150 con 0x7f731c159c90 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c15a060 0x7f731c0ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c15a060 0x7f731c0ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47300/0 (socket says 172.21.15.7:47300) 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731bfff700 1 -- 172.21.15.7:0/3876282529 learned_addr learned my addr 172.21.15.7:0/3876282529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731b7fe700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c069320 0x7f731c16d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731affd700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c159c90 0x7f731c16d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731bfff700 1 -- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c159c90 msgr2=0x7f731c16d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731bfff700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c159c90 0x7f731c16d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731bfff700 1 -- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c069320 msgr2=0x7f731c16d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731bfff700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c069320 0x7f731c16d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731bfff700 1 -- 172.21.15.7:0/3876282529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314009ae0 con 0x7f731c15a060 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.450+0000 7f731b7fe700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c069320 0x7f731c16d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.451+0000 7f731bfff700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c15a060 0x7f731c0ade60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f731c06a1d0 tx=0x7f7314017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.451+0000 7f73137fe700 1 -- 172.21.15.7:0/3876282529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140032f0 con 0x7f731c15a060 2024-01-11T07:08:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.451+0000 7f73137fe700 1 -- 172.21.15.7:0/3876282529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314012e60 con 0x7f731c15a060 2024-01-11T07:08:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.451+0000 7f73137fe700 1 -- 172.21.15.7:0/3876282529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731401d650 con 0x7f731c15a060 2024-01-11T07:08:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.451+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c0af430 con 0x7f731c15a060 2024-01-11T07:08:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.451+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c154340 con 0x7f731c15a060 2024-01-11T07:08:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.452+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c0afa80 con 0x7f731c15a060 2024-01-11T07:08:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.452+0000 7f73137fe700 1 -- 172.21.15.7:0/3876282529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f731401da80 con 0x7f731c15a060 2024-01-11T07:08:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.453+0000 7f73137fe700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72fc066f30 0x7f72fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.453+0000 7f73137fe700 1 -- 172.21.15.7:0/3876282529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7314085d10 con 0x7f731c15a060 2024-01-11T07:08:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.453+0000 7f731b7fe700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72fc066f30 0x7f72fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.454+0000 7f731b7fe700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72fc066f30 0x7f72fc0693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f730c006fd0 tx=0x7f730c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.457+0000 7f73137fe700 1 -- 172.21.15.7:0/3876282529 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7314052f80 con 0x7f731c15a060 2024-01-11T07:08:26.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.713+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f731c04c3c0 con 0x7f731c15a060 2024-01-11T07:08:26.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.713+0000 7f73137fe700 1 -- 172.21.15.7:0/3876282529 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f73140565a0 con 0x7f731c15a060 2024-01-11T07:08:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:08:26.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72fc066f30 msgr2=0x7f72fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72fc066f30 0x7f72fc0693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f730c006fd0 tx=0x7f730c008040 comp rx=0 tx=0).stop 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c15a060 msgr2=0x7f731c0ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c15a060 0x7f731c0ade60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f731c06a1d0 tx=0x7f7314017000 comp rx=0 tx=0).stop 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 shutdown_connections 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72fc066f30 0x7f72fc0693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c15a060 0x7f731c0ade60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f731c069320 0x7f731c16d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 --2- 172.21.15.7:0/3876282529 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f731c159c90 0x7f731c16d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 >> 172.21.15.7:0/3876282529 conn(0x7f731c071100 msgr2=0x7f731c067b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 shutdown_connections 2024-01-11T07:08:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:26.716+0000 7f7321e4a700 1 -- 172.21.15.7:0/3876282529 wait complete. 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:08:26.730 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:08:26.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:26.731 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:08:26.750 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:08:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:08:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:08:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42311sec preferred_lft 42311sec 2024-01-11T07:08:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:26.752 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:08:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:08:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:08:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:26.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:26.753 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:08:26.821 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:08:26.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:26.823 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:08:26.887 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:08:26.888 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:08:26.888 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:08:26.888 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:08:26.888 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:08:26.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:26.888 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:08:26.888 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:08:26.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:08:26.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:08:26.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:08:26.889 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:08:27.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:27.012 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:08:27.033 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:08:27.033 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:08:27.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:27.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:08:27.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:08:27.109 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:08:27.171 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:08:27.172 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:08:27.172 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:08:27.234 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:08:27.235 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:08:27.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:08:27.329 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:08:27.329 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:08:27.330 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:08:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:08:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:08:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:08:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:08:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:08:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:08:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:08:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:08:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:08:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:08:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:08:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:08:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:08:27.360 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:08:27.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:27.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:08:27.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:08:27.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.570+0000 7f709fcd2700 1 Processor -- start 2024-01-11T07:08:27.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.570+0000 7f709fcd2700 1 -- start start 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.571+0000 7f709fcd2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f709815ae60 0x7f709815c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.571+0000 7f709fcd2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f709806aa10 0x7f709806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.571+0000 7f709fcd2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709815c6a0 con 0x7f709806ece0 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.571+0000 7f709fcd2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70980b0490 con 0x7f709806aa10 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.571+0000 7f709fcd2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709806e310 con 0x7f709815ae60 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.571+0000 7f709d26d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f709806aa10 0x7f709806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.571+0000 7f709d26d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f709806aa10 0x7f709806e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41370/0 (socket says 172.21.15.7:41370) 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.571+0000 7f709d26d700 1 -- 172.21.15.7:0/1444820256 learned_addr learned my addr 172.21.15.7:0/1444820256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709da6e700 1 --2- 172.21.15.7:0/1444820256 >> v2:172.21.15.113:3300/0 conn(0x7f709815ae60 0x7f709815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709d26d700 1 -- 172.21.15.7:0/1444820256 >> v2:172.21.15.113:3300/0 conn(0x7f709815ae60 msgr2=0x7f709815c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709d26d700 1 --2- 172.21.15.7:0/1444820256 >> v2:172.21.15.113:3300/0 conn(0x7f709815ae60 0x7f709815c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709d26d700 1 -- 172.21.15.7:0/1444820256 >> v1:172.21.15.113:6789/0 conn(0x7f709806ece0 legacy=0x7f7098063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:27.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709d26d700 1 -- 172.21.15.7:0/1444820256 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709806dd30 con 0x7f709806aa10 2024-01-11T07:08:27.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709d26d700 1 --2- 172.21.15.7:0/1444820256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709806aa10 0x7f709806e7a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7098069570 tx=0x7f708c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790382d946fae3bf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:27.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709ca6c700 1 -- 172.21.15.7:0/1444820256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00b820 con 0x7f709806aa10 2024-01-11T07:08:27.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709ca6c700 1 -- 172.21.15.7:0/1444820256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c00b980 con 0x7f709806aa10 2024-01-11T07:08:27.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.572+0000 7f709ca6c700 1 -- 172.21.15.7:0/1444820256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00bd20 con 0x7f709806aa10 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.573+0000 7f709fcd2700 1 -- 172.21.15.7:0/1444820256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709806aa10 msgr2=0x7f709806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.573+0000 7f709fcd2700 1 --2- 172.21.15.7:0/1444820256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709806aa10 0x7f709806e7a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7098069570 tx=0x7f708c009250 comp rx=0 tx=0).stop 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.573+0000 7f709fcd2700 1 -- 172.21.15.7:0/1444820256 shutdown_connections 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.573+0000 7f709fcd2700 1 --2- 172.21.15.7:0/1444820256 >> v2:172.21.15.113:3300/0 conn(0x7f709815ae60 0x7f709815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.573+0000 7f709fcd2700 1 --2- 172.21.15.7:0/1444820256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709806aa10 0x7f709806e7a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.573+0000 7f709fcd2700 1 -- 172.21.15.7:0/1444820256 >> 172.21.15.7:0/1444820256 conn(0x7f7098067610 msgr2=0x7f70980679e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.573+0000 7f709fcd2700 1 -- 172.21.15.7:0/1444820256 shutdown_connections 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.573+0000 7f709fcd2700 1 -- 172.21.15.7:0/1444820256 wait complete. 2024-01-11T07:08:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.574+0000 7f709fcd2700 1 Processor -- start 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.574+0000 7f709fcd2700 1 -- start start 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709fcd2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f709806aa10 0x7f70980ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709fcd2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f709806ece0 0x7f70980c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709fcd2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709815ae60 0x7f70980cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709fcd2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c8b90 con 0x7f709815ae60 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709fcd2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c98b0 con 0x7f709806aa10 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709fcd2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c8fc0 con 0x7f709806ece0 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709e26f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709815ae60 0x7f70980cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709e26f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709815ae60 0x7f70980cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41378/0 (socket says 172.21.15.7:41378) 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709e26f700 1 -- 172.21.15.7:0/2909313256 learned_addr learned my addr 172.21.15.7:0/2909313256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709da6e700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f709806aa10 0x7f70980ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709e26f700 1 -- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f709806ece0 msgr2=0x7f70980c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709d26d700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f709806ece0 0x7f70980c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709e26f700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f709806ece0 0x7f70980c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709e26f700 1 -- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f709806aa10 msgr2=0x7f70980ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709e26f700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f709806aa10 0x7f70980ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.575+0000 7f709e26f700 1 -- 172.21.15.7:0/2909313256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0094f0 con 0x7f709815ae60 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.576+0000 7f709e26f700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709815ae60 0x7f70980cafa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f709000c960 tx=0x7f70900026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.576+0000 7f7089ffb700 1 -- 172.21.15.7:0/2909313256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000cea0 con 0x7f709815ae60 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.576+0000 7f7089ffb700 1 -- 172.21.15.7:0/2909313256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709000a740 con 0x7f709815ae60 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.576+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70980c9260 con 0x7f709815ae60 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.576+0000 7f7089ffb700 1 -- 172.21.15.7:0/2909313256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000ab00 con 0x7f709815ae60 2024-01-11T07:08:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.576+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70980c9d40 con 0x7f709815ae60 2024-01-11T07:08:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.577+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70980c8d10 con 0x7f709815ae60 2024-01-11T07:08:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.578+0000 7f7089ffb700 1 -- 172.21.15.7:0/2909313256 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7090012420 con 0x7f709815ae60 2024-01-11T07:08:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.578+0000 7f7089ffb700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f707c066f30 0x7f707c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.578+0000 7f7089ffb700 1 -- 172.21.15.7:0/2909313256 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7090084950 con 0x7f709815ae60 2024-01-11T07:08:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.578+0000 7f709da6e700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f707c066f30 0x7f707c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.579+0000 7f709da6e700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f707c066f30 0x7f707c0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7094006fd0 tx=0x7f7094008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:27.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.583+0000 7f7089ffb700 1 -- 172.21.15.7:0/2909313256 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7090051bc0 con 0x7f709815ae60 2024-01-11T07:08:27.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.842+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f709804c3c0 con 0x7f709815ae60 2024-01-11T07:08:27.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.842+0000 7f7089ffb700 1 -- 172.21.15.7:0/2909313256 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f70900551e0 con 0x7f709815ae60 2024-01-11T07:08:27.843 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:08:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f707c066f30 msgr2=0x7f707c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f707c066f30 0x7f707c0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7094006fd0 tx=0x7f7094008040 comp rx=0 tx=0).stop 2024-01-11T07:08:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709815ae60 msgr2=0x7f70980cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709815ae60 0x7f70980cafa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f709000c960 tx=0x7f70900026e0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 shutdown_connections 2024-01-11T07:08:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f707c066f30 0x7f707c0693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709815ae60 0x7f70980cafa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f709806aa10 0x7f70980ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 --2- 172.21.15.7:0/2909313256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f709806ece0 0x7f70980c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 >> 172.21.15.7:0/2909313256 conn(0x7f7098067610 msgr2=0x7f70980b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.844+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 shutdown_connections 2024-01-11T07:08:27.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:27.845+0000 7f709fcd2700 1 -- 172.21.15.7:0/2909313256 wait complete. 2024-01-11T07:08:27.858 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:08:27.858 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:08:27.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:08:27.859 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:08:27.859 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:08:27.859 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:08:27.859 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:08:27.859 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:08:27.859 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:08:27.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:27.859 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:08:27.881 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:08:27.881 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:08:27.881 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:08:27.881 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42353sec preferred_lft 42353sec 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:08:27.882 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:08:27.883 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:08:27.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:27.884 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:08:27.951 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:08:27.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:27.952 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:08:28.013 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:08:28.013 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:08:28.013 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:08:28.013 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:08:28.013 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:08:28.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:28.014 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:08:28.014 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:08:28.014 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:08:28.014 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:08:28.014 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:08:28.014 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:08:28.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:28.123 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:08:28.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:08:28.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:08:28.141 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:08:28.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:28.141 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:08:28.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:08:28.209 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:08:28.266 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:08:28.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:08:28.267 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:08:28.323 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:08:28.323 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:08:28.323 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:08:28.405 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:08:28.406 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:08:28.406 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:08:28.415 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:08:28.415 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:08:28.415 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:08:28.415 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:08:28.415 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:08:28.433 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:08:28.433 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:08:28.433 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:08:28.433 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:08:28.433 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:08:28.433 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:08:28.433 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:08:28.433 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:08:28.435 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:08:28.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:28.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:08:28.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.642+0000 7f3bee4c3700 1 Processor -- start 2024-01-11T07:08:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.642+0000 7f3bee4c3700 1 -- start start 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.643+0000 7f3bee4c3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3be8158280 0x7f3be80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.643+0000 7f3bee4c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3be815aa80 0x7f3be80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.643+0000 7f3bee4c3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be8159300 con 0x7f3be80cafb0 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.643+0000 7f3bee4c3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be80cf000 con 0x7f3be815aa80 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.643+0000 7f3bee4c3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be80cf260 con 0x7f3be8158280 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.643+0000 7f3beca60700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3be815aa80 0x7f3be80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3be7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3be80cafb0 0x7f3be80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50212/0 (socket says 172.21.15.7:50212) 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3beca60700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3be815aa80 0x7f3be80b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41392/0 (socket says 172.21.15.7:41392) 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3be7fff700 1 -- 172.21.15.7:0/3131914863 learned_addr learned my addr 172.21.15.7:0/3131914863 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3be77fe700 1 --2- 172.21.15.7:0/3131914863 >> v2:172.21.15.113:3300/0 conn(0x7f3be8158280 0x7f3be80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3beca60700 1 -- 172.21.15.7:0/3131914863 >> v2:172.21.15.113:3300/0 conn(0x7f3be8158280 msgr2=0x7f3be80b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3beca60700 1 --2- 172.21.15.7:0/3131914863 >> v2:172.21.15.113:3300/0 conn(0x7f3be8158280 0x7f3be80b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3beca60700 1 -- 172.21.15.7:0/3131914863 >> v1:172.21.15.7:6789/0 conn(0x7f3be80cafb0 legacy=0x7f3be80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3beca60700 1 -- 172.21.15.7:0/3131914863 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be80cf540 con 0x7f3be815aa80 2024-01-11T07:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3be6ffd700 1 -- 172.21.15.7:0/3131914863 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50535673 0 0) 0x7f3be8159300 con 0x7f3be80cafb0 2024-01-11T07:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3beca60700 1 --2- 172.21.15.7:0/3131914863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be815aa80 0x7f3be80b0970 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3be806a660 tx=0x7f3be0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e2f193013bc962 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3be6ffd700 1 -- 172.21.15.7:0/3131914863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000e070 con 0x7f3be815aa80 2024-01-11T07:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3be6ffd700 1 -- 172.21.15.7:0/3131914863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000bb60 con 0x7f3be815aa80 2024-01-11T07:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.644+0000 7f3be6ffd700 1 -- 172.21.15.7:0/3131914863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0012430 con 0x7f3be815aa80 2024-01-11T07:08:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.645+0000 7f3bee4c3700 1 -- 172.21.15.7:0/3131914863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be815aa80 msgr2=0x7f3be80b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.645+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/3131914863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be815aa80 0x7f3be80b0970 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3be806a660 tx=0x7f3be0009780 comp rx=0 tx=0).stop 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.645+0000 7f3bee4c3700 1 -- 172.21.15.7:0/3131914863 shutdown_connections 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.645+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/3131914863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be815aa80 0x7f3be80b0970 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.645+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/3131914863 >> v2:172.21.15.113:3300/0 conn(0x7f3be8158280 0x7f3be80b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.645+0000 7f3bee4c3700 1 -- 172.21.15.7:0/3131914863 >> 172.21.15.7:0/3131914863 conn(0x7f3be8071100 msgr2=0x7f3be806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.645+0000 7f3bee4c3700 1 -- 172.21.15.7:0/3131914863 shutdown_connections 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.645+0000 7f3bee4c3700 1 -- 172.21.15.7:0/3131914863 wait complete. 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.646+0000 7f3bee4c3700 1 Processor -- start 2024-01-11T07:08:28.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.646+0000 7f3bee4c3700 1 -- start start 2024-01-11T07:08:28.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3bee4c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be80cafb0 0x7f3be80b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:28.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3bee4c3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3be8158280 0x7f3be80c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:28.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3bee4c3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3be815aa80 0x7f3be80c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3bee4c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80af2c0 con 0x7f3be80cafb0 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3bee4c3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80af5a0 con 0x7f3be8158280 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3bee4c3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80af880 con 0x7f3be815aa80 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3be7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be80cafb0 0x7f3be80b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3be7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be80cafb0 0x7f3be80b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41406/0 (socket says 172.21.15.7:41406) 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.647+0000 7f3be7fff700 1 -- 172.21.15.7:0/1836723651 learned_addr learned my addr 172.21.15.7:0/1836723651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3be77fe700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3be8158280 0x7f3be80c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3beca60700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3be815aa80 0x7f3be80c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3be7fff700 1 -- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3be815aa80 msgr2=0x7f3be80c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3be7fff700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3be815aa80 0x7f3be80c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3be7fff700 1 -- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3be8158280 msgr2=0x7f3be80c4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3be7fff700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3be8158280 0x7f3be80c4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3be7fff700 1 -- 172.21.15.7:0/1836723651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0009a20 con 0x7f3be80cafb0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3be7fff700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be80cafb0 0x7f3be80b0640 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c7e0 tx=0x7f3bd8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3bcffff700 1 -- 172.21.15.7:0/1836723651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8005b30 con 0x7f3be80cafb0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3bcffff700 1 -- 172.21.15.7:0/1836723651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd8005c90 con 0x7f3be80cafb0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3bcffff700 1 -- 172.21.15.7:0/1836723651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8007300 con 0x7f3be80cafb0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be80afb60 con 0x7f3be80cafb0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.648+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be80b0120 con 0x7f3be80cafb0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.649+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be81574d0 con 0x7f3be80cafb0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.650+0000 7f3bcffff700 1 -- 172.21.15.7:0/1836723651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3bd801b400 con 0x7f3be80cafb0 2024-01-11T07:08:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.650+0000 7f3bcffff700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3bd4066f30 0x7f3bd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:28.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.650+0000 7f3bcffff700 1 -- 172.21.15.7:0/1836723651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3bd8084040 con 0x7f3be80cafb0 2024-01-11T07:08:28.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.650+0000 7f3be77fe700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3bd4066f30 0x7f3bd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:28.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.651+0000 7f3be77fe700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3bd4066f30 0x7f3bd40693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3be8157700 tx=0x7f3bdc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:28.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.654+0000 7f3bcffff700 1 -- 172.21.15.7:0/1836723651 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bd8051330 con 0x7f3be80cafb0 2024-01-11T07:08:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.905+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3be80b0340 con 0x7f3be80cafb0 2024-01-11T07:08:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.905+0000 7f3bcffff700 1 -- 172.21.15.7:0/1836723651 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f3bd8054950 con 0x7f3be80cafb0 2024-01-11T07:08:28.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:28.906 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:28.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3bd4066f30 msgr2=0x7f3bd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:28.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3bd4066f30 0x7f3bd40693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3be8157700 tx=0x7f3bdc008040 comp rx=0 tx=0).stop 2024-01-11T07:08:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be80cafb0 msgr2=0x7f3be80b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be80cafb0 0x7f3be80b0640 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c7e0 tx=0x7f3bd8007df0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 shutdown_connections 2024-01-11T07:08:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3bd4066f30 0x7f3bd40693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3be815aa80 0x7f3be80c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be80cafb0 0x7f3be80b0640 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 --2- 172.21.15.7:0/1836723651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3be8158280 0x7f3be80c4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 >> 172.21.15.7:0/1836723651 conn(0x7f3be8071100 msgr2=0x7f3be81565f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:28.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 shutdown_connections 2024-01-11T07:08:28.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:28.908+0000 7f3bee4c3700 1 -- 172.21.15.7:0/1836723651 wait complete. 2024-01-11T07:08:28.911 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:28.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:08:29.077 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":5310,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8834555129999995,"mdsmap_epoch":25,"osdmap_epoch":47,"osdmap_epoch_barrier":47,"uptime":14.138920826} 2024-01-11T07:08:29.077 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 5310, "mdsmap_epoch": 25, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "rank_uptime": 8.883455513, "state": "up:active", "uptime": 14.138920826, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:08:29.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-11T07:08:29.226 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-11T07:08:29.226 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-11T07:08:29.226 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-01-11T07:08:29.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:29.227 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-01-11T07:08:30.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:08:30.400 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:08:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:265+0 records in 2024-01-11T07:08:30.735 INFO:teuthology.orchestra.run.smithi007.stderr:265+0 records out 2024-01-11T07:08:30.735 INFO:teuthology.orchestra.run.smithi007.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.48607 s, 187 MB/s 2024-01-11T07:08:30.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:30.736 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-01-11T07:08:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:292+0 records in 2024-01-11T07:08:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:292+0 records out 2024-01-11T07:08:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.65905 s, 185 MB/s 2024-01-11T07:08:32.417 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-01-11T07:08:32.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:08:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.552+0000 7fb9d01d3700 1 Processor -- start 2024-01-11T07:08:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.552+0000 7fb9d01d3700 1 -- start start 2024-01-11T07:08:32.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9d01d3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c8164f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:32.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9d01d3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c80d0220 con 0x7fb9c8159d80 2024-01-11T07:08:32.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9d01d3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c80cf610 con 0x7fb9c806e340 2024-01-11T07:08:32.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9d01d3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80d06b0 con 0x7fb9c8162ba0 2024-01-11T07:08:32.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9cdf6f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb9c8159d80 0x7fb9c806dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50222/0 (socket says 172.21.15.7:50222) 2024-01-11T07:08:32.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9cdf6f700 1 -- 172.21.15.7:0/4206793967 learned_addr learned my addr 172.21.15.7:0/4206793967 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9ce770700 1 --2- 172.21.15.7:0/4206793967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c8164f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9ccf6d700 1 -- 172.21.15.7:0/4206793967 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918867235 0 0) 0x7fb9c80d0220 con 0x7fb9c8159d80 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.553+0000 7fb9ccf6d700 1 -- 172.21.15.7:0/4206793967 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9b4003570 con 0x7fb9c8159d80 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ccf6d700 1 -- 172.21.15.7:0/4206793967 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928341408 0 0) 0x7fb9c80cf610 con 0x7fb9c806e340 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ccf6d700 1 -- 172.21.15.7:0/4206793967 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c80d0220 con 0x7fb9c806e340 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ce770700 1 -- 172.21.15.7:0/4206793967 >> v1:172.21.15.113:6789/0 conn(0x7fb9c806e340 legacy=0x7fb9c815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ce770700 1 -- 172.21.15.7:0/4206793967 >> v1:172.21.15.7:6789/0 conn(0x7fb9c8159d80 legacy=0x7fb9c806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ce770700 1 -- 172.21.15.7:0/4206793967 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80cf8f0 con 0x7fb9c8162ba0 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ce770700 1 --2- 172.21.15.7:0/4206793967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c8164f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009e90 tx=0x7fb9c0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0fb25f04d287ac server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ccf6d700 1 -- 172.21.15.7:0/4206793967 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000e070 con 0x7fb9c8162ba0 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ccf6d700 1 -- 172.21.15.7:0/4206793967 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c000bb80 con 0x7fb9c8162ba0 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.554+0000 7fb9ccf6d700 1 -- 172.21.15.7:0/4206793967 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0012430 con 0x7fb9c8162ba0 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.555+0000 7fb9d01d3700 1 -- 172.21.15.7:0/4206793967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 msgr2=0x7fb9c8164f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.555+0000 7fb9d01d3700 1 --2- 172.21.15.7:0/4206793967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c8164f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009e90 tx=0x7fb9c0009780 comp rx=0 tx=0).stop 2024-01-11T07:08:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.555+0000 7fb9d01d3700 1 -- 172.21.15.7:0/4206793967 shutdown_connections 2024-01-11T07:08:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.555+0000 7fb9d01d3700 1 --2- 172.21.15.7:0/4206793967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c8164f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.555+0000 7fb9d01d3700 1 -- 172.21.15.7:0/4206793967 >> 172.21.15.7:0/4206793967 conn(0x7fb9c8071130 msgr2=0x7fb9c80b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.555+0000 7fb9d01d3700 1 -- 172.21.15.7:0/4206793967 shutdown_connections 2024-01-11T07:08:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.555+0000 7fb9d01d3700 1 -- 172.21.15.7:0/4206793967 wait complete. 2024-01-11T07:08:32.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.556+0000 7fb9d01d3700 1 Processor -- start 2024-01-11T07:08:32.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.556+0000 7fb9d01d3700 1 -- start start 2024-01-11T07:08:32.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9d01d3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c806e340 0x7fb9c814c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:32.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9d01d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c8159d80 0x7fb9c80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:32.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9d01d3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c80b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9d01d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c814cd50 con 0x7fb9c8159d80 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9d01d3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c816c510 con 0x7fb9c806e340 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9d01d3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c816b900 con 0x7fb9c8162ba0 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c8159d80 0x7fb9c80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c8159d80 0x7fb9c80b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41410/0 (socket says 172.21.15.7:41410) 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 -- 172.21.15.7:0/1093234856 learned_addr learned my addr 172.21.15.7:0/1093234856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cdf6f700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c806e340 0x7fb9c814c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9ce770700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c80b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 -- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 msgr2=0x7fb9c80b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c80b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 -- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c806e340 msgr2=0x7fb9c814c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c806e340 0x7fb9c814c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 -- 172.21.15.7:0/1093234856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0009a20 con 0x7fb9c8159d80 2024-01-11T07:08:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.557+0000 7fb9cd76e700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c8159d80 0x7fb9c80b4d50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c7c0 tx=0x7fb9bc007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.558+0000 7fb9b9ffb700 1 -- 172.21.15.7:0/1093234856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc010510 con 0x7fb9c8159d80 2024-01-11T07:08:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.558+0000 7fb9b9ffb700 1 -- 172.21.15.7:0/1093234856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc0106b0 con 0x7fb9c8159d80 2024-01-11T07:08:32.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.558+0000 7fb9b9ffb700 1 -- 172.21.15.7:0/1093234856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc010a70 con 0x7fb9c8159d80 2024-01-11T07:08:32.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.558+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c816bbe0 con 0x7fb9c8159d80 2024-01-11T07:08:32.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.558+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c816c1a0 con 0x7fb9c8159d80 2024-01-11T07:08:32.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.559+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c8089080 con 0x7fb9c8159d80 2024-01-11T07:08:32.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.559+0000 7fb9b9ffb700 1 -- 172.21.15.7:0/1093234856 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb9bc012440 con 0x7fb9c8159d80 2024-01-11T07:08:32.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.559+0000 7fb9b9ffb700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9ac066f30 0x7fb9ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:32.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.560+0000 7fb9b9ffb700 1 -- 172.21.15.7:0/1093234856 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb9bc083b80 con 0x7fb9c8159d80 2024-01-11T07:08:32.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.560+0000 7fb9cdf6f700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9ac066f30 0x7fb9ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:32.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.560+0000 7fb9cdf6f700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9ac066f30 0x7fb9ac0693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4007fd0 tx=0x7fb9c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:32.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.565+0000 7fb9b9ffb700 1 -- 172.21.15.7:0/1093234856 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9bc04b5a0 con 0x7fb9c8159d80 2024-01-11T07:08:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.770+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb9c804c3c0 con 0x7fb9c8159d80 2024-01-11T07:08:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.771+0000 7fb9b9ffb700 1 -- 172.21.15.7:0/1093234856 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v47) v1 ==== 81+0+32212 (secure 0 0 0) 0x7fb9bc017070 con 0x7fb9c8159d80 2024-01-11T07:08:32.772 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:32.772 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:32.772 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 47, 2024-01-11T07:08:32.772 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:08:21.182993+0000", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:08:32.773 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:32.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:08:32.775 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "18", 2024-01-11T07:08:32.776 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:32.777 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.778 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:08:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:17.698588+0000", 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:32.780 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:32.781 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:32.782 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:32.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:32.784 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-11T07:08:32.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:18.590636+0000", 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:08:32.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "47", 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:32.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:32.788 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:32.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:32.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:08:32.792 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:08:32.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:08:32.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:32.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:32.802 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:32.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:32.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:32.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:32.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:32.808 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.809 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.810 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:32.813 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:08:32.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:08:32.818 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:32.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:08:32.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:32.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:32.824 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:32.825 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.826 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:08:32.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:08:32.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:32.830 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:32.831 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:32.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.836 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:32.837 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:08:32.838 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:32.839 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:32.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:08:32.842 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:08:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:08:32.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:08:32.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:08:32.844 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:08:32.844 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:08:32.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:08:32.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:32.844 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9ac066f30 msgr2=0x7fb9ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9ac066f30 0x7fb9ac0693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4007fd0 tx=0x7fb9c4009040 comp rx=0 tx=0).stop 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c8159d80 msgr2=0x7fb9c80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c8159d80 0x7fb9c80b4d50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00c7c0 tx=0x7fb9bc007de0 comp rx=0 tx=0).stop 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 shutdown_connections 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9ac066f30 0x7fb9ac0693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9c8162ba0 0x7fb9c80b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c806e340 0x7fb9c814c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 --2- 172.21.15.7:0/1093234856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9c8159d80 0x7fb9c80b4d50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 >> 172.21.15.7:0/1093234856 conn(0x7fb9c8071130 msgr2=0x7fb9c8153420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 shutdown_connections 2024-01-11T07:08:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:32.774+0000 7fb9d01d3700 1 -- 172.21.15.7:0/1093234856 wait complete. 2024-01-11T07:08:32.847 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:08:34.295 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:08:34.295+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-01-11T07:08:37.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:08:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.988+0000 7f2cc3fd6700 1 Processor -- start 2024-01-11T07:08:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.988+0000 7f2cc3fd6700 1 -- start start 2024-01-11T07:08:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.989+0000 7f2cc3fd6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc0aff20 0x7f2cbc0aab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.989+0000 7f2cc3fd6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2cbc0636a0 0x7f2cbc06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.989+0000 7f2cc3fd6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc0ab040 con 0x7f2cbc063230 2024-01-11T07:08:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.989+0000 7f2cc3fd6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b02f0 con 0x7f2cbc0636a0 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.989+0000 7f2cc3fd6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc154860 con 0x7f2cbc0aff20 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.989+0000 7f2cc2573700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2cbc0636a0 0x7f2cbc06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.989+0000 7f2cc1571700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2cbc063230 0x7f2cbc06dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49436/0 (socket says 172.21.15.7:49436) 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.989+0000 7f2cc1571700 1 -- 172.21.15.7:0/3160333015 learned_addr learned my addr 172.21.15.7:0/3160333015 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc1d72700 1 --2- 172.21.15.7:0/3160333015 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc0aff20 0x7f2cbc0aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc2573700 1 -- 172.21.15.7:0/3160333015 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc0aff20 msgr2=0x7f2cbc0aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc2573700 1 --2- 172.21.15.7:0/3160333015 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc0aff20 0x7f2cbc0aab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc2573700 1 -- 172.21.15.7:0/3160333015 >> v1:172.21.15.7:6789/0 conn(0x7f2cbc063230 legacy=0x7f2cbc06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc2573700 1 -- 172.21.15.7:0/3160333015 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc15d910 con 0x7f2cbc0636a0 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc0d70700 1 -- 172.21.15.7:0/3160333015 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569719018 0 0) 0x7f2cbc0ab040 con 0x7f2cbc063230 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc2573700 1 --2- 172.21.15.7:0/3160333015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0636a0 0x7f2cbc06e1f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009e60 tx=0x7f2cb4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89d2b3aaa90288e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc0d70700 1 -- 172.21.15.7:0/3160333015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400e070 con 0x7f2cbc0636a0 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc0d70700 1 -- 172.21.15.7:0/3160333015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb400bc30 con 0x7f2cbc0636a0 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.990+0000 7f2cc0d70700 1 -- 172.21.15.7:0/3160333015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4012430 con 0x7f2cbc0636a0 2024-01-11T07:08:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.991+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/3160333015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0636a0 msgr2=0x7f2cbc06e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.991+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/3160333015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0636a0 0x7f2cbc06e1f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009e60 tx=0x7f2cb4009780 comp rx=0 tx=0).stop 2024-01-11T07:08:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.991+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/3160333015 shutdown_connections 2024-01-11T07:08:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.991+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/3160333015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0636a0 0x7f2cbc06e1f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.991+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/3160333015 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc0aff20 0x7f2cbc0aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.991+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/3160333015 >> 172.21.15.7:0/3160333015 conn(0x7f2cbc068090 msgr2=0x7f2cbc0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.991+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/3160333015 shutdown_connections 2024-01-11T07:08:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.991+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/3160333015 wait complete. 2024-01-11T07:08:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.992+0000 7f2cc3fd6700 1 Processor -- start 2024-01-11T07:08:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.992+0000 7f2cc3fd6700 1 -- start start 2024-01-11T07:08:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc3fd6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc063230 0x7f2cbc153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc3fd6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cbc0636a0 0x7f2cbc06a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc3fd6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0aff20 0x7f2cbc0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc3fd6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc06bb40 con 0x7f2cbc0aff20 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc3fd6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b3060 con 0x7f2cbc0636a0 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc3fd6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc06bfd0 con 0x7f2cbc063230 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0aff20 0x7f2cbc0b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0aff20 0x7f2cbc0b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48336/0 (socket says 172.21.15.7:48336) 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 -- 172.21.15.7:0/2093046856 learned_addr learned my addr 172.21.15.7:0/2093046856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc1d72700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc063230 0x7f2cbc153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc1571700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cbc0636a0 0x7f2cbc06a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 -- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc063230 msgr2=0x7f2cbc153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc063230 0x7f2cbc153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 -- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cbc0636a0 msgr2=0x7f2cbc06a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cbc0636a0 0x7f2cbc06a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 -- 172.21.15.7:0/2093046856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4009a20 con 0x7f2cbc0aff20 2024-01-11T07:08:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.993+0000 7f2cc2573700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0aff20 0x7f2cbc0b0b20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4013000 tx=0x7f2cb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:37.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.994+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4012e50 con 0x7f2cbc0aff20 2024-01-11T07:08:37.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.994+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb401d630 con 0x7f2cbc0aff20 2024-01-11T07:08:37.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.994+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc06af30 con 0x7f2cbc0aff20 2024-01-11T07:08:37.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.994+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb401d9f0 con 0x7f2cbc0aff20 2024-01-11T07:08:37.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.994+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc06b4f0 con 0x7f2cbc0aff20 2024-01-11T07:08:37.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.995+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc04c3c0 con 0x7f2cbc0aff20 2024-01-11T07:08:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.995+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2cb4024070 con 0x7f2cbc0aff20 2024-01-11T07:08:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.996+0000 7f2cb1ffb700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.996+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2cb4086ef0 con 0x7f2cbc0aff20 2024-01-11T07:08:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.996+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7f2ca006ea50 con 0x7f2cbc0aff20 2024-01-11T07:08:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.996+0000 7f2cc1d72700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:37.997+0000 7f2cc1d72700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2ca0066f30 0x7f2ca00693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8006fd0 tx=0x7f2cb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.000+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cb40540c0 con 0x7f2cbc0aff20 2024-01-11T07:08:38.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.180+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(50..50 src has 1..50) v4 ==== 288+0+0 (secure 0 0 0) 0x7f2cb4058480 con 0x7f2cbc0aff20 2024-01-11T07:08:38.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.181+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7f2ca006f3a0 con 0x7f2cbc0aff20 2024-01-11T07:08:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.210+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2cbc06bc80 con 0x7f2cbc0aff20 2024-01-11T07:08:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.211+0000 7f2cb1ffb700 1 -- 172.21.15.7:0/2093046856 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+32355 (secure 0 0 0) 0x7f2cb40576e0 con 0x7f2cbc0aff20 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 50, 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:08:38.177756+0000", 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:08:38.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:08:38.212 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:38.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:38.214 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:38.215 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:38.217 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:17.698588+0000", 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:38.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:38.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:38.221 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:38.222 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.223 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:18.590636+0000", 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:38.224 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:08:38.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.230 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:08:38.234 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.235 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.239 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.240 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:08:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:08:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:08:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:08:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.242 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:38.242 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.242 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.242 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.242 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:08:38.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:08:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:08:38.248 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.249 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.256 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:08:38.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:08:38.261 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.262 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.264 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:08:38.268 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.271 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:08:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.291 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:08:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:08:38.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.295 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:38.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2ca0066f30 msgr2=0x7f2ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2ca0066f30 0x7f2ca00693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8006fd0 tx=0x7f2cb8008040 comp rx=0 tx=0).stop 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0aff20 msgr2=0x7f2cbc0b0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0aff20 0x7f2cbc0b0b20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4013000 tx=0x7f2cb4019000 comp rx=0 tx=0).stop 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 shutdown_connections 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2ca0066f30 0x7f2ca00693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cbc0aff20 0x7f2cbc0b0b20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2cbc063230 0x7f2cbc153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 --2- 172.21.15.7:0/2093046856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2cbc0636a0 0x7f2cbc06a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 >> 172.21.15.7:0/2093046856 conn(0x7f2cbc068090 msgr2=0x7f2cbc0cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 shutdown_connections 2024-01-11T07:08:38.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.213+0000 7f2cc3fd6700 1 -- 172.21.15.7:0/2093046856 wait complete. 2024-01-11T07:08:38.298 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-01-11T07:08:38.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:38.298 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-01-11T07:08:38.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:08:38.585 INFO:teuthology.orchestra.run.smithi007.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-01-11T07:08:38.586 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-11T07:08:38.586 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-11T07:08:38.586 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.260919 s, 201 MB/s 2024-01-11T07:08:38.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:08:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.717+0000 7fb10d0e1700 1 Processor -- start 2024-01-11T07:08:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.717+0000 7fb10d0e1700 1 -- start start 2024-01-11T07:08:38.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10d0e1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080cafb0 0x7fb1080ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:38.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10d0e1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb108158280 0x7fb1080b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:38.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10d0e1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb1080b0970 0x7fb108159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:38.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10d0e1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1080b0df0 con 0x7fb1080b0970 2024-01-11T07:08:38.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10d0e1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb108159300 con 0x7fb108158280 2024-01-11T07:08:38.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10d0e1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080cf260 con 0x7fb1080cafb0 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb1080b0970 0x7fb108159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb1080b0970 0x7fb108159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48348/0 (socket says 172.21.15.7:48348) 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 -- 172.21.15.7:0/1025563586 learned_addr learned my addr 172.21.15.7:0/1025563586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb106d9d700 1 --2- 172.21.15.7:0/1025563586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080cafb0 0x7fb1080ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10659c700 1 --2- 172.21.15.7:0/1025563586 >> v2:172.21.15.113:3300/0 conn(0x7fb108158280 0x7fb1080b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 -- 172.21.15.7:0/1025563586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080cafb0 msgr2=0x7fb1080ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 --2- 172.21.15.7:0/1025563586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080cafb0 0x7fb1080ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 -- 172.21.15.7:0/1025563586 >> v2:172.21.15.113:3300/0 conn(0x7fb108158280 msgr2=0x7fb1080b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 --2- 172.21.15.7:0/1025563586 >> v2:172.21.15.113:3300/0 conn(0x7fb108158280 0x7fb1080b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 -- 172.21.15.7:0/1025563586 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080cf540 con 0x7fb1080b0970 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.719+0000 7fb10759e700 1 --2- 172.21.15.7:0/1025563586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1080b0970 0x7fb108159a70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb108064280 tx=0x7fb100009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f4802cb44523cb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb105d9b700 1 -- 172.21.15.7:0/1025563586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000e070 con 0x7fb1080b0970 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb105d9b700 1 -- 172.21.15.7:0/1025563586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10000ba50 con 0x7fb1080b0970 2024-01-11T07:08:38.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb105d9b700 1 -- 172.21.15.7:0/1025563586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000092e0 con 0x7fb1080b0970 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 -- 172.21.15.7:0/1025563586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1080b0970 msgr2=0x7fb108159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/1025563586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1080b0970 0x7fb108159a70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb108064280 tx=0x7fb100009840 comp rx=0 tx=0).stop 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 -- 172.21.15.7:0/1025563586 shutdown_connections 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/1025563586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1080b0970 0x7fb108159a70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/1025563586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080cafb0 0x7fb1080ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/1025563586 >> v2:172.21.15.113:3300/0 conn(0x7fb108158280 0x7fb1080b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 -- 172.21.15.7:0/1025563586 >> 172.21.15.7:0/1025563586 conn(0x7fb108071100 msgr2=0x7fb10806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 -- 172.21.15.7:0/1025563586 shutdown_connections 2024-01-11T07:08:38.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.720+0000 7fb10d0e1700 1 -- 172.21.15.7:0/1025563586 wait complete. 2024-01-11T07:08:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.722+0000 7fb10d0e1700 1 Processor -- start 2024-01-11T07:08:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.722+0000 7fb10d0e1700 1 -- start start 2024-01-11T07:08:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10d0e1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080b0970 0x7fb108148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10d0e1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1080cafb0 0x7fb1080c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10d0e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb108158280 0x7fb1080c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10d0e1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080c62d0 con 0x7fb108158280 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10d0e1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080c6820 con 0x7fb1080cafb0 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10d0e1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1080c4c60 con 0x7fb1080b0970 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb108158280 0x7fb1080c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb108158280 0x7fb1080c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48356/0 (socket says 172.21.15.7:48356) 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.723+0000 7fb10759e700 1 -- 172.21.15.7:0/3214662420 learned_addr learned my addr 172.21.15.7:0/3214662420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb106d9d700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080b0970 0x7fb108148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10659c700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1080cafb0 0x7fb1080c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10759e700 1 -- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080b0970 msgr2=0x7fb108148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10759e700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080b0970 0x7fb108148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10759e700 1 -- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1080cafb0 msgr2=0x7fb1080c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10759e700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1080cafb0 0x7fb1080c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10759e700 1 -- 172.21.15.7:0/3214662420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb100009ae0 con 0x7fb108158280 2024-01-11T07:08:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10659c700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1080cafb0 0x7fb1080c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10759e700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb108158280 0x7fb1080c8520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb100009fd0 tx=0x7fb100018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb0feffd700 1 -- 172.21.15.7:0/3214662420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000032f0 con 0x7fb108158280 2024-01-11T07:08:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb0feffd700 1 -- 172.21.15.7:0/3214662420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb100008e70 con 0x7fb108158280 2024-01-11T07:08:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb0feffd700 1 -- 172.21.15.7:0/3214662420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100026690 con 0x7fb108158280 2024-01-11T07:08:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1080c4f40 con 0x7fb108158280 2024-01-11T07:08:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.724+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb108148960 con 0x7fb108158280 2024-01-11T07:08:38.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.725+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1080c5130 con 0x7fb108158280 2024-01-11T07:08:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.726+0000 7fb0feffd700 1 -- 172.21.15.7:0/3214662420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb100003040 con 0x7fb108158280 2024-01-11T07:08:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.726+0000 7fb0feffd700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0e8066f30 0x7fb0e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.726+0000 7fb0feffd700 1 -- 172.21.15.7:0/3214662420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb1000865a0 con 0x7fb108158280 2024-01-11T07:08:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.726+0000 7fb0feffd700 1 -- 172.21.15.7:0/3214662420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7fb0e806ea50 con 0x7fb108158280 2024-01-11T07:08:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.726+0000 7fb106d9d700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0e8066f30 0x7fb0e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:38.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.727+0000 7fb106d9d700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0e8066f30 0x7fb0e80693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0006fd0 tx=0x7fb0f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:38.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.731+0000 7fb0feffd700 1 -- 172.21.15.7:0/3214662420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb100053810 con 0x7fb108158280 2024-01-11T07:08:38.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.935+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb10804c3c0 con 0x7fb108158280 2024-01-11T07:08:38.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.936+0000 7fb0feffd700 1 -- 172.21.15.7:0/3214662420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+32355 (secure 0 0 0) 0x7fb10002b020 con 0x7fb108158280 2024-01-11T07:08:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:38.936 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 50, 2024-01-11T07:08:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:08:38.177756+0000", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:08:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:08:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:08:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:17.698588+0000", 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:08:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:18.590636+0000", 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:08:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:08:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:08:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:08:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:08:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:08:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:08:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:39.005 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.938+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0e8066f30 msgr2=0x7fb0e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.938+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0e8066f30 0x7fb0e80693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0006fd0 tx=0x7fb0f0008040 comp rx=0 tx=0).stop 2024-01-11T07:08:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb108158280 msgr2=0x7fb1080c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb108158280 0x7fb1080c8520 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb100009fd0 tx=0x7fb100018000 comp rx=0 tx=0).stop 2024-01-11T07:08:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 shutdown_connections 2024-01-11T07:08:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0e8066f30 0x7fb0e80693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb108158280 0x7fb1080c8520 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1080b0970 0x7fb108148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 --2- 172.21.15.7:0/3214662420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1080cafb0 0x7fb1080c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 >> 172.21.15.7:0/3214662420 conn(0x7fb108071100 msgr2=0x7fb108155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 shutdown_connections 2024-01-11T07:08:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:38.939+0000 7fb10d0e1700 1 -- 172.21.15.7:0/3214662420 wait complete. 2024-01-11T07:08:39.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.143+0000 7f292138a700 1 Processor -- start 2024-01-11T07:08:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.144+0000 7f292138a700 1 -- start start 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.144+0000 7f292138a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f291c1560d0 0x7f291c069e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.144+0000 7f292138a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0c7520 0x7f291c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.144+0000 7f292138a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291c0699a0 con 0x7f291c0cadf0 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.144+0000 7f292138a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291c0cc2f0 con 0x7f291c1560d0 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.144+0000 7f292138a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c068d90 con 0x7f291c0c7520 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f291c1560d0 0x7f291c069e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f291c1560d0 0x7f291c069e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48372/0 (socket says 172.21.15.7:48372) 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291affd700 1 -- 172.21.15.7:0/888602615 learned_addr learned my addr 172.21.15.7:0/888602615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291b7fe700 1 --2- 172.21.15.7:0/888602615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0c7520 0x7f291c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291affd700 1 -- 172.21.15.7:0/888602615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0c7520 msgr2=0x7f291c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291affd700 1 --2- 172.21.15.7:0/888602615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0c7520 0x7f291c0cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291affd700 1 -- 172.21.15.7:0/888602615 >> v1:172.21.15.113:6789/0 conn(0x7f291c0cadf0 legacy=0x7f291c0c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291affd700 1 -- 172.21.15.7:0/888602615 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c069070 con 0x7f291c1560d0 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f291affd700 1 --2- 172.21.15.7:0/888602615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c1560d0 0x7f291c069e30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f290400bc40 tx=0x7f290400b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8c15e4edb64b0c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f2919ffb700 1 -- 172.21.15.7:0/888602615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400db40 con 0x7f291c1560d0 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f2919ffb700 1 -- 172.21.15.7:0/888602615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290400dca0 con 0x7f291c1560d0 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.145+0000 7f2919ffb700 1 -- 172.21.15.7:0/888602615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904017640 con 0x7f291c1560d0 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.146+0000 7f292138a700 1 -- 172.21.15.7:0/888602615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c1560d0 msgr2=0x7f291c069e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.146+0000 7f292138a700 1 --2- 172.21.15.7:0/888602615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c1560d0 0x7f291c069e30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f290400bc40 tx=0x7f290400b4e0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.146+0000 7f292138a700 1 -- 172.21.15.7:0/888602615 shutdown_connections 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.146+0000 7f292138a700 1 --2- 172.21.15.7:0/888602615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0c7520 0x7f291c0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.146+0000 7f292138a700 1 --2- 172.21.15.7:0/888602615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c1560d0 0x7f291c069e30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.146+0000 7f292138a700 1 -- 172.21.15.7:0/888602615 >> 172.21.15.7:0/888602615 conn(0x7f291c063230 msgr2=0x7f291c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.146+0000 7f292138a700 1 -- 172.21.15.7:0/888602615 shutdown_connections 2024-01-11T07:08:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.146+0000 7f292138a700 1 -- 172.21.15.7:0/888602615 wait complete. 2024-01-11T07:08:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.148+0000 7f292138a700 1 Processor -- start 2024-01-11T07:08:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.148+0000 7f292138a700 1 -- start start 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f292138a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c0c7520 0x7f291c0c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f292138a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0cadf0 0x7f291c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f292138a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291c1560d0 0x7f291c14cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f292138a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0b0ca0 con 0x7f291c0c7520 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f292138a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0b0f80 con 0x7f291c1560d0 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f292138a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0b1260 con 0x7f291c0cadf0 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f291affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c0c7520 0x7f291c0c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f291affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c0c7520 0x7f291c0c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48384/0 (socket says 172.21.15.7:48384) 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f291affd700 1 -- 172.21.15.7:0/3851620992 learned_addr learned my addr 172.21.15.7:0/3851620992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:39.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f291b7fe700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291c1560d0 0x7f291c14cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f291a7fc700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0cadf0 0x7f291c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f291affd700 1 -- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0cadf0 msgr2=0x7f291c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f291affd700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0cadf0 0x7f291c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.149+0000 7f291affd700 1 -- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291c1560d0 msgr2=0x7f291c14cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.150+0000 7f291affd700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291c1560d0 0x7f291c14cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.150+0000 7f291affd700 1 -- 172.21.15.7:0/3851620992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290400b780 con 0x7f291c0c7520 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.150+0000 7f291affd700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c0c7520 0x7f291c0c53e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f291c06ac50 tx=0x7f2904019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.150+0000 7f2912ffd700 1 -- 172.21.15.7:0/3851620992 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904022610 con 0x7f291c0c7520 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.150+0000 7f2912ffd700 1 -- 172.21.15.7:0/3851620992 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2904022790 con 0x7f291c0c7520 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.150+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c0b1540 con 0x7f291c0c7520 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.150+0000 7f2912ffd700 1 -- 172.21.15.7:0/3851620992 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904022b50 con 0x7f291c0c7520 2024-01-11T07:08:39.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.150+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c0c5920 con 0x7f291c0c7520 2024-01-11T07:08:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.151+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c0b19e0 con 0x7f291c0c7520 2024-01-11T07:08:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.152+0000 7f2912ffd700 1 -- 172.21.15.7:0/3851620992 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2904015070 con 0x7f291c0c7520 2024-01-11T07:08:39.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.152+0000 7f2912ffd700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f28fc066f30 0x7f28fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.152+0000 7f2912ffd700 1 -- 172.21.15.7:0/3851620992 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2904088fd0 con 0x7f291c0c7520 2024-01-11T07:08:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.152+0000 7f2912ffd700 1 -- 172.21.15.7:0/3851620992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7f28fc06ea50 con 0x7f291c0c7520 2024-01-11T07:08:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.153+0000 7f291a7fc700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f28fc066f30 0x7f28fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.153+0000 7f291a7fc700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f28fc066f30 0x7f28fc0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f291c06e030 tx=0x7f290c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:39.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.156+0000 7f2912ffd700 1 -- 172.21.15.7:0/3851620992 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29040561a0 con 0x7f291c0c7520 2024-01-11T07:08:39.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.410+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f291c0b1b70 con 0x7f291c0c7520 2024-01-11T07:08:39.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.411+0000 7f2912ffd700 1 -- 172.21.15.7:0/3851620992 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f29040597c0 con 0x7f291c0c7520 2024-01-11T07:08:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:39.410 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f28fc066f30 msgr2=0x7f28fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f28fc066f30 0x7f28fc0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f291c06e030 tx=0x7f290c009040 comp rx=0 tx=0).stop 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c0c7520 msgr2=0x7f291c0c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c0c7520 0x7f291c0c53e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f291c06ac50 tx=0x7f2904019000 comp rx=0 tx=0).stop 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 shutdown_connections 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f28fc066f30 0x7f28fc0693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f291c1560d0 0x7f291c14cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f291c0c7520 0x7f291c0c53e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 --2- 172.21.15.7:0/3851620992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f291c0cadf0 0x7f291c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 >> 172.21.15.7:0/3851620992 conn(0x7f291c063230 msgr2=0x7f291c063600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 shutdown_connections 2024-01-11T07:08:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.413+0000 7f292138a700 1 -- 172.21.15.7:0/3851620992 wait complete. 2024-01-11T07:08:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:39.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:08:39.578 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":5310,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.384813207000001,"mdsmap_epoch":25,"osdmap_epoch":50,"osdmap_epoch_barrier":50,"uptime":24.640277966999999} 2024-01-11T07:08:39.578 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 5310, "mdsmap_epoch": 25, "osdmap_epoch": 50, "osdmap_epoch_barrier": 50, "rank_uptime": 19.384813207, "state": "up:active", "uptime": 24.640277967, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:08:39.578 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:08:39.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:39.579 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-01-11T07:08:39.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:08:39.598 INFO:teuthology.orchestra.run.smithi007.stderr:dd: failed to open 'small_file_1': Permission denied 2024-01-11T07:08:39.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:39.599 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-01-11T07:08:39.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:39.660 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-01-11T07:08:39.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:08:39.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.906+0000 7f161c599700 1 Processor -- start 2024-01-11T07:08:39.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.906+0000 7f161c599700 1 -- start start 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.907+0000 7f161c599700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161415a060 0x7f16140b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.907+0000 7f161c599700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1614069320 0x7f16140b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.907+0000 7f161c599700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16140ce9b0 con 0x7f1614159c90 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.907+0000 7f161c599700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16140b1f30 con 0x7f1614069320 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.907+0000 7f161c599700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16140cf530 con 0x7f161415a060 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.907+0000 7f161a335700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1614159c90 0x7f16140b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49444/0 (socket says 172.21.15.7:49444) 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.907+0000 7f161a335700 1 -- 172.21.15.7:0/3465234884 learned_addr learned my addr 172.21.15.7:0/3465234884 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.907+0000 7f161ab36700 1 --2- 172.21.15.7:0/3465234884 >> v2:172.21.15.7:3300/0 conn(0x7f1614069320 0x7f16140b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f1619b34700 1 --2- 172.21.15.7:0/3465234884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161415a060 0x7f16140b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f1619333700 1 -- 172.21.15.7:0/3465234884 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796827455 0 0) 0x7f16140ce9b0 con 0x7f1614159c90 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f1619333700 1 -- 172.21.15.7:0/3465234884 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1600003570 con 0x7f1614159c90 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f161ab36700 1 -- 172.21.15.7:0/3465234884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161415a060 msgr2=0x7f16140b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:39.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f161ab36700 1 --2- 172.21.15.7:0/3465234884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161415a060 0x7f16140b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f161ab36700 1 -- 172.21.15.7:0/3465234884 >> v1:172.21.15.7:6789/0 conn(0x7f1614159c90 legacy=0x7f16140b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f161ab36700 1 -- 172.21.15.7:0/3465234884 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140cf9c0 con 0x7f1614069320 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f1619b34700 1 --2- 172.21.15.7:0/3465234884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161415a060 0x7f16140b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f161ab36700 1 --2- 172.21.15.7:0/3465234884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1614069320 0x7f16140b0970 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f160c009e60 tx=0x7f160c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d235431806ff13 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f1619333700 1 -- 172.21.15.7:0/3465234884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c00e070 con 0x7f1614069320 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f1619333700 1 -- 172.21.15.7:0/3465234884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c00bb60 con 0x7f1614069320 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.908+0000 7f1619333700 1 -- 172.21.15.7:0/3465234884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c012430 con 0x7f1614069320 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.909+0000 7f161c599700 1 -- 172.21.15.7:0/3465234884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1614069320 msgr2=0x7f16140b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.909+0000 7f161c599700 1 --2- 172.21.15.7:0/3465234884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1614069320 0x7f16140b0970 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f160c009e60 tx=0x7f160c009780 comp rx=0 tx=0).stop 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.909+0000 7f161c599700 1 -- 172.21.15.7:0/3465234884 shutdown_connections 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.909+0000 7f161c599700 1 --2- 172.21.15.7:0/3465234884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1614069320 0x7f16140b0970 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.909+0000 7f161c599700 1 --2- 172.21.15.7:0/3465234884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f161415a060 0x7f16140b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.909+0000 7f161c599700 1 -- 172.21.15.7:0/3465234884 >> 172.21.15.7:0/3465234884 conn(0x7f1614071100 msgr2=0x7f161406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:39.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.909+0000 7f161c599700 1 -- 172.21.15.7:0/3465234884 shutdown_connections 2024-01-11T07:08:39.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.909+0000 7f161c599700 1 -- 172.21.15.7:0/3465234884 wait complete. 2024-01-11T07:08:39.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.910+0000 7f161c599700 1 Processor -- start 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.910+0000 7f161c599700 1 -- start start 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161c599700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1614069320 0x7f16140c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161c599700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1614159c90 0x7f161415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161c599700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161415a060 0x7f161415dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161c599700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161415ef90 con 0x7f161415a060 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161c599700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161415e110 con 0x7f1614069320 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161c599700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161415e310 con 0x7f1614159c90 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161ab36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161415a060 0x7f161415dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161ab36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161415a060 0x7f161415dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48408/0 (socket says 172.21.15.7:48408) 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161ab36700 1 -- 172.21.15.7:0/1870073370 learned_addr learned my addr 172.21.15.7:0/1870073370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161a335700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1614069320 0x7f16140c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f1619b34700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1614159c90 0x7f161415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.911+0000 7f161ab36700 1 -- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1614159c90 msgr2=0x7f161415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f161ab36700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1614159c90 0x7f161415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f161ab36700 1 -- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1614069320 msgr2=0x7f16140c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f161ab36700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1614069320 0x7f16140c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f161ab36700 1 -- 172.21.15.7:0/1870073370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c009a20 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f161a335700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1614069320 0x7f16140c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f161ab36700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161415a060 0x7f161415dbd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f160c013000 tx=0x7f160c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f16067fc700 1 -- 172.21.15.7:0/1870073370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c0032f0 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f16067fc700 1 -- 172.21.15.7:0/1870073370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c027470 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161415e5f0 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f16067fc700 1 -- 172.21.15.7:0/1870073370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c027810 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.912+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161415ebb0 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.913+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161404c3c0 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.913+0000 7f16067fc700 1 -- 172.21.15.7:0/1870073370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f160c024070 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.914+0000 7f16067fc700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15f8066f30 0x7f15f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.914+0000 7f16067fc700 1 -- 172.21.15.7:0/1870073370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f160c086ef0 con 0x7f161415a060 2024-01-11T07:08:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.914+0000 7f161a335700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15f8066f30 0x7f15f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:39.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.914+0000 7f16067fc700 1 -- 172.21.15.7:0/1870073370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7f15f806ea50 con 0x7f161415a060 2024-01-11T07:08:39.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.915+0000 7f161a335700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15f8066f30 0x7f15f80693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f16080058d0 tx=0x7f1608009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:39.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:39.919+0000 7f16067fc700 1 -- 172.21.15.7:0/1870073370 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f160c054210 con 0x7f161415a060 2024-01-11T07:08:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.125+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f161406f510 con 0x7f161415a060 2024-01-11T07:08:40.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.126+0000 7f16067fc700 1 -- 172.21.15.7:0/1870073370 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+32355 (secure 0 0 0) 0x7f160c02c020 con 0x7f161415a060 2024-01-11T07:08:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:40.126 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:40.126 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 50, 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:08:38.177756+0000", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:40.127 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:08:40.128 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:40.129 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:40.130 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:40.131 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:40.132 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:40.133 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:17.698588+0000", 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:40.134 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:40.135 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:40.136 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:40.137 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:40.138 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:18.590636+0000", 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:40.140 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:40.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:40.142 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.143 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:08:40.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:40.145 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:08:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:08:40.152 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:08:40.155 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:08:40.159 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:08:40.160 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:08:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:08:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:08:40.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:40.192 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:40.193 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:40.194 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:08:40.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:40.196 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15f8066f30 msgr2=0x7f15f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15f8066f30 0x7f15f80693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f16080058d0 tx=0x7f1608009040 comp rx=0 tx=0).stop 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161415a060 msgr2=0x7f161415dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161415a060 0x7f161415dbd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f160c013000 tx=0x7f160c019000 comp rx=0 tx=0).stop 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 shutdown_connections 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15f8066f30 0x7f15f80693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161415a060 0x7f161415dbd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1614069320 0x7f16140c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 --2- 172.21.15.7:0/1870073370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1614159c90 0x7f161415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:40.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 >> 172.21.15.7:0/1870073370 conn(0x7f1614071100 msgr2=0x7f161415c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:40.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 shutdown_connections 2024-01-11T07:08:40.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:40.129+0000 7f161c599700 1 -- 172.21.15.7:0/1870073370 wait complete. 2024-01-11T07:08:40.199 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:08:42.175 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:08:42.175+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-01-11T07:08:45.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:08:45.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.350+0000 7fed5e158700 1 Processor -- start 2024-01-11T07:08:45.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.350+0000 7fed5e158700 1 -- start start 2024-01-11T07:08:45.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.351+0000 7fed5e158700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed58157f20 0x7fed580cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.351+0000 7fed5e158700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fed5815b4b0 0x7fed580b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.351+0000 7fed5e158700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5806a580 con 0x7fed580cce70 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.351+0000 7fed5e158700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5806a860 con 0x7fed58157f20 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.351+0000 7fed5e158700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5806a390 con 0x7fed5815b4b0 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.351+0000 7fed56ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed58157f20 0x7fed580cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.351+0000 7fed56ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed58157f20 0x7fed580cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48424/0 (socket says 172.21.15.7:48424) 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.351+0000 7fed56ffd700 1 -- 172.21.15.7:0/358082089 learned_addr learned my addr 172.21.15.7:0/358082089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed57fff700 1 --2- 172.21.15.7:0/358082089 >> v2:172.21.15.113:3300/0 conn(0x7fed5815b4b0 0x7fed580b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed56ffd700 1 -- 172.21.15.7:0/358082089 >> v2:172.21.15.113:3300/0 conn(0x7fed5815b4b0 msgr2=0x7fed580b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed56ffd700 1 --2- 172.21.15.7:0/358082089 >> v2:172.21.15.113:3300/0 conn(0x7fed5815b4b0 0x7fed580b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed56ffd700 1 -- 172.21.15.7:0/358082089 >> v1:172.21.15.113:6789/0 conn(0x7fed580cce70 legacy=0x7fed58069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed56ffd700 1 -- 172.21.15.7:0/358082089 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5815a9d0 con 0x7fed58157f20 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed56ffd700 1 --2- 172.21.15.7:0/358082089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed58157f20 0x7fed580cd240 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed580b24a0 tx=0x7fed40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbcc0804f152ca4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:45.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed567fc700 1 -- 172.21.15.7:0/358082089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4000b820 con 0x7fed58157f20 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed567fc700 1 -- 172.21.15.7:0/358082089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4000b980 con 0x7fed58157f20 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.352+0000 7fed567fc700 1 -- 172.21.15.7:0/358082089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4000bd20 con 0x7fed58157f20 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.353+0000 7fed5e158700 1 -- 172.21.15.7:0/358082089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed58157f20 msgr2=0x7fed580cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.353+0000 7fed5e158700 1 --2- 172.21.15.7:0/358082089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed58157f20 0x7fed580cd240 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed580b24a0 tx=0x7fed40009250 comp rx=0 tx=0).stop 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.353+0000 7fed5e158700 1 -- 172.21.15.7:0/358082089 shutdown_connections 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.353+0000 7fed5e158700 1 --2- 172.21.15.7:0/358082089 >> v2:172.21.15.113:3300/0 conn(0x7fed5815b4b0 0x7fed580b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.353+0000 7fed5e158700 1 --2- 172.21.15.7:0/358082089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed58157f20 0x7fed580cd240 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.353+0000 7fed5e158700 1 -- 172.21.15.7:0/358082089 >> 172.21.15.7:0/358082089 conn(0x7fed580cd910 msgr2=0x7fed580cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.353+0000 7fed5e158700 1 -- 172.21.15.7:0/358082089 shutdown_connections 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.353+0000 7fed5e158700 1 -- 172.21.15.7:0/358082089 wait complete. 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.354+0000 7fed5e158700 1 Processor -- start 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.354+0000 7fed5e158700 1 -- start start 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed5e158700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed580cce70 0x7fed5815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed5e158700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed58157f20 0x7fed5816b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed5e158700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed5815b4b0 0x7fed5806b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed5e158700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed581592e0 con 0x7fed580cce70 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed5e158700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5815c4b0 con 0x7fed58157f20 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed5e158700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed581586d0 con 0x7fed5815b4b0 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed580cce70 0x7fed5815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed580cce70 0x7fed5815d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48426/0 (socket says 172.21.15.7:48426) 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed577fe700 1 -- 172.21.15.7:0/853269181 learned_addr learned my addr 172.21.15.7:0/853269181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed56ffd700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed58157f20 0x7fed5816b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.355+0000 7fed57fff700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed5815b4b0 0x7fed5806b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed577fe700 1 -- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed5815b4b0 msgr2=0x7fed5806b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed577fe700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed5815b4b0 0x7fed5806b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed577fe700 1 -- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed58157f20 msgr2=0x7fed5816b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed577fe700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed58157f20 0x7fed5816b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed577fe700 1 -- 172.21.15.7:0/853269181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed400094f0 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed577fe700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed580cce70 0x7fed5815d2a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed4800c5d0 tx=0x7fed48007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed4f7fe700 1 -- 172.21.15.7:0/853269181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48011590 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed4f7fe700 1 -- 172.21.15.7:0/853269181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed48011710 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed581589b0 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed4f7fe700 1 -- 172.21.15.7:0/853269181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48011ad0 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.356+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed58158f70 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.357+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed58064a40 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.358+0000 7fed4f7fe700 1 -- 172.21.15.7:0/853269181 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fed4801c460 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.358+0000 7fed4f7fe700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fed38066f30 0x7fed380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.358+0000 7fed4f7fe700 1 -- 172.21.15.7:0/853269181 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fed480841f0 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.358+0000 7fed56ffd700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fed38066f30 0x7fed380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.358+0000 7fed4f7fe700 1 -- 172.21.15.7:0/853269181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7fed3806ea50 con 0x7fed580cce70 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.359+0000 7fed56ffd700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fed38066f30 0x7fed380693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed580b24a0 tx=0x7fed40016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.363+0000 7fed4f7fe700 1 -- 172.21.15.7:0/853269181 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed48051460 con 0x7fed580cce70 2024-01-11T07:08:45.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.570+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fed5804c3c0 con 0x7fed580cce70 2024-01-11T07:08:45.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.571+0000 7fed4f7fe700 1 -- 172.21.15.7:0/853269181 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+32355 (secure 0 0 0) 0x7fed48054a80 con 0x7fed580cce70 2024-01-11T07:08:45.571 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:45.571 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:45.571 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 50, 2024-01-11T07:08:45.571 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:08:45.571 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:08:45.571 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:08:38.177756+0000", 2024-01-11T07:08:45.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:08:45.571 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:08:45.572 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:08:45.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:08:45.574 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:45.575 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:45.576 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:45.577 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.578 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:17.698588+0000", 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:45.579 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:45.580 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:45.581 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:45.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:45.583 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:18.590636+0000", 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:45.585 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:45.586 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "48", 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:45.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:45.588 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:45.589 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:45.591 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:45.596 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:45.597 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.598 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.599 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.600 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.601 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.602 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:45.603 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.604 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:08:45.605 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.608 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:08:45.608 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:45.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.608 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:08:45.609 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:45.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.609 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:08:45.609 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:08:45.609 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:08:45.610 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:08:45.610 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:45.610 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:45.610 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:45.610 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:45.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.611 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:45.611 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:45.611 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:45.611 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:45.612 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:45.612 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:45.612 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:45.612 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:45.612 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:45.612 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:45.612 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:45.613 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:45.613 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:45.613 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.613 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:08:45.614 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:45.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.614 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:08:45.614 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:45.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.615 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.615 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:45.615 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.616 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:08:45.616 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:45.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.616 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:08:45.617 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:45.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.617 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:45.617 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.618 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:08:45.618 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:45.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.620 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:45.621 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.622 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.624 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:45.624 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.624 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:08:45.624 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:45.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.625 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:08:45.625 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:45.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.625 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:45.625 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:08:45.626 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:45.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.627 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.627 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:45.627 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.627 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:08:45.628 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:45.629 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:08:45.630 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:08:45.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:08:45.632 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:08:45.633 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:45.634 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:45.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.639 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:45.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.645 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.646 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:45.647 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:45.650 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:45.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:45.652 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:08:45.653 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:08:45.654 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:08:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:08:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:08:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:08:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:08:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:08:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:45.655 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fed38066f30 msgr2=0x7fed380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fed38066f30 0x7fed380693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fed580b24a0 tx=0x7fed40016000 comp rx=0 tx=0).stop 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed580cce70 msgr2=0x7fed5815d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed580cce70 0x7fed5815d2a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed4800c5d0 tx=0x7fed48007f60 comp rx=0 tx=0).stop 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 shutdown_connections 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fed38066f30 0x7fed380693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fed5815b4b0 0x7fed5806b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed580cce70 0x7fed5815d2a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 --2- 172.21.15.7:0/853269181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed58157f20 0x7fed5816b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 >> 172.21.15.7:0/853269181 conn(0x7fed580cd910 msgr2=0x7fed5814f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 shutdown_connections 2024-01-11T07:08:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:45.573+0000 7fed5e158700 1 -- 172.21.15.7:0/853269181 wait complete. 2024-01-11T07:08:45.658 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:08:47.199 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:08:47.200+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-01-11T07:08:50.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:08:50.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.801+0000 7f428be2c700 1 Processor -- start 2024-01-11T07:08:50.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.801+0000 7f428be2c700 1 -- start start 2024-01-11T07:08:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f428be2c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42840cafb0 0x7f42840ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f428be2c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 0x7f42840b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f428be2c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4284159300 con 0x7f42840b0970 2024-01-11T07:08:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f428be2c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42840b0df0 con 0x7f42840cafb0 2024-01-11T07:08:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f428be2c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42840cf260 con 0x7f4284158280 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f4289bc8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42840cafb0 0x7f42840ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f428a3c9700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f42840b0970 0x7f4284159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49108/0 (socket says 172.21.15.7:49108) 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f428a3c9700 1 -- 172.21.15.7:0/777427572 learned_addr learned my addr 172.21.15.7:0/777427572 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.802+0000 7f42893c7700 1 --2- 172.21.15.7:0/777427572 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 0x7f42840b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4288bc6700 1 -- 172.21.15.7:0/777427572 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115752543 0 0) 0x7f4284159300 con 0x7f42840b0970 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4288bc6700 1 -- 172.21.15.7:0/777427572 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4270003570 con 0x7f42840b0970 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4289bc8700 1 -- 172.21.15.7:0/777427572 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 msgr2=0x7f42840b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4289bc8700 1 --2- 172.21.15.7:0/777427572 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 0x7f42840b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4289bc8700 1 -- 172.21.15.7:0/777427572 >> v1:172.21.15.7:6789/0 conn(0x7f42840b0970 legacy=0x7f4284159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4289bc8700 1 -- 172.21.15.7:0/777427572 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42840cf540 con 0x7f42840cafb0 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4289bc8700 1 --2- 172.21.15.7:0/777427572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840cafb0 0x7f42840ce8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f42800098e0 tx=0x7f42800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bab246d9353cf0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4288bc6700 1 -- 172.21.15.7:0/777427572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280011630 con 0x7f42840cafb0 2024-01-11T07:08:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4288bc6700 1 -- 172.21.15.7:0/777427572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280011790 con 0x7f42840cafb0 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.803+0000 7f4288bc6700 1 -- 172.21.15.7:0/777427572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280011b30 con 0x7f42840cafb0 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.804+0000 7f428be2c700 1 -- 172.21.15.7:0/777427572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840cafb0 msgr2=0x7f42840ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.804+0000 7f428be2c700 1 --2- 172.21.15.7:0/777427572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840cafb0 0x7f42840ce8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f42800098e0 tx=0x7f42800091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.804+0000 7f428be2c700 1 -- 172.21.15.7:0/777427572 shutdown_connections 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.804+0000 7f428be2c700 1 --2- 172.21.15.7:0/777427572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840cafb0 0x7f42840ce8f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.804+0000 7f428be2c700 1 --2- 172.21.15.7:0/777427572 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 0x7f42840b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.804+0000 7f428be2c700 1 -- 172.21.15.7:0/777427572 >> 172.21.15.7:0/777427572 conn(0x7f4284071100 msgr2=0x7f428406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.804+0000 7f428be2c700 1 -- 172.21.15.7:0/777427572 shutdown_connections 2024-01-11T07:08:50.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.804+0000 7f428be2c700 1 -- 172.21.15.7:0/777427572 wait complete. 2024-01-11T07:08:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.805+0000 7f428be2c700 1 Processor -- start 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.805+0000 7f428be2c700 1 -- start start 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f428be2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840b0970 0x7f42840ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f428be2c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42840cafb0 0x7f4284148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f428be2c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 0x7f42840af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f428be2c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428416a400 con 0x7f42840b0970 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f428be2c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4284157fc0 con 0x7f42840cafb0 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f428be2c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42840c88e0 con 0x7f4284158280 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840b0970 0x7f42840ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840b0970 0x7f42840ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51022/0 (socket says 172.21.15.7:51022) 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 -- 172.21.15.7:0/577151419 learned_addr learned my addr 172.21.15.7:0/577151419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f42893c7700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42840cafb0 0x7f4284148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f428a3c9700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 0x7f42840af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 -- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 msgr2=0x7f42840af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 0x7f42840af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 -- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42840cafb0 msgr2=0x7f4284148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42840cafb0 0x7f4284148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 -- 172.21.15.7:0/577151419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4280009420 con 0x7f42840b0970 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f42893c7700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42840cafb0 0x7f4284148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.806+0000 7f4289bc8700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840b0970 0x7f42840ca050 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f428406a1d0 tx=0x7f4280017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.807+0000 7f4279ffb700 1 -- 172.21.15.7:0/577151419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280011b10 con 0x7f42840b0970 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.807+0000 7f4279ffb700 1 -- 172.21.15.7:0/577151419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42800253f0 con 0x7f42840b0970 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.807+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42840c8bc0 con 0x7f42840b0970 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.807+0000 7f4279ffb700 1 -- 172.21.15.7:0/577151419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280025790 con 0x7f42840b0970 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.807+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42840c9180 con 0x7f42840b0970 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.808+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428416a6e0 con 0x7f42840b0970 2024-01-11T07:08:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.808+0000 7f4279ffb700 1 -- 172.21.15.7:0/577151419 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4280025b80 con 0x7f42840b0970 2024-01-11T07:08:50.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.809+0000 7f4279ffb700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4268066f30 0x7f42680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:50.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.809+0000 7f4279ffb700 1 -- 172.21.15.7:0/577151419 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4280085e60 con 0x7f42840b0970 2024-01-11T07:08:50.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.809+0000 7f42893c7700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4268066f30 0x7f42680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:50.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.810+0000 7f42893c7700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4268066f30 0x7f42680693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4274006f00 tx=0x7f4274008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:50.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:50.814+0000 7f4279ffb700 1 -- 172.21.15.7:0/577151419 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4280052ff0 con 0x7f42840b0970 2024-01-11T07:08:51.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.020+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f428404c3c0 con 0x7f42840b0970 2024-01-11T07:08:51.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.021+0000 7f4279ffb700 1 -- 172.21.15.7:0/577151419 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+32587 (secure 0 0 0) 0x7f4280056610 con 0x7f42840b0970 2024-01-11T07:08:51.021 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:51.021 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:51.021 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 53, 2024-01-11T07:08:51.021 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:08:49.209265+0000", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.022 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:08:51.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:08:51.024 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "52", 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:51.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:51.026 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:51.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:51.028 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:17.698588+0000", 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:51.029 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:51.030 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "52", 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:51.031 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:51.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:51.033 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-11T07:08:51.034 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:18.590636+0000", 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:08:51.035 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "52", 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:51.036 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:51.037 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:51.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:51.039 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:51.040 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:08:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.046 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:08:51.047 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:08:51.048 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:08:51.049 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:08:51.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.052 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:08:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:08:51.060 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:08:51.061 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.062 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.063 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 52, 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:08:51.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.069 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:08:51.073 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.074 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.080 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:08:51.084 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.085 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.088 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.089 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.090 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "7.16", 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-11T07:08:51.091 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "7.1b", 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 4 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.092 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:08:51.093 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.094 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4268066f30 msgr2=0x7f42680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4268066f30 0x7f42680693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4274006f00 tx=0x7f4274008040 comp rx=0 tx=0).stop 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840b0970 msgr2=0x7f42840ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840b0970 0x7f42840ca050 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f428406a1d0 tx=0x7f4280017000 comp rx=0 tx=0).stop 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 shutdown_connections 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4268066f30 0x7f42680693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4284158280 0x7f42840af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42840b0970 0x7f42840ca050 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 --2- 172.21.15.7:0/577151419 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42840cafb0 0x7f4284148380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 >> 172.21.15.7:0/577151419 conn(0x7f4284071100 msgr2=0x7f42840ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 shutdown_connections 2024-01-11T07:08:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.024+0000 7f428be2c700 1 -- 172.21.15.7:0/577151419 wait complete. 2024-01-11T07:08:51.097 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-01-11T07:08:51.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:08:51.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.235+0000 7f466e480700 1 Processor -- start 2024-01-11T07:08:51.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.236+0000 7f466e480700 1 -- start start 2024-01-11T07:08:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.236+0000 7f466e480700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680cafb0 0x7f46680ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.236+0000 7f466e480700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4668158280 0x7f46680b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.236+0000 7f466e480700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4668159300 con 0x7f46680b0970 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.236+0000 7f466e480700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46680b0df0 con 0x7f4668158280 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.236+0000 7f466e480700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46680cf260 con 0x7f46680cafb0 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f46677fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4668158280 0x7f46680b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f46677fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4668158280 0x7f46680b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51028/0 (socket says 172.21.15.7:51028) 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f46677fe700 1 -- 172.21.15.7:0/2501428552 learned_addr learned my addr 172.21.15.7:0/2501428552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f4667fff700 1 --2- 172.21.15.7:0/2501428552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680cafb0 0x7f46680ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f46677fe700 1 -- 172.21.15.7:0/2501428552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680cafb0 msgr2=0x7f46680ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f46677fe700 1 --2- 172.21.15.7:0/2501428552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680cafb0 0x7f46680ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f46677fe700 1 -- 172.21.15.7:0/2501428552 >> v1:172.21.15.113:6789/0 conn(0x7f46680b0970 legacy=0x7f4668159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f46677fe700 1 -- 172.21.15.7:0/2501428552 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46680cf540 con 0x7f4668158280 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f46677fe700 1 --2- 172.21.15.7:0/2501428552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680b0430 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f465c0099b0 tx=0x7f465c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388a6004a7230dbe server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f4666ffd700 1 -- 172.21.15.7:0/2501428552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c009ea0 con 0x7f4668158280 2024-01-11T07:08:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f4666ffd700 1 -- 172.21.15.7:0/2501428552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c00b860 con 0x7f4668158280 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.237+0000 7f4666ffd700 1 -- 172.21.15.7:0/2501428552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c00bc00 con 0x7f4668158280 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.238+0000 7f466e480700 1 -- 172.21.15.7:0/2501428552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 msgr2=0x7f46680b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.238+0000 7f466e480700 1 --2- 172.21.15.7:0/2501428552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680b0430 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f465c0099b0 tx=0x7f465c009250 comp rx=0 tx=0).stop 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.238+0000 7f466e480700 1 -- 172.21.15.7:0/2501428552 shutdown_connections 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.238+0000 7f466e480700 1 --2- 172.21.15.7:0/2501428552 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680cafb0 0x7f46680ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.238+0000 7f466e480700 1 --2- 172.21.15.7:0/2501428552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680b0430 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.238+0000 7f466e480700 1 -- 172.21.15.7:0/2501428552 >> 172.21.15.7:0/2501428552 conn(0x7f4668071100 msgr2=0x7f466806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.238+0000 7f466e480700 1 -- 172.21.15.7:0/2501428552 shutdown_connections 2024-01-11T07:08:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.238+0000 7f466e480700 1 -- 172.21.15.7:0/2501428552 wait complete. 2024-01-11T07:08:51.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.239+0000 7f466e480700 1 Processor -- start 2024-01-11T07:08:51.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.239+0000 7f466e480700 1 -- start start 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466e480700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680b0970 0x7f46680b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466e480700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46680cafb0 0x7f46681483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466e480700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466e480700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46680b4220 con 0x7f4668158280 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466e480700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46680b2750 con 0x7f46680cafb0 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466e480700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46680b3610 con 0x7f46680b0970 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466ca1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466ca1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51036/0 (socket says 172.21.15.7:51036) 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f466ca1d700 1 -- 172.21.15.7:0/1535449662 learned_addr learned my addr 172.21.15.7:0/1535449662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f46677fe700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46680cafb0 0x7f46681483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.240+0000 7f4667fff700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680b0970 0x7f46680b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f466ca1d700 1 -- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680b0970 msgr2=0x7f46680b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f466ca1d700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680b0970 0x7f46680b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f466ca1d700 1 -- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46680cafb0 msgr2=0x7f46681483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f466ca1d700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46680cafb0 0x7f46681483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f466ca1d700 1 -- 172.21.15.7:0/1535449662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0094f0 con 0x7f4668158280 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f466ca1d700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680af2c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f466000c8e0 tx=0x7f4660001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f464ffff700 1 -- 172.21.15.7:0/1535449662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660012630 con 0x7f4668158280 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f464ffff700 1 -- 172.21.15.7:0/1535449662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660012790 con 0x7f4668158280 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46680b38f0 con 0x7f4668158280 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f464ffff700 1 -- 172.21.15.7:0/1535449662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660012b50 con 0x7f4668158280 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.241+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46680b3eb0 con 0x7f4668158280 2024-01-11T07:08:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.242+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4668167b70 con 0x7f4668158280 2024-01-11T07:08:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.243+0000 7f464ffff700 1 -- 172.21.15.7:0/1535449662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f466001a3a0 con 0x7f4668158280 2024-01-11T07:08:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.243+0000 7f464ffff700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4654066f30 0x7f46540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.243+0000 7f464ffff700 1 -- 172.21.15.7:0/1535449662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f46600846e0 con 0x7f4668158280 2024-01-11T07:08:51.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.243+0000 7f4667fff700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4654066f30 0x7f46540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.244+0000 7f4667fff700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4654066f30 0x7f46540693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4658009db0 tx=0x7f46580091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:51.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.247+0000 7f464ffff700 1 -- 172.21.15.7:0/1535449662 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4668167b70 con 0x7f4668158280 2024-01-11T07:08:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.451+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4668167b70 con 0x7f4668158280 2024-01-11T07:08:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.452+0000 7f464ffff700 1 -- 172.21.15.7:0/1535449662 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+32587 (secure 0 0 0) 0x7f4668167b70 con 0x7f4668158280 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 53, 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:08:49.209265+0000", 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:08:51.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:08:51.453 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 7, 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:51.454 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:51.455 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "52", 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:51.456 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:51.457 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:51.458 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 6, 2024-01-11T07:08:51.459 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:17.698588+0000", 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:51.460 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.461 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "52", 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:51.462 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:51.463 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:51.464 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:51.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:51.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:51.464 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:51.464 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:51.464 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:51.464 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:51.464 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:08:18.590636+0000", 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:08:51.466 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:08:51.467 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "52", 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:08:51.468 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:08:51.469 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:08:51.470 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:08:51.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.472 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:08:51.476 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.477 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.483 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.486 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:08:51.489 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.491 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:08:51.495 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:08:51.496 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.497 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 52, 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:08:51.499 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.502 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:51.503 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.504 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.509 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.510 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.514 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 45, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:08:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.519 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:08:51.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:08:51.522 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:08:51.523 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.524 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.525 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.526 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.528 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:08:51.529 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "7.16", 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-01-11T07:08:51.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "7.1b", 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 0, 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 4 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.531 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:08:51.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:08:51.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:08:51.534 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4654066f30 msgr2=0x7f46540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4654066f30 0x7f46540693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4658009db0 tx=0x7f46580091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 msgr2=0x7f46680af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680af2c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f466000c8e0 tx=0x7f4660001710 comp rx=0 tx=0).stop 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 shutdown_connections 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4654066f30 0x7f46540693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4668158280 0x7f46680af2c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46680b0970 0x7f46680b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 --2- 172.21.15.7:0/1535449662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46680cafb0 0x7f46681483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 >> 172.21.15.7:0/1535449662 conn(0x7f4668071100 msgr2=0x7f46680cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 shutdown_connections 2024-01-11T07:08:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.454+0000 7f466e480700 1 -- 172.21.15.7:0/1535449662 wait complete. 2024-01-11T07:08:51.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:51.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.672+0000 7fda09333700 1 Processor -- start 2024-01-11T07:08:51.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.672+0000 7fda09333700 1 -- start start 2024-01-11T07:08:51.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.673+0000 7fda09333700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fda0415a060 0x7fda040b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.673+0000 7fda09333700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda04069320 0x7fda040b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.673+0000 7fda09333700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda040ce9b0 con 0x7fda04159c90 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.673+0000 7fda09333700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040b1f30 con 0x7fda04069320 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.673+0000 7fda09333700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040cf530 con 0x7fda0415a060 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.673+0000 7fda037fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda04069320 0x7fda040b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda037fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda04069320 0x7fda040b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51040/0 (socket says 172.21.15.7:51040) 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda037fe700 1 -- 172.21.15.7:0/272747047 learned_addr learned my addr 172.21.15.7:0/272747047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda027fc700 1 --2- 172.21.15.7:0/272747047 >> v2:172.21.15.113:3300/0 conn(0x7fda0415a060 0x7fda040b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda037fe700 1 -- 172.21.15.7:0/272747047 >> v2:172.21.15.113:3300/0 conn(0x7fda0415a060 msgr2=0x7fda040b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda037fe700 1 --2- 172.21.15.7:0/272747047 >> v2:172.21.15.113:3300/0 conn(0x7fda0415a060 0x7fda040b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda037fe700 1 -- 172.21.15.7:0/272747047 >> v1:172.21.15.113:6789/0 conn(0x7fda04159c90 legacy=0x7fda040b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda037fe700 1 -- 172.21.15.7:0/272747047 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040cf9c0 con 0x7fda04069320 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda037fe700 1 --2- 172.21.15.7:0/272747047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04069320 0x7fda040b0970 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fda040699e0 tx=0x7fd9fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ff4a575ed85723 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda01ffb700 1 -- 172.21.15.7:0/272747047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00e070 con 0x7fda04069320 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda01ffb700 1 -- 172.21.15.7:0/272747047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc00ba50 con 0x7fda04069320 2024-01-11T07:08:51.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.674+0000 7fda01ffb700 1 -- 172.21.15.7:0/272747047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0092e0 con 0x7fda04069320 2024-01-11T07:08:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.675+0000 7fda09333700 1 -- 172.21.15.7:0/272747047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04069320 msgr2=0x7fda040b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.675+0000 7fda09333700 1 --2- 172.21.15.7:0/272747047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04069320 0x7fda040b0970 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fda040699e0 tx=0x7fd9fc009840 comp rx=0 tx=0).stop 2024-01-11T07:08:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.675+0000 7fda09333700 1 -- 172.21.15.7:0/272747047 shutdown_connections 2024-01-11T07:08:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.675+0000 7fda09333700 1 --2- 172.21.15.7:0/272747047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04069320 0x7fda040b0970 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.675+0000 7fda09333700 1 --2- 172.21.15.7:0/272747047 >> v2:172.21.15.113:3300/0 conn(0x7fda0415a060 0x7fda040b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.675+0000 7fda09333700 1 -- 172.21.15.7:0/272747047 >> 172.21.15.7:0/272747047 conn(0x7fda04071100 msgr2=0x7fda0406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.675+0000 7fda09333700 1 -- 172.21.15.7:0/272747047 shutdown_connections 2024-01-11T07:08:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.675+0000 7fda09333700 1 -- 172.21.15.7:0/272747047 wait complete. 2024-01-11T07:08:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.677+0000 7fda09333700 1 Processor -- start 2024-01-11T07:08:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.677+0000 7fda09333700 1 -- start start 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda09333700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda04069320 0x7fda0413fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda09333700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04159c90 0x7fda0415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda09333700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda0415a060 0x7fda0416d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda09333700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04155b30 con 0x7fda04159c90 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda09333700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0416dea0 con 0x7fda0415a060 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda09333700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0416e180 con 0x7fda04069320 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04159c90 0x7fda0415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04159c90 0x7fda0415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51056/0 (socket says 172.21.15.7:51056) 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 -- 172.21.15.7:0/1177148019 learned_addr learned my addr 172.21.15.7:0/1177148019 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda02ffd700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda04069320 0x7fda0413fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda037fe700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda0415a060 0x7fda0416d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 -- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda04069320 msgr2=0x7fda0413fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda04069320 0x7fda0413fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 -- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda0415a060 msgr2=0x7fda0416d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda0415a060 0x7fda0416d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 -- 172.21.15.7:0/1177148019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc009ae0 con 0x7fda04159c90 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.678+0000 7fda027fc700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04159c90 0x7fda0415b050 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400d890 tx=0x7fd9f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.679+0000 7fd9faffd700 1 -- 172.21.15.7:0/1177148019 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40125f0 con 0x7fda04159c90 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.679+0000 7fd9faffd700 1 -- 172.21.15.7:0/1177148019 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4012750 con 0x7fda04159c90 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.679+0000 7fd9faffd700 1 -- 172.21.15.7:0/1177148019 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4012af0 con 0x7fda04159c90 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.679+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0415a850 con 0x7fda04159c90 2024-01-11T07:08:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.679+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0415b590 con 0x7fda04159c90 2024-01-11T07:08:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.680+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0415acf0 con 0x7fda04159c90 2024-01-11T07:08:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.680+0000 7fd9faffd700 1 -- 172.21.15.7:0/1177148019 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd9f401e460 con 0x7fda04159c90 2024-01-11T07:08:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.681+0000 7fd9faffd700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9e4066f30 0x7fd9e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.681+0000 7fd9faffd700 1 -- 172.21.15.7:0/1177148019 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fd9f4084b40 con 0x7fda04159c90 2024-01-11T07:08:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.681+0000 7fda02ffd700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9e4066f30 0x7fd9e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.681+0000 7fda02ffd700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9e4066f30 0x7fd9e40693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec003a80 tx=0x7fd9ec007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:51.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.685+0000 7fd9faffd700 1 -- 172.21.15.7:0/1177148019 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9f4051cd0 con 0x7fda04159c90 2024-01-11T07:08:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.934+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda0404c3c0 con 0x7fda04159c90 2024-01-11T07:08:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.934+0000 7fd9faffd700 1 -- 172.21.15.7:0/1177148019 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7fd9f40552f0 con 0x7fda04159c90 2024-01-11T07:08:51.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:51.934 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9e4066f30 msgr2=0x7fd9e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9e4066f30 0x7fd9e40693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec003a80 tx=0x7fd9ec007ec0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04159c90 msgr2=0x7fda0415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04159c90 0x7fda0415b050 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400d890 tx=0x7fd9f4008040 comp rx=0 tx=0).stop 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 shutdown_connections 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9e4066f30 0x7fd9e40693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda0415a060 0x7fda0416d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fda04069320 0x7fda0413fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 --2- 172.21.15.7:0/1177148019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04159c90 0x7fda0415b050 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 >> 172.21.15.7:0/1177148019 conn(0x7fda04071100 msgr2=0x7fda04157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 shutdown_connections 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:51.936+0000 7fda09333700 1 -- 172.21.15.7:0/1177148019 wait complete. 2024-01-11T07:08:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:51.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:08:52.100 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":5310,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.907543450999999,"mdsmap_epoch":25,"osdmap_epoch":53,"osdmap_epoch_barrier":50,"uptime":37.163013831999997} 2024-01-11T07:08:52.101 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 5310, "mdsmap_epoch": 25, "osdmap_epoch": 53, "osdmap_epoch_barrier": 50, "rank_uptime": 31.907543451, "state": "up:active", "uptime": 37.163013832, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:08:52.101 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:08:52.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:52.101 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-01-11T07:08:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-11T07:08:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-11T07:08:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.299572 s, 175 MB/s 2024-01-11T07:08:52.422 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:08:52.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:08:52.466 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:08:52.512 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:08:52.512 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:08:52.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:52.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:08:52.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:08:52.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:52.596 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:08:52.621 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:08:52.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:08:52.622 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:08:52.661 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:08:52.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:08:52.661 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:08:52.719 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:52.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.859+0000 7f6075388700 1 Processor -- start 2024-01-11T07:08:52.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.859+0000 7f6075388700 1 -- start start 2024-01-11T07:08:52.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f6075388700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60700ac3a0 0x7f60700cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:52.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f6075388700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f607015d920 0x7f607015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:52.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f6075388700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607015e2d0 con 0x7f607015a2c0 2024-01-11T07:08:52.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f6075388700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607006ac30 con 0x7f60700ac3a0 2024-01-11T07:08:52.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f6075388700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60700ac770 con 0x7f607015d920 2024-01-11T07:08:52.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f606effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60700ac3a0 0x7f60700cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:52.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f606effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60700ac3a0 0x7f60700cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51066/0 (socket says 172.21.15.7:51066) 2024-01-11T07:08:52.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f606effd700 1 -- 172.21.15.7:0/2232450621 learned_addr learned my addr 172.21.15.7:0/2232450621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.860+0000 7f606e7fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f607015d920 0x7f607015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606effd700 1 -- 172.21.15.7:0/2232450621 >> v2:172.21.15.113:3300/0 conn(0x7f607015d920 msgr2=0x7f607015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606effd700 1 --2- 172.21.15.7:0/2232450621 >> v2:172.21.15.113:3300/0 conn(0x7f607015d920 0x7f607015dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606effd700 1 -- 172.21.15.7:0/2232450621 >> v1:172.21.15.113:6789/0 conn(0x7f607015a2c0 legacy=0x7f607015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606effd700 1 -- 172.21.15.7:0/2232450621 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607006a300 con 0x7f60700ac3a0 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606e7fc700 1 --2- 172.21.15.7:0/2232450621 >> v2:172.21.15.113:3300/0 conn(0x7f607015d920 0x7f607015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606effd700 1 --2- 172.21.15.7:0/2232450621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60700ac3a0 0x7f60700cac60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f607006ce10 tx=0x7f6060009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de3bb1e2c9512a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606dffb700 1 -- 172.21.15.7:0/2232450621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060009e70 con 0x7f60700ac3a0 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606dffb700 1 -- 172.21.15.7:0/2232450621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606000b860 con 0x7f60700ac3a0 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f606dffb700 1 -- 172.21.15.7:0/2232450621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000bc00 con 0x7f60700ac3a0 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f6075388700 1 -- 172.21.15.7:0/2232450621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60700ac3a0 msgr2=0x7f60700cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.861+0000 7f6075388700 1 --2- 172.21.15.7:0/2232450621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60700ac3a0 0x7f60700cac60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f607006ce10 tx=0x7f6060009250 comp rx=0 tx=0).stop 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.862+0000 7f6075388700 1 -- 172.21.15.7:0/2232450621 shutdown_connections 2024-01-11T07:08:52.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.862+0000 7f6075388700 1 --2- 172.21.15.7:0/2232450621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60700ac3a0 0x7f60700cac60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:52.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.862+0000 7f6075388700 1 --2- 172.21.15.7:0/2232450621 >> v2:172.21.15.113:3300/0 conn(0x7f607015d920 0x7f607015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:52.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.862+0000 7f6075388700 1 -- 172.21.15.7:0/2232450621 >> 172.21.15.7:0/2232450621 conn(0x7f60700cd090 msgr2=0x7f60700cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:52.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.862+0000 7f6075388700 1 -- 172.21.15.7:0/2232450621 shutdown_connections 2024-01-11T07:08:52.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.862+0000 7f6075388700 1 -- 172.21.15.7:0/2232450621 wait complete. 2024-01-11T07:08:52.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.863+0000 7f6075388700 1 Processor -- start 2024-01-11T07:08:52.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.863+0000 7f6075388700 1 -- start start 2024-01-11T07:08:52.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f6075388700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60700ac3a0 0x7f6070142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f6075388700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f607015a2c0 0x7f607016ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f6075388700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607015d920 0x7f607013c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f6075388700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701423f0 con 0x7f607015d920 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f6075388700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070141540 con 0x7f60700ac3a0 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f6075388700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607016b1c0 con 0x7f607015a2c0 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607015d920 0x7f607013c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607015d920 0x7f607013c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51072/0 (socket says 172.21.15.7:51072) 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 -- 172.21.15.7:0/1022137273 learned_addr learned my addr 172.21.15.7:0/1022137273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606effd700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60700ac3a0 0x7f6070142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606e7fc700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f607015a2c0 0x7f607016ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 -- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f607015a2c0 msgr2=0x7f607016ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f607015a2c0 0x7f607016ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 -- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60700ac3a0 msgr2=0x7f6070142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:52.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60700ac3a0 0x7f6070142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 -- 172.21.15.7:0/1022137273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058009420 con 0x7f607015d920 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.864+0000 7f606f7fe700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607015d920 0x7f607013c1d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f606800c840 tx=0x7f6068001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.865+0000 7f6066ffd700 1 -- 172.21.15.7:0/1022137273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068012630 con 0x7f607015d920 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.865+0000 7f6066ffd700 1 -- 172.21.15.7:0/1022137273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60680127b0 con 0x7f607015d920 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.865+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60600094f0 con 0x7f607015d920 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.865+0000 7f6066ffd700 1 -- 172.21.15.7:0/1022137273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068012b70 con 0x7f607015d920 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.865+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6070141ac0 con 0x7f607015d920 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.866+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60701426d0 con 0x7f607015d920 2024-01-11T07:08:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.866+0000 7f6066ffd700 1 -- 172.21.15.7:0/1022137273 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f606801a3a0 con 0x7f607015d920 2024-01-11T07:08:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.866+0000 7f6066ffd700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6050066f30 0x7f60500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.867+0000 7f6066ffd700 1 -- 172.21.15.7:0/1022137273 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f60680846d0 con 0x7f607015d920 2024-01-11T07:08:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.867+0000 7f606effd700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6050066f30 0x7f60500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:52.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.867+0000 7f606effd700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6050066f30 0x7f60500693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f607006ce10 tx=0x7f6060017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:52.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:52.871+0000 7f6066ffd700 1 -- 172.21.15.7:0/1022137273 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6068051900 con 0x7f607015d920 2024-01-11T07:08:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.125+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f607004c3c0 con 0x7f607015d920 2024-01-11T07:08:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.125+0000 7f6066ffd700 1 -- 172.21.15.7:0/1022137273 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f6068054f20 con 0x7f607015d920 2024-01-11T07:08:53.125 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:53.125 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6050066f30 msgr2=0x7f60500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6050066f30 0x7f60500693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f607006ce10 tx=0x7f6060017000 comp rx=0 tx=0).stop 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607015d920 msgr2=0x7f607013c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607015d920 0x7f607013c1d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f606800c840 tx=0x7f6068001670 comp rx=0 tx=0).stop 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 shutdown_connections 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6050066f30 0x7f60500693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607015d920 0x7f607013c1d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60700ac3a0 0x7f6070142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 --2- 172.21.15.7:0/1022137273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f607015a2c0 0x7f607016ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 >> 172.21.15.7:0/1022137273 conn(0x7f60700cd090 msgr2=0x7f60700cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 shutdown_connections 2024-01-11T07:08:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.128+0000 7f6075388700 1 -- 172.21.15.7:0/1022137273 wait complete. 2024-01-11T07:08:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:53.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.274+0000 7faa2b117700 1 Processor -- start 2024-01-11T07:08:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.274+0000 7faa2b117700 1 -- start start 2024-01-11T07:08:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.275+0000 7faa2b117700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faa24063230 0x7faa2415e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.275+0000 7faa2b117700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa2415aa40 0x7faa2406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.275+0000 7faa2b117700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2415df80 con 0x7faa24063600 2024-01-11T07:08:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.275+0000 7faa2b117700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2415e950 con 0x7faa2415aa40 2024-01-11T07:08:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.275+0000 7faa2b117700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa240aab70 con 0x7faa24063230 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.275+0000 7faa296b4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa2415aa40 0x7faa2406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.275+0000 7faa23fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faa24063600 0x7faa2415a330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49116/0 (socket says 172.21.15.7:49116) 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.275+0000 7faa23fff700 1 -- 172.21.15.7:0/191406940 learned_addr learned my addr 172.21.15.7:0/191406940 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa28eb3700 1 --2- 172.21.15.7:0/191406940 >> v2:172.21.15.113:3300/0 conn(0x7faa24063230 0x7faa2415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa296b4700 1 -- 172.21.15.7:0/191406940 >> v2:172.21.15.113:3300/0 conn(0x7faa24063230 msgr2=0x7faa2415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa296b4700 1 --2- 172.21.15.7:0/191406940 >> v2:172.21.15.113:3300/0 conn(0x7faa24063230 0x7faa2415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa296b4700 1 -- 172.21.15.7:0/191406940 >> v1:172.21.15.7:6789/0 conn(0x7faa24063600 legacy=0x7faa2415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa296b4700 1 -- 172.21.15.7:0/191406940 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2415af10 con 0x7faa2415aa40 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa296b4700 1 --2- 172.21.15.7:0/191406940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2415aa40 0x7faa2406b530 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa2406ca90 tx=0x7faa1c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68be5943aee128 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa237fe700 1 -- 172.21.15.7:0/191406940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c00e070 con 0x7faa2415aa40 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa237fe700 1 -- 172.21.15.7:0/191406940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c00bc30 con 0x7faa2415aa40 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.276+0000 7faa237fe700 1 -- 172.21.15.7:0/191406940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c012430 con 0x7faa2415aa40 2024-01-11T07:08:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.277+0000 7faa2b117700 1 -- 172.21.15.7:0/191406940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2415aa40 msgr2=0x7faa2406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.277+0000 7faa2b117700 1 --2- 172.21.15.7:0/191406940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2415aa40 0x7faa2406b530 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa2406ca90 tx=0x7faa1c009780 comp rx=0 tx=0).stop 2024-01-11T07:08:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.277+0000 7faa2b117700 1 -- 172.21.15.7:0/191406940 shutdown_connections 2024-01-11T07:08:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.277+0000 7faa2b117700 1 --2- 172.21.15.7:0/191406940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2415aa40 0x7faa2406b530 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.277+0000 7faa2b117700 1 --2- 172.21.15.7:0/191406940 >> v2:172.21.15.113:3300/0 conn(0x7faa24063230 0x7faa2415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.277+0000 7faa2b117700 1 -- 172.21.15.7:0/191406940 >> 172.21.15.7:0/191406940 conn(0x7faa240cd910 msgr2=0x7faa240cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.277+0000 7faa2b117700 1 -- 172.21.15.7:0/191406940 shutdown_connections 2024-01-11T07:08:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.277+0000 7faa2b117700 1 -- 172.21.15.7:0/191406940 wait complete. 2024-01-11T07:08:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.278+0000 7faa2b117700 1 Processor -- start 2024-01-11T07:08:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.278+0000 7faa2b117700 1 -- start start 2024-01-11T07:08:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa2b117700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa24063230 0x7faa240b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa2b117700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa24063600 0x7faa2414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa2b117700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2415aa40 0x7faa240ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa2b117700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa240b0860 con 0x7faa24063600 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa2b117700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa240b13a0 con 0x7faa2415aa40 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa2b117700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa240b0a70 con 0x7faa24063230 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa24063600 0x7faa2414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa24063600 0x7faa2414f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51092/0 (socket says 172.21.15.7:51092) 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa23fff700 1 -- 172.21.15.7:0/3357227332 learned_addr learned my addr 172.21.15.7:0/3357227332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa296b4700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2415aa40 0x7faa240ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa28eb3700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa24063230 0x7faa240b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa23fff700 1 -- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa24063230 msgr2=0x7faa240b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa23fff700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa24063230 0x7faa240b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa23fff700 1 -- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2415aa40 msgr2=0x7faa240ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa23fff700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2415aa40 0x7faa240ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.279+0000 7faa23fff700 1 -- 172.21.15.7:0/3357227332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c009a20 con 0x7faa24063600 2024-01-11T07:08:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.280+0000 7faa23fff700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa24063600 0x7faa2414f380 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c7c0 tx=0x7faa10007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.280+0000 7faa20ff9700 1 -- 172.21.15.7:0/3357227332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10005b30 con 0x7faa24063600 2024-01-11T07:08:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.280+0000 7faa20ff9700 1 -- 172.21.15.7:0/3357227332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa10005c90 con 0x7faa24063600 2024-01-11T07:08:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.280+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa240b0d50 con 0x7faa24063600 2024-01-11T07:08:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.280+0000 7faa20ff9700 1 -- 172.21.15.7:0/3357227332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10004a80 con 0x7faa24063600 2024-01-11T07:08:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.280+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa240b2b70 con 0x7faa24063600 2024-01-11T07:08:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.281+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa240ac6c0 con 0x7faa24063600 2024-01-11T07:08:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.281+0000 7faa20ff9700 1 -- 172.21.15.7:0/3357227332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faa100070d0 con 0x7faa24063600 2024-01-11T07:08:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.282+0000 7faa20ff9700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.282+0000 7faa20ff9700 1 -- 172.21.15.7:0/3357227332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7faa10083870 con 0x7faa24063600 2024-01-11T07:08:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.282+0000 7faa28eb3700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.282+0000 7faa28eb3700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa08066f30 0x7faa080693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa18006fd0 tx=0x7faa18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.286+0000 7faa20ff9700 1 -- 172.21.15.7:0/3357227332 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa1004b5a0 con 0x7faa24063600 2024-01-11T07:08:53.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.493+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa240b1510 con 0x7faa24063600 2024-01-11T07:08:53.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.494+0000 7faa20ff9700 1 -- 172.21.15.7:0/3357227332 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+15398 (secure 0 0 0) 0x7faa10083070 con 0x7faa24063600 2024-01-11T07:08:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:53.494 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:49.209265+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:08:17.698588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-11T07:08:18.590636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa08066f30 msgr2=0x7faa080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa08066f30 0x7faa080693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa18006fd0 tx=0x7faa18008040 comp rx=0 tx=0).stop 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa24063600 msgr2=0x7faa2414f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa24063600 0x7faa2414f380 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c7c0 tx=0x7faa10007f70 comp rx=0 tx=0).stop 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 shutdown_connections 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa08066f30 0x7faa080693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2415aa40 0x7faa240ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa24063230 0x7faa240b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 --2- 172.21.15.7:0/3357227332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa24063600 0x7faa2414f380 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 >> 172.21.15.7:0/3357227332 conn(0x7faa240cd910 msgr2=0x7faa24157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 shutdown_connections 2024-01-11T07:08:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.497+0000 7faa2b117700 1 -- 172.21.15.7:0/3357227332 wait complete. 2024-01-11T07:08:53.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:08:17.698588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:08:18.590636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:08:53.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:08:53.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.645+0000 7ff3dbd5b700 1 Processor -- start 2024-01-11T07:08:53.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.645+0000 7ff3dbd5b700 1 -- start start 2024-01-11T07:08:53.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.646+0000 7ff3dbd5b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3d415a1a0 0x7ff3d40cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.646+0000 7ff3dbd5b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d40ced10 con 0x7ff3d40b0030 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.646+0000 7ff3dbd5b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d40cc140 con 0x7ff3d40cbca0 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.646+0000 7ff3dbd5b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d40714b0 con 0x7ff3d415a1a0 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.646+0000 7ff3d9af7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3d415a1a0 0x7ff3d40cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.646+0000 7ff3da2f8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff3d40b0030 0x7ff3d4163d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49118/0 (socket says 172.21.15.7:49118) 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.646+0000 7ff3da2f8700 1 -- 172.21.15.7:0/3939454407 learned_addr learned my addr 172.21.15.7:0/3939454407 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.647+0000 7ff3d9af7700 1 -- 172.21.15.7:0/3939454407 >> v1:172.21.15.113:6789/0 conn(0x7ff3d40cbca0 legacy=0x7ff3d40af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.647+0000 7ff3d9af7700 1 -- 172.21.15.7:0/3939454407 >> v1:172.21.15.7:6789/0 conn(0x7ff3d40b0030 legacy=0x7ff3d4163d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.647+0000 7ff3d9af7700 1 -- 172.21.15.7:0/3939454407 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40cc960 con 0x7ff3d415a1a0 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.647+0000 7ff3d8af5700 1 -- 172.21.15.7:0/3939454407 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375429884 0 0) 0x7ff3d40ced10 con 0x7ff3d40b0030 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.647+0000 7ff3d9af7700 1 --2- 172.21.15.7:0/3939454407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d415a1a0 0x7ff3d40cddf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00098e0 tx=0x7ff3d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8136dfb26ff19f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.647+0000 7ff3d8af5700 1 -- 172.21.15.7:0/3939454407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0009e80 con 0x7ff3d415a1a0 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.647+0000 7ff3d8af5700 1 -- 172.21.15.7:0/3939454407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d000b860 con 0x7ff3d415a1a0 2024-01-11T07:08:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.647+0000 7ff3d8af5700 1 -- 172.21.15.7:0/3939454407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000bbe0 con 0x7ff3d415a1a0 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.648+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/3939454407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d415a1a0 msgr2=0x7ff3d40cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.648+0000 7ff3dbd5b700 1 --2- 172.21.15.7:0/3939454407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d415a1a0 0x7ff3d40cddf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00098e0 tx=0x7ff3d00091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.648+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/3939454407 shutdown_connections 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.648+0000 7ff3dbd5b700 1 --2- 172.21.15.7:0/3939454407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d415a1a0 0x7ff3d40cddf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.648+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/3939454407 >> 172.21.15.7:0/3939454407 conn(0x7ff3d406e6c0 msgr2=0x7ff3d4070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.648+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/3939454407 shutdown_connections 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.648+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/3939454407 wait complete. 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.649+0000 7ff3dbd5b700 1 Processor -- start 2024-01-11T07:08:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.649+0000 7ff3dbd5b700 1 -- start start 2024-01-11T07:08:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3dbd5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d40b0030 0x7ff3d41544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3dbd5b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d40cbca0 0x7ff3d416ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3dbd5b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d415a1a0 0x7ff3d40b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3dbd5b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d416b370 con 0x7ff3d40b0030 2024-01-11T07:08:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3dbd5b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d416b900 con 0x7ff3d415a1a0 2024-01-11T07:08:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3dbd5b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d416bbe0 con 0x7ff3d40cbca0 2024-01-11T07:08:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d40b0030 0x7ff3d41544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d40b0030 0x7ff3d41544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51102/0 (socket says 172.21.15.7:51102) 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 -- 172.21.15.7:0/2543997845 learned_addr learned my addr 172.21.15.7:0/2543997845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d92f6700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d40cbca0 0x7ff3d416ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3da2f8700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d415a1a0 0x7ff3d40b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 -- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d40cbca0 msgr2=0x7ff3d416ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d40cbca0 0x7ff3d416ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 -- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d415a1a0 msgr2=0x7ff3d40b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d415a1a0 0x7ff3d40b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 -- 172.21.15.7:0/2543997845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0009420 con 0x7ff3d40b0030 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3d9af7700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d40b0030 0x7ff3d41544d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4067fc0 tx=0x7ff3d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.650+0000 7ff3c9ffb700 1 -- 172.21.15.7:0/2543997845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d001f070 con 0x7ff3d40b0030 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.651+0000 7ff3c9ffb700 1 -- 172.21.15.7:0/2543997845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d0000c90 con 0x7ff3d40b0030 2024-01-11T07:08:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.651+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d416bec0 con 0x7ff3d40b0030 2024-01-11T07:08:53.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.651+0000 7ff3c9ffb700 1 -- 172.21.15.7:0/2543997845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0025420 con 0x7ff3d40b0030 2024-01-11T07:08:53.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.651+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d416c480 con 0x7ff3d40b0030 2024-01-11T07:08:53.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.652+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d416c7f0 con 0x7ff3d40b0030 2024-01-11T07:08:53.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.652+0000 7ff3c9ffb700 1 -- 172.21.15.7:0/2543997845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff3d0023070 con 0x7ff3d40b0030 2024-01-11T07:08:53.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.653+0000 7ff3c9ffb700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3b8066f30 0x7ff3b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:53.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.653+0000 7ff3c9ffb700 1 -- 172.21.15.7:0/2543997845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7ff3d0085aa0 con 0x7ff3d40b0030 2024-01-11T07:08:53.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.653+0000 7ff3d92f6700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3b8066f30 0x7ff3b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:53.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.653+0000 7ff3d92f6700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3b8066f30 0x7ff3b80693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40caf20 tx=0x7ff3c4007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:53.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.657+0000 7ff3c9ffb700 1 -- 172.21.15.7:0/2543997845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3d0052c30 con 0x7ff3d40b0030 2024-01-11T07:08:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.862+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3d4089350 con 0x7ff3d40b0030 2024-01-11T07:08:53.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.862+0000 7ff3c9ffb700 1 -- 172.21.15.7:0/2543997845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3d002a020 con 0x7ff3d40b0030 2024-01-11T07:08:53.862 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:08:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3b8066f30 msgr2=0x7ff3b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3b8066f30 0x7ff3b80693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40caf20 tx=0x7ff3c4007ec0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d40b0030 msgr2=0x7ff3d41544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d40b0030 0x7ff3d41544d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4067fc0 tx=0x7ff3d0017000 comp rx=0 tx=0).stop 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 shutdown_connections 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3b8066f30 0x7ff3b80693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d415a1a0 0x7ff3d40b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d40b0030 0x7ff3d41544d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 --2- 172.21.15.7:0/2543997845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d40cbca0 0x7ff3d416ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 >> 172.21.15.7:0/2543997845 conn(0x7ff3d406e6c0 msgr2=0x7ff3d40cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 shutdown_connections 2024-01-11T07:08:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:53.865+0000 7ff3dbd5b700 1 -- 172.21.15.7:0/2543997845 wait complete. 2024-01-11T07:08:53.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:08:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.013+0000 7f4c05a9e700 1 Processor -- start 2024-01-11T07:08:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.013+0000 7f4c05a9e700 1 -- start start 2024-01-11T07:08:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4c05a9e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c000b1790 0x7f4c000afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4c05a9e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c000b0a80 con 0x7f4c00157ab0 2024-01-11T07:08:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4c05a9e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c000cab10 con 0x7f4c0006ba80 2024-01-11T07:08:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4c05a9e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c000b02e0 con 0x7f4c000b1790 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4bfffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c000b1790 0x7f4c000afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4bff7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4c00157ab0 0x7f4c0006b370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49128/0 (socket says 172.21.15.7:49128) 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4bff7fe700 1 -- 172.21.15.7:0/2561838857 learned_addr learned my addr 172.21.15.7:0/2561838857 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4bfffff700 1 -- 172.21.15.7:0/2561838857 >> v1:172.21.15.113:6789/0 conn(0x7f4c0006ba80 legacy=0x7f4c000af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4bfffff700 1 -- 172.21.15.7:0/2561838857 >> v1:172.21.15.7:6789/0 conn(0x7f4c00157ab0 legacy=0x7f4c0006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4bfffff700 1 -- 172.21.15.7:0/2561838857 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c000b04c0 con 0x7f4c000b1790 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.014+0000 7f4bfe7fc700 1 -- 172.21.15.7:0/2561838857 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986765380 0 0) 0x7f4c000b0a80 con 0x7f4c00157ab0 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4bfffff700 1 --2- 172.21.15.7:0/2561838857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c000b1790 0x7f4c000afda0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009e60 tx=0x7f4bf8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bb7326ea1ea261 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4bfe7fc700 1 -- 172.21.15.7:0/2561838857 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800e070 con 0x7f4c000b1790 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4bfe7fc700 1 -- 172.21.15.7:0/2561838857 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf800bc30 con 0x7f4c000b1790 2024-01-11T07:08:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4bfe7fc700 1 -- 172.21.15.7:0/2561838857 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8012430 con 0x7f4c000b1790 2024-01-11T07:08:54.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2561838857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c000b1790 msgr2=0x7f4c000afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4c05a9e700 1 --2- 172.21.15.7:0/2561838857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c000b1790 0x7f4c000afda0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009e60 tx=0x7f4bf8009780 comp rx=0 tx=0).stop 2024-01-11T07:08:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2561838857 shutdown_connections 2024-01-11T07:08:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4c05a9e700 1 --2- 172.21.15.7:0/2561838857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c000b1790 0x7f4c000afda0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2561838857 >> 172.21.15.7:0/2561838857 conn(0x7f4c00070690 msgr2=0x7f4c00070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.015+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2561838857 shutdown_connections 2024-01-11T07:08:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.016+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2561838857 wait complete. 2024-01-11T07:08:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.017+0000 7f4c05a9e700 1 Processor -- start 2024-01-11T07:08:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.017+0000 7f4c05a9e700 1 -- start start 2024-01-11T07:08:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4c05a9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c0006ba80 0x7f4c000c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4c05a9e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000b1790 0x7f4c001681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4c05a9e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c00157ab0 0x7f4c000c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4c05a9e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c001567c0 con 0x7f4c0006ba80 2024-01-11T07:08:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4c05a9e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c000c7f10 con 0x7f4c000b1790 2024-01-11T07:08:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4c05a9e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c000a6a00 con 0x7f4c00157ab0 2024-01-11T07:08:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c0006ba80 0x7f4c000c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c0006ba80 0x7f4c000c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51112/0 (socket says 172.21.15.7:51112) 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 -- 172.21.15.7:0/2882858955 learned_addr learned my addr 172.21.15.7:0/2882858955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bfffff700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c00157ab0 0x7f4c000c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bfeffd700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000b1790 0x7f4c001681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 -- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c00157ab0 msgr2=0x7f4c000c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c00157ab0 0x7f4c000c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 -- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000b1790 msgr2=0x7f4c001681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000b1790 0x7f4c001681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 -- 172.21.15.7:0/2882858955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8009a20 con 0x7f4c0006ba80 2024-01-11T07:08:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.018+0000 7f4bff7fe700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c0006ba80 0x7f4c000c6520 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000c5b0 tx=0x7f4bf0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.019+0000 7f4be77fe700 1 -- 172.21.15.7:0/2882858955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0005b30 con 0x7f4c0006ba80 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.019+0000 7f4be77fe700 1 -- 172.21.15.7:0/2882858955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0005c90 con 0x7f4c0006ba80 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.019+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c000c81b0 con 0x7f4c0006ba80 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.019+0000 7f4be77fe700 1 -- 172.21.15.7:0/2882858955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0007300 con 0x7f4c0006ba80 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.019+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c000c8770 con 0x7f4c0006ba80 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.020+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c000a6d50 con 0x7f4c0006ba80 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.020+0000 7f4be77fe700 1 -- 172.21.15.7:0/2882858955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4bf001b3b0 con 0x7f4c0006ba80 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.021+0000 7f4be77fe700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4bec066f30 0x7f4bec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.021+0000 7f4be77fe700 1 -- 172.21.15.7:0/2882858955 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4bf0083970 con 0x7f4c0006ba80 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.021+0000 7f4bfeffd700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4bec066f30 0x7f4bec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.021+0000 7f4bfeffd700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4bec066f30 0x7f4bec0693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c00156d40 tx=0x7f4bf4007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.025+0000 7f4be77fe700 1 -- 172.21.15.7:0/2882858955 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bf0050ba0 con 0x7f4c0006ba80 2024-01-11T07:08:54.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.228+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c000b0d10 con 0x7f4c0006ba80 2024-01-11T07:08:54.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.229+0000 7f4be77fe700 1 -- 172.21.15.7:0/2882858955 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4bf0017020 con 0x7f4c0006ba80 2024-01-11T07:08:54.232 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.231+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4bec066f30 msgr2=0x7f4bec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.231+0000 7f4c05a9e700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4bec066f30 0x7f4bec0693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c00156d40 tx=0x7f4bf4007ec0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.231+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c0006ba80 msgr2=0x7f4c000c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.231+0000 7f4c05a9e700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c0006ba80 0x7f4c000c6520 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000c5b0 tx=0x7f4bf0001070 comp rx=0 tx=0).stop 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.231+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 shutdown_connections 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.232+0000 7f4c05a9e700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4bec066f30 0x7f4bec0693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.232+0000 7f4c05a9e700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c00157ab0 0x7f4c000c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.232+0000 7f4c05a9e700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c0006ba80 0x7f4c000c6520 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.232+0000 7f4c05a9e700 1 --2- 172.21.15.7:0/2882858955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c000b1790 0x7f4c001681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.232+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 >> 172.21.15.7:0/2882858955 conn(0x7f4c00070690 msgr2=0x7f4c000b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.232+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 shutdown_connections 2024-01-11T07:08:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.232+0000 7f4c05a9e700 1 -- 172.21.15.7:0/2882858955 wait complete. 2024-01-11T07:08:54.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:08:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.380+0000 7f2e8f4eb700 1 Processor -- start 2024-01-11T07:08:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.381+0000 7f2e8f4eb700 1 -- start start 2024-01-11T07:08:54.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8f4eb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8806ad10 0x7f2e88164c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8f4eb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8815b860 con 0x7f2e880af7a0 2024-01-11T07:08:54.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8f4eb700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8815a9d0 con 0x7f2e880ada70 2024-01-11T07:08:54.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8f4eb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8815b530 con 0x7f2e8806ad10 2024-01-11T07:08:54.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8ca86700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2e880af7a0 0x7f2e8806a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49134/0 (socket says 172.21.15.7:49134) 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8ca86700 1 -- 172.21.15.7:0/297693239 learned_addr learned my addr 172.21.15.7:0/297693239 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8da88700 1 --2- 172.21.15.7:0/297693239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8806ad10 0x7f2e88164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e7ffff700 1 -- 172.21.15.7:0/297693239 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843361342 0 0) 0x7f2e8815b860 con 0x7f2e880af7a0 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e7ffff700 1 -- 172.21.15.7:0/297693239 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e74003570 con 0x7f2e880af7a0 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e7ffff700 1 -- 172.21.15.7:0/297693239 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715992722 0 0) 0x7f2e8815a9d0 con 0x7f2e880ada70 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e7ffff700 1 -- 172.21.15.7:0/297693239 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e8815b860 con 0x7f2e880ada70 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8da88700 1 -- 172.21.15.7:0/297693239 >> v1:172.21.15.113:6789/0 conn(0x7f2e880ada70 legacy=0x7f2e8815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8da88700 1 -- 172.21.15.7:0/297693239 >> v1:172.21.15.7:6789/0 conn(0x7f2e880af7a0 legacy=0x7f2e8806a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.382+0000 7f2e8da88700 1 -- 172.21.15.7:0/297693239 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8815ac30 con 0x7f2e8806ad10 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e8da88700 1 --2- 172.21.15.7:0/297693239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8806ad10 0x7f2e88164c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009e90 tx=0x7f2e80009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2cfedfc1706568 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e7ffff700 1 -- 172.21.15.7:0/297693239 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000e070 con 0x7f2e8806ad10 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e7ffff700 1 -- 172.21.15.7:0/297693239 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8000bb80 con 0x7f2e8806ad10 2024-01-11T07:08:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e7ffff700 1 -- 172.21.15.7:0/297693239 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80012430 con 0x7f2e8806ad10 2024-01-11T07:08:54.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/297693239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8806ad10 msgr2=0x7f2e88164c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e8f4eb700 1 --2- 172.21.15.7:0/297693239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8806ad10 0x7f2e88164c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009e90 tx=0x7f2e80009780 comp rx=0 tx=0).stop 2024-01-11T07:08:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/297693239 shutdown_connections 2024-01-11T07:08:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e8f4eb700 1 --2- 172.21.15.7:0/297693239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8806ad10 0x7f2e88164c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/297693239 >> 172.21.15.7:0/297693239 conn(0x7f2e880cd070 msgr2=0x7f2e880cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/297693239 shutdown_connections 2024-01-11T07:08:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.383+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/297693239 wait complete. 2024-01-11T07:08:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.385+0000 7f2e8f4eb700 1 Processor -- start 2024-01-11T07:08:54.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.385+0000 7f2e8f4eb700 1 -- start start 2024-01-11T07:08:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8f4eb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8806ad10 0x7f2e881549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8f4eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ada70 0x7f2e88156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8f4eb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880af7a0 0x7f2e88156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8f4eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e88142a80 con 0x7f2e880ada70 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8f4eb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880b1bd0 con 0x7f2e8806ad10 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8f4eb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880b1eb0 con 0x7f2e880af7a0 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ada70 0x7f2e88156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ada70 0x7f2e88156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51124/0 (socket says 172.21.15.7:51124) 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 -- 172.21.15.7:0/555483997 learned_addr learned my addr 172.21.15.7:0/555483997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8d287700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8806ad10 0x7f2e881549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8da88700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880af7a0 0x7f2e88156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 -- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880af7a0 msgr2=0x7f2e88156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880af7a0 0x7f2e88156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 -- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8806ad10 msgr2=0x7f2e881549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8806ad10 0x7f2e881549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 -- 172.21.15.7:0/555483997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e80009a20 con 0x7f2e880ada70 2024-01-11T07:08:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8ca86700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ada70 0x7f2e88156690 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800c580 tx=0x7f2e78007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e7d7fa700 1 -- 172.21.15.7:0/555483997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78005b30 con 0x7f2e880ada70 2024-01-11T07:08:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e880b2190 con 0x7f2e880ada70 2024-01-11T07:08:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e7d7fa700 1 -- 172.21.15.7:0/555483997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78005cb0 con 0x7f2e880ada70 2024-01-11T07:08:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.386+0000 7f2e7d7fa700 1 -- 172.21.15.7:0/555483997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78007320 con 0x7f2e880ada70 2024-01-11T07:08:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.387+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e880b27d0 con 0x7f2e880ada70 2024-01-11T07:08:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.388+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8804c3c0 con 0x7f2e880ada70 2024-01-11T07:08:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.388+0000 7f2e7d7fa700 1 -- 172.21.15.7:0/555483997 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2e7801b3b0 con 0x7f2e880ada70 2024-01-11T07:08:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.388+0000 7f2e7d7fa700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.389+0000 7f2e7d7fa700 1 -- 172.21.15.7:0/555483997 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f2e78083960 con 0x7f2e880ada70 2024-01-11T07:08:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.389+0000 7f2e8d287700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.389+0000 7f2e8d287700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e84007fd0 tx=0x7f2e84007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.393+0000 7f2e7d7fa700 1 -- 172.21.15.7:0/555483997 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e78050b90 con 0x7f2e880ada70 2024-01-11T07:08:54.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.599+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e880b2960 con 0x7f2e880ada70 2024-01-11T07:08:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.599+0000 7f2e7d7fa700 1 -- 172.21.15.7:0/555483997 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e78017020 con 0x7f2e880ada70 2024-01-11T07:08:54.599 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:08:54.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.601+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 msgr2=0x7f2e6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.601+0000 7f2e8f4eb700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e84007fd0 tx=0x7f2e84007d50 comp rx=0 tx=0).stop 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.601+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ada70 msgr2=0x7f2e88156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.601+0000 7f2e8f4eb700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ada70 0x7f2e88156690 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800c580 tx=0x7f2e78007e20 comp rx=0 tx=0).stop 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.602+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 shutdown_connections 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.602+0000 7f2e8f4eb700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.602+0000 7f2e8f4eb700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880af7a0 0x7f2e88156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.602+0000 7f2e8f4eb700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8806ad10 0x7f2e881549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.602+0000 7f2e8f4eb700 1 --2- 172.21.15.7:0/555483997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880ada70 0x7f2e88156690 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.602+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 >> 172.21.15.7:0/555483997 conn(0x7f2e880cd070 msgr2=0x7f2e8806d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.602+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 shutdown_connections 2024-01-11T07:08:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.602+0000 7f2e8f4eb700 1 -- 172.21.15.7:0/555483997 wait complete. 2024-01-11T07:08:54.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.745+0000 7fecd8df5700 1 Processor -- start 2024-01-11T07:08:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.745+0000 7fecd8df5700 1 -- start start 2024-01-11T07:08:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd8df5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecd4157da0 0x7fecd40cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd8df5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd406d270 con 0x7fecd41579d0 2024-01-11T07:08:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd8df5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40cda40 con 0x7fecd40ce220 2024-01-11T07:08:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd8df5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd40cd5b0 con 0x7fecd4157da0 2024-01-11T07:08:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecd4157da0 0x7fecd40cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fecd41579d0 0x7fecd415c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49140/0 (socket says 172.21.15.7:49140) 2024-01-11T07:08:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd259c700 1 -- 172.21.15.7:0/491073254 learned_addr learned my addr 172.21.15.7:0/491073254 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecd4157da0 0x7fecd40cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51136/0 (socket says 172.21.15.7:51136) 2024-01-11T07:08:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd1d9b700 1 -- 172.21.15.7:0/491073254 >> v1:172.21.15.113:6789/0 conn(0x7fecd40ce220 legacy=0x7fecd40ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd1d9b700 1 -- 172.21.15.7:0/491073254 >> v1:172.21.15.7:6789/0 conn(0x7fecd41579d0 legacy=0x7fecd415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd1d9b700 1 -- 172.21.15.7:0/491073254 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40ce610 con 0x7fecd4157da0 2024-01-11T07:08:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd159a700 1 -- 172.21.15.7:0/491073254 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753503287 0 0) 0x7fecd406d270 con 0x7fecd41579d0 2024-01-11T07:08:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd1d9b700 1 --2- 172.21.15.7:0/491073254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd40cdce0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098e0 tx=0x7fecc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a017cc63747a6f05 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd159a700 1 -- 172.21.15.7:0/491073254 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4009e80 con 0x7fecd4157da0 2024-01-11T07:08:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.747+0000 7fecd159a700 1 -- 172.21.15.7:0/491073254 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc400b860 con 0x7fecd4157da0 2024-01-11T07:08:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.748+0000 7fecd159a700 1 -- 172.21.15.7:0/491073254 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc400bbe0 con 0x7fecd4157da0 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.748+0000 7fecd8df5700 1 -- 172.21.15.7:0/491073254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 msgr2=0x7fecd40cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.748+0000 7fecd8df5700 1 --2- 172.21.15.7:0/491073254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd40cdce0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098e0 tx=0x7fecc40091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.748+0000 7fecd8df5700 1 -- 172.21.15.7:0/491073254 shutdown_connections 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.748+0000 7fecd8df5700 1 --2- 172.21.15.7:0/491073254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd40cdce0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.748+0000 7fecd8df5700 1 -- 172.21.15.7:0/491073254 >> 172.21.15.7:0/491073254 conn(0x7fecd406ae00 msgr2=0x7fecd406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.748+0000 7fecd8df5700 1 -- 172.21.15.7:0/491073254 shutdown_connections 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.748+0000 7fecd8df5700 1 -- 172.21.15.7:0/491073254 wait complete. 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.749+0000 7fecd8df5700 1 Processor -- start 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.749+0000 7fecd8df5700 1 -- start start 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd8df5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecd40ce220 0x7fecd40c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd8df5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd41579d0 0x7fecd40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd8df5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd4158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd8df5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40b2e30 con 0x7fecd4157da0 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd8df5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40c5fe0 con 0x7fecd40ce220 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd8df5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40b4410 con 0x7fecd41579d0 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd4158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd4158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51150/0 (socket says 172.21.15.7:51150) 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd2d9d700 1 -- 172.21.15.7:0/3241810169 learned_addr learned my addr 172.21.15.7:0/3241810169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd259c700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecd40ce220 0x7fecd40c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.750+0000 7fecd1d9b700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd41579d0 0x7fecd40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd2d9d700 1 -- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd41579d0 msgr2=0x7fecd40c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd2d9d700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd41579d0 0x7fecd40c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd2d9d700 1 -- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecd40ce220 msgr2=0x7fecd40c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd2d9d700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecd40ce220 0x7fecd40c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd2d9d700 1 -- 172.21.15.7:0/3241810169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4009420 con 0x7fecd4157da0 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd259c700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecd40ce220 0x7fecd40c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd2d9d700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd4158d40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feccc00c8e0 tx=0x7feccc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecca7fc700 1 -- 172.21.15.7:0/3241810169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc00ce80 con 0x7fecd4157da0 2024-01-11T07:08:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecca7fc700 1 -- 172.21.15.7:0/3241810169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc00a7e0 con 0x7fecd4157da0 2024-01-11T07:08:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd415aa90 con 0x7fecd4157da0 2024-01-11T07:08:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecca7fc700 1 -- 172.21.15.7:0/3241810169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc00ab80 con 0x7fecd4157da0 2024-01-11T07:08:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.751+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd40b2b50 con 0x7fecd4157da0 2024-01-11T07:08:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.752+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd4068100 con 0x7fecd4157da0 2024-01-11T07:08:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.753+0000 7fecca7fc700 1 -- 172.21.15.7:0/3241810169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feccc012420 con 0x7fecd4157da0 2024-01-11T07:08:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.753+0000 7fecca7fc700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.753+0000 7fecca7fc700 1 -- 172.21.15.7:0/3241810169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7feccc0846e0 con 0x7fecd4157da0 2024-01-11T07:08:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.753+0000 7fecd259c700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.754+0000 7fecd259c700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fecb4066f30 0x7fecb40693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0011e0 tx=0x7fecbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:54.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:54.757+0000 7fecca7fc700 1 -- 172.21.15.7:0/3241810169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feccc051910 con 0x7fecd4157da0 2024-01-11T07:08:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.011+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecd4068290 con 0x7fecd4157da0 2024-01-11T07:08:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.012+0000 7fecca7fc700 1 -- 172.21.15.7:0/3241810169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7feccc054f30 con 0x7fecd4157da0 2024-01-11T07:08:55.011 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:55.012 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fecb4066f30 msgr2=0x7fecb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fecb4066f30 0x7fecb40693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0011e0 tx=0x7fecbc009040 comp rx=0 tx=0).stop 2024-01-11T07:08:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 msgr2=0x7fecd4158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd4158d40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feccc00c8e0 tx=0x7feccc001710 comp rx=0 tx=0).stop 2024-01-11T07:08:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 shutdown_connections 2024-01-11T07:08:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd4157da0 0x7fecd4158d40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fecd40ce220 0x7fecd40c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 --2- 172.21.15.7:0/3241810169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fecd41579d0 0x7fecd40c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 >> 172.21.15.7:0/3241810169 conn(0x7fecd406ae00 msgr2=0x7fecd40cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 shutdown_connections 2024-01-11T07:08:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.014+0000 7fecd8df5700 1 -- 172.21.15.7:0/3241810169 wait complete. 2024-01-11T07:08:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:55.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.159+0000 7f9b9303e700 1 Processor -- start 2024-01-11T07:08:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.159+0000 7f9b9303e700 1 -- start start 2024-01-11T07:08:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.159+0000 7f9b9303e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b8c0cef00 0x7f9b8c164ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b9303e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8c0d18c0 con 0x7f9b8c159ca0 2024-01-11T07:08:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b9303e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8c0d0010 con 0x7f9b8c06e340 2024-01-11T07:08:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b9303e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c0cf400 con 0x7f9b8c0cef00 2024-01-11T07:08:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b90dda700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9b8c159ca0 0x7f9b8c06dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49146/0 (socket says 172.21.15.7:49146) 2024-01-11T07:08:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b90dda700 1 -- 172.21.15.7:0/2216923821 learned_addr learned my addr 172.21.15.7:0/2216923821 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b915db700 1 --2- 172.21.15.7:0/2216923821 >> v2:172.21.15.113:3300/0 conn(0x7f9b8c0cef00 0x7f9b8c164ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b8b7fe700 1 -- 172.21.15.7:0/2216923821 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435547468 0 0) 0x7f9b8c0d18c0 con 0x7f9b8c159ca0 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b8b7fe700 1 -- 172.21.15.7:0/2216923821 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b7c003570 con 0x7f9b8c159ca0 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b915db700 1 -- 172.21.15.7:0/2216923821 >> v1:172.21.15.113:6789/0 conn(0x7f9b8c06e340 legacy=0x7f9b8c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b8b7fe700 1 -- 172.21.15.7:0/2216923821 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2509074295 0 0) 0x7f9b7c003570 con 0x7f9b8c159ca0 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b915db700 1 -- 172.21.15.7:0/2216923821 >> v1:172.21.15.7:6789/0 conn(0x7f9b8c159ca0 legacy=0x7f9b8c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.160+0000 7f9b915db700 1 -- 172.21.15.7:0/2216923821 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c0cf6e0 con 0x7f9b8c0cef00 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.161+0000 7f9b915db700 1 --2- 172.21.15.7:0/2216923821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c164ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009e60 tx=0x7f9b84009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f342e9040e9832d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.161+0000 7f9b8b7fe700 1 -- 172.21.15.7:0/2216923821 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400e070 con 0x7f9b8c0cef00 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.161+0000 7f9b8b7fe700 1 -- 172.21.15.7:0/2216923821 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8400b850 con 0x7f9b8c0cef00 2024-01-11T07:08:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.161+0000 7f9b8b7fe700 1 -- 172.21.15.7:0/2216923821 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80001470 con 0x7f9b8c0cef00 2024-01-11T07:08:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.161+0000 7f9b9303e700 1 -- 172.21.15.7:0/2216923821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c0cef00 msgr2=0x7f9b8c164ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.161+0000 7f9b9303e700 1 --2- 172.21.15.7:0/2216923821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c164ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009e60 tx=0x7f9b84009780 comp rx=0 tx=0).stop 2024-01-11T07:08:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.161+0000 7f9b9303e700 1 -- 172.21.15.7:0/2216923821 shutdown_connections 2024-01-11T07:08:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.162+0000 7f9b9303e700 1 --2- 172.21.15.7:0/2216923821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c164ab0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.162+0000 7f9b9303e700 1 -- 172.21.15.7:0/2216923821 >> 172.21.15.7:0/2216923821 conn(0x7f9b8c063cc0 msgr2=0x7f9b8c0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.162+0000 7f9b9303e700 1 -- 172.21.15.7:0/2216923821 shutdown_connections 2024-01-11T07:08:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.162+0000 7f9b9303e700 1 -- 172.21.15.7:0/2216923821 wait complete. 2024-01-11T07:08:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.163+0000 7f9b9303e700 1 Processor -- start 2024-01-11T07:08:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.163+0000 7f9b9303e700 1 -- start start 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b9303e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b8c06e340 0x7f9b8c14f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b9303e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b9303e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c159ca0 0x7f9b8c157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b9303e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0c9b30 con 0x7f9b8c0cef00 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b9303e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c150370 con 0x7f9b8c159ca0 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b9303e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c150610 con 0x7f9b8c06e340 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c1570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51166/0 (socket says 172.21.15.7:51166) 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b8bfff700 1 -- 172.21.15.7:0/2262222511 learned_addr learned my addr 172.21.15.7:0/2262222511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b90dda700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b8c06e340 0x7f9b8c14f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b915db700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c159ca0 0x7f9b8c157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b8bfff700 1 -- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b8c06e340 msgr2=0x7f9b8c14f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b8bfff700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b8c06e340 0x7f9b8c14f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b8bfff700 1 -- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c159ca0 msgr2=0x7f9b8c157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b8bfff700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c159ca0 0x7f9b8c157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.164+0000 7f9b8bfff700 1 -- 172.21.15.7:0/2262222511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b84009a20 con 0x7f9b8c0cef00 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.165+0000 7f9b8bfff700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c1570e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800c590 tx=0x7f9b78007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.165+0000 7f9b88ff9700 1 -- 172.21.15.7:0/2262222511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78011590 con 0x7f9b8c0cef00 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.165+0000 7f9b88ff9700 1 -- 172.21.15.7:0/2262222511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78011710 con 0x7f9b8c0cef00 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.165+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c0b5020 con 0x7f9b8c0cef00 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.165+0000 7f9b88ff9700 1 -- 172.21.15.7:0/2262222511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78011ad0 con 0x7f9b8c0cef00 2024-01-11T07:08:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.165+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c0b53b0 con 0x7f9b8c0cef00 2024-01-11T07:08:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.166+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c0b5980 con 0x7f9b8c0cef00 2024-01-11T07:08:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.166+0000 7f9b88ff9700 1 -- 172.21.15.7:0/2262222511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9b7801a360 con 0x7f9b8c0cef00 2024-01-11T07:08:55.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.167+0000 7f9b88ff9700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b70066f30 0x7f9b700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.167+0000 7f9b88ff9700 1 -- 172.21.15.7:0/2262222511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f9b78083460 con 0x7f9b8c0cef00 2024-01-11T07:08:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.167+0000 7f9b90dda700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b70066f30 0x7f9b700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.167+0000 7f9b90dda700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b70066f30 0x7f9b700693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b80001ed0 tx=0x7f9b800019d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.171+0000 7f9b88ff9700 1 -- 172.21.15.7:0/2262222511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b78050690 con 0x7f9b8c0cef00 2024-01-11T07:08:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.378+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b8c04c3c0 con 0x7f9b8c0cef00 2024-01-11T07:08:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.379+0000 7f9b88ff9700 1 -- 172.21.15.7:0/2262222511 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f9b78053cb0 con 0x7f9b8c0cef00 2024-01-11T07:08:55.379 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:55.379 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:49.209265+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:08:17.698588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-11T07:08:18.590636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b70066f30 msgr2=0x7f9b700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b70066f30 0x7f9b700693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b80001ed0 tx=0x7f9b800019d0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0cef00 msgr2=0x7f9b8c1570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c1570e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800c590 tx=0x7f9b78007f50 comp rx=0 tx=0).stop 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 shutdown_connections 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b70066f30 0x7f9b700693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b8c159ca0 0x7f9b8c157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9b8c06e340 0x7f9b8c14f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 --2- 172.21.15.7:0/2262222511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b8c0cef00 0x7f9b8c1570e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 >> 172.21.15.7:0/2262222511 conn(0x7f9b8c063cc0 msgr2=0x7f9b8c0c7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 shutdown_connections 2024-01-11T07:08:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.381+0000 7f9b9303e700 1 -- 172.21.15.7:0/2262222511 wait complete. 2024-01-11T07:08:55.394 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:08:55.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:55.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.526+0000 7faf55fcb700 1 Processor -- start 2024-01-11T07:08:55.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.526+0000 7faf55fcb700 1 -- start start 2024-01-11T07:08:55.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf55fcb700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf50159c90 0x7faf500b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf55fcb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf50069320 0x7faf500b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf55fcb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500ce9b0 con 0x7faf5015a060 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf55fcb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500cf530 con 0x7faf50069320 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf55fcb700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500cf9c0 con 0x7faf50159c90 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf4ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf50069320 0x7faf500b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf4effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faf5015a060 0x7faf500b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49148/0 (socket says 172.21.15.7:49148) 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf4effd700 1 -- 172.21.15.7:0/175184522 learned_addr learned my addr 172.21.15.7:0/175184522 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf4f7fe700 1 --2- 172.21.15.7:0/175184522 >> v2:172.21.15.113:3300/0 conn(0x7faf50159c90 0x7faf500b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf4ffff700 1 -- 172.21.15.7:0/175184522 >> v2:172.21.15.113:3300/0 conn(0x7faf50159c90 msgr2=0x7faf500b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf4ffff700 1 --2- 172.21.15.7:0/175184522 >> v2:172.21.15.113:3300/0 conn(0x7faf50159c90 0x7faf500b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.527+0000 7faf4ffff700 1 -- 172.21.15.7:0/175184522 >> v1:172.21.15.7:6789/0 conn(0x7faf5015a060 legacy=0x7faf500b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf4ffff700 1 -- 172.21.15.7:0/175184522 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500cec00 con 0x7faf50069320 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf4e7fc700 1 -- 172.21.15.7:0/175184522 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585379734 0 0) 0x7faf500ce9b0 con 0x7faf5015a060 2024-01-11T07:08:55.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf4ffff700 1 --2- 172.21.15.7:0/175184522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50069320 0x7faf500b3520 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faf5005f2e0 tx=0x7faf48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272364f76ce6817d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf4e7fc700 1 -- 172.21.15.7:0/175184522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800e070 con 0x7faf50069320 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf4e7fc700 1 -- 172.21.15.7:0/175184522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4800bc30 con 0x7faf50069320 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf4e7fc700 1 -- 172.21.15.7:0/175184522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48012430 con 0x7faf50069320 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf55fcb700 1 -- 172.21.15.7:0/175184522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50069320 msgr2=0x7faf500b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf55fcb700 1 --2- 172.21.15.7:0/175184522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50069320 0x7faf500b3520 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faf5005f2e0 tx=0x7faf48009780 comp rx=0 tx=0).stop 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf55fcb700 1 -- 172.21.15.7:0/175184522 shutdown_connections 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf55fcb700 1 --2- 172.21.15.7:0/175184522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf50069320 0x7faf500b3520 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf55fcb700 1 --2- 172.21.15.7:0/175184522 >> v2:172.21.15.113:3300/0 conn(0x7faf50159c90 0x7faf500b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.528+0000 7faf55fcb700 1 -- 172.21.15.7:0/175184522 >> 172.21.15.7:0/175184522 conn(0x7faf50071100 msgr2=0x7faf5006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.529+0000 7faf55fcb700 1 -- 172.21.15.7:0/175184522 shutdown_connections 2024-01-11T07:08:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.529+0000 7faf55fcb700 1 -- 172.21.15.7:0/175184522 wait complete. 2024-01-11T07:08:55.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.530+0000 7faf55fcb700 1 Processor -- start 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.530+0000 7faf55fcb700 1 -- start start 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.531+0000 7faf55fcb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf50069320 0x7faf50155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.531+0000 7faf55fcb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf50159c90 0x7faf50169e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.531+0000 7faf55fcb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5015a060 0x7faf500af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.531+0000 7faf55fcb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5016b4a0 con 0x7faf5015a060 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.531+0000 7faf55fcb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50155780 con 0x7faf50159c90 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.531+0000 7faf55fcb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5016a890 con 0x7faf50069320 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5015a060 0x7faf500af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5015a060 0x7faf500af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51190/0 (socket says 172.21.15.7:51190) 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 -- 172.21.15.7:0/3555719586 learned_addr learned my addr 172.21.15.7:0/3555719586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4effd700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf50159c90 0x7faf50169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4f7fe700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf50069320 0x7faf50155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 -- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf50069320 msgr2=0x7faf50155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf50069320 0x7faf50155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 -- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf50159c90 msgr2=0x7faf50169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf50159c90 0x7faf50169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 -- 172.21.15.7:0/3555719586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48009a20 con 0x7faf5015a060 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf4ffff700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5015a060 0x7faf500af3c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faf5006a1d0 tx=0x7faf48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf377fe700 1 -- 172.21.15.7:0/3555719586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48012e60 con 0x7faf5015a060 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf377fe700 1 -- 172.21.15.7:0/3555719586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4801d630 con 0x7faf5015a060 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5016ab70 con 0x7faf5015a060 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.532+0000 7faf377fe700 1 -- 172.21.15.7:0/3555719586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4801d9f0 con 0x7faf5015a060 2024-01-11T07:08:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.533+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5016b130 con 0x7faf5015a060 2024-01-11T07:08:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.533+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5016b650 con 0x7faf5015a060 2024-01-11T07:08:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.534+0000 7faf377fe700 1 -- 172.21.15.7:0/3555719586 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faf48024070 con 0x7faf5015a060 2024-01-11T07:08:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.534+0000 7faf377fe700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faf3c066f30 0x7faf3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.534+0000 7faf377fe700 1 -- 172.21.15.7:0/3555719586 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7faf48086d70 con 0x7faf5015a060 2024-01-11T07:08:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.535+0000 7faf4f7fe700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faf3c066f30 0x7faf3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.535+0000 7faf4f7fe700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faf3c066f30 0x7faf3c0693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf40006fd0 tx=0x7faf40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.539+0000 7faf377fe700 1 -- 172.21.15.7:0/3555719586 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf5016b650 con 0x7faf5015a060 2024-01-11T07:08:55.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.791+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf5016b650 con 0x7faf5015a060 2024-01-11T07:08:55.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.791+0000 7faf377fe700 1 -- 172.21.15.7:0/3555719586 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7faf5016b650 con 0x7faf5015a060 2024-01-11T07:08:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faf3c066f30 msgr2=0x7faf3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faf3c066f30 0x7faf3c0693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf40006fd0 tx=0x7faf40008040 comp rx=0 tx=0).stop 2024-01-11T07:08:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5015a060 msgr2=0x7faf500af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5015a060 0x7faf500af3c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faf5006a1d0 tx=0x7faf48019000 comp rx=0 tx=0).stop 2024-01-11T07:08:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 shutdown_connections 2024-01-11T07:08:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faf3c066f30 0x7faf3c0693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5015a060 0x7faf500af3c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faf50069320 0x7faf50155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 --2- 172.21.15.7:0/3555719586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf50159c90 0x7faf50169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 >> 172.21.15.7:0/3555719586 conn(0x7faf50071100 msgr2=0x7faf501570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 shutdown_connections 2024-01-11T07:08:55.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.793+0000 7faf55fcb700 1 -- 172.21.15.7:0/3555719586 wait complete. 2024-01-11T07:08:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:55.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:08:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.941+0000 7f4cb632d700 1 Processor -- start 2024-01-11T07:08:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.941+0000 7f4cb632d700 1 -- start start 2024-01-11T07:08:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.941+0000 7f4cb632d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cb01579d0 0x7f4cb015c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.941+0000 7f4cb632d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4cb0157da0 0x7f4cb00cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.941+0000 7f4cb632d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb006d270 con 0x7f4cb00ce220 2024-01-11T07:08:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.941+0000 7f4cb632d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb00cda40 con 0x7f4cb01579d0 2024-01-11T07:08:55.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.941+0000 7f4cb632d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb00cd5b0 con 0x7f4cb0157da0 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cb01579d0 0x7f4cb015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4cb48ca700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4cb00ce220 0x7f4cb006ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49156/0 (socket says 172.21.15.7:49156) 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4cb48ca700 1 -- 172.21.15.7:0/870220971 learned_addr learned my addr 172.21.15.7:0/870220971 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caf7fe700 1 --2- 172.21.15.7:0/870220971 >> v2:172.21.15.113:3300/0 conn(0x7f4cb0157da0 0x7f4cb00cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caffff700 1 -- 172.21.15.7:0/870220971 >> v2:172.21.15.113:3300/0 conn(0x7f4cb0157da0 msgr2=0x7f4cb00cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caffff700 1 --2- 172.21.15.7:0/870220971 >> v2:172.21.15.113:3300/0 conn(0x7f4cb0157da0 0x7f4cb00cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caffff700 1 -- 172.21.15.7:0/870220971 >> v1:172.21.15.7:6789/0 conn(0x7f4cb00ce220 legacy=0x7f4cb006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caeffd700 1 -- 172.21.15.7:0/870220971 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905964351 0 0) 0x7f4cb006d270 con 0x7f4cb00ce220 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caffff700 1 -- 172.21.15.7:0/870220971 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb00ce610 con 0x7f4cb01579d0 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caffff700 1 --2- 172.21.15.7:0/870220971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb015c100 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00b0ff0 tx=0x7f4ca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f663c75a48444c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caeffd700 1 -- 172.21.15.7:0/870220971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0011630 con 0x7f4cb01579d0 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caeffd700 1 -- 172.21.15.7:0/870220971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca0011790 con 0x7f4cb01579d0 2024-01-11T07:08:55.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.942+0000 7f4caeffd700 1 -- 172.21.15.7:0/870220971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0011b30 con 0x7f4cb01579d0 2024-01-11T07:08:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.943+0000 7f4cb632d700 1 -- 172.21.15.7:0/870220971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 msgr2=0x7f4cb015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.943+0000 7f4cb632d700 1 --2- 172.21.15.7:0/870220971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb015c100 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00b0ff0 tx=0x7f4ca00091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.943+0000 7f4cb632d700 1 -- 172.21.15.7:0/870220971 shutdown_connections 2024-01-11T07:08:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.943+0000 7f4cb632d700 1 --2- 172.21.15.7:0/870220971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb015c100 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.943+0000 7f4cb632d700 1 --2- 172.21.15.7:0/870220971 >> v2:172.21.15.113:3300/0 conn(0x7f4cb0157da0 0x7f4cb00cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.943+0000 7f4cb632d700 1 -- 172.21.15.7:0/870220971 >> 172.21.15.7:0/870220971 conn(0x7f4cb006ae00 msgr2=0x7f4cb006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.943+0000 7f4cb632d700 1 -- 172.21.15.7:0/870220971 shutdown_connections 2024-01-11T07:08:55.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.943+0000 7f4cb632d700 1 -- 172.21.15.7:0/870220971 wait complete. 2024-01-11T07:08:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.945+0000 7f4cb632d700 1 Processor -- start 2024-01-11T07:08:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.945+0000 7f4cb632d700 1 -- start start 2024-01-11T07:08:55.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4cb632d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb00ce220 0x7f4cb0152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4cb632d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb00c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4cb632d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb0157da0 0x7f4cb00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4cb632d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00c85c0 con 0x7f4cb01579d0 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4cb632d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00c6550 con 0x7f4cb0157da0 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4cb632d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0152ed0 con 0x7f4cb00ce220 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4caf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb00c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4caf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb00c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51206/0 (socket says 172.21.15.7:51206) 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4caf7fe700 1 -- 172.21.15.7:0/1898574009 learned_addr learned my addr 172.21.15.7:0/1898574009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4cb48ca700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb0157da0 0x7f4cb00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4caffff700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb00ce220 0x7f4cb0152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4caf7fe700 1 -- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb00ce220 msgr2=0x7f4cb0152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4caf7fe700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb00ce220 0x7f4cb0152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4caf7fe700 1 -- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb0157da0 msgr2=0x7f4cb00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.946+0000 7f4caf7fe700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb0157da0 0x7f4cb00b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.947+0000 7f4caf7fe700 1 -- 172.21.15.7:0/1898574009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca0009420 con 0x7f4cb01579d0 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.947+0000 7f4cb48ca700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb0157da0 0x7f4cb00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.947+0000 7f4caf7fe700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb00c7ce0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400e930 tx=0x7f4ca40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.947+0000 7f4c97fff700 1 -- 172.21.15.7:0/1898574009 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400eed0 con 0x7f4cb01579d0 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.947+0000 7f4c97fff700 1 -- 172.21.15.7:0/1898574009 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca400c850 con 0x7f4cb01579d0 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.947+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb00c5c20 con 0x7f4cb01579d0 2024-01-11T07:08:55.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.947+0000 7f4c97fff700 1 -- 172.21.15.7:0/1898574009 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400cbf0 con 0x7f4cb01579d0 2024-01-11T07:08:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.947+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb00c61e0 con 0x7f4cb01579d0 2024-01-11T07:08:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.948+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb00c66e0 con 0x7f4cb01579d0 2024-01-11T07:08:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.948+0000 7f4c97fff700 1 -- 172.21.15.7:0/1898574009 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4ca4013470 con 0x7f4cb01579d0 2024-01-11T07:08:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.949+0000 7f4c97fff700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c9c066f30 0x7f4c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.949+0000 7f4c97fff700 1 -- 172.21.15.7:0/1898574009 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4ca4084730 con 0x7f4cb01579d0 2024-01-11T07:08:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.949+0000 7f4caffff700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c9c066f30 0x7f4c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:55.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.949+0000 7f4caffff700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c9c066f30 0x7f4c9c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0003080 tx=0x7f4ca0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:55.953+0000 7f4c97fff700 1 -- 172.21.15.7:0/1898574009 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ca4051960 con 0x7f4cb01579d0 2024-01-11T07:08:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.206+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cb004c3c0 con 0x7f4cb01579d0 2024-01-11T07:08:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.207+0000 7f4c97fff700 1 -- 172.21.15.7:0/1898574009 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f4ca4054f80 con 0x7f4cb01579d0 2024-01-11T07:08:56.206 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:56.207 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:08:19.620845+0000","modified":"2024-01-11T07:08:23.209392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2351618262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2351618262},{"type":"v1","addr":"172.21.15.7:6835","nonce":2351618262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c9c066f30 msgr2=0x7f4c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c9c066f30 0x7f4c9c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0003080 tx=0x7f4ca0017000 comp rx=0 tx=0).stop 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 msgr2=0x7f4cb00c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb00c7ce0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400e930 tx=0x7f4ca40091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 shutdown_connections 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c9c066f30 0x7f4c9c0693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cb0157da0 0x7f4cb00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cb00ce220 0x7f4cb0152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 --2- 172.21.15.7:0/1898574009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cb01579d0 0x7f4cb00c7ce0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 >> 172.21.15.7:0/1898574009 conn(0x7f4cb006ae00 msgr2=0x7f4cb00c4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 shutdown_connections 2024-01-11T07:08:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.209+0000 7f4cb632d700 1 -- 172.21.15.7:0/1898574009 wait complete. 2024-01-11T07:08:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-01-11T07:08:56.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:08:56.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.355+0000 7f6b4d8d6700 1 Processor -- start 2024-01-11T07:08:56.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.355+0000 7f6b4d8d6700 1 -- start start 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.356+0000 7f6b4d8d6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b4806e340 0x7f6b4815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.356+0000 7f6b4d8d6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b480d0220 con 0x7f6b48159d80 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.356+0000 7f6b4d8d6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b480cf610 con 0x7f6b4815b3f0 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.356+0000 7f6b4d8d6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480d06b0 con 0x7f6b4806e340 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.356+0000 7f6b46ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6b48159d80 0x7f6b4806dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49168/0 (socket says 172.21.15.7:49168) 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.356+0000 7f6b46ffd700 1 -- 172.21.15.7:0/106426092 learned_addr learned my addr 172.21.15.7:0/106426092 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.356+0000 7f6b3ffff700 1 --2- 172.21.15.7:0/106426092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b4806e340 0x7f6b4815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b467fc700 1 -- 172.21.15.7:0/106426092 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587701127 0 0) 0x7f6b480d0220 con 0x7f6b48159d80 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b467fc700 1 -- 172.21.15.7:0/106426092 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b34003570 con 0x7f6b48159d80 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b467fc700 1 -- 172.21.15.7:0/106426092 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222120554 0 0) 0x7f6b480cf610 con 0x7f6b4815b3f0 2024-01-11T07:08:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b467fc700 1 -- 172.21.15.7:0/106426092 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b480d0220 con 0x7f6b4815b3f0 2024-01-11T07:08:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b3ffff700 1 -- 172.21.15.7:0/106426092 >> v1:172.21.15.113:6789/0 conn(0x7f6b4815b3f0 legacy=0x7f6b4815b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b3ffff700 1 -- 172.21.15.7:0/106426092 >> v1:172.21.15.7:6789/0 conn(0x7f6b48159d80 legacy=0x7f6b4806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b3ffff700 1 -- 172.21.15.7:0/106426092 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480cf8f0 con 0x7f6b4806e340 2024-01-11T07:08:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b3ffff700 1 --2- 172.21.15.7:0/106426092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b4806e340 0x7f6b4815aeb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009910 tx=0x7f6b300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b88ef6170143b226 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b467fc700 1 -- 172.21.15.7:0/106426092 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30011630 con 0x7f6b4806e340 2024-01-11T07:08:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b467fc700 1 -- 172.21.15.7:0/106426092 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b30011790 con 0x7f6b4806e340 2024-01-11T07:08:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.357+0000 7f6b467fc700 1 -- 172.21.15.7:0/106426092 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30011b30 con 0x7f6b4806e340 2024-01-11T07:08:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.358+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/106426092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b4806e340 msgr2=0x7f6b4815aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.358+0000 7f6b4d8d6700 1 --2- 172.21.15.7:0/106426092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b4806e340 0x7f6b4815aeb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009910 tx=0x7f6b300091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.358+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/106426092 shutdown_connections 2024-01-11T07:08:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.358+0000 7f6b4d8d6700 1 --2- 172.21.15.7:0/106426092 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b4806e340 0x7f6b4815aeb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.358+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/106426092 >> 172.21.15.7:0/106426092 conn(0x7f6b48071130 msgr2=0x7f6b480b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.358+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/106426092 shutdown_connections 2024-01-11T07:08:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.358+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/106426092 wait complete. 2024-01-11T07:08:56.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.359+0000 7f6b4d8d6700 1 Processor -- start 2024-01-11T07:08:56.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.359+0000 7f6b4d8d6700 1 -- start start 2024-01-11T07:08:56.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b4d8d6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b4806e340 0x7f6b48150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b4d8d6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b48159d80 0x7f6b4815d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b4d8d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b4815b3f0 0x7f6b480b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b4d8d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4816ad20 con 0x7f6b4815b3f0 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b4d8d6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4815e0c0 con 0x7f6b4806e340 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b4d8d6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4816a6d0 con 0x7f6b48159d80 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b4815b3f0 0x7f6b480b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b4815b3f0 0x7f6b480b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51222/0 (socket says 172.21.15.7:51222) 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b477fe700 1 -- 172.21.15.7:0/349983476 learned_addr learned my addr 172.21.15.7:0/349983476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b46ffd700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b4806e340 0x7f6b48150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.360+0000 7f6b3ffff700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b48159d80 0x7f6b4815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b477fe700 1 -- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b48159d80 msgr2=0x7f6b4815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b477fe700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b48159d80 0x7f6b4815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b477fe700 1 -- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b4806e340 msgr2=0x7f6b48150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b477fe700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b4806e340 0x7f6b48150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b477fe700 1 -- 172.21.15.7:0/349983476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b30009420 con 0x7f6b4815b3f0 2024-01-11T07:08:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b477fe700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b4815b3f0 0x7f6b480b5c60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000c860 tx=0x7f6b40002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b3effd700 1 -- 172.21.15.7:0/349983476 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40002030 con 0x7f6b4815b3f0 2024-01-11T07:08:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b3effd700 1 -- 172.21.15.7:0/349983476 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b400021b0 con 0x7f6b4815b3f0 2024-01-11T07:08:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4816a9b0 con 0x7f6b4815b3f0 2024-01-11T07:08:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b3effd700 1 -- 172.21.15.7:0/349983476 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40012630 con 0x7f6b4815b3f0 2024-01-11T07:08:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.361+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b48150b60 con 0x7f6b4815b3f0 2024-01-11T07:08:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.362+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b48089080 con 0x7f6b4815b3f0 2024-01-11T07:08:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.362+0000 7f6b3effd700 1 -- 172.21.15.7:0/349983476 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6b40012a20 con 0x7f6b4815b3f0 2024-01-11T07:08:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.363+0000 7f6b3effd700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b28066f30 0x7f6b280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.363+0000 7f6b3effd700 1 -- 172.21.15.7:0/349983476 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6b40084690 con 0x7f6b4815b3f0 2024-01-11T07:08:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.363+0000 7f6b46ffd700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b28066f30 0x7f6b280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.364+0000 7f6b46ffd700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b28066f30 0x7f6b280693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6b48154870 tx=0x7f6b38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.367+0000 7f6b3effd700 1 -- 172.21.15.7:0/349983476 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b40051820 con 0x7f6b4815b3f0 2024-01-11T07:08:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.570+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b4804c3c0 con 0x7f6b4815b3f0 2024-01-11T07:08:56.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.571+0000 7f6b3effd700 1 -- 172.21.15.7:0/349983476 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f6b40054e40 con 0x7f6b4815b3f0 2024-01-11T07:08:56.571 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:08:56.571 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:08:49.209265+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:08:17.698588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-11T07:08:18.590636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.1b","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:08:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b28066f30 msgr2=0x7f6b280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b28066f30 0x7f6b280693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6b48154870 tx=0x7f6b38009040 comp rx=0 tx=0).stop 2024-01-11T07:08:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b4815b3f0 msgr2=0x7f6b480b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b4815b3f0 0x7f6b480b5c60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000c860 tx=0x7f6b40002700 comp rx=0 tx=0).stop 2024-01-11T07:08:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 shutdown_connections 2024-01-11T07:08:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b28066f30 0x7f6b280693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b4815b3f0 0x7f6b480b5c60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b4806e340 0x7f6b48150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 --2- 172.21.15.7:0/349983476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b48159d80 0x7f6b4815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 >> 172.21.15.7:0/349983476 conn(0x7f6b48071130 msgr2=0x7f6b480b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 shutdown_connections 2024-01-11T07:08:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.573+0000 7f6b4d8d6700 1 -- 172.21.15.7:0/349983476 wait complete. 2024-01-11T07:08:56.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:08:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.718+0000 7fc1fee54700 1 Processor -- start 2024-01-11T07:08:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.718+0000 7fc1fee54700 1 -- start start 2024-01-11T07:08:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1fee54700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f81579d0 0x7fc1f815c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1fee54700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1f8157da0 0x7fc1f80cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1fee54700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f806d270 con 0x7fc1f80ce220 2024-01-11T07:08:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1fee54700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f80ce6a0 con 0x7fc1f8157da0 2024-01-11T07:08:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1fee54700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80cda40 con 0x7fc1f81579d0 2024-01-11T07:08:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1f7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1f8157da0 0x7fc1f80cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1f7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1f8157da0 0x7fc1f80cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51226/0 (socket says 172.21.15.7:51226) 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1f7fff700 1 -- 172.21.15.7:0/2419786974 learned_addr learned my addr 172.21.15.7:0/2419786974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1fcbf0700 1 --2- 172.21.15.7:0/2419786974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f81579d0 0x7fc1f815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1f7fff700 1 -- 172.21.15.7:0/2419786974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f81579d0 msgr2=0x7fc1f815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1f7fff700 1 --2- 172.21.15.7:0/2419786974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f81579d0 0x7fc1f815c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1f7fff700 1 -- 172.21.15.7:0/2419786974 >> v1:172.21.15.113:6789/0 conn(0x7fc1f80ce220 legacy=0x7fc1f806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.719+0000 7fc1f7fff700 1 -- 172.21.15.7:0/2419786974 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f80cd5b0 con 0x7fc1f8157da0 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.720+0000 7fc1f7fff700 1 --2- 172.21.15.7:0/2419786974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f8157da0 0x7fc1f80cdce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40099b0 tx=0x7fc1e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f07e3de81e9014 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.720+0000 7fc1f77fe700 1 -- 172.21.15.7:0/2419786974 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e400b860 con 0x7fc1f8157da0 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.720+0000 7fc1f77fe700 1 -- 172.21.15.7:0/2419786974 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e400b9c0 con 0x7fc1f8157da0 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.720+0000 7fc1f77fe700 1 -- 172.21.15.7:0/2419786974 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e400bd60 con 0x7fc1f8157da0 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.720+0000 7fc1fee54700 1 -- 172.21.15.7:0/2419786974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f8157da0 msgr2=0x7fc1f80cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.720+0000 7fc1fee54700 1 --2- 172.21.15.7:0/2419786974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f8157da0 0x7fc1f80cdce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40099b0 tx=0x7fc1e4009250 comp rx=0 tx=0).stop 2024-01-11T07:08:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.721+0000 7fc1fee54700 1 -- 172.21.15.7:0/2419786974 shutdown_connections 2024-01-11T07:08:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.721+0000 7fc1fee54700 1 --2- 172.21.15.7:0/2419786974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f81579d0 0x7fc1f815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.721+0000 7fc1fee54700 1 --2- 172.21.15.7:0/2419786974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f8157da0 0x7fc1f80cdce0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.721+0000 7fc1fee54700 1 -- 172.21.15.7:0/2419786974 >> 172.21.15.7:0/2419786974 conn(0x7fc1f806ae00 msgr2=0x7fc1f806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.721+0000 7fc1fee54700 1 -- 172.21.15.7:0/2419786974 shutdown_connections 2024-01-11T07:08:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.721+0000 7fc1fee54700 1 -- 172.21.15.7:0/2419786974 wait complete. 2024-01-11T07:08:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.722+0000 7fc1fee54700 1 Processor -- start 2024-01-11T07:08:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.722+0000 7fc1fee54700 1 -- start start 2024-01-11T07:08:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fee54700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f80ce220 0x7fc1f815d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fee54700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1f81579d0 0x7fc1f8149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fee54700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f8157da0 0x7fc1f80c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fee54700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80c79d0 con 0x7fc1f80ce220 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fee54700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80c6dc0 con 0x7fc1f81579d0 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fee54700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80c70a0 con 0x7fc1f8157da0 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f80ce220 0x7fc1f815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f80ce220 0x7fc1f815d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51236/0 (socket says 172.21.15.7:51236) 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 -- 172.21.15.7:0/1859288183 learned_addr learned my addr 172.21.15.7:0/1859288183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1f7fff700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1f81579d0 0x7fc1f8149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fd3f1700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f8157da0 0x7fc1f80c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 -- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f8157da0 msgr2=0x7fc1f80c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f8157da0 0x7fc1f80c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 -- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1f81579d0 msgr2=0x7fc1f8149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1f81579d0 0x7fc1f8149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 -- 172.21.15.7:0/1859288183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40094f0 con 0x7fc1f80ce220 2024-01-11T07:08:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.723+0000 7fc1fcbf0700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f80ce220 0x7fc1f815d670 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00d890 tx=0x7fc1ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.724+0000 7fc1f4ff9700 1 -- 172.21.15.7:0/1859288183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec0125f0 con 0x7fc1f80ce220 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.724+0000 7fc1f4ff9700 1 -- 172.21.15.7:0/1859288183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec012770 con 0x7fc1f80ce220 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.724+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f80c7380 con 0x7fc1f80ce220 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.724+0000 7fc1f4ff9700 1 -- 172.21.15.7:0/1859288183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec012b30 con 0x7fc1f80ce220 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.724+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f815dbb0 con 0x7fc1f80ce220 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.725+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f804c3c0 con 0x7fc1f80ce220 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.725+0000 7fc1f4ff9700 1 -- 172.21.15.7:0/1859288183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc1ec01e460 con 0x7fc1f80ce220 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.726+0000 7fc1f4ff9700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1dc066f30 0x7fc1dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.726+0000 7fc1f4ff9700 1 -- 172.21.15.7:0/1859288183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fc1ec084b40 con 0x7fc1f80ce220 2024-01-11T07:08:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.726+0000 7fc1f7fff700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1dc066f30 0x7fc1dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:56.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.727+0000 7fc1f7fff700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1dc066f30 0x7fc1dc0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40099b0 tx=0x7fc1e4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.730+0000 7fc1f4ff9700 1 -- 172.21.15.7:0/1859288183 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1ec051d70 con 0x7fc1f80ce220 2024-01-11T07:08:56.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.979+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc1f80c7cb0 con 0x7fc1f80ce220 2024-01-11T07:08:56.979 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:08:56.979+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:08:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.983+0000 7fc1f4ff9700 1 -- 172.21.15.7:0/1859288183 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc1ec055390 con 0x7fc1f80ce220 2024-01-11T07:08:56.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1dc066f30 msgr2=0x7fc1dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1dc066f30 0x7fc1dc0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40099b0 tx=0x7fc1e4017000 comp rx=0 tx=0).stop 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f80ce220 msgr2=0x7fc1f815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f80ce220 0x7fc1f815d670 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00d890 tx=0x7fc1ec008040 comp rx=0 tx=0).stop 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 shutdown_connections 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc1dc066f30 0x7fc1dc0693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1f8157da0 0x7fc1f80c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1f80ce220 0x7fc1f815d670 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 --2- 172.21.15.7:0/1859288183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1f81579d0 0x7fc1f8149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 >> 172.21.15.7:0/1859288183 conn(0x7fc1f806ae00 msgr2=0x7fc1f8153360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 shutdown_connections 2024-01-11T07:08:56.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:56.986+0000 7fc1fee54700 1 -- 172.21.15.7:0/1859288183 wait complete. 2024-01-11T07:08:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:08:56.988 INFO:tasks.ceph.mds.c.smithi007.stderr: -9137> 2024-01-11T07:08:25.748+0000 7f46b3937700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:08:57.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:08:57.028 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:08:57.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.139+0000 7ff3d9c3d700 1 Processor -- start 2024-01-11T07:08:57.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.140+0000 7ff3d9c3d700 1 -- start start 2024-01-11T07:08:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d9c3d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3d406dd00 0x7ff3d40c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d9c3d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406e0d0 0x7ff3d4159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:57.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d9c3d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d415b770 con 0x7ff3d4159ff0 2024-01-11T07:08:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d9c3d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d415abf0 con 0x7ff3d406dd00 2024-01-11T07:08:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d9c3d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d415aed0 con 0x7ff3d406e0d0 2024-01-11T07:08:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3d406dd00 0x7ff3d40c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3d406dd00 0x7ff3d40c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51254/0 (socket says 172.21.15.7:51254) 2024-01-11T07:08:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d37fe700 1 -- 172.21.15.7:0/4210645977 learned_addr learned my addr 172.21.15.7:0/4210645977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:57.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d2ffd700 1 --2- 172.21.15.7:0/4210645977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406e0d0 0x7ff3d4159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d37fe700 1 -- 172.21.15.7:0/4210645977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406e0d0 msgr2=0x7ff3d4159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d37fe700 1 --2- 172.21.15.7:0/4210645977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406e0d0 0x7ff3d4159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d37fe700 1 -- 172.21.15.7:0/4210645977 >> v1:172.21.15.113:6789/0 conn(0x7ff3d4159ff0 legacy=0x7ff3d40c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d27fc700 1 -- 172.21.15.7:0/4210645977 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129858917 0 0) 0x7ff3d415b770 con 0x7ff3d4159ff0 2024-01-11T07:08:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.141+0000 7ff3d37fe700 1 -- 172.21.15.7:0/4210645977 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d415b1b0 con 0x7ff3d406dd00 2024-01-11T07:08:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d37fe700 1 --2- 172.21.15.7:0/4210645977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406dd00 0x7ff3d40c98d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40099b0 tx=0x7ff3c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171145ca81b52559 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d27fc700 1 -- 172.21.15.7:0/4210645977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4009ea0 con 0x7ff3d406dd00 2024-01-11T07:08:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d27fc700 1 -- 172.21.15.7:0/4210645977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c400b860 con 0x7ff3d406dd00 2024-01-11T07:08:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d27fc700 1 -- 172.21.15.7:0/4210645977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c400bc00 con 0x7ff3d406dd00 2024-01-11T07:08:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/4210645977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406dd00 msgr2=0x7ff3d40c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:57.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/4210645977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406dd00 0x7ff3d40c98d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40099b0 tx=0x7ff3c4009250 comp rx=0 tx=0).stop 2024-01-11T07:08:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/4210645977 shutdown_connections 2024-01-11T07:08:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/4210645977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406dd00 0x7ff3d40c98d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.142+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/4210645977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406e0d0 0x7ff3d4159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.143+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/4210645977 >> 172.21.15.7:0/4210645977 conn(0x7ff3d40680e0 msgr2=0x7ff3d40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.143+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/4210645977 shutdown_connections 2024-01-11T07:08:57.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.143+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/4210645977 wait complete. 2024-01-11T07:08:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.144+0000 7ff3d9c3d700 1 Processor -- start 2024-01-11T07:08:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.144+0000 7ff3d9c3d700 1 -- start start 2024-01-11T07:08:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d9c3d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406dd00 0x7ff3d40cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d9c3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406e0d0 0x7ff3d40b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:57.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d9c3d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4159ff0 0x7ff3d414f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d9c3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4151490 con 0x7ff3d406e0d0 2024-01-11T07:08:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d9c3d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4151730 con 0x7ff3d4159ff0 2024-01-11T07:08:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d9c3d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d40b1af0 con 0x7ff3d406dd00 2024-01-11T07:08:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406e0d0 0x7ff3d40b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406e0d0 0x7ff3d40b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51270/0 (socket says 172.21.15.7:51270) 2024-01-11T07:08:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 -- 172.21.15.7:0/2314738654 learned_addr learned my addr 172.21.15.7:0/2314738654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d37fe700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406dd00 0x7ff3d40cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d3fff700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4159ff0 0x7ff3d414f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 -- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406dd00 msgr2=0x7ff3d40cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406dd00 0x7ff3d40cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 -- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4159ff0 msgr2=0x7ff3d414f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4159ff0 0x7ff3d414f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 -- 172.21.15.7:0/2314738654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40094f0 con 0x7ff3d406e0d0 2024-01-11T07:08:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.145+0000 7ff3d2ffd700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406e0d0 0x7ff3d40b0770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800ece0 tx=0x7ff3c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.146+0000 7ff3bb7fe700 1 -- 172.21.15.7:0/2314738654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800c7f0 con 0x7ff3d406e0d0 2024-01-11T07:08:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.146+0000 7ff3bb7fe700 1 -- 172.21.15.7:0/2314738654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c800c950 con 0x7ff3d406e0d0 2024-01-11T07:08:57.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.146+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d40b1250 con 0x7ff3d406e0d0 2024-01-11T07:08:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.146+0000 7ff3bb7fe700 1 -- 172.21.15.7:0/2314738654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800ccf0 con 0x7ff3d406e0d0 2024-01-11T07:08:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.146+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d40c74f0 con 0x7ff3d406e0d0 2024-01-11T07:08:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.147+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d404c3c0 con 0x7ff3d406e0d0 2024-01-11T07:08:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.147+0000 7ff3bb7fe700 1 -- 172.21.15.7:0/2314738654 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff3c8013680 con 0x7ff3d406e0d0 2024-01-11T07:08:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.148+0000 7ff3bb7fe700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3c0066f30 0x7ff3c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.148+0000 7ff3bb7fe700 1 -- 172.21.15.7:0/2314738654 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5849+0+0 (secure 0 0 0) 0x7ff3c8084f90 con 0x7ff3d406e0d0 2024-01-11T07:08:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.148+0000 7ff3d37fe700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3c0066f30 0x7ff3c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.149+0000 7ff3d37fe700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3c0066f30 0x7ff3c00693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40099b0 tx=0x7ff3c4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.152+0000 7ff3bb7fe700 1 -- 172.21.15.7:0/2314738654 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3c80521f0 con 0x7ff3d406e0d0 2024-01-11T07:08:57.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.404+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff3d4151870 con 0x7ff3d406e0d0 2024-01-11T07:08:57.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.985+0000 7ff3bb7fe700 1 -- 172.21.15.7:0/2314738654 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff3c8055810 con 0x7ff3d406e0d0 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3c0066f30 msgr2=0x7ff3c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3c0066f30 0x7ff3c00693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40099b0 tx=0x7ff3c4017000 comp rx=0 tx=0).stop 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406e0d0 msgr2=0x7ff3d40b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406e0d0 0x7ff3d40b0770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800ece0 tx=0x7ff3c80091a0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 shutdown_connections 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3c0066f30 0x7ff3c00693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d4159ff0 0x7ff3d414f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3d406dd00 0x7ff3d40cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 --2- 172.21.15.7:0/2314738654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3d406e0d0 0x7ff3d40b0770 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 >> 172.21.15.7:0/2314738654 conn(0x7ff3d40680e0 msgr2=0x7ff3d40af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 shutdown_connections 2024-01-11T07:08:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:57.988+0000 7ff3d9c3d700 1 -- 172.21.15.7:0/2314738654 wait complete. 2024-01-11T07:08:58.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:08:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.187+0000 7f9eefd0f700 1 Processor -- start 2024-01-11T07:08:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.187+0000 7f9eefd0f700 1 -- start start 2024-01-11T07:08:58.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.188+0000 7f9eefd0f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ee8158540 0x7f9ee80aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:58.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.188+0000 7f9eefd0f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee81590a0 con 0x7f9ee80acfb0 2024-01-11T07:08:58.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.188+0000 7f9eefd0f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee8158ae0 con 0x7f9ee8063480 2024-01-11T07:08:58.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.188+0000 7f9eefd0f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee8158dc0 con 0x7f9ee8158540 2024-01-11T07:08:58.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.188+0000 7f9eee2ac700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ee8158540 0x7f9ee80aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.188+0000 7f9eed2aa700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9ee80acfb0 0x7f9ee80ae740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36958/0 (socket says 172.21.15.7:36958) 2024-01-11T07:08:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eed2aa700 1 -- 172.21.15.7:0/1023972118 learned_addr learned my addr 172.21.15.7:0/1023972118 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:08:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eee2ac700 1 -- 172.21.15.7:0/1023972118 >> v1:172.21.15.113:6789/0 conn(0x7f9ee8063480 legacy=0x7f9ee806e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eee2ac700 1 -- 172.21.15.7:0/1023972118 >> v1:172.21.15.7:6789/0 conn(0x7f9ee80acfb0 legacy=0x7f9ee80ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eee2ac700 1 -- 172.21.15.7:0/1023972118 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee806dc80 con 0x7f9ee8158540 2024-01-11T07:08:58.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eecaa9700 1 -- 172.21.15.7:0/1023972118 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813754718 0 0) 0x7f9ee81590a0 con 0x7f9ee80acfb0 2024-01-11T07:08:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eee2ac700 1 --2- 172.21.15.7:0/1023972118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8158540 0x7f9ee80aee50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009e60 tx=0x7f9ee0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4172f78fbfeb47b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eecaa9700 1 -- 172.21.15.7:0/1023972118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000e070 con 0x7f9ee8158540 2024-01-11T07:08:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eecaa9700 1 -- 172.21.15.7:0/1023972118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee000bc30 con 0x7f9ee8158540 2024-01-11T07:08:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.189+0000 7f9eecaa9700 1 -- 172.21.15.7:0/1023972118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0012430 con 0x7f9ee8158540 2024-01-11T07:08:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.190+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1023972118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8158540 msgr2=0x7f9ee80aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:58.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.190+0000 7f9eefd0f700 1 --2- 172.21.15.7:0/1023972118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8158540 0x7f9ee80aee50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009e60 tx=0x7f9ee0009780 comp rx=0 tx=0).stop 2024-01-11T07:08:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.190+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1023972118 shutdown_connections 2024-01-11T07:08:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.190+0000 7f9eefd0f700 1 --2- 172.21.15.7:0/1023972118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8158540 0x7f9ee80aee50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.190+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1023972118 >> 172.21.15.7:0/1023972118 conn(0x7f9ee80680a0 msgr2=0x7f9ee80cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.190+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1023972118 shutdown_connections 2024-01-11T07:08:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.190+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1023972118 wait complete. 2024-01-11T07:08:58.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.191+0000 7f9eefd0f700 1 Processor -- start 2024-01-11T07:08:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.191+0000 7f9eefd0f700 1 -- start start 2024-01-11T07:08:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eefd0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8063480 0x7f9ee8150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eefd0f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee80acfb0 0x7f9ee80cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eefd0f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee8158540 0x7f9ee8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eefd0f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80cce70 con 0x7f9ee8063480 2024-01-11T07:08:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eefd0f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81513e0 con 0x7f9ee8158540 2024-01-11T07:08:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eefd0f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80c58e0 con 0x7f9ee80acfb0 2024-01-11T07:08:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eedaab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8063480 0x7f9ee8150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eedaab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8063480 0x7f9ee8150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38406/0 (socket says 172.21.15.7:38406) 2024-01-11T07:08:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eedaab700 1 -- 172.21.15.7:0/1254275115 learned_addr learned my addr 172.21.15.7:0/1254275115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eee2ac700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee8158540 0x7f9ee8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eed2aa700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee80acfb0 0x7f9ee80cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eedaab700 1 -- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee80acfb0 msgr2=0x7f9ee80cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eedaab700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee80acfb0 0x7f9ee80cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eedaab700 1 -- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee8158540 msgr2=0x7f9ee8155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eedaab700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee8158540 0x7f9ee8155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eedaab700 1 -- 172.21.15.7:0/1254275115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0009a20 con 0x7f9ee8063480 2024-01-11T07:08:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eee2ac700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee8158540 0x7f9ee8155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.192+0000 7f9eed2aa700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee80acfb0 0x7f9ee80cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:08:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.193+0000 7f9eedaab700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8063480 0x7f9ee8150ea0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400c4e0 tx=0x7f9ee400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.193+0000 7f9eddffb700 1 -- 172.21.15.7:0/1254275115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4011590 con 0x7f9ee8063480 2024-01-11T07:08:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.193+0000 7f9eddffb700 1 -- 172.21.15.7:0/1254275115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee4011710 con 0x7f9ee8063480 2024-01-11T07:08:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.193+0000 7f9eddffb700 1 -- 172.21.15.7:0/1254275115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4011ad0 con 0x7f9ee8063480 2024-01-11T07:08:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.193+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee80c5bc0 con 0x7f9ee8063480 2024-01-11T07:08:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.193+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee80c6180 con 0x7f9ee8063480 2024-01-11T07:08:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.194+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee804c3c0 con 0x7f9ee8063480 2024-01-11T07:08:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.194+0000 7f9eddffb700 1 -- 172.21.15.7:0/1254275115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9ee401b3b0 con 0x7f9ee8063480 2024-01-11T07:08:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.195+0000 7f9eddffb700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9ecc066f30 0x7f9ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.195+0000 7f9eddffb700 1 -- 172.21.15.7:0/1254275115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f9ee4083af0 con 0x7f9ee8063480 2024-01-11T07:08:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.195+0000 7f9eed2aa700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9ecc066f30 0x7f9ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.195+0000 7f9eed2aa700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9ecc066f30 0x7f9ecc0693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9ee816b640 tx=0x7f9ed8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.199+0000 7f9eddffb700 1 -- 172.21.15.7:0/1254275115 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ee4050cd0 con 0x7f9ee8063480 2024-01-11T07:08:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:58.402+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9ee8158320 con 0x7f9ee8063480 2024-01-11T07:08:59.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.244+0000 7f9eddffb700 1 -- 172.21.15.7:0/1254275115 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9ee4017070 con 0x7f9ee8063480 2024-01-11T07:08:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.248+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9ecc066f30 msgr2=0x7f9ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.248+0000 7f9eefd0f700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9ecc066f30 0x7f9ecc0693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9ee816b640 tx=0x7f9ed8009040 comp rx=0 tx=0).stop 2024-01-11T07:08:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.248+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8063480 msgr2=0x7f9ee8150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.248+0000 7f9eefd0f700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8063480 0x7f9ee8150ea0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400c4e0 tx=0x7f9ee400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.248+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 shutdown_connections 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.248+0000 7f9eefd0f700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9ecc066f30 0x7f9ecc0693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.249+0000 7f9eefd0f700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee8158540 0x7f9ee8155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.249+0000 7f9eefd0f700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ee8063480 0x7f9ee8150ea0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.249+0000 7f9eefd0f700 1 --2- 172.21.15.7:0/1254275115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee80acfb0 0x7f9ee80cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.249+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 >> 172.21.15.7:0/1254275115 conn(0x7f9ee80680a0 msgr2=0x7f9ee815a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.249+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 shutdown_connections 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.249+0000 7f9eefd0f700 1 -- 172.21.15.7:0/1254275115 wait complete. 2024-01-11T07:08:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:08:59.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:08:59.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.404+0000 7ff04169a700 1 Processor -- start 2024-01-11T07:08:59.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.405+0000 7ff04169a700 1 -- start start 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.405+0000 7ff04169a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c157620 0x7ff03c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.405+0000 7ff04169a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff03c067610 0x7ff03c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.405+0000 7ff04169a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff03c1579f0 0x7ff03c15de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.405+0000 7ff04169a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03c0cc7f0 con 0x7ff03c1579f0 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.405+0000 7ff04169a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03c068790 con 0x7ff03c067610 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.405+0000 7ff04169a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c067b80 con 0x7ff03c157620 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff03c1579f0 0x7ff03c15de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff03c1579f0 0x7ff03c15de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38414/0 (socket says 172.21.15.7:38414) 2024-01-11T07:08:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 -- 172.21.15.7:0/2327039826 learned_addr learned my addr 172.21.15.7:0/2327039826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03a7fc700 1 --2- 172.21.15.7:0/2327039826 >> v2:172.21.15.113:3300/0 conn(0x7ff03c067610 0x7ff03c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03affd700 1 --2- 172.21.15.7:0/2327039826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c157620 0x7ff03c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 -- 172.21.15.7:0/2327039826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c157620 msgr2=0x7ff03c06d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 --2- 172.21.15.7:0/2327039826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c157620 0x7ff03c06d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 -- 172.21.15.7:0/2327039826 >> v2:172.21.15.113:3300/0 conn(0x7ff03c067610 msgr2=0x7ff03c15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 --2- 172.21.15.7:0/2327039826 >> v2:172.21.15.113:3300/0 conn(0x7ff03c067610 0x7ff03c15d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 -- 172.21.15.7:0/2327039826 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c067e60 con 0x7ff03c1579f0 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff03b7fe700 1 --2- 172.21.15.7:0/2327039826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c15de20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06be50 tx=0x7ff034009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff7e68c78a3e2bd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff039ffb700 1 -- 172.21.15.7:0/2327039826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03400e070 con 0x7ff03c1579f0 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff039ffb700 1 -- 172.21.15.7:0/2327039826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03400b970 con 0x7ff03c1579f0 2024-01-11T07:08:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.406+0000 7ff039ffb700 1 -- 172.21.15.7:0/2327039826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0340092e0 con 0x7ff03c1579f0 2024-01-11T07:08:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 -- 172.21.15.7:0/2327039826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 msgr2=0x7ff03c15de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 --2- 172.21.15.7:0/2327039826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c15de20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06be50 tx=0x7ff034009840 comp rx=0 tx=0).stop 2024-01-11T07:08:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 -- 172.21.15.7:0/2327039826 shutdown_connections 2024-01-11T07:08:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 --2- 172.21.15.7:0/2327039826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c15de20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 --2- 172.21.15.7:0/2327039826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c157620 0x7ff03c06d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 --2- 172.21.15.7:0/2327039826 >> v2:172.21.15.113:3300/0 conn(0x7ff03c067610 0x7ff03c15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 -- 172.21.15.7:0/2327039826 >> 172.21.15.7:0/2327039826 conn(0x7ff03c06ed50 msgr2=0x7ff03c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:08:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 -- 172.21.15.7:0/2327039826 shutdown_connections 2024-01-11T07:08:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.407+0000 7ff04169a700 1 -- 172.21.15.7:0/2327039826 wait complete. 2024-01-11T07:08:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.409+0000 7ff04169a700 1 Processor -- start 2024-01-11T07:08:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.409+0000 7ff04169a700 1 -- start start 2024-01-11T07:08:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.409+0000 7ff04169a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c067610 0x7ff03c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.409+0000 7ff04169a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03c157620 0x7ff03c172790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.409+0000 7ff04169a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.409+0000 7ff04169a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c172cd0 con 0x7ff03c1579f0 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.409+0000 7ff04169a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0c9bc0 con 0x7ff03c157620 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.409+0000 7ff04169a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0c8fb0 con 0x7ff03c067610 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38430/0 (socket says 172.21.15.7:38430) 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 -- 172.21.15.7:0/4095100036 learned_addr learned my addr 172.21.15.7:0/4095100036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03affd700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c067610 0x7ff03c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03a7fc700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03c157620 0x7ff03c172790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 -- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c067610 msgr2=0x7ff03c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c067610 0x7ff03c152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 -- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03c157620 msgr2=0x7ff03c172790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03c157620 0x7ff03c172790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 -- 172.21.15.7:0/4095100036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff034009ae0 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff03b7fe700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c155930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff034009880 tx=0x7ff034018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff032ffd700 1 -- 172.21.15.7:0/4095100036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034008b10 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff032ffd700 1 -- 172.21.15.7:0/4095100036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff034008c70 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c0c9290 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff032ffd700 1 -- 172.21.15.7:0/4095100036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034025460 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.410+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c0c9850 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.411+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff03c0c44c0 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.412+0000 7ff032ffd700 1 -- 172.21.15.7:0/4095100036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff034003040 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.412+0000 7ff032ffd700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff01c066f30 0x7ff01c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.412+0000 7ff032ffd700 1 -- 172.21.15.7:0/4095100036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5411+0+0 (secure 0 0 0) 0x7ff034085870 con 0x7ff03c1579f0 2024-01-11T07:08:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.412+0000 7ff03affd700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff01c066f30 0x7ff01c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:08:59.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.413+0000 7ff03affd700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff01c066f30 0x7ff01c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff03c171ce0 tx=0x7ff02c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:08:59.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.417+0000 7ff032ffd700 1 -- 172.21.15.7:0/4095100036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff034052cb0 con 0x7ff03c1579f0 2024-01-11T07:08:59.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:08:59.619+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff03c04c3c0 con 0x7ff03c1579f0 2024-01-11T07:09:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.252+0000 7ff032ffd700 1 -- 172.21.15.7:0/4095100036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff0340562d0 con 0x7ff03c1579f0 2024-01-11T07:09:00.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.257+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff01c066f30 msgr2=0x7ff01c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.257+0000 7ff04169a700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff01c066f30 0x7ff01c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff03c171ce0 tx=0x7ff02c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.257+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 msgr2=0x7ff03c155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.257+0000 7ff04169a700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c155930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff034009880 tx=0x7ff034018000 comp rx=0 tx=0).stop 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.258+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 shutdown_connections 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.258+0000 7ff04169a700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff01c066f30 0x7ff01c0693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.258+0000 7ff04169a700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c1579f0 0x7ff03c155930 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.258+0000 7ff04169a700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff03c067610 0x7ff03c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.258+0000 7ff04169a700 1 --2- 172.21.15.7:0/4095100036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff03c157620 0x7ff03c172790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.258+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 >> 172.21.15.7:0/4095100036 conn(0x7ff03c06ed50 msgr2=0x7ff03c06ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.258+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 shutdown_connections 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.258+0000 7ff04169a700 1 -- 172.21.15.7:0/4095100036 wait complete. 2024-01-11T07:09:00.258 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:09:00.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:09:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.414+0000 7f033d7bc700 1 Processor -- start 2024-01-11T07:09:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.414+0000 7f033d7bc700 1 -- start start 2024-01-11T07:09:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f033d7bc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f033815a090 0x7f03380cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f033d7bc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f03380cc140 0x7f03380b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f033d7bc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380d00c0 con 0x7f0338159cc0 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f033d7bc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03380cc570 con 0x7f033815a090 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f033d7bc700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03380d0550 con 0x7f03380cc140 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f03367fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f033815a090 0x7f03380cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f0336ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0338159cc0 0x7f03380cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36960/0 (socket says 172.21.15.7:36960) 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f0336ffd700 1 -- 172.21.15.7:0/1282864293 learned_addr learned my addr 172.21.15.7:0/1282864293 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.415+0000 7f03377fe700 1 --2- 172.21.15.7:0/1282864293 >> v2:172.21.15.113:3300/0 conn(0x7f03380cc140 0x7f03380b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f03367fc700 1 -- 172.21.15.7:0/1282864293 >> v2:172.21.15.113:3300/0 conn(0x7f03380cc140 msgr2=0x7f03380b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f03367fc700 1 --2- 172.21.15.7:0/1282864293 >> v2:172.21.15.113:3300/0 conn(0x7f03380cc140 0x7f03380b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f03367fc700 1 -- 172.21.15.7:0/1282864293 >> v1:172.21.15.7:6789/0 conn(0x7f0338159cc0 legacy=0x7f03380cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f03367fc700 1 -- 172.21.15.7:0/1282864293 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380ce030 con 0x7f033815a090 2024-01-11T07:09:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f0335ffb700 1 -- 172.21.15.7:0/1282864293 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886017279 0 0) 0x7f03380d00c0 con 0x7f0338159cc0 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f03367fc700 1 --2- 172.21.15.7:0/1282864293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033815a090 0x7f03380cbc00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f03380b0880 tx=0x7f03280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2c33132eeb213b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f0335ffb700 1 -- 172.21.15.7:0/1282864293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032800b6b0 con 0x7f033815a090 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f0335ffb700 1 -- 172.21.15.7:0/1282864293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032800b810 con 0x7f033815a090 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f0335ffb700 1 -- 172.21.15.7:0/1282864293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032800bbb0 con 0x7f033815a090 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f033d7bc700 1 -- 172.21.15.7:0/1282864293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033815a090 msgr2=0x7f03380cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.416+0000 7f033d7bc700 1 --2- 172.21.15.7:0/1282864293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033815a090 0x7f03380cbc00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f03380b0880 tx=0x7f03280091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.417+0000 7f033d7bc700 1 -- 172.21.15.7:0/1282864293 shutdown_connections 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.417+0000 7f033d7bc700 1 --2- 172.21.15.7:0/1282864293 >> v2:172.21.15.113:3300/0 conn(0x7f03380cc140 0x7f03380b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.417+0000 7f033d7bc700 1 --2- 172.21.15.7:0/1282864293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033815a090 0x7f03380cbc00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.417+0000 7f033d7bc700 1 -- 172.21.15.7:0/1282864293 >> 172.21.15.7:0/1282864293 conn(0x7f0338071120 msgr2=0x7f033806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.417+0000 7f033d7bc700 1 -- 172.21.15.7:0/1282864293 shutdown_connections 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.417+0000 7f033d7bc700 1 -- 172.21.15.7:0/1282864293 wait complete. 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.418+0000 7f033d7bc700 1 Processor -- start 2024-01-11T07:09:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.418+0000 7f033d7bc700 1 -- start start 2024-01-11T07:09:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f033d7bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380cc140 0x7f03380b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f033d7bc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0338159cc0 0x7f033813b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f033d7bc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033815a090 0x7f033813c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f033d7bc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033813bd90 con 0x7f03380cc140 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f033d7bc700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b1690 con 0x7f033815a090 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f033d7bc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380b0a80 con 0x7f0338159cc0 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f0336ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380cc140 0x7f03380b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f0336ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380cc140 0x7f03380b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38444/0 (socket says 172.21.15.7:38444) 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f0336ffd700 1 -- 172.21.15.7:0/624469237 learned_addr learned my addr 172.21.15.7:0/624469237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f03377fe700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033815a090 0x7f033813c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f03367fc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0338159cc0 0x7f033813b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f0336ffd700 1 -- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0338159cc0 msgr2=0x7f033813b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f0336ffd700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0338159cc0 0x7f033813b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f0336ffd700 1 -- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033815a090 msgr2=0x7f033813c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f0336ffd700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033815a090 0x7f033813c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.419+0000 7f0336ffd700 1 -- 172.21.15.7:0/624469237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328009420 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.420+0000 7f0336ffd700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380cc140 0x7f03380b1b20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f032000c7e0 tx=0x7f0320007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.420+0000 7f032effd700 1 -- 172.21.15.7:0/624469237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320005b30 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.420+0000 7f032effd700 1 -- 172.21.15.7:0/624469237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0320005c90 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.420+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03380b0d60 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.420+0000 7f032effd700 1 -- 172.21.15.7:0/624469237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320007300 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.420+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03380b1320 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.421+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0338088fe0 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.421+0000 7f032effd700 1 -- 172.21.15.7:0/624469237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f032001b3b0 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.422+0000 7f032effd700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0318066f30 0x7f03180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.422+0000 7f032effd700 1 -- 172.21.15.7:0/624469237 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0320083750 con 0x7f03380cc140 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.422+0000 7f03367fc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0318066f30 0x7f03180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.422+0000 7f03367fc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0318066f30 0x7f03180693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f03280091e0 tx=0x7f0328017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.426+0000 7f032effd700 1 -- 172.21.15.7:0/624469237 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0320050cc0 con 0x7f03380cc140 2024-01-11T07:09:00.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:09:00.469 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:09:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.631+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f033804c3c0 con 0x7f03380cc140 2024-01-11T07:09:00.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.635+0000 7f032effd700 1 -- 172.21.15.7:0/624469237 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0320001d60 con 0x7f03380cc140 2024-01-11T07:09:00.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0318066f30 msgr2=0x7f03180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0318066f30 0x7f03180693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f03280091e0 tx=0x7f0328017000 comp rx=0 tx=0).stop 2024-01-11T07:09:00.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380cc140 msgr2=0x7f03380b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380cc140 0x7f03380b1b20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f032000c7e0 tx=0x7f0320007df0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 shutdown_connections 2024-01-11T07:09:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0318066f30 0x7f03180693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033815a090 0x7f033813c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03380cc140 0x7f03380b1b20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 --2- 172.21.15.7:0/624469237 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0338159cc0 0x7f033813b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 >> 172.21.15.7:0/624469237 conn(0x7f0338071120 msgr2=0x7f033806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 shutdown_connections 2024-01-11T07:09:00.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.637+0000 7f033d7bc700 1 -- 172.21.15.7:0/624469237 wait complete. 2024-01-11T07:09:00.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-01-11T07:09:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.785+0000 7ff27337a700 1 Processor -- start 2024-01-11T07:09:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.785+0000 7ff27337a700 1 -- start start 2024-01-11T07:09:00.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff27337a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff26c15c330 0x7ff26c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff27337a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c06e1b0 0x7ff26c06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff27337a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0c87a0 con 0x7ff26c15b850 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff27337a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c0c9cb0 con 0x7ff26c15c330 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff27337a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c069280 con 0x7ff26c06e1b0 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff270915700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff26c15c330 0x7ff26c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff271116700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff26c15b850 0x7ff26c15bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36966/0 (socket says 172.21.15.7:36966) 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff271116700 1 -- 172.21.15.7:0/2430620843 learned_addr learned my addr 172.21.15.7:0/2430620843 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff271917700 1 --2- 172.21.15.7:0/2430620843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c06e1b0 0x7ff26c06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff270915700 1 -- 172.21.15.7:0/2430620843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c06e1b0 msgr2=0x7ff26c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff270915700 1 --2- 172.21.15.7:0/2430620843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c06e1b0 0x7ff26c06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff270915700 1 -- 172.21.15.7:0/2430620843 >> v1:172.21.15.7:6789/0 conn(0x7ff26c15b850 legacy=0x7ff26c15bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff270915700 1 -- 172.21.15.7:0/2430620843 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c069560 con 0x7ff26c15c330 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.786+0000 7ff263fff700 1 -- 172.21.15.7:0/2430620843 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37843693 0 0) 0x7ff26c0c87a0 con 0x7ff26c15b850 2024-01-11T07:09:00.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff270915700 1 --2- 172.21.15.7:0/2430620843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c15c330 0x7ff26c068bf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0098e0 tx=0x7ff25c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c05e4683dcc0a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff263fff700 1 -- 172.21.15.7:0/2430620843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00b640 con 0x7ff26c15c330 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff263fff700 1 -- 172.21.15.7:0/2430620843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c00b7a0 con 0x7ff26c15c330 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff263fff700 1 -- 172.21.15.7:0/2430620843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00bb40 con 0x7ff26c15c330 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff27337a700 1 -- 172.21.15.7:0/2430620843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c15c330 msgr2=0x7ff26c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff27337a700 1 --2- 172.21.15.7:0/2430620843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c15c330 0x7ff26c068bf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0098e0 tx=0x7ff25c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff27337a700 1 -- 172.21.15.7:0/2430620843 shutdown_connections 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff27337a700 1 --2- 172.21.15.7:0/2430620843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c06e1b0 0x7ff26c06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff27337a700 1 --2- 172.21.15.7:0/2430620843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c15c330 0x7ff26c068bf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff27337a700 1 -- 172.21.15.7:0/2430620843 >> 172.21.15.7:0/2430620843 conn(0x7ff26c063230 msgr2=0x7ff26c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff27337a700 1 -- 172.21.15.7:0/2430620843 shutdown_connections 2024-01-11T07:09:00.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.787+0000 7ff27337a700 1 -- 172.21.15.7:0/2430620843 wait complete. 2024-01-11T07:09:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.789+0000 7ff27337a700 1 Processor -- start 2024-01-11T07:09:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.789+0000 7ff27337a700 1 -- start start 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff27337a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c06e1b0 0x7ff26c167d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff27337a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c15b850 0x7ff26c0a8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff27337a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff26c15c330 0x7ff26c156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff27337a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c168da0 con 0x7ff26c06e1b0 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff27337a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c1583b0 con 0x7ff26c15c330 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff27337a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c168270 con 0x7ff26c15b850 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c06e1b0 0x7ff26c167d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c06e1b0 0x7ff26c167d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38448/0 (socket says 172.21.15.7:38448) 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 -- 172.21.15.7:0/2485347999 learned_addr learned my addr 172.21.15.7:0/2485347999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff270915700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c15b850 0x7ff26c0a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271917700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff26c15c330 0x7ff26c156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 -- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c15b850 msgr2=0x7ff26c0a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c15b850 0x7ff26c0a8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 -- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff26c15c330 msgr2=0x7ff26c156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff26c15c330 0x7ff26c156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 -- 172.21.15.7:0/2485347999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c009420 con 0x7ff26c06e1b0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271917700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff26c15c330 0x7ff26c156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.790+0000 7ff271116700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c06e1b0 0x7ff26c167d30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c7e0 tx=0x7ff268007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.791+0000 7ff2617fa700 1 -- 172.21.15.7:0/2485347999 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268005b30 con 0x7ff26c06e1b0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.791+0000 7ff2617fa700 1 -- 172.21.15.7:0/2485347999 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268005c90 con 0x7ff26c06e1b0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.791+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c0a8bd0 con 0x7ff26c06e1b0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.791+0000 7ff2617fa700 1 -- 172.21.15.7:0/2485347999 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268007300 con 0x7ff26c06e1b0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.791+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c168580 con 0x7ff26c06e1b0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.792+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c158690 con 0x7ff26c06e1b0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.792+0000 7ff2617fa700 1 -- 172.21.15.7:0/2485347999 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff2680076b0 con 0x7ff26c06e1b0 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.793+0000 7ff2617fa700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff250066f30 0x7ff2500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.793+0000 7ff2617fa700 1 -- 172.21.15.7:0/2485347999 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff2680837a0 con 0x7ff26c06e1b0 2024-01-11T07:09:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.793+0000 7ff270915700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff250066f30 0x7ff2500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:00.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.794+0000 7ff270915700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff250066f30 0x7ff2500693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0093f0 tx=0x7ff25c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:00.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:00.797+0000 7ff2617fa700 1 -- 172.21.15.7:0/2485347999 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff268050d10 con 0x7ff26c06e1b0 2024-01-11T07:09:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.088+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7ff26c158180 con 0x7ff26c06e1b0 2024-01-11T07:09:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.258+0000 7ff2617fa700 1 -- 172.21.15.7:0/2485347999 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v129) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff268054330 con 0x7ff26c06e1b0 2024-01-11T07:09:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.260+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff250066f30 msgr2=0x7ff2500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.260+0000 7ff27337a700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff250066f30 0x7ff2500693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0093f0 tx=0x7ff25c017000 comp rx=0 tx=0).stop 2024-01-11T07:09:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.260+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c06e1b0 msgr2=0x7ff26c167d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.260+0000 7ff27337a700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c06e1b0 0x7ff26c167d30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c7e0 tx=0x7ff268007df0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.261+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 shutdown_connections 2024-01-11T07:09:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.261+0000 7ff27337a700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff250066f30 0x7ff2500693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.261+0000 7ff27337a700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff26c15c330 0x7ff26c156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.261+0000 7ff27337a700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff26c06e1b0 0x7ff26c167d30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.261+0000 7ff27337a700 1 --2- 172.21.15.7:0/2485347999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff26c15b850 0x7ff26c0a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.261+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 >> 172.21.15.7:0/2485347999 conn(0x7ff26c063230 msgr2=0x7ff26c15bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.261+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 shutdown_connections 2024-01-11T07:09:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.261+0000 7ff27337a700 1 -- 172.21.15.7:0/2485347999 wait complete. 2024-01-11T07:09:01.275 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-01-11T07:09:01.275 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2024-01-11T07:09:01.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-01-11T07:09:01.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.411+0000 7f9fde2a3700 1 Processor -- start 2024-01-11T07:09:01.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.412+0000 7f9fde2a3700 1 -- start start 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.412+0000 7f9fde2a3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fd806dd00 0x7f9fd80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.412+0000 7f9fde2a3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806e0d0 0x7f9fd8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.412+0000 7f9fde2a3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd815b770 con 0x7f9fd8159ff0 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.412+0000 7f9fde2a3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd815abf0 con 0x7f9fd806dd00 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.412+0000 7f9fde2a3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd815aed0 con 0x7f9fd806e0d0 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fd806dd00 0x7f9fd80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fd806dd00 0x7f9fd80c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38462/0 (socket says 172.21.15.7:38462) 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd7fff700 1 -- 172.21.15.7:0/1323639424 learned_addr learned my addr 172.21.15.7:0/1323639424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:01.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd77fe700 1 --2- 172.21.15.7:0/1323639424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806e0d0 0x7f9fd8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd7fff700 1 -- 172.21.15.7:0/1323639424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806e0d0 msgr2=0x7f9fd8159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd7fff700 1 --2- 172.21.15.7:0/1323639424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806e0d0 0x7f9fd8159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd7fff700 1 -- 172.21.15.7:0/1323639424 >> v1:172.21.15.113:6789/0 conn(0x7f9fd8159ff0 legacy=0x7f9fd80c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd7fff700 1 -- 172.21.15.7:0/1323639424 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd815b1b0 con 0x7f9fd806dd00 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd77fe700 1 --2- 172.21.15.7:0/1323639424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806e0d0 0x7f9fd8159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd7fff700 1 --2- 172.21.15.7:0/1323639424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806dd00 0x7f9fd80c98d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80099b0 tx=0x7f9fc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cd779db19b2829 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd6ffd700 1 -- 172.21.15.7:0/1323639424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800b860 con 0x7f9fd806dd00 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd6ffd700 1 -- 172.21.15.7:0/1323639424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc800b9c0 con 0x7f9fd806dd00 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.413+0000 7f9fd6ffd700 1 -- 172.21.15.7:0/1323639424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800bd60 con 0x7f9fd806dd00 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.414+0000 7f9fde2a3700 1 -- 172.21.15.7:0/1323639424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806dd00 msgr2=0x7f9fd80c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.414+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/1323639424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806dd00 0x7f9fd80c98d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80099b0 tx=0x7f9fc8009250 comp rx=0 tx=0).stop 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.414+0000 7f9fde2a3700 1 -- 172.21.15.7:0/1323639424 shutdown_connections 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.414+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/1323639424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806dd00 0x7f9fd80c98d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.414+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/1323639424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806e0d0 0x7f9fd8159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.414+0000 7f9fde2a3700 1 -- 172.21.15.7:0/1323639424 >> 172.21.15.7:0/1323639424 conn(0x7f9fd80680e0 msgr2=0x7f9fd80cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.414+0000 7f9fde2a3700 1 -- 172.21.15.7:0/1323639424 shutdown_connections 2024-01-11T07:09:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.414+0000 7f9fde2a3700 1 -- 172.21.15.7:0/1323639424 wait complete. 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.415+0000 7f9fde2a3700 1 Processor -- start 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.415+0000 7f9fde2a3700 1 -- start start 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fde2a3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806dd00 0x7f9fd8153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fde2a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806e0d0 0x7f9fd80c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fde2a3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd8159ff0 0x7f9fd80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fde2a3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80c1690 con 0x7f9fd806e0d0 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fde2a3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80c1db0 con 0x7f9fd8159ff0 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fde2a3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80c7350 con 0x7f9fd806dd00 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fd77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806e0d0 0x7f9fd80c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fd77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806e0d0 0x7f9fd80c3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38470/0 (socket says 172.21.15.7:38470) 2024-01-11T07:09:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.416+0000 7f9fd77fe700 1 -- 172.21.15.7:0/4049112577 learned_addr learned my addr 172.21.15.7:0/4049112577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fdc840700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd8159ff0 0x7f9fd80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fd7fff700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806dd00 0x7f9fd8153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fd77fe700 1 -- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806dd00 msgr2=0x7f9fd8153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fd77fe700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806dd00 0x7f9fd8153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fd77fe700 1 -- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd8159ff0 msgr2=0x7f9fd80b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fd77fe700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd8159ff0 0x7f9fd80b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fd77fe700 1 -- 172.21.15.7:0/4049112577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80094f0 con 0x7f9fd806e0d0 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fd77fe700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806e0d0 0x7f9fd80c3af0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e8d0 tx=0x7f9fcc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fbffff700 1 -- 172.21.15.7:0/4049112577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc013410 con 0x7f9fd806e0d0 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fbffff700 1 -- 172.21.15.7:0/4049112577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc013570 con 0x7f9fd806e0d0 2024-01-11T07:09:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd80c6d90 con 0x7f9fd806e0d0 2024-01-11T07:09:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fbffff700 1 -- 172.21.15.7:0/4049112577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc013910 con 0x7f9fd806e0d0 2024-01-11T07:09:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.417+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd80c1a40 con 0x7f9fd806e0d0 2024-01-11T07:09:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.418+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd80c1f20 con 0x7f9fd806e0d0 2024-01-11T07:09:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.419+0000 7f9fbffff700 1 -- 172.21.15.7:0/4049112577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9fcc013d40 con 0x7f9fd806e0d0 2024-01-11T07:09:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.419+0000 7f9fbffff700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fc4066f30 0x7f9fc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.419+0000 7f9fbffff700 1 -- 172.21.15.7:0/4049112577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9fcc083f10 con 0x7f9fd806e0d0 2024-01-11T07:09:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.419+0000 7f9fd7fff700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fc4066f30 0x7f9fc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.420+0000 7f9fd7fff700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fc4066f30 0x7f9fc40693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80099b0 tx=0x7f9fc8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.424+0000 7f9fbffff700 1 -- 172.21.15.7:0/4049112577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fcc051480 con 0x7f9fd806e0d0 2024-01-11T07:09:01.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:01.713+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f9fd804c3c0 con 0x7f9fd806e0d0 2024-01-11T07:09:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.261+0000 7f9fbffff700 1 -- 172.21.15.7:0/4049112577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v130) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9fcc054aa0 con 0x7f9fd806e0d0 2024-01-11T07:09:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fc4066f30 msgr2=0x7f9fc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fc4066f30 0x7f9fc40693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80099b0 tx=0x7f9fc8017000 comp rx=0 tx=0).stop 2024-01-11T07:09:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806e0d0 msgr2=0x7f9fd80c3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806e0d0 0x7f9fd80c3af0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e8d0 tx=0x7f9fcc008040 comp rx=0 tx=0).stop 2024-01-11T07:09:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 shutdown_connections 2024-01-11T07:09:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9fc4066f30 0x7f9fc40693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fd8159ff0 0x7f9fd80b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fd806dd00 0x7f9fd8153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 --2- 172.21.15.7:0/4049112577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd806e0d0 0x7f9fd80c3af0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 >> 172.21.15.7:0/4049112577 conn(0x7f9fd80680e0 msgr2=0x7f9fd80ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 shutdown_connections 2024-01-11T07:09:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.264+0000 7f9fde2a3700 1 -- 172.21.15.7:0/4049112577 wait complete. 2024-01-11T07:09:02.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-11T07:09:02.431 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-11T07:09:02.432 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-11T07:09:02.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:09:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.567+0000 7fe3fb546700 1 Processor -- start 2024-01-11T07:09:02.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.568+0000 7fe3fb546700 1 -- start start 2024-01-11T07:09:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.568+0000 7fe3fb546700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f41579a0 0x7fe3f415af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3fb546700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f415b4b0 con 0x7fe3f406a9c0 2024-01-11T07:09:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3fb546700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f406bb60 con 0x7fe3f4063230 2024-01-11T07:09:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3fb546700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f406e8b0 con 0x7fe3f41579a0 2024-01-11T07:09:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3f8ae1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe3f406a9c0 0x7fe3f406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36972/0 (socket says 172.21.15.7:36972) 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3f8ae1700 1 -- 172.21.15.7:0/417657081 learned_addr learned my addr 172.21.15.7:0/417657081 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3f92e2700 1 --2- 172.21.15.7:0/417657081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f41579a0 0x7fe3f415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3e7fff700 1 -- 172.21.15.7:0/417657081 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577739841 0 0) 0x7fe3f0001110 con 0x7fe3f406a9c0 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3e7fff700 1 -- 172.21.15.7:0/417657081 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e0003570 con 0x7fe3f406a9c0 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3e7fff700 1 -- 172.21.15.7:0/417657081 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667611594 0 0) 0x7fe3f406bb60 con 0x7fe3f4063230 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3e7fff700 1 -- 172.21.15.7:0/417657081 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f0001110 con 0x7fe3f4063230 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3f92e2700 1 -- 172.21.15.7:0/417657081 >> v1:172.21.15.113:6789/0 conn(0x7fe3f4063230 legacy=0x7fe3f40c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3e7fff700 1 -- 172.21.15.7:0/417657081 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567987236 0 0) 0x7fe3e0003570 con 0x7fe3f406a9c0 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3f92e2700 1 -- 172.21.15.7:0/417657081 >> v1:172.21.15.7:6789/0 conn(0x7fe3f406a9c0 legacy=0x7fe3f406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.569+0000 7fe3f92e2700 1 -- 172.21.15.7:0/417657081 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f406af50 con 0x7fe3f41579a0 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3f92e2700 1 --2- 172.21.15.7:0/417657081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f41579a0 0x7fe3f415af70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009910 tx=0x7fe3e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690e097b0cea628f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3e7fff700 1 -- 172.21.15.7:0/417657081 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0001600 con 0x7fe3f41579a0 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3e7fff700 1 -- 172.21.15.7:0/417657081 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e8011420 con 0x7fe3f41579a0 2024-01-11T07:09:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3e7fff700 1 -- 172.21.15.7:0/417657081 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80117c0 con 0x7fe3f41579a0 2024-01-11T07:09:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3fb546700 1 -- 172.21.15.7:0/417657081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f41579a0 msgr2=0x7fe3f415af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3fb546700 1 --2- 172.21.15.7:0/417657081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f41579a0 0x7fe3f415af70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009910 tx=0x7fe3e80091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3fb546700 1 -- 172.21.15.7:0/417657081 shutdown_connections 2024-01-11T07:09:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3fb546700 1 --2- 172.21.15.7:0/417657081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f41579a0 0x7fe3f415af70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.570+0000 7fe3fb546700 1 -- 172.21.15.7:0/417657081 >> 172.21.15.7:0/417657081 conn(0x7fe3f40cd8f0 msgr2=0x7fe3f40cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.571+0000 7fe3fb546700 1 -- 172.21.15.7:0/417657081 shutdown_connections 2024-01-11T07:09:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.571+0000 7fe3fb546700 1 -- 172.21.15.7:0/417657081 wait complete. 2024-01-11T07:09:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.572+0000 7fe3fb546700 1 Processor -- start 2024-01-11T07:09:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.572+0000 7fe3fb546700 1 -- start start 2024-01-11T07:09:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3fb546700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f4063230 0x7fe3f40afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3fb546700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f406a9c0 0x7fe3f416d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3fb546700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f41579a0 0x7fe3f40638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3fb546700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40ad960 con 0x7fe3f41579a0 2024-01-11T07:09:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3fb546700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f416dba0 con 0x7fe3f4063230 2024-01-11T07:09:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3fb546700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f416de80 con 0x7fe3f406a9c0 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f41579a0 0x7fe3f40638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f41579a0 0x7fe3f40638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38482/0 (socket says 172.21.15.7:38482) 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 -- 172.21.15.7:0/1937735103 learned_addr learned my addr 172.21.15.7:0/1937735103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f92e2700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f4063230 0x7fe3f40afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f8ae1700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f406a9c0 0x7fe3f416d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 -- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f406a9c0 msgr2=0x7fe3f416d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f406a9c0 0x7fe3f416d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 -- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f4063230 msgr2=0x7fe3f40afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f4063230 0x7fe3f40afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 -- 172.21.15.7:0/1937735103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8009420 con 0x7fe3f41579a0 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.573+0000 7fe3f9ae3700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f41579a0 0x7fe3f40638e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00c910 tx=0x7fe3ec0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.574+0000 7fe3e57fa700 1 -- 172.21.15.7:0/1937735103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec012630 con 0x7fe3f41579a0 2024-01-11T07:09:02.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.574+0000 7fe3e57fa700 1 -- 172.21.15.7:0/1937735103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec0127b0 con 0x7fe3f41579a0 2024-01-11T07:09:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.574+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f416e160 con 0x7fe3f41579a0 2024-01-11T07:09:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.574+0000 7fe3e57fa700 1 -- 172.21.15.7:0/1937735103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec012b70 con 0x7fe3f41579a0 2024-01-11T07:09:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.574+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f416d8b0 con 0x7fe3f41579a0 2024-01-11T07:09:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.575+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f4064a40 con 0x7fe3f41579a0 2024-01-11T07:09:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.575+0000 7fe3e57fa700 1 -- 172.21.15.7:0/1937735103 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe3ec01b400 con 0x7fe3f41579a0 2024-01-11T07:09:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.576+0000 7fe3e57fa700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3d8066f30 0x7fe3d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.576+0000 7fe3e57fa700 1 -- 172.21.15.7:0/1937735103 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe3ec0845e0 con 0x7fe3f41579a0 2024-01-11T07:09:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.576+0000 7fe3f92e2700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3d8066f30 0x7fe3d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:02.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.576+0000 7fe3f92e2700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3d8066f30 0x7fe3d80693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80030c0 tx=0x7fe3e8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.580+0000 7fe3e57fa700 1 -- 172.21.15.7:0/1937735103 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3ec051ab0 con 0x7fe3f41579a0 2024-01-11T07:09:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.783+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe3f404c3c0 con 0x7fe3f41579a0 2024-01-11T07:09:02.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.787+0000 7fe3e57fa700 1 -- 172.21.15.7:0/1937735103 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe3ec0550d0 con 0x7fe3f41579a0 2024-01-11T07:09:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3d8066f30 msgr2=0x7fe3d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3d8066f30 0x7fe3d80693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80030c0 tx=0x7fe3e8016000 comp rx=0 tx=0).stop 2024-01-11T07:09:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f41579a0 msgr2=0x7fe3f40638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f41579a0 0x7fe3f40638e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00c910 tx=0x7fe3ec0027a0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 shutdown_connections 2024-01-11T07:09:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3d8066f30 0x7fe3d80693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f41579a0 0x7fe3f40638e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f4063230 0x7fe3f40afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 --2- 172.21.15.7:0/1937735103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f406a9c0 0x7fe3f416d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 >> 172.21.15.7:0/1937735103 conn(0x7fe3f40cd8f0 msgr2=0x7fe3f40c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 shutdown_connections 2024-01-11T07:09:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:02.789+0000 7fe3fb546700 1 -- 172.21.15.7:0/1937735103 wait complete. 2024-01-11T07:09:02.802 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:09:02.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:09:02.846 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:09:02.901 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:09:02.902 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:09:02.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:09:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.040+0000 7f017d29a700 1 Processor -- start 2024-01-11T07:09:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.040+0000 7f017d29a700 1 -- start start 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f017d29a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0178159ff0 0x7f01780b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f017d29a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f017806f4d0 0x7f01780b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f017d29a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0178067de0 con 0x7f0178159c20 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f017d29a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0178069570 con 0x7f0178159ff0 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f017d29a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01780b20e0 con 0x7f017806f4d0 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01767fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0178159ff0 0x7f01780b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01767fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0178159ff0 0x7f01780b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38490/0 (socket says 172.21.15.7:38490) 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01767fc700 1 -- 172.21.15.7:0/1821363434 learned_addr learned my addr 172.21.15.7:0/1821363434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01777fe700 1 --2- 172.21.15.7:0/1821363434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f017806f4d0 0x7f01780b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01767fc700 1 -- 172.21.15.7:0/1821363434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f017806f4d0 msgr2=0x7f01780b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01767fc700 1 --2- 172.21.15.7:0/1821363434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f017806f4d0 0x7f01780b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01767fc700 1 -- 172.21.15.7:0/1821363434 >> v1:172.21.15.113:6789/0 conn(0x7f0178159c20 legacy=0x7f0178068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01767fc700 1 -- 172.21.15.7:0/1821363434 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017806f950 con 0x7f0178159ff0 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f01767fc700 1 --2- 172.21.15.7:0/1821363434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159ff0 0x7f01780b1820 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f016c0099b0 tx=0x7f016c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f367c2daf4b8301d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.041+0000 7f0175ffb700 1 -- 172.21.15.7:0/1821363434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c00b820 con 0x7f0178159ff0 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f0175ffb700 1 -- 172.21.15.7:0/1821363434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016c00b980 con 0x7f0178159ff0 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f0175ffb700 1 -- 172.21.15.7:0/1821363434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c00bd20 con 0x7f0178159ff0 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f017d29a700 1 -- 172.21.15.7:0/1821363434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159ff0 msgr2=0x7f01780b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f017d29a700 1 --2- 172.21.15.7:0/1821363434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159ff0 0x7f01780b1820 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f016c0099b0 tx=0x7f016c009250 comp rx=0 tx=0).stop 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f017d29a700 1 -- 172.21.15.7:0/1821363434 shutdown_connections 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f017d29a700 1 --2- 172.21.15.7:0/1821363434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f017806f4d0 0x7f01780b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f017d29a700 1 --2- 172.21.15.7:0/1821363434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159ff0 0x7f01780b1820 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f017d29a700 1 -- 172.21.15.7:0/1821363434 >> 172.21.15.7:0/1821363434 conn(0x7f0178071160 msgr2=0x7f017806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f017d29a700 1 -- 172.21.15.7:0/1821363434 shutdown_connections 2024-01-11T07:09:03.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.042+0000 7f017d29a700 1 -- 172.21.15.7:0/1821363434 wait complete. 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.043+0000 7f017d29a700 1 Processor -- start 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f017d29a700 1 -- start start 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f017d29a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806f4d0 0x7f0178153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f017d29a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159c20 0x7f017815e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f017d29a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0178159ff0 0x7f017815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f017d29a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01780cadc0 con 0x7f0178159c20 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f017d29a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01780cc550 con 0x7f017806f4d0 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f017d29a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01780ca520 con 0x7f0178159ff0 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f01767fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159c20 0x7f017815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f01767fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159c20 0x7f017815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38500/0 (socket says 172.21.15.7:38500) 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.044+0000 7f01767fc700 1 -- 172.21.15.7:0/4017107605 learned_addr learned my addr 172.21.15.7:0/4017107605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f01777fe700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0178159ff0 0x7f017815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f0176ffd700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806f4d0 0x7f0178153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f01767fc700 1 -- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0178159ff0 msgr2=0x7f017815e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f01767fc700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0178159ff0 0x7f017815e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f01767fc700 1 -- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806f4d0 msgr2=0x7f0178153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f01767fc700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806f4d0 0x7f0178153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f01767fc700 1 -- 172.21.15.7:0/4017107605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0094f0 con 0x7f0178159c20 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f01767fc700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159c20 0x7f017815e480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f016c009290 tx=0x7f016c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f015effd700 1 -- 172.21.15.7:0/4017107605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c011e40 con 0x7f0178159c20 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f015effd700 1 -- 172.21.15.7:0/4017107605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016c01b470 con 0x7f0178159c20 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01780ca800 con 0x7f0178159c20 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f015effd700 1 -- 172.21.15.7:0/4017107605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c01b830 con 0x7f0178159c20 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.045+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01780cc1b0 con 0x7f0178159c20 2024-01-11T07:09:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.046+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017804c3c0 con 0x7f0178159c20 2024-01-11T07:09:03.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.047+0000 7f015effd700 1 -- 172.21.15.7:0/4017107605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f016c01bbe0 con 0x7f0178159c20 2024-01-11T07:09:03.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.047+0000 7f015effd700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0164066f30 0x7f01640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.047+0000 7f015effd700 1 -- 172.21.15.7:0/4017107605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f016c085ca0 con 0x7f0178159c20 2024-01-11T07:09:03.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.048+0000 7f0176ffd700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0164066f30 0x7f01640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.048+0000 7f0176ffd700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0164066f30 0x7f01640693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f017816b170 tx=0x7f0168007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.052+0000 7f015effd700 1 -- 172.21.15.7:0/4017107605 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f016c053210 con 0x7f0178159c20 2024-01-11T07:09:03.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.306+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01780cc690 con 0x7f0178159c20 2024-01-11T07:09:03.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.307+0000 7f015effd700 1 -- 172.21.15.7:0/4017107605 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f016c02a020 con 0x7f0178159c20 2024-01-11T07:09:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:03.307 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4961,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/781558691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":781558691},{"type":"v1","addr":"172.21.15.7:6834","nonce":781558691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/803319955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":803319955},{"type":"v1","addr":"172.21.15.113:6836","nonce":803319955}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":4970,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2200560714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2200560714},{"type":"v1","addr":"172.21.15.113:6837","nonce":2200560714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1476490709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1476490709},{"type":"v1","addr":"172.21.15.7:6835","nonce":1476490709}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2024-01-11T07:09:03.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0164066f30 msgr2=0x7f01640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0164066f30 0x7f01640693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f017816b170 tx=0x7f0168007ea0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159c20 msgr2=0x7f017815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159c20 0x7f017815e480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f016c009290 tx=0x7f016c016000 comp rx=0 tx=0).stop 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 shutdown_connections 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0164066f30 0x7f01640693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0178159ff0 0x7f017815e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806f4d0 0x7f0178153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 --2- 172.21.15.7:0/4017107605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0178159c20 0x7f017815e480 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 >> 172.21.15.7:0/4017107605 conn(0x7f0178071160 msgr2=0x7f01780b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 shutdown_connections 2024-01-11T07:09:03.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.309+0000 7f017d29a700 1 -- 172.21.15.7:0/4017107605 wait complete. 2024-01-11T07:09:03.310 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-01-11T07:09:03.323 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:09:03.323 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:09:03.323 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:09:03.323 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:09:03.324 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:09:03.324 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:09:03.324 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:09:03.324 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:09:03.325 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:09:03.325 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:09:03.325 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:09:03.325 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:09:03.325 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:09:03.325 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:09:03.325 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:09:03.325 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:09:03.333 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:09:03.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:09:03.334 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:09:03.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:09:03.335 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:09:03.336 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:09:03.336 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:09:03.337 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:09:03.376 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:09:03.377 DEBUG:teuthology.parallel:result is None 2024-01-11T07:09:03.378 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:09:03.378 DEBUG:teuthology.parallel:result is None 2024-01-11T07:09:03.379 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:09:03.379 DEBUG:teuthology.parallel:result is None 2024-01-11T07:09:03.379 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:09:03.380 DEBUG:teuthology.parallel:result is None 2024-01-11T07:09:03.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:09:03.462 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:09:03.463 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:09:03.464 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:09:03.464 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:09:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.521+0000 7fca1ab79700 1 Processor -- start 2024-01-11T07:09:03.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.521+0000 7fca1ab79700 1 -- start start 2024-01-11T07:09:03.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca1ab79700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca140cd6d0 0x7fca140cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca1ab79700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca1415e450 0x7fca14159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca1ab79700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1415acf0 con 0x7fca1415d8a0 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca1ab79700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca1415afd0 con 0x7fca1415e450 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca1ab79700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca140ce980 con 0x7fca140cd6d0 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca19116700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca1415e450 0x7fca14159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca13fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fca1415d8a0 0x7fca1415dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37004/0 (socket says 172.21.15.7:37004) 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca13fff700 1 -- 172.21.15.7:0/443940800 learned_addr learned my addr 172.21.15.7:0/443940800 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca18915700 1 --2- 172.21.15.7:0/443940800 >> v2:172.21.15.113:3300/0 conn(0x7fca140cd6d0 0x7fca140cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca18915700 1 -- 172.21.15.7:0/443940800 >> v2:172.21.15.7:3300/0 conn(0x7fca1415e450 msgr2=0x7fca14159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca18915700 1 --2- 172.21.15.7:0/443940800 >> v2:172.21.15.7:3300/0 conn(0x7fca1415e450 0x7fca14159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.522+0000 7fca18915700 1 -- 172.21.15.7:0/443940800 >> v1:172.21.15.7:6789/0 conn(0x7fca1415d8a0 legacy=0x7fca1415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.523+0000 7fca18915700 1 -- 172.21.15.7:0/443940800 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca140cec60 con 0x7fca140cd6d0 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.523+0000 7fca19116700 1 --2- 172.21.15.7:0/443940800 >> v2:172.21.15.7:3300/0 conn(0x7fca1415e450 0x7fca14159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.523+0000 7fca18915700 1 --2- 172.21.15.7:0/443940800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca140cd6d0 0x7fca140cdaa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca140677c0 tx=0x7fca080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c037d6de231ea2f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:03.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.523+0000 7fca137fe700 1 -- 172.21.15.7:0/443940800 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08011630 con 0x7fca140cd6d0 2024-01-11T07:09:03.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.523+0000 7fca137fe700 1 -- 172.21.15.7:0/443940800 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca08011790 con 0x7fca140cd6d0 2024-01-11T07:09:03.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.523+0000 7fca137fe700 1 -- 172.21.15.7:0/443940800 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08011b30 con 0x7fca140cd6d0 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.523+0000 7fca1ab79700 1 -- 172.21.15.7:0/443940800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca140cd6d0 msgr2=0x7fca140cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.523+0000 7fca1ab79700 1 --2- 172.21.15.7:0/443940800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca140cd6d0 0x7fca140cdaa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca140677c0 tx=0x7fca080091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.524+0000 7fca1ab79700 1 -- 172.21.15.7:0/443940800 shutdown_connections 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.524+0000 7fca1ab79700 1 --2- 172.21.15.7:0/443940800 >> v2:172.21.15.7:3300/0 conn(0x7fca1415e450 0x7fca14159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.524+0000 7fca1ab79700 1 --2- 172.21.15.7:0/443940800 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca140cd6d0 0x7fca140cdaa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.524+0000 7fca1ab79700 1 -- 172.21.15.7:0/443940800 >> 172.21.15.7:0/443940800 conn(0x7fca14071120 msgr2=0x7fca1406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.524+0000 7fca1ab79700 1 -- 172.21.15.7:0/443940800 shutdown_connections 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.524+0000 7fca1ab79700 1 -- 172.21.15.7:0/443940800 wait complete. 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.525+0000 7fca1ab79700 1 Processor -- start 2024-01-11T07:09:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.526+0000 7fca1ab79700 1 -- start start 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.526+0000 7fca1ab79700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca140cd6d0 0x7fca1414a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.526+0000 7fca1ab79700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1415d8a0 0x7fca140cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.526+0000 7fca1ab79700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca1415e450 0x7fca14159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.526+0000 7fca1ab79700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca140b39b0 con 0x7fca1415d8a0 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.526+0000 7fca1ab79700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca140b3f70 con 0x7fca1415e450 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.526+0000 7fca1ab79700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca140cb8a0 con 0x7fca140cd6d0 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1415d8a0 0x7fca140cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1415d8a0 0x7fca140cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38548/0 (socket says 172.21.15.7:38548) 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 -- 172.21.15.7:0/851802870 learned_addr learned my addr 172.21.15.7:0/851802870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca18915700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca140cd6d0 0x7fca1414a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca19116700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca1415e450 0x7fca14159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 -- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca140cd6d0 msgr2=0x7fca1414a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca140cd6d0 0x7fca1414a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 -- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca1415e450 msgr2=0x7fca14159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca1415e450 0x7fca14159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 -- 172.21.15.7:0/851802870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c009a20 con 0x7fca1415d8a0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca13fff700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1415d8a0 0x7fca140cb000 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c7c0 tx=0x7fca00007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca10ff9700 1 -- 172.21.15.7:0/851802870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00005b30 con 0x7fca1415d8a0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca10ff9700 1 -- 172.21.15.7:0/851802870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca00005c90 con 0x7fca1415d8a0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca10ff9700 1 -- 172.21.15.7:0/851802870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00004a80 con 0x7fca1415d8a0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca08009420 con 0x7fca1415d8a0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.527+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca140b3c00 con 0x7fca1415d8a0 2024-01-11T07:09:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.528+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca140b40e0 con 0x7fca1415d8a0 2024-01-11T07:09:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.529+0000 7fca10ff9700 1 -- 172.21.15.7:0/851802870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fca000070d0 con 0x7fca1415d8a0 2024-01-11T07:09:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.529+0000 7fca10ff9700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc9f8066f30 0x7fc9f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.529+0000 7fca10ff9700 1 -- 172.21.15.7:0/851802870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fca000835e0 con 0x7fca1415d8a0 2024-01-11T07:09:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.529+0000 7fca18915700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc9f8066f30 0x7fc9f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.530+0000 7fca18915700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc9f8066f30 0x7fc9f80693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fca080093f0 tx=0x7fca08017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.534+0000 7fca10ff9700 1 -- 172.21.15.7:0/851802870 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca00050c10 con 0x7fca1415d8a0 2024-01-11T07:09:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:03.755+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fca1404c3c0 con 0x7fca1415d8a0 2024-01-11T07:09:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.272+0000 7fca10ff9700 1 -- 172.21.15.7:0/851802870 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7fca00054230 con 0x7fca1415d8a0 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc9f8066f30 msgr2=0x7fc9f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc9f8066f30 0x7fc9f80693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fca080093f0 tx=0x7fca08017000 comp rx=0 tx=0).stop 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1415d8a0 msgr2=0x7fca140cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1415d8a0 0x7fca140cb000 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fca0000c7c0 tx=0x7fca00007f70 comp rx=0 tx=0).stop 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 shutdown_connections 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc9f8066f30 0x7fc9f80693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca1415e450 0x7fca14159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca140cd6d0 0x7fca1414a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 --2- 172.21.15.7:0/851802870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca1415d8a0 0x7fca140cb000 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 >> 172.21.15.7:0/851802870 conn(0x7fca14071120 msgr2=0x7fca140d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 shutdown_connections 2024-01-11T07:09:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.276+0000 7fca1ab79700 1 -- 172.21.15.7:0/851802870 wait complete. 2024-01-11T07:09:04.277 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:09:04.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:09:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.429+0000 7fc91c472700 1 Processor -- start 2024-01-11T07:09:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.429+0000 7fc91c472700 1 -- start start 2024-01-11T07:09:04.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc91c472700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914155650 0x7fc9140ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc91c472700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc914069d60 0x7fc9140b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc91c472700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc914063db0 0x7fc9140b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc91c472700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9140ad300 con 0x7fc914069d60 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc91c472700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc914063920 con 0x7fc914063db0 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc91c472700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9140ad020 con 0x7fc914155650 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc919a0d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc914069d60 0x7fc9140b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc919a0d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc914069d60 0x7fc9140b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38562/0 (socket says 172.21.15.7:38562) 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc919a0d700 1 -- 172.21.15.7:0/3767519335 learned_addr learned my addr 172.21.15.7:0/3767519335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc91a20e700 1 --2- 172.21.15.7:0/3767519335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914155650 0x7fc9140ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc91aa0f700 1 --2- 172.21.15.7:0/3767519335 >> v2:172.21.15.113:3300/0 conn(0x7fc914063db0 0x7fc9140b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc919a0d700 1 -- 172.21.15.7:0/3767519335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914155650 msgr2=0x7fc9140ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc919a0d700 1 --2- 172.21.15.7:0/3767519335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914155650 0x7fc9140ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc919a0d700 1 -- 172.21.15.7:0/3767519335 >> v2:172.21.15.113:3300/0 conn(0x7fc914063db0 msgr2=0x7fc9140b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc919a0d700 1 --2- 172.21.15.7:0/3767519335 >> v2:172.21.15.113:3300/0 conn(0x7fc914063db0 0x7fc9140b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.430+0000 7fc919a0d700 1 -- 172.21.15.7:0/3767519335 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9140632d0 con 0x7fc914069d60 2024-01-11T07:09:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc919a0d700 1 --2- 172.21.15.7:0/3767519335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc9140b2b70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc91406aed0 tx=0x7fc904009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79117f39f32b3932 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91920c700 1 -- 172.21.15.7:0/3767519335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90400b860 con 0x7fc914069d60 2024-01-11T07:09:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91920c700 1 -- 172.21.15.7:0/3767519335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90400b9c0 con 0x7fc914069d60 2024-01-11T07:09:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91920c700 1 -- 172.21.15.7:0/3767519335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90400bd60 con 0x7fc914069d60 2024-01-11T07:09:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91c472700 1 -- 172.21.15.7:0/3767519335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 msgr2=0x7fc9140b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91c472700 1 --2- 172.21.15.7:0/3767519335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc9140b2b70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc91406aed0 tx=0x7fc904009250 comp rx=0 tx=0).stop 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91c472700 1 -- 172.21.15.7:0/3767519335 shutdown_connections 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91c472700 1 --2- 172.21.15.7:0/3767519335 >> v2:172.21.15.113:3300/0 conn(0x7fc914063db0 0x7fc9140b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91c472700 1 --2- 172.21.15.7:0/3767519335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914155650 0x7fc9140ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91c472700 1 --2- 172.21.15.7:0/3767519335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc9140b2b70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.431+0000 7fc91c472700 1 -- 172.21.15.7:0/3767519335 >> 172.21.15.7:0/3767519335 conn(0x7fc9140cd910 msgr2=0x7fc9140cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.432+0000 7fc91c472700 1 -- 172.21.15.7:0/3767519335 shutdown_connections 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.432+0000 7fc91c472700 1 -- 172.21.15.7:0/3767519335 wait complete. 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.433+0000 7fc91c472700 1 Processor -- start 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.433+0000 7fc91c472700 1 -- start start 2024-01-11T07:09:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.433+0000 7fc91c472700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914063db0 0x7fc914168530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.433+0000 7fc91c472700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc914159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.433+0000 7fc91c472700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc914155650 0x7fc91416b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc91c472700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914152de0 con 0x7fc914069d60 2024-01-11T07:09:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc91c472700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141680a0 con 0x7fc914155650 2024-01-11T07:09:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc91c472700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc914167a70 con 0x7fc914063db0 2024-01-11T07:09:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc914159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc914159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38578/0 (socket says 172.21.15.7:38578) 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 -- 172.21.15.7:0/1952078045 learned_addr learned my addr 172.21.15.7:0/1952078045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc91a20e700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914063db0 0x7fc914168530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc91aa0f700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc914155650 0x7fc91416b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 -- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914063db0 msgr2=0x7fc914168530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914063db0 0x7fc914168530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 -- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc914155650 msgr2=0x7fc91416b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc914155650 0x7fc91416b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 -- 172.21.15.7:0/1952078045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040094f0 con 0x7fc914069d60 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc919a0d700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc914159f90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc91406cfd0 tx=0x7fc904017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc90a7fc700 1 -- 172.21.15.7:0/1952078045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904024440 con 0x7fc914069d60 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc90a7fc700 1 -- 172.21.15.7:0/1952078045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9040245a0 con 0x7fc914069d60 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91416bac0 con 0x7fc914069d60 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc90a7fc700 1 -- 172.21.15.7:0/1952078045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904024960 con 0x7fc914069d60 2024-01-11T07:09:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.434+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc914167d30 con 0x7fc914069d60 2024-01-11T07:09:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.435+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc914168230 con 0x7fc914069d60 2024-01-11T07:09:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.436+0000 7fc90a7fc700 1 -- 172.21.15.7:0/1952078045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc90401a420 con 0x7fc914069d60 2024-01-11T07:09:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.436+0000 7fc90a7fc700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc8f8066f30 0x7fc8f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.436+0000 7fc90a7fc700 1 -- 172.21.15.7:0/1952078045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc9040860c0 con 0x7fc914069d60 2024-01-11T07:09:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.436+0000 7fc91a20e700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc8f8066f30 0x7fc8f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.437+0000 7fc91a20e700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc8f8066f30 0x7fc8f80693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc910006fd0 tx=0x7fc910008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:04.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.441+0000 7fc90a7fc700 1 -- 172.21.15.7:0/1952078045 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc904053680 con 0x7fc914069d60 2024-01-11T07:09:04.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.725+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc91404c3c0 con 0x7fc914069d60 2024-01-11T07:09:04.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.726+0000 7fc90a7fc700 1 -- 172.21.15.7:0/1952078045 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc904056ca0 con 0x7fc914069d60 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:09:04.726 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:09:04.727 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:09:04.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.729 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:09:04.730 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:09:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.734 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:09:04.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:09:04.737 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:09:04.738 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:09:04.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc8f8066f30 msgr2=0x7fc8f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc8f8066f30 0x7fc8f80693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc910006fd0 tx=0x7fc910008040 comp rx=0 tx=0).stop 2024-01-11T07:09:04.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 msgr2=0x7fc914159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc914159f90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc91406cfd0 tx=0x7fc904017000 comp rx=0 tx=0).stop 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 shutdown_connections 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc8f8066f30 0x7fc8f80693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc914155650 0x7fc91416b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc914063db0 0x7fc914168530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 --2- 172.21.15.7:0/1952078045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc914069d60 0x7fc914159f90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 >> 172.21.15.7:0/1952078045 conn(0x7fc9140cd910 msgr2=0x7fc914153080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 shutdown_connections 2024-01-11T07:09:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.728+0000 7fc91c472700 1 -- 172.21.15.7:0/1952078045 wait complete. 2024-01-11T07:09:04.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:09:04.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.873+0000 7f4199ce8700 1 Processor -- start 2024-01-11T07:09:04.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.873+0000 7f4199ce8700 1 -- start start 2024-01-11T07:09:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f4199ce8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41940cafb0 0x7f41940ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f4199ce8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41940b0b40 0x7f4194161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f4199ce8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4194159300 con 0x7f4194158280 2024-01-11T07:09:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f4199ce8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41940cf260 con 0x7f41940cafb0 2024-01-11T07:09:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f4199ce8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41940cf540 con 0x7f41940b0b40 2024-01-11T07:09:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41940cafb0 0x7f41940ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41940cafb0 0x7f41940ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38584/0 (socket says 172.21.15.7:38584) 2024-01-11T07:09:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41937fe700 1 -- 172.21.15.7:0/2462857261 learned_addr learned my addr 172.21.15.7:0/2462857261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f4193fff700 1 --2- 172.21.15.7:0/2462857261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41940b0b40 0x7f4194161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41937fe700 1 -- 172.21.15.7:0/2462857261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41940b0b40 msgr2=0x7f4194161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41937fe700 1 --2- 172.21.15.7:0/2462857261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41940b0b40 0x7f4194161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41937fe700 1 -- 172.21.15.7:0/2462857261 >> v1:172.21.15.113:6789/0 conn(0x7f4194158280 legacy=0x7f41940b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41937fe700 1 -- 172.21.15.7:0/2462857261 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41940cf820 con 0x7f41940cafb0 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f4193fff700 1 --2- 172.21.15.7:0/2462857261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41940b0b40 0x7f4194161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41937fe700 1 --2- 172.21.15.7:0/2462857261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f41940ce8f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f41840099b0 tx=0x7f4184009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c8ea58bc00b7bb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.874+0000 7f41927fc700 1 -- 172.21.15.7:0/2462857261 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418400b860 con 0x7f41940cafb0 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f41927fc700 1 -- 172.21.15.7:0/2462857261 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418400b9c0 con 0x7f41940cafb0 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f41927fc700 1 -- 172.21.15.7:0/2462857261 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418400bd60 con 0x7f41940cafb0 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f4199ce8700 1 -- 172.21.15.7:0/2462857261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 msgr2=0x7f41940ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f4199ce8700 1 --2- 172.21.15.7:0/2462857261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f41940ce8f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f41840099b0 tx=0x7f4184009250 comp rx=0 tx=0).stop 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f4199ce8700 1 -- 172.21.15.7:0/2462857261 shutdown_connections 2024-01-11T07:09:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f4199ce8700 1 --2- 172.21.15.7:0/2462857261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41940b0b40 0x7f4194161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f4199ce8700 1 --2- 172.21.15.7:0/2462857261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f41940ce8f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f4199ce8700 1 -- 172.21.15.7:0/2462857261 >> 172.21.15.7:0/2462857261 conn(0x7f4194071100 msgr2=0x7f419406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f4199ce8700 1 -- 172.21.15.7:0/2462857261 shutdown_connections 2024-01-11T07:09:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.875+0000 7f4199ce8700 1 -- 172.21.15.7:0/2462857261 wait complete. 2024-01-11T07:09:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4199ce8700 1 Processor -- start 2024-01-11T07:09:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4199ce8700 1 -- start start 2024-01-11T07:09:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4199ce8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41940b0b40 0x7f419416e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4199ce8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f419416a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4199ce8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4194158280 0x7f41940b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4199ce8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41940ca7c0 con 0x7f41940cafb0 2024-01-11T07:09:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4199ce8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41940c85a0 con 0x7f41940b0b40 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4199ce8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41940ca4e0 con 0x7f4194158280 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.877+0000 7f4192ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f419416a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4192ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f419416a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38586/0 (socket says 172.21.15.7:38586) 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4192ffd700 1 -- 172.21.15.7:0/1995108492 learned_addr learned my addr 172.21.15.7:0/1995108492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f41937fe700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41940b0b40 0x7f419416e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4193fff700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4194158280 0x7f41940b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4192ffd700 1 -- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4194158280 msgr2=0x7f41940b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4192ffd700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4194158280 0x7f41940b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4192ffd700 1 -- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41940b0b40 msgr2=0x7f419416e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4192ffd700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41940b0b40 0x7f419416e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4192ffd700 1 -- 172.21.15.7:0/1995108492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840094f0 con 0x7f41940cafb0 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4192ffd700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f419416a0e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f418800c590 tx=0x7f4188007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f417b7fe700 1 -- 172.21.15.7:0/1995108492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188011590 con 0x7f41940cafb0 2024-01-11T07:09:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f417b7fe700 1 -- 172.21.15.7:0/1995108492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4188011710 con 0x7f41940cafb0 2024-01-11T07:09:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41940c7f50 con 0x7f41940cafb0 2024-01-11T07:09:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f417b7fe700 1 -- 172.21.15.7:0/1995108492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188011ad0 con 0x7f41940cafb0 2024-01-11T07:09:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.878+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419416e560 con 0x7f41940cafb0 2024-01-11T07:09:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.879+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419404c3c0 con 0x7f41940cafb0 2024-01-11T07:09:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.880+0000 7f417b7fe700 1 -- 172.21.15.7:0/1995108492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f418801b3b0 con 0x7f41940cafb0 2024-01-11T07:09:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.880+0000 7f417b7fe700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.880+0000 7f417b7fe700 1 -- 172.21.15.7:0/1995108492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4188083490 con 0x7f41940cafb0 2024-01-11T07:09:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.881+0000 7f41937fe700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.881+0000 7f41937fe700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4180066f30 0x7f41800693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f41840099b0 tx=0x7f4184017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:04.885+0000 7f417b7fe700 1 -- 172.21.15.7:0/1995108492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4188050a50 con 0x7f41940cafb0 2024-01-11T07:09:05.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.089+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4194159550 con 0x7f41940cafb0 2024-01-11T07:09:05.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.090+0000 7f417b7fe700 1 -- 172.21.15.7:0/1995108492 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+12086 (secure 0 0 0) 0x7f4188017020 con 0x7f41940cafb0 2024-01-11T07:09:05.090 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:05.090 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:09:04.277992+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4180066f30 msgr2=0x7f41800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4180066f30 0x7f41800693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f41840099b0 tx=0x7f4184017000 comp rx=0 tx=0).stop 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 msgr2=0x7f419416a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f419416a0e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f418800c590 tx=0x7f4188007f50 comp rx=0 tx=0).stop 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 shutdown_connections 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4194158280 0x7f41940b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41940b0b40 0x7f419416e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 --2- 172.21.15.7:0/1995108492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41940cafb0 0x7f419416a0e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 >> 172.21.15.7:0/1995108492 conn(0x7f4194071100 msgr2=0x7f41940cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 shutdown_connections 2024-01-11T07:09:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.093+0000 7f4199ce8700 1 -- 172.21.15.7:0/1995108492 wait complete. 2024-01-11T07:09:05.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:09:05.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:09:05.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.241+0000 7f4cc3336700 1 Processor -- start 2024-01-11T07:09:05.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.241+0000 7f4cc3336700 1 -- start start 2024-01-11T07:09:05.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.242+0000 7f4cc3336700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc15e040 0x7f4cbc15e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.242+0000 7f4cc3336700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4cbc06a370 0x7f4cbc06c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.242+0000 7f4cc3336700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cbc162080 con 0x7f4cbc06bbe0 2024-01-11T07:09:05.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.242+0000 7f4cc3336700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cbc06c680 con 0x7f4cbc06a370 2024-01-11T07:09:05.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.242+0000 7f4cc3336700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc162e40 con 0x7f4cbc15e040 2024-01-11T07:09:05.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.242+0000 7f4cc10d2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc15e040 0x7f4cbc15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.242+0000 7f4cc18d3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4cbc06a370 0x7f4cbc06c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.242+0000 7f4cc10d2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc15e040 0x7f4cbc15e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:44656/0 (socket says 172.21.15.7:44656) 2024-01-11T07:09:05.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cc10d2700 1 -- 172.21.15.7:0/2939418035 learned_addr learned my addr 172.21.15.7:0/2939418035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:05.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cc08d1700 1 --1- 172.21.15.7:0/2939418035 >> v1:172.21.15.113:6789/0 conn(0x7f4cbc06bbe0 0x7f4cbc069c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:46130/0 (socket says 172.21.15.7:46130) 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cc18d3700 1 -- 172.21.15.7:0/2939418035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc15e040 msgr2=0x7f4cbc15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cc18d3700 1 --2- 172.21.15.7:0/2939418035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc15e040 0x7f4cbc15e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cc18d3700 1 -- 172.21.15.7:0/2939418035 >> v1:172.21.15.113:6789/0 conn(0x7f4cbc06bbe0 legacy=0x7f4cbc069c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cc18d3700 1 -- 172.21.15.7:0/2939418035 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc0c9450 con 0x7f4cbc06a370 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cb3fff700 1 -- 172.21.15.7:0/2939418035 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331143606 0 0) 0x7f4cbc162080 con 0x7f4cbc06bbe0 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cc10d2700 1 --2- 172.21.15.7:0/2939418035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc15e040 0x7f4cbc15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cc18d3700 1 --2- 172.21.15.7:0/2939418035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 0x7f4cbc06c050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009e60 tx=0x7f4cb4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120e6deb9a6b9cbf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cb3fff700 1 -- 172.21.15.7:0/2939418035 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400e070 con 0x7f4cbc06a370 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cb3fff700 1 -- 172.21.15.7:0/2939418035 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb400bb60 con 0x7f4cbc06a370 2024-01-11T07:09:05.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.243+0000 7f4cb3fff700 1 -- 172.21.15.7:0/2939418035 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4012430 con 0x7f4cbc06a370 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.244+0000 7f4cc3336700 1 -- 172.21.15.7:0/2939418035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 msgr2=0x7f4cbc06c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.244+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2939418035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 0x7f4cbc06c050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009e60 tx=0x7f4cb4009780 comp rx=0 tx=0).stop 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.244+0000 7f4cc3336700 1 -- 172.21.15.7:0/2939418035 shutdown_connections 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.244+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2939418035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 0x7f4cbc06c050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.244+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2939418035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc15e040 0x7f4cbc15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.244+0000 7f4cc3336700 1 -- 172.21.15.7:0/2939418035 >> 172.21.15.7:0/2939418035 conn(0x7f4cbc0cd8f0 msgr2=0x7f4cbc0cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.244+0000 7f4cc3336700 1 -- 172.21.15.7:0/2939418035 shutdown_connections 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.244+0000 7f4cc3336700 1 -- 172.21.15.7:0/2939418035 wait complete. 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.245+0000 7f4cc3336700 1 Processor -- start 2024-01-11T07:09:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.245+0000 7f4cc3336700 1 -- start start 2024-01-11T07:09:05.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc3336700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 0x7f4cbc0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc3336700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc06bbe0 0x7f4cbc0c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc3336700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc15e040 0x7f4cbc0c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc3336700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc0c4f80 con 0x7f4cbc15e040 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc3336700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc1610a0 con 0x7f4cbc06a370 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc3336700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc160520 con 0x7f4cbc06bbe0 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc18d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc15e040 0x7f4cbc0c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc18d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc15e040 0x7f4cbc0c3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38592/0 (socket says 172.21.15.7:38592) 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.246+0000 7f4cc18d3700 1 -- 172.21.15.7:0/2962765437 learned_addr learned my addr 172.21.15.7:0/2962765437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc08d1700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc06bbe0 0x7f4cbc0c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc10d2700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 0x7f4cbc0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc18d3700 1 -- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc06bbe0 msgr2=0x7f4cbc0c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc18d3700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc06bbe0 0x7f4cbc0c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc18d3700 1 -- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 msgr2=0x7f4cbc0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc18d3700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 0x7f4cbc0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc18d3700 1 -- 172.21.15.7:0/2962765437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb80094f0 con 0x7f4cbc15e040 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc08d1700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc06bbe0 0x7f4cbc0c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc18d3700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc15e040 0x7f4cbc0c3780 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc06d5c0 tx=0x7f4cb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cb17fa700 1 -- 172.21.15.7:0/2962765437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4012e40 con 0x7f4cbc15e040 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cb17fa700 1 -- 172.21.15.7:0/2962765437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb401d630 con 0x7f4cbc15e040 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb4009a20 con 0x7f4cbc15e040 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cb17fa700 1 -- 172.21.15.7:0/2962765437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb401d9f0 con 0x7f4cbc15e040 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.247+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc160dc0 con 0x7f4cbc15e040 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.248+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cbc064a40 con 0x7f4cbc15e040 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.249+0000 7f4cb17fa700 1 -- 172.21.15.7:0/2962765437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4cb4024070 con 0x7f4cbc15e040 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.249+0000 7f4cb17fa700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ca0066f30 0x7f4ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.249+0000 7f4cb17fa700 1 -- 172.21.15.7:0/2962765437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4cb4086b30 con 0x7f4cbc15e040 2024-01-11T07:09:05.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.249+0000 7f4cc10d2700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ca0066f30 0x7f4ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.250+0000 7f4cc10d2700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ca0066f30 0x7f4ca00693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8006d40 tx=0x7f4cb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.254+0000 7f4cb17fa700 1 -- 172.21.15.7:0/2962765437 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cb4054050 con 0x7f4cbc15e040 2024-01-11T07:09:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.457+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cbc160910 con 0x7f4cbc15e040 2024-01-11T07:09:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.458+0000 7f4cb17fa700 1 -- 172.21.15.7:0/2962765437 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cb402c020 con 0x7f4cbc15e040 2024-01-11T07:09:05.458 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:09:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ca0066f30 msgr2=0x7f4ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ca0066f30 0x7f4ca00693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8006d40 tx=0x7f4cb8008040 comp rx=0 tx=0).stop 2024-01-11T07:09:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc15e040 msgr2=0x7f4cbc0c3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc15e040 0x7f4cbc0c3780 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc06d5c0 tx=0x7f4cb4019000 comp rx=0 tx=0).stop 2024-01-11T07:09:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 shutdown_connections 2024-01-11T07:09:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ca0066f30 0x7f4ca00693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cbc15e040 0x7f4cbc0c3780 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4cbc06a370 0x7f4cbc0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 --2- 172.21.15.7:0/2962765437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4cbc06bbe0 0x7f4cbc0c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 >> 172.21.15.7:0/2962765437 conn(0x7f4cbc0cd8f0 msgr2=0x7f4cbc06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 shutdown_connections 2024-01-11T07:09:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.460+0000 7f4cc3336700 1 -- 172.21.15.7:0/2962765437 wait complete. 2024-01-11T07:09:05.473 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:09:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.604+0000 7f61fa8d3700 1 Processor -- start 2024-01-11T07:09:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.605+0000 7f61fa8d3700 1 -- start start 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.605+0000 7f61fa8d3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61f406dd00 0x7f61f40c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.605+0000 7f61fa8d3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61f406e0d0 0x7f61f4159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.605+0000 7f61fa8d3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f415b770 con 0x7f61f4159ff0 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.605+0000 7f61fa8d3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f415abf0 con 0x7f61f406dd00 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.605+0000 7f61fa8d3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f415aed0 con 0x7f61f406e0d0 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61f406dd00 0x7f61f40c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61f406dd00 0x7f61f40c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38606/0 (socket says 172.21.15.7:38606) 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f3fff700 1 -- 172.21.15.7:0/3874693328 learned_addr learned my addr 172.21.15.7:0/3874693328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f37fe700 1 --2- 172.21.15.7:0/3874693328 >> v2:172.21.15.113:3300/0 conn(0x7f61f406e0d0 0x7f61f4159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f3fff700 1 -- 172.21.15.7:0/3874693328 >> v2:172.21.15.113:3300/0 conn(0x7f61f406e0d0 msgr2=0x7f61f4159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f3fff700 1 --2- 172.21.15.7:0/3874693328 >> v2:172.21.15.113:3300/0 conn(0x7f61f406e0d0 0x7f61f4159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f3fff700 1 -- 172.21.15.7:0/3874693328 >> v1:172.21.15.113:6789/0 conn(0x7f61f4159ff0 legacy=0x7f61f40c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f3fff700 1 -- 172.21.15.7:0/3874693328 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f415b1b0 con 0x7f61f406dd00 2024-01-11T07:09:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f3fff700 1 --2- 172.21.15.7:0/3874693328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406dd00 0x7f61f40c98d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f61f406d2e0 tx=0x7f61e400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65507d8d79856392 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f2ffd700 1 -- 172.21.15.7:0/3874693328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e400dba0 con 0x7f61f406dd00 2024-01-11T07:09:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f2ffd700 1 -- 172.21.15.7:0/3874693328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e400dd00 con 0x7f61f406dd00 2024-01-11T07:09:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.606+0000 7f61f2ffd700 1 -- 172.21.15.7:0/3874693328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40196d0 con 0x7f61f406dd00 2024-01-11T07:09:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.607+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3874693328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406dd00 msgr2=0x7f61f40c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.607+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3874693328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406dd00 0x7f61f40c98d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f61f406d2e0 tx=0x7f61e400b4f0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.607+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3874693328 shutdown_connections 2024-01-11T07:09:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.607+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3874693328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406dd00 0x7f61f40c98d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.607+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3874693328 >> v2:172.21.15.113:3300/0 conn(0x7f61f406e0d0 0x7f61f4159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.607+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3874693328 >> 172.21.15.7:0/3874693328 conn(0x7f61f40680e0 msgr2=0x7f61f40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.607+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3874693328 shutdown_connections 2024-01-11T07:09:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.607+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3874693328 wait complete. 2024-01-11T07:09:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.608+0000 7f61fa8d3700 1 Processor -- start 2024-01-11T07:09:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61fa8d3700 1 -- start start 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61fa8d3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61f406dd00 0x7f61f40b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61fa8d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406e0d0 0x7f61f4166a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61fa8d3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61f4159ff0 0x7f61f40c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61fa8d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f41571b0 con 0x7f61f406e0d0 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61fa8d3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40b06c0 con 0x7f61f406dd00 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61fa8d3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40c6ee0 con 0x7f61f4159ff0 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61f37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406e0d0 0x7f61f4166a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61f37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406e0d0 0x7f61f4166a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38622/0 (socket says 172.21.15.7:38622) 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.609+0000 7f61f37fe700 1 -- 172.21.15.7:0/3147229592 learned_addr learned my addr 172.21.15.7:0/3147229592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61f3fff700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61f406dd00 0x7f61f40b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61f8e70700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61f4159ff0 0x7f61f40c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61f37fe700 1 -- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61f4159ff0 msgr2=0x7f61f40c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61f37fe700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61f4159ff0 0x7f61f40c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61f37fe700 1 -- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61f406dd00 msgr2=0x7f61f40b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61f37fe700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61f406dd00 0x7f61f40b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61f37fe700 1 -- 172.21.15.7:0/3147229592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e400b790 con 0x7f61f406e0d0 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61f37fe700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406e0d0 0x7f61f4166a60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f61e800ec80 tx=0x7f61e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61dbfff700 1 -- 172.21.15.7:0/3147229592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e800c850 con 0x7f61f406e0d0 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61dbfff700 1 -- 172.21.15.7:0/3147229592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e800c9b0 con 0x7f61f406e0d0 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f40b0910 con 0x7f61f406e0d0 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61dbfff700 1 -- 172.21.15.7:0/3147229592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e800cd50 con 0x7f61f406e0d0 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.610+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f40b0ed0 con 0x7f61f406e0d0 2024-01-11T07:09:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.611+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f404c3c0 con 0x7f61f406e0d0 2024-01-11T07:09:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.612+0000 7f61dbfff700 1 -- 172.21.15.7:0/3147229592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f61e8013680 con 0x7f61f406e0d0 2024-01-11T07:09:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.612+0000 7f61dbfff700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61e0066f30 0x7f61e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.612+0000 7f61dbfff700 1 -- 172.21.15.7:0/3147229592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f61e8084440 con 0x7f61f406e0d0 2024-01-11T07:09:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.612+0000 7f61f3fff700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61e0066f30 0x7f61e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.613+0000 7f61f3fff700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61e0066f30 0x7f61e00693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f61f406d2e0 tx=0x7f61e401b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.617+0000 7f61dbfff700 1 -- 172.21.15.7:0/3147229592 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61e8051a00 con 0x7f61f406e0d0 2024-01-11T07:09:05.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.823+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f61f415b490 con 0x7f61f406e0d0 2024-01-11T07:09:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.823+0000 7f61dbfff700 1 -- 172.21.15.7:0/3147229592 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+65 (secure 0 0 0) 0x7f61e8055020 con 0x7f61f406e0d0 2024-01-11T07:09:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:06.019 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:09:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:09:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:09:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:09:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:09:06.019 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61e0066f30 msgr2=0x7f61e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61e0066f30 0x7f61e00693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f61f406d2e0 tx=0x7f61e401b000 comp rx=0 tx=0).stop 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406e0d0 msgr2=0x7f61f4166a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406e0d0 0x7f61f4166a60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f61e800ec80 tx=0x7f61e80091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 shutdown_connections 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f61e0066f30 0x7f61e00693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61f4159ff0 0x7f61f40c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61f406dd00 0x7f61f40b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 --2- 172.21.15.7:0/3147229592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f406e0d0 0x7f61f4166a60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 >> 172.21.15.7:0/3147229592 conn(0x7f61f40680e0 msgr2=0x7f61f4158ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:06.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 shutdown_connections 2024-01-11T07:09:06.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:05.825+0000 7f61fa8d3700 1 -- 172.21.15.7:0/3147229592 wait complete. 2024-01-11T07:09:06.022 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:09:06.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:09:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.167+0000 7fbdc3f0e700 1 Processor -- start 2024-01-11T07:09:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.167+0000 7fbdc3f0e700 1 -- start start 2024-01-11T07:09:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc3f0e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdbc0afcf0 0x7fbdbc06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc3f0e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc063bf0 0x7fbdbc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc3f0e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdbc0cd090 0x7fbdbc06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc3f0e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc06b710 con 0x7fbdbc0afcf0 2024-01-11T07:09:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc3f0e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc161d20 con 0x7fbdbc0cd090 2024-01-11T07:09:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc3f0e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc162000 con 0x7fbdbc063bf0 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc1caa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdbc0afcf0 0x7fbdbc06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc1caa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdbc0afcf0 0x7fbdbc06b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38634/0 (socket says 172.21.15.7:38634) 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc1caa700 1 -- 172.21.15.7:0/3054748161 learned_addr learned my addr 172.21.15.7:0/3054748161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc14a9700 1 --2- 172.21.15.7:0/3054748161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc063bf0 0x7fbdbc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc24ab700 1 --2- 172.21.15.7:0/3054748161 >> v2:172.21.15.113:3300/0 conn(0x7fbdbc0cd090 0x7fbdbc06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc1caa700 1 -- 172.21.15.7:0/3054748161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc063bf0 msgr2=0x7fbdbc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc1caa700 1 --2- 172.21.15.7:0/3054748161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc063bf0 0x7fbdbc0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc1caa700 1 -- 172.21.15.7:0/3054748161 >> v2:172.21.15.113:3300/0 conn(0x7fbdbc0cd090 msgr2=0x7fbdbc06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc1caa700 1 --2- 172.21.15.7:0/3054748161 >> v2:172.21.15.113:3300/0 conn(0x7fbdbc0cd090 0x7fbdbc06dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.168+0000 7fbdc1caa700 1 -- 172.21.15.7:0/3054748161 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc1622e0 con 0x7fbdbc0afcf0 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc1caa700 1 --2- 172.21.15.7:0/3054748161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc0afcf0 0x7fbdbc06b0c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0c9ca0 tx=0x7fbdb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4d82b0cc4ab7ad server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc0ca8700 1 -- 172.21.15.7:0/3054748161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800b860 con 0x7fbdbc0afcf0 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc0ca8700 1 -- 172.21.15.7:0/3054748161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb800b9c0 con 0x7fbdbc0afcf0 2024-01-11T07:09:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc0ca8700 1 -- 172.21.15.7:0/3054748161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800bd60 con 0x7fbdbc0afcf0 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/3054748161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc0afcf0 msgr2=0x7fbdbc06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/3054748161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc0afcf0 0x7fbdbc06b0c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0c9ca0 tx=0x7fbdb8009250 comp rx=0 tx=0).stop 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/3054748161 shutdown_connections 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/3054748161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc0afcf0 0x7fbdbc06b0c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/3054748161 >> v2:172.21.15.113:3300/0 conn(0x7fbdbc0cd090 0x7fbdbc06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/3054748161 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc063bf0 0x7fbdbc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.169+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/3054748161 >> 172.21.15.7:0/3054748161 conn(0x7fbdbc0680a0 msgr2=0x7fbdbc0cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.170+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/3054748161 shutdown_connections 2024-01-11T07:09:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.170+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/3054748161 wait complete. 2024-01-11T07:09:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.171+0000 7fbdc3f0e700 1 Processor -- start 2024-01-11T07:09:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.171+0000 7fbdc3f0e700 1 -- start start 2024-01-11T07:09:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.171+0000 7fbdc3f0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc063bf0 0x7fbdbc149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc3f0e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc0afcf0 0x7fbdbc15e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc3f0e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdbc0cd090 0x7fbdbc15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc3f0e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0b2890 con 0x7fbdbc063bf0 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc3f0e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0b2b70 con 0x7fbdbc0cd090 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc3f0e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc0b2e50 con 0x7fbdbc0afcf0 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc063bf0 0x7fbdbc149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc063bf0 0x7fbdbc149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38638/0 (socket says 172.21.15.7:38638) 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 -- 172.21.15.7:0/1849062905 learned_addr learned my addr 172.21.15.7:0/1849062905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc24ab700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdbc0cd090 0x7fbdbc15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc14a9700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc0afcf0 0x7fbdbc15e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 -- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc0afcf0 msgr2=0x7fbdbc15e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc0afcf0 0x7fbdbc15e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 -- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdbc0cd090 msgr2=0x7fbdbc15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdbc0cd090 0x7fbdbc15ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 -- 172.21.15.7:0/1849062905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb80094f0 con 0x7fbdbc063bf0 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc1caa700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc063bf0 0x7fbdbc149890 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0cb400 tx=0x7fbdb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdb1ffb700 1 -- 172.21.15.7:0/1849062905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8024440 con 0x7fbdbc063bf0 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdb1ffb700 1 -- 172.21.15.7:0/1849062905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb80245a0 con 0x7fbdbc063bf0 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc0b3130 con 0x7fbdbc063bf0 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.172+0000 7fbdb1ffb700 1 -- 172.21.15.7:0/1849062905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8024960 con 0x7fbdbc063bf0 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.173+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc149dd0 con 0x7fbdbc063bf0 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.174+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdbc068700 con 0x7fbdbc063bf0 2024-01-11T07:09:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.174+0000 7fbdb1ffb700 1 -- 172.21.15.7:0/1849062905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbdb801a420 con 0x7fbdbc063bf0 2024-01-11T07:09:06.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.175+0000 7fbdb1ffb700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbda0066f30 0x7fbda00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:06.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.175+0000 7fbdb1ffb700 1 -- 172.21.15.7:0/1849062905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbdb8086000 con 0x7fbdbc063bf0 2024-01-11T07:09:06.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.175+0000 7fbdc14a9700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbda0066f30 0x7fbda00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:06.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.175+0000 7fbdc14a9700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbda0066f30 0x7fbda00693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc15c370 tx=0x7fbdac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.179+0000 7fbdb1ffb700 1 -- 172.21.15.7:0/1849062905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdb80535c0 con 0x7fbdbc063bf0 2024-01-11T07:09:06.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:06.383+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbdbc04c3c0 con 0x7fbdbc063bf0 2024-01-11T07:09:07.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.280+0000 7fbdb1ffb700 1 -- 172.21.15.7:0/1849062905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbdb8056be0 con 0x7fbdbc063bf0 2024-01-11T07:09:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.285+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbda0066f30 msgr2=0x7fbda00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.285+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbda0066f30 0x7fbda00693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc15c370 tx=0x7fbdac008040 comp rx=0 tx=0).stop 2024-01-11T07:09:07.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.285+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc063bf0 msgr2=0x7fbdbc149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc063bf0 0x7fbdbc149890 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0cb400 tx=0x7fbdb8017000 comp rx=0 tx=0).stop 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 shutdown_connections 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbda0066f30 0x7fbda00693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdbc0cd090 0x7fbdbc15ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdbc063bf0 0x7fbdbc149890 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 --2- 172.21.15.7:0/1849062905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdbc0afcf0 0x7fbdbc15e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 >> 172.21.15.7:0/1849062905 conn(0x7fbdbc0680a0 msgr2=0x7fbdbc0c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 shutdown_connections 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.286+0000 7fbdc3f0e700 1 -- 172.21.15.7:0/1849062905 wait complete. 2024-01-11T07:09:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:09:07.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:09:07.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.444+0000 7fc02b936700 1 Processor -- start 2024-01-11T07:09:07.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.444+0000 7fc02b936700 1 -- start start 2024-01-11T07:09:07.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc02b936700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc0241579b0 0x7fc02415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc02b936700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0240aeab0 con 0x7fc024157d80 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc02b936700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0240aed90 con 0x7fc024063940 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc02b936700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0240af070 con 0x7fc0241579b0 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc028ed1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc024157d80 0x7fc024063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60460/0 (socket says 172.21.15.7:60460) 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc028ed1700 1 -- 172.21.15.7:0/3366298221 learned_addr learned my addr 172.21.15.7:0/3366298221 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc0296d2700 1 --2- 172.21.15.7:0/3366298221 >> v2:172.21.15.113:3300/0 conn(0x7fc0241579b0 0x7fc02415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc01bfff700 1 -- 172.21.15.7:0/3366298221 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510068946 0 0) 0x7fc0240aeab0 con 0x7fc024157d80 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.445+0000 7fc01bfff700 1 -- 172.21.15.7:0/3366298221 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc010003570 con 0x7fc024157d80 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc01bfff700 1 -- 172.21.15.7:0/3366298221 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856252997 0 0) 0x7fc0240aed90 con 0x7fc024063940 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc01bfff700 1 -- 172.21.15.7:0/3366298221 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0240aeab0 con 0x7fc024063940 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc01bfff700 1 -- 172.21.15.7:0/3366298221 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3249606255 0 0) 0x7fc010003570 con 0x7fc024157d80 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc0296d2700 1 -- 172.21.15.7:0/3366298221 >> v1:172.21.15.113:6789/0 conn(0x7fc024063940 legacy=0x7fc024069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc0296d2700 1 -- 172.21.15.7:0/3366298221 >> v1:172.21.15.7:6789/0 conn(0x7fc024157d80 legacy=0x7fc024063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:07.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc0296d2700 1 -- 172.21.15.7:0/3366298221 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0240af350 con 0x7fc0241579b0 2024-01-11T07:09:07.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc0296d2700 1 --2- 172.21.15.7:0/3366298221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0241579b0 0x7fc02415bf50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc0200098e0 tx=0x7fc0200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe126c365eb70201 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:07.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc01bfff700 1 -- 172.21.15.7:0/3366298221 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014001470 con 0x7fc0241579b0 2024-01-11T07:09:07.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc01bfff700 1 -- 172.21.15.7:0/3366298221 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc020011420 con 0x7fc0241579b0 2024-01-11T07:09:07.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.446+0000 7fc01bfff700 1 -- 172.21.15.7:0/3366298221 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0200117c0 con 0x7fc0241579b0 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.447+0000 7fc02b936700 1 -- 172.21.15.7:0/3366298221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0241579b0 msgr2=0x7fc02415bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.447+0000 7fc02b936700 1 --2- 172.21.15.7:0/3366298221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0241579b0 0x7fc02415bf50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc0200098e0 tx=0x7fc0200091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.447+0000 7fc02b936700 1 -- 172.21.15.7:0/3366298221 shutdown_connections 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.447+0000 7fc02b936700 1 --2- 172.21.15.7:0/3366298221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0241579b0 0x7fc02415bf50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.447+0000 7fc02b936700 1 -- 172.21.15.7:0/3366298221 >> 172.21.15.7:0/3366298221 conn(0x7fc0240680e0 msgr2=0x7fc0240cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.447+0000 7fc02b936700 1 -- 172.21.15.7:0/3366298221 shutdown_connections 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.447+0000 7fc02b936700 1 -- 172.21.15.7:0/3366298221 wait complete. 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.448+0000 7fc02b936700 1 Processor -- start 2024-01-11T07:09:07.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.448+0000 7fc02b936700 1 -- start start 2024-01-11T07:09:07.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc02b936700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc024063940 0x7fc0240c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:07.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc02b936700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0241579b0 0x7fc02415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:07.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc02b936700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc024157d80 0x7fc0240c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc02b936700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240c2750 con 0x7fc024157d80 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc02b936700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0241684d0 con 0x7fc024063940 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc02b936700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc024167c00 con 0x7fc0241579b0 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc029ed3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc024157d80 0x7fc0240c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc029ed3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc024157d80 0x7fc0240c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47052/0 (socket says 172.21.15.7:47052) 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc029ed3700 1 -- 172.21.15.7:0/3996146836 learned_addr learned my addr 172.21.15.7:0/3996146836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc028ed1700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0241579b0 0x7fc02415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.449+0000 7fc0296d2700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc024063940 0x7fc0240c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc029ed3700 1 -- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0241579b0 msgr2=0x7fc02415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc029ed3700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0241579b0 0x7fc02415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc029ed3700 1 -- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc024063940 msgr2=0x7fc0240c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc029ed3700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc024063940 0x7fc0240c1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:07.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc029ed3700 1 -- 172.21.15.7:0/3996146836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020009420 con 0x7fc024157d80 2024-01-11T07:09:07.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc028ed1700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0241579b0 0x7fc02415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:09:07.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc029ed3700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc024157d80 0x7fc0240c6a30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00c910 tx=0x7fc01c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:07.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc0197fa700 1 -- 172.21.15.7:0/3996146836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00a7a0 con 0x7fc024157d80 2024-01-11T07:09:07.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc0197fa700 1 -- 172.21.15.7:0/3996146836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c00a920 con 0x7fc024157d80 2024-01-11T07:09:07.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0240c24d0 con 0x7fc024157d80 2024-01-11T07:09:07.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc0197fa700 1 -- 172.21.15.7:0/3996146836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00ace0 con 0x7fc024157d80 2024-01-11T07:09:07.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.450+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0240c20f0 con 0x7fc024157d80 2024-01-11T07:09:07.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.451+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc024168740 con 0x7fc024157d80 2024-01-11T07:09:07.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.452+0000 7fc0197fa700 1 -- 172.21.15.7:0/3996146836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc01c012630 con 0x7fc024157d80 2024-01-11T07:09:07.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.452+0000 7fc0197fa700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc008066f30 0x7fc0080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:07.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.452+0000 7fc0197fa700 1 -- 172.21.15.7:0/3996146836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fc01c084780 con 0x7fc024157d80 2024-01-11T07:09:07.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.452+0000 7fc0296d2700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc008066f30 0x7fc0080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:07.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.453+0000 7fc0296d2700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc008066f30 0x7fc0080693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0200097b0 tx=0x7fc020016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:07.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.457+0000 7fc0197fa700 1 -- 172.21.15.7:0/3996146836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc01c051c80 con 0x7fc024157d80 2024-01-11T07:09:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:07.662+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc0240af630 con 0x7fc024157d80 2024-01-11T07:09:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.284+0000 7fc0197fa700 1 -- 172.21.15.7:0/3996146836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc01c0552a0 con 0x7fc024157d80 2024-01-11T07:09:08.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.292+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc008066f30 msgr2=0x7fc0080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:08.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.292+0000 7fc02b936700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc008066f30 0x7fc0080693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0200097b0 tx=0x7fc020016000 comp rx=0 tx=0).stop 2024-01-11T07:09:08.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.292+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc024157d80 msgr2=0x7fc0240c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:08.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.292+0000 7fc02b936700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc024157d80 0x7fc0240c6a30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc01c00c910 tx=0x7fc01c007f90 comp rx=0 tx=0).stop 2024-01-11T07:09:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.296+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 shutdown_connections 2024-01-11T07:09:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.296+0000 7fc02b936700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc008066f30 0x7fc0080693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.296+0000 7fc02b936700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc024157d80 0x7fc0240c6a30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.296+0000 7fc02b936700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc024063940 0x7fc0240c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.296+0000 7fc02b936700 1 --2- 172.21.15.7:0/3996146836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0241579b0 0x7fc02415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.296+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 >> 172.21.15.7:0/3996146836 conn(0x7fc0240680e0 msgr2=0x7fc0240b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:08.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.297+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 shutdown_connections 2024-01-11T07:09:08.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.298+0000 7fc02b936700 1 -- 172.21.15.7:0/3996146836 wait complete. 2024-01-11T07:09:08.300 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:09:08.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:09:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.454+0000 7fe7bc6aa700 1 Processor -- start 2024-01-11T07:09:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.454+0000 7fe7bc6aa700 1 -- start start 2024-01-11T07:09:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bc6aa700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7b40cbb60 0x7fe7b415a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bc6aa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7b415e5a0 0x7fe7b415a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bc6aa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b415e110 con 0x7fe7b40cb790 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bc6aa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b415b330 con 0x7fe7b415e5a0 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bc6aa700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b415ad90 con 0x7fe7b40cbb60 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bac47700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7b415e5a0 0x7fe7b415a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7ba446700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe7b40cb790 0x7fe7b40cdc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60466/0 (socket says 172.21.15.7:60466) 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7ba446700 1 -- 172.21.15.7:0/2400424713 learned_addr learned my addr 172.21.15.7:0/2400424713 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7b9c45700 1 --2- 172.21.15.7:0/2400424713 >> v2:172.21.15.113:3300/0 conn(0x7fe7b40cbb60 0x7fe7b415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bac47700 1 -- 172.21.15.7:0/2400424713 >> v2:172.21.15.113:3300/0 conn(0x7fe7b40cbb60 msgr2=0x7fe7b415a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bac47700 1 --2- 172.21.15.7:0/2400424713 >> v2:172.21.15.113:3300/0 conn(0x7fe7b40cbb60 0x7fe7b415a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bac47700 1 -- 172.21.15.7:0/2400424713 >> v1:172.21.15.7:6789/0 conn(0x7fe7b40cb790 legacy=0x7fe7b40cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7bac47700 1 -- 172.21.15.7:0/2400424713 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b415b050 con 0x7fe7b415e5a0 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.455+0000 7fe7b9444700 1 -- 172.21.15.7:0/2400424713 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397286767 0 0) 0x7fe7b415e110 con 0x7fe7b40cb790 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.456+0000 7fe7bac47700 1 --2- 172.21.15.7:0/2400424713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b415e5a0 0x7fe7b415a850 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40697f0 tx=0x7fe7ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3c0678ecad8c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.456+0000 7fe7b9444700 1 -- 172.21.15.7:0/2400424713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac00e070 con 0x7fe7b415e5a0 2024-01-11T07:09:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.456+0000 7fe7b9444700 1 -- 172.21.15.7:0/2400424713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac00bb60 con 0x7fe7b415e5a0 2024-01-11T07:09:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.456+0000 7fe7b9444700 1 -- 172.21.15.7:0/2400424713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac012430 con 0x7fe7b415e5a0 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.456+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/2400424713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b415e5a0 msgr2=0x7fe7b415a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.456+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/2400424713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b415e5a0 0x7fe7b415a850 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40697f0 tx=0x7fe7ac009780 comp rx=0 tx=0).stop 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.456+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/2400424713 shutdown_connections 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.456+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/2400424713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b415e5a0 0x7fe7b415a850 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.457+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/2400424713 >> v2:172.21.15.113:3300/0 conn(0x7fe7b40cbb60 0x7fe7b415a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.457+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/2400424713 >> 172.21.15.7:0/2400424713 conn(0x7fe7b406ae00 msgr2=0x7fe7b406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.457+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/2400424713 shutdown_connections 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.457+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/2400424713 wait complete. 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.458+0000 7fe7bc6aa700 1 Processor -- start 2024-01-11T07:09:08.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.458+0000 7fe7bc6aa700 1 -- start start 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7bc6aa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7b40cb790 0x7fe7b4157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7bc6aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b40cbb60 0x7fe7b40c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7bc6aa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7b415e5a0 0x7fe7b4158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7bc6aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40c7520 con 0x7fe7b40cbb60 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7bc6aa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40c6480 con 0x7fe7b415e5a0 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7bc6aa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40c6760 con 0x7fe7b40cb790 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7b9c45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b40cbb60 0x7fe7b40c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7b9c45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b40cbb60 0x7fe7b40c5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47060/0 (socket says 172.21.15.7:47060) 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.459+0000 7fe7b9c45700 1 -- 172.21.15.7:0/772292295 learned_addr learned my addr 172.21.15.7:0/772292295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7bac47700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7b415e5a0 0x7fe7b4158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7ba446700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7b40cb790 0x7fe7b4157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7b9c45700 1 -- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7b40cb790 msgr2=0x7fe7b4157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7b9c45700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7b40cb790 0x7fe7b4157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7b9c45700 1 -- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7b415e5a0 msgr2=0x7fe7b4158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7b9c45700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7b415e5a0 0x7fe7b4158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7b9c45700 1 -- 172.21.15.7:0/772292295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac009a20 con 0x7fe7b40cbb60 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7b9c45700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b40cbb60 0x7fe7b40c5aa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400ec80 tx=0x7fe7a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7aa7fc700 1 -- 172.21.15.7:0/772292295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400c850 con 0x7fe7b40cbb60 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7aa7fc700 1 -- 172.21.15.7:0/772292295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a400c9b0 con 0x7fe7b40cbb60 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b40c6a40 con 0x7fe7b40cbb60 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7aa7fc700 1 -- 172.21.15.7:0/772292295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400cd50 con 0x7fe7b40cbb60 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.460+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b40c7000 con 0x7fe7b40cbb60 2024-01-11T07:09:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.461+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b404c3c0 con 0x7fe7b40cbb60 2024-01-11T07:09:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.462+0000 7fe7aa7fc700 1 -- 172.21.15.7:0/772292295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe7a4013680 con 0x7fe7b40cbb60 2024-01-11T07:09:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.462+0000 7fe7aa7fc700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe798066f30 0x7fe7980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.463+0000 7fe7aa7fc700 1 -- 172.21.15.7:0/772292295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fe7a4084700 con 0x7fe7b40cbb60 2024-01-11T07:09:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.463+0000 7fe7ba446700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe798066f30 0x7fe7980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:08.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.463+0000 7fe7ba446700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe798066f30 0x7fe7980693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007fd0 tx=0x7fe7b0007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.467+0000 7fe7aa7fc700 1 -- 172.21.15.7:0/772292295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7a40519d0 con 0x7fe7b40cbb60 2024-01-11T07:09:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:08.715+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe7b415dac0 con 0x7fe7b40cbb60 2024-01-11T07:09:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.306+0000 7fe7aa7fc700 1 -- 172.21.15.7:0/772292295 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe7a4054ff0 con 0x7fe7b40cbb60 2024-01-11T07:09:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe798066f30 msgr2=0x7fe7980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe798066f30 0x7fe7980693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007fd0 tx=0x7fe7b0007d30 comp rx=0 tx=0).stop 2024-01-11T07:09:09.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b40cbb60 msgr2=0x7fe7b40c5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b40cbb60 0x7fe7b40c5aa0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400ec80 tx=0x7fe7a40091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 shutdown_connections 2024-01-11T07:09:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe798066f30 0x7fe7980693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7b415e5a0 0x7fe7b4158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7b40cb790 0x7fe7b4157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 --2- 172.21.15.7:0/772292295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7b40cbb60 0x7fe7b40c5aa0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 >> 172.21.15.7:0/772292295 conn(0x7fe7b406ae00 msgr2=0x7fe7b406f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.322+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 shutdown_connections 2024-01-11T07:09:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.323+0000 7fe7bc6aa700 1 -- 172.21.15.7:0/772292295 wait complete. 2024-01-11T07:09:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 8 and data pool 9 2024-01-11T07:09:09.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:09:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.480+0000 7f6aec188700 1 Processor -- start 2024-01-11T07:09:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.480+0000 7f6aec188700 1 -- start start 2024-01-11T07:09:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.481+0000 7f6aec188700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae415b260 0x7f6ae40b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.481+0000 7f6aec188700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ae4069c80 0x7f6ae406a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.481+0000 7f6aec188700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae406e3f0 con 0x7f6ae4156760 2024-01-11T07:09:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.481+0000 7f6aec188700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae406ef90 con 0x7f6ae4069c80 2024-01-11T07:09:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.481+0000 7f6aec188700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae406a640 con 0x7f6ae415b260 2024-01-11T07:09:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6aea725700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ae4069c80 0x7f6ae406a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6ae9723700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae415b260 0x7f6ae40b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6aea725700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ae4069c80 0x7f6ae406a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:41962/0 (socket says 172.21.15.7:41962) 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6aea725700 1 -- 172.21.15.7:0/1347880091 learned_addr learned my addr 172.21.15.7:0/1347880091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6ae9723700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae415b260 0x7f6ae40b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:60090/0 (socket says 172.21.15.7:60090) 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6ae9f24700 1 --1- 172.21.15.7:0/1347880091 >> v1:172.21.15.113:6789/0 conn(0x7f6ae4156760 0x7f6ae406e880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:56752/0 (socket says 172.21.15.7:56752) 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6aea725700 1 -- 172.21.15.7:0/1347880091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae415b260 msgr2=0x7f6ae40b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6aea725700 1 --2- 172.21.15.7:0/1347880091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae415b260 0x7f6ae40b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6aea725700 1 -- 172.21.15.7:0/1347880091 >> v1:172.21.15.113:6789/0 conn(0x7f6ae4156760 legacy=0x7f6ae406e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6aea725700 1 -- 172.21.15.7:0/1347880091 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae4157b40 con 0x7f6ae4069c80 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6ae8f22700 1 -- 172.21.15.7:0/1347880091 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064385766 0 0) 0x7f6ae406e3f0 con 0x7f6ae4156760 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6aea725700 1 --2- 172.21.15.7:0/1347880091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4069c80 0x7f6ae406a100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c040 tx=0x7f6adc00d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a7e097187786c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6ae8f22700 1 -- 172.21.15.7:0/1347880091 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00b2e0 con 0x7f6ae4069c80 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6ae8f22700 1 -- 172.21.15.7:0/1347880091 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00b440 con 0x7f6ae4069c80 2024-01-11T07:09:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.482+0000 7f6ae8f22700 1 -- 172.21.15.7:0/1347880091 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00ccb0 con 0x7f6ae4069c80 2024-01-11T07:09:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.483+0000 7f6aec188700 1 -- 172.21.15.7:0/1347880091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4069c80 msgr2=0x7f6ae406a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.483+0000 7f6aec188700 1 --2- 172.21.15.7:0/1347880091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4069c80 0x7f6ae406a100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c040 tx=0x7f6adc00d040 comp rx=0 tx=0).stop 2024-01-11T07:09:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.483+0000 7f6aec188700 1 -- 172.21.15.7:0/1347880091 shutdown_connections 2024-01-11T07:09:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.483+0000 7f6aec188700 1 --2- 172.21.15.7:0/1347880091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4069c80 0x7f6ae406a100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.483+0000 7f6aec188700 1 --2- 172.21.15.7:0/1347880091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae415b260 0x7f6ae40b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.483+0000 7f6aec188700 1 -- 172.21.15.7:0/1347880091 >> 172.21.15.7:0/1347880091 conn(0x7f6ae40cd910 msgr2=0x7f6ae40cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.483+0000 7f6aec188700 1 -- 172.21.15.7:0/1347880091 shutdown_connections 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.483+0000 7f6aec188700 1 -- 172.21.15.7:0/1347880091 wait complete. 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.484+0000 7f6aec188700 1 Processor -- start 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.484+0000 7f6aec188700 1 -- start start 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aec188700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae4069c80 0x7f6ae414c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aec188700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4156760 0x7f6ae4153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aec188700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae415b260 0x7f6ae40ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aec188700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4166460 con 0x7f6ae415b260 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aec188700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40caa80 con 0x7f6ae4156760 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aec188700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4166180 con 0x7f6ae4069c80 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aea725700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae415b260 0x7f6ae40ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aea725700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae415b260 0x7f6ae40ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47066/0 (socket says 172.21.15.7:47066) 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6aea725700 1 -- 172.21.15.7:0/2680767905 learned_addr learned my addr 172.21.15.7:0/2680767905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6ae9723700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4156760 0x7f6ae4153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.485+0000 7f6ae9f24700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae4069c80 0x7f6ae414c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6aea725700 1 -- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae4069c80 msgr2=0x7f6ae414c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6aea725700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae4069c80 0x7f6ae414c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6aea725700 1 -- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4156760 msgr2=0x7f6ae4153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6aea725700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4156760 0x7f6ae4153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6aea725700 1 -- 172.21.15.7:0/2680767905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc00bb00 con 0x7f6ae415b260 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6aea725700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae415b260 0x7f6ae40ccb10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c6e0 tx=0x7f6adc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6ad5ffb700 1 -- 172.21.15.7:0/2680767905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc01d620 con 0x7f6ae415b260 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6ad5ffb700 1 -- 172.21.15.7:0/2680767905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc01d780 con 0x7f6ae415b260 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6ad5ffb700 1 -- 172.21.15.7:0/2680767905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc01db20 con 0x7f6ae415b260 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae40ca430 con 0x7f6ae415b260 2024-01-11T07:09:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.486+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae414cb00 con 0x7f6ae415b260 2024-01-11T07:09:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.488+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae40ca290 con 0x7f6ae415b260 2024-01-11T07:09:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.488+0000 7f6ad5ffb700 1 -- 172.21.15.7:0/2680767905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6adc01a070 con 0x7f6ae415b260 2024-01-11T07:09:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.489+0000 7f6ad5ffb700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac8066f30 0x7f6ac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.489+0000 7f6ad5ffb700 1 -- 172.21.15.7:0/2680767905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6adc086080 con 0x7f6ae415b260 2024-01-11T07:09:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.489+0000 7f6ae9f24700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac8066f30 0x7f6ac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.489+0000 7f6ae9f24700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac8066f30 0x7f6ac80693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40b1e10 tx=0x7f6ae0007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:09.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.493+0000 7f6ad5ffb700 1 -- 172.21.15.7:0/2680767905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6adc0b75a0 con 0x7f6ae415b260 2024-01-11T07:09:09.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.696+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ae404c3c0 con 0x7f6ae415b260 2024-01-11T07:09:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.696+0000 7f6ad5ffb700 1 -- 172.21.15.7:0/2680767905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f6adc018090 con 0x7f6ae415b260 2024-01-11T07:09:09.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:09.697 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":63,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:09:08.716713+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:09:06.415283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-11T07:09:07.694952+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:09:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac8066f30 msgr2=0x7f6ac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac8066f30 0x7f6ac80693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40b1e10 tx=0x7f6ae0007d10 comp rx=0 tx=0).stop 2024-01-11T07:09:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae415b260 msgr2=0x7f6ae40ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae415b260 0x7f6ae40ccb10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c6e0 tx=0x7f6adc016000 comp rx=0 tx=0).stop 2024-01-11T07:09:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 shutdown_connections 2024-01-11T07:09:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac8066f30 0x7f6ac80693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae415b260 0x7f6ae40ccb10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae4069c80 0x7f6ae414c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 --2- 172.21.15.7:0/2680767905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae4156760 0x7f6ae4153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 >> 172.21.15.7:0/2680767905 conn(0x7f6ae40cd910 msgr2=0x7f6ae406b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 shutdown_connections 2024-01-11T07:09:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.699+0000 7f6aec188700 1 -- 172.21.15.7:0/2680767905 wait complete. 2024-01-11T07:09:09.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:09:09.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.845+0000 7f79ff029700 1 Processor -- start 2024-01-11T07:09:09.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.845+0000 7f79ff029700 1 -- start start 2024-01-11T07:09:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.846+0000 7f79ff029700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79f80ac3a0 0x7f79f80cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.846+0000 7f79ff029700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79f80ab8d0 0x7f79f815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.846+0000 7f79ff029700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f815e2d0 con 0x7f79f815d920 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.846+0000 7f79ff029700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f806ac30 con 0x7f79f80ac3a0 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.846+0000 7f79ff029700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f80ac770 con 0x7f79f80ab8d0 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.846+0000 7f79fcdc5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79f80ac3a0 0x7f79f80cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.846+0000 7f79fcdc5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79f80ac3a0 0x7f79f80cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47082/0 (socket says 172.21.15.7:47082) 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.846+0000 7f79fcdc5700 1 -- 172.21.15.7:0/2554804064 learned_addr learned my addr 172.21.15.7:0/2554804064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79fd5c6700 1 --2- 172.21.15.7:0/2554804064 >> v2:172.21.15.113:3300/0 conn(0x7f79f80ab8d0 0x7f79f815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79fcdc5700 1 -- 172.21.15.7:0/2554804064 >> v2:172.21.15.113:3300/0 conn(0x7f79f80ab8d0 msgr2=0x7f79f815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79fcdc5700 1 --2- 172.21.15.7:0/2554804064 >> v2:172.21.15.113:3300/0 conn(0x7f79f80ab8d0 0x7f79f815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79fcdc5700 1 -- 172.21.15.7:0/2554804064 >> v1:172.21.15.113:6789/0 conn(0x7f79f815d920 legacy=0x7f79f815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79fcdc5700 1 -- 172.21.15.7:0/2554804064 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f806a300 con 0x7f79f80ac3a0 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79fd5c6700 1 --2- 172.21.15.7:0/2554804064 >> v2:172.21.15.113:3300/0 conn(0x7f79f80ab8d0 0x7f79f815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79fcdc5700 1 --2- 172.21.15.7:0/2554804064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f80ac3a0 0x7f79f80cac60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f79f806fda0 tx=0x7f79ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc67c5b790312ae server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79f77fe700 1 -- 172.21.15.7:0/2554804064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00b860 con 0x7f79f80ac3a0 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79f77fe700 1 -- 172.21.15.7:0/2554804064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec00b9c0 con 0x7f79f80ac3a0 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.847+0000 7f79f77fe700 1 -- 172.21.15.7:0/2554804064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00bd60 con 0x7f79f80ac3a0 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.848+0000 7f79ff029700 1 -- 172.21.15.7:0/2554804064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f80ac3a0 msgr2=0x7f79f80cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.848+0000 7f79ff029700 1 --2- 172.21.15.7:0/2554804064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f80ac3a0 0x7f79f80cac60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f79f806fda0 tx=0x7f79ec009250 comp rx=0 tx=0).stop 2024-01-11T07:09:09.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.848+0000 7f79ff029700 1 -- 172.21.15.7:0/2554804064 shutdown_connections 2024-01-11T07:09:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.848+0000 7f79ff029700 1 --2- 172.21.15.7:0/2554804064 >> v2:172.21.15.113:3300/0 conn(0x7f79f80ab8d0 0x7f79f815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.848+0000 7f79ff029700 1 --2- 172.21.15.7:0/2554804064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f80ac3a0 0x7f79f80cac60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.848+0000 7f79ff029700 1 -- 172.21.15.7:0/2554804064 >> 172.21.15.7:0/2554804064 conn(0x7f79f80cd090 msgr2=0x7f79f80cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.848+0000 7f79ff029700 1 -- 172.21.15.7:0/2554804064 shutdown_connections 2024-01-11T07:09:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.848+0000 7f79ff029700 1 -- 172.21.15.7:0/2554804064 wait complete. 2024-01-11T07:09:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.849+0000 7f79ff029700 1 Processor -- start 2024-01-11T07:09:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.849+0000 7f79ff029700 1 -- start start 2024-01-11T07:09:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79ff029700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80ab8d0 0x7f79f813a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79ff029700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80ac3a0 0x7f79f80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79ff029700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f815d920 0x7f79f8069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79ff029700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f80b0630 con 0x7f79f815d920 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79ff029700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f813a6f0 con 0x7f79f80ac3a0 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79ff029700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f813fe80 con 0x7f79f80ab8d0 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79fd5c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f815d920 0x7f79f8069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79fd5c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f815d920 0x7f79f8069610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47096/0 (socket says 172.21.15.7:47096) 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.850+0000 7f79fd5c6700 1 -- 172.21.15.7:0/789945571 learned_addr learned my addr 172.21.15.7:0/789945571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79fcdc5700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80ab8d0 0x7f79f813a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79f7fff700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80ac3a0 0x7f79f80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79fd5c6700 1 -- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80ab8d0 msgr2=0x7f79f813a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79fd5c6700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80ab8d0 0x7f79f813a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79fd5c6700 1 -- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80ac3a0 msgr2=0x7f79f80b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79fd5c6700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80ac3a0 0x7f79f80b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79fd5c6700 1 -- 172.21.15.7:0/789945571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0094f0 con 0x7f79f815d920 2024-01-11T07:09:09.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79fd5c6700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f815d920 0x7f79f8069610 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f79f000d150 tx=0x7f79f0009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:09.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79f4ff9700 1 -- 172.21.15.7:0/789945571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f000dc90 con 0x7f79f815d920 2024-01-11T07:09:09.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79f4ff9700 1 -- 172.21.15.7:0/789945571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f000f040 con 0x7f79f815d920 2024-01-11T07:09:09.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f8140160 con 0x7f79f815d920 2024-01-11T07:09:09.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79f4ff9700 1 -- 172.21.15.7:0/789945571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0015690 con 0x7f79f815d920 2024-01-11T07:09:09.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.851+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f80b02c0 con 0x7f79f815d920 2024-01-11T07:09:09.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.852+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f804c3c0 con 0x7f79f815d920 2024-01-11T07:09:09.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.853+0000 7f79f4ff9700 1 -- 172.21.15.7:0/789945571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f79f001e3b0 con 0x7f79f815d920 2024-01-11T07:09:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.853+0000 7f79f4ff9700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f79dc066f30 0x7f79dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.853+0000 7f79f4ff9700 1 -- 172.21.15.7:0/789945571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f79f0084a10 con 0x7f79f815d920 2024-01-11T07:09:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.853+0000 7f79fcdc5700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f79dc066f30 0x7f79dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.854+0000 7f79fcdc5700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f79dc066f30 0x7f79dc0693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f79f806fda0 tx=0x7f79ec017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:09.858+0000 7f79f4ff9700 1 -- 172.21.15.7:0/789945571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79f0051c80 con 0x7f79f815d920 2024-01-11T07:09:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.062+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79f80b0770 con 0x7f79f815d920 2024-01-11T07:09:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.063+0000 7f79f4ff9700 1 -- 172.21.15.7:0/789945571 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+15315 (secure 0 0 0) 0x7f79f001a090 con 0x7f79f815d920 2024-01-11T07:09:10.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:10.063 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":63,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:09:08.716713+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:09:06.415283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-11T07:09:07.694952+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f79dc066f30 msgr2=0x7f79dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f79dc066f30 0x7f79dc0693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f79f806fda0 tx=0x7f79ec017000 comp rx=0 tx=0).stop 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f815d920 msgr2=0x7f79f8069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f815d920 0x7f79f8069610 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f79f000d150 tx=0x7f79f0009a50 comp rx=0 tx=0).stop 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 shutdown_connections 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f79dc066f30 0x7f79dc0693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79f815d920 0x7f79f8069610 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f79f80ab8d0 0x7f79f813a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 --2- 172.21.15.7:0/789945571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79f80ac3a0 0x7f79f80b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 >> 172.21.15.7:0/789945571 conn(0x7f79f80cd090 msgr2=0x7f79f80c3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 shutdown_connections 2024-01-11T07:09:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.066+0000 7f79ff029700 1 -- 172.21.15.7:0/789945571 wait complete. 2024-01-11T07:09:10.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:09:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.213+0000 7fd525a48700 1 Processor -- start 2024-01-11T07:09:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.213+0000 7fd525a48700 1 -- start start 2024-01-11T07:09:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.214+0000 7fd525a48700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd52015d4a0 0x7fd5200b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.214+0000 7fd525a48700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd52015d870 0x7fd5200cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.214+0000 7fd525a48700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5200cca30 0x7fd520068570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.214+0000 7fd525a48700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5200cced0 con 0x7fd52015d4a0 2024-01-11T07:09:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.214+0000 7fd525a48700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5200680e0 con 0x7fd52015d870 2024-01-11T07:09:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.214+0000 7fd525a48700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200677b0 con 0x7fd5200cca30 2024-01-11T07:09:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.214+0000 7fd51effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd52015d4a0 0x7fd5200b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd52015d4a0 0x7fd5200b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47098/0 (socket says 172.21.15.7:47098) 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51effd700 1 -- 172.21.15.7:0/1796054125 learned_addr learned my addr 172.21.15.7:0/1796054125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd5167fc700 1 --2- 172.21.15.7:0/1796054125 >> v2:172.21.15.113:3300/0 conn(0x7fd52015d870 0x7fd5200cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51f7fe700 1 --2- 172.21.15.7:0/1796054125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5200cca30 0x7fd520068570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51effd700 1 -- 172.21.15.7:0/1796054125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5200cca30 msgr2=0x7fd520068570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51effd700 1 --2- 172.21.15.7:0/1796054125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5200cca30 0x7fd520068570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51effd700 1 -- 172.21.15.7:0/1796054125 >> v2:172.21.15.113:3300/0 conn(0x7fd52015d870 msgr2=0x7fd5200cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51effd700 1 --2- 172.21.15.7:0/1796054125 >> v2:172.21.15.113:3300/0 conn(0x7fd52015d870 0x7fd5200cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51effd700 1 -- 172.21.15.7:0/1796054125 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520067a90 con 0x7fd52015d4a0 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51effd700 1 --2- 172.21.15.7:0/1796054125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52015d4a0 0x7fd5200b1790 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd510009990 tx=0x7fd510009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c58d2c465894218 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51e7fc700 1 -- 172.21.15.7:0/1796054125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510005c10 con 0x7fd52015d4a0 2024-01-11T07:09:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51e7fc700 1 -- 172.21.15.7:0/1796054125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510005d70 con 0x7fd52015d4a0 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.215+0000 7fd51e7fc700 1 -- 172.21.15.7:0/1796054125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510006260 con 0x7fd52015d4a0 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 -- 172.21.15.7:0/1796054125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52015d4a0 msgr2=0x7fd5200b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 --2- 172.21.15.7:0/1796054125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52015d4a0 0x7fd5200b1790 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd510009990 tx=0x7fd510009220 comp rx=0 tx=0).stop 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 -- 172.21.15.7:0/1796054125 shutdown_connections 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 --2- 172.21.15.7:0/1796054125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52015d4a0 0x7fd5200b1790 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 --2- 172.21.15.7:0/1796054125 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd5200cca30 0x7fd520068570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 --2- 172.21.15.7:0/1796054125 >> v2:172.21.15.113:3300/0 conn(0x7fd52015d870 0x7fd5200cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 -- 172.21.15.7:0/1796054125 >> 172.21.15.7:0/1796054125 conn(0x7fd520097a50 msgr2=0x7fd5200ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 -- 172.21.15.7:0/1796054125 shutdown_connections 2024-01-11T07:09:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.216+0000 7fd525a48700 1 -- 172.21.15.7:0/1796054125 wait complete. 2024-01-11T07:09:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.218+0000 7fd525a48700 1 Processor -- start 2024-01-11T07:09:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.218+0000 7fd525a48700 1 -- start start 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd525a48700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5200cca30 0x7fd5200c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd525a48700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd52015d4a0 0x7fd5200c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd525a48700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd520152f10 0x7fd5201594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd525a48700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200c8100 con 0x7fd5200cca30 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd525a48700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200b2010 con 0x7fd52015d4a0 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd525a48700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200c47f0 con 0x7fd520152f10 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5200cca30 0x7fd5200c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5200cca30 0x7fd5200c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47100/0 (socket says 172.21.15.7:47100) 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51effd700 1 -- 172.21.15.7:0/2062098699 learned_addr learned my addr 172.21.15.7:0/2062098699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd5167fc700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd52015d4a0 0x7fd5200c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51f7fe700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd520152f10 0x7fd5201594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51effd700 1 -- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd520152f10 msgr2=0x7fd5201594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51effd700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd520152f10 0x7fd5201594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51effd700 1 -- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd52015d4a0 msgr2=0x7fd5200c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51effd700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd52015d4a0 0x7fd5200c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.219+0000 7fd51effd700 1 -- 172.21.15.7:0/2062098699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510009480 con 0x7fd5200cca30 2024-01-11T07:09:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.220+0000 7fd51effd700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5200cca30 0x7fd5200c3970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd510009f80 tx=0x7fd510016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.220+0000 7fd5177fe700 1 -- 172.21.15.7:0/2062098699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100116c0 con 0x7fd5200cca30 2024-01-11T07:09:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.220+0000 7fd5177fe700 1 -- 172.21.15.7:0/2062098699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510020070 con 0x7fd5200cca30 2024-01-11T07:09:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.220+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5200b22d0 con 0x7fd5200cca30 2024-01-11T07:09:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.220+0000 7fd5177fe700 1 -- 172.21.15.7:0/2062098699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510011a90 con 0x7fd5200cca30 2024-01-11T07:09:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.220+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5200c4230 con 0x7fd5200cca30 2024-01-11T07:09:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.221+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52004c3c0 con 0x7fd5200cca30 2024-01-11T07:09:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.221+0000 7fd5177fe700 1 -- 172.21.15.7:0/2062098699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd510022030 con 0x7fd5200cca30 2024-01-11T07:09:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.222+0000 7fd5177fe700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd500066f30 0x7fd5000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.222+0000 7fd5177fe700 1 -- 172.21.15.7:0/2062098699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd510085f30 con 0x7fd5200cca30 2024-01-11T07:09:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.222+0000 7fd5167fc700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd500066f30 0x7fd5000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.223+0000 7fd5167fc700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd500066f30 0x7fd5000693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5200c8500 tx=0x7fd5080091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.226+0000 7fd5177fe700 1 -- 172.21.15.7:0/2062098699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5100531a0 con 0x7fd5200cca30 2024-01-11T07:09:10.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:10.481+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd520067d70 con 0x7fd5200cca30 2024-01-11T07:09:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.325+0000 7fd5177fe700 1 -- 172.21.15.7:0/2062098699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd5100567c0 con 0x7fd5200cca30 2024-01-11T07:09:11.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd500066f30 msgr2=0x7fd5000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd500066f30 0x7fd5000693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd5200c8500 tx=0x7fd5080091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5200cca30 msgr2=0x7fd5200c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5200cca30 0x7fd5200c3970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd510009f80 tx=0x7fd510016000 comp rx=0 tx=0).stop 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 shutdown_connections 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd500066f30 0x7fd5000693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd520152f10 0x7fd5201594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5200cca30 0x7fd5200c3970 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 --2- 172.21.15.7:0/2062098699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd52015d4a0 0x7fd5200c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 >> 172.21.15.7:0/2062098699 conn(0x7fd520097a50 msgr2=0x7fd52006a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 shutdown_connections 2024-01-11T07:09:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.328+0000 7fd525a48700 1 -- 172.21.15.7:0/2062098699 wait complete. 2024-01-11T07:09:11.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:09:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.483+0000 7f55fee32700 1 Processor -- start 2024-01-11T07:09:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.483+0000 7f55fee32700 1 -- start start 2024-01-11T07:09:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.484+0000 7f55fee32700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.484+0000 7f55fee32700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55f80ace30 0x7f55f81571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.484+0000 7f55fee32700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f806e590 con 0x7f55f806e120 2024-01-11T07:09:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.484+0000 7f55fee32700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f806ab50 con 0x7f55f80ace30 2024-01-11T07:09:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.484+0000 7f55fee32700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f806a6c0 con 0x7f55f815b1a0 2024-01-11T07:09:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fcbce700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fd3cf700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55f80ace30 0x7f55f81571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fcbce700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:60146/0 (socket says 172.21.15.7:60146) 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fcbce700 1 -- 172.21.15.7:0/2614226159 learned_addr learned my addr 172.21.15.7:0/2614226159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55f7fff700 1 --1- 172.21.15.7:0/2614226159 >> v1:172.21.15.113:6789/0 conn(0x7f55f806e120 0x7f55f80ac720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:56768/0 (socket says 172.21.15.7:56768) 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55f77fe700 1 -- 172.21.15.7:0/2614226159 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837818170 0 0) 0x7f55f806e590 con 0x7f55f806e120 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55f77fe700 1 -- 172.21.15.7:0/2614226159 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55e8003570 con 0x7f55f806e120 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fcbce700 1 -- 172.21.15.7:0/2614226159 >> v2:172.21.15.113:3300/0 conn(0x7f55f80ace30 msgr2=0x7f55f81571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fcbce700 1 --2- 172.21.15.7:0/2614226159 >> v2:172.21.15.113:3300/0 conn(0x7f55f80ace30 0x7f55f81571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fcbce700 1 -- 172.21.15.7:0/2614226159 >> v1:172.21.15.113:6789/0 conn(0x7f55f806e120 legacy=0x7f55f80ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fcbce700 1 -- 172.21.15.7:0/2614226159 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f815a7d0 con 0x7f55f815b1a0 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fd3cf700 1 --2- 172.21.15.7:0/2614226159 >> v2:172.21.15.113:3300/0 conn(0x7f55f80ace30 0x7f55f81571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55fcbce700 1 --2- 172.21.15.7:0/2614226159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f806dbe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00bc50 tx=0x7f55ec00b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1363a2e697295fab server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55f77fe700 1 -- 172.21.15.7:0/2614226159 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00db40 con 0x7f55f815b1a0 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55f77fe700 1 -- 172.21.15.7:0/2614226159 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec00dca0 con 0x7f55f815b1a0 2024-01-11T07:09:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.485+0000 7f55f77fe700 1 -- 172.21.15.7:0/2614226159 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec019690 con 0x7f55f815b1a0 2024-01-11T07:09:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.486+0000 7f55fee32700 1 -- 172.21.15.7:0/2614226159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 msgr2=0x7f55f806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.486+0000 7f55fee32700 1 --2- 172.21.15.7:0/2614226159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f806dbe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00bc50 tx=0x7f55ec00b4f0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.486+0000 7f55fee32700 1 -- 172.21.15.7:0/2614226159 shutdown_connections 2024-01-11T07:09:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.486+0000 7f55fee32700 1 --2- 172.21.15.7:0/2614226159 >> v2:172.21.15.113:3300/0 conn(0x7f55f80ace30 0x7f55f81571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.486+0000 7f55fee32700 1 --2- 172.21.15.7:0/2614226159 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f806dbe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.486+0000 7f55fee32700 1 -- 172.21.15.7:0/2614226159 >> 172.21.15.7:0/2614226159 conn(0x7f55f80cd8f0 msgr2=0x7f55f80cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.486+0000 7f55fee32700 1 -- 172.21.15.7:0/2614226159 shutdown_connections 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.486+0000 7f55fee32700 1 -- 172.21.15.7:0/2614226159 wait complete. 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.487+0000 7f55fee32700 1 Processor -- start 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.488+0000 7f55fee32700 1 -- start start 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.488+0000 7f55fee32700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55f806e120 0x7f55f80b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.488+0000 7f55fee32700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55f80ace30 0x7f55f80c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.488+0000 7f55fee32700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f80c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.488+0000 7f55fee32700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80b32e0 con 0x7f55f806e120 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.488+0000 7f55fee32700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80b29f0 con 0x7f55f80ace30 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.488+0000 7f55fee32700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80b2c90 con 0x7f55f815b1a0 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55f806e120 0x7f55f80b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55f806e120 0x7f55f80b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47104/0 (socket says 172.21.15.7:47104) 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 -- 172.21.15.7:0/155035457 learned_addr learned my addr 172.21.15.7:0/155035457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55f7fff700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55f80ace30 0x7f55f80c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fd3cf700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f80c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 -- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 msgr2=0x7f55f80c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f80c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 -- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55f80ace30 msgr2=0x7f55f80c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55f80ace30 0x7f55f80c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 -- 172.21.15.7:0/155035457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0009a20 con 0x7f55f806e120 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fcbce700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55f806e120 0x7f55f80b0dc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b530 tx=0x7f55ec01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55f4ff9700 1 -- 172.21.15.7:0/155035457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec010680 con 0x7f55f806e120 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55f4ff9700 1 -- 172.21.15.7:0/155035457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec01e040 con 0x7f55f806e120 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ec00b790 con 0x7f55f806e120 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55f4ff9700 1 -- 172.21.15.7:0/155035457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec010a90 con 0x7f55f806e120 2024-01-11T07:09:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.489+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f816aa80 con 0x7f55f806e120 2024-01-11T07:09:11.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.490+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f8064a40 con 0x7f55f806e120 2024-01-11T07:09:11.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.491+0000 7f55f4ff9700 1 -- 172.21.15.7:0/155035457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f55ec02c430 con 0x7f55f806e120 2024-01-11T07:09:11.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.491+0000 7f55f4ff9700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55dc066f30 0x7f55dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:11.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.491+0000 7f55f4ff9700 1 -- 172.21.15.7:0/155035457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55ec014070 con 0x7f55f806e120 2024-01-11T07:09:11.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.491+0000 7f55f7fff700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55dc066f30 0x7f55dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:11.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.492+0000 7f55f7fff700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55dc066f30 0x7f55dc0693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55f816d0b0 tx=0x7f55e4000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:11.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.496+0000 7f55f4ff9700 1 -- 172.21.15.7:0/155035457 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55ec0539b0 con 0x7f55f806e120 2024-01-11T07:09:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:11.745+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f55f804c3c0 con 0x7f55f806e120 2024-01-11T07:09:12.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.329+0000 7f55f4ff9700 1 -- 172.21.15.7:0/155035457 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f55ec056fd0 con 0x7f55f806e120 2024-01-11T07:09:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55dc066f30 msgr2=0x7f55dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55dc066f30 0x7f55dc0693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55f816d0b0 tx=0x7f55e4000bc0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55f806e120 msgr2=0x7f55f80b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55f806e120 0x7f55f80b0dc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b530 tx=0x7f55ec01b000 comp rx=0 tx=0).stop 2024-01-11T07:09:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 shutdown_connections 2024-01-11T07:09:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55dc066f30 0x7f55dc0693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55f815b1a0 0x7f55f80c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55f806e120 0x7f55f80b0dc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 --2- 172.21.15.7:0/155035457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55f80ace30 0x7f55f80c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.332+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 >> 172.21.15.7:0/155035457 conn(0x7f55f80cd8f0 msgr2=0x7f55f81543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.333+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 shutdown_connections 2024-01-11T07:09:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.333+0000 7f55fee32700 1 -- 172.21.15.7:0/155035457 wait complete. 2024-01-11T07:09:12.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:09:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.484+0000 7f5e374c9700 1 Processor -- start 2024-01-11T07:09:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.484+0000 7f5e374c9700 1 -- start start 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e374c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e30063230 0x7f5e30063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e374c9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 0x7f5e300aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e374c9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3015d880 con 0x7f5e300ac5b0 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e374c9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3006c5b0 con 0x7f5e30063230 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e374c9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30063b40 con 0x7f5e3015a280 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e35265700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e30063230 0x7f5e30063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e35a66700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5e300ac5b0 0x7f5e300aca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60474/0 (socket says 172.21.15.7:60474) 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e35a66700 1 -- 172.21.15.7:0/1907914911 learned_addr learned my addr 172.21.15.7:0/1907914911 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e34a64700 1 --2- 172.21.15.7:0/1907914911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 0x7f5e300aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e35265700 1 -- 172.21.15.7:0/1907914911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 msgr2=0x7f5e300aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e35265700 1 --2- 172.21.15.7:0/1907914911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 0x7f5e300aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e35265700 1 -- 172.21.15.7:0/1907914911 >> v1:172.21.15.7:6789/0 conn(0x7f5e300ac5b0 legacy=0x7f5e300aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e35265700 1 -- 172.21.15.7:0/1907914911 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3015a7f0 con 0x7f5e30063230 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.485+0000 7f5e27fff700 1 -- 172.21.15.7:0/1907914911 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070581429 0 0) 0x7f5e3015d880 con 0x7f5e300ac5b0 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e35265700 1 --2- 172.21.15.7:0/1907914911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e30063600 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009910 tx=0x7f5e2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8772360798abcb25 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e27fff700 1 -- 172.21.15.7:0/1907914911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00b640 con 0x7f5e30063230 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e27fff700 1 -- 172.21.15.7:0/1907914911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c00b7a0 con 0x7f5e30063230 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e27fff700 1 -- 172.21.15.7:0/1907914911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00bb60 con 0x7f5e30063230 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e374c9700 1 -- 172.21.15.7:0/1907914911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 msgr2=0x7f5e30063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1907914911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e30063600 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009910 tx=0x7f5e2c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e374c9700 1 -- 172.21.15.7:0/1907914911 shutdown_connections 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1907914911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e30063600 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1907914911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 0x7f5e300aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.486+0000 7f5e374c9700 1 -- 172.21.15.7:0/1907914911 >> 172.21.15.7:0/1907914911 conn(0x7f5e300680e0 msgr2=0x7f5e300cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:12.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.487+0000 7f5e374c9700 1 -- 172.21.15.7:0/1907914911 shutdown_connections 2024-01-11T07:09:12.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.487+0000 7f5e374c9700 1 -- 172.21.15.7:0/1907914911 wait complete. 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.488+0000 7f5e374c9700 1 Processor -- start 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.488+0000 7f5e374c9700 1 -- start start 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e374c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e3016b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e374c9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e300ac5b0 0x7f5e301536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e374c9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 0x7f5e300b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e374c9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300ad1f0 con 0x7f5e30063230 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e374c9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300add70 con 0x7f5e300ac5b0 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e374c9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300ad440 con 0x7f5e3015a280 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e3016b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e3016b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47116/0 (socket says 172.21.15.7:47116) 2024-01-11T07:09:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 -- 172.21.15.7:0/1914300732 learned_addr learned my addr 172.21.15.7:0/1914300732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e34a64700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e300ac5b0 0x7f5e301536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35a66700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 0x7f5e300b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 -- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 msgr2=0x7f5e300b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 0x7f5e300b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 -- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e300ac5b0 msgr2=0x7f5e301536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e300ac5b0 0x7f5e301536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 -- 172.21.15.7:0/1914300732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009420 con 0x7f5e30063230 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.489+0000 7f5e35265700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e3016b350 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009fb0 tx=0x7f5e2c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.490+0000 7f5e257fa700 1 -- 172.21.15.7:0/1914300732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c01f070 con 0x7f5e30063230 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.490+0000 7f5e257fa700 1 -- 172.21.15.7:0/1914300732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c000bc0 con 0x7f5e30063230 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.490+0000 7f5e257fa700 1 -- 172.21.15.7:0/1914300732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00bb60 con 0x7f5e30063230 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.490+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e300ad720 con 0x7f5e30063230 2024-01-11T07:09:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.490+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e30153bf0 con 0x7f5e30063230 2024-01-11T07:09:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.491+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3004c3c0 con 0x7f5e30063230 2024-01-11T07:09:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.491+0000 7f5e257fa700 1 -- 172.21.15.7:0/1914300732 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5e2c023070 con 0x7f5e30063230 2024-01-11T07:09:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.491+0000 7f5e257fa700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e14066f60 0x7f5e14069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.491+0000 7f5e257fa700 1 -- 172.21.15.7:0/1914300732 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e2c0857e0 con 0x7f5e30063230 2024-01-11T07:09:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.492+0000 7f5e34a64700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e14066f60 0x7f5e14069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.492+0000 7f5e34a64700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e14066f60 0x7f5e14069410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e3015f560 tx=0x7f5e200091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:12.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.496+0000 7f5e257fa700 1 -- 172.21.15.7:0/1914300732 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e2c052a90 con 0x7f5e30063230 2024-01-11T07:09:12.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.744+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e3015aad0 con 0x7f5e30063230 2024-01-11T07:09:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.745+0000 7f5e257fa700 1 -- 172.21.15.7:0/1914300732 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f5e2c02a090 con 0x7f5e30063230 2024-01-11T07:09:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:12.744 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1734166534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1734166534},{"type":"v1","addr":"172.21.15.113:6836","nonce":1734166534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3195231529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3195231529},{"type":"v1","addr":"172.21.15.113:6837","nonce":3195231529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/338342475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":338342475},{"type":"v1","addr":"172.21.15.7:6834","nonce":338342475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:09:08.717048+0000","modified":"2024-01-11T07:09:12.324714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1812078530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1812078530},{"type":"v1","addr":"172.21.15.7:6835","nonce":1812078530}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e14066f60 msgr2=0x7f5e14069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e14066f60 0x7f5e14069410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5e3015f560 tx=0x7f5e200091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 msgr2=0x7f5e3016b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e3016b350 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009fb0 tx=0x7f5e2c017000 comp rx=0 tx=0).stop 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 shutdown_connections 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e14066f60 0x7f5e14069410 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e3015a280 0x7f5e300b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e30063230 0x7f5e3016b350 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 --2- 172.21.15.7:0/1914300732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e300ac5b0 0x7f5e301536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 >> 172.21.15.7:0/1914300732 conn(0x7f5e300680e0 msgr2=0x7f5e300cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 shutdown_connections 2024-01-11T07:09:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.747+0000 7f5e374c9700 1 -- 172.21.15.7:0/1914300732 wait complete. 2024-01-11T07:09:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-11T07:09:12.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:09:12.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.893+0000 7fd3c3c82700 1 Processor -- start 2024-01-11T07:09:12.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.893+0000 7fd3c3c82700 1 -- start start 2024-01-11T07:09:12.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c3c82700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc157d80 0x7fd3bc063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c3c82700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3bc063770 0x7fd3bc069c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c3c82700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3bc15c660 con 0x7fd3bc1579b0 2024-01-11T07:09:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c3c82700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3bc063bb0 con 0x7fd3bc063770 2024-01-11T07:09:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c3c82700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0aeab0 con 0x7fd3bc157d80 2024-01-11T07:09:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c221f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3bc063770 0x7fd3bc069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c121d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc157d80 0x7fd3bc063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c121d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc157d80 0x7fd3bc063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:60192/0 (socket says 172.21.15.7:60192) 2024-01-11T07:09:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.894+0000 7fd3c121d700 1 -- 172.21.15.7:0/2623004062 learned_addr learned my addr 172.21.15.7:0/2623004062 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c121d700 1 -- 172.21.15.7:0/2623004062 >> v2:172.21.15.113:3300/0 conn(0x7fd3bc063770 msgr2=0x7fd3bc069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c121d700 1 --2- 172.21.15.7:0/2623004062 >> v2:172.21.15.113:3300/0 conn(0x7fd3bc063770 0x7fd3bc069c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c121d700 1 -- 172.21.15.7:0/2623004062 >> v1:172.21.15.113:6789/0 conn(0x7fd3bc1579b0 legacy=0x7fd3bc15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c121d700 1 -- 172.21.15.7:0/2623004062 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc0aed90 con 0x7fd3bc157d80 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c0a1c700 1 -- 172.21.15.7:0/2623004062 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187329433 0 0) 0x7fd3bc15c660 con 0x7fd3bc1579b0 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c121d700 1 --2- 172.21.15.7:0/2623004062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc157d80 0x7fd3bc063230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0099b0 tx=0x7fd3ac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a2b72832c30e56 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c0a1c700 1 -- 172.21.15.7:0/2623004062 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac009ea0 con 0x7fd3bc157d80 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c0a1c700 1 -- 172.21.15.7:0/2623004062 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac00b860 con 0x7fd3bc157d80 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.895+0000 7fd3c0a1c700 1 -- 172.21.15.7:0/2623004062 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00bc00 con 0x7fd3bc157d80 2024-01-11T07:09:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.896+0000 7fd3c3c82700 1 -- 172.21.15.7:0/2623004062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc157d80 msgr2=0x7fd3bc063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.896+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/2623004062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc157d80 0x7fd3bc063230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0099b0 tx=0x7fd3ac009250 comp rx=0 tx=0).stop 2024-01-11T07:09:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.896+0000 7fd3c3c82700 1 -- 172.21.15.7:0/2623004062 shutdown_connections 2024-01-11T07:09:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.896+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/2623004062 >> v2:172.21.15.113:3300/0 conn(0x7fd3bc063770 0x7fd3bc069c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.896+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/2623004062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc157d80 0x7fd3bc063230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.896+0000 7fd3c3c82700 1 -- 172.21.15.7:0/2623004062 >> 172.21.15.7:0/2623004062 conn(0x7fd3bc0680e0 msgr2=0x7fd3bc0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.896+0000 7fd3c3c82700 1 -- 172.21.15.7:0/2623004062 shutdown_connections 2024-01-11T07:09:12.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.896+0000 7fd3c3c82700 1 -- 172.21.15.7:0/2623004062 wait complete. 2024-01-11T07:09:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.897+0000 7fd3c3c82700 1 Processor -- start 2024-01-11T07:09:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.897+0000 7fd3c3c82700 1 -- start start 2024-01-11T07:09:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c3c82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3bc063770 0x7fd3bc0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c3c82700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc1579b0 0x7fd3bc167620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c3c82700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3bc157d80 0x7fd3bc0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c3c82700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0c69b0 con 0x7fd3bc063770 2024-01-11T07:09:12.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c3c82700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0c74c0 con 0x7fd3bc157d80 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c3c82700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc168670 con 0x7fd3bc1579b0 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c1a1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3bc063770 0x7fd3bc0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c1a1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3bc063770 0x7fd3bc0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47124/0 (socket says 172.21.15.7:47124) 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c1a1e700 1 -- 172.21.15.7:0/256785245 learned_addr learned my addr 172.21.15.7:0/256785245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c221f700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3bc157d80 0x7fd3bc0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c121d700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc1579b0 0x7fd3bc167620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c1a1e700 1 -- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc1579b0 msgr2=0x7fd3bc167620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c1a1e700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc1579b0 0x7fd3bc167620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c1a1e700 1 -- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3bc157d80 msgr2=0x7fd3bc0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c1a1e700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3bc157d80 0x7fd3bc0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.898+0000 7fd3c1a1e700 1 -- 172.21.15.7:0/256785245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac0094f0 con 0x7fd3bc063770 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.899+0000 7fd3c1a1e700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3bc063770 0x7fd3bc0c6470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800c7e0 tx=0x7fd3b8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.899+0000 7fd3b1ffb700 1 -- 172.21.15.7:0/256785245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80105c0 con 0x7fd3bc063770 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.899+0000 7fd3b1ffb700 1 -- 172.21.15.7:0/256785245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b8010740 con 0x7fd3bc063770 2024-01-11T07:09:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.899+0000 7fd3b1ffb700 1 -- 172.21.15.7:0/256785245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8010b00 con 0x7fd3bc063770 2024-01-11T07:09:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.899+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3bc0c56c0 con 0x7fd3bc063770 2024-01-11T07:09:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.899+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3bc168b00 con 0x7fd3bc063770 2024-01-11T07:09:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.900+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3bc064280 con 0x7fd3bc063770 2024-01-11T07:09:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.900+0000 7fd3b1ffb700 1 -- 172.21.15.7:0/256785245 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd3b801b3b0 con 0x7fd3bc063770 2024-01-11T07:09:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.901+0000 7fd3b1ffb700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3a0066f30 0x7fd3a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.901+0000 7fd3b1ffb700 1 -- 172.21.15.7:0/256785245 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd3b8083ae0 con 0x7fd3bc063770 2024-01-11T07:09:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.901+0000 7fd3c121d700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3a0066f30 0x7fd3a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.902+0000 7fd3c121d700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3a0066f30 0x7fd3a00693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0099b0 tx=0x7fd3ac017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:12.905+0000 7fd3b1ffb700 1 -- 172.21.15.7:0/256785245 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3b8050d50 con 0x7fd3bc063770 2024-01-11T07:09:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.112+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3bc04c3c0 con 0x7fd3bc063770 2024-01-11T07:09:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.113+0000 7fd3b1ffb700 1 -- 172.21.15.7:0/256785245 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+15302 (secure 0 0 0) 0x7fd3b8017070 con 0x7fd3bc063770 2024-01-11T07:09:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:13.112 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":64,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:09:10.303732+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:09:06.415283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-11T07:09:07.694952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3a0066f30 msgr2=0x7fd3a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3a0066f30 0x7fd3a00693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0099b0 tx=0x7fd3ac017000 comp rx=0 tx=0).stop 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3bc063770 msgr2=0x7fd3bc0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3bc063770 0x7fd3bc0c6470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800c7e0 tx=0x7fd3b8007df0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 shutdown_connections 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd3a0066f30 0x7fd3a00693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3bc157d80 0x7fd3bc0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3bc063770 0x7fd3bc0c6470 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 --2- 172.21.15.7:0/256785245 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3bc1579b0 0x7fd3bc167620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 >> 172.21.15.7:0/256785245 conn(0x7fd3bc0680e0 msgr2=0x7fd3bc157000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 shutdown_connections 2024-01-11T07:09:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.115+0000 7fd3c3c82700 1 -- 172.21.15.7:0/256785245 wait complete. 2024-01-11T07:09:13.128 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:09:13.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:09:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.260+0000 7ffa3964b700 1 Processor -- start 2024-01-11T07:09:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.260+0000 7ffa3964b700 1 -- start start 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.261+0000 7ffa3964b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa34159ca0 0x7ffa3406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.261+0000 7ffa3964b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa340cef00 0x7ffa3406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.261+0000 7ffa3964b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa340d18c0 con 0x7ffa3415a070 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.261+0000 7ffa3964b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa340d0010 con 0x7ffa340cef00 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.261+0000 7ffa3964b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340cf400 con 0x7ffa34159ca0 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.261+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa34159ca0 0x7ffa3406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa34159ca0 0x7ffa3406dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:60208/0 (socket says 172.21.15.7:60208) 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa2bfff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffa3415a070 0x7ffa340ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:56794/0 (socket says 172.21.15.7:56794) 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa32ffd700 1 -- 172.21.15.7:0/2729002997 learned_addr learned my addr 172.21.15.7:0/2729002997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa337fe700 1 --2- 172.21.15.7:0/2729002997 >> v2:172.21.15.113:3300/0 conn(0x7ffa340cef00 0x7ffa3406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa32ffd700 1 -- 172.21.15.7:0/2729002997 >> v2:172.21.15.113:3300/0 conn(0x7ffa340cef00 msgr2=0x7ffa3406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/2729002997 >> v2:172.21.15.113:3300/0 conn(0x7ffa340cef00 0x7ffa3406e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa32ffd700 1 -- 172.21.15.7:0/2729002997 >> v1:172.21.15.113:6789/0 conn(0x7ffa3415a070 legacy=0x7ffa340ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa32ffd700 1 -- 172.21.15.7:0/2729002997 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa340cf6e0 con 0x7ffa34159ca0 2024-01-11T07:09:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa327fc700 1 -- 172.21.15.7:0/2729002997 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531833173 0 0) 0x7ffa340d18c0 con 0x7ffa3415a070 2024-01-11T07:09:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/2729002997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa34159ca0 0x7ffa3406dc30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa240099b0 tx=0x7ffa24009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4221051903288ea8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa327fc700 1 -- 172.21.15.7:0/2729002997 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24009ea0 con 0x7ffa34159ca0 2024-01-11T07:09:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa327fc700 1 -- 172.21.15.7:0/2729002997 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2400b860 con 0x7ffa34159ca0 2024-01-11T07:09:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.262+0000 7ffa327fc700 1 -- 172.21.15.7:0/2729002997 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400bc00 con 0x7ffa34159ca0 2024-01-11T07:09:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.263+0000 7ffa3964b700 1 -- 172.21.15.7:0/2729002997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa34159ca0 msgr2=0x7ffa3406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.263+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2729002997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa34159ca0 0x7ffa3406dc30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa240099b0 tx=0x7ffa24009250 comp rx=0 tx=0).stop 2024-01-11T07:09:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.263+0000 7ffa3964b700 1 -- 172.21.15.7:0/2729002997 shutdown_connections 2024-01-11T07:09:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.263+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2729002997 >> v2:172.21.15.113:3300/0 conn(0x7ffa340cef00 0x7ffa3406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.263+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2729002997 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa34159ca0 0x7ffa3406dc30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.263+0000 7ffa3964b700 1 -- 172.21.15.7:0/2729002997 >> 172.21.15.7:0/2729002997 conn(0x7ffa34063cc0 msgr2=0x7ffa340d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.263+0000 7ffa3964b700 1 -- 172.21.15.7:0/2729002997 shutdown_connections 2024-01-11T07:09:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.263+0000 7ffa3964b700 1 -- 172.21.15.7:0/2729002997 wait complete. 2024-01-11T07:09:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.264+0000 7ffa3964b700 1 Processor -- start 2024-01-11T07:09:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.264+0000 7ffa3964b700 1 -- start start 2024-01-11T07:09:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa3964b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cef00 0x7ffa340b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa3964b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa34159ca0 0x7ffa3415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa3964b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3415a070 0x7ffa3415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa3964b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340b26f0 con 0x7ffa3415a070 2024-01-11T07:09:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa3964b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340b1380 con 0x7ffa34159ca0 2024-01-11T07:09:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa3964b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340b1650 con 0x7ffa340cef00 2024-01-11T07:09:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3415a070 0x7ffa3415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3415a070 0x7ffa3415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47140/0 (socket says 172.21.15.7:47140) 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.265+0000 7ffa337fe700 1 -- 172.21.15.7:0/2664859243 learned_addr learned my addr 172.21.15.7:0/2664859243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cef00 0x7ffa340b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa2bfff700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa34159ca0 0x7ffa3415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa337fe700 1 -- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cef00 msgr2=0x7ffa340b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa337fe700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cef00 0x7ffa340b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa337fe700 1 -- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa34159ca0 msgr2=0x7ffa3415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa337fe700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa34159ca0 0x7ffa3415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa337fe700 1 -- 172.21.15.7:0/2664859243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa240094f0 con 0x7ffa3415a070 2024-01-11T07:09:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa337fe700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3415a070 0x7ffa3415c6f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009e90 tx=0x7ffa2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa2affd700 1 -- 172.21.15.7:0/2664859243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00cb70 con 0x7ffa3415a070 2024-01-11T07:09:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa2affd700 1 -- 172.21.15.7:0/2664859243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c00f040 con 0x7ffa3415a070 2024-01-11T07:09:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa340b1930 con 0x7ffa3415a070 2024-01-11T07:09:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa2affd700 1 -- 172.21.15.7:0/2664859243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c014480 con 0x7ffa3415a070 2024-01-11T07:09:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.266+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa340b1ef0 con 0x7ffa3415a070 2024-01-11T07:09:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.267+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3404c3c0 con 0x7ffa3415a070 2024-01-11T07:09:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.268+0000 7ffa2affd700 1 -- 172.21.15.7:0/2664859243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffa2c014830 con 0x7ffa3415a070 2024-01-11T07:09:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.268+0000 7ffa2affd700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffa14066f60 0x7ffa14069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.268+0000 7ffa2affd700 1 -- 172.21.15.7:0/2664859243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffa2c084eb0 con 0x7ffa3415a070 2024-01-11T07:09:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.269+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffa14066f60 0x7ffa14069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.269+0000 7ffa32ffd700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffa14066f60 0x7ffa14069410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffa240099b0 tx=0x7ffa24017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.273+0000 7ffa2affd700 1 -- 172.21.15.7:0/2664859243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa2c052120 con 0x7ffa3415a070 2024-01-11T07:09:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.475+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa340cf9c0 con 0x7ffa14066f60 2024-01-11T07:09:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.483+0000 7ffa2affd700 1 -- 172.21.15.7:0/2664859243 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282055 (secure 0 0 0) 0x7ffa340cf9c0 con 0x7ffa14066f60 2024-01-11T07:09:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.486+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffa14066f60 msgr2=0x7ffa14069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.486+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffa14066f60 0x7ffa14069410 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffa240099b0 tx=0x7ffa24017000 comp rx=0 tx=0).stop 2024-01-11T07:09:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3415a070 msgr2=0x7ffa3415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3415a070 0x7ffa3415c6f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009e90 tx=0x7ffa2c00b040 comp rx=0 tx=0).stop 2024-01-11T07:09:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 shutdown_connections 2024-01-11T07:09:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffa14066f60 0x7ffa14069410 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3415a070 0x7ffa3415c6f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa340cef00 0x7ffa340b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 --2- 172.21.15.7:0/2664859243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa34159ca0 0x7ffa3415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 >> 172.21.15.7:0/2664859243 conn(0x7ffa34063cc0 msgr2=0x7ffa340c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 shutdown_connections 2024-01-11T07:09:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.487+0000 7ffa3964b700 1 -- 172.21.15.7:0/2664859243 wait complete. 2024-01-11T07:09:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:09:13.500 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-01-11T07:09:11.870154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3556,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558943,"statfs":{"total":1600000000,"available":1596358043,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.589132"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.288736+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:22:43.835881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.292201+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:52:35.878435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287910+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:36:56.988470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287694+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:36:56.988470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.288928+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:37:37.524593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.292100+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:14:16.262334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.286116+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:27:13.551988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.292575+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:22:08.210127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.284876+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:45:48.056386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.288954+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:54:53.370334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287225+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:54:41.006083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.286750+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:35:47.171584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.289029+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:16:40.381505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.284456+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:40:39.248740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.285121+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:09:53.056402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.291680+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:36:56.988470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.291768+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:58:01.237181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287151+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:49.957640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287203+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:09:20.643598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.291738+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:54:07.755392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.289900+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:52.445820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.295773+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:59:26.020277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287532+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:35:47.171584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.288194+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:58.402399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.289356+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:45:04.399484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.285649+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:49.957640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.289270+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:49.957640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287954+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:52.145903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.291028+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:54:47.107951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.286821+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:12:40.831991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287353+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:53:17.216834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.288788+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:12:40.831991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.290216+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:52:06.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.288788+0000","last_change":"2024-01-11T07:09:08.288788+0000","last_active":"2024-01-11T07:09:08.288788+0000","last_peered":"2024-01-11T07:09:08.288788+0000","last_clean":"2024-01-11T07:09:08.288788+0000","last_became_active":"2024-01-11T07:09:08.288521+0000","last_became_peered":"2024-01-11T07:09:08.288521+0000","last_unstale":"2024-01-11T07:09:08.288788+0000","last_undegraded":"2024-01-11T07:09:08.288788+0000","last_fullsized":"2024-01-11T07:09:08.288788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:40:38.246668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.289683+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:18:07.532362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.297354+0000","last_change":"2024-01-11T07:09:08.297354+0000","last_active":"2024-01-11T07:09:08.297354+0000","last_peered":"2024-01-11T07:09:08.297354+0000","last_clean":"2024-01-11T07:09:08.297354+0000","last_became_active":"2024-01-11T07:09:08.297059+0000","last_became_peered":"2024-01-11T07:09:08.297059+0000","last_unstale":"2024-01-11T07:09:08.297354+0000","last_undegraded":"2024-01-11T07:09:08.297354+0000","last_fullsized":"2024-01-11T07:09:08.297354+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:09:01.014913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.291283+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:14:34.579163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.296590+0000","last_change":"2024-01-11T07:09:08.296590+0000","last_active":"2024-01-11T07:09:08.296590+0000","last_peered":"2024-01-11T07:09:08.296590+0000","last_clean":"2024-01-11T07:09:08.296590+0000","last_became_active":"2024-01-11T07:09:08.296279+0000","last_became_peered":"2024-01-11T07:09:08.296279+0000","last_unstale":"2024-01-11T07:09:08.296590+0000","last_undegraded":"2024-01-11T07:09:08.296590+0000","last_fullsized":"2024-01-11T07:09:08.296590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:09:52.054331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.293562+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:29.290268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.292606+0000","last_change":"2024-01-11T07:09:08.292606+0000","last_active":"2024-01-11T07:09:08.292606+0000","last_peered":"2024-01-11T07:09:08.292606+0000","last_clean":"2024-01-11T07:09:08.292606+0000","last_became_active":"2024-01-11T07:09:08.292359+0000","last_became_peered":"2024-01-11T07:09:08.292359+0000","last_unstale":"2024-01-11T07:09:08.292606+0000","last_undegraded":"2024-01-11T07:09:08.292606+0000","last_fullsized":"2024-01-11T07:09:08.292606+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:09:52.054331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.291998+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:59:52.925015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.291550+0000","last_change":"2024-01-11T07:09:08.291550+0000","last_active":"2024-01-11T07:09:08.291550+0000","last_peered":"2024-01-11T07:09:08.291550+0000","last_clean":"2024-01-11T07:09:08.291550+0000","last_became_active":"2024-01-11T07:09:08.291318+0000","last_became_peered":"2024-01-11T07:09:08.291318+0000","last_unstale":"2024-01-11T07:09:08.291550+0000","last_undegraded":"2024-01-11T07:09:08.291550+0000","last_fullsized":"2024-01-11T07:09:08.291550+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:33:14.250241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.295381+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:46:55.793938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.296464+0000","last_change":"2024-01-11T07:09:08.296464+0000","last_active":"2024-01-11T07:09:08.296464+0000","last_peered":"2024-01-11T07:09:08.296464+0000","last_clean":"2024-01-11T07:09:08.296464+0000","last_became_active":"2024-01-11T07:09:08.294350+0000","last_became_peered":"2024-01-11T07:09:08.294350+0000","last_unstale":"2024-01-11T07:09:08.296464+0000","last_undegraded":"2024-01-11T07:09:08.296464+0000","last_fullsized":"2024-01-11T07:09:08.296464+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:45:47.054314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.288545+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:47:37.370118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.287875+0000","last_change":"2024-01-11T07:09:08.287875+0000","last_active":"2024-01-11T07:09:08.287875+0000","last_peered":"2024-01-11T07:09:08.287875+0000","last_clean":"2024-01-11T07:09:08.287875+0000","last_became_active":"2024-01-11T07:09:08.287278+0000","last_became_peered":"2024-01-11T07:09:08.287278+0000","last_unstale":"2024-01-11T07:09:08.287875+0000","last_undegraded":"2024-01-11T07:09:08.287875+0000","last_fullsized":"2024-01-11T07:09:08.287875+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:33:54.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.295420+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:26:09.536862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.297633+0000","last_change":"2024-01-11T07:09:08.297633+0000","last_active":"2024-01-11T07:09:08.297633+0000","last_peered":"2024-01-11T07:09:08.297633+0000","last_clean":"2024-01-11T07:09:08.297633+0000","last_became_active":"2024-01-11T07:09:08.297375+0000","last_became_peered":"2024-01-11T07:09:08.297375+0000","last_unstale":"2024-01-11T07:09:08.297633+0000","last_undegraded":"2024-01-11T07:09:08.297633+0000","last_fullsized":"2024-01-11T07:09:08.297633+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:27.924756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.285710+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:48:28.141412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.296186+0000","last_change":"2024-01-11T07:09:08.296186+0000","last_active":"2024-01-11T07:09:08.296186+0000","last_peered":"2024-01-11T07:09:08.296186+0000","last_clean":"2024-01-11T07:09:08.296186+0000","last_became_active":"2024-01-11T07:09:08.296017+0000","last_became_peered":"2024-01-11T07:09:08.296017+0000","last_unstale":"2024-01-11T07:09:08.296186+0000","last_undegraded":"2024-01-11T07:09:08.296186+0000","last_fullsized":"2024-01-11T07:09:08.296186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:33:54.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.289506+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:29.290268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.301111+0000","last_change":"2024-01-11T07:09:08.301111+0000","last_active":"2024-01-11T07:09:08.301111+0000","last_peered":"2024-01-11T07:09:08.301111+0000","last_clean":"2024-01-11T07:09:08.301111+0000","last_became_active":"2024-01-11T07:09:08.300126+0000","last_became_peered":"2024-01-11T07:09:08.300126+0000","last_unstale":"2024-01-11T07:09:08.301111+0000","last_undegraded":"2024-01-11T07:09:08.301111+0000","last_fullsized":"2024-01-11T07:09:08.301111+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:14:15.260263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287639+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:36:15.503114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.297768+0000","last_change":"2024-01-11T07:09:08.297768+0000","last_active":"2024-01-11T07:09:08.297768+0000","last_peered":"2024-01-11T07:09:08.297768+0000","last_clean":"2024-01-11T07:09:08.297768+0000","last_became_active":"2024-01-11T07:09:08.297544+0000","last_became_peered":"2024-01-11T07:09:08.297544+0000","last_unstale":"2024-01-11T07:09:08.297768+0000","last_undegraded":"2024-01-11T07:09:08.297768+0000","last_fullsized":"2024-01-11T07:09:08.297768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:40:38.246668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.286095+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:48:28.141412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.286837+0000","last_change":"2024-01-11T07:09:08.286837+0000","last_active":"2024-01-11T07:09:08.286837+0000","last_peered":"2024-01-11T07:09:08.286837+0000","last_clean":"2024-01-11T07:09:08.286837+0000","last_became_active":"2024-01-11T07:09:08.286549+0000","last_became_peered":"2024-01-11T07:09:08.286549+0000","last_unstale":"2024-01-11T07:09:08.286837+0000","last_undegraded":"2024-01-11T07:09:08.286837+0000","last_fullsized":"2024-01-11T07:09:08.286837+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:54:02.650387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.285298+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:16:22.826955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.296154+0000","last_change":"2024-01-11T07:09:08.296154+0000","last_active":"2024-01-11T07:09:08.296154+0000","last_peered":"2024-01-11T07:09:08.296154+0000","last_clean":"2024-01-11T07:09:08.296154+0000","last_became_active":"2024-01-11T07:09:08.295921+0000","last_became_peered":"2024-01-11T07:09:08.295921+0000","last_unstale":"2024-01-11T07:09:08.296154+0000","last_undegraded":"2024-01-11T07:09:08.296154+0000","last_fullsized":"2024-01-11T07:09:08.296154+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:36:55.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.286502+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:16:22.826955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.293054+0000","last_change":"2024-01-11T07:09:08.293054+0000","last_active":"2024-01-11T07:09:08.293054+0000","last_peered":"2024-01-11T07:09:08.293054+0000","last_clean":"2024-01-11T07:09:08.293054+0000","last_became_active":"2024-01-11T07:09:08.292805+0000","last_became_peered":"2024-01-11T07:09:08.292805+0000","last_unstale":"2024-01-11T07:09:08.293054+0000","last_undegraded":"2024-01-11T07:09:08.293054+0000","last_fullsized":"2024-01-11T07:09:08.293054+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:32:13.616888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.284159+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:33:55.863179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.294439+0000","last_change":"2024-01-11T07:09:08.294439+0000","last_active":"2024-01-11T07:09:08.294439+0000","last_peered":"2024-01-11T07:09:08.294439+0000","last_clean":"2024-01-11T07:09:08.294439+0000","last_became_active":"2024-01-11T07:09:08.294268+0000","last_became_peered":"2024-01-11T07:09:08.294268+0000","last_unstale":"2024-01-11T07:09:08.294439+0000","last_undegraded":"2024-01-11T07:09:08.294439+0000","last_fullsized":"2024-01-11T07:09:08.294439+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:40:38.246668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.289133+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:52:06.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.294974+0000","last_change":"2024-01-11T07:09:08.294974+0000","last_active":"2024-01-11T07:09:08.294974+0000","last_peered":"2024-01-11T07:09:08.294974+0000","last_clean":"2024-01-11T07:09:08.294974+0000","last_became_active":"2024-01-11T07:09:08.293520+0000","last_became_peered":"2024-01-11T07:09:08.293520+0000","last_unstale":"2024-01-11T07:09:08.294974+0000","last_undegraded":"2024-01-11T07:09:08.294974+0000","last_fullsized":"2024-01-11T07:09:08.294974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:17.642140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.286112+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:27:13.551988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.300909+0000","last_change":"2024-01-11T07:09:08.300909+0000","last_active":"2024-01-11T07:09:08.300909+0000","last_peered":"2024-01-11T07:09:08.300909+0000","last_clean":"2024-01-11T07:09:08.300909+0000","last_became_active":"2024-01-11T07:09:08.300672+0000","last_became_peered":"2024-01-11T07:09:08.300672+0000","last_unstale":"2024-01-11T07:09:08.300909+0000","last_undegraded":"2024-01-11T07:09:08.300909+0000","last_fullsized":"2024-01-11T07:09:08.300909+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:51.443749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.285581+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:12:40.831991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.294135+0000","last_change":"2024-01-11T07:09:08.294135+0000","last_active":"2024-01-11T07:09:08.294135+0000","last_peered":"2024-01-11T07:09:08.294135+0000","last_clean":"2024-01-11T07:09:08.294135+0000","last_became_active":"2024-01-11T07:09:08.293857+0000","last_became_peered":"2024-01-11T07:09:08.293857+0000","last_unstale":"2024-01-11T07:09:08.294135+0000","last_undegraded":"2024-01-11T07:09:08.294135+0000","last_fullsized":"2024-01-11T07:09:08.294135+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:07:35.400318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.292631+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:38.567510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.294253+0000","last_change":"2024-01-11T07:09:08.294253+0000","last_active":"2024-01-11T07:09:08.294253+0000","last_peered":"2024-01-11T07:09:08.294253+0000","last_clean":"2024-01-11T07:09:08.294253+0000","last_became_active":"2024-01-11T07:09:08.294023+0000","last_became_peered":"2024-01-11T07:09:08.294023+0000","last_unstale":"2024-01-11T07:09:08.294253+0000","last_undegraded":"2024-01-11T07:09:08.294253+0000","last_fullsized":"2024-01-11T07:09:08.294253+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:45:47.054314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.289104+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:14:16.262334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.299652+0000","last_change":"2024-01-11T07:09:08.299652+0000","last_active":"2024-01-11T07:09:08.299652+0000","last_peered":"2024-01-11T07:09:08.299652+0000","last_clean":"2024-01-11T07:09:08.299652+0000","last_became_active":"2024-01-11T07:09:08.299406+0000","last_became_peered":"2024-01-11T07:09:08.299406+0000","last_unstale":"2024-01-11T07:09:08.299652+0000","last_undegraded":"2024-01-11T07:09:08.299652+0000","last_fullsized":"2024-01-11T07:09:08.299652+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:17.642140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.285148+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:27:13.551988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.293907+0000","last_change":"2024-01-11T07:09:08.293907+0000","last_active":"2024-01-11T07:09:08.293907+0000","last_peered":"2024-01-11T07:09:08.293907+0000","last_clean":"2024-01-11T07:09:08.293907+0000","last_became_active":"2024-01-11T07:09:08.293649+0000","last_became_peered":"2024-01-11T07:09:08.293649+0000","last_unstale":"2024-01-11T07:09:08.293907+0000","last_undegraded":"2024-01-11T07:09:08.293907+0000","last_fullsized":"2024-01-11T07:09:08.293907+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:28.288196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.288169+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:16:22.826955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.294615+0000","last_change":"2024-01-11T07:09:08.294615+0000","last_active":"2024-01-11T07:09:08.294615+0000","last_peered":"2024-01-11T07:09:08.294615+0000","last_clean":"2024-01-11T07:09:08.294615+0000","last_became_active":"2024-01-11T07:09:08.292997+0000","last_became_peered":"2024-01-11T07:09:08.292997+0000","last_unstale":"2024-01-11T07:09:08.294615+0000","last_undegraded":"2024-01-11T07:09:08.294615+0000","last_fullsized":"2024-01-11T07:09:08.294615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:16:39.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.293137+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:06.681281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.297120+0000","last_change":"2024-01-11T07:09:08.297120+0000","last_active":"2024-01-11T07:09:08.297120+0000","last_peered":"2024-01-11T07:09:08.297120+0000","last_clean":"2024-01-11T07:09:08.297120+0000","last_became_active":"2024-01-11T07:09:08.296916+0000","last_became_peered":"2024-01-11T07:09:08.296916+0000","last_unstale":"2024-01-11T07:09:08.297120+0000","last_undegraded":"2024-01-11T07:09:08.297120+0000","last_fullsized":"2024-01-11T07:09:08.297120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:17.642140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287254+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:14:34.579163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.300670+0000","last_change":"2024-01-11T07:09:08.300670+0000","last_active":"2024-01-11T07:09:08.300670+0000","last_peered":"2024-01-11T07:09:08.300670+0000","last_clean":"2024-01-11T07:09:08.300670+0000","last_became_active":"2024-01-11T07:09:08.300418+0000","last_became_peered":"2024-01-11T07:09:08.300418+0000","last_unstale":"2024-01-11T07:09:08.300670+0000","last_undegraded":"2024-01-11T07:09:08.300670+0000","last_fullsized":"2024-01-11T07:09:08.300670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:55:39.579113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.286386+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:52:06.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.296273+0000","last_change":"2024-01-11T07:09:08.296273+0000","last_active":"2024-01-11T07:09:08.296273+0000","last_peered":"2024-01-11T07:09:08.296273+0000","last_clean":"2024-01-11T07:09:08.296273+0000","last_became_active":"2024-01-11T07:09:08.293753+0000","last_became_peered":"2024-01-11T07:09:08.293753+0000","last_unstale":"2024-01-11T07:09:08.296273+0000","last_undegraded":"2024-01-11T07:09:08.296273+0000","last_fullsized":"2024-01-11T07:09:08.296273+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:33:54.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.288570+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:43:23.665806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.293675+0000","last_change":"2024-01-11T07:09:08.293675+0000","last_active":"2024-01-11T07:09:08.293675+0000","last_peered":"2024-01-11T07:09:08.293675+0000","last_clean":"2024-01-11T07:09:08.293675+0000","last_became_active":"2024-01-11T07:09:08.293449+0000","last_became_peered":"2024-01-11T07:09:08.293449+0000","last_unstale":"2024-01-11T07:09:08.293675+0000","last_undegraded":"2024-01-11T07:09:08.293675+0000","last_fullsized":"2024-01-11T07:09:08.293675+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:43.682968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287997+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:27:13.551988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.288814+0000","last_change":"2024-01-11T07:09:08.288814+0000","last_active":"2024-01-11T07:09:08.288814+0000","last_peered":"2024-01-11T07:09:08.288814+0000","last_clean":"2024-01-11T07:09:08.288814+0000","last_became_active":"2024-01-11T07:09:08.288593+0000","last_became_peered":"2024-01-11T07:09:08.288593+0000","last_unstale":"2024-01-11T07:09:08.288814+0000","last_undegraded":"2024-01-11T07:09:08.288814+0000","last_fullsized":"2024-01-11T07:09:08.288814+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:03:52.919770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.294994+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:22:46.243279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.294795+0000","last_change":"2024-01-11T07:09:08.294795+0000","last_active":"2024-01-11T07:09:08.294795+0000","last_peered":"2024-01-11T07:09:08.294795+0000","last_clean":"2024-01-11T07:09:08.294795+0000","last_became_active":"2024-01-11T07:09:08.293581+0000","last_became_peered":"2024-01-11T07:09:08.293581+0000","last_unstale":"2024-01-11T07:09:08.294795+0000","last_undegraded":"2024-01-11T07:09:08.294795+0000","last_fullsized":"2024-01-11T07:09:08.294795+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:57.400327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287723+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:14:34.579163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.287822+0000","last_change":"2024-01-11T07:09:08.287822+0000","last_active":"2024-01-11T07:09:08.287822+0000","last_peered":"2024-01-11T07:09:08.287822+0000","last_clean":"2024-01-11T07:09:08.287822+0000","last_became_active":"2024-01-11T07:09:08.287592+0000","last_became_peered":"2024-01-11T07:09:08.287592+0000","last_unstale":"2024-01-11T07:09:08.287822+0000","last_undegraded":"2024-01-11T07:09:08.287822+0000","last_fullsized":"2024-01-11T07:09:08.287822+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:07:35.400318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.287202+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:18:44.510596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.298120+0000","last_change":"2024-01-11T07:09:08.298120+0000","last_active":"2024-01-11T07:09:08.298120+0000","last_peered":"2024-01-11T07:09:08.298120+0000","last_clean":"2024-01-11T07:09:08.298120+0000","last_became_active":"2024-01-11T07:09:08.297739+0000","last_became_peered":"2024-01-11T07:09:08.297739+0000","last_unstale":"2024-01-11T07:09:08.298120+0000","last_undegraded":"2024-01-11T07:09:08.298120+0000","last_fullsized":"2024-01-11T07:09:08.298120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:47:36.368046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.291087+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:18:15.830741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.296860+0000","last_change":"2024-01-11T07:09:08.296860+0000","last_active":"2024-01-11T07:09:08.296860+0000","last_peered":"2024-01-11T07:09:08.296860+0000","last_clean":"2024-01-11T07:09:08.296860+0000","last_became_active":"2024-01-11T07:09:08.296386+0000","last_became_peered":"2024-01-11T07:09:08.296386+0000","last_unstale":"2024-01-11T07:09:08.296860+0000","last_undegraded":"2024-01-11T07:09:08.296860+0000","last_fullsized":"2024-01-11T07:09:08.296860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:31:41.910438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:09:08.279367+0000","last_change":"2024-01-11T07:09:08.292060+0000","last_active":"2024-01-11T07:09:08.279367+0000","last_peered":"2024-01-11T07:09:08.279367+0000","last_clean":"2024-01-11T07:09:08.279367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:09:08.279367+0000","last_undegraded":"2024-01-11T07:09:08.279367+0000","last_fullsized":"2024-01-11T07:09:08.279367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:57.551943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-01-11T07:09:08.290931+0000","last_change":"2024-01-11T07:09:08.290931+0000","last_active":"2024-01-11T07:09:08.290931+0000","last_peered":"2024-01-11T07:09:08.290931+0000","last_clean":"2024-01-11T07:09:08.290931+0000","last_became_active":"2024-01-11T07:09:08.290691+0000","last_became_peered":"2024-01-11T07:09:08.290691+0000","last_unstale":"2024-01-11T07:09:08.290931+0000","last_undegraded":"2024-01-11T07:09:08.290931+0000","last_fullsized":"2024-01-11T07:09:08.290931+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:44:51.315800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":199,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-01-11T07:09:07.286521+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:09:07.286521+0000","last_peered":"2024-01-11T07:09:07.286521+0000","last_clean":"2024-01-11T07:09:07.286521+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:09:07.286521+0000","last_undegraded":"2024-01-11T07:09:07.286521+0000","last_fullsized":"2024-01-11T07:09:07.286521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":874,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194437,"statfs":{"total":200000000,"available":199104325,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.745},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.40999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.5},"min":{"1min":0.433,"5min":0.311,"15min":0.311},"max":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.441,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.628}]},{"osd":3,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.432,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.32600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.437}]},{"osd":5,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46000000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.497},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":304,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195008,"statfs":{"total":200000000,"available":199688438,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76500000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70299999999999996,"5min":1.157,"15min":1.157},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.62,"15min":0.622},"min":{"1min":0.42899999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.32100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.749,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63900000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.622}]},{"osd":7,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.82399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":304,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195008,"statfs":{"total":200000000,"available":199688438,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":0.62,"15min":0.48599999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":20.41,"5min":20.41,"15min":20.41},"last":0.374},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45200000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.57799999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":0.56799999999999995,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":15.611000000000001,"5min":15.611000000000001,"15min":15.611000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":1.482,"5min":0.67900000000000005,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":20.199999999999999,"5min":20.199999999999999,"15min":20.199999999999999},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":0.67400000000000004,"15min":0.54100000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":20.369,"5min":20.369,"15min":20.369},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.435,"15min":0.439},"min":{"1min":0.313,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.749,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.499}]},{"osd":4,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.70199999999999996,"15min":0.64200000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":9.9570000000000007,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.226,"5min":0.73999999999999999,"15min":0.65900000000000003},"min":{"1min":0.498,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.73099999999999998,"15min":0.67500000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":9.891,"5min":9.891,"15min":9.891},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.77800000000000002,"15min":0.72799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":9.9900000000000002,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":0.90200000000000002,"15min":0.77600000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":20.324000000000002,"5min":20.324000000000002,"15min":20.324000000000002},"last":0.871},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.5209999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":880,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194431,"statfs":{"total":200000000,"available":199098070,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.47399999999999998,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.59699999999999998,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55400000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.73599999999999999,"15min":0.78900000000000003},"min":{"1min":0.307,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.73199999999999998,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.53100000000000003,"15min":0.55200000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.62,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.75,"15min":0.80700000000000005},"min":{"1min":0.29699999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57699999999999996,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.57399999999999995,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77700000000000002,"5min":2.855,"15min":2.855},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.71799999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.90600000000000003,"15min":0.95799999999999996},"min":{"1min":0.432,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83299999999999996,"5min":2.819,"15min":2.819},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.97099999999999997,"15min":1.0249999999999999},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.879,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.62},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.81699999999999995,"15min":0.78800000000000003},"min":{"1min":0.53700000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":7.4690000000000003,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.87,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.96099999999999997,"15min":1.01},"min":{"1min":0.52400000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89500000000000002,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":298,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195014,"statfs":{"total":200000000,"available":199694693,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41999999999999998,"15min":0.42399999999999999},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.624,"5min":1.069,"15min":1.069},"last":0.37},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39300000000000002,"15min":0.39400000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.443,"15min":0.443},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68700000000000006,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.48099999999999998},"min":{"1min":0.377,"5min":0.313,"15min":0.313},"max":{"1min":0.68400000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76800000000000002,"5min":1.139,"15min":1.139},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.48799999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71999999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89400000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.878,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70699999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.85799999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82799999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82099999999999995,"5min":1.052,"15min":1.052},"last":0.628}]},{"osd":7,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68000000000000005,"15min":0.68799999999999994},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.748,"5min":1.038,"15min":1.038},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":298,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195014,"statfs":{"total":200000000,"available":199694693,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.81899999999999995,"15min":0.89100000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54300000000000004,"5min":11.481,"15min":11.481},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.61599999999999999,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":1.026,"15min":1.133},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51900000000000002,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.65000000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42499999999999999,"15min":0.42699999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.54500000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.434},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.0820000000000001,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59899999999999998,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88600000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":1.321,"15min":1.4239999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92900000000000005,"5min":17.474,"15min":17.474},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.256,"15min":1.3560000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95299999999999996,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.0449999999999999,"15min":1.111},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":11.394,"15min":11.394},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":1.0609999999999999,"15min":1.129},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85799999999999998,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.288,"15min":1.3859999999999999},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97099999999999997,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.68999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":298,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195014,"statfs":{"total":200000000,"available":199694693,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":1.0780000000000001,"15min":1.1559999999999999},"min":{"1min":0.40799999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.78200000000000003,"5min":8.484,"15min":8.484},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.139,"15min":1.228},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79900000000000004,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.88800000000000001,"15min":0.93799999999999994},"min":{"1min":0.437,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":1.0329999999999999,"15min":1.1100000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81399999999999995,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.98899999999999999,"15min":1.054},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85899999999999999,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.90000000000000002,"15min":0.95699999999999996},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72199999999999998,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.0860000000000001,"15min":1.173},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74399999999999999,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":1.1479999999999999,"15min":1.24},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79300000000000004,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.98199999999999998,"15min":1.0720000000000001},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.89100000000000001,"15min":0.95699999999999996},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67000000000000004,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.48799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.95099999999999996,"15min":1.0169999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82599999999999996,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":1.119,"15min":1.119},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.91600000000000004,"15min":0.98399999999999999},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66400000000000003,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.628,"5min":1.024,"15min":1.0900000000000001},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.73099999999999998,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":298,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195014,"statfs":{"total":200000000,"available":199694693,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82699999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":0.87,"15min":0.87},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81599999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.874,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84199999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.60099999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.40100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53600000000000003,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.76500000000000001}]}]}],"pool_statfs":[]}} 2024-01-11T07:09:13.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:09:13.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.675+0000 7f9c000e6700 1 Processor -- start 2024-01-11T07:09:13.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.675+0000 7f9c000e6700 1 -- start start 2024-01-11T07:09:13.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.676+0000 7f9c000e6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bf80ac3a0 0x7f9bf80cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.676+0000 7f9c000e6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf80ab8d0 0x7f9bf815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.676+0000 7f9c000e6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf815e2d0 con 0x7f9bf815d920 2024-01-11T07:09:13.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.676+0000 7f9c000e6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf806ac30 con 0x7f9bf80ac3a0 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.676+0000 7f9c000e6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80ac770 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.676+0000 7f9bfde82700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bf80ac3a0 0x7f9bf80cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.676+0000 7f9bfde82700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bf80ac3a0 0x7f9bf80cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47154/0 (socket says 172.21.15.7:47154) 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.676+0000 7f9bfde82700 1 -- 172.21.15.7:0/1366181744 learned_addr learned my addr 172.21.15.7:0/1366181744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfe683700 1 --2- 172.21.15.7:0/1366181744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf80ab8d0 0x7f9bf815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfde82700 1 -- 172.21.15.7:0/1366181744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf80ab8d0 msgr2=0x7f9bf815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfde82700 1 --2- 172.21.15.7:0/1366181744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf80ab8d0 0x7f9bf815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfde82700 1 -- 172.21.15.7:0/1366181744 >> v1:172.21.15.113:6789/0 conn(0x7f9bf815d920 legacy=0x7f9bf815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfde82700 1 -- 172.21.15.7:0/1366181744 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf806a300 con 0x7f9bf80ac3a0 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfde82700 1 --2- 172.21.15.7:0/1366181744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ac3a0 0x7f9bf80cac60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0099b0 tx=0x7f9bec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf415cecbb74f4e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfce80700 1 -- 172.21.15.7:0/1366181744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec009ea0 con 0x7f9bf80ac3a0 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfce80700 1 -- 172.21.15.7:0/1366181744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec00b860 con 0x7f9bf80ac3a0 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.677+0000 7f9bfce80700 1 -- 172.21.15.7:0/1366181744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00bc00 con 0x7f9bf80ac3a0 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.678+0000 7f9c000e6700 1 -- 172.21.15.7:0/1366181744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ac3a0 msgr2=0x7f9bf80cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.678+0000 7f9c000e6700 1 --2- 172.21.15.7:0/1366181744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ac3a0 0x7f9bf80cac60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0099b0 tx=0x7f9bec009250 comp rx=0 tx=0).stop 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.678+0000 7f9c000e6700 1 -- 172.21.15.7:0/1366181744 shutdown_connections 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.678+0000 7f9c000e6700 1 --2- 172.21.15.7:0/1366181744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf80ab8d0 0x7f9bf815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.678+0000 7f9c000e6700 1 --2- 172.21.15.7:0/1366181744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ac3a0 0x7f9bf80cac60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.678+0000 7f9c000e6700 1 -- 172.21.15.7:0/1366181744 >> 172.21.15.7:0/1366181744 conn(0x7f9bf80cd090 msgr2=0x7f9bf80cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.678+0000 7f9c000e6700 1 -- 172.21.15.7:0/1366181744 shutdown_connections 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.678+0000 7f9c000e6700 1 -- 172.21.15.7:0/1366181744 wait complete. 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.679+0000 7f9c000e6700 1 Processor -- start 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.679+0000 7f9c000e6700 1 -- start start 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9c000e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ab8d0 0x7f9bf813e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9c000e6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bf80ac3a0 0x7f9bf8169510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9c000e6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf815d920 0x7f9bf8169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9c000e6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf813f940 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9c000e6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8142860 con 0x7f9bf80ac3a0 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9c000e6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf813ed30 con 0x7f9bf815d920 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ab8d0 0x7f9bf813e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ab8d0 0x7f9bf813e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47162/0 (socket says 172.21.15.7:47162) 2024-01-11T07:09:13.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 -- 172.21.15.7:0/204178600 learned_addr learned my addr 172.21.15.7:0/204178600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfd681700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bf80ac3a0 0x7f9bf8169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfe683700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf815d920 0x7f9bf8169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 -- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf815d920 msgr2=0x7f9bf8169a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf815d920 0x7f9bf8169a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 -- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bf80ac3a0 msgr2=0x7f9bf8169510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bf80ac3a0 0x7f9bf8169510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 -- 172.21.15.7:0/204178600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0094f0 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.680+0000 7f9bfde82700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ab8d0 0x7f9bf813e2e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806fdf0 tx=0x7f9bec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.681+0000 7f9be9ffb700 1 -- 172.21.15.7:0/204178600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00bbe0 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.681+0000 7f9be9ffb700 1 -- 172.21.15.7:0/204178600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec011420 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.681+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf813f010 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.681+0000 7f9be9ffb700 1 -- 172.21.15.7:0/204178600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0117c0 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.681+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf813f5d0 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.682+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf804c3c0 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.682+0000 7f9be9ffb700 1 -- 172.21.15.7:0/204178600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9bec011bf0 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.683+0000 7f9be9ffb700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bdc066f30 0x7f9bdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:13.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.683+0000 7f9be9ffb700 1 -- 172.21.15.7:0/204178600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9bec085e70 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.683+0000 7f9bfd681700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bdc066f30 0x7f9bdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.684+0000 7f9bfd681700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bdc066f30 0x7f9bdc0693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80b2fb0 tx=0x7f9bf4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.687+0000 7f9be9ffb700 1 -- 172.21.15.7:0/204178600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bec0530e0 con 0x7f9bf80ab8d0 2024-01-11T07:09:13.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.886+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bf80c7520 con 0x7f9bdc066f30 2024-01-11T07:09:13.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.895+0000 7f9be9ffb700 1 -- 172.21.15.7:0/204178600 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284969 (secure 0 0 0) 0x7f9bf80c7520 con 0x7f9bdc066f30 2024-01-11T07:09:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:13.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bdc066f30 msgr2=0x7f9bdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bdc066f30 0x7f9bdc0693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80b2fb0 tx=0x7f9bf4009040 comp rx=0 tx=0).stop 2024-01-11T07:09:13.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ab8d0 msgr2=0x7f9bf813e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:13.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ab8d0 0x7f9bf813e2e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806fdf0 tx=0x7f9bec017000 comp rx=0 tx=0).stop 2024-01-11T07:09:13.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 shutdown_connections 2024-01-11T07:09:13.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bdc066f30 0x7f9bdc0693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9bf815d920 0x7f9bf8169a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf80ab8d0 0x7f9bf813e2e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 --2- 172.21.15.7:0/204178600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bf80ac3a0 0x7f9bf8169510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:13.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 >> 172.21.15.7:0/204178600 conn(0x7f9bf80cd090 msgr2=0x7f9bf80c35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:13.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 shutdown_connections 2024-01-11T07:09:13.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:13.899+0000 7f9c000e6700 1 -- 172.21.15.7:0/204178600 wait complete. 2024-01-11T07:09:13.902 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:09:13.912 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-01-11T07:09:13.870673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3693,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558806,"statfs":{"total":1600000000,"available":1596217748,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001417"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.328060+0000","last_change":"2024-01-11T07:09:09.327990+0000","last_active":"2024-01-11T07:09:09.328060+0000","last_peered":"2024-01-11T07:09:09.328060+0000","last_clean":"2024-01-11T07:09:09.328060+0000","last_became_active":"2024-01-11T07:09:09.327748+0000","last_became_peered":"2024-01-11T07:09:09.327748+0000","last_unstale":"2024-01-11T07:09:09.328060+0000","last_undegraded":"2024-01-11T07:09:09.328060+0000","last_fullsized":"2024-01-11T07:09:09.328060+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:09:48.890233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.332348+0000","last_change":"2024-01-11T07:09:09.332291+0000","last_active":"2024-01-11T07:09:09.332348+0000","last_peered":"2024-01-11T07:09:09.332348+0000","last_clean":"2024-01-11T07:09:09.332348+0000","last_became_active":"2024-01-11T07:09:09.330713+0000","last_became_peered":"2024-01-11T07:09:09.330713+0000","last_unstale":"2024-01-11T07:09:09.332348+0000","last_undegraded":"2024-01-11T07:09:09.332348+0000","last_fullsized":"2024-01-11T07:09:09.332348+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:17:31.292878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.315060+0000","last_change":"2024-01-11T07:09:09.314981+0000","last_active":"2024-01-11T07:09:09.315060+0000","last_peered":"2024-01-11T07:09:09.315060+0000","last_clean":"2024-01-11T07:09:09.315060+0000","last_became_active":"2024-01-11T07:09:09.314691+0000","last_became_peered":"2024-01-11T07:09:09.314691+0000","last_unstale":"2024-01-11T07:09:09.315060+0000","last_undegraded":"2024-01-11T07:09:09.315060+0000","last_fullsized":"2024-01-11T07:09:09.315060+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:54:47.107951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.322080+0000","last_change":"2024-01-11T07:09:09.321147+0000","last_active":"2024-01-11T07:09:09.322080+0000","last_peered":"2024-01-11T07:09:09.322080+0000","last_clean":"2024-01-11T07:09:09.322080+0000","last_became_active":"2024-01-11T07:09:09.320869+0000","last_became_peered":"2024-01-11T07:09:09.320869+0000","last_unstale":"2024-01-11T07:09:09.322080+0000","last_undegraded":"2024-01-11T07:09:09.322080+0000","last_fullsized":"2024-01-11T07:09:09.322080+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:06.681281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.320628+0000","last_change":"2024-01-11T07:09:09.320576+0000","last_active":"2024-01-11T07:09:09.320628+0000","last_peered":"2024-01-11T07:09:09.320628+0000","last_clean":"2024-01-11T07:09:09.320628+0000","last_became_active":"2024-01-11T07:09:09.320371+0000","last_became_peered":"2024-01-11T07:09:09.320371+0000","last_unstale":"2024-01-11T07:09:09.320628+0000","last_undegraded":"2024-01-11T07:09:09.320628+0000","last_fullsized":"2024-01-11T07:09:09.320628+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:53:29.845913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.323432+0000","last_change":"2024-01-11T07:09:09.323290+0000","last_active":"2024-01-11T07:09:09.323432+0000","last_peered":"2024-01-11T07:09:09.323432+0000","last_clean":"2024-01-11T07:09:09.323432+0000","last_became_active":"2024-01-11T07:09:09.323011+0000","last_became_peered":"2024-01-11T07:09:09.323011+0000","last_unstale":"2024-01-11T07:09:09.323432+0000","last_undegraded":"2024-01-11T07:09:09.323432+0000","last_fullsized":"2024-01-11T07:09:09.323432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:59.669567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.319890+0000","last_change":"2024-01-11T07:09:09.319822+0000","last_active":"2024-01-11T07:09:09.319890+0000","last_peered":"2024-01-11T07:09:09.319890+0000","last_clean":"2024-01-11T07:09:09.319890+0000","last_became_active":"2024-01-11T07:09:09.319583+0000","last_became_peered":"2024-01-11T07:09:09.319583+0000","last_unstale":"2024-01-11T07:09:09.319890+0000","last_undegraded":"2024-01-11T07:09:09.319890+0000","last_fullsized":"2024-01-11T07:09:09.319890+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:12:40.831991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.327089+0000","last_change":"2024-01-11T07:09:09.326810+0000","last_active":"2024-01-11T07:09:09.327089+0000","last_peered":"2024-01-11T07:09:09.327089+0000","last_clean":"2024-01-11T07:09:09.327089+0000","last_became_active":"2024-01-11T07:09:09.326506+0000","last_became_peered":"2024-01-11T07:09:09.326506+0000","last_unstale":"2024-01-11T07:09:09.327089+0000","last_undegraded":"2024-01-11T07:09:09.327089+0000","last_fullsized":"2024-01-11T07:09:09.327089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:05:29.398927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.323147+0000","last_change":"2024-01-11T07:09:09.323080+0000","last_active":"2024-01-11T07:09:09.323147+0000","last_peered":"2024-01-11T07:09:09.323147+0000","last_clean":"2024-01-11T07:09:09.323147+0000","last_became_active":"2024-01-11T07:09:09.322759+0000","last_became_peered":"2024-01-11T07:09:09.322759+0000","last_unstale":"2024-01-11T07:09:09.323147+0000","last_undegraded":"2024-01-11T07:09:09.323147+0000","last_fullsized":"2024-01-11T07:09:09.323147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:16:22.826955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.325060+0000","last_change":"2024-01-11T07:09:09.324921+0000","last_active":"2024-01-11T07:09:09.325060+0000","last_peered":"2024-01-11T07:09:09.325060+0000","last_clean":"2024-01-11T07:09:09.325060+0000","last_became_active":"2024-01-11T07:09:09.324457+0000","last_became_peered":"2024-01-11T07:09:09.324457+0000","last_unstale":"2024-01-11T07:09:09.325060+0000","last_undegraded":"2024-01-11T07:09:09.325060+0000","last_fullsized":"2024-01-11T07:09:09.325060+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:48:53.685787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.333549+0000","last_change":"2024-01-11T07:09:09.333203+0000","last_active":"2024-01-11T07:09:09.333549+0000","last_peered":"2024-01-11T07:09:09.333549+0000","last_clean":"2024-01-11T07:09:09.333549+0000","last_became_active":"2024-01-11T07:09:09.329413+0000","last_became_peered":"2024-01-11T07:09:09.329413+0000","last_unstale":"2024-01-11T07:09:09.333549+0000","last_undegraded":"2024-01-11T07:09:09.333549+0000","last_fullsized":"2024-01-11T07:09:09.333549+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:33:15.252313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.317441+0000","last_change":"2024-01-11T07:09:09.317361+0000","last_active":"2024-01-11T07:09:09.317441+0000","last_peered":"2024-01-11T07:09:09.317441+0000","last_clean":"2024-01-11T07:09:09.317441+0000","last_became_active":"2024-01-11T07:09:09.317011+0000","last_became_peered":"2024-01-11T07:09:09.317011+0000","last_unstale":"2024-01-11T07:09:09.317441+0000","last_undegraded":"2024-01-11T07:09:09.317441+0000","last_fullsized":"2024-01-11T07:09:09.317441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:58.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.319688+0000","last_change":"2024-01-11T07:09:09.319617+0000","last_active":"2024-01-11T07:09:09.319688+0000","last_peered":"2024-01-11T07:09:09.319688+0000","last_clean":"2024-01-11T07:09:09.319688+0000","last_became_active":"2024-01-11T07:09:09.319195+0000","last_became_peered":"2024-01-11T07:09:09.319195+0000","last_unstale":"2024-01-11T07:09:09.319688+0000","last_undegraded":"2024-01-11T07:09:09.319688+0000","last_fullsized":"2024-01-11T07:09:09.319688+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:18:44.510596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.322101+0000","last_change":"2024-01-11T07:09:09.321244+0000","last_active":"2024-01-11T07:09:09.322101+0000","last_peered":"2024-01-11T07:09:09.322101+0000","last_clean":"2024-01-11T07:09:09.322101+0000","last_became_active":"2024-01-11T07:09:09.321024+0000","last_became_peered":"2024-01-11T07:09:09.321024+0000","last_unstale":"2024-01-11T07:09:09.322101+0000","last_undegraded":"2024-01-11T07:09:09.322101+0000","last_fullsized":"2024-01-11T07:09:09.322101+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:52:06.865459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.325742+0000","last_change":"2024-01-11T07:09:09.325657+0000","last_active":"2024-01-11T07:09:09.325742+0000","last_peered":"2024-01-11T07:09:09.325742+0000","last_clean":"2024-01-11T07:09:09.325742+0000","last_became_active":"2024-01-11T07:09:09.325375+0000","last_became_peered":"2024-01-11T07:09:09.325375+0000","last_unstale":"2024-01-11T07:09:09.325742+0000","last_undegraded":"2024-01-11T07:09:09.325742+0000","last_fullsized":"2024-01-11T07:09:09.325742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:35:47.171584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.311173+0000","last_change":"2024-01-11T07:09:09.310969+0000","last_active":"2024-01-11T07:09:09.311173+0000","last_peered":"2024-01-11T07:09:09.311173+0000","last_clean":"2024-01-11T07:09:09.311173+0000","last_became_active":"2024-01-11T07:09:09.310660+0000","last_became_peered":"2024-01-11T07:09:09.310660+0000","last_unstale":"2024-01-11T07:09:09.311173+0000","last_undegraded":"2024-01-11T07:09:09.311173+0000","last_fullsized":"2024-01-11T07:09:09.311173+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:18:15.830741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.332067+0000","last_change":"2024-01-11T07:09:09.331974+0000","last_active":"2024-01-11T07:09:09.332067+0000","last_peered":"2024-01-11T07:09:09.332067+0000","last_clean":"2024-01-11T07:09:09.332067+0000","last_became_active":"2024-01-11T07:09:09.328046+0000","last_became_peered":"2024-01-11T07:09:09.328046+0000","last_unstale":"2024-01-11T07:09:09.332067+0000","last_undegraded":"2024-01-11T07:09:09.332067+0000","last_fullsized":"2024-01-11T07:09:09.332067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:05:42.413757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.313873+0000","last_change":"2024-01-11T07:09:09.312947+0000","last_active":"2024-01-11T07:09:09.313873+0000","last_peered":"2024-01-11T07:09:09.313873+0000","last_clean":"2024-01-11T07:09:09.313873+0000","last_became_active":"2024-01-11T07:09:09.312715+0000","last_became_peered":"2024-01-11T07:09:09.312715+0000","last_unstale":"2024-01-11T07:09:09.313873+0000","last_undegraded":"2024-01-11T07:09:09.313873+0000","last_fullsized":"2024-01-11T07:09:09.313873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:59.669567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.328647+0000","last_change":"2024-01-11T07:09:09.328571+0000","last_active":"2024-01-11T07:09:09.328647+0000","last_peered":"2024-01-11T07:09:09.328647+0000","last_clean":"2024-01-11T07:09:09.328647+0000","last_became_active":"2024-01-11T07:09:09.328323+0000","last_became_peered":"2024-01-11T07:09:09.328323+0000","last_unstale":"2024-01-11T07:09:09.328647+0000","last_undegraded":"2024-01-11T07:09:09.328647+0000","last_fullsized":"2024-01-11T07:09:09.328647+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:45:53.606265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.327007+0000","last_change":"2024-01-11T07:09:09.325273+0000","last_active":"2024-01-11T07:09:09.327007+0000","last_peered":"2024-01-11T07:09:09.327007+0000","last_clean":"2024-01-11T07:09:09.327007+0000","last_became_active":"2024-01-11T07:09:09.324979+0000","last_became_peered":"2024-01-11T07:09:09.324979+0000","last_unstale":"2024-01-11T07:09:09.327007+0000","last_undegraded":"2024-01-11T07:09:09.327007+0000","last_fullsized":"2024-01-11T07:09:09.327007+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:07:12.971171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.321560+0000","last_change":"2024-01-11T07:09:09.318174+0000","last_active":"2024-01-11T07:09:09.321560+0000","last_peered":"2024-01-11T07:09:09.321560+0000","last_clean":"2024-01-11T07:09:09.321560+0000","last_became_active":"2024-01-11T07:09:09.317954+0000","last_became_peered":"2024-01-11T07:09:09.317954+0000","last_unstale":"2024-01-11T07:09:09.321560+0000","last_undegraded":"2024-01-11T07:09:09.321560+0000","last_fullsized":"2024-01-11T07:09:09.321560+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:58.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.332822+0000","last_change":"2024-01-11T07:09:09.332737+0000","last_active":"2024-01-11T07:09:09.332822+0000","last_peered":"2024-01-11T07:09:09.332822+0000","last_clean":"2024-01-11T07:09:09.332822+0000","last_became_active":"2024-01-11T07:09:09.331622+0000","last_became_peered":"2024-01-11T07:09:09.331622+0000","last_unstale":"2024-01-11T07:09:09.332822+0000","last_undegraded":"2024-01-11T07:09:09.332822+0000","last_fullsized":"2024-01-11T07:09:09.332822+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:40:48.826715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.327529+0000","last_change":"2024-01-11T07:09:09.327484+0000","last_active":"2024-01-11T07:09:09.327529+0000","last_peered":"2024-01-11T07:09:09.327529+0000","last_clean":"2024-01-11T07:09:09.327529+0000","last_became_active":"2024-01-11T07:09:09.327295+0000","last_became_peered":"2024-01-11T07:09:09.327295+0000","last_unstale":"2024-01-11T07:09:09.327529+0000","last_undegraded":"2024-01-11T07:09:09.327529+0000","last_fullsized":"2024-01-11T07:09:09.327529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:54:07.755392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.319105+0000","last_change":"2024-01-11T07:09:09.319037+0000","last_active":"2024-01-11T07:09:09.319105+0000","last_peered":"2024-01-11T07:09:09.319105+0000","last_clean":"2024-01-11T07:09:09.319105+0000","last_became_active":"2024-01-11T07:09:09.318722+0000","last_became_peered":"2024-01-11T07:09:09.318722+0000","last_unstale":"2024-01-11T07:09:09.319105+0000","last_undegraded":"2024-01-11T07:09:09.319105+0000","last_fullsized":"2024-01-11T07:09:09.319105+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:44.462853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326706+0000","last_change":"2024-01-11T07:09:09.326611+0000","last_active":"2024-01-11T07:09:09.326706+0000","last_peered":"2024-01-11T07:09:09.326706+0000","last_clean":"2024-01-11T07:09:09.326706+0000","last_became_active":"2024-01-11T07:09:09.326325+0000","last_became_peered":"2024-01-11T07:09:09.326325+0000","last_unstale":"2024-01-11T07:09:09.326706+0000","last_undegraded":"2024-01-11T07:09:09.326706+0000","last_fullsized":"2024-01-11T07:09:09.326706+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:40:50.521189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.317907+0000","last_change":"2024-01-11T07:09:09.317810+0000","last_active":"2024-01-11T07:09:09.317907+0000","last_peered":"2024-01-11T07:09:09.317907+0000","last_clean":"2024-01-11T07:09:09.317907+0000","last_became_active":"2024-01-11T07:09:09.317597+0000","last_became_peered":"2024-01-11T07:09:09.317597+0000","last_unstale":"2024-01-11T07:09:09.317907+0000","last_undegraded":"2024-01-11T07:09:09.317907+0000","last_fullsized":"2024-01-11T07:09:09.317907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:14:16.262334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.319754+0000","last_change":"2024-01-11T07:09:09.311683+0000","last_active":"2024-01-11T07:09:09.319754+0000","last_peered":"2024-01-11T07:09:09.319754+0000","last_clean":"2024-01-11T07:09:09.319754+0000","last_became_active":"2024-01-11T07:09:09.311437+0000","last_became_peered":"2024-01-11T07:09:09.311437+0000","last_unstale":"2024-01-11T07:09:09.319754+0000","last_undegraded":"2024-01-11T07:09:09.319754+0000","last_fullsized":"2024-01-11T07:09:09.319754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:31:42.912510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.322173+0000","last_change":"2024-01-11T07:09:09.321444+0000","last_active":"2024-01-11T07:09:09.322173+0000","last_peered":"2024-01-11T07:09:09.322173+0000","last_clean":"2024-01-11T07:09:09.322173+0000","last_became_active":"2024-01-11T07:09:09.321207+0000","last_became_peered":"2024-01-11T07:09:09.321207+0000","last_unstale":"2024-01-11T07:09:09.322173+0000","last_undegraded":"2024-01-11T07:09:09.322173+0000","last_fullsized":"2024-01-11T07:09:09.322173+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:18:58.548770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.327023+0000","last_change":"2024-01-11T07:09:09.325614+0000","last_active":"2024-01-11T07:09:09.327023+0000","last_peered":"2024-01-11T07:09:09.327023+0000","last_clean":"2024-01-11T07:09:09.327023+0000","last_became_active":"2024-01-11T07:09:09.325334+0000","last_became_peered":"2024-01-11T07:09:09.325334+0000","last_unstale":"2024-01-11T07:09:09.327023+0000","last_undegraded":"2024-01-11T07:09:09.327023+0000","last_fullsized":"2024-01-11T07:09:09.327023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:47:40.570974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.315911+0000","last_change":"2024-01-11T07:09:09.315730+0000","last_active":"2024-01-11T07:09:09.315911+0000","last_peered":"2024-01-11T07:09:09.315911+0000","last_clean":"2024-01-11T07:09:09.315911+0000","last_became_active":"2024-01-11T07:09:09.315331+0000","last_became_peered":"2024-01-11T07:09:09.315331+0000","last_unstale":"2024-01-11T07:09:09.315911+0000","last_undegraded":"2024-01-11T07:09:09.315911+0000","last_fullsized":"2024-01-11T07:09:09.315911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:31:42.912510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.330421+0000","last_change":"2024-01-11T07:09:09.330352+0000","last_active":"2024-01-11T07:09:09.330421+0000","last_peered":"2024-01-11T07:09:09.330421+0000","last_clean":"2024-01-11T07:09:09.330421+0000","last_became_active":"2024-01-11T07:09:09.330059+0000","last_became_peered":"2024-01-11T07:09:09.330059+0000","last_unstale":"2024-01-11T07:09:09.330421+0000","last_undegraded":"2024-01-11T07:09:09.330421+0000","last_fullsized":"2024-01-11T07:09:09.330421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:08:15.754949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.332521+0000","last_change":"2024-01-11T07:09:09.332456+0000","last_active":"2024-01-11T07:09:09.332521+0000","last_peered":"2024-01-11T07:09:09.332521+0000","last_clean":"2024-01-11T07:09:09.332521+0000","last_became_active":"2024-01-11T07:09:09.332231+0000","last_became_peered":"2024-01-11T07:09:09.332231+0000","last_unstale":"2024-01-11T07:09:09.332521+0000","last_undegraded":"2024-01-11T07:09:09.332521+0000","last_fullsized":"2024-01-11T07:09:09.332521+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:54:07.755392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.319886+0000","last_change":"2024-01-11T07:09:09.317748+0000","last_active":"2024-01-11T07:09:09.319886+0000","last_peered":"2024-01-11T07:09:09.319886+0000","last_clean":"2024-01-11T07:09:09.319886+0000","last_became_active":"2024-01-11T07:09:09.317106+0000","last_became_peered":"2024-01-11T07:09:09.317106+0000","last_unstale":"2024-01-11T07:09:09.319886+0000","last_undegraded":"2024-01-11T07:09:09.319886+0000","last_fullsized":"2024-01-11T07:09:09.319886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:57.551943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"63'4","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.328471+0000","last_change":"2024-01-11T07:09:08.288788+0000","last_active":"2024-01-11T07:09:09.328471+0000","last_peered":"2024-01-11T07:09:09.328471+0000","last_clean":"2024-01-11T07:09:09.328471+0000","last_became_active":"2024-01-11T07:09:08.288521+0000","last_became_peered":"2024-01-11T07:09:08.288521+0000","last_unstale":"2024-01-11T07:09:09.328471+0000","last_undegraded":"2024-01-11T07:09:09.328471+0000","last_fullsized":"2024-01-11T07:09:09.328471+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:40:38.246668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.323195+0000","last_change":"2024-01-11T07:09:09.323125+0000","last_active":"2024-01-11T07:09:09.323195+0000","last_peered":"2024-01-11T07:09:09.323195+0000","last_clean":"2024-01-11T07:09:09.323195+0000","last_became_active":"2024-01-11T07:09:09.322485+0000","last_became_peered":"2024-01-11T07:09:09.322485+0000","last_unstale":"2024-01-11T07:09:09.323195+0000","last_undegraded":"2024-01-11T07:09:09.323195+0000","last_fullsized":"2024-01-11T07:09:09.323195+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:07:36.402389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326730+0000","last_change":"2024-01-11T07:09:08.297354+0000","last_active":"2024-01-11T07:09:09.326730+0000","last_peered":"2024-01-11T07:09:09.326730+0000","last_clean":"2024-01-11T07:09:09.326730+0000","last_became_active":"2024-01-11T07:09:08.297059+0000","last_became_peered":"2024-01-11T07:09:08.297059+0000","last_unstale":"2024-01-11T07:09:09.326730+0000","last_undegraded":"2024-01-11T07:09:09.326730+0000","last_fullsized":"2024-01-11T07:09:09.326730+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:09:01.014913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326696+0000","last_change":"2024-01-11T07:09:09.326623+0000","last_active":"2024-01-11T07:09:09.326696+0000","last_peered":"2024-01-11T07:09:09.326696+0000","last_clean":"2024-01-11T07:09:09.326696+0000","last_became_active":"2024-01-11T07:09:09.326383+0000","last_became_peered":"2024-01-11T07:09:09.326383+0000","last_unstale":"2024-01-11T07:09:09.326696+0000","last_undegraded":"2024-01-11T07:09:09.326696+0000","last_fullsized":"2024-01-11T07:09:09.326696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:05:58.298969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.324021+0000","last_change":"2024-01-11T07:09:08.296590+0000","last_active":"2024-01-11T07:09:09.324021+0000","last_peered":"2024-01-11T07:09:09.324021+0000","last_clean":"2024-01-11T07:09:09.324021+0000","last_became_active":"2024-01-11T07:09:08.296279+0000","last_became_peered":"2024-01-11T07:09:08.296279+0000","last_unstale":"2024-01-11T07:09:09.324021+0000","last_undegraded":"2024-01-11T07:09:09.324021+0000","last_fullsized":"2024-01-11T07:09:09.324021+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:09:52.054331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.321985+0000","last_change":"2024-01-11T07:09:09.319886+0000","last_active":"2024-01-11T07:09:09.321985+0000","last_peered":"2024-01-11T07:09:09.321985+0000","last_clean":"2024-01-11T07:09:09.321985+0000","last_became_active":"2024-01-11T07:09:09.319605+0000","last_became_peered":"2024-01-11T07:09:09.319605+0000","last_unstale":"2024-01-11T07:09:09.321985+0000","last_undegraded":"2024-01-11T07:09:09.321985+0000","last_fullsized":"2024-01-11T07:09:09.321985+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:54:47.107951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.325263+0000","last_change":"2024-01-11T07:09:08.292606+0000","last_active":"2024-01-11T07:09:09.325263+0000","last_peered":"2024-01-11T07:09:09.325263+0000","last_clean":"2024-01-11T07:09:09.325263+0000","last_became_active":"2024-01-11T07:09:08.292359+0000","last_became_peered":"2024-01-11T07:09:08.292359+0000","last_unstale":"2024-01-11T07:09:09.325263+0000","last_undegraded":"2024-01-11T07:09:09.325263+0000","last_fullsized":"2024-01-11T07:09:09.325263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:09:52.054331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.327055+0000","last_change":"2024-01-11T07:09:09.326260+0000","last_active":"2024-01-11T07:09:09.327055+0000","last_peered":"2024-01-11T07:09:09.327055+0000","last_clean":"2024-01-11T07:09:09.327055+0000","last_became_active":"2024-01-11T07:09:09.326029+0000","last_became_peered":"2024-01-11T07:09:09.326029+0000","last_unstale":"2024-01-11T07:09:09.327055+0000","last_undegraded":"2024-01-11T07:09:09.327055+0000","last_fullsized":"2024-01-11T07:09:09.327055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:35:40.843768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.319917+0000","last_change":"2024-01-11T07:09:08.291550+0000","last_active":"2024-01-11T07:09:09.319917+0000","last_peered":"2024-01-11T07:09:09.319917+0000","last_clean":"2024-01-11T07:09:09.319917+0000","last_became_active":"2024-01-11T07:09:08.291318+0000","last_became_peered":"2024-01-11T07:09:08.291318+0000","last_unstale":"2024-01-11T07:09:09.319917+0000","last_undegraded":"2024-01-11T07:09:09.319917+0000","last_fullsized":"2024-01-11T07:09:09.319917+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:33:14.250241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.332655+0000","last_change":"2024-01-11T07:09:09.332570+0000","last_active":"2024-01-11T07:09:09.332655+0000","last_peered":"2024-01-11T07:09:09.332655+0000","last_clean":"2024-01-11T07:09:09.332655+0000","last_became_active":"2024-01-11T07:09:09.331367+0000","last_became_peered":"2024-01-11T07:09:09.331367+0000","last_unstale":"2024-01-11T07:09:09.332655+0000","last_undegraded":"2024-01-11T07:09:09.332655+0000","last_fullsized":"2024-01-11T07:09:09.332655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:20:03.424637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.328586+0000","last_change":"2024-01-11T07:09:08.296464+0000","last_active":"2024-01-11T07:09:09.328586+0000","last_peered":"2024-01-11T07:09:09.328586+0000","last_clean":"2024-01-11T07:09:09.328586+0000","last_became_active":"2024-01-11T07:09:08.294350+0000","last_became_peered":"2024-01-11T07:09:08.294350+0000","last_unstale":"2024-01-11T07:09:09.328586+0000","last_undegraded":"2024-01-11T07:09:09.328586+0000","last_fullsized":"2024-01-11T07:09:09.328586+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:45:47.054314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.311559+0000","last_change":"2024-01-11T07:09:09.311444+0000","last_active":"2024-01-11T07:09:09.311559+0000","last_peered":"2024-01-11T07:09:09.311559+0000","last_clean":"2024-01-11T07:09:09.311559+0000","last_became_active":"2024-01-11T07:09:09.311210+0000","last_became_peered":"2024-01-11T07:09:09.311210+0000","last_unstale":"2024-01-11T07:09:09.311559+0000","last_undegraded":"2024-01-11T07:09:09.311559+0000","last_fullsized":"2024-01-11T07:09:09.311559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:55:40.581185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.316605+0000","last_change":"2024-01-11T07:09:08.287875+0000","last_active":"2024-01-11T07:09:09.316605+0000","last_peered":"2024-01-11T07:09:09.316605+0000","last_clean":"2024-01-11T07:09:09.316605+0000","last_became_active":"2024-01-11T07:09:08.287278+0000","last_became_peered":"2024-01-11T07:09:08.287278+0000","last_unstale":"2024-01-11T07:09:09.316605+0000","last_undegraded":"2024-01-11T07:09:09.316605+0000","last_fullsized":"2024-01-11T07:09:09.316605+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:33:54.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326990+0000","last_change":"2024-01-11T07:09:09.325088+0000","last_active":"2024-01-11T07:09:09.326990+0000","last_peered":"2024-01-11T07:09:09.326990+0000","last_clean":"2024-01-11T07:09:09.326990+0000","last_became_active":"2024-01-11T07:09:09.318204+0000","last_became_peered":"2024-01-11T07:09:09.318204+0000","last_unstale":"2024-01-11T07:09:09.326990+0000","last_undegraded":"2024-01-11T07:09:09.326990+0000","last_fullsized":"2024-01-11T07:09:09.326990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:09:01.951552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.314262+0000","last_change":"2024-01-11T07:09:08.297633+0000","last_active":"2024-01-11T07:09:09.314262+0000","last_peered":"2024-01-11T07:09:09.314262+0000","last_clean":"2024-01-11T07:09:09.314262+0000","last_became_active":"2024-01-11T07:09:08.297375+0000","last_became_peered":"2024-01-11T07:09:08.297375+0000","last_unstale":"2024-01-11T07:09:09.314262+0000","last_undegraded":"2024-01-11T07:09:09.314262+0000","last_fullsized":"2024-01-11T07:09:09.314262+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:27.924756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.325123+0000","last_change":"2024-01-11T07:09:09.325051+0000","last_active":"2024-01-11T07:09:09.325123+0000","last_peered":"2024-01-11T07:09:09.325123+0000","last_clean":"2024-01-11T07:09:09.325123+0000","last_became_active":"2024-01-11T07:09:09.324812+0000","last_became_peered":"2024-01-11T07:09:09.324812+0000","last_unstale":"2024-01-11T07:09:09.325123+0000","last_undegraded":"2024-01-11T07:09:09.325123+0000","last_fullsized":"2024-01-11T07:09:09.325123+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:14:34.579163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.311593+0000","last_change":"2024-01-11T07:09:08.296186+0000","last_active":"2024-01-11T07:09:09.311593+0000","last_peered":"2024-01-11T07:09:09.311593+0000","last_clean":"2024-01-11T07:09:09.311593+0000","last_became_active":"2024-01-11T07:09:08.296017+0000","last_became_peered":"2024-01-11T07:09:08.296017+0000","last_unstale":"2024-01-11T07:09:09.311593+0000","last_undegraded":"2024-01-11T07:09:09.311593+0000","last_fullsized":"2024-01-11T07:09:09.311593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:33:54.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326532+0000","last_change":"2024-01-11T07:09:09.326464+0000","last_active":"2024-01-11T07:09:09.326532+0000","last_peered":"2024-01-11T07:09:09.326532+0000","last_clean":"2024-01-11T07:09:09.326532+0000","last_became_active":"2024-01-11T07:09:09.325914+0000","last_became_peered":"2024-01-11T07:09:09.325914+0000","last_unstale":"2024-01-11T07:09:09.326532+0000","last_undegraded":"2024-01-11T07:09:09.326532+0000","last_fullsized":"2024-01-11T07:09:09.326532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:58:01.237181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.327866+0000","last_change":"2024-01-11T07:09:08.301111+0000","last_active":"2024-01-11T07:09:09.327866+0000","last_peered":"2024-01-11T07:09:09.327866+0000","last_clean":"2024-01-11T07:09:09.327866+0000","last_became_active":"2024-01-11T07:09:08.300126+0000","last_became_peered":"2024-01-11T07:09:08.300126+0000","last_unstale":"2024-01-11T07:09:09.327866+0000","last_undegraded":"2024-01-11T07:09:09.327866+0000","last_fullsized":"2024-01-11T07:09:09.327866+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:14:15.260263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.323342+0000","last_change":"2024-01-11T07:09:09.323281+0000","last_active":"2024-01-11T07:09:09.323342+0000","last_peered":"2024-01-11T07:09:09.323342+0000","last_clean":"2024-01-11T07:09:09.323342+0000","last_became_active":"2024-01-11T07:09:09.322832+0000","last_became_peered":"2024-01-11T07:09:09.322832+0000","last_unstale":"2024-01-11T07:09:09.323342+0000","last_undegraded":"2024-01-11T07:09:09.323342+0000","last_fullsized":"2024-01-11T07:09:09.323342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:28.926828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.324110+0000","last_change":"2024-01-11T07:09:08.297768+0000","last_active":"2024-01-11T07:09:09.324110+0000","last_peered":"2024-01-11T07:09:09.324110+0000","last_clean":"2024-01-11T07:09:09.324110+0000","last_became_active":"2024-01-11T07:09:08.297544+0000","last_became_peered":"2024-01-11T07:09:08.297544+0000","last_unstale":"2024-01-11T07:09:09.324110+0000","last_undegraded":"2024-01-11T07:09:09.324110+0000","last_fullsized":"2024-01-11T07:09:09.324110+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:40:38.246668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.323617+0000","last_change":"2024-01-11T07:09:09.323544+0000","last_active":"2024-01-11T07:09:09.323617+0000","last_peered":"2024-01-11T07:09:09.323617+0000","last_clean":"2024-01-11T07:09:09.323617+0000","last_became_active":"2024-01-11T07:09:09.323167+0000","last_became_peered":"2024-01-11T07:09:09.323167+0000","last_unstale":"2024-01-11T07:09:09.323617+0000","last_undegraded":"2024-01-11T07:09:09.323617+0000","last_fullsized":"2024-01-11T07:09:09.323617+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:59:52.925015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.328334+0000","last_change":"2024-01-11T07:09:08.286837+0000","last_active":"2024-01-11T07:09:09.328334+0000","last_peered":"2024-01-11T07:09:09.328334+0000","last_clean":"2024-01-11T07:09:09.328334+0000","last_became_active":"2024-01-11T07:09:08.286549+0000","last_became_peered":"2024-01-11T07:09:08.286549+0000","last_unstale":"2024-01-11T07:09:09.328334+0000","last_undegraded":"2024-01-11T07:09:09.328334+0000","last_fullsized":"2024-01-11T07:09:09.328334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:54:02.650387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326715+0000","last_change":"2024-01-11T07:09:09.326578+0000","last_active":"2024-01-11T07:09:09.326715+0000","last_peered":"2024-01-11T07:09:09.326715+0000","last_clean":"2024-01-11T07:09:09.326715+0000","last_became_active":"2024-01-11T07:09:09.326390+0000","last_became_peered":"2024-01-11T07:09:09.326390+0000","last_unstale":"2024-01-11T07:09:09.326715+0000","last_undegraded":"2024-01-11T07:09:09.326715+0000","last_fullsized":"2024-01-11T07:09:09.326715+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:52.445820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326945+0000","last_change":"2024-01-11T07:09:08.296154+0000","last_active":"2024-01-11T07:09:09.326945+0000","last_peered":"2024-01-11T07:09:09.326945+0000","last_clean":"2024-01-11T07:09:09.326945+0000","last_became_active":"2024-01-11T07:09:08.295921+0000","last_became_peered":"2024-01-11T07:09:08.295921+0000","last_unstale":"2024-01-11T07:09:09.326945+0000","last_undegraded":"2024-01-11T07:09:09.326945+0000","last_fullsized":"2024-01-11T07:09:09.326945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:36:55.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.311686+0000","last_change":"2024-01-11T07:09:09.311480+0000","last_active":"2024-01-11T07:09:09.311686+0000","last_peered":"2024-01-11T07:09:09.311686+0000","last_clean":"2024-01-11T07:09:09.311686+0000","last_became_active":"2024-01-11T07:09:09.311253+0000","last_became_peered":"2024-01-11T07:09:09.311253+0000","last_unstale":"2024-01-11T07:09:09.311686+0000","last_undegraded":"2024-01-11T07:09:09.311686+0000","last_fullsized":"2024-01-11T07:09:09.311686+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:09:02.016984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.309751+0000","last_change":"2024-01-11T07:09:08.293054+0000","last_active":"2024-01-11T07:09:09.309751+0000","last_peered":"2024-01-11T07:09:09.309751+0000","last_clean":"2024-01-11T07:09:09.309751+0000","last_became_active":"2024-01-11T07:09:08.292805+0000","last_became_peered":"2024-01-11T07:09:08.292805+0000","last_unstale":"2024-01-11T07:09:09.309751+0000","last_undegraded":"2024-01-11T07:09:09.309751+0000","last_fullsized":"2024-01-11T07:09:09.309751+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:32:13.616888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.315950+0000","last_change":"2024-01-11T07:09:09.315754+0000","last_active":"2024-01-11T07:09:09.315950+0000","last_peered":"2024-01-11T07:09:09.315950+0000","last_clean":"2024-01-11T07:09:09.315950+0000","last_became_active":"2024-01-11T07:09:09.315286+0000","last_became_peered":"2024-01-11T07:09:09.315286+0000","last_unstale":"2024-01-11T07:09:09.315950+0000","last_undegraded":"2024-01-11T07:09:09.315950+0000","last_fullsized":"2024-01-11T07:09:09.315950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:58:49.957640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.312729+0000","last_change":"2024-01-11T07:09:08.294439+0000","last_active":"2024-01-11T07:09:09.312729+0000","last_peered":"2024-01-11T07:09:09.312729+0000","last_clean":"2024-01-11T07:09:09.312729+0000","last_became_active":"2024-01-11T07:09:08.294268+0000","last_became_peered":"2024-01-11T07:09:08.294268+0000","last_unstale":"2024-01-11T07:09:09.312729+0000","last_undegraded":"2024-01-11T07:09:09.312729+0000","last_fullsized":"2024-01-11T07:09:09.312729+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:40:38.246668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.333550+0000","last_change":"2024-01-11T07:09:09.333507+0000","last_active":"2024-01-11T07:09:09.333550+0000","last_peered":"2024-01-11T07:09:09.333550+0000","last_clean":"2024-01-11T07:09:09.333550+0000","last_became_active":"2024-01-11T07:09:09.333364+0000","last_became_peered":"2024-01-11T07:09:09.333364+0000","last_unstale":"2024-01-11T07:09:09.333550+0000","last_undegraded":"2024-01-11T07:09:09.333550+0000","last_fullsized":"2024-01-11T07:09:09.333550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:52:35.878435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.322791+0000","last_change":"2024-01-11T07:09:08.294974+0000","last_active":"2024-01-11T07:09:09.322791+0000","last_peered":"2024-01-11T07:09:09.322791+0000","last_clean":"2024-01-11T07:09:09.322791+0000","last_became_active":"2024-01-11T07:09:08.293520+0000","last_became_peered":"2024-01-11T07:09:08.293520+0000","last_unstale":"2024-01-11T07:09:09.322791+0000","last_undegraded":"2024-01-11T07:09:09.322791+0000","last_fullsized":"2024-01-11T07:09:09.322791+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:17.642140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.315891+0000","last_change":"2024-01-11T07:09:09.315552+0000","last_active":"2024-01-11T07:09:09.315891+0000","last_peered":"2024-01-11T07:09:09.315891+0000","last_clean":"2024-01-11T07:09:09.315891+0000","last_became_active":"2024-01-11T07:09:09.314952+0000","last_became_peered":"2024-01-11T07:09:09.314952+0000","last_unstale":"2024-01-11T07:09:09.315891+0000","last_undegraded":"2024-01-11T07:09:09.315891+0000","last_fullsized":"2024-01-11T07:09:09.315891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:06.681281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326676+0000","last_change":"2024-01-11T07:09:08.300909+0000","last_active":"2024-01-11T07:09:09.326676+0000","last_peered":"2024-01-11T07:09:09.326676+0000","last_clean":"2024-01-11T07:09:09.326676+0000","last_became_active":"2024-01-11T07:09:08.300672+0000","last_became_peered":"2024-01-11T07:09:08.300672+0000","last_unstale":"2024-01-11T07:09:09.326676+0000","last_undegraded":"2024-01-11T07:09:09.326676+0000","last_fullsized":"2024-01-11T07:09:09.326676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:51.443749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.324230+0000","last_change":"2024-01-11T07:09:09.324140+0000","last_active":"2024-01-11T07:09:09.324230+0000","last_peered":"2024-01-11T07:09:09.324230+0000","last_clean":"2024-01-11T07:09:09.324230+0000","last_became_active":"2024-01-11T07:09:09.323581+0000","last_became_peered":"2024-01-11T07:09:09.323581+0000","last_unstale":"2024-01-11T07:09:09.324230+0000","last_undegraded":"2024-01-11T07:09:09.324230+0000","last_fullsized":"2024-01-11T07:09:09.324230+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:09:02.016984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.324232+0000","last_change":"2024-01-11T07:09:08.294135+0000","last_active":"2024-01-11T07:09:09.324232+0000","last_peered":"2024-01-11T07:09:09.324232+0000","last_clean":"2024-01-11T07:09:09.324232+0000","last_became_active":"2024-01-11T07:09:08.293857+0000","last_became_peered":"2024-01-11T07:09:08.293857+0000","last_unstale":"2024-01-11T07:09:09.324232+0000","last_undegraded":"2024-01-11T07:09:09.324232+0000","last_fullsized":"2024-01-11T07:09:09.324232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:07:35.400318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.332183+0000","last_change":"2024-01-11T07:09:09.332124+0000","last_active":"2024-01-11T07:09:09.332183+0000","last_peered":"2024-01-11T07:09:09.332183+0000","last_clean":"2024-01-11T07:09:09.332183+0000","last_became_active":"2024-01-11T07:09:09.330039+0000","last_became_peered":"2024-01-11T07:09:09.330039+0000","last_unstale":"2024-01-11T07:09:09.332183+0000","last_undegraded":"2024-01-11T07:09:09.332183+0000","last_fullsized":"2024-01-11T07:09:09.332183+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:54:55.118561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.325373+0000","last_change":"2024-01-11T07:09:08.294253+0000","last_active":"2024-01-11T07:09:09.325373+0000","last_peered":"2024-01-11T07:09:09.325373+0000","last_clean":"2024-01-11T07:09:09.325373+0000","last_became_active":"2024-01-11T07:09:08.294023+0000","last_became_peered":"2024-01-11T07:09:08.294023+0000","last_unstale":"2024-01-11T07:09:09.325373+0000","last_undegraded":"2024-01-11T07:09:09.325373+0000","last_fullsized":"2024-01-11T07:09:09.325373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:45:47.054314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326737+0000","last_change":"2024-01-11T07:09:09.326612+0000","last_active":"2024-01-11T07:09:09.326737+0000","last_peered":"2024-01-11T07:09:09.326737+0000","last_clean":"2024-01-11T07:09:09.326737+0000","last_became_active":"2024-01-11T07:09:09.326232+0000","last_became_peered":"2024-01-11T07:09:09.326232+0000","last_unstale":"2024-01-11T07:09:09.326737+0000","last_undegraded":"2024-01-11T07:09:09.326737+0000","last_fullsized":"2024-01-11T07:09:09.326737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:57.551943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.306008+0000","last_change":"2024-01-11T07:09:08.299652+0000","last_active":"2024-01-11T07:09:09.306008+0000","last_peered":"2024-01-11T07:09:09.306008+0000","last_clean":"2024-01-11T07:09:09.306008+0000","last_became_active":"2024-01-11T07:09:08.299406+0000","last_became_peered":"2024-01-11T07:09:08.299406+0000","last_unstale":"2024-01-11T07:09:09.306008+0000","last_undegraded":"2024-01-11T07:09:09.306008+0000","last_fullsized":"2024-01-11T07:09:09.306008+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:17.642140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.324346+0000","last_change":"2024-01-11T07:09:09.324284+0000","last_active":"2024-01-11T07:09:09.324346+0000","last_peered":"2024-01-11T07:09:09.324346+0000","last_clean":"2024-01-11T07:09:09.324346+0000","last_became_active":"2024-01-11T07:09:09.323905+0000","last_became_peered":"2024-01-11T07:09:09.323905+0000","last_unstale":"2024-01-11T07:09:09.324346+0000","last_undegraded":"2024-01-11T07:09:09.324346+0000","last_fullsized":"2024-01-11T07:09:09.324346+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:31:42.912510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326585+0000","last_change":"2024-01-11T07:09:08.293907+0000","last_active":"2024-01-11T07:09:09.326585+0000","last_peered":"2024-01-11T07:09:09.326585+0000","last_clean":"2024-01-11T07:09:09.326585+0000","last_became_active":"2024-01-11T07:09:08.293649+0000","last_became_peered":"2024-01-11T07:09:08.293649+0000","last_unstale":"2024-01-11T07:09:09.326585+0000","last_undegraded":"2024-01-11T07:09:09.326585+0000","last_fullsized":"2024-01-11T07:09:09.326585+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:28.288196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.310828+0000","last_change":"2024-01-11T07:09:09.310529+0000","last_active":"2024-01-11T07:09:09.310828+0000","last_peered":"2024-01-11T07:09:09.310828+0000","last_clean":"2024-01-11T07:09:09.310828+0000","last_became_active":"2024-01-11T07:09:09.310242+0000","last_became_peered":"2024-01-11T07:09:09.310242+0000","last_unstale":"2024-01-11T07:09:09.310828+0000","last_undegraded":"2024-01-11T07:09:09.310828+0000","last_fullsized":"2024-01-11T07:09:09.310828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:52.445820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.319694+0000","last_change":"2024-01-11T07:09:08.294615+0000","last_active":"2024-01-11T07:09:09.319694+0000","last_peered":"2024-01-11T07:09:09.319694+0000","last_clean":"2024-01-11T07:09:09.319694+0000","last_became_active":"2024-01-11T07:09:08.292997+0000","last_became_peered":"2024-01-11T07:09:08.292997+0000","last_unstale":"2024-01-11T07:09:09.319694+0000","last_undegraded":"2024-01-11T07:09:09.319694+0000","last_fullsized":"2024-01-11T07:09:09.319694+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:16:39.379434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.311332+0000","last_change":"2024-01-11T07:09:09.311266+0000","last_active":"2024-01-11T07:09:09.311332+0000","last_peered":"2024-01-11T07:09:09.311332+0000","last_clean":"2024-01-11T07:09:09.311332+0000","last_became_active":"2024-01-11T07:09:09.311006+0000","last_became_peered":"2024-01-11T07:09:09.311006+0000","last_unstale":"2024-01-11T07:09:09.311332+0000","last_undegraded":"2024-01-11T07:09:09.311332+0000","last_fullsized":"2024-01-11T07:09:09.311332+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:09:02.016984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.317278+0000","last_change":"2024-01-11T07:09:08.297120+0000","last_active":"2024-01-11T07:09:09.317278+0000","last_peered":"2024-01-11T07:09:09.317278+0000","last_clean":"2024-01-11T07:09:09.317278+0000","last_became_active":"2024-01-11T07:09:08.296916+0000","last_became_peered":"2024-01-11T07:09:08.296916+0000","last_unstale":"2024-01-11T07:09:09.317278+0000","last_undegraded":"2024-01-11T07:09:09.317278+0000","last_fullsized":"2024-01-11T07:09:09.317278+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:17.642140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.322153+0000","last_change":"2024-01-11T07:09:09.322001+0000","last_active":"2024-01-11T07:09:09.322153+0000","last_peered":"2024-01-11T07:09:09.322153+0000","last_clean":"2024-01-11T07:09:09.322153+0000","last_became_active":"2024-01-11T07:09:09.321693+0000","last_became_peered":"2024-01-11T07:09:09.321693+0000","last_unstale":"2024-01-11T07:09:09.322153+0000","last_undegraded":"2024-01-11T07:09:09.322153+0000","last_fullsized":"2024-01-11T07:09:09.322153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:29.290268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.333272+0000","last_change":"2024-01-11T07:09:08.300670+0000","last_active":"2024-01-11T07:09:09.333272+0000","last_peered":"2024-01-11T07:09:09.333272+0000","last_clean":"2024-01-11T07:09:09.333272+0000","last_became_active":"2024-01-11T07:09:08.300418+0000","last_became_peered":"2024-01-11T07:09:08.300418+0000","last_unstale":"2024-01-11T07:09:09.333272+0000","last_undegraded":"2024-01-11T07:09:09.333272+0000","last_fullsized":"2024-01-11T07:09:09.333272+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:55:39.579113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.317423+0000","last_change":"2024-01-11T07:09:09.317228+0000","last_active":"2024-01-11T07:09:09.317423+0000","last_peered":"2024-01-11T07:09:09.317423+0000","last_clean":"2024-01-11T07:09:09.317423+0000","last_became_active":"2024-01-11T07:09:09.316705+0000","last_became_peered":"2024-01-11T07:09:09.316705+0000","last_unstale":"2024-01-11T07:09:09.317423+0000","last_undegraded":"2024-01-11T07:09:09.317423+0000","last_fullsized":"2024-01-11T07:09:09.317423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:59.669567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-01-11T07:09:10.306625+0000","last_change":"2024-01-11T07:09:08.296273+0000","last_active":"2024-01-11T07:09:10.306625+0000","last_peered":"2024-01-11T07:09:10.306625+0000","last_clean":"2024-01-11T07:09:10.306625+0000","last_became_active":"2024-01-11T07:09:08.293753+0000","last_became_peered":"2024-01-11T07:09:08.293753+0000","last_unstale":"2024-01-11T07:09:10.306625+0000","last_undegraded":"2024-01-11T07:09:10.306625+0000","last_fullsized":"2024-01-11T07:09:10.306625+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:33:54.861107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.325043+0000","last_change":"2024-01-11T07:09:09.324807+0000","last_active":"2024-01-11T07:09:09.325043+0000","last_peered":"2024-01-11T07:09:09.325043+0000","last_clean":"2024-01-11T07:09:09.325043+0000","last_became_active":"2024-01-11T07:09:09.324224+0000","last_became_peered":"2024-01-11T07:09:09.324224+0000","last_unstale":"2024-01-11T07:09:09.325043+0000","last_undegraded":"2024-01-11T07:09:09.325043+0000","last_fullsized":"2024-01-11T07:09:09.325043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:54:03.652459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.309456+0000","last_change":"2024-01-11T07:09:08.293675+0000","last_active":"2024-01-11T07:09:09.309456+0000","last_peered":"2024-01-11T07:09:09.309456+0000","last_clean":"2024-01-11T07:09:09.309456+0000","last_became_active":"2024-01-11T07:09:08.293449+0000","last_became_peered":"2024-01-11T07:09:08.293449+0000","last_unstale":"2024-01-11T07:09:09.309456+0000","last_undegraded":"2024-01-11T07:09:09.309456+0000","last_fullsized":"2024-01-11T07:09:09.309456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:43.682968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.321947+0000","last_change":"2024-01-11T07:09:09.319103+0000","last_active":"2024-01-11T07:09:09.321947+0000","last_peered":"2024-01-11T07:09:09.321947+0000","last_clean":"2024-01-11T07:09:09.321947+0000","last_became_active":"2024-01-11T07:09:09.318854+0000","last_became_peered":"2024-01-11T07:09:09.318854+0000","last_unstale":"2024-01-11T07:09:09.321947+0000","last_undegraded":"2024-01-11T07:09:09.321947+0000","last_fullsized":"2024-01-11T07:09:09.321947+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:35:47.171584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.315489+0000","last_change":"2024-01-11T07:09:08.288814+0000","last_active":"2024-01-11T07:09:09.315489+0000","last_peered":"2024-01-11T07:09:09.315489+0000","last_clean":"2024-01-11T07:09:09.315489+0000","last_became_active":"2024-01-11T07:09:08.288593+0000","last_became_peered":"2024-01-11T07:09:08.288593+0000","last_unstale":"2024-01-11T07:09:09.315489+0000","last_undegraded":"2024-01-11T07:09:09.315489+0000","last_fullsized":"2024-01-11T07:09:09.315489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:03:52.919770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.332475+0000","last_change":"2024-01-11T07:09:09.332429+0000","last_active":"2024-01-11T07:09:09.332475+0000","last_peered":"2024-01-11T07:09:09.332475+0000","last_clean":"2024-01-11T07:09:09.332475+0000","last_became_active":"2024-01-11T07:09:09.331045+0000","last_became_peered":"2024-01-11T07:09:09.331045+0000","last_unstale":"2024-01-11T07:09:09.332475+0000","last_undegraded":"2024-01-11T07:09:09.332475+0000","last_fullsized":"2024-01-11T07:09:09.332475+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:12:39.270967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.322251+0000","last_change":"2024-01-11T07:09:08.294795+0000","last_active":"2024-01-11T07:09:09.322251+0000","last_peered":"2024-01-11T07:09:09.322251+0000","last_clean":"2024-01-11T07:09:09.322251+0000","last_became_active":"2024-01-11T07:09:08.293581+0000","last_became_peered":"2024-01-11T07:09:08.293581+0000","last_unstale":"2024-01-11T07:09:09.322251+0000","last_undegraded":"2024-01-11T07:09:09.322251+0000","last_fullsized":"2024-01-11T07:09:09.322251+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:53:57.400327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.327867+0000","last_change":"2024-01-11T07:09:09.327807+0000","last_active":"2024-01-11T07:09:09.327867+0000","last_peered":"2024-01-11T07:09:09.327867+0000","last_clean":"2024-01-11T07:09:09.327867+0000","last_became_active":"2024-01-11T07:09:09.327569+0000","last_became_peered":"2024-01-11T07:09:09.327569+0000","last_unstale":"2024-01-11T07:09:09.327867+0000","last_undegraded":"2024-01-11T07:09:09.327867+0000","last_fullsized":"2024-01-11T07:09:09.327867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:18:15.830741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.313400+0000","last_change":"2024-01-11T07:09:08.287822+0000","last_active":"2024-01-11T07:09:09.313400+0000","last_peered":"2024-01-11T07:09:09.313400+0000","last_clean":"2024-01-11T07:09:09.313400+0000","last_became_active":"2024-01-11T07:09:08.287592+0000","last_became_peered":"2024-01-11T07:09:08.287592+0000","last_unstale":"2024-01-11T07:09:09.313400+0000","last_undegraded":"2024-01-11T07:09:09.313400+0000","last_fullsized":"2024-01-11T07:09:09.313400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:07:35.400318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.323216+0000","last_change":"2024-01-11T07:09:09.323136+0000","last_active":"2024-01-11T07:09:09.323216+0000","last_peered":"2024-01-11T07:09:09.323216+0000","last_clean":"2024-01-11T07:09:09.323216+0000","last_became_active":"2024-01-11T07:09:09.322564+0000","last_became_peered":"2024-01-11T07:09:09.322564+0000","last_unstale":"2024-01-11T07:09:09.323216+0000","last_undegraded":"2024-01-11T07:09:09.323216+0000","last_fullsized":"2024-01-11T07:09:09.323216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:20:24.440561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.318915+0000","last_change":"2024-01-11T07:09:08.298120+0000","last_active":"2024-01-11T07:09:09.318915+0000","last_peered":"2024-01-11T07:09:09.318915+0000","last_clean":"2024-01-11T07:09:09.318915+0000","last_became_active":"2024-01-11T07:09:08.297739+0000","last_became_peered":"2024-01-11T07:09:08.297739+0000","last_unstale":"2024-01-11T07:09:09.318915+0000","last_undegraded":"2024-01-11T07:09:09.318915+0000","last_fullsized":"2024-01-11T07:09:09.318915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:47:36.368046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.326793+0000","last_change":"2024-01-11T07:09:09.317573+0000","last_active":"2024-01-11T07:09:09.326793+0000","last_peered":"2024-01-11T07:09:09.326793+0000","last_clean":"2024-01-11T07:09:09.326793+0000","last_became_active":"2024-01-11T07:09:09.317307+0000","last_became_peered":"2024-01-11T07:09:09.317307+0000","last_unstale":"2024-01-11T07:09:09.326793+0000","last_undegraded":"2024-01-11T07:09:09.326793+0000","last_fullsized":"2024-01-11T07:09:09.326793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:37:53.704653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.312567+0000","last_change":"2024-01-11T07:09:08.296860+0000","last_active":"2024-01-11T07:09:09.312567+0000","last_peered":"2024-01-11T07:09:09.312567+0000","last_clean":"2024-01-11T07:09:09.312567+0000","last_became_active":"2024-01-11T07:09:08.296386+0000","last_became_peered":"2024-01-11T07:09:08.296386+0000","last_unstale":"2024-01-11T07:09:09.312567+0000","last_undegraded":"2024-01-11T07:09:09.312567+0000","last_fullsized":"2024-01-11T07:09:09.312567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:31:41.910438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.327039+0000","last_change":"2024-01-11T07:09:09.325907+0000","last_active":"2024-01-11T07:09:09.327039+0000","last_peered":"2024-01-11T07:09:09.327039+0000","last_clean":"2024-01-11T07:09:09.327039+0000","last_became_active":"2024-01-11T07:09:09.325633+0000","last_became_peered":"2024-01-11T07:09:09.325633+0000","last_unstale":"2024-01-11T07:09:09.327039+0000","last_undegraded":"2024-01-11T07:09:09.327039+0000","last_fullsized":"2024-01-11T07:09:09.327039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:08.279367+0000","last_clean_scrub_stamp":"2024-01-11T07:09:08.279367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:37.672881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.310080+0000","last_change":"2024-01-11T07:09:08.290931+0000","last_active":"2024-01-11T07:09:09.310080+0000","last_peered":"2024-01-11T07:09:09.310080+0000","last_clean":"2024-01-11T07:09:09.310080+0000","last_became_active":"2024-01-11T07:09:08.290691+0000","last_became_peered":"2024-01-11T07:09:08.290691+0000","last_unstale":"2024-01-11T07:09:09.310080+0000","last_undegraded":"2024-01-11T07:09:09.310080+0000","last_fullsized":"2024-01-11T07:09:09.310080+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:09:07.277296+0000","last_clean_scrub_stamp":"2024-01-11T07:09:07.277296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:44:51.315800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-01-11T07:09:09.311768+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:09:09.311768+0000","last_peered":"2024-01-11T07:09:09.311768+0000","last_clean":"2024-01-11T07:09:09.311768+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:09:09.311768+0000","last_undegraded":"2024-01-11T07:09:09.311768+0000","last_fullsized":"2024-01-11T07:09:09.311768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":893,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084859,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.504},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.40999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.73999999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.5},"min":{"1min":0.433,"5min":0.311,"15min":0.311},"max":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.441,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.432,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.47999999999999998,"15min":0.47399999999999998},"min":{"1min":0.32600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34300000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.91600000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.90300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194995,"statfs":{"total":200000000,"available":199675387,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76500000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70299999999999996,"5min":1.157,"15min":1.157},"last":0.434}]},{"osd":1,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.622}]},{"osd":2,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77200000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.62,"15min":0.622},"min":{"1min":0.42899999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.32100000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.749,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.63200000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.63900000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194995,"statfs":{"total":200000000,"available":199675297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":0.62,"15min":0.48599999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":20.41,"5min":20.41,"15min":20.41},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45200000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.57799999999999996,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.373}]},{"osd":2,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":0.56799999999999995,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":15.611000000000001,"5min":15.611000000000001,"15min":15.611000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":1.482,"5min":0.67900000000000005,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":20.199999999999999,"5min":20.199999999999999,"15min":20.199999999999999},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":0.67400000000000004,"15min":0.54100000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":20.369,"5min":20.369,"15min":20.369},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.435,"15min":0.439},"min":{"1min":0.313,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.749,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.70199999999999996,"15min":0.64200000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":9.9570000000000007,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.226,"5min":0.73999999999999999,"15min":0.65900000000000003},"min":{"1min":0.498,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.379,"5min":0.376,"15min":0.376},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.73099999999999998,"15min":0.67500000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":9.891,"5min":9.891,"15min":9.891},"last":0.48599999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.0820000000000001,"5min":0.77800000000000002,"15min":0.72799999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":9.9900000000000002,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:08:49 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":0.90200000000000002,"15min":0.77600000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":20.324000000000002,"5min":20.324000000000002,"15min":20.324000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.5209999999999999,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":895,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194416,"statfs":{"total":200000000,"available":199082929,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.47399999999999998,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.59699999999999998,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55400000000000005,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.73599999999999999,"15min":0.78900000000000003},"min":{"1min":0.307,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.73199999999999998,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.53100000000000003,"15min":0.55200000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.62,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.75,"15min":0.80700000000000005},"min":{"1min":0.29699999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.57699999999999996,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.57399999999999995,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77700000000000002,"5min":2.855,"15min":2.855},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.71799999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.90600000000000003,"15min":0.95799999999999996},"min":{"1min":0.432,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83299999999999996,"5min":2.819,"15min":2.819},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.85399999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.97099999999999997,"15min":1.0249999999999999},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.879,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.81699999999999995,"15min":0.78800000000000003},"min":{"1min":0.53700000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":7.4690000000000003,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.87,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.96099999999999997,"15min":1.01},"min":{"1min":0.52400000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89500000000000002,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.64100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194995,"statfs":{"total":200000000,"available":199675331,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41999999999999998,"15min":0.42399999999999999},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.624,"5min":1.069,"15min":1.069},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39300000000000002,"15min":0.39400000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.28799999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.443,"15min":0.443},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68700000000000006,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.438},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.28299999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.311}]},{"osd":3,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.48099999999999998},"min":{"1min":0.377,"5min":0.313,"15min":0.313},"max":{"1min":0.68400000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76800000000000002,"5min":1.139,"15min":1.139},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.48799999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71999999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89400000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.878,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70699999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.85799999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65200000000000002,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82799999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.66400000000000003,"15min":0.67000000000000004},"min":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82099999999999995,"5min":1.052,"15min":1.052},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68000000000000005,"15min":0.68799999999999994},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.748,"5min":1.038,"15min":1.038},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194995,"statfs":{"total":200000000,"available":199675331,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.81899999999999995,"15min":0.89100000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54300000000000004,"5min":11.481,"15min":11.481},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.61599999999999999,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":1.026,"15min":1.133},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51900000000000002,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.65000000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42499999999999999,"15min":0.42699999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.54500000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.44},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.0820000000000001,"15min":1.1839999999999999},"min":{"1min":0.33400000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59899999999999998,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88600000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":1.321,"15min":1.4239999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92900000000000005,"5min":17.474,"15min":17.474},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.256,"15min":1.3560000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95299999999999996,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.0449999999999999,"15min":1.111},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":11.394,"15min":11.394},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":1.0609999999999999,"15min":1.129},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85799999999999998,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.873,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.288,"15min":1.3859999999999999},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97099999999999997,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.60899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194993,"statfs":{"total":200000000,"available":199673851,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":1.0780000000000001,"15min":1.1559999999999999},"min":{"1min":0.40799999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.78200000000000003,"5min":8.484,"15min":8.484},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.139,"15min":1.228},"min":{"1min":0.42799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79900000000000004,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.88800000000000001,"15min":0.93799999999999994},"min":{"1min":0.437,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81299999999999994,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":1.0329999999999999,"15min":1.1100000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81399999999999995,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.98899999999999999,"15min":1.054},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85899999999999999,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.90000000000000002,"15min":0.95699999999999996},"min":{"1min":0.379,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72199999999999998,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.0860000000000001,"15min":1.173},"min":{"1min":0.39000000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74399999999999999,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":1.1479999999999999,"15min":1.24},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79300000000000004,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.98199999999999998,"15min":1.0720000000000001},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.89100000000000001,"15min":0.95699999999999996},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67000000000000004,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.95099999999999996,"15min":1.0169999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82599999999999996,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71899999999999997,"5min":1.119,"15min":1.119},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 11 07:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.91600000000000004,"15min":0.98399999999999999},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66400000000000003,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.628,"5min":1.024,"15min":1.0900000000000001},"min":{"1min":0.47299999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.73099999999999998,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.71599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194994,"statfs":{"total":200000000,"available":199674763,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82699999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80000000000000004,"5min":0.87,"15min":0.87},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76600000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60799999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81599999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.874,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84199999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.60099999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.505},"min":{"1min":0.40100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53600000000000003,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.77500000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:08:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82499999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71799999999999997}]}]}],"pool_statfs":[]}} 2024-01-11T07:09:13.917 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:09:13.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:09:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.094+0000 7fc3ba521700 1 Processor -- start 2024-01-11T07:09:14.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.094+0000 7fc3ba521700 1 -- start start 2024-01-11T07:09:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.095+0000 7fc3ba521700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3b4156760 0x7fc3b406e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.095+0000 7fc3ba521700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3b4069c80 0x7fc3b406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.095+0000 7fc3ba521700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b406e3f0 con 0x7fc3b415b260 2024-01-11T07:09:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.095+0000 7fc3ba521700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b40b1d20 con 0x7fc3b4069c80 2024-01-11T07:09:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.095+0000 7fc3ba521700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b406a660 con 0x7fc3b4156760 2024-01-11T07:09:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.095+0000 7fc3b8abe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3b4069c80 0x7fc3b406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.095+0000 7fc3b8abe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3b4069c80 0x7fc3b406a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47172/0 (socket says 172.21.15.7:47172) 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.095+0000 7fc3b8abe700 1 -- 172.21.15.7:0/963652804 learned_addr learned my addr 172.21.15.7:0/963652804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b3fff700 1 --2- 172.21.15.7:0/963652804 >> v2:172.21.15.113:3300/0 conn(0x7fc3b4156760 0x7fc3b406e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b8abe700 1 -- 172.21.15.7:0/963652804 >> v2:172.21.15.113:3300/0 conn(0x7fc3b4156760 msgr2=0x7fc3b406e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b8abe700 1 --2- 172.21.15.7:0/963652804 >> v2:172.21.15.113:3300/0 conn(0x7fc3b4156760 0x7fc3b406e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b8abe700 1 -- 172.21.15.7:0/963652804 >> v1:172.21.15.113:6789/0 conn(0x7fc3b415b260 legacy=0x7fc3b40b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b8abe700 1 -- 172.21.15.7:0/963652804 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b4157b40 con 0x7fc3b4069c80 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b8abe700 1 --2- 172.21.15.7:0/963652804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b406a120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc3b406c780 tx=0x7fc3ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14791b72640ddd40 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b2ffd700 1 -- 172.21.15.7:0/963652804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00e070 con 0x7fc3b4069c80 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b2ffd700 1 -- 172.21.15.7:0/963652804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac00ba20 con 0x7fc3b4069c80 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.096+0000 7fc3b2ffd700 1 -- 172.21.15.7:0/963652804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0092e0 con 0x7fc3b4069c80 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.097+0000 7fc3ba521700 1 -- 172.21.15.7:0/963652804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 msgr2=0x7fc3b406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.097+0000 7fc3ba521700 1 --2- 172.21.15.7:0/963652804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b406a120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc3b406c780 tx=0x7fc3ac009840 comp rx=0 tx=0).stop 2024-01-11T07:09:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.097+0000 7fc3ba521700 1 -- 172.21.15.7:0/963652804 shutdown_connections 2024-01-11T07:09:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.097+0000 7fc3ba521700 1 --2- 172.21.15.7:0/963652804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b406a120 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.097+0000 7fc3ba521700 1 --2- 172.21.15.7:0/963652804 >> v2:172.21.15.113:3300/0 conn(0x7fc3b4156760 0x7fc3b406e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.097+0000 7fc3ba521700 1 -- 172.21.15.7:0/963652804 >> 172.21.15.7:0/963652804 conn(0x7fc3b40cd910 msgr2=0x7fc3b40cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.097+0000 7fc3ba521700 1 -- 172.21.15.7:0/963652804 shutdown_connections 2024-01-11T07:09:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.097+0000 7fc3ba521700 1 -- 172.21.15.7:0/963652804 wait complete. 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.098+0000 7fc3ba521700 1 Processor -- start 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.098+0000 7fc3ba521700 1 -- start start 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3ba521700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3ba521700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3b4156760 0x7fc3b40aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3ba521700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3b415b260 0x7fc3b40af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3ba521700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b414e110 con 0x7fc3b4069c80 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3ba521700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b415c550 con 0x7fc3b415b260 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3ba521700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40afae0 con 0x7fc3b4156760 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b415b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47178/0 (socket says 172.21.15.7:47178) 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3b3fff700 1 -- 172.21.15.7:0/3495454877 learned_addr learned my addr 172.21.15.7:0/3495454877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3b8abe700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3b415b260 0x7fc3b40af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.099+0000 7fc3b37fe700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3b4156760 0x7fc3b40aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc3b3fff700 1 -- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3b4156760 msgr2=0x7fc3b40aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc3b3fff700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3b4156760 0x7fc3b40aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc3b3fff700 1 -- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3b415b260 msgr2=0x7fc3b40af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc3b3fff700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3b415b260 0x7fc3b40af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc3b3fff700 1 -- 172.21.15.7:0/3495454877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac009ae0 con 0x7fc3b4069c80 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc3b3fff700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b415b950 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400ec80 tx=0x7fc3a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc39bfff700 1 -- 172.21.15.7:0/3495454877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a400c850 con 0x7fc3b4069c80 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc39bfff700 1 -- 172.21.15.7:0/3495454877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a400c9b0 con 0x7fc3b4069c80 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b40afdc0 con 0x7fc3b4069c80 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc39bfff700 1 -- 172.21.15.7:0/3495454877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a400cd50 con 0x7fc3b4069c80 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.100+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b40b0380 con 0x7fc3b4069c80 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.101+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b414dee0 con 0x7fc3b4069c80 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.101+0000 7fc39bfff700 1 -- 172.21.15.7:0/3495454877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc3a4013680 con 0x7fc3b4069c80 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.102+0000 7fc39bfff700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3a0066f30 0x7fc3a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.102+0000 7fc39bfff700 1 -- 172.21.15.7:0/3495454877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc3a4084760 con 0x7fc3b4069c80 2024-01-11T07:09:14.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.102+0000 7fc3b37fe700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3a0066f30 0x7fc3a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.103+0000 7fc3b37fe700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3a0066f30 0x7fc3a00693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc3b415c950 tx=0x7fc3a8007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.107+0000 7fc39bfff700 1 -- 172.21.15.7:0/3495454877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3b414dee0 con 0x7fc3b4069c80 2024-01-11T07:09:14.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.395+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc3b414dee0 con 0x7fc3b4069c80 2024-01-11T07:09:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.398+0000 7fc39bfff700 1 -- 172.21.15.7:0/3495454877 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc3b414dee0 con 0x7fc3b4069c80 2024-01-11T07:09:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3a0066f30 msgr2=0x7fc3a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3a0066f30 0x7fc3a00693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc3b415c950 tx=0x7fc3a8007ec0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 msgr2=0x7fc3b415b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b415b950 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400ec80 tx=0x7fc3a40091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 shutdown_connections 2024-01-11T07:09:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3a0066f30 0x7fc3a00693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3b415b260 0x7fc3b40af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3b4069c80 0x7fc3b415b950 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 --2- 172.21.15.7:0/3495454877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3b4156760 0x7fc3b40aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.400+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 >> 172.21.15.7:0/3495454877 conn(0x7fc3b40cd910 msgr2=0x7fc3b41580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.401+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 shutdown_connections 2024-01-11T07:09:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.401+0000 7fc3ba521700 1 -- 172.21.15.7:0/3495454877 wait complete. 2024-01-11T07:09:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:09:14.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:09:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.546+0000 7fb7e0396700 1 Processor -- start 2024-01-11T07:09:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.546+0000 7fb7e0396700 1 -- start start 2024-01-11T07:09:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7e0396700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7d80ac580 0x7fb7d806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7e0396700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d806a090 con 0x7fb7d81546b0 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7e0396700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d80acca0 con 0x7fb7d81542e0 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7e0396700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d80acf80 con 0x7fb7d80ac580 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7dd931700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb7d81546b0 0x7fb7d806a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60488/0 (socket says 172.21.15.7:60488) 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7dd931700 1 -- 172.21.15.7:0/4115649054 learned_addr learned my addr 172.21.15.7:0/4115649054 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7de933700 1 --2- 172.21.15.7:0/4115649054 >> v2:172.21.15.113:3300/0 conn(0x7fb7d80ac580 0x7fb7d806bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7dd130700 1 -- 172.21.15.7:0/4115649054 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265285138 0 0) 0x7fb7d806a090 con 0x7fb7d81546b0 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.547+0000 7fb7dd130700 1 -- 172.21.15.7:0/4115649054 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c4003570 con 0x7fb7d81546b0 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.548+0000 7fb7de933700 1 -- 172.21.15.7:0/4115649054 >> v1:172.21.15.113:6789/0 conn(0x7fb7d81542e0 legacy=0x7fb7d80aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.548+0000 7fb7de933700 1 -- 172.21.15.7:0/4115649054 >> v1:172.21.15.7:6789/0 conn(0x7fb7d81546b0 legacy=0x7fb7d806a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.548+0000 7fb7de933700 1 -- 172.21.15.7:0/4115649054 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d8069d20 con 0x7fb7d80ac580 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.548+0000 7fb7de933700 1 --2- 172.21.15.7:0/4115649054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d80ac580 0x7fb7d806bc50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009e60 tx=0x7fb7d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c60057096c07dd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.548+0000 7fb7dd130700 1 -- 172.21.15.7:0/4115649054 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000e070 con 0x7fb7d80ac580 2024-01-11T07:09:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.548+0000 7fb7dd130700 1 -- 172.21.15.7:0/4115649054 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d000bb60 con 0x7fb7d80ac580 2024-01-11T07:09:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.548+0000 7fb7dd130700 1 -- 172.21.15.7:0/4115649054 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0012430 con 0x7fb7d80ac580 2024-01-11T07:09:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.549+0000 7fb7e0396700 1 -- 172.21.15.7:0/4115649054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d80ac580 msgr2=0x7fb7d806bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.549+0000 7fb7e0396700 1 --2- 172.21.15.7:0/4115649054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d80ac580 0x7fb7d806bc50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009e60 tx=0x7fb7d0009780 comp rx=0 tx=0).stop 2024-01-11T07:09:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.549+0000 7fb7e0396700 1 -- 172.21.15.7:0/4115649054 shutdown_connections 2024-01-11T07:09:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.549+0000 7fb7e0396700 1 --2- 172.21.15.7:0/4115649054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d80ac580 0x7fb7d806bc50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.549+0000 7fb7e0396700 1 -- 172.21.15.7:0/4115649054 >> 172.21.15.7:0/4115649054 conn(0x7fb7d80cd910 msgr2=0x7fb7d80cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.549+0000 7fb7e0396700 1 -- 172.21.15.7:0/4115649054 shutdown_connections 2024-01-11T07:09:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.549+0000 7fb7e0396700 1 -- 172.21.15.7:0/4115649054 wait complete. 2024-01-11T07:09:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.550+0000 7fb7e0396700 1 Processor -- start 2024-01-11T07:09:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.550+0000 7fb7e0396700 1 -- start start 2024-01-11T07:09:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7e0396700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7d80ac580 0x7fb7d814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7e0396700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d81542e0 0x7fb7d80cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7e0396700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d81546b0 0x7fb7d80cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7e0396700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80cbb70 con 0x7fb7d81546b0 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7e0396700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80c3490 con 0x7fb7d81542e0 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7e0396700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d8150310 con 0x7fb7d80ac580 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d81546b0 0x7fb7d80cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d81546b0 0x7fb7d80cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47194/0 (socket says 172.21.15.7:47194) 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 -- 172.21.15.7:0/127315611 learned_addr learned my addr 172.21.15.7:0/127315611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7dd931700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d81542e0 0x7fb7d80cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de132700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7d80ac580 0x7fb7d814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 -- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7d80ac580 msgr2=0x7fb7d814fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7d80ac580 0x7fb7d814fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 -- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d81542e0 msgr2=0x7fb7d80cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d81542e0 0x7fb7d80cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 -- 172.21.15.7:0/127315611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d0009a20 con 0x7fb7d81546b0 2024-01-11T07:09:14.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.551+0000 7fb7de933700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d81546b0 0x7fb7d80cc460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000b6c0 tx=0x7fb7d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.552+0000 7fb7ce7fc700 1 -- 172.21.15.7:0/127315611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0027470 con 0x7fb7d81546b0 2024-01-11T07:09:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.552+0000 7fb7ce7fc700 1 -- 172.21.15.7:0/127315611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d00275f0 con 0x7fb7d81546b0 2024-01-11T07:09:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.552+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d80c2880 con 0x7fb7d81546b0 2024-01-11T07:09:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.552+0000 7fb7ce7fc700 1 -- 172.21.15.7:0/127315611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00279b0 con 0x7fb7d81546b0 2024-01-11T07:09:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.552+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d80c2e40 con 0x7fb7d81546b0 2024-01-11T07:09:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.553+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d80c3640 con 0x7fb7d81546b0 2024-01-11T07:09:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.553+0000 7fb7ce7fc700 1 -- 172.21.15.7:0/127315611 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb7d0024070 con 0x7fb7d81546b0 2024-01-11T07:09:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.553+0000 7fb7ce7fc700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb7bc066f30 0x7fb7bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.554+0000 7fb7ce7fc700 1 -- 172.21.15.7:0/127315611 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb7d0086f30 con 0x7fb7d81546b0 2024-01-11T07:09:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.554+0000 7fb7de132700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb7bc066f30 0x7fb7bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.554+0000 7fb7de132700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb7bc066f30 0x7fb7bc0693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4003ac0 tx=0x7fb7d4007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:14.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.558+0000 7fb7ce7fc700 1 -- 172.21.15.7:0/127315611 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7d80c3640 con 0x7fb7d81546b0 2024-01-11T07:09:14.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.807+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7d80c3640 con 0x7fb7d81546b0 2024-01-11T07:09:14.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.808+0000 7fb7ce7fc700 1 -- 172.21.15.7:0/127315611 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fb7d80c3640 con 0x7fb7d81546b0 2024-01-11T07:09:14.808 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:14.808 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1734166534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1734166534},{"type":"v1","addr":"172.21.15.113:6836","nonce":1734166534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3195231529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3195231529},{"type":"v1","addr":"172.21.15.113:6837","nonce":3195231529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/338342475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":338342475},{"type":"v1","addr":"172.21.15.7:6834","nonce":338342475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:09:08.717048+0000","modified":"2024-01-11T07:09:12.324714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1812078530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1812078530},{"type":"v1","addr":"172.21.15.7:6835","nonce":1812078530}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-11T07:09:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb7bc066f30 msgr2=0x7fb7bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb7bc066f30 0x7fb7bc0693e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4003ac0 tx=0x7fb7d4007ea0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d81546b0 msgr2=0x7fb7d80cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d81546b0 0x7fb7d80cc460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000b6c0 tx=0x7fb7d0019000 comp rx=0 tx=0).stop 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 shutdown_connections 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb7bc066f30 0x7fb7bc0693e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7d81546b0 0x7fb7d80cc460 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7d80ac580 0x7fb7d814fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 --2- 172.21.15.7:0/127315611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7d81542e0 0x7fb7d80cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 >> 172.21.15.7:0/127315611 conn(0x7fb7d80cd910 msgr2=0x7fb7d80653b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 shutdown_connections 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:14.810+0000 7fb7e0396700 1 -- 172.21.15.7:0/127315611 wait complete. 2024-01-11T07:09:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-11T07:09:15.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:09:08.717048+0000', 'modified': '2024-01-11T07:09:12.324714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5712': {'gid': 5712, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/1812078530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 1812078530}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 1812078530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:09:15.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:09:15.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-11T07:09:15.276 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-11T07:09:15.277+0000 7f9917c19700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:09:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.371+0000 7fef1574d700 1 Processor -- start 2024-01-11T07:09:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.371+0000 7fef1574d700 1 -- start start 2024-01-11T07:09:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.372+0000 7fef1574d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 0x7fef10063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.372+0000 7fef1574d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef100ab100 con 0x7fef1015a280 2024-01-11T07:09:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.372+0000 7fef1574d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1015d880 con 0x7fef100ac5b0 2024-01-11T07:09:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.372+0000 7fef1574d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1006c5b0 con 0x7fef10063230 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.372+0000 7fef0e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fef1015a280 0x7fef100aa9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60490/0 (socket says 172.21.15.7:60490) 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.372+0000 7fef0e7fc700 1 -- 172.21.15.7:0/2920136103 learned_addr learned my addr 172.21.15.7:0/2920136103 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.372+0000 7fef0effd700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 0x7fef10063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.372+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197020272 0 0) 0x7fef100ab100 con 0x7fef1015a280 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feef8003570 con 0x7fef1015a280 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1122787268 0 0) 0x7feef8003570 con 0x7fef1015a280 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 msgr2=0x7fef10063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 0x7fef10063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 >> v1:172.21.15.113:6789/0 conn(0x7fef100ac5b0 legacy=0x7fef100aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10063b40 con 0x7fef1015a280 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fef04001470 con 0x7fef1015a280 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0effd700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 0x7fef10063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 >> v1:172.21.15.7:6789/0 conn(0x7fef1015a280 legacy=0x7fef100aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef8003a90 0x7feef8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef8004810 0x7feef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 0x7feef80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1006c5b0 con 0x7feef8003a90 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef800bb40 con 0x7feef8004810 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef800bd00 con 0x7fef10063230 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0effd700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef8003a90 0x7feef8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef8004810 0x7feef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 0x7feef80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0effd700 1 -- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 msgr2=0x7feef80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0effd700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef10063230 0x7feef80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0effd700 1 -- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef8004810 msgr2=0x7feef8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0effd700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feef8004810 0x7feef8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.373+0000 7fef0effd700 1 -- 172.21.15.7:0/2920136103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00009420 con 0x7feef8003a90 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef0effd700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef8003a90 0x7feef8003ea0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fef00009df0 tx=0x7fef000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21932151fe953d71 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00009740 con 0x7feef8003a90 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0000e920 con 0x7feef8003a90 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef0dffb700 1 -- 172.21.15.7:0/2920136103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0000ece0 con 0x7feef8003a90 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef1574d700 1 -- 172.21.15.7:0/2920136103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef8003a90 msgr2=0x7feef8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef1574d700 1 --2- 172.21.15.7:0/2920136103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feef8003a90 0x7feef8003ea0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fef00009df0 tx=0x7fef000091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef0f7fe700 1 -- 172.21.15.7:0/2920136103 reap_dead start 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef1574d700 1 -- 172.21.15.7:0/2920136103 shutdown_connections 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef1574d700 1 -- 172.21.15.7:0/2920136103 >> 172.21.15.7:0/2920136103 conn(0x7fef100680e0 msgr2=0x7fef100cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef1574d700 1 -- 172.21.15.7:0/2920136103 shutdown_connections 2024-01-11T07:09:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.374+0000 7fef1574d700 1 -- 172.21.15.7:0/2920136103 wait complete. 2024-01-11T07:09:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.376+0000 7fef1574d700 1 Processor -- start 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.376+0000 7fef1574d700 1 -- start start 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef1574d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef1016d730 0x7fef1016a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef1574d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef100b2110 0x7fef100b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef1574d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100c3240 0x7fef100c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef1574d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef100b17b0 con 0x7fef100c3240 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef1574d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef100b15c0 con 0x7fef100b2110 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef1574d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef100c3bf0 con 0x7fef1016d730 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100c3240 0x7fef100c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100c3240 0x7fef100c36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47208/0 (socket says 172.21.15.7:47208) 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 -- 172.21.15.7:0/1490026747 learned_addr learned my addr 172.21.15.7:0/1490026747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef100b2110 0x7fef100b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0effd700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef1016d730 0x7fef1016a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 -- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef1016d730 msgr2=0x7fef1016a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef1016d730 0x7fef1016a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 -- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef100b2110 msgr2=0x7fef100b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef100b2110 0x7fef100b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 -- 172.21.15.7:0/1490026747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100adc50 con 0x7fef100c3240 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100c3240 0x7fef100c36b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fef1006d3e0 tx=0x7fef08001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7feef6ffd700 1 -- 172.21.15.7:0/1490026747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08006db0 con 0x7fef100c3240 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7feef6ffd700 1 -- 172.21.15.7:0/1490026747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0800c5a0 con 0x7fef100c3240 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7fef1574d700 1 -- 172.21.15.7:0/1490026747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef100b2580 con 0x7fef100c3240 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.377+0000 7feef6ffd700 1 -- 172.21.15.7:0/1490026747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0800c960 con 0x7fef100c3240 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.378+0000 7fef1574d700 1 -- 172.21.15.7:0/1490026747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef100ad600 con 0x7fef100c3240 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.379+0000 7feef6ffd700 1 -- 172.21.15.7:0/1490026747 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fef0800ccd0 con 0x7fef100c3240 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.379+0000 7feef6ffd700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feefc066f30 0x7feefc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.379+0000 7feef6ffd700 1 -- 172.21.15.7:0/1490026747 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef08083c80 con 0x7fef100c3240 2024-01-11T07:09:15.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.380+0000 7fef0effd700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feefc066f30 0x7feefc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.380+0000 7fef0effd700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feefc066f30 0x7feefc0693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef100cb390 tx=0x7fef0001a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.381+0000 7fef1ea4db80 1 Processor -- start 2024-01-11T07:09:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.382+0000 7fef1ea4db80 1 -- start start 2024-01-11T07:09:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.382+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 0x55c6bc64e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.382+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc649c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.382+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc66a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.382+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc57c3e0 con 0x55c6bc64a170 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.382+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc579be0 con 0x55c6bc64e8b0 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.382+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc5759d0 con 0x55c6bc64dfa0 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc66a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc66a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47210/0 (socket says 172.21.15.7:47210) 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 -- 172.21.15.7:0/3273915590 learned_addr learned my addr 172.21.15.7:0/3273915590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc649c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0effd700 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 0x55c6bc64e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 -- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 msgr2=0x55c6bc64e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 0x55c6bc64e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 -- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 msgr2=0x55c6bc649c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc649c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 -- 172.21.15.7:0/3273915590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6bc409360 con 0x55c6bc64a170 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc649c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc66a360 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fef080821a0 tx=0x7fef08009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5621e9e2c07e2391 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7feeeffff700 1 -- 172.21.15.7:0/3273915590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08050ef0 con 0x55c6bc64a170 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7feeeffff700 1 -- 172.21.15.7:0/3273915590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef080552b0 con 0x55c6bc64a170 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.383+0000 7feeeffff700 1 -- 172.21.15.7:0/3273915590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08054bb0 con 0x55c6bc64a170 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 -- 172.21.15.7:0/3273915590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 msgr2=0x55c6bc66a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc66a360 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fef080821a0 tx=0x7fef08009040 comp rx=0 tx=0).stop 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 -- 172.21.15.7:0/3273915590 shutdown_connections 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc66a360 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 0x55c6bc64e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/3273915590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc649c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 -- 172.21.15.7:0/3273915590 >> 172.21.15.7:0/3273915590 conn(0x55c6bc624440 msgr2=0x55c6bc66fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 -- 172.21.15.7:0/3273915590 shutdown_connections 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.384+0000 7fef1ea4db80 1 -- 172.21.15.7:0/3273915590 wait complete. 2024-01-11T07:09:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.385+0000 7fef1ea4db80 1 -- start start 2024-01-11T07:09:15.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.385+0000 7fef1ea4db80 1 Processor -- start 2024-01-11T07:09:15.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc67f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 0x55c6bc67f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc64c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc5221b0 con 0x55c6bc64a170 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc521e50 con 0x55c6bc64e8b0 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc46ee40 con 0x55c6bc64dfa0 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef1ea4db80 10 client.0 _ll_register_callbacks cb 0x55c6bc4717b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc67f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc67f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47226/0 (socket says 172.21.15.7:47226) 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0effd700 1 -- 172.21.15.7:0/4197205986 learned_addr learned my addr 172.21.15.7:0/4197205986 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 0x55c6bc67f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc64c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0effd700 1 -- 172.21.15.7:0/4197205986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 msgr2=0x55c6bc67f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0effd700 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 0x55c6bc67f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0effd700 1 -- 172.21.15.7:0/4197205986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 msgr2=0x55c6bc64c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0effd700 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc64c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.386+0000 7fef0effd700 1 -- 172.21.15.7:0/4197205986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef10063b40 con 0x55c6bc64a170 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef0effd700 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc67f130 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fef0000e790 tx=0x7fef0001f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4317438c4c73771 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7feeed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7feeed7fa700 1 -- 172.21.15.7:0/4197205986 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00013dd0 con 0x55c6bc64a170 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7feeed7fa700 1 -- 172.21.15.7:0/4197205986 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef0001c450 con 0x55c6bc64a170 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 1 -- 172.21.15.7:0/4197205986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6bc5f6eb0 con 0x55c6bc64a170 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7feeed7fa700 1 -- 172.21.15.7:0/4197205986 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef0001c820 con 0x55c6bc64a170 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 10 client.5826 fetch_fsmap learned FSMap version 34 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 1 -- 172.21.15.7:0/4197205986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x7fef100ad600 con 0x55c6bc64a170 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7feeed7fa700 1 -- 172.21.15.7:0/4197205986 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c6bc5f6eb0 con 0x55c6bc64a170 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7feeed7fa700 1 -- 172.21.15.7:0/4197205986 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fef0001c980 con 0x55c6bc64a170 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 10 client.5826 fetch_fsmap finished waiting for FSMap version 34 2024-01-11T07:09:15.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 10 client.5826 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 20 client.5826 populate_metadata read hostname 'smithi007' 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] conn(0x55c6bc64ce80 0x55c6bc665c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 4 client.5826 mds_command: new command op to 5712 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef1ea4db80 1 -- 172.21.15.7:0/4197205986 --> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6bc54c880 con 0x55c6bc64ce80 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.387+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] conn(0x55c6bc64ce80 0x55c6bc665c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.388+0000 7feeed7fa700 10 client.5826 ms_handle_connect on v2:172.21.15.7:6833/1812078530 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.388+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] conn(0x55c6bc64ce80 0x55c6bc665c20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5014733c9ce29dba server_cookie=514adb604de72561 in_seq=0 out_seq=0 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.388+0000 7feeed7fa700 1 -- 172.21.15.7:0/4197205986 <== mds.0 v2:172.21.15.7:6833/1812078530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c6bc54c880 con 0x55c6bc64ce80 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.388+0000 7feeed7fa700 10 client.5826 handle_command_reply: tid=0 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.388+0000 7fef1ea4db80 1 client.5826 shutdown 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 -- 172.21.15.7:0/4197205986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 msgr2=0x55c6bc67f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc67f130 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fef0000e790 tx=0x7fef0001f050 comp rx=0 tx=0).stop 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 -- 172.21.15.7:0/4197205986 shutdown_connections 2024-01-11T07:09:15.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc64c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64a170 0x55c6bc67f130 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] conn(0x55c6bc64ce80 0x55c6bc665c20 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/4197205986 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64dfa0 0x55c6bc67f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 -- 172.21.15.7:0/4197205986 >> 172.21.15.7:0/4197205986 conn(0x55c6bc624440 msgr2=0x55c6bc624810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7feeed7fa700 0 client.5826 ms_handle_reset on v2:172.21.15.7:6833/1812078530 2024-01-11T07:09:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 -- 172.21.15.7:0/4197205986 shutdown_connections 2024-01-11T07:09:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 1 -- 172.21.15.7:0/4197205986 wait complete. 2024-01-11T07:09:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.389+0000 7fef1ea4db80 20 client.5826 trim_cache size 0 max 16384 2024-01-11T07:09:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.407+0000 7fef1ea4db80 1 Processor -- start 2024-01-11T07:09:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.407+0000 7fef1ea4db80 1 -- start start 2024-01-11T07:09:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64a170 0x55c6bc67b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc66aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc66afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc68d470 con 0x55c6bc64dfa0 2024-01-11T07:09:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc64a580 con 0x55c6bc64e8b0 2024-01-11T07:09:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc655a50 con 0x55c6bc64a170 2024-01-11T07:09:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc66aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc66aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47232/0 (socket says 172.21.15.7:47232) 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0effd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64a170 0x55c6bc67b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 -- 172.21.15.7:0/78108895 learned_addr learned my addr 172.21.15.7:0/78108895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/78108895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc66afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 -- 172.21.15.7:0/78108895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64a170 msgr2=0x55c6bc67b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/78108895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64a170 0x55c6bc67b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 -- 172.21.15.7:0/78108895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 msgr2=0x55c6bc66afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/78108895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc66afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 -- 172.21.15.7:0/78108895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6bc3b5b10 con 0x55c6bc64dfa0 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.408+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/78108895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc66aa80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fef04004890 tx=0x7fef1015b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951d04da73e19b28 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7feef4ff9700 1 -- 172.21.15.7:0/78108895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04007550 con 0x55c6bc64dfa0 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7feef4ff9700 1 -- 172.21.15.7:0/78108895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0401c070 con 0x55c6bc64dfa0 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7feef4ff9700 1 -- 172.21.15.7:0/78108895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04020450 con 0x55c6bc64dfa0 2024-01-11T07:09:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 -- 172.21.15.7:0/78108895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 msgr2=0x55c6bc66aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/78108895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc66aa80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fef04004890 tx=0x7fef1015b370 comp rx=0 tx=0).stop 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 -- 172.21.15.7:0/78108895 shutdown_connections 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/78108895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64e8b0 0x55c6bc66afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/78108895 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64a170 0x55c6bc67b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/78108895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc66aa80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 -- 172.21.15.7:0/78108895 >> 172.21.15.7:0/78108895 conn(0x55c6bc624440 msgr2=0x55c6bc664380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 -- 172.21.15.7:0/78108895 shutdown_connections 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.409+0000 7fef1ea4db80 1 -- 172.21.15.7:0/78108895 wait complete. 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.410+0000 7fef1ea4db80 1 -- start start 2024-01-11T07:09:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.410+0000 7fef1ea4db80 1 Processor -- start 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64a170 0x55c6bc66ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc67ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef1ea4db80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64e8b0 0x55c6bc67f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc6512a0 con 0x55c6bc64dfa0 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc61bb10 con 0x55c6bc64a170 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef1ea4db80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc652d50 con 0x55c6bc64e8b0 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef1ea4db80 10 client.0 _ll_register_callbacks cb 0x7fef0000ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc67ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc67ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47234/0 (socket says 172.21.15.7:47234) 2024-01-11T07:09:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0e7fc700 1 -- 172.21.15.7:0/561700930 learned_addr learned my addr 172.21.15.7:0/561700930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0f7fe700 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64e8b0 0x55c6bc67f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0effd700 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64a170 0x55c6bc66ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0e7fc700 1 -- 172.21.15.7:0/561700930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64e8b0 msgr2=0x55c6bc67f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64e8b0 0x55c6bc67f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0e7fc700 1 -- 172.21.15.7:0/561700930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64a170 msgr2=0x55c6bc66ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64a170 0x55c6bc66ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.411+0000 7fef0e7fc700 1 -- 172.21.15.7:0/561700930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef04004a10 con 0x55c6bc64dfa0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef0e7fc700 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc67ed30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fef040070f0 tx=0x7fef04006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e965fb77272c56f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7feeed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7feeed7fa700 1 -- 172.21.15.7:0/561700930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04007580 con 0x55c6bc64dfa0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7feeed7fa700 1 -- 172.21.15.7:0/561700930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef040076e0 con 0x55c6bc64dfa0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 1 -- 172.21.15.7:0/561700930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6bc61a490 con 0x55c6bc64dfa0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7feeed7fa700 1 -- 172.21.15.7:0/561700930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef04034d00 con 0x55c6bc64dfa0 2024-01-11T07:09:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 10 client.5832 fetch_fsmap learned FSMap version 34 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 1 -- 172.21.15.7:0/561700930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c6bc544290 con 0x55c6bc64dfa0 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7feeed7fa700 1 -- 172.21.15.7:0/561700930 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c6bc61a490 con 0x55c6bc64dfa0 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7feeed7fa700 1 -- 172.21.15.7:0/561700930 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fef04012050 con 0x55c6bc64dfa0 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 10 client.5832 fetch_fsmap finished waiting for FSMap version 34 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 10 client.5832 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 20 client.5832 populate_metadata read hostname 'smithi007' 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] conn(0x55c6bc6d4f20 0x55c6bc6d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 4 client.5832 mds_command: new command op to 5712 tid=0[{"prefix": "status"}] 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef1ea4db80 1 -- 172.21.15.7:0/561700930 --> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c6bc518910 con 0x55c6bc6d4f20 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.412+0000 7fef0effd700 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] conn(0x55c6bc6d4f20 0x55c6bc6d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7feeed7fa700 10 client.5832 ms_handle_connect on v2:172.21.15.7:6833/1812078530 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef0effd700 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] conn(0x55c6bc6d4f20 0x55c6bc6d52f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53f33ec9bd9be8e server_cookie=74b06fd1843a46ff in_seq=0 out_seq=0 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7feeed7fa700 1 -- 172.21.15.7:0/561700930 <== mds.0 v2:172.21.15.7:6833/1812078530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55c6bc518910 con 0x55c6bc6d4f20 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7feeed7fa700 10 client.5832 handle_command_reply: tid=0 2024-01-11T07:09:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 client.5832 shutdown 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 -- 172.21.15.7:0/561700930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 msgr2=0x55c6bc67ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc67ed30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fef040070f0 tx=0x7fef04006000 comp rx=0 tx=0).stop 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 -- 172.21.15.7:0/561700930 shutdown_connections 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.7:6833/1812078530,v1:172.21.15.7:6835/1812078530] conn(0x55c6bc6d4f20 0x55c6bc6d52f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c6bc64e8b0 0x55c6bc67f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c6bc64a170 0x55c6bc66ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7feeed7fa700 0 client.5832 ms_handle_reset on v2:172.21.15.7:6833/1812078530 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 --2- 172.21.15.7:0/561700930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6bc64dfa0 0x55c6bc67ed30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 -- 172.21.15.7:0/561700930 >> 172.21.15.7:0/561700930 conn(0x55c6bc624440 msgr2=0x55c6bc67b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 -- 172.21.15.7:0/561700930 shutdown_connections 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.413+0000 7fef1ea4db80 1 -- 172.21.15.7:0/561700930 wait complete. 2024-01-11T07:09:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1ea4db80 20 client.5832 trim_cache size 0 max 16384 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5712, 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1099278699999999, 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 34, 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 64, 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 64, 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.946825215 2024-01-11T07:09:15.419 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:09:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 -- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feefc066f30 msgr2=0x7feefc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feefc066f30 0x7feefc0693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef100cb390 tx=0x7fef0001a000 comp rx=0 tx=0).stop 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 -- 172.21.15.7:0/1490026747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100c3240 msgr2=0x7fef100c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100c3240 0x7fef100c36b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fef1006d3e0 tx=0x7fef08001640 comp rx=0 tx=0).stop 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 -- 172.21.15.7:0/1490026747 shutdown_connections 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feefc066f30 0x7feefc0693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef100c3240 0x7fef100c36b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef1016d730 0x7fef1016a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 --2- 172.21.15.7:0/1490026747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef100b2110 0x7fef100b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 -- 172.21.15.7:0/1490026747 >> 172.21.15.7:0/1490026747 conn(0x7fef100680e0 msgr2=0x7fef100cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 -- 172.21.15.7:0/1490026747 shutdown_connections 2024-01-11T07:09:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.414+0000 7fef1574d700 1 -- 172.21.15.7:0/1490026747 wait complete. 2024-01-11T07:09:15.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:09:15.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.559+0000 7fa6c93d8700 1 Processor -- start 2024-01-11T07:09:15.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.559+0000 7fa6c93d8700 1 -- start start 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c93d8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6c40ac3a0 0x7fa6c40cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c93d8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ab8d0 0x7fa6c415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c93d8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c415e2d0 con 0x7fa6c415d920 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c93d8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c406ac30 con 0x7fa6c40ac3a0 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c93d8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40ac770 con 0x7fa6c40ab8d0 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6c40ac3a0 0x7fa6c40cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c27fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa6c415d920 0x7fa6c415a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60500/0 (socket says 172.21.15.7:60500) 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c27fc700 1 -- 172.21.15.7:0/3470623394 learned_addr learned my addr 172.21.15.7:0/3470623394 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c37fe700 1 --2- 172.21.15.7:0/3470623394 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ab8d0 0x7fa6c415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c2ffd700 1 -- 172.21.15.7:0/3470623394 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ab8d0 msgr2=0x7fa6c415dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c2ffd700 1 --2- 172.21.15.7:0/3470623394 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ab8d0 0x7fa6c415dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c2ffd700 1 -- 172.21.15.7:0/3470623394 >> v1:172.21.15.7:6789/0 conn(0x7fa6c415d920 legacy=0x7fa6c415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/3470623394 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423542640 0 0) 0x7fa6c415e2d0 con 0x7fa6c415d920 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c2ffd700 1 -- 172.21.15.7:0/3470623394 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c406a300 con 0x7fa6c40ac3a0 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.560+0000 7fa6c2ffd700 1 --2- 172.21.15.7:0/3470623394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c40ac3a0 0x7fa6c40cac60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40098e0 tx=0x7fa6b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd7d86c4a2787e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/3470623394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400b640 con 0x7fa6c40ac3a0 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/3470623394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b400b7a0 con 0x7fa6c40ac3a0 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c1ffb700 1 -- 172.21.15.7:0/3470623394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400bb40 con 0x7fa6c40ac3a0 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c93d8700 1 -- 172.21.15.7:0/3470623394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c40ac3a0 msgr2=0x7fa6c40cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/3470623394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c40ac3a0 0x7fa6c40cac60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40098e0 tx=0x7fa6b40091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c93d8700 1 -- 172.21.15.7:0/3470623394 shutdown_connections 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/3470623394 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ab8d0 0x7fa6c415dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/3470623394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c40ac3a0 0x7fa6c40cac60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c93d8700 1 -- 172.21.15.7:0/3470623394 >> 172.21.15.7:0/3470623394 conn(0x7fa6c40cd090 msgr2=0x7fa6c40cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:15.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c93d8700 1 -- 172.21.15.7:0/3470623394 shutdown_connections 2024-01-11T07:09:15.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.561+0000 7fa6c93d8700 1 -- 172.21.15.7:0/3470623394 wait complete. 2024-01-11T07:09:15.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c93d8700 1 Processor -- start 2024-01-11T07:09:15.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c93d8700 1 -- start start 2024-01-11T07:09:15.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c93d8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c40ab8d0 0x7fa6c416ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c93d8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ac3a0 0x7fa6c413e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c93d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c415d920 0x7fa6c413ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c93d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c416a360 con 0x7fa6c415d920 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c93d8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c413f460 con 0x7fa6c40ab8d0 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c93d8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c413f740 con 0x7fa6c40ac3a0 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c415d920 0x7fa6c413ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.563+0000 7fa6c37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c415d920 0x7fa6c413ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47254/0 (socket says 172.21.15.7:47254) 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c37fe700 1 -- 172.21.15.7:0/1661563690 learned_addr learned my addr 172.21.15.7:0/1661563690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c27fc700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ac3a0 0x7fa6c413e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c2ffd700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c40ab8d0 0x7fa6c416ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c37fe700 1 -- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ac3a0 msgr2=0x7fa6c413e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c37fe700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ac3a0 0x7fa6c413e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c37fe700 1 -- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c40ab8d0 msgr2=0x7fa6c416ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c37fe700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c40ab8d0 0x7fa6c416ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c37fe700 1 -- 172.21.15.7:0/1661563690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4009420 con 0x7fa6c415d920 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c37fe700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c415d920 0x7fa6c413ede0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009d40 tx=0x7fa6bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6aaffd700 1 -- 172.21.15.7:0/1661563690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00cb50 con 0x7fa6c415d920 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6aaffd700 1 -- 172.21.15.7:0/1661563690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc00f040 con 0x7fa6c415d920 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c413fa20 con 0x7fa6c415d920 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6aaffd700 1 -- 172.21.15.7:0/1661563690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc014480 con 0x7fa6c415d920 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.564+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c413ffe0 con 0x7fa6c415d920 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.565+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c404c3c0 con 0x7fa6c415d920 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.566+0000 7fa6aaffd700 1 -- 172.21.15.7:0/1661563690 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa6bc014830 con 0x7fa6c415d920 2024-01-11T07:09:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.566+0000 7fa6aaffd700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6b0066f30 0x7fa6b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:15.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.566+0000 7fa6aaffd700 1 -- 172.21.15.7:0/1661563690 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa6bc085160 con 0x7fa6c415d920 2024-01-11T07:09:15.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.566+0000 7fa6c2ffd700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6b0066f30 0x7fa6b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:15.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.567+0000 7fa6c2ffd700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6b0066f30 0x7fa6b00693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40093f0 tx=0x7fa6b4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.571+0000 7fa6aaffd700 1 -- 172.21.15.7:0/1661563690 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6bc0523d0 con 0x7fa6c415d920 2024-01-11T07:09:15.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.823+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa6c406a5e0 con 0x7fa6c415d920 2024-01-11T07:09:15.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.823+0000 7fa6aaffd700 1 -- 172.21.15.7:0/1661563690 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa6bc0559f0 con 0x7fa6c415d920 2024-01-11T07:09:15.823 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:09:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6b0066f30 msgr2=0x7fa6b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6b0066f30 0x7fa6b00693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40093f0 tx=0x7fa6b4017000 comp rx=0 tx=0).stop 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c415d920 msgr2=0x7fa6c413ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c415d920 0x7fa6c413ede0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009d40 tx=0x7fa6bc00b040 comp rx=0 tx=0).stop 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 shutdown_connections 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa6b0066f30 0x7fa6b00693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6c415d920 0x7fa6c413ede0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6c40ab8d0 0x7fa6c416ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 --2- 172.21.15.7:0/1661563690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6c40ac3a0 0x7fa6c413e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 >> 172.21.15.7:0/1661563690 conn(0x7fa6c40cd090 msgr2=0x7fa6c413b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 shutdown_connections 2024-01-11T07:09:15.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:15.826+0000 7fa6c93d8700 1 -- 172.21.15.7:0/1661563690 wait complete. 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:09:15.839 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:09:15.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:15.840 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:09:15.859 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:09:15.859 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:09:15.859 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:09:15.859 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:15.859 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42262sec preferred_lft 42262sec 2024-01-11T07:09:15.860 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:09:15.861 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:15.862 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:09:15.862 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:09:15.862 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:09:15.862 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:15.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:15.862 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:09:15.930 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:09:15.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:15.931 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:09:15.995 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:09:15.995 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:09:15.995 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:09:15.995 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:09:15.995 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:09:15.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:15.996 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:09:15.996 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:09:15.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:09:15.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:09:15.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:09:15.996 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:09:16.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:16.121 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:09:16.142 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:09:16.142 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:09:16.142 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:09:16.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:16.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:09:16.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:09:16.218 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:09:16.280 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:09:16.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:09:16.281 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:09:16.342 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:09:16.342 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:09:16.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:09:16.436 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:09:16.436 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:09:16.436 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:09:16.446 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:09:16.447 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:09:16.447 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:09:16.447 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:09:16.447 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:09:16.466 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:09:16.466 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:09:16.466 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:09:16.466 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:09:16.466 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:09:16.466 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:09:16.466 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:09:16.466 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:09:16.468 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:09:16.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:16.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:09:16.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:09:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.680+0000 7f12e050a700 1 Processor -- start 2024-01-11T07:09:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.680+0000 7f12e050a700 1 -- start start 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.681+0000 7f12e050a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12d81546b0 0x7f12d80ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.681+0000 7f12e050a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d806a520 0x7f12d806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.681+0000 7f12e050a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d806a090 con 0x7f12d81542e0 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.681+0000 7f12e050a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80acac0 con 0x7f12d81546b0 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.681+0000 7f12e050a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80acca0 con 0x7f12d806a520 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12ddaa5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12d81546b0 0x7f12d80ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12de2a6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f12d81542e0 0x7f12d80aaae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60502/0 (socket says 172.21.15.7:60502) 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12de2a6700 1 -- 172.21.15.7:0/187693910 learned_addr learned my addr 172.21.15.7:0/187693910 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12deaa7700 1 --2- 172.21.15.7:0/187693910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d806a520 0x7f12d806bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12ddaa5700 1 -- 172.21.15.7:0/187693910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d806a520 msgr2=0x7f12d806bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12ddaa5700 1 --2- 172.21.15.7:0/187693910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d806a520 0x7f12d806bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12ddaa5700 1 -- 172.21.15.7:0/187693910 >> v1:172.21.15.7:6789/0 conn(0x7f12d81542e0 legacy=0x7f12d80aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12dd2a4700 1 -- 172.21.15.7:0/187693910 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925864534 0 0) 0x7f12d806a090 con 0x7f12d81542e0 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12ddaa5700 1 -- 172.21.15.7:0/187693910 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d80acf80 con 0x7f12d81546b0 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12ddaa5700 1 --2- 172.21.15.7:0/187693910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d80ac580 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f12c80098e0 tx=0x7f12c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c527f9467604c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12dd2a4700 1 -- 172.21.15.7:0/187693910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800b640 con 0x7f12d81546b0 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12dd2a4700 1 -- 172.21.15.7:0/187693910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c800b7a0 con 0x7f12d81546b0 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.682+0000 7f12dd2a4700 1 -- 172.21.15.7:0/187693910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800bb40 con 0x7f12d81546b0 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.683+0000 7f12e050a700 1 -- 172.21.15.7:0/187693910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 msgr2=0x7f12d80ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.683+0000 7f12e050a700 1 --2- 172.21.15.7:0/187693910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d80ac580 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f12c80098e0 tx=0x7f12c80091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.683+0000 7f12e050a700 1 -- 172.21.15.7:0/187693910 shutdown_connections 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.683+0000 7f12e050a700 1 --2- 172.21.15.7:0/187693910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d806a520 0x7f12d806bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.683+0000 7f12e050a700 1 --2- 172.21.15.7:0/187693910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d80ac580 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.683+0000 7f12e050a700 1 -- 172.21.15.7:0/187693910 >> 172.21.15.7:0/187693910 conn(0x7f12d80cd910 msgr2=0x7f12d80cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.683+0000 7f12e050a700 1 -- 172.21.15.7:0/187693910 shutdown_connections 2024-01-11T07:09:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.683+0000 7f12e050a700 1 -- 172.21.15.7:0/187693910 wait complete. 2024-01-11T07:09:16.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.684+0000 7f12e050a700 1 Processor -- start 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.684+0000 7f12e050a700 1 -- start start 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12e050a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d806a520 0x7f12d80af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12e050a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d81542e0 0x7f12d814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12e050a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d814e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12e050a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8156e80 con 0x7f12d81546b0 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12e050a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8156550 con 0x7f12d806a520 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12e050a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8156830 con 0x7f12d81542e0 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12deaa7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d814e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12deaa7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d814e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47282/0 (socket says 172.21.15.7:47282) 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.685+0000 7f12deaa7700 1 -- 172.21.15.7:0/3397568135 learned_addr learned my addr 172.21.15.7:0/3397568135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12de2a6700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d806a520 0x7f12d80af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12ddaa5700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d81542e0 0x7f12d814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12deaa7700 1 -- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d81542e0 msgr2=0x7f12d814dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12deaa7700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d81542e0 0x7f12d814dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12deaa7700 1 -- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d806a520 msgr2=0x7f12d80af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12deaa7700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d806a520 0x7f12d80af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12deaa7700 1 -- 172.21.15.7:0/3397568135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c8009420 con 0x7f12d81546b0 2024-01-11T07:09:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12de2a6700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d806a520 0x7f12d80af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:09:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12deaa7700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d814e510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009e90 tx=0x7f12d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12ce7fc700 1 -- 172.21.15.7:0/3397568135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000cc20 con 0x7f12d81546b0 2024-01-11T07:09:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12ce7fc700 1 -- 172.21.15.7:0/3397568135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d000f040 con 0x7f12d81546b0 2024-01-11T07:09:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12ce7fc700 1 -- 172.21.15.7:0/3397568135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0014480 con 0x7f12d81546b0 2024-01-11T07:09:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d8156b10 con 0x7f12d81546b0 2024-01-11T07:09:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.686+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d80aff20 con 0x7f12d81546b0 2024-01-11T07:09:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.687+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d80b01a0 con 0x7f12d81546b0 2024-01-11T07:09:16.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.688+0000 7f12ce7fc700 1 -- 172.21.15.7:0/3397568135 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f12d0014830 con 0x7f12d81546b0 2024-01-11T07:09:16.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.688+0000 7f12ce7fc700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f12bc066f30 0x7f12bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:16.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.688+0000 7f12ce7fc700 1 -- 172.21.15.7:0/3397568135 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12d00852d0 con 0x7f12d81546b0 2024-01-11T07:09:16.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.688+0000 7f12de2a6700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f12bc066f30 0x7f12bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:16.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.689+0000 7f12de2a6700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f12bc066f30 0x7f12bc0693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f12d8153350 tx=0x7f12d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:16.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.693+0000 7f12ce7fc700 1 -- 172.21.15.7:0/3397568135 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12d0055f30 con 0x7f12d81546b0 2024-01-11T07:09:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.943+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f12d8064a40 con 0x7f12d81546b0 2024-01-11T07:09:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.944+0000 7f12ce7fc700 1 -- 172.21.15.7:0/3397568135 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f12d0055ac0 con 0x7f12d81546b0 2024-01-11T07:09:16.943 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f12bc066f30 msgr2=0x7f12bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f12bc066f30 0x7f12bc0693e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f12d8153350 tx=0x7f12d4009040 comp rx=0 tx=0).stop 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 msgr2=0x7f12d814e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d814e510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009e90 tx=0x7f12d000b040 comp rx=0 tx=0).stop 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 shutdown_connections 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f12bc066f30 0x7f12bc0693e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12d81546b0 0x7f12d814e510 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12d806a520 0x7f12d80af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 --2- 172.21.15.7:0/3397568135 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12d81542e0 0x7f12d814dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 >> 172.21.15.7:0/3397568135 conn(0x7f12d80cd910 msgr2=0x7f12d806ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 shutdown_connections 2024-01-11T07:09:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:16.946+0000 7f12e050a700 1 -- 172.21.15.7:0/3397568135 wait complete. 2024-01-11T07:09:16.959 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:09:16.959 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:09:16.959 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:09:16.959 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:09:16.959 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:09:16.959 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:09:16.959 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:09:16.959 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:09:16.960 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:09:16.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:16.960 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:09:16.982 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:09:16.983 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42304sec preferred_lft 42304sec 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:09:16.984 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:09:16.985 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:09:16.985 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:16.985 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:09:16.985 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:09:16.985 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:09:16.985 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:09:16.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:16.985 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:09:17.056 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:09:17.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:17.058 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:09:17.120 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:09:17.120 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:09:17.120 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:09:17.120 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:09:17.120 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:09:17.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:17.121 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:09:17.121 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:09:17.121 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:09:17.121 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:09:17.121 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:09:17.121 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:09:17.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:17.232 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:09:17.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:09:17.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:09:17.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:09:17.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:17.250 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:09:17.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:09:17.318 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:09:17.376 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:09:17.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:09:17.376 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:09:17.433 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:09:17.433 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:09:17.433 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:09:17.515 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:09:17.515 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:09:17.515 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:09:17.523 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:09:17.523 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:09:17.524 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:09:17.524 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:09:17.524 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:09:17.541 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:09:17.541 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:09:17.541 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:09:17.542 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:09:17.542 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:09:17.542 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:09:17.542 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:09:17.542 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:09:17.545 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:09:17.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:09:17.545 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:09:17.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:09:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.754+0000 7f7fe13b8700 1 Processor -- start 2024-01-11T07:09:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.754+0000 7f7fe13b8700 1 -- start start 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fe13b8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fdc0693a0 0x7f7fdc06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fe13b8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7fdc06f1d0 0x7f7fdc0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fe13b8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc06f5c0 con 0x7f7fdc159d10 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fe13b8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc15d9a0 con 0x7f7fdc0693a0 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fe13b8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc15dbb0 con 0x7f7fdc06f1d0 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fda7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fdc0693a0 0x7f7fdc06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fdaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7fdc159d10 0x7f7fdc068c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58176/0 (socket says 172.21.15.7:58176) 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fdaffd700 1 -- 172.21.15.7:0/736763676 learned_addr learned my addr 172.21.15.7:0/736763676 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:09:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fdb7fe700 1 --2- 172.21.15.7:0/736763676 >> v2:172.21.15.113:3300/0 conn(0x7f7fdc06f1d0 0x7f7fdc0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fda7fc700 1 -- 172.21.15.7:0/736763676 >> v2:172.21.15.113:3300/0 conn(0x7f7fdc06f1d0 msgr2=0x7f7fdc0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fda7fc700 1 --2- 172.21.15.7:0/736763676 >> v2:172.21.15.113:3300/0 conn(0x7f7fdc06f1d0 0x7f7fdc0b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fda7fc700 1 -- 172.21.15.7:0/736763676 >> v1:172.21.15.7:6789/0 conn(0x7f7fdc159d10 legacy=0x7f7fdc068c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fda7fc700 1 -- 172.21.15.7:0/736763676 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc15de90 con 0x7f7fdc0693a0 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fd9ffb700 1 -- 172.21.15.7:0/736763676 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312337767 0 0) 0x7f7fdc06f5c0 con 0x7f7fdc159d10 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.755+0000 7f7fda7fc700 1 --2- 172.21.15.7:0/736763676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc06ec90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0ce830 tx=0x7f7fcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7103c71919fb1465 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fd9ffb700 1 -- 172.21.15.7:0/736763676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011630 con 0x7f7fdc0693a0 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fd9ffb700 1 -- 172.21.15.7:0/736763676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc011790 con 0x7f7fdc0693a0 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fd9ffb700 1 -- 172.21.15.7:0/736763676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011b30 con 0x7f7fdc0693a0 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fe13b8700 1 -- 172.21.15.7:0/736763676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 msgr2=0x7f7fdc06ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/736763676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc06ec90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0ce830 tx=0x7f7fcc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fe13b8700 1 -- 172.21.15.7:0/736763676 shutdown_connections 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/736763676 >> v2:172.21.15.113:3300/0 conn(0x7f7fdc06f1d0 0x7f7fdc0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/736763676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc06ec90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:17.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fe13b8700 1 -- 172.21.15.7:0/736763676 >> 172.21.15.7:0/736763676 conn(0x7f7fdc0ce9f0 msgr2=0x7f7fdc0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:17.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fe13b8700 1 -- 172.21.15.7:0/736763676 shutdown_connections 2024-01-11T07:09:17.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.756+0000 7f7fe13b8700 1 -- 172.21.15.7:0/736763676 wait complete. 2024-01-11T07:09:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.758+0000 7f7fe13b8700 1 Processor -- start 2024-01-11T07:09:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.758+0000 7f7fe13b8700 1 -- start start 2024-01-11T07:09:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fe13b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc14aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fe13b8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fdc06f1d0 0x7f7fdc157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fe13b8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fdc159d10 0x7f7fdc15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fe13b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0c6d10 con 0x7f7fdc0693a0 2024-01-11T07:09:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fe13b8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc157b10 con 0x7f7fdc159d10 2024-01-11T07:09:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fe13b8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc1584e0 con 0x7f7fdc06f1d0 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fdaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc14aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fdaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc14aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37192/0 (socket says 172.21.15.7:37192) 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fdaffd700 1 -- 172.21.15.7:0/3213365296 learned_addr learned my addr 172.21.15.7:0/3213365296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fdb7fe700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fdc159d10 0x7f7fdc15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.759+0000 7f7fda7fc700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fdc06f1d0 0x7f7fdc157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fdaffd700 1 -- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fdc06f1d0 msgr2=0x7f7fdc157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fdaffd700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fdc06f1d0 0x7f7fdc157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fdaffd700 1 -- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fdc159d10 msgr2=0x7f7fdc15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fdaffd700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fdc159d10 0x7f7fdc15cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fdaffd700 1 -- 172.21.15.7:0/3213365296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc009420 con 0x7f7fdc0693a0 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fda7fc700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fdc06f1d0 0x7f7fdc157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fdaffd700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc14aff0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000c5e0 tx=0x7f7fd0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fc2ffd700 1 -- 172.21.15.7:0/3213365296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0005b30 con 0x7f7fdc0693a0 2024-01-11T07:09:17.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fc2ffd700 1 -- 172.21.15.7:0/3213365296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd0005c90 con 0x7f7fdc0693a0 2024-01-11T07:09:17.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc0c6f60 con 0x7f7fdc0693a0 2024-01-11T07:09:17.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fc2ffd700 1 -- 172.21.15.7:0/3213365296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0007300 con 0x7f7fdc0693a0 2024-01-11T07:09:17.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.760+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc1574c0 con 0x7f7fdc0693a0 2024-01-11T07:09:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.761+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc069770 con 0x7f7fdc0693a0 2024-01-11T07:09:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.761+0000 7f7fc2ffd700 1 -- 172.21.15.7:0/3213365296 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7fd001b3b0 con 0x7f7fdc0693a0 2024-01-11T07:09:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.762+0000 7f7fc2ffd700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7fc8066f30 0x7f7fc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:09:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.762+0000 7f7fc2ffd700 1 -- 172.21.15.7:0/3213365296 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7fd0083a10 con 0x7f7fdc0693a0 2024-01-11T07:09:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.762+0000 7f7fda7fc700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7fc8066f30 0x7f7fc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:09:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.763+0000 7f7fda7fc700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7fc8066f30 0x7f7fc80693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0093f0 tx=0x7f7fcc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:09:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:17.767+0000 7f7fc2ffd700 1 -- 172.21.15.7:0/3213365296 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fd0050c80 con 0x7f7fdc0693a0 2024-01-11T07:09:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.018+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fdc04c3c0 con 0x7f7fdc0693a0 2024-01-11T07:09:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.019+0000 7f7fc2ffd700 1 -- 172.21.15.7:0/3213365296 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f7fd0017090 con 0x7f7fdc0693a0 2024-01-11T07:09:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:09:18.019 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1734166534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1734166534},{"type":"v1","addr":"172.21.15.113:6836","nonce":1734166534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3195231529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3195231529},{"type":"v1","addr":"172.21.15.113:6837","nonce":3195231529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/338342475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":338342475},{"type":"v1","addr":"172.21.15.7:6834","nonce":338342475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:09:08.717048+0000","modified":"2024-01-11T07:09:12.324714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1812078530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1812078530},{"type":"v1","addr":"172.21.15.7:6835","nonce":1812078530}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-11T07:09:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7fc8066f30 msgr2=0x7f7fc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7fc8066f30 0x7f7fc80693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0093f0 tx=0x7f7fcc017000 comp rx=0 tx=0).stop 2024-01-11T07:09:18.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 msgr2=0x7f7fdc14aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc14aff0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000c5e0 tx=0x7f7fd0001070 comp rx=0 tx=0).stop 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 shutdown_connections 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7fc8066f30 0x7f7fc80693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fdc159d10 0x7f7fdc15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fdc0693a0 0x7f7fdc14aff0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 --2- 172.21.15.7:0/3213365296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fdc06f1d0 0x7f7fdc157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 >> 172.21.15.7:0/3213365296 conn(0x7f7fdc0ce9f0 msgr2=0x7f7fdc0cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 shutdown_connections 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:09:18.021+0000 7f7fe13b8700 1 -- 172.21.15.7:0/3213365296 wait complete. 2024-01-11T07:09:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-11T07:09:18.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-11T07:09:18.187 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":5712,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8729610349999994,"mdsmap_epoch":34,"osdmap_epoch":64,"osdmap_epoch_barrier":64,"uptime":14.709861181999999} 2024-01-11T07:09:18.188 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 5712, "mdsmap_epoch": 34, "osdmap_epoch": 64, "osdmap_epoch_barrier": 64, "rank_uptime": 8.872961035, "state": "up:active", "uptime": 14.709861182, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:09:18.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-11T07:09:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-11T07:09:18.337 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-11T07:09:18.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-01-11T07:09:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:{"mon_tick_interval":"5"} 2024-01-11T07:09:18.479 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-01-11T07:09:18.479 INFO:tasks.cephfs.test_full:pool capacity 557041216, 531MB should be enough to fill it 2024-01-11T07:09:18.479 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:09:18.479 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-01-11T07:09:18.479 DEBUG:teuthology.orchestra.run.smithi007:> import datetime 2024-01-11T07:09:18.479 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-11T07:09:18.479 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> # Write some buffered data through before going full, all should be well 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> print("writing some data through which we expect to succeed") 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> bytes = 0 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> # buffered IO, the error should always only appear from fsync and not 2024-01-11T07:09:18.480 DEBUG:teuthology.orchestra.run.smithi007:> # from write 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> full = False 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(531 * 0.9)): 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> print("done writing {0} bytes".format(bytes)) 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> # OK, now we should sneak in under the full condition 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> # due to the time it takes the OSDs to report to the 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> # mons, and get a successful fsync on our full-making data 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> # buffered write, add more dirty data to the buffer 2024-01-11T07:09:18.481 DEBUG:teuthology.orchestra.run.smithi007:> print("starting buffered write") 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(531 * 0.2)): 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(15.0) 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> except OSError: 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> pass; 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> # Wait long enough for a background flush that should fail 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(30) 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> # ...and check that the failed background flush is reflected in fclose 2024-01-11T07:09:18.482 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> except OSError: 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> print("close() returned an error as expected") 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("close() failed to raise error") 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> # The kernel cephfs client does not raise errors on fclose 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-01-11T07:09:18.483 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:writing some data through which we expect to succeed 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-01-11T07:09:23.547 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-01-11T07:09:23.548 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 30932992 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 31981568 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 33030144 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 34078720 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 35127296 bytes via buffered write, may repeat 2024-01-11T07:09:23.549 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 36175872 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 37224448 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 38273024 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 39321600 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 40370176 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 41418752 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 42467328 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 43515904 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 44564480 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 45613056 bytes via buffered write, may repeat 2024-01-11T07:09:23.550 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 46661632 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 47710208 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 48758784 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 49807360 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 50855936 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 51904512 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 52953088 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 54001664 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 55050240 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 56098816 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 57147392 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 58195968 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 59244544 bytes via buffered write, may repeat 2024-01-11T07:09:23.551 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 60293120 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 61341696 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 62390272 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 63438848 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 64487424 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 65536000 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 66584576 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 67633152 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 68681728 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 69730304 bytes via buffered write, may repeat 2024-01-11T07:09:23.552 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 70778880 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 71827456 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 72876032 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 73924608 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 74973184 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 76021760 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 77070336 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 78118912 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 79167488 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 80216064 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 81264640 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 82313216 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 83361792 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 84410368 bytes via buffered write, may repeat 2024-01-11T07:09:23.553 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 85458944 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 86507520 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 87556096 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 88604672 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 89653248 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 90701824 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 91750400 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 92798976 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 93847552 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 94896128 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 95944704 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 96993280 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 98041856 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 99090432 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 100139008 bytes via buffered write, may repeat 2024-01-11T07:09:23.554 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 101187584 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 102236160 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 103284736 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 104333312 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 105381888 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 106430464 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 107479040 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 108527616 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 109576192 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 110624768 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 111673344 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 112721920 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 113770496 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 114819072 bytes via buffered write, may repeat 2024-01-11T07:09:23.555 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 115867648 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 116916224 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 117964800 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 119013376 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 120061952 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 121110528 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 122159104 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 123207680 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 124256256 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 125304832 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 126353408 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 127401984 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 128450560 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 129499136 bytes via buffered write, may repeat 2024-01-11T07:09:23.556 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 130547712 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 131596288 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 132644864 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 133693440 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 134742016 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 135790592 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 136839168 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 137887744 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 138936320 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 139984896 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 141033472 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 142082048 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 143130624 bytes via buffered write, may repeat 2024-01-11T07:09:23.557 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 144179200 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 145227776 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 146276352 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 147324928 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 148373504 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 149422080 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 150470656 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 151519232 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 152567808 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 153616384 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 154664960 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 155713536 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 156762112 bytes via buffered write, may repeat 2024-01-11T07:09:23.558 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 157810688 bytes via buffered write, may repeat 2024-01-11T07:09:23.559 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 158859264 bytes via buffered write, may repeat 2024-01-11T07:09:23.559 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 159907840 bytes via buffered write, may repeat 2024-01-11T07:09:23.559 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 160956416 bytes via buffered write, may repeat 2024-01-11T07:09:23.559 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 162004992 bytes via buffered write, may repeat 2024-01-11T07:09:23.602 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 163053568 bytes via buffered write, may repeat 2024-01-11T07:09:23.602 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 164102144 bytes via buffered write, may repeat 2024-01-11T07:09:23.602 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 165150720 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 166199296 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 167247872 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 168296448 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 169345024 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 170393600 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 171442176 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 172490752 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 173539328 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 174587904 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 175636480 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 176685056 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 177733632 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 178782208 bytes via buffered write, may repeat 2024-01-11T07:09:23.603 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 179830784 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 180879360 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 181927936 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 182976512 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 184025088 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 185073664 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 186122240 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 187170816 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 188219392 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 189267968 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 190316544 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 191365120 bytes via buffered write, may repeat 2024-01-11T07:09:23.604 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 192413696 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 193462272 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 194510848 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 195559424 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 196608000 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 197656576 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 198705152 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 199753728 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 200802304 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 201850880 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 202899456 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 203948032 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 204996608 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 206045184 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 207093760 bytes via buffered write, may repeat 2024-01-11T07:09:23.605 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 208142336 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 209190912 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 210239488 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 211288064 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 212336640 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 213385216 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 214433792 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 215482368 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 216530944 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 217579520 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 218628096 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 219676672 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 220725248 bytes via buffered write, may repeat 2024-01-11T07:09:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 221773824 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 222822400 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 223870976 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 224919552 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 225968128 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 227016704 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 228065280 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 229113856 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 230162432 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 231211008 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 232259584 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 233308160 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 234356736 bytes via buffered write, may repeat 2024-01-11T07:09:23.607 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 235405312 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 236453888 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 237502464 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 238551040 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 239599616 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 240648192 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 241696768 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 242745344 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 243793920 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 244842496 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 245891072 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 246939648 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 247988224 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 249036800 bytes via buffered write, may repeat 2024-01-11T07:09:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 250085376 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 251133952 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 252182528 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 253231104 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 254279680 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 255328256 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 256376832 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 257425408 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 258473984 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 259522560 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 260571136 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 261619712 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 262668288 bytes via buffered write, may repeat 2024-01-11T07:09:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 263716864 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 264765440 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 265814016 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 266862592 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 267911168 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 268959744 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 270008320 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 271056896 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 272105472 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 273154048 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 274202624 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 275251200 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 276299776 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 277348352 bytes via buffered write, may repeat 2024-01-11T07:09:23.610 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 278396928 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 279445504 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 280494080 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 281542656 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 282591232 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 283639808 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 284688384 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 285736960 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 286785536 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 287834112 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 288882688 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 289931264 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 290979840 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 292028416 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 293076992 bytes via buffered write, may repeat 2024-01-11T07:09:23.611 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 294125568 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 295174144 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 296222720 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 297271296 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 298319872 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 299368448 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 300417024 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 301465600 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 302514176 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 303562752 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 304611328 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 305659904 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 306708480 bytes via buffered write, may repeat 2024-01-11T07:09:23.612 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 307757056 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 308805632 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 309854208 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 310902784 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 311951360 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 312999936 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 314048512 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 315097088 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 316145664 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 317194240 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 318242816 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 319291392 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 320339968 bytes via buffered write, may repeat 2024-01-11T07:09:23.613 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 321388544 bytes via buffered write, may repeat 2024-01-11T07:09:23.614 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 322437120 bytes via buffered write, may repeat 2024-01-11T07:09:23.614 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 323485696 bytes via buffered write, may repeat 2024-01-11T07:09:23.614 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 324534272 bytes via buffered write, may repeat 2024-01-11T07:09:23.656 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 325582848 bytes via buffered write, may repeat 2024-01-11T07:09:23.656 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 326631424 bytes via buffered write, may repeat 2024-01-11T07:09:23.656 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 327680000 bytes via buffered write, may repeat 2024-01-11T07:09:23.656 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 328728576 bytes via buffered write, may repeat 2024-01-11T07:09:23.656 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 329777152 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 330825728 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 331874304 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 332922880 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 333971456 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 335020032 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 336068608 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 337117184 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 338165760 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 339214336 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 340262912 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 341311488 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 342360064 bytes via buffered write, may repeat 2024-01-11T07:09:23.657 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 343408640 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 344457216 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 345505792 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 346554368 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 347602944 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 348651520 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 349700096 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 350748672 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 351797248 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 352845824 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 353894400 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 354942976 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 355991552 bytes via buffered write, may repeat 2024-01-11T07:09:23.658 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 357040128 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 358088704 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 359137280 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 360185856 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 361234432 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 362283008 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 363331584 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 364380160 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 365428736 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 366477312 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 367525888 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 368574464 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 369623040 bytes via buffered write, may repeat 2024-01-11T07:09:23.659 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 370671616 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 371720192 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 372768768 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 373817344 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 374865920 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 375914496 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 376963072 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 378011648 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 379060224 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 380108800 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 381157376 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 382205952 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 383254528 bytes via buffered write, may repeat 2024-01-11T07:09:23.660 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 384303104 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 385351680 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 386400256 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 387448832 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 388497408 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 389545984 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 390594560 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 391643136 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 392691712 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 393740288 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 394788864 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 395837440 bytes via buffered write, may repeat 2024-01-11T07:09:23.661 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 396886016 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 397934592 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 398983168 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 400031744 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 401080320 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 402128896 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 403177472 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 404226048 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 405274624 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 406323200 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 407371776 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 408420352 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 409468928 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 410517504 bytes via buffered write, may repeat 2024-01-11T07:09:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 411566080 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 412614656 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 413663232 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 414711808 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 415760384 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 416808960 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 417857536 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 418906112 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 419954688 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 421003264 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 422051840 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 423100416 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 424148992 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 425197568 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 426246144 bytes via buffered write, may repeat 2024-01-11T07:09:23.663 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 427294720 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 428343296 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 429391872 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 430440448 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 431489024 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 432537600 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 433586176 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 434634752 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 435683328 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 436731904 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 437780480 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 438829056 bytes via buffered write, may repeat 2024-01-11T07:09:23.664 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 439877632 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 440926208 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 441974784 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 443023360 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 444071936 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 445120512 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 446169088 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 447217664 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 448266240 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 449314816 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 450363392 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 451411968 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 452460544 bytes via buffered write, may repeat 2024-01-11T07:09:23.665 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 453509120 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 454557696 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 455606272 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 456654848 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 457703424 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 458752000 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 459800576 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 460849152 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 461897728 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 462946304 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 463994880 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 465043456 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 466092032 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 467140608 bytes via buffered write, may repeat 2024-01-11T07:09:23.666 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 468189184 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 469237760 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 470286336 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 471334912 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 472383488 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 473432064 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 474480640 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 475529216 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 476577792 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 477626368 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 478674944 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 479723520 bytes via buffered write, may repeat 2024-01-11T07:09:23.667 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 480772096 bytes via buffered write, may repeat 2024-01-11T07:09:23.668 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 481820672 bytes via buffered write, may repeat 2024-01-11T07:09:23.668 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 482869248 bytes via buffered write, may repeat 2024-01-11T07:09:23.668 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 483917824 bytes via buffered write, may repeat 2024-01-11T07:09:23.668 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 484966400 bytes via buffered write, may repeat 2024-01-11T07:09:23.668 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 486014976 bytes via buffered write, may repeat 2024-01-11T07:09:23.668 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 487063552 bytes via buffered write, may repeat 2024-01-11T07:09:26.354 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:09:26.355+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-01-11T07:09:30.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:09:30.562 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:09:32.186 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:09:32.186+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-01-11T07:09:59.999 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:09:59.999+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 backfillfull osd(s); 4 full osd(s); 3 pool(s) full 2024-01-11T07:10:00.640 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:10:00.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:10:30.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:10:30.732 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:10:39.088 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 488112128 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 489160704 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 490209280 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 491257856 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 492306432 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 493355008 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 494403584 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 495452160 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 496500736 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 497549312 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 498597888 bytes via buffered write, may repeat 2024-01-11T07:10:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 499646464 bytes via buffered write, may repeat 2024-01-11T07:10:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 500695040 bytes via buffered write, may repeat 2024-01-11T07:10:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:done writing 500695040 bytes 2024-01-11T07:10:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:successfully fsync'ed prior to getting full state reported 2024-01-11T07:10:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:starting buffered write 2024-01-11T07:10:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:10:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:10:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:10:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-01-11T07:10:39.268 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:10:39.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:10:39.319 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:10:39.364 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:10:39.365 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:10:39.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:10:39.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:10:39.423 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:10:39.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:10:39.424 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:10:39.446 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:10:39.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:10:39.447 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:10:39.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:10:39.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:10:39.493 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:10:39.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:10:39.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.651+0000 7eff849a8700 1 Processor -- start 2024-01-11T07:10:39.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.652+0000 7eff849a8700 1 -- start start 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff849a8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff80159c90 0x7eff800b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff849a8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff80069320 0x7eff800b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff849a8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff800ce9b0 con 0x7eff8015a060 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff849a8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff800cf530 con 0x7eff80069320 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff849a8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff800cf9c0 con 0x7eff80159c90 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff80069320 0x7eff800b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7eff8015a060 0x7eff800b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50228/0 (socket says 172.21.15.7:50228) 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7dd9b700 1 -- 172.21.15.7:0/1094547535 learned_addr learned my addr 172.21.15.7:0/1094547535 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7e59c700 1 --2- 172.21.15.7:0/1094547535 >> v2:172.21.15.113:3300/0 conn(0x7eff80159c90 0x7eff800b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7ed9d700 1 -- 172.21.15.7:0/1094547535 >> v2:172.21.15.113:3300/0 conn(0x7eff80159c90 msgr2=0x7eff800b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:39.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7ed9d700 1 --2- 172.21.15.7:0/1094547535 >> v2:172.21.15.113:3300/0 conn(0x7eff80159c90 0x7eff800b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7ed9d700 1 -- 172.21.15.7:0/1094547535 >> v1:172.21.15.7:6789/0 conn(0x7eff8015a060 legacy=0x7eff800b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7ed9d700 1 -- 172.21.15.7:0/1094547535 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800cec00 con 0x7eff80069320 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7d59a700 1 -- 172.21.15.7:0/1094547535 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504298519 0 0) 0x7eff800ce9b0 con 0x7eff8015a060 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.653+0000 7eff7ed9d700 1 --2- 172.21.15.7:0/1094547535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80069320 0x7eff800b3520 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7eff8005f2e0 tx=0x7eff78009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97fbd80dce88558 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff7d59a700 1 -- 172.21.15.7:0/1094547535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800e070 con 0x7eff80069320 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff7d59a700 1 -- 172.21.15.7:0/1094547535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7800bb60 con 0x7eff80069320 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff7d59a700 1 -- 172.21.15.7:0/1094547535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78012430 con 0x7eff80069320 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff849a8700 1 -- 172.21.15.7:0/1094547535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80069320 msgr2=0x7eff800b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff849a8700 1 --2- 172.21.15.7:0/1094547535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80069320 0x7eff800b3520 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7eff8005f2e0 tx=0x7eff78009780 comp rx=0 tx=0).stop 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff849a8700 1 -- 172.21.15.7:0/1094547535 shutdown_connections 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff849a8700 1 --2- 172.21.15.7:0/1094547535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80069320 0x7eff800b3520 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff849a8700 1 --2- 172.21.15.7:0/1094547535 >> v2:172.21.15.113:3300/0 conn(0x7eff80159c90 0x7eff800b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff849a8700 1 -- 172.21.15.7:0/1094547535 >> 172.21.15.7:0/1094547535 conn(0x7eff80071100 msgr2=0x7eff8006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff849a8700 1 -- 172.21.15.7:0/1094547535 shutdown_connections 2024-01-11T07:10:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.654+0000 7eff849a8700 1 -- 172.21.15.7:0/1094547535 wait complete. 2024-01-11T07:10:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff849a8700 1 Processor -- start 2024-01-11T07:10:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff849a8700 1 -- start start 2024-01-11T07:10:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff849a8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff80069320 0x7eff80155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff849a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80159c90 0x7eff80169e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff849a8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff8015a060 0x7eff800af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff849a8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8016b4a0 con 0x7eff80159c90 2024-01-11T07:10:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff849a8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff80155780 con 0x7eff80069320 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff849a8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8016a890 con 0x7eff8015a060 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.656+0000 7eff7dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80159c90 0x7eff80169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80159c90 0x7eff80169e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35164/0 (socket says 172.21.15.7:35164) 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7dd9b700 1 -- 172.21.15.7:0/2760378033 learned_addr learned my addr 172.21.15.7:0/2760378033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7ed9d700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff8015a060 0x7eff800af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7e59c700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff80069320 0x7eff80155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7dd9b700 1 -- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff8015a060 msgr2=0x7eff800af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7dd9b700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff8015a060 0x7eff800af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7dd9b700 1 -- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff80069320 msgr2=0x7eff80155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7dd9b700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff80069320 0x7eff80155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7dd9b700 1 -- 172.21.15.7:0/2760378033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78009a20 con 0x7eff80159c90 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff7dd9b700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80159c90 0x7eff80169e10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c5e0 tx=0x7eff74001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff6e7fc700 1 -- 172.21.15.7:0/2760378033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74005b30 con 0x7eff80159c90 2024-01-11T07:10:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff6e7fc700 1 -- 172.21.15.7:0/2760378033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff74005c90 con 0x7eff80159c90 2024-01-11T07:10:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8016ab70 con 0x7eff80159c90 2024-01-11T07:10:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff6e7fc700 1 -- 172.21.15.7:0/2760378033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff740072e0 con 0x7eff80159c90 2024-01-11T07:10:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.657+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8016b130 con 0x7eff80159c90 2024-01-11T07:10:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.658+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8016b650 con 0x7eff80159c90 2024-01-11T07:10:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.659+0000 7eff6e7fc700 1 -- 172.21.15.7:0/2760378033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7eff7401b3b0 con 0x7eff80159c90 2024-01-11T07:10:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.659+0000 7eff6e7fc700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.659+0000 7eff6e7fc700 1 -- 172.21.15.7:0/2760378033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff74083a30 con 0x7eff80159c90 2024-01-11T07:10:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.659+0000 7eff6e7fc700 1 -- 172.21.15.7:0/2760378033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7eff6006ea50 con 0x7eff80159c90 2024-01-11T07:10:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.659+0000 7eff7e59c700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.660+0000 7eff7e59c700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff60066f30 0x7eff600693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff70009d40 tx=0x7eff70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:39.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.664+0000 7eff6e7fc700 1 -- 172.21.15.7:0/2760378033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff7404b5a0 con 0x7eff80159c90 2024-01-11T07:10:39.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.911+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff800b1020 con 0x7eff80159c90 2024-01-11T07:10:39.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.912+0000 7eff6e7fc700 1 -- 172.21.15.7:0/2760378033 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4594 (secure 0 0 0) 0x7eff74017070 con 0x7eff80159c90 2024-01-11T07:10:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:39.913 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1734166534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1734166534},{"type":"v1","addr":"172.21.15.113:6836","nonce":1734166534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3195231529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3195231529},{"type":"v1","addr":"172.21.15.113:6837","nonce":3195231529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/338342475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":338342475},{"type":"v1","addr":"172.21.15.7:6834","nonce":338342475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:09:08.717048+0000","modified":"2024-01-11T07:09:12.324714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1812078530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1812078530},{"type":"v1","addr":"172.21.15.7:6835","nonce":1812078530}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff60066f30 msgr2=0x7eff600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff60066f30 0x7eff600693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff70009d40 tx=0x7eff70008040 comp rx=0 tx=0).stop 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80159c90 msgr2=0x7eff80169e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80159c90 0x7eff80169e10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c5e0 tx=0x7eff74001070 comp rx=0 tx=0).stop 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 shutdown_connections 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff8015a060 0x7eff800af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff80069320 0x7eff80155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 --2- 172.21.15.7:0/2760378033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff80159c90 0x7eff80169e10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:39.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 >> 172.21.15.7:0/2760378033 conn(0x7eff80071100 msgr2=0x7eff801570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:39.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 shutdown_connections 2024-01-11T07:10:39.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:39.914+0000 7eff849a8700 1 -- 172.21.15.7:0/2760378033 wait complete. 2024-01-11T07:10:39.916 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-11T07:10:39.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:10:40.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.060+0000 7f18e6880700 1 Processor -- start 2024-01-11T07:10:40.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.060+0000 7f18e6880700 1 -- start start 2024-01-11T07:10:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18e6880700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18e01567f0 0x7f18e00c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18e6880700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e00c8c00 0x7f18e006a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18e6880700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e006e0d0 con 0x7f18e006f170 2024-01-11T07:10:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18e6880700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e00ccb00 con 0x7f18e01567f0 2024-01-11T07:10:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18e6880700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e006e3b0 con 0x7f18e00c8c00 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18e4e1d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f18e006f170 0x7f18e015a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50242/0 (socket says 172.21.15.7:50242) 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18e4e1d700 1 -- 172.21.15.7:0/1762028775 learned_addr learned my addr 172.21.15.7:0/1762028775 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18df7fe700 1 --2- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e00c8c00 0x7f18e006a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18dffff700 1 --2- 172.21.15.7:0/1762028775 >> v2:172.21.15.113:3300/0 conn(0x7f18e01567f0 0x7f18e00c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18deffd700 1 -- 172.21.15.7:0/1762028775 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052519871 0 0) 0x7f18e006e0d0 con 0x7f18e006f170 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18deffd700 1 -- 172.21.15.7:0/1762028775 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c8003570 con 0x7f18e006f170 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18dffff700 1 -- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e00c8c00 msgr2=0x7f18e006a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18dffff700 1 --2- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e00c8c00 0x7f18e006a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18dffff700 1 -- 172.21.15.7:0/1762028775 >> v1:172.21.15.7:6789/0 conn(0x7f18e006f170 legacy=0x7f18e015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18dffff700 1 -- 172.21.15.7:0/1762028775 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e006e690 con 0x7f18e01567f0 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.061+0000 7f18df7fe700 1 --2- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e00c8c00 0x7f18e006a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.062+0000 7f18dffff700 1 --2- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 0x7f18e00c86c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f18d00098e0 tx=0x7f18d00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac4c792d70200d94 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.062+0000 7f18deffd700 1 -- 172.21.15.7:0/1762028775 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0011630 con 0x7f18e01567f0 2024-01-11T07:10:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.062+0000 7f18deffd700 1 -- 172.21.15.7:0/1762028775 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d0011790 con 0x7f18e01567f0 2024-01-11T07:10:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.062+0000 7f18deffd700 1 -- 172.21.15.7:0/1762028775 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0011b30 con 0x7f18e01567f0 2024-01-11T07:10:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.062+0000 7f18e6880700 1 -- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 msgr2=0x7f18e00c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.062+0000 7f18e6880700 1 --2- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 0x7f18e00c86c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f18d00098e0 tx=0x7f18d00091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.063+0000 7f18e6880700 1 -- 172.21.15.7:0/1762028775 shutdown_connections 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.063+0000 7f18e6880700 1 --2- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 0x7f18e00c86c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.063+0000 7f18e6880700 1 --2- 172.21.15.7:0/1762028775 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e00c8c00 0x7f18e006a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.063+0000 7f18e6880700 1 -- 172.21.15.7:0/1762028775 >> 172.21.15.7:0/1762028775 conn(0x7f18e00cd090 msgr2=0x7f18e00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.063+0000 7f18e6880700 1 -- 172.21.15.7:0/1762028775 shutdown_connections 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.063+0000 7f18e6880700 1 -- 172.21.15.7:0/1762028775 wait complete. 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.064+0000 7f18e6880700 1 Processor -- start 2024-01-11T07:10:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.064+0000 7f18e6880700 1 -- start start 2024-01-11T07:10:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18e6880700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e006f170 0x7f18e00c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18e6880700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18e00c8c00 0x7f18e0154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18e6880700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 0x7f18e00a8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18e6880700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e0155320 con 0x7f18e00c8c00 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18e6880700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00a79d0 con 0x7f18e01567f0 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18e6880700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e00c7460 con 0x7f18e006f170 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18e00c8c00 0x7f18e0154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18e00c8c00 0x7f18e0154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35174/0 (socket says 172.21.15.7:35174) 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 -- 172.21.15.7:0/3210894762 learned_addr learned my addr 172.21.15.7:0/3210894762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18dffff700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e006f170 0x7f18e00c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18e4e1d700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 0x7f18e00a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 -- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e006f170 msgr2=0x7f18e00c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e006f170 0x7f18e00c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 -- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 msgr2=0x7f18e00a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 0x7f18e00a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 -- 172.21.15.7:0/3210894762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4009420 con 0x7f18e00c8c00 2024-01-11T07:10:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.065+0000 7f18df7fe700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18e00c8c00 0x7f18e0154670 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f18d400ec80 tx=0x7f18d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.066+0000 7f18c7fff700 1 -- 172.21.15.7:0/3210894762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400c920 con 0x7f18e00c8c00 2024-01-11T07:10:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.066+0000 7f18c7fff700 1 -- 172.21.15.7:0/3210894762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d400ca80 con 0x7f18e00c8c00 2024-01-11T07:10:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.066+0000 7f18c7fff700 1 -- 172.21.15.7:0/3210894762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400ce00 con 0x7f18e00c8c00 2024-01-11T07:10:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.066+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d0009420 con 0x7f18e00c8c00 2024-01-11T07:10:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.066+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e00c41e0 con 0x7f18e00c8c00 2024-01-11T07:10:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.067+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e004c3c0 con 0x7f18e00c8c00 2024-01-11T07:10:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.067+0000 7f18c7fff700 1 -- 172.21.15.7:0/3210894762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f18d4013680 con 0x7f18e00c8c00 2024-01-11T07:10:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.068+0000 7f18c7fff700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18cc066f00 0x7f18cc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.068+0000 7f18c7fff700 1 -- 172.21.15.7:0/3210894762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f18d4084c20 con 0x7f18e00c8c00 2024-01-11T07:10:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.068+0000 7f18dffff700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18cc066f00 0x7f18cc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.068+0000 7f18c7fff700 1 -- 172.21.15.7:0/3210894762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f18cc06ea20 con 0x7f18e00c8c00 2024-01-11T07:10:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.068+0000 7f18dffff700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18cc066f00 0x7f18cc0693b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f18d00091e0 tx=0x7f18d0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.072+0000 7f18c7fff700 1 -- 172.21.15.7:0/3210894762 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18d4017060 con 0x7f18e00c8c00 2024-01-11T07:10:40.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.279+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18e006e970 con 0x7f18e00c8c00 2024-01-11T07:10:40.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.280+0000 7f18c7fff700 1 -- 172.21.15.7:0/3210894762 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+15360 (secure 0 0 0) 0x7f18d4070c10 con 0x7f18e00c8c00 2024-01-11T07:10:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:40.281 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:09:30.370524+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:09:06.415283+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-11T07:09:07.694952+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18cc066f00 msgr2=0x7f18cc0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18cc066f00 0x7f18cc0693b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f18d00091e0 tx=0x7f18d0017000 comp rx=0 tx=0).stop 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18e00c8c00 msgr2=0x7f18e0154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18e00c8c00 0x7f18e0154670 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f18d400ec80 tx=0x7f18d40091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 shutdown_connections 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18cc066f00 0x7f18cc0693b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e01567f0 0x7f18e00a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18e006f170 0x7f18e00c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 --2- 172.21.15.7:0/3210894762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18e00c8c00 0x7f18e0154670 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 >> 172.21.15.7:0/3210894762 conn(0x7f18e00cd090 msgr2=0x7f18e006aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 shutdown_connections 2024-01-11T07:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.283+0000 7f18e6880700 1 -- 172.21.15.7:0/3210894762 wait complete. 2024-01-11T07:10:40.297 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:09:06.415283+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:09:07.694952+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:10:40.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:10:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.490+0000 7f5891fd7700 1 Processor -- start 2024-01-11T07:10:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.490+0000 7f5891fd7700 1 -- start start 2024-01-11T07:10:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.491+0000 7f5891fd7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c063230 0x7f588c063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.491+0000 7f5891fd7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f588c0ac5b0 0x7f588c0aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.491+0000 7f5891fd7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c0ab100 con 0x7f588c15a280 2024-01-11T07:10:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.491+0000 7f5891fd7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c15d880 con 0x7f588c0ac5b0 2024-01-11T07:10:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.491+0000 7f5891fd7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c06c5b0 con 0x7f588c063230 2024-01-11T07:10:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f5882ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f588c15a280 0x7f588c0aa9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50248/0 (socket says 172.21.15.7:50248) 2024-01-11T07:10:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f5882ffd700 1 -- 172.21.15.7:0/246162129 learned_addr learned my addr 172.21.15.7:0/246162129 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588b7fe700 1 --2- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c063230 0x7f588c063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588bfff700 1 --2- 172.21.15.7:0/246162129 >> v2:172.21.15.113:3300/0 conn(0x7f588c0ac5b0 0x7f588c0aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588affd700 1 -- 172.21.15.7:0/246162129 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930090814 0 0) 0x7f588c0ab100 con 0x7f588c15a280 2024-01-11T07:10:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588affd700 1 -- 172.21.15.7:0/246162129 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5878003570 con 0x7f588c15a280 2024-01-11T07:10:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588bfff700 1 -- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c063230 msgr2=0x7f588c063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588bfff700 1 --2- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c063230 0x7f588c063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588bfff700 1 -- 172.21.15.7:0/246162129 >> v1:172.21.15.7:6789/0 conn(0x7f588c15a280 legacy=0x7f588c0aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588affd700 1 -- 172.21.15.7:0/246162129 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901248388 0 0) 0x7f5878003570 con 0x7f588c15a280 2024-01-11T07:10:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588bfff700 1 -- 172.21.15.7:0/246162129 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c063b40 con 0x7f588c0ac5b0 2024-01-11T07:10:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588b7fe700 1 --2- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c063230 0x7f588c063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.492+0000 7f588bfff700 1 --2- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c0ac5b0 0x7f588c0aca30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e60 tx=0x7f5884009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ced6e78ec2a03f58 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f588affd700 1 -- 172.21.15.7:0/246162129 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588400e070 con 0x7f588c0ac5b0 2024-01-11T07:10:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f588affd700 1 -- 172.21.15.7:0/246162129 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588400b7a0 con 0x7f588c0ac5b0 2024-01-11T07:10:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f588affd700 1 -- 172.21.15.7:0/246162129 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874001470 con 0x7f588c0ac5b0 2024-01-11T07:10:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f5891fd7700 1 -- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c0ac5b0 msgr2=0x7f588c0aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f5891fd7700 1 --2- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c0ac5b0 0x7f588c0aca30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e60 tx=0x7f5884009780 comp rx=0 tx=0).stop 2024-01-11T07:10:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f5891fd7700 1 -- 172.21.15.7:0/246162129 shutdown_connections 2024-01-11T07:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f5891fd7700 1 --2- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c0ac5b0 0x7f588c0aca30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f5891fd7700 1 --2- 172.21.15.7:0/246162129 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c063230 0x7f588c063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f5891fd7700 1 -- 172.21.15.7:0/246162129 >> 172.21.15.7:0/246162129 conn(0x7f588c0680e0 msgr2=0x7f588c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f5891fd7700 1 -- 172.21.15.7:0/246162129 shutdown_connections 2024-01-11T07:10:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.493+0000 7f5891fd7700 1 -- 172.21.15.7:0/246162129 wait complete. 2024-01-11T07:10:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.494+0000 7f5891fd7700 1 Processor -- start 2024-01-11T07:10:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.495+0000 7f5891fd7700 1 -- start start 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.495+0000 7f5891fd7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f588c063230 0x7f588c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.495+0000 7f5891fd7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c0ac5b0 0x7f588c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5891fd7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c15a280 0x7f588c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5891fd7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c16a0b0 con 0x7f588c063230 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5891fd7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c16aed0 con 0x7f588c15a280 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5891fd7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c16a2c0 con 0x7f588c0ac5b0 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f588c063230 0x7f588c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f588c063230 0x7f588c0b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35176/0 (socket says 172.21.15.7:35176) 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 -- 172.21.15.7:0/2136343442 learned_addr learned my addr 172.21.15.7:0/2136343442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5882ffd700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c0ac5b0 0x7f588c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588bfff700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c15a280 0x7f588c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 -- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c0ac5b0 msgr2=0x7f588c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c0ac5b0 0x7f588c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 -- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c15a280 msgr2=0x7f588c0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c15a280 0x7f588c0adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 -- 172.21.15.7:0/2136343442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009420 con 0x7f588c063230 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f588b7fe700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f588c063230 0x7f588c0b2810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f587c00ec80 tx=0x7f587c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5883fff700 1 -- 172.21.15.7:0/2136343442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00c850 con 0x7f588c063230 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5883fff700 1 -- 172.21.15.7:0/2136343442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00c9b0 con 0x7f588c063230 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5884009a20 con 0x7f588c063230 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5883fff700 1 -- 172.21.15.7:0/2136343442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00cd30 con 0x7f588c063230 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.496+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c16a880 con 0x7f588c063230 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.498+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c06c3f0 con 0x7f588c063230 2024-01-11T07:10:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.498+0000 7f5883fff700 1 -- 172.21.15.7:0/2136343442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f587c013680 con 0x7f588c063230 2024-01-11T07:10:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.498+0000 7f5883fff700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f586c066f30 0x7f586c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.498+0000 7f5883fff700 1 -- 172.21.15.7:0/2136343442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f587c084af0 con 0x7f588c063230 2024-01-11T07:10:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.499+0000 7f5883fff700 1 -- 172.21.15.7:0/2136343442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f586c06ea50 con 0x7f588c063230 2024-01-11T07:10:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.499+0000 7f5882ffd700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f586c066f30 0x7f586c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.499+0000 7f5882ffd700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f586c066f30 0x7f586c0693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5874001ed0 tx=0x7f58740019d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.503+0000 7f5883fff700 1 -- 172.21.15.7:0/2136343442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f587c017060 con 0x7f588c063230 2024-01-11T07:10:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.704+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f588c04c3c0 con 0x7f588c063230 2024-01-11T07:10:40.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.704+0000 7f5883fff700 1 -- 172.21.15.7:0/2136343442 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f587c021020 con 0x7f588c063230 2024-01-11T07:10:40.705 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:10:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f586c066f30 msgr2=0x7f586c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f586c066f30 0x7f586c0693e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5874001ed0 tx=0x7f58740019d0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f588c063230 msgr2=0x7f588c0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f588c063230 0x7f588c0b2810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f587c00ec80 tx=0x7f587c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 shutdown_connections 2024-01-11T07:10:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f586c066f30 0x7f586c0693e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f588c15a280 0x7f588c0adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f588c063230 0x7f588c0b2810 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 --2- 172.21.15.7:0/2136343442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f588c0ac5b0 0x7f588c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 >> 172.21.15.7:0/2136343442 conn(0x7f588c0680e0 msgr2=0x7f588c15b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 shutdown_connections 2024-01-11T07:10:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.706+0000 7f5891fd7700 1 -- 172.21.15.7:0/2136343442 wait complete. 2024-01-11T07:10:40.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:10:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.854+0000 7fde04dc4700 1 Processor -- start 2024-01-11T07:10:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.854+0000 7fde04dc4700 1 -- start start 2024-01-11T07:10:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.855+0000 7fde04dc4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde00159d70 0x7fde0015a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.855+0000 7fde04dc4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde000cf0d0 0x7fde000c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.855+0000 7fde04dc4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde00067840 con 0x7fde000cf540 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.855+0000 7fde04dc4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde000cb0c0 con 0x7fde00159d70 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.855+0000 7fde04dc4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000ca820 con 0x7fde000cf0d0 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.855+0000 7fddfed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fde000cf540 0x7fde000c9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50260/0 (socket says 172.21.15.7:50260) 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.855+0000 7fddfed9d700 1 -- 172.21.15.7:0/2639195682 learned_addr learned my addr 172.21.15.7:0/2639195682 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfdd9b700 1 --2- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde000cf0d0 0x7fde000c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfe59c700 1 --2- 172.21.15.7:0/2639195682 >> v2:172.21.15.113:3300/0 conn(0x7fde00159d70 0x7fde0015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfd59a700 1 -- 172.21.15.7:0/2639195682 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137407657 0 0) 0x7fde00067840 con 0x7fde000cf540 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfd59a700 1 -- 172.21.15.7:0/2639195682 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdde8003570 con 0x7fde000cf540 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfe59c700 1 -- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde000cf0d0 msgr2=0x7fde000c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfe59c700 1 --2- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde000cf0d0 0x7fde000c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfe59c700 1 -- 172.21.15.7:0/2639195682 >> v1:172.21.15.7:6789/0 conn(0x7fde000cf540 legacy=0x7fde000c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfe59c700 1 -- 172.21.15.7:0/2639195682 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000cab00 con 0x7fde00159d70 2024-01-11T07:10:40.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfdd9b700 1 --2- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde000cf0d0 0x7fde000c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfe59c700 1 --2- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde00159d70 0x7fde0015a140 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fddf00098d0 tx=0x7fddf0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d792fefe88ffad9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfd59a700 1 -- 172.21.15.7:0/2639195682 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0005c10 con 0x7fde00159d70 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfd59a700 1 -- 172.21.15.7:0/2639195682 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf0005d70 con 0x7fde00159d70 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.856+0000 7fddfd59a700 1 -- 172.21.15.7:0/2639195682 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0006260 con 0x7fde00159d70 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.857+0000 7fde04dc4700 1 -- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde00159d70 msgr2=0x7fde0015a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.857+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde00159d70 0x7fde0015a140 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fddf00098d0 tx=0x7fddf0009120 comp rx=0 tx=0).stop 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.857+0000 7fde04dc4700 1 -- 172.21.15.7:0/2639195682 shutdown_connections 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.857+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde00159d70 0x7fde0015a140 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.857+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2639195682 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde000cf0d0 0x7fde000c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.857+0000 7fde04dc4700 1 -- 172.21.15.7:0/2639195682 >> 172.21.15.7:0/2639195682 conn(0x7fde00097a50 msgr2=0x7fde000ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.857+0000 7fde04dc4700 1 -- 172.21.15.7:0/2639195682 shutdown_connections 2024-01-11T07:10:40.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.857+0000 7fde04dc4700 1 -- 172.21.15.7:0/2639195682 wait complete. 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.859+0000 7fde04dc4700 1 Processor -- start 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.859+0000 7fde04dc4700 1 -- start start 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fde04dc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000cf0d0 0x7fde0016d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fde04dc4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde000cf540 0x7fde0016a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fde04dc4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00159d70 0x7fde0009b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fde04dc4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde001577d0 con 0x7fde000cf0d0 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fde04dc4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0009a0b0 con 0x7fde000cf540 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fde04dc4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000b2870 con 0x7fde00159d70 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000cf0d0 0x7fde0016d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000cf0d0 0x7fde0016d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35184/0 (socket says 172.21.15.7:35184) 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 -- 172.21.15.7:0/2886461130 learned_addr learned my addr 172.21.15.7:0/2886461130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfdd9b700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde000cf540 0x7fde0016a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfed9d700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00159d70 0x7fde0009b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 -- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00159d70 msgr2=0x7fde0009b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00159d70 0x7fde0009b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 -- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde000cf540 msgr2=0x7fde0016a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde000cf540 0x7fde0016a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 -- 172.21.15.7:0/2886461130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4009420 con 0x7fde000cf0d0 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.860+0000 7fddfe59c700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000cf0d0 0x7fde0016d190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009ec0 tx=0x7fddf0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.861+0000 7fddee7fc700 1 -- 172.21.15.7:0/2886461130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf001d780 con 0x7fde000cf0d0 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.861+0000 7fddee7fc700 1 -- 172.21.15.7:0/2886461130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf001d8e0 con 0x7fde000cf0d0 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.861+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf00093c0 con 0x7fde000cf0d0 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.861+0000 7fddee7fc700 1 -- 172.21.15.7:0/2886461130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf001dca0 con 0x7fde000cf0d0 2024-01-11T07:10:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.861+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde000b2e30 con 0x7fde000cf0d0 2024-01-11T07:10:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.862+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0004c3c0 con 0x7fde000cf0d0 2024-01-11T07:10:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.862+0000 7fddee7fc700 1 -- 172.21.15.7:0/2886461130 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fddf001b690 con 0x7fde000cf0d0 2024-01-11T07:10:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.863+0000 7fddee7fc700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde0066f30 0x7fdde00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.863+0000 7fddee7fc700 1 -- 172.21.15.7:0/2886461130 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fddf0086470 con 0x7fde000cf0d0 2024-01-11T07:10:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.863+0000 7fddfdd9b700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde0066f30 0x7fdde00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.863+0000 7fddee7fc700 1 -- 172.21.15.7:0/2886461130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fdde006ea50 con 0x7fde000cf0d0 2024-01-11T07:10:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.863+0000 7fddfdd9b700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde0066f30 0x7fdde00693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fde00069580 tx=0x7fddf40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:40.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:40.867+0000 7fddee7fc700 1 -- 172.21.15.7:0/2886461130 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddf0053640 con 0x7fde000cf0d0 2024-01-11T07:10:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.068+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde000cade0 con 0x7fde000cf0d0 2024-01-11T07:10:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.069+0000 7fddee7fc700 1 -- 172.21.15.7:0/2886461130 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7fddf0056c60 con 0x7fde000cf0d0 2024-01-11T07:10:41.069 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde0066f30 msgr2=0x7fdde00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde0066f30 0x7fdde00693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fde00069580 tx=0x7fddf40091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000cf0d0 msgr2=0x7fde0016d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000cf0d0 0x7fde0016d190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009ec0 tx=0x7fddf0016000 comp rx=0 tx=0).stop 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 shutdown_connections 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde0066f30 0x7fdde00693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00159d70 0x7fde0009b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000cf0d0 0x7fde0016d190 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 --2- 172.21.15.7:0/2886461130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde000cf540 0x7fde0016a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 >> 172.21.15.7:0/2886461130 conn(0x7fde00097a50 msgr2=0x7fde000c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 shutdown_connections 2024-01-11T07:10:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.071+0000 7fde04dc4700 1 -- 172.21.15.7:0/2886461130 wait complete. 2024-01-11T07:10:41.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:10:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.218+0000 7f42a75e3700 1 Processor -- start 2024-01-11T07:10:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.218+0000 7f42a75e3700 1 -- start start 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.219+0000 7f42a75e3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f42a0063940 0x7f42a015af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.219+0000 7f42a75e3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a015b4b0 con 0x7f42a01579a0 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.219+0000 7f42a75e3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a006bb60 con 0x7f42a006a9c0 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.219+0000 7f42a75e3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a006e8b0 con 0x7f42a0063940 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.219+0000 7f42a537f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f42a01579a0 0x7f42a006bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50276/0 (socket says 172.21.15.7:50276) 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.219+0000 7f42a537f700 1 -- 172.21.15.7:0/3056918706 learned_addr learned my addr 172.21.15.7:0/3056918706 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f42a5b80700 1 --2- 172.21.15.7:0/3056918706 >> v2:172.21.15.113:3300/0 conn(0x7f42a0063940 0x7f42a015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013618488 0 0) 0x7f429c001110 con 0x7f42a01579a0 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f428c003570 con 0x7f42a01579a0 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835683338 0 0) 0x7f428c003570 con 0x7f42a01579a0 2024-01-11T07:10:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 >> v2:172.21.15.113:3300/0 conn(0x7f42a0063940 msgr2=0x7f42a015af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 --2- 172.21.15.7:0/3056918706 >> v2:172.21.15.113:3300/0 conn(0x7f42a0063940 0x7f42a015af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 >> v1:172.21.15.113:6789/0 conn(0x7f42a006a9c0 legacy=0x7f42a0063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a006af50 con 0x7f42a01579a0 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f429c001600 con 0x7f42a01579a0 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 >> v1:172.21.15.7:6789/0 conn(0x7f42a01579a0 legacy=0x7f42a006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c003a60 0x7f428c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c004810 0x7f428c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c0051c0 0x7f428c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a006e8b0 con 0x7f428c004810 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c00c160 con 0x7f428c0051c0 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c00c320 con 0x7f428c003a60 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f42a4b7e700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c004810 0x7f428c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f42a537f700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c003a60 0x7f428c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f42a5b80700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c0051c0 0x7f428c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f42a4b7e700 1 -- 172.21.15.7:0/3056918706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c003a60 msgr2=0x7f428c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.220+0000 7f42a4b7e700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c003a60 0x7f428c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a4b7e700 1 -- 172.21.15.7:0/3056918706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c0051c0 msgr2=0x7f428c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a4b7e700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f428c0051c0 0x7f428c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a5b80700 1 -- 172.21.15.7:0/3056918706 reap_dead start 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a4b7e700 1 -- 172.21.15.7:0/3056918706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c003570 con 0x7f428c004810 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a4b7e700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c004810 0x7f428c004c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f42900018b0 tx=0x7f4290007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d149fe295d046f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290012420 con 0x7f428c004810 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42900125a0 con 0x7f428c004810 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f4297fff700 1 -- 172.21.15.7:0/3056918706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290012960 con 0x7f428c004810 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a75e3700 1 -- 172.21.15.7:0/3056918706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c004810 msgr2=0x7f428c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a75e3700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c004810 0x7f428c004c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f42900018b0 tx=0x7f4290007d30 comp rx=0 tx=0).stop 2024-01-11T07:10:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a75e3700 1 -- 172.21.15.7:0/3056918706 shutdown_connections 2024-01-11T07:10:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a75e3700 1 --2- 172.21.15.7:0/3056918706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f428c004810 0x7f428c004c80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.221+0000 7f42a75e3700 1 -- 172.21.15.7:0/3056918706 >> 172.21.15.7:0/3056918706 conn(0x7f42a00cd8f0 msgr2=0x7f42a00cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.222+0000 7f42a75e3700 1 -- 172.21.15.7:0/3056918706 shutdown_connections 2024-01-11T07:10:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.222+0000 7f42a75e3700 1 -- 172.21.15.7:0/3056918706 wait complete. 2024-01-11T07:10:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.223+0000 7f42a75e3700 1 Processor -- start 2024-01-11T07:10:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.223+0000 7f42a75e3700 1 -- start start 2024-01-11T07:10:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a75e3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c004810 0x7f42a0152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a75e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a00b35f0 0x7f42a016b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a75e3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42a016b660 0x7f42a0155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a75e3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00b1720 con 0x7f42a00b35f0 2024-01-11T07:10:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a75e3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00b0b10 con 0x7f42a016b660 2024-01-11T07:10:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a75e3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a00b0df0 con 0x7f428c004810 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a4b7e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a00b35f0 0x7f42a016b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a4b7e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a00b35f0 0x7f42a016b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35214/0 (socket says 172.21.15.7:35214) 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a4b7e700 1 -- 172.21.15.7:0/2705707872 learned_addr learned my addr 172.21.15.7:0/2705707872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a537f700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c004810 0x7f42a0152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a5b80700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42a016b660 0x7f42a0155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a4b7e700 1 -- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c004810 msgr2=0x7f42a0152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a4b7e700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c004810 0x7f42a0152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a4b7e700 1 -- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42a016b660 msgr2=0x7f42a0155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a4b7e700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42a016b660 0x7f42a0155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.224+0000 7f42a4b7e700 1 -- 172.21.15.7:0/2705707872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a00b10d0 con 0x7f42a00b35f0 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.225+0000 7f42a4b7e700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a00b35f0 0x7f42a016b120 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c130 tx=0x7f4290015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.225+0000 7f42957fa700 1 -- 172.21.15.7:0/2705707872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429001f6f0 con 0x7f42a00b35f0 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.225+0000 7f42957fa700 1 -- 172.21.15.7:0/2705707872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429001f850 con 0x7f42a00b35f0 2024-01-11T07:10:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.225+0000 7f42957fa700 1 -- 172.21.15.7:0/2705707872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429001fbf0 con 0x7f42a00b35f0 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.225+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a00b1720 con 0x7f42a00b35f0 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.225+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a00b1bb0 con 0x7f42a00b35f0 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.226+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a0064a40 con 0x7f42a00b35f0 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.227+0000 7f42957fa700 1 -- 172.21.15.7:0/2705707872 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4290032480 con 0x7f42a00b35f0 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.227+0000 7f42957fa700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.227+0000 7f42957fa700 1 -- 172.21.15.7:0/2705707872 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4290086010 con 0x7f42a00b35f0 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.228+0000 7f42957fa700 1 -- 172.21.15.7:0/2705707872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f428406ea50 con 0x7f42a00b35f0 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.228+0000 7f42a537f700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.228+0000 7f42a537f700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4284066f30 0x7f42840693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f429c000c40 tx=0x7f429c007a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:41.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.231+0000 7f42957fa700 1 -- 172.21.15.7:0/2705707872 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42900533c0 con 0x7f42a00b35f0 2024-01-11T07:10:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.438+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42a00b18b0 con 0x7f42a00b35f0 2024-01-11T07:10:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.438+0000 7f42957fa700 1 -- 172.21.15.7:0/2705707872 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4290017070 con 0x7f42a00b35f0 2024-01-11T07:10:41.439 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:10:41.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4284066f30 msgr2=0x7f42840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4284066f30 0x7f42840693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f429c000c40 tx=0x7f429c007a50 comp rx=0 tx=0).stop 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a00b35f0 msgr2=0x7f42a016b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a00b35f0 0x7f42a016b120 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c130 tx=0x7f4290015000 comp rx=0 tx=0).stop 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 shutdown_connections 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42a016b660 0x7f42a0155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f428c004810 0x7f42a0152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 --2- 172.21.15.7:0/2705707872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a00b35f0 0x7f42a016b120 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 >> 172.21.15.7:0/2705707872 conn(0x7f42a00cd8f0 msgr2=0x7f42a006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 shutdown_connections 2024-01-11T07:10:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.440+0000 7f42a75e3700 1 -- 172.21.15.7:0/2705707872 wait complete. 2024-01-11T07:10:41.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:10:41.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.585+0000 7f267ce4d700 1 Processor -- start 2024-01-11T07:10:41.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.585+0000 7f267ce4d700 1 -- start start 2024-01-11T07:10:41.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.586+0000 7f267ce4d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267815b260 0x7f26780b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.586+0000 7f267ce4d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2678069c80 0x7f267806a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.586+0000 7f267ce4d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f267806e3f0 con 0x7f2678156760 2024-01-11T07:10:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.586+0000 7f267ce4d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267806ef90 con 0x7f2678069c80 2024-01-11T07:10:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.586+0000 7f267ce4d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267806a640 con 0x7f267815b260 2024-01-11T07:10:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f267659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2678156760 0x7f267806e880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50288/0 (socket says 172.21.15.7:50288) 2024-01-11T07:10:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f267659c700 1 -- 172.21.15.7:0/3041219617 learned_addr learned my addr 172.21.15.7:0/3041219617 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f2676d9d700 1 --2- 172.21.15.7:0/3041219617 >> v2:172.21.15.7:3300/0 conn(0x7f2678069c80 0x7f267806a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f2675d9b700 1 --2- 172.21.15.7:0/3041219617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267815b260 0x7f26780b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f267559a700 1 -- 172.21.15.7:0/3041219617 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089630468 0 0) 0x7f267806e3f0 con 0x7f2678156760 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f267559a700 1 -- 172.21.15.7:0/3041219617 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2664003570 con 0x7f2678156760 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f2676d9d700 1 -- 172.21.15.7:0/3041219617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267815b260 msgr2=0x7f26780b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f2676d9d700 1 --2- 172.21.15.7:0/3041219617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267815b260 0x7f26780b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f2676d9d700 1 -- 172.21.15.7:0/3041219617 >> v1:172.21.15.7:6789/0 conn(0x7f2678156760 legacy=0x7f267806e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f2676d9d700 1 -- 172.21.15.7:0/3041219617 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678157b40 con 0x7f2678069c80 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f2676d9d700 1 --2- 172.21.15.7:0/3041219617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678069c80 0x7f267806a100 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2670009e90 tx=0x7f2670009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0f522f9d80fd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f267559a700 1 -- 172.21.15.7:0/3041219617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267000e070 con 0x7f2678069c80 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f267559a700 1 -- 172.21.15.7:0/3041219617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267000bb40 con 0x7f2678069c80 2024-01-11T07:10:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.587+0000 7f267559a700 1 -- 172.21.15.7:0/3041219617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670012430 con 0x7f2678069c80 2024-01-11T07:10:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.588+0000 7f267ce4d700 1 -- 172.21.15.7:0/3041219617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678069c80 msgr2=0x7f267806a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.588+0000 7f267ce4d700 1 --2- 172.21.15.7:0/3041219617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678069c80 0x7f267806a100 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2670009e90 tx=0x7f2670009780 comp rx=0 tx=0).stop 2024-01-11T07:10:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.588+0000 7f267ce4d700 1 -- 172.21.15.7:0/3041219617 shutdown_connections 2024-01-11T07:10:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.588+0000 7f267ce4d700 1 --2- 172.21.15.7:0/3041219617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678069c80 0x7f267806a100 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.588+0000 7f267ce4d700 1 --2- 172.21.15.7:0/3041219617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267815b260 0x7f26780b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.588+0000 7f267ce4d700 1 -- 172.21.15.7:0/3041219617 >> 172.21.15.7:0/3041219617 conn(0x7f26780cd910 msgr2=0x7f26780cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.588+0000 7f267ce4d700 1 -- 172.21.15.7:0/3041219617 shutdown_connections 2024-01-11T07:10:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.588+0000 7f267ce4d700 1 -- 172.21.15.7:0/3041219617 wait complete. 2024-01-11T07:10:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.589+0000 7f267ce4d700 1 Processor -- start 2024-01-11T07:10:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.589+0000 7f267ce4d700 1 -- start start 2024-01-11T07:10:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f267ce4d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2678069c80 0x7f2678149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f267ce4d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678156760 0x7f267814b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f267ce4d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267815b260 0x7f267815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f267ce4d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267814bc10 con 0x7f2678156760 2024-01-11T07:10:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f267ce4d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267814bef0 con 0x7f267815b260 2024-01-11T07:10:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f267ce4d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267814c1d0 con 0x7f2678069c80 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f2675d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678156760 0x7f267814b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f2675d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678156760 0x7f267814b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35234/0 (socket says 172.21.15.7:35234) 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f2675d9b700 1 -- 172.21.15.7:0/1077771973 learned_addr learned my addr 172.21.15.7:0/1077771973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f2675d9b700 1 -- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2678069c80 msgr2=0x7f2678149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f2675d9b700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2678069c80 0x7f2678149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f2676d9d700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267815b260 0x7f267815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.590+0000 7f2675d9b700 1 -- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267815b260 msgr2=0x7f267815a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.591+0000 7f2675d9b700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267815b260 0x7f267815a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.591+0000 7f2675d9b700 1 -- 172.21.15.7:0/1077771973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670009a20 con 0x7f2678156760 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.591+0000 7f2675d9b700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678156760 0x7f267814b510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f266000e930 tx=0x7f26600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.591+0000 7f266e7fc700 1 -- 172.21.15.7:0/1077771973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000eed0 con 0x7f2678156760 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.591+0000 7f266e7fc700 1 -- 172.21.15.7:0/1077771973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266000c850 con 0x7f2678156760 2024-01-11T07:10:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.591+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26780ca810 con 0x7f2678156760 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.591+0000 7f266e7fc700 1 -- 172.21.15.7:0/1077771973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266000cbf0 con 0x7f2678156760 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.591+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267815aca0 con 0x7f2678156760 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.592+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267815af20 con 0x7f2678156760 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.592+0000 7f266e7fc700 1 -- 172.21.15.7:0/1077771973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2660013470 con 0x7f2678156760 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.593+0000 7f266e7fc700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2658066f30 0x7f26580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.593+0000 7f266e7fc700 1 -- 172.21.15.7:0/1077771973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2660084760 con 0x7f2678156760 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.593+0000 7f267659c700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2658066f30 0x7f26580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.593+0000 7f266e7fc700 1 -- 172.21.15.7:0/1077771973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7f265806ea50 con 0x7f2678156760 2024-01-11T07:10:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.594+0000 7f267659c700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2658066f30 0x7f26580693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26680058d0 tx=0x7f26680079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:41.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.598+0000 7f266e7fc700 1 -- 172.21.15.7:0/1077771973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2660021060 con 0x7f2678156760 2024-01-11T07:10:41.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.848+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26780cad30 con 0x7f2678156760 2024-01-11T07:10:41.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.849+0000 7f266e7fc700 1 -- 172.21.15.7:0/1077771973 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f26780cad30 con 0x7f2678156760 2024-01-11T07:10:41.849 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:41.849 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1734166534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1734166534},{"type":"v1","addr":"172.21.15.113:6836","nonce":1734166534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3195231529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3195231529},{"type":"v1","addr":"172.21.15.113:6837","nonce":3195231529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/338342475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":338342475},{"type":"v1","addr":"172.21.15.7:6834","nonce":338342475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:09:08.717048+0000","modified":"2024-01-11T07:09:12.324714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1812078530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1812078530},{"type":"v1","addr":"172.21.15.7:6835","nonce":1812078530}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2658066f30 msgr2=0x7f26580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2658066f30 0x7f26580693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26680058d0 tx=0x7f26680079d0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678156760 msgr2=0x7f267814b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678156760 0x7f267814b510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f266000e930 tx=0x7f26600091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 shutdown_connections 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2658066f30 0x7f26580693e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267815b260 0x7f267815a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2678069c80 0x7f2678149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 --2- 172.21.15.7:0/1077771973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2678156760 0x7f267814b510 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 >> 172.21.15.7:0/1077771973 conn(0x7f26780cd910 msgr2=0x7f26780c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 shutdown_connections 2024-01-11T07:10:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.851+0000 7f267ce4d700 1 -- 172.21.15.7:0/1077771973 wait complete. 2024-01-11T07:10:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-11T07:10:41.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:10:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.997+0000 7fbb3b3a3700 1 Processor -- start 2024-01-11T07:10:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.998+0000 7fbb3b3a3700 1 -- start start 2024-01-11T07:10:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.998+0000 7fbb3b3a3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157e80 0x7fbb340b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.998+0000 7fbb3b3a3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb3406f3c0 0x7fbb340b0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.998+0000 7fbb3b3a3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb340c8f80 con 0x7fbb34157ab0 2024-01-11T07:10:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.998+0000 7fbb3b3a3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb340b0ba0 con 0x7fbb3406f3c0 2024-01-11T07:10:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.998+0000 7fbb3b3a3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c9de0 con 0x7fbb34157e80 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3893e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157e80 0x7fbb340b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb39940700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb3406f3c0 0x7fbb340b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3893e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157e80 0x7fbb340b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:44408/0 (socket says 172.21.15.7:44408) 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3893e700 1 -- 172.21.15.7:0/3568089303 learned_addr learned my addr 172.21.15.7:0/3568089303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb39940700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb3406f3c0 0x7fbb340b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:39826/0 (socket says 172.21.15.7:39826) 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3913f700 1 --1- 172.21.15.7:0/3568089303 >> v1:172.21.15.113:6789/0 conn(0x7fbb34157ab0 0x7fbb3406ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:57846/0 (socket says 172.21.15.7:57846) 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3893e700 1 -- 172.21.15.7:0/3568089303 >> v2:172.21.15.113:3300/0 conn(0x7fbb3406f3c0 msgr2=0x7fbb340b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3893e700 1 --2- 172.21.15.7:0/3568089303 >> v2:172.21.15.113:3300/0 conn(0x7fbb3406f3c0 0x7fbb340b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3893e700 1 -- 172.21.15.7:0/3568089303 >> v1:172.21.15.113:6789/0 conn(0x7fbb34157ab0 legacy=0x7fbb3406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3893e700 1 -- 172.21.15.7:0/3568089303 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340c91d0 con 0x7fbb34157e80 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:41.999+0000 7fbb3893e700 1 --2- 172.21.15.7:0/3568089303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157e80 0x7fbb340b0120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb240099b0 tx=0x7fbb24009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbedea3431a1b0b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb2bfff700 1 -- 172.21.15.7:0/3568089303 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24009ea0 con 0x7fbb34157e80 2024-01-11T07:10:42.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb2bfff700 1 -- 172.21.15.7:0/3568089303 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2400b860 con 0x7fbb34157e80 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb2bfff700 1 -- 172.21.15.7:0/3568089303 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400bc00 con 0x7fbb34157e80 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/3568089303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157e80 msgr2=0x7fbb340b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/3568089303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157e80 0x7fbb340b0120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb240099b0 tx=0x7fbb24009250 comp rx=0 tx=0).stop 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/3568089303 shutdown_connections 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/3568089303 >> v2:172.21.15.113:3300/0 conn(0x7fbb3406f3c0 0x7fbb340b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/3568089303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157e80 0x7fbb340b0120 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/3568089303 >> 172.21.15.7:0/3568089303 conn(0x7fbb34070690 msgr2=0x7fbb34070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/3568089303 shutdown_connections 2024-01-11T07:10:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.000+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/3568089303 wait complete. 2024-01-11T07:10:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.001+0000 7fbb3b3a3700 1 Processor -- start 2024-01-11T07:10:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.002+0000 7fbb3b3a3700 1 -- start start 2024-01-11T07:10:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.002+0000 7fbb3b3a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3406f3c0 0x7fbb340c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.002+0000 7fbb3b3a3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157ab0 0x7fbb34153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3b3a3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34157e80 0x7fbb3415a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3b3a3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb341544f0 con 0x7fbb3406f3c0 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3b3a3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c8650 con 0x7fbb34157e80 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3b3a3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3415a600 con 0x7fbb34157ab0 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3406f3c0 0x7fbb340c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3406f3c0 0x7fbb340c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35248/0 (socket says 172.21.15.7:35248) 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 -- 172.21.15.7:0/4037987817 learned_addr learned my addr 172.21.15.7:0/4037987817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3893e700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157ab0 0x7fbb34153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb39940700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34157e80 0x7fbb3415a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 -- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157ab0 msgr2=0x7fbb34153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157ab0 0x7fbb34153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 -- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34157e80 msgr2=0x7fbb3415a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34157e80 0x7fbb3415a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 -- 172.21.15.7:0/4037987817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240094f0 con 0x7fbb3406f3c0 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3893e700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157ab0 0x7fbb34153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3913f700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3406f3c0 0x7fbb340c73d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c7c0 tx=0x7fbb30007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb297fa700 1 -- 172.21.15.7:0/4037987817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30005a70 con 0x7fbb3406f3c0 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb297fa700 1 -- 172.21.15.7:0/4037987817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb30005bd0 con 0x7fbb3406f3c0 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb340c7a40 con 0x7fbb3406f3c0 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.003+0000 7fbb297fa700 1 -- 172.21.15.7:0/4037987817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb300070d0 con 0x7fbb3406f3c0 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.004+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb340c8000 con 0x7fbb3406f3c0 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.004+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb340c2860 con 0x7fbb3406f3c0 2024-01-11T07:10:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.005+0000 7fbb297fa700 1 -- 172.21.15.7:0/4037987817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbb300074c0 con 0x7fbb3406f3c0 2024-01-11T07:10:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.005+0000 7fbb297fa700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbb18066f30 0x7fbb180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.005+0000 7fbb297fa700 1 -- 172.21.15.7:0/4037987817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb30083a50 con 0x7fbb3406f3c0 2024-01-11T07:10:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.006+0000 7fbb297fa700 1 -- 172.21.15.7:0/4037987817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7fbb1806ea50 con 0x7fbb3406f3c0 2024-01-11T07:10:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.006+0000 7fbb3893e700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbb18066f30 0x7fbb180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.006+0000 7fbb3893e700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbb18066f30 0x7fbb180693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbb240099b0 tx=0x7fbb24017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.010+0000 7fbb297fa700 1 -- 172.21.15.7:0/4037987817 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb30017070 con 0x7fbb3406f3c0 2024-01-11T07:10:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.220+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb3404c3c0 con 0x7fbb3406f3c0 2024-01-11T07:10:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.220+0000 7fbb297fa700 1 -- 172.21.15.7:0/4037987817 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+15360 (secure 0 0 0) 0x7fbb340c2860 con 0x7fbb3406f3c0 2024-01-11T07:10:42.221 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:42.221 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:09:30.370524+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:09:06.415283+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-11T07:09:07.694952+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbb18066f30 msgr2=0x7fbb180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbb18066f30 0x7fbb180693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbb240099b0 tx=0x7fbb24017000 comp rx=0 tx=0).stop 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3406f3c0 msgr2=0x7fbb340c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3406f3c0 0x7fbb340c73d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000c7c0 tx=0x7fbb30007f70 comp rx=0 tx=0).stop 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 shutdown_connections 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbb18066f30 0x7fbb180693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34157e80 0x7fbb3415a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb3406f3c0 0x7fbb340c73d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 --2- 172.21.15.7:0/4037987817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb34157ab0 0x7fbb34153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 >> 172.21.15.7:0/4037987817 conn(0x7fbb34070690 msgr2=0x7fbb34070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 shutdown_connections 2024-01-11T07:10:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.223+0000 7fbb3b3a3700 1 -- 172.21.15.7:0/4037987817 wait complete. 2024-01-11T07:10:42.238 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:10:42.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:10:42.283 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:10:42.282+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-01-11T07:10:42.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.372+0000 7f6ea0fea700 1 Processor -- start 2024-01-11T07:10:42.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.372+0000 7f6ea0fea700 1 -- start start 2024-01-11T07:10:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.373+0000 7f6ea0fea700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 0x7f6e9c0aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.373+0000 7f6ea0fea700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c15d880 con 0x7f6e9c0ac5b0 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.373+0000 7f6ea0fea700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c06c5b0 con 0x7f6e9c063230 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.373+0000 7f6ea0fea700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c15a7f0 con 0x7f6e9c15a280 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.373+0000 7f6e9ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6e9c0ac5b0 0x7f6e9c0aca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50298/0 (socket says 172.21.15.7:50298) 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.373+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/669331103 learned_addr learned my addr 172.21.15.7:0/669331103 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e91d9b700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 0x7f6e9c0aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742806626 0 0) 0x7f6e9c15d880 con 0x7f6e9c0ac5b0 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e88003570 con 0x7f6e9c0ac5b0 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649024642 0 0) 0x7f6e9c06c5b0 con 0x7f6e9c063230 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e9c15d880 con 0x7f6e9c063230 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3790370541 0 0) 0x7f6e88003570 con 0x7f6e9c0ac5b0 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 msgr2=0x7f6e9c0aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 0x7f6e9c0aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 >> v1:172.21.15.113:6789/0 conn(0x7f6e9c063230 legacy=0x7f6e9c063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c15aad0 con 0x7f6e9c0ac5b0 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f6e940017f0 con 0x7f6e9c0ac5b0 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e91d9b700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 0x7f6e9c0aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 >> v1:172.21.15.7:6789/0 conn(0x7f6e9c0ac5b0 legacy=0x7f6e9c0aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e88003c20 0x7f6e88004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 0x7f6e88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e88004d50 0x7f6e880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c15a7f0 con 0x7f6e88004d50 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8800bb10 con 0x7f6e88003c20 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8800bcd0 con 0x7f6e9c15a280 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e9ad9d700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e88004d50 0x7f6e880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e9a59c700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e88003c20 0x7f6e88004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.374+0000 7f6e91d9b700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 0x7f6e88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 msgr2=0x7f6e88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:42.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e9ad9d700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e9c15a280 0x7f6e88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e88003c20 msgr2=0x7f6e88004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e9ad9d700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e88003c20 0x7f6e88004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/669331103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e84009420 con 0x7f6e88004d50 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e9ad9d700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e88004d50 0x7f6e880051c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400a8d0 tx=0x7f6e94002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5861bd5410241789 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e940070d0 con 0x7f6e88004d50 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e94007230 con 0x7f6e88004d50 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6e99d9b700 1 -- 172.21.15.7:0/669331103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e940075f0 con 0x7f6e88004d50 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6ea0fea700 1 -- 172.21.15.7:0/669331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e88004d50 msgr2=0x7f6e880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.375+0000 7f6ea0fea700 1 --2- 172.21.15.7:0/669331103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e88004d50 0x7f6e880051c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400a8d0 tx=0x7f6e94002190 comp rx=0 tx=0).stop 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.376+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/669331103 reap_dead start 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.376+0000 7f6ea0fea700 1 -- 172.21.15.7:0/669331103 shutdown_connections 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.376+0000 7f6ea0fea700 1 -- 172.21.15.7:0/669331103 >> 172.21.15.7:0/669331103 conn(0x7f6e9c0680e0 msgr2=0x7f6e9c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.376+0000 7f6ea0fea700 1 -- 172.21.15.7:0/669331103 shutdown_connections 2024-01-11T07:10:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.376+0000 7f6ea0fea700 1 -- 172.21.15.7:0/669331103 wait complete. 2024-01-11T07:10:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.377+0000 7f6ea0fea700 1 Processor -- start 2024-01-11T07:10:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.377+0000 7f6ea0fea700 1 -- start start 2024-01-11T07:10:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6ea0fea700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e88004d50 0x7f6e9c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6ea0fea700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c153310 0x7f6e9c14f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6ea0fea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c14fde0 0x7f6e9c0af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6ea0fea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0cd270 con 0x7f6e9c14fde0 2024-01-11T07:10:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6ea0fea700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0b0240 con 0x7f6e9c153310 2024-01-11T07:10:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6ea0fea700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c16e320 con 0x7f6e88004d50 2024-01-11T07:10:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c14fde0 0x7f6e9c0af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c14fde0 0x7f6e9c0af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35266/0 (socket says 172.21.15.7:35266) 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/2060393223 learned_addr learned my addr 172.21.15.7:0/2060393223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e91d9b700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c153310 0x7f6e9c14f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9a59c700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e88004d50 0x7f6e9c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e88004d50 msgr2=0x7f6e9c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9ad9d700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e88004d50 0x7f6e9c152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c153310 msgr2=0x7f6e9c14f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9ad9d700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c153310 0x7f6e9c14f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.378+0000 7f6e9ad9d700 1 -- 172.21.15.7:0/2060393223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c16d560 con 0x7f6e9c14fde0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.379+0000 7f6e9ad9d700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c14fde0 0x7f6e9c0af560 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400a8a0 tx=0x7f6e94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.379+0000 7f6e92ffd700 1 -- 172.21.15.7:0/2060393223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94023070 con 0x7f6e9c14fde0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.379+0000 7f6e92ffd700 1 -- 172.21.15.7:0/2060393223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e94004bb0 con 0x7f6e9c14fde0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.379+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9c16d840 con 0x7f6e9c14fde0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.379+0000 7f6e92ffd700 1 -- 172.21.15.7:0/2060393223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9401e430 con 0x7f6e9c14fde0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.379+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c16de00 con 0x7f6e9c14fde0 2024-01-11T07:10:42.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.380+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9c04c3c0 con 0x7f6e9c14fde0 2024-01-11T07:10:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.380+0000 7f6e92ffd700 1 -- 172.21.15.7:0/2060393223 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6e9401e7a0 con 0x7f6e9c14fde0 2024-01-11T07:10:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.381+0000 7f6e92ffd700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6e7c066f30 0x7f6e7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.381+0000 7f6e92ffd700 1 -- 172.21.15.7:0/2060393223 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e940865c0 con 0x7f6e9c14fde0 2024-01-11T07:10:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.381+0000 7f6e92ffd700 1 -- 172.21.15.7:0/2060393223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f6e7c06ea50 con 0x7f6e9c14fde0 2024-01-11T07:10:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.381+0000 7f6e9a59c700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6e7c066f30 0x7f6e7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.381+0000 7f6e9a59c700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6e7c066f30 0x7f6e7c0693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0010e0 tx=0x7f6e8c007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.385+0000 7f6e92ffd700 1 -- 172.21.15.7:0/2060393223 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e94053830 con 0x7f6e9c14fde0 2024-01-11T07:10:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.636+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e9c16dff0 con 0x7f6e9c14fde0 2024-01-11T07:10:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.637+0000 7f6e92ffd700 1 -- 172.21.15.7:0/2060393223 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f6e94056e50 con 0x7f6e9c14fde0 2024-01-11T07:10:42.637 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1734166534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1734166534},{"type":"v1","addr":"172.21.15.113:6836","nonce":1734166534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3195231529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3195231529},{"type":"v1","addr":"172.21.15.113:6837","nonce":3195231529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/338342475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":338342475},{"type":"v1","addr":"172.21.15.7:6834","nonce":338342475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:09:08.717048+0000","modified":"2024-01-11T07:09:12.324714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1812078530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1812078530},{"type":"v1","addr":"172.21.15.7:6835","nonce":1812078530}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-11T07:10:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6e7c066f30 msgr2=0x7f6e7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6e7c066f30 0x7f6e7c0693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0010e0 tx=0x7f6e8c007d30 comp rx=0 tx=0).stop 2024-01-11T07:10:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c14fde0 msgr2=0x7f6e9c0af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c14fde0 0x7f6e9c0af560 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400a8a0 tx=0x7f6e94016000 comp rx=0 tx=0).stop 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 shutdown_connections 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6e7c066f30 0x7f6e7c0693e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e9c14fde0 0x7f6e9c0af560 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e88004d50 0x7f6e9c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 --2- 172.21.15.7:0/2060393223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e9c153310 0x7f6e9c14f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 >> 172.21.15.7:0/2060393223 conn(0x7f6e9c0680e0 msgr2=0x7f6e9c06a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 shutdown_connections 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.639+0000 7f6ea0fea700 1 -- 172.21.15.7:0/2060393223 wait complete. 2024-01-11T07:10:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-11T07:10:42.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:10:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.826+0000 7f4219a13700 1 Processor -- start 2024-01-11T07:10:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.826+0000 7f4219a13700 1 -- start start 2024-01-11T07:10:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4219a13700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42140cd6d0 0x7f42140cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4219a13700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f421415d8a0 0x7f421415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4219a13700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 0x7f42140cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4219a13700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421415e700 con 0x7f42140cd6d0 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4219a13700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421415acf0 con 0x7f421415d8a0 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4219a13700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421415afd0 con 0x7f421415e280 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4212ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42140cd6d0 0x7f42140cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4212ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42140cd6d0 0x7f42140cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35272/0 (socket says 172.21.15.7:35272) 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f4212ffd700 1 -- 172.21.15.7:0/3762890738 learned_addr learned my addr 172.21.15.7:0/3762890738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f42127fc700 1 --2- 172.21.15.7:0/3762890738 >> v2:172.21.15.113:3300/0 conn(0x7f421415d8a0 0x7f421415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.827+0000 7f42137fe700 1 --2- 172.21.15.7:0/3762890738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 0x7f42140cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4212ffd700 1 -- 172.21.15.7:0/3762890738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 msgr2=0x7f42140cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4212ffd700 1 --2- 172.21.15.7:0/3762890738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 0x7f42140cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4212ffd700 1 -- 172.21.15.7:0/3762890738 >> v2:172.21.15.113:3300/0 conn(0x7f421415d8a0 msgr2=0x7f421415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4212ffd700 1 --2- 172.21.15.7:0/3762890738 >> v2:172.21.15.113:3300/0 conn(0x7f421415d8a0 0x7f421415dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4212ffd700 1 -- 172.21.15.7:0/3762890738 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42140ce980 con 0x7f42140cd6d0 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4212ffd700 1 --2- 172.21.15.7:0/3762890738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140cdaa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f42140cc620 tx=0x7f4204009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b63659067b603ab server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4211ffb700 1 -- 172.21.15.7:0/3762890738 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420400b640 con 0x7f42140cd6d0 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4211ffb700 1 -- 172.21.15.7:0/3762890738 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420400b7a0 con 0x7f42140cd6d0 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.828+0000 7f4211ffb700 1 -- 172.21.15.7:0/3762890738 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420400bb60 con 0x7f42140cd6d0 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 -- 172.21.15.7:0/3762890738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 msgr2=0x7f42140cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 --2- 172.21.15.7:0/3762890738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140cdaa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f42140cc620 tx=0x7f4204009250 comp rx=0 tx=0).stop 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 -- 172.21.15.7:0/3762890738 shutdown_connections 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 --2- 172.21.15.7:0/3762890738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140cdaa0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 --2- 172.21.15.7:0/3762890738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 0x7f42140cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 --2- 172.21.15.7:0/3762890738 >> v2:172.21.15.113:3300/0 conn(0x7f421415d8a0 0x7f421415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 -- 172.21.15.7:0/3762890738 >> 172.21.15.7:0/3762890738 conn(0x7f4214071120 msgr2=0x7f421406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 -- 172.21.15.7:0/3762890738 shutdown_connections 2024-01-11T07:10:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.829+0000 7f4219a13700 1 -- 172.21.15.7:0/3762890738 wait complete. 2024-01-11T07:10:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.830+0000 7f4219a13700 1 Processor -- start 2024-01-11T07:10:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.830+0000 7f4219a13700 1 -- start start 2024-01-11T07:10:42.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4219a13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4219a13700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f421415d8a0 0x7f421415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4219a13700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 0x7f4214148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4219a13700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421415d550 con 0x7f42140cd6d0 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4219a13700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214169e90 con 0x7f421415d8a0 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4219a13700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42140b4020 con 0x7f421415e280 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4212ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4212ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35288/0 (socket says 172.21.15.7:35288) 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f4212ffd700 1 -- 172.21.15.7:0/3014922079 learned_addr learned my addr 172.21.15.7:0/3014922079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f42127fc700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f421415d8a0 0x7f421415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.831+0000 7f42137fe700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 0x7f4214148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f4212ffd700 1 -- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 msgr2=0x7f4214148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f4212ffd700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 0x7f4214148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f4212ffd700 1 -- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f421415d8a0 msgr2=0x7f421415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f4212ffd700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f421415d8a0 0x7f421415d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f4212ffd700 1 -- 172.21.15.7:0/3014922079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42040094f0 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f4212ffd700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140b39b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4214063db0 tx=0x7f4204017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f420affd700 1 -- 172.21.15.7:0/3014922079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420400b910 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f420affd700 1 -- 172.21.15.7:0/3014922079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420400ba70 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42140b4300 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f420affd700 1 -- 172.21.15.7:0/3014922079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204011630 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.832+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42140b48c0 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.833+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421416a030 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.833+0000 7f420affd700 1 -- 172.21.15.7:0/3014922079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f42040119a0 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.834+0000 7f420affd700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f41f4066f30 0x7f41f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.834+0000 7f420affd700 1 -- 172.21.15.7:0/3014922079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42040856d0 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.834+0000 7f420affd700 1 -- 172.21.15.7:0/3014922079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f41f406ea50 con 0x7f42140cd6d0 2024-01-11T07:10:42.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.834+0000 7f42127fc700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f41f4066f30 0x7f41f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:42.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.835+0000 7f42127fc700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f41f4066f30 0x7f41f40693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42140b4e60 tx=0x7f41fc0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:42.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:42.838+0000 7f420affd700 1 -- 172.21.15.7:0/3014922079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4204015060 con 0x7f42140cd6d0 2024-01-11T07:10:43.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.093+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42140b4ab0 con 0x7f42140cd6d0 2024-01-11T07:10:43.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.093+0000 7f420affd700 1 -- 172.21.15.7:0/3014922079 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f4204028070 con 0x7f42140cd6d0 2024-01-11T07:10:43.095 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:43.095 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1734166534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1734166534},{"type":"v1","addr":"172.21.15.113:6836","nonce":1734166534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3195231529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3195231529},{"type":"v1","addr":"172.21.15.113:6837","nonce":3195231529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/338342475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":338342475},{"type":"v1","addr":"172.21.15.7:6834","nonce":338342475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:09:08.717048+0000","modified":"2024-01-11T07:09:12.324714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/1812078530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":1812078530},{"type":"v1","addr":"172.21.15.7:6835","nonce":1812078530}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2024-01-11T07:10:43.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f41f4066f30 msgr2=0x7f41f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f41f4066f30 0x7f41f40693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42140b4e60 tx=0x7f41fc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 msgr2=0x7f42140b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140b39b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4214063db0 tx=0x7f4204017000 comp rx=0 tx=0).stop 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 shutdown_connections 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f41f4066f30 0x7f41f40693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f421415e280 0x7f4214148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42140cd6d0 0x7f42140b39b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 --2- 172.21.15.7:0/3014922079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f421415d8a0 0x7f421415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 >> 172.21.15.7:0/3014922079 conn(0x7f4214071120 msgr2=0x7f42140cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 shutdown_connections 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.096+0000 7f4219a13700 1 -- 172.21.15.7:0/3014922079 wait complete. 2024-01-11T07:10:43.098 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-01-11T07:10:43.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:10:43.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.242+0000 7f24ce87b700 1 Processor -- start 2024-01-11T07:10:43.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.242+0000 7f24ce87b700 1 -- start start 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.243+0000 7f24ce87b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24c806dd00 0x7f24c80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.243+0000 7f24ce87b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c815b770 con 0x7f24c806e0d0 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.243+0000 7f24ce87b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c815a620 con 0x7f24c815a1c0 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.243+0000 7f24ce87b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c815abf0 con 0x7f24c806dd00 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.243+0000 7f24c7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24c806dd00 0x7f24c80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.243+0000 7f24c77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f24c806e0d0 0x7f24c8159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50302/0 (socket says 172.21.15.7:50302) 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.243+0000 7f24c77fe700 1 -- 172.21.15.7:0/2523168682 learned_addr learned my addr 172.21.15.7:0/2523168682 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.244+0000 7f24c7fff700 1 -- 172.21.15.7:0/2523168682 >> v1:172.21.15.113:6789/0 conn(0x7f24c815a1c0 legacy=0x7f24c8069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.244+0000 7f24c7fff700 1 -- 172.21.15.7:0/2523168682 >> v1:172.21.15.7:6789/0 conn(0x7f24c806e0d0 legacy=0x7f24c8159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.244+0000 7f24c7fff700 1 -- 172.21.15.7:0/2523168682 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c815aed0 con 0x7f24c806dd00 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.244+0000 7f24c6ffd700 1 -- 172.21.15.7:0/2523168682 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295192686 0 0) 0x7f24c815b770 con 0x7f24c806e0d0 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.244+0000 7f24c7fff700 1 --2- 172.21.15.7:0/2523168682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c806dd00 0x7f24c80c98d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f24b80098e0 tx=0x7f24b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aadc424a1c7628 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:43.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.244+0000 7f24c6ffd700 1 -- 172.21.15.7:0/2523168682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8009e80 con 0x7f24c806dd00 2024-01-11T07:10:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.244+0000 7f24c6ffd700 1 -- 172.21.15.7:0/2523168682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b800b860 con 0x7f24c806dd00 2024-01-11T07:10:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.244+0000 7f24c6ffd700 1 -- 172.21.15.7:0/2523168682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800bbe0 con 0x7f24c806dd00 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.245+0000 7f24ce87b700 1 -- 172.21.15.7:0/2523168682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c806dd00 msgr2=0x7f24c80c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.245+0000 7f24ce87b700 1 --2- 172.21.15.7:0/2523168682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c806dd00 0x7f24c80c98d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f24b80098e0 tx=0x7f24b80091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.245+0000 7f24ce87b700 1 -- 172.21.15.7:0/2523168682 shutdown_connections 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.245+0000 7f24ce87b700 1 --2- 172.21.15.7:0/2523168682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c806dd00 0x7f24c80c98d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.245+0000 7f24ce87b700 1 -- 172.21.15.7:0/2523168682 >> 172.21.15.7:0/2523168682 conn(0x7f24c80680e0 msgr2=0x7f24c80cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.245+0000 7f24ce87b700 1 -- 172.21.15.7:0/2523168682 shutdown_connections 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.245+0000 7f24ce87b700 1 -- 172.21.15.7:0/2523168682 wait complete. 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.246+0000 7f24ce87b700 1 Processor -- start 2024-01-11T07:10:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.246+0000 7f24ce87b700 1 -- start start 2024-01-11T07:10:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24ce87b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806dd00 0x7f24c814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24ce87b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24c806e0d0 0x7f24c814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24ce87b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c815a1c0 0x7f24c8147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24ce87b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80c4100 con 0x7f24c815a1c0 2024-01-11T07:10:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24ce87b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8147370 con 0x7f24c806dd00 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24ce87b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8147800 con 0x7f24c806e0d0 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24cce18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c815a1c0 0x7f24c8147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24cce18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c815a1c0 0x7f24c8147d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35308/0 (socket says 172.21.15.7:35308) 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.247+0000 7f24cce18700 1 -- 172.21.15.7:0/819040393 learned_addr learned my addr 172.21.15.7:0/819040393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24c7fff700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806dd00 0x7f24c814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24c77fe700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24c806e0d0 0x7f24c814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24cce18700 1 -- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24c806e0d0 msgr2=0x7f24c814a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24cce18700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24c806e0d0 0x7f24c814a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24cce18700 1 -- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806dd00 msgr2=0x7f24c814a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24cce18700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806dd00 0x7f24c814a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24cce18700 1 -- 172.21.15.7:0/819040393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8009420 con 0x7f24c815a1c0 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24cce18700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c815a1c0 0x7f24c8147d90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c840 tx=0x7f24c0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000a650 con 0x7f24c815a1c0 2024-01-11T07:10:43.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c000a7d0 con 0x7f24c815a1c0 2024-01-11T07:10:43.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c8146760 con 0x7f24c815a1c0 2024-01-11T07:10:43.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000ab90 con 0x7f24c815a1c0 2024-01-11T07:10:43.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.248+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c8146d20 con 0x7f24c815a1c0 2024-01-11T07:10:43.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.249+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c804c3c0 con 0x7f24c815a1c0 2024-01-11T07:10:43.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.250+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f24c0003030 con 0x7f24c815a1c0 2024-01-11T07:10:43.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.250+0000 7f24affff700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f24b4066f30 0x7f24b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.250+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f24c0084510 con 0x7f24c815a1c0 2024-01-11T07:10:43.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.250+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f24b406ea50 con 0x7f24c815a1c0 2024-01-11T07:10:43.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.250+0000 7f24c7fff700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f24b4066f30 0x7f24b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.251+0000 7f24c7fff700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f24b4066f30 0x7f24b40693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f24b80093f0 tx=0x7f24b8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:43.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.254+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24c0051780 con 0x7f24c815a1c0 2024-01-11T07:10:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.292+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(71..71 src has 1..71) v4 ==== 288+0+0 (secure 0 0 0) 0x7f24c0055b40 con 0x7f24c815a1c0 2024-01-11T07:10:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.292+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7f24b406f3a0 con 0x7f24c815a1c0 2024-01-11T07:10:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.461+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24c815b1b0 con 0x7f24c815a1c0 2024-01-11T07:10:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.462+0000 7f24affff700 1 -- 172.21.15.7:0/819040393 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f24c0054da0 con 0x7f24c815a1c0 2024-01-11T07:10:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:43.463 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":71,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:10:43.289207+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:09:06.415283+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-11T07:09:07.694952+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f24b4066f30 msgr2=0x7f24b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f24b4066f30 0x7f24b40693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f24b80093f0 tx=0x7f24b8017000 comp rx=0 tx=0).stop 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c815a1c0 msgr2=0x7f24c8147d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c815a1c0 0x7f24c8147d90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c840 tx=0x7f24c0001670 comp rx=0 tx=0).stop 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 shutdown_connections 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f24b4066f30 0x7f24b40693e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c815a1c0 0x7f24c8147d90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24c806dd00 0x7f24c814a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 --2- 172.21.15.7:0/819040393 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f24c806e0d0 0x7f24c814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 >> 172.21.15.7:0/819040393 conn(0x7f24c80680e0 msgr2=0x7f24c8158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 shutdown_connections 2024-01-11T07:10:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.465+0000 7f24ce87b700 1 -- 172.21.15.7:0/819040393 wait complete. 2024-01-11T07:10:43.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:10:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.612+0000 7fa52438c700 1 Processor -- start 2024-01-11T07:10:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.612+0000 7fa52438c700 1 -- start start 2024-01-11T07:10:43.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa52438c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa51c0cafb0 0x7fa51c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa52438c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51c0b0b40 0x7fa51c161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa52438c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c159300 con 0x7fa51c158280 2024-01-11T07:10:43.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa52438c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51c0cf260 con 0x7fa51c0cafb0 2024-01-11T07:10:43.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa52438c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0cf540 con 0x7fa51c0b0b40 2024-01-11T07:10:43.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521927700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa51c158280 0x7fa51c0b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50318/0 (socket says 172.21.15.7:50318) 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521927700 1 -- 172.21.15.7:0/599816817 learned_addr learned my addr 172.21.15.7:0/599816817 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa522929700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51c0b0b40 0x7fa51c161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa522128700 1 --2- 172.21.15.7:0/599816817 >> v2:172.21.15.113:3300/0 conn(0x7fa51c0cafb0 0x7fa51c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698300427 0 0) 0x7fa51c159300 con 0x7fa51c158280 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa508003570 con 0x7fa51c158280 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2129401982 0 0) 0x7fa508003570 con 0x7fa51c158280 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51c0b0b40 msgr2=0x7fa51c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51c0b0b40 0x7fa51c161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 >> v2:172.21.15.113:3300/0 conn(0x7fa51c0cafb0 msgr2=0x7fa51c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 --2- 172.21.15.7:0/599816817 >> v2:172.21.15.113:3300/0 conn(0x7fa51c0cafb0 0x7fa51c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0cf820 con 0x7fa51c158280 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa522128700 1 --2- 172.21.15.7:0/599816817 >> v2:172.21.15.113:3300/0 conn(0x7fa51c0cafb0 0x7fa51c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa518001470 con 0x7fa51c158280 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 >> v1:172.21.15.7:6789/0 conn(0x7fa51c158280 legacy=0x7fa51c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa508003a60 0x7fa508003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51c0b0b40 0x7fa508004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa508004d50 0x7fa5080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0cf260 con 0x7fa508004d50 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0cf540 con 0x7fa508003a60 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50800ba60 con 0x7fa51c0b0b40 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa522929700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa508004d50 0x7fa5080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa522128700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa508003a60 0x7fa508003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.614+0000 7fa521927700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51c0b0b40 0x7fa508004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa522929700 1 -- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51c0b0b40 msgr2=0x7fa508004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa522929700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51c0b0b40 0x7fa508004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa522929700 1 -- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa508003a60 msgr2=0x7fa508003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa522929700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa508003a60 0x7fa508003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa522929700 1 -- 172.21.15.7:0/599816817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa508003570 con 0x7fa508004d50 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa522128700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa508003a60 0x7fa508003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:43.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa522929700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa508004d50 0x7fa5080051c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa51400e210 tx=0x7fa51400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133ec69b2fc70f32 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400ea60 con 0x7fa508004d50 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51400f040 con 0x7fa508004d50 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa521126700 1 -- 172.21.15.7:0/599816817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400ee10 con 0x7fa508004d50 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.615+0000 7fa52438c700 1 -- 172.21.15.7:0/599816817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa508004d50 msgr2=0x7fa5080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.616+0000 7fa52438c700 1 --2- 172.21.15.7:0/599816817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa508004d50 0x7fa5080051c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa51400e210 tx=0x7fa51400b040 comp rx=0 tx=0).stop 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.616+0000 7fa522929700 1 -- 172.21.15.7:0/599816817 reap_dead start 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.616+0000 7fa52438c700 1 -- 172.21.15.7:0/599816817 shutdown_connections 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.616+0000 7fa52438c700 1 -- 172.21.15.7:0/599816817 >> 172.21.15.7:0/599816817 conn(0x7fa51c071100 msgr2=0x7fa51c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.616+0000 7fa52438c700 1 -- 172.21.15.7:0/599816817 shutdown_connections 2024-01-11T07:10:43.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.616+0000 7fa52438c700 1 -- 172.21.15.7:0/599816817 wait complete. 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa52438c700 1 Processor -- start 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa52438c700 1 -- start start 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa52438c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa508004d50 0x7fa51c0c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa52438c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51c153540 0x7fa51c0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa52438c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c0b2d10 0x7fa51c0c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa52438c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0b3180 con 0x7fa51c153540 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa52438c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0b3390 con 0x7fa51c0b2d10 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa52438c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0b3a30 con 0x7fa508004d50 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.618+0000 7fa521927700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51c153540 0x7fa51c0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa521927700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51c153540 0x7fa51c0b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35332/0 (socket says 172.21.15.7:35332) 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa521927700 1 -- 172.21.15.7:0/3150007162 learned_addr learned my addr 172.21.15.7:0/3150007162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa522128700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa508004d50 0x7fa51c0c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa522929700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c0b2d10 0x7fa51c0c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa521927700 1 -- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa508004d50 msgr2=0x7fa51c0c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa521927700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa508004d50 0x7fa51c0c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa521927700 1 -- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c0b2d10 msgr2=0x7fa51c0c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa521927700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c0b2d10 0x7fa51c0c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa521927700 1 -- 172.21.15.7:0/3150007162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c0c9750 con 0x7fa51c153540 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa521927700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51c153540 0x7fa51c0b27d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5180073c0 tx=0x7fa518001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518018030 con 0x7fa51c153540 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5180077a0 con 0x7fa51c153540 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c0c8ba0 con 0x7fa51c153540 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51800fb20 con 0x7fa51c153540 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.619+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c0c9100 con 0x7fa51c153540 2024-01-11T07:10:43.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.620+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c0cef60 con 0x7fa51c153540 2024-01-11T07:10:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.621+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa51801b070 con 0x7fa51c153540 2024-01-11T07:10:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.621+0000 7fa50e7fc700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa500066f30 0x7fa5000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.621+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa518019070 con 0x7fa51c153540 2024-01-11T07:10:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.622+0000 7fa522128700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa500066f30 0x7fa5000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.622+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=72}) v3 -- 0x7fa50006ea50 con 0x7fa51c153540 2024-01-11T07:10:43.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.622+0000 7fa522128700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa500066f30 0x7fa5000693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa510006f00 tx=0x7fa5100091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:43.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.627+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5180516e0 con 0x7fa51c153540 2024-01-11T07:10:43.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.876+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa51c0c98e0 con 0x7fa51c153540 2024-01-11T07:10:43.877 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:10:43.876+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:10:43.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.878+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(72..72 src has 1..72) v4 ==== 366+0+0 (secure 0 0 0) 0x7fa518058540 con 0x7fa51c153540 2024-01-11T07:10:43.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.878+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fa50006f3a0 con 0x7fa51c153540 2024-01-11T07:10:43.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.881+0000 7fa50e7fc700 1 -- 172.21.15.7:0/3150007162 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa518054d00 con 0x7fa51c153540 2024-01-11T07:10:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa500066f30 msgr2=0x7fa5000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa500066f30 0x7fa5000693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa510006f00 tx=0x7fa5100091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51c153540 msgr2=0x7fa51c0b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51c153540 0x7fa51c0b27d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa5180073c0 tx=0x7fa518001e90 comp rx=0 tx=0).stop 2024-01-11T07:10:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 shutdown_connections 2024-01-11T07:10:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa500066f30 0x7fa5000693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa51c0b2d10 0x7fa51c0c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa508004d50 0x7fa51c0c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 --2- 172.21.15.7:0/3150007162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51c153540 0x7fa51c0b27d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 >> 172.21.15.7:0/3150007162 conn(0x7fa51c071100 msgr2=0x7fa51c159970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 shutdown_connections 2024-01-11T07:10:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:43.884+0000 7fa52438c700 1 -- 172.21.15.7:0/3150007162 wait complete. 2024-01-11T07:10:43.886 INFO:tasks.ceph.mds.a.smithi007.stderr: -9666> 2024-01-11T07:09:15.277+0000 7f9917c19700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:10:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:10:43.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:10:43.920 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:10:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.032+0000 7fa3cefd4700 1 Processor -- start 2024-01-11T07:10:44.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.033+0000 7fa3cefd4700 1 -- start start 2024-01-11T07:10:44.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.033+0000 7fa3cefd4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3c806de30 0x7fa3c8154210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:44.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.033+0000 7fa3cefd4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806e200 0x7fa3c8154750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:44.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.033+0000 7fa3cefd4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c815a790 con 0x7fa3c806ad40 2024-01-11T07:10:44.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.033+0000 7fa3cefd4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c815b7a0 con 0x7fa3c806de30 2024-01-11T07:10:44.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.033+0000 7fa3cefd4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c815a9e0 con 0x7fa3c806e200 2024-01-11T07:10:44.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3cd571700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa3c806ad40 0x7fa3c806b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50320/0 (socket says 172.21.15.7:50320) 2024-01-11T07:10:44.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3cd571700 1 -- 172.21.15.7:0/1201446313 learned_addr learned my addr 172.21.15.7:0/1201446313 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:44.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3ccd70700 1 --2- 172.21.15.7:0/1201446313 >> v2:172.21.15.113:3300/0 conn(0x7fa3c806de30 0x7fa3c8154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3c7fff700 1 --2- 172.21.15.7:0/1201446313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806e200 0x7fa3c8154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3c77fe700 1 -- 172.21.15.7:0/1201446313 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330299438 0 0) 0x7fa3c815a790 con 0x7fa3c806ad40 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3c77fe700 1 -- 172.21.15.7:0/1201446313 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b8003570 con 0x7fa3c806ad40 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3ccd70700 1 -- 172.21.15.7:0/1201446313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806e200 msgr2=0x7fa3c8154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3ccd70700 1 --2- 172.21.15.7:0/1201446313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806e200 0x7fa3c8154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3ccd70700 1 -- 172.21.15.7:0/1201446313 >> v1:172.21.15.7:6789/0 conn(0x7fa3c806ad40 legacy=0x7fa3c806b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3ccd70700 1 -- 172.21.15.7:0/1201446313 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c815acc0 con 0x7fa3c806de30 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.034+0000 7fa3ccd70700 1 --2- 172.21.15.7:0/1201446313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 0x7fa3c8154210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098e0 tx=0x7fa3bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=929e8648008c0ac0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3c77fe700 1 -- 172.21.15.7:0/1201446313 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc011630 con 0x7fa3c806de30 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3c77fe700 1 -- 172.21.15.7:0/1201446313 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc011790 con 0x7fa3c806de30 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3c77fe700 1 -- 172.21.15.7:0/1201446313 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc011b10 con 0x7fa3c806de30 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3cefd4700 1 -- 172.21.15.7:0/1201446313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 msgr2=0x7fa3c8154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/1201446313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 0x7fa3c8154210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0098e0 tx=0x7fa3bc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3cefd4700 1 -- 172.21.15.7:0/1201446313 shutdown_connections 2024-01-11T07:10:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/1201446313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 0x7fa3c8154210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/1201446313 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806e200 0x7fa3c8154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3cefd4700 1 -- 172.21.15.7:0/1201446313 >> 172.21.15.7:0/1201446313 conn(0x7fa3c80cd8f0 msgr2=0x7fa3c80cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3cefd4700 1 -- 172.21.15.7:0/1201446313 shutdown_connections 2024-01-11T07:10:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.035+0000 7fa3cefd4700 1 -- 172.21.15.7:0/1201446313 wait complete. 2024-01-11T07:10:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.037+0000 7fa3cefd4700 1 Processor -- start 2024-01-11T07:10:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.037+0000 7fa3cefd4700 1 -- start start 2024-01-11T07:10:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cefd4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806ad40 0x7fa3c816c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:44.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cefd4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 0x7fa3c8150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cefd4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c806e200 0x7fa3c8168040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cefd4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8156090 con 0x7fa3c806e200 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cefd4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8158050 con 0x7fa3c806de30 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cefd4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c816c540 con 0x7fa3c806ad40 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cd571700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c806e200 0x7fa3c8168040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cd571700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c806e200 0x7fa3c8168040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35350/0 (socket says 172.21.15.7:35350) 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3cd571700 1 -- 172.21.15.7:0/263584720 learned_addr learned my addr 172.21.15.7:0/263584720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3ccd70700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806ad40 0x7fa3c816c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.038+0000 7fa3c7fff700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 0x7fa3c8150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3cd571700 1 -- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806ad40 msgr2=0x7fa3c816c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3cd571700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806ad40 0x7fa3c816c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3cd571700 1 -- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 msgr2=0x7fa3c8150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3cd571700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 0x7fa3c8150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3cd571700 1 -- 172.21.15.7:0/263584720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc009420 con 0x7fa3c806e200 2024-01-11T07:10:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3cd571700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c806e200 0x7fa3c8168040 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000cca0 tx=0x7fa3c0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000a8d0 con 0x7fa3c806e200 2024-01-11T07:10:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c000aa30 con 0x7fa3c806e200 2024-01-11T07:10:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c8155240 con 0x7fa3c806e200 2024-01-11T07:10:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000adb0 con 0x7fa3c806e200 2024-01-11T07:10:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.039+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c8155a40 con 0x7fa3c806e200 2024-01-11T07:10:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.040+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c8064a40 con 0x7fa3c806e200 2024-01-11T07:10:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.041+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa3c0007300 con 0x7fa3c806e200 2024-01-11T07:10:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.041+0000 7fa3c4ff9700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3ac066f30 0x7fa3ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.041+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fa3c0084900 con 0x7fa3c806e200 2024-01-11T07:10:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.041+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fa3ac06eaf0 con 0x7fa3c806e200 2024-01-11T07:10:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.041+0000 7fa3ccd70700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3ac066f30 0x7fa3ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.042+0000 7fa3ccd70700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3ac066f30 0x7fa3ac0693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0091e0 tx=0x7fa3bc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.045+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3c0051ba0 con 0x7fa3c806e200 2024-01-11T07:10:44.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.297+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa3c804c3c0 con 0x7fa3c806e200 2024-01-11T07:10:44.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.880+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(73..73 src has 1..73) v4 ==== 1460+0+0 (secure 0 0 0) 0x7fa3c0055f60 con 0x7fa3c806e200 2024-01-11T07:10:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.880+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7fa3ac06fce0 con 0x7fa3c806e200 2024-01-11T07:10:44.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.887+0000 7fa3c4ff9700 1 -- 172.21.15.7:0/263584720 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa3c00551c0 con 0x7fa3c806e200 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3ac066f30 msgr2=0x7fa3ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3ac066f30 0x7fa3ac0693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0091e0 tx=0x7fa3bc017000 comp rx=0 tx=0).stop 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c806e200 msgr2=0x7fa3c8168040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c806e200 0x7fa3c8168040 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000cca0 tx=0x7fa3c0001ae0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 shutdown_connections 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3ac066f30 0x7fa3ac0693e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c806e200 0x7fa3c8168040 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3c806ad40 0x7fa3c816c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 --2- 172.21.15.7:0/263584720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3c806de30 0x7fa3c8150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 >> 172.21.15.7:0/263584720 conn(0x7fa3c80cd8f0 msgr2=0x7fa3c8150d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 shutdown_connections 2024-01-11T07:10:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:44.890+0000 7fa3cefd4700 1 -- 172.21.15.7:0/263584720 wait complete. 2024-01-11T07:10:44.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:10:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.088+0000 7f159449b700 1 Processor -- start 2024-01-11T07:10:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.089+0000 7f159449b700 1 -- start start 2024-01-11T07:10:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f159449b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f158c0cdf70 0x7f158c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f159449b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f158c0ce340 0x7f158c0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f159449b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c067b20 con 0x7f158c0684f0 2024-01-11T07:10:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f159449b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c15e3c0 con 0x7f158c0cdf70 2024-01-11T07:10:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f159449b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c15e6a0 con 0x7f158c0ce340 2024-01-11T07:10:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592237700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f158c0cdf70 0x7f158c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592237700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f158c0cdf70 0x7f158c067fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35362/0 (socket says 172.21.15.7:35362) 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592237700 1 -- 172.21.15.7:0/58472516 learned_addr learned my addr 172.21.15.7:0/58472516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592a38700 1 --2- 172.21.15.7:0/58472516 >> v2:172.21.15.113:3300/0 conn(0x7f158c0ce340 0x7f158c0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592237700 1 -- 172.21.15.7:0/58472516 >> v2:172.21.15.113:3300/0 conn(0x7f158c0ce340 msgr2=0x7f158c0aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592237700 1 --2- 172.21.15.7:0/58472516 >> v2:172.21.15.113:3300/0 conn(0x7f158c0ce340 0x7f158c0aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592237700 1 -- 172.21.15.7:0/58472516 >> v1:172.21.15.113:6789/0 conn(0x7f158c0684f0 legacy=0x7f158c15d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592237700 1 -- 172.21.15.7:0/58472516 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0677b0 con 0x7f158c0cdf70 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592a38700 1 --2- 172.21.15.7:0/58472516 >> v2:172.21.15.113:3300/0 conn(0x7f158c0ce340 0x7f158c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1592237700 1 --2- 172.21.15.7:0/58472516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0cdf70 0x7f158c067fb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f158c15d880 tx=0x7f1588009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e4782691893872 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1591235700 1 -- 172.21.15.7:0/58472516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800b860 con 0x7f158c0cdf70 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1591235700 1 -- 172.21.15.7:0/58472516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158800b9c0 con 0x7f158c0cdf70 2024-01-11T07:10:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.090+0000 7f1591235700 1 -- 172.21.15.7:0/58472516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800bd60 con 0x7f158c0cdf70 2024-01-11T07:10:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.091+0000 7f159449b700 1 -- 172.21.15.7:0/58472516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0cdf70 msgr2=0x7f158c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.091+0000 7f159449b700 1 --2- 172.21.15.7:0/58472516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0cdf70 0x7f158c067fb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f158c15d880 tx=0x7f1588009250 comp rx=0 tx=0).stop 2024-01-11T07:10:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.091+0000 7f159449b700 1 -- 172.21.15.7:0/58472516 shutdown_connections 2024-01-11T07:10:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.091+0000 7f159449b700 1 --2- 172.21.15.7:0/58472516 >> v2:172.21.15.113:3300/0 conn(0x7f158c0ce340 0x7f158c0aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.091+0000 7f159449b700 1 --2- 172.21.15.7:0/58472516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0cdf70 0x7f158c067fb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.091+0000 7f159449b700 1 -- 172.21.15.7:0/58472516 >> 172.21.15.7:0/58472516 conn(0x7f158c06edc0 msgr2=0x7f158c06f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.091+0000 7f159449b700 1 -- 172.21.15.7:0/58472516 shutdown_connections 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.091+0000 7f159449b700 1 -- 172.21.15.7:0/58472516 wait complete. 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.092+0000 7f159449b700 1 Processor -- start 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.092+0000 7f159449b700 1 -- start start 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f159449b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0684f0 0x7f158c14ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f159449b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f158c0cdf70 0x7f158c0af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f159449b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f158c0ce340 0x7f158c0c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f159449b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0b2770 con 0x7f158c0684f0 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f159449b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0b2a10 con 0x7f158c0cdf70 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f159449b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c0b2cf0 con 0x7f158c0ce340 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f1592237700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0684f0 0x7f158c14ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f1592237700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0684f0 0x7f158c14ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35364/0 (socket says 172.21.15.7:35364) 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f1592237700 1 -- 172.21.15.7:0/3154363672 learned_addr learned my addr 172.21.15.7:0/3154363672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f1591a36700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f158c0cdf70 0x7f158c0af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.093+0000 7f1592a38700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f158c0ce340 0x7f158c0c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f1592237700 1 -- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f158c0ce340 msgr2=0x7f158c0c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f1592237700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f158c0ce340 0x7f158c0c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f1592237700 1 -- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f158c0cdf70 msgr2=0x7f158c0af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f1592237700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f158c0cdf70 0x7f158c0af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f1592237700 1 -- 172.21.15.7:0/3154363672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880094f0 con 0x7f158c0684f0 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f1592237700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0684f0 0x7f158c14ce90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1588009290 tx=0x7f1588017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f15827fc700 1 -- 172.21.15.7:0/3154363672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588026480 con 0x7f158c0684f0 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f15827fc700 1 -- 172.21.15.7:0/3154363672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158801f070 con 0x7f158c0684f0 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c0b2fd0 con 0x7f158c0684f0 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f15827fc700 1 -- 172.21.15.7:0/3154363672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588026850 con 0x7f158c0684f0 2024-01-11T07:10:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.094+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c0b3590 con 0x7f158c0684f0 2024-01-11T07:10:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.095+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c14e990 con 0x7f158c0684f0 2024-01-11T07:10:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.095+0000 7f15827fc700 1 -- 172.21.15.7:0/3154363672 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f158801a430 con 0x7f158c0684f0 2024-01-11T07:10:45.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.096+0000 7f15827fc700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1570066f30 0x7f15700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:45.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.096+0000 7f15827fc700 1 -- 172.21.15.7:0/3154363672 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f1588021070 con 0x7f158c0684f0 2024-01-11T07:10:45.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.096+0000 7f1591a36700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1570066f30 0x7f15700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:45.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.097+0000 7f1591a36700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1570066f30 0x7f15700693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f158c153cd0 tx=0x7f157c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.101+0000 7f15827fc700 1 -- 172.21.15.7:0/3154363672 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1588053690 con 0x7f158c0684f0 2024-01-11T07:10:45.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:45.305+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f158c14e800 con 0x7f158c0684f0 2024-01-11T07:10:46.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.307+0000 7f15827fc700 1 -- 172.21.15.7:0/3154363672 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1588056cb0 con 0x7f158c0684f0 2024-01-11T07:10:46.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.311+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1570066f30 msgr2=0x7f15700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:46.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.311+0000 7f159449b700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1570066f30 0x7f15700693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f158c153cd0 tx=0x7f157c007ec0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.311+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0684f0 msgr2=0x7f158c14ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:46.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.311+0000 7f159449b700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0684f0 0x7f158c14ce90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1588009290 tx=0x7f1588017000 comp rx=0 tx=0).stop 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.312+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 shutdown_connections 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.312+0000 7f159449b700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1570066f30 0x7f15700693e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.312+0000 7f159449b700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f158c0ce340 0x7f158c0c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.312+0000 7f159449b700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f158c0684f0 0x7f158c14ce90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.312+0000 7f159449b700 1 --2- 172.21.15.7:0/3154363672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f158c0cdf70 0x7f158c0af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.312+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 >> 172.21.15.7:0/3154363672 conn(0x7f158c06edc0 msgr2=0x7f158c0ab3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.312+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 shutdown_connections 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.312+0000 7f159449b700 1 -- 172.21.15.7:0/3154363672 wait complete. 2024-01-11T07:10:46.312 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:10:46.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:10:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.468+0000 7f0c574d9700 1 Processor -- start 2024-01-11T07:10:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.468+0000 7f0c574d9700 1 -- start start 2024-01-11T07:10:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.469+0000 7f0c574d9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c50063480 0x7f0c5006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.469+0000 7f0c574d9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 0x7f0c50069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.469+0000 7f0c574d9700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5006ac80 con 0x7f0c50063850 2024-01-11T07:10:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.469+0000 7f0c574d9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5006e630 con 0x7f0c50063480 2024-01-11T07:10:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.469+0000 7f0c574d9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5006a9a0 con 0x7f0c50154540 2024-01-11T07:10:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.469+0000 7f0c55275700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c50063480 0x7f0c5006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.469+0000 7f0c55275700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c50063480 0x7f0c5006eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35366/0 (socket says 172.21.15.7:35366) 2024-01-11T07:10:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.469+0000 7f0c55275700 1 -- 172.21.15.7:0/4026135403 learned_addr learned my addr 172.21.15.7:0/4026135403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c55a76700 1 --2- 172.21.15.7:0/4026135403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 0x7f0c50069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c55275700 1 -- 172.21.15.7:0/4026135403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 msgr2=0x7f0c50069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c55275700 1 --2- 172.21.15.7:0/4026135403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 0x7f0c50069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c55275700 1 -- 172.21.15.7:0/4026135403 >> v1:172.21.15.113:6789/0 conn(0x7f0c50063850 legacy=0x7f0c50153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c55275700 1 -- 172.21.15.7:0/4026135403 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5006dfe0 con 0x7f0c50063480 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c55275700 1 --2- 172.21.15.7:0/4026135403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063480 0x7f0c5006eac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0099b0 tx=0x7f0c4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcb5f725aa43e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c43fff700 1 -- 172.21.15.7:0/4026135403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00b860 con 0x7f0c50063480 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c43fff700 1 -- 172.21.15.7:0/4026135403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c00b9c0 con 0x7f0c50063480 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.470+0000 7f0c43fff700 1 -- 172.21.15.7:0/4026135403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00bd60 con 0x7f0c50063480 2024-01-11T07:10:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.471+0000 7f0c574d9700 1 -- 172.21.15.7:0/4026135403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063480 msgr2=0x7f0c5006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:46.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.471+0000 7f0c574d9700 1 --2- 172.21.15.7:0/4026135403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063480 0x7f0c5006eac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0099b0 tx=0x7f0c4c009250 comp rx=0 tx=0).stop 2024-01-11T07:10:46.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.471+0000 7f0c574d9700 1 -- 172.21.15.7:0/4026135403 shutdown_connections 2024-01-11T07:10:46.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.471+0000 7f0c574d9700 1 --2- 172.21.15.7:0/4026135403 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 0x7f0c50069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.471+0000 7f0c574d9700 1 --2- 172.21.15.7:0/4026135403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063480 0x7f0c5006eac0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.471+0000 7f0c574d9700 1 -- 172.21.15.7:0/4026135403 >> 172.21.15.7:0/4026135403 conn(0x7f0c5006bce0 msgr2=0x7f0c5006c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:46.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.471+0000 7f0c574d9700 1 -- 172.21.15.7:0/4026135403 shutdown_connections 2024-01-11T07:10:46.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.471+0000 7f0c574d9700 1 -- 172.21.15.7:0/4026135403 wait complete. 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.472+0000 7f0c574d9700 1 Processor -- start 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.472+0000 7f0c574d9700 1 -- start start 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c574d9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c50063480 0x7f0c50156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c574d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063850 0x7f0c50151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c574d9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 0x7f0c500c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c574d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c500c8360 con 0x7f0c50063850 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c574d9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c500c7750 con 0x7f0c50063480 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c574d9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c500c7a30 con 0x7f0c50154540 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c54a74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063850 0x7f0c50151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:46.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c54a74700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063850 0x7f0c50151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35368/0 (socket says 172.21.15.7:35368) 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c54a74700 1 -- 172.21.15.7:0/729720041 learned_addr learned my addr 172.21.15.7:0/729720041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.473+0000 7f0c55275700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c50063480 0x7f0c50156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c55a76700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 0x7f0c500c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c54a74700 1 -- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 msgr2=0x7f0c500c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c54a74700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 0x7f0c500c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c54a74700 1 -- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c50063480 msgr2=0x7f0c50156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c54a74700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c50063480 0x7f0c50156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c54a74700 1 -- 172.21.15.7:0/729720041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0094f0 con 0x7f0c50063850 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c55275700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c50063480 0x7f0c50156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c54a74700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063850 0x7f0c50151af0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400c590 tx=0x7f0c4400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c417fa700 1 -- 172.21.15.7:0/729720041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44011590 con 0x7f0c50063850 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c417fa700 1 -- 172.21.15.7:0/729720041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c440116f0 con 0x7f0c50063850 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c500c7d10 con 0x7f0c50063850 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c417fa700 1 -- 172.21.15.7:0/729720041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44011ab0 con 0x7f0c50063850 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.474+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c501574b0 con 0x7f0c50063850 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.475+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5004c3c0 con 0x7f0c50063850 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.476+0000 7f0c417fa700 1 -- 172.21.15.7:0/729720041 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0c4401b3b0 con 0x7f0c50063850 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.476+0000 7f0c417fa700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0c34066f30 0x7f0c340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.476+0000 7f0c417fa700 1 -- 172.21.15.7:0/729720041 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f0c440838d0 con 0x7f0c50063850 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.476+0000 7f0c55275700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0c34066f30 0x7f0c340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:46.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.477+0000 7f0c55275700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0c34066f30 0x7f0c340693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0099b0 tx=0x7f0c4c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:46.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.480+0000 7f0c417fa700 1 -- 172.21.15.7:0/729720041 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c44050ca0 con 0x7f0c50063850 2024-01-11T07:10:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:46.685+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c5006e2c0 con 0x7f0c50063850 2024-01-11T07:10:47.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.308+0000 7f0c417fa700 1 -- 172.21.15.7:0/729720041 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0c44017070 con 0x7f0c50063850 2024-01-11T07:10:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0c34066f30 msgr2=0x7f0c340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0c34066f30 0x7f0c340693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0099b0 tx=0x7f0c4c017000 comp rx=0 tx=0).stop 2024-01-11T07:10:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063850 msgr2=0x7f0c50151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063850 0x7f0c50151af0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400c590 tx=0x7f0c4400c950 comp rx=0 tx=0).stop 2024-01-11T07:10:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 shutdown_connections 2024-01-11T07:10:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0c34066f30 0x7f0c340693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c50154540 0x7f0c500c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c50063480 0x7f0c50156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 --2- 172.21.15.7:0/729720041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c50063850 0x7f0c50151af0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 >> 172.21.15.7:0/729720041 conn(0x7f0c5006bce0 msgr2=0x7f0c500cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:47.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 shutdown_connections 2024-01-11T07:10:47.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.312+0000 7f0c574d9700 1 -- 172.21.15.7:0/729720041 wait complete. 2024-01-11T07:10:47.313 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:10:47.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:10:47.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.467+0000 7fa058a0a700 1 Processor -- start 2024-01-11T07:10:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.467+0000 7fa058a0a700 1 -- start start 2024-01-11T07:10:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa058a0a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa05415b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa058a0a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa05415a1e0 0x7fa0540b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa058a0a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0540ca2c0 0x7fa05406e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa058a0a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0540b2e40 con 0x7fa0540ca2c0 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa058a0a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0540b1a00 con 0x7fa05415a1e0 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa058a0a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05406e110 con 0x7fa054159e10 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa052d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0540ca2c0 0x7fa05406e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa052d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0540ca2c0 0x7fa05406e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41424/0 (socket says 172.21.15.7:41424) 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa052d9d700 1 -- 172.21.15.7:0/3441529969 learned_addr learned my addr 172.21.15.7:0/3441529969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa051d9b700 1 --2- 172.21.15.7:0/3441529969 >> v2:172.21.15.113:3300/0 conn(0x7fa05415a1e0 0x7fa0540b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa05259c700 1 --2- 172.21.15.7:0/3441529969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa05415b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa052d9d700 1 -- 172.21.15.7:0/3441529969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 msgr2=0x7fa05415b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa052d9d700 1 --2- 172.21.15.7:0/3441529969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa05415b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa052d9d700 1 -- 172.21.15.7:0/3441529969 >> v2:172.21.15.113:3300/0 conn(0x7fa05415a1e0 msgr2=0x7fa0540b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa052d9d700 1 --2- 172.21.15.7:0/3441529969 >> v2:172.21.15.113:3300/0 conn(0x7fa05415a1e0 0x7fa0540b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa052d9d700 1 -- 172.21.15.7:0/3441529969 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540b1c30 con 0x7fa0540ca2c0 2024-01-11T07:10:47.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa051d9b700 1 --2- 172.21.15.7:0/3441529969 >> v2:172.21.15.113:3300/0 conn(0x7fa05415a1e0 0x7fa0540b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.468+0000 7fa05259c700 1 --2- 172.21.15.7:0/3441529969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa05415b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.469+0000 7fa052d9d700 1 --2- 172.21.15.7:0/3441529969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa05406e5a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa054069000 tx=0x7fa04c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03f8c3b57247236 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.469+0000 7fa05159a700 1 -- 172.21.15.7:0/3441529969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00e070 con 0x7fa0540ca2c0 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.469+0000 7fa05159a700 1 -- 172.21.15.7:0/3441529969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c00ba50 con 0x7fa0540ca2c0 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.469+0000 7fa05159a700 1 -- 172.21.15.7:0/3441529969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0092e0 con 0x7fa0540ca2c0 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.469+0000 7fa058a0a700 1 -- 172.21.15.7:0/3441529969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 msgr2=0x7fa05406e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.469+0000 7fa058a0a700 1 --2- 172.21.15.7:0/3441529969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa05406e5a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa054069000 tx=0x7fa04c009840 comp rx=0 tx=0).stop 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.470+0000 7fa058a0a700 1 -- 172.21.15.7:0/3441529969 shutdown_connections 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.470+0000 7fa058a0a700 1 --2- 172.21.15.7:0/3441529969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa05406e5a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.470+0000 7fa058a0a700 1 --2- 172.21.15.7:0/3441529969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa05415b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.470+0000 7fa058a0a700 1 --2- 172.21.15.7:0/3441529969 >> v2:172.21.15.113:3300/0 conn(0x7fa05415a1e0 0x7fa0540b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.470+0000 7fa058a0a700 1 -- 172.21.15.7:0/3441529969 >> 172.21.15.7:0/3441529969 conn(0x7fa0540cd910 msgr2=0x7fa0540cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.470+0000 7fa058a0a700 1 -- 172.21.15.7:0/3441529969 shutdown_connections 2024-01-11T07:10:47.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.470+0000 7fa058a0a700 1 -- 172.21.15.7:0/3441529969 wait complete. 2024-01-11T07:10:47.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.471+0000 7fa058a0a700 1 Processor -- start 2024-01-11T07:10:47.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.471+0000 7fa058a0a700 1 -- start start 2024-01-11T07:10:47.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa058a0a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa0540ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa058a0a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa0540ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa058a0a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05415a1e0 0x7fa054143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa058a0a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540cac40 con 0x7fa0540ca2c0 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa058a0a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c7580 con 0x7fa05415a1e0 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa058a0a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa054169ce0 con 0x7fa054159e10 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa0540ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa0540ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41436/0 (socket says 172.21.15.7:41436) 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 -- 172.21.15.7:0/2565982621 learned_addr learned my addr 172.21.15.7:0/2565982621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa051d9b700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa0540ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa052d9d700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05415a1e0 0x7fa054143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 -- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 msgr2=0x7fa0540ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa0540ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 -- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05415a1e0 msgr2=0x7fa054143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05415a1e0 0x7fa054143490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 -- 172.21.15.7:0/2565982621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c009ae0 con 0x7fa0540ca2c0 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa052d9d700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05415a1e0 0x7fa054143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa05259c700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa0540ccbb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c880 tx=0x7fa044008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.472+0000 7fa0427fc700 1 -- 172.21.15.7:0/2565982621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044011660 con 0x7fa0540ca2c0 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.473+0000 7fa0427fc700 1 -- 172.21.15.7:0/2565982621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0440117c0 con 0x7fa0540ca2c0 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.473+0000 7fa0427fc700 1 -- 172.21.15.7:0/2565982621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044011b60 con 0x7fa0540ca2c0 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.473+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0540c6c90 con 0x7fa0540ca2c0 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.473+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0540c6f30 con 0x7fa0540ca2c0 2024-01-11T07:10:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.474+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05404c3c0 con 0x7fa0540ca2c0 2024-01-11T07:10:47.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.474+0000 7fa0427fc700 1 -- 172.21.15.7:0/2565982621 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa044021350 con 0x7fa0540ca2c0 2024-01-11T07:10:47.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.475+0000 7fa0427fc700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa034066f30 0x7fa0340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.475+0000 7fa0427fc700 1 -- 172.21.15.7:0/2565982621 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa0440841e0 con 0x7fa0540ca2c0 2024-01-11T07:10:47.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.475+0000 7fa051d9b700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa034066f30 0x7fa0340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.475+0000 7fa051d9b700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa034066f30 0x7fa0340693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa0540c9970 tx=0x7fa048008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:47.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.479+0000 7fa0427fc700 1 -- 172.21.15.7:0/2565982621 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0440551e0 con 0x7fa0540ca2c0 2024-01-11T07:10:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.683+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa0540b1f10 con 0x7fa0540ca2c0 2024-01-11T07:10:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.687+0000 7fa0427fc700 1 -- 172.21.15.7:0/2565982621 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa044054d70 con 0x7fa0540ca2c0 2024-01-11T07:10:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa034066f30 msgr2=0x7fa0340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa034066f30 0x7fa0340693e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa0540c9970 tx=0x7fa048008040 comp rx=0 tx=0).stop 2024-01-11T07:10:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 msgr2=0x7fa0540ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa0540ccbb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c880 tx=0x7fa044008040 comp rx=0 tx=0).stop 2024-01-11T07:10:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 shutdown_connections 2024-01-11T07:10:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa034066f30 0x7fa0340693e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa05415a1e0 0x7fa054143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0540ca2c0 0x7fa0540ccbb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 --2- 172.21.15.7:0/2565982621 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa054159e10 0x7fa0540ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.689+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 >> 172.21.15.7:0/2565982621 conn(0x7fa0540cd910 msgr2=0x7fa054157740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.690+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 shutdown_connections 2024-01-11T07:10:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.690+0000 7fa058a0a700 1 -- 172.21.15.7:0/2565982621 wait complete. 2024-01-11T07:10:47.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-01-11T07:10:47.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.840+0000 7ff88cec1700 1 Processor -- start 2024-01-11T07:10:47.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.840+0000 7ff88cec1700 1 -- start start 2024-01-11T07:10:47.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.841+0000 7ff88cec1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8880d0170 0x7ff8880cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.841+0000 7ff88cec1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88815dfb0 0x7ff888159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.841+0000 7ff88cec1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88815acf0 con 0x7ff8880cd6d0 2024-01-11T07:10:47.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.841+0000 7ff88cec1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88815afd0 con 0x7ff8880d0170 2024-01-11T07:10:47.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.841+0000 7ff88cec1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880ce980 con 0x7ff88815dfb0 2024-01-11T07:10:47.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.841+0000 7ff88659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff8880cd6d0 0x7ff88815d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.841+0000 7ff88659c700 1 -- 172.21.15.7:0/1593385589 learned_addr learned my addr 172.21.15.7:0/1593385589 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff885d9b700 1 --2- 172.21.15.7:0/1593385589 >> v2:172.21.15.113:3300/0 conn(0x7ff8880d0170 0x7ff8880cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff886d9d700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88815dfb0 0x7ff888159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227374680 0 0) 0x7ff88815acf0 con 0x7ff8880cd6d0 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff870003570 con 0x7ff8880cd6d0 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3308087839 0 0) 0x7ff870003570 con 0x7ff8880cd6d0 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88815dfb0 msgr2=0x7ff888159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88815dfb0 0x7ff888159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 >> v2:172.21.15.113:3300/0 conn(0x7ff8880d0170 msgr2=0x7ff8880cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 --2- 172.21.15.7:0/1593385589 >> v2:172.21.15.113:3300/0 conn(0x7ff8880d0170 0x7ff8880cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880cec60 con 0x7ff8880cd6d0 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7ff878001470 con 0x7ff8880cd6d0 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 >> v1:172.21.15.7:6789/0 conn(0x7ff8880cd6d0 legacy=0x7ff88815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff870003a60 0x7ff870003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff870004810 0x7ff870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88815dfb0 0x7ff8700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88815afd0 con 0x7ff870003a60 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880ce980 con 0x7ff870004810 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000ba60 con 0x7ff88815dfb0 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88659c700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff870003a60 0x7ff870003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff885d9b700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff870004810 0x7ff870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff886d9d700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88815dfb0 0x7ff8700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88659c700 1 -- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88815dfb0 msgr2=0x7ff8700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88659c700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff88815dfb0 0x7ff8700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88659c700 1 -- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff870004810 msgr2=0x7ff870004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88659c700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff870004810 0x7ff870004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.842+0000 7ff88659c700 1 -- 172.21.15.7:0/1593385589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870003570 con 0x7ff870003a60 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.843+0000 7ff88659c700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff870003a60 0x7ff870003e70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c3f0 tx=0x7ff878001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb845582ec167b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.843+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87800cc80 con 0x7ff870003a60 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.843+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87800ce20 con 0x7ff870003a60 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.843+0000 7ff88559a700 1 -- 172.21.15.7:0/1593385589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878005b70 con 0x7ff870003a60 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.843+0000 7ff88cec1700 1 -- 172.21.15.7:0/1593385589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff870003a60 msgr2=0x7ff870003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.843+0000 7ff88cec1700 1 --2- 172.21.15.7:0/1593385589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff870003a60 0x7ff870003e70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c3f0 tx=0x7ff878001e90 comp rx=0 tx=0).stop 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.843+0000 7ff886d9d700 1 -- 172.21.15.7:0/1593385589 reap_dead start 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.844+0000 7ff88cec1700 1 -- 172.21.15.7:0/1593385589 shutdown_connections 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.844+0000 7ff88cec1700 1 -- 172.21.15.7:0/1593385589 >> 172.21.15.7:0/1593385589 conn(0x7ff888071120 msgr2=0x7ff88806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.844+0000 7ff88cec1700 1 -- 172.21.15.7:0/1593385589 shutdown_connections 2024-01-11T07:10:47.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.844+0000 7ff88cec1700 1 -- 172.21.15.7:0/1593385589 wait complete. 2024-01-11T07:10:47.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.845+0000 7ff88cec1700 1 Processor -- start 2024-01-11T07:10:47.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.845+0000 7ff88cec1700 1 -- start start 2024-01-11T07:10:47.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff88cec1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff870003a60 0x7ff8880b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff88cec1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880cadd0 0x7ff888155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff88cec1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8880b4050 0x7ff8880b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff88cec1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880b5c00 con 0x7ff8880cadd0 2024-01-11T07:10:47.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff88cec1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880b56b0 con 0x7ff8880b4050 2024-01-11T07:10:47.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff88cec1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880cb350 con 0x7ff870003a60 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880cadd0 0x7ff888155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880cadd0 0x7ff888155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41456/0 (socket says 172.21.15.7:41456) 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 -- 172.21.15.7:0/1405574072 learned_addr learned my addr 172.21.15.7:0/1405574072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff88659c700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff870003a60 0x7ff8880b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff886d9d700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8880b4050 0x7ff8880b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 -- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff870003a60 msgr2=0x7ff8880b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff870003a60 0x7ff8880b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 -- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8880b4050 msgr2=0x7ff8880b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8880b4050 0x7ff8880b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 -- 172.21.15.7:0/1405574072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880cb630 con 0x7ff8880cadd0 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff885d9b700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880cadd0 0x7ff888155be0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff87c011d40 tx=0x7ff87c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.846+0000 7ff8767fc700 1 -- 172.21.15.7:0/1405574072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00f990 con 0x7ff8880cadd0 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.847+0000 7ff8767fc700 1 -- 172.21.15.7:0/1405574072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c00faf0 con 0x7ff8880cadd0 2024-01-11T07:10:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.847+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8880cb910 con 0x7ff8880cadd0 2024-01-11T07:10:47.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.847+0000 7ff8767fc700 1 -- 172.21.15.7:0/1405574072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00feb0 con 0x7ff8880cadd0 2024-01-11T07:10:47.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.847+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff888156120 con 0x7ff8880cadd0 2024-01-11T07:10:47.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.848+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8880b3cf0 con 0x7ff8880cadd0 2024-01-11T07:10:47.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.848+0000 7ff8767fc700 1 -- 172.21.15.7:0/1405574072 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff87c014650 con 0x7ff8880cadd0 2024-01-11T07:10:47.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.848+0000 7ff8767fc700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff868066f30 0x7ff8680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:47.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.849+0000 7ff8767fc700 1 -- 172.21.15.7:0/1405574072 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff87c018030 con 0x7ff8880cadd0 2024-01-11T07:10:47.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.849+0000 7ff88659c700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff868066f30 0x7ff8680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:47.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.849+0000 7ff88659c700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff868066f30 0x7ff8680693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c3f0 tx=0x7ff878015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:47.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:47.853+0000 7ff8767fc700 1 -- 172.21.15.7:0/1405574072 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff87c052450 con 0x7ff8880cadd0 2024-01-11T07:10:48.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.139+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7ff888156400 con 0x7ff8880cadd0 2024-01-11T07:10:48.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.310+0000 7ff8767fc700 1 -- 172.21.15.7:0/1405574072 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v199) v1 ==== 134+0+0 (secure 0 0 0) 0x7ff888156400 con 0x7ff8880cadd0 2024-01-11T07:10:48.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff868066f30 msgr2=0x7ff8680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:48.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff868066f30 0x7ff8680693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff87800c3f0 tx=0x7ff878015000 comp rx=0 tx=0).stop 2024-01-11T07:10:48.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880cadd0 msgr2=0x7ff888155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:48.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880cadd0 0x7ff888155be0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff87c011d40 tx=0x7ff87c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:48.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 shutdown_connections 2024-01-11T07:10:48.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff868066f30 0x7ff8680693e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:48.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8880b4050 0x7ff8880b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:48.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff870003a60 0x7ff8880b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:48.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 --2- 172.21.15.7:0/1405574072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880cadd0 0x7ff888155be0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:48.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 >> 172.21.15.7:0/1405574072 conn(0x7ff888071120 msgr2=0x7ff8880b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:48.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 shutdown_connections 2024-01-11T07:10:48.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.312+0000 7ff88cec1700 1 -- 172.21.15.7:0/1405574072 wait complete. 2024-01-11T07:10:48.327 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-01-11T07:10:48.328 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2024-01-11T07:10:48.328 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2024-01-11T07:10:48.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-01-11T07:10:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.463+0000 7f8477c50700 1 Processor -- start 2024-01-11T07:10:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.464+0000 7f8477c50700 1 -- start start 2024-01-11T07:10:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f8477c50700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f847015dd90 0x7f84700aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f8477c50700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847006ac30 con 0x7f847015d920 2024-01-11T07:10:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f8477c50700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84700cb370 con 0x7f84700ac3a0 2024-01-11T07:10:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f8477c50700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84700ac770 con 0x7f847015dd90 2024-01-11T07:10:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84761ed700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f847015dd90 0x7f84700aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84751eb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f847015d920 0x7f84700aa8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40306/0 (socket says 172.21.15.7:40306) 2024-01-11T07:10:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84751eb700 1 -- 172.21.15.7:0/4093099101 learned_addr learned my addr 172.21.15.7:0/4093099101 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84761ed700 1 -- 172.21.15.7:0/4093099101 >> v1:172.21.15.113:6789/0 conn(0x7f84700ac3a0 legacy=0x7f84700cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84761ed700 1 -- 172.21.15.7:0/4093099101 >> v1:172.21.15.7:6789/0 conn(0x7f847015d920 legacy=0x7f84700aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84761ed700 1 -- 172.21.15.7:0/4093099101 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847006a300 con 0x7f847015dd90 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84749ea700 1 -- 172.21.15.7:0/4093099101 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232656650 0 0) 0x7f847006ac30 con 0x7f847015d920 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84761ed700 1 --2- 172.21.15.7:0/4093099101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015dd90 0x7f84700aafd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8468009e60 tx=0x7f8468009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c38c93637627273 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.465+0000 7f84749ea700 1 -- 172.21.15.7:0/4093099101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846800e070 con 0x7f847015dd90 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f84749ea700 1 -- 172.21.15.7:0/4093099101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846800bc30 con 0x7f847015dd90 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f84749ea700 1 -- 172.21.15.7:0/4093099101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468012430 con 0x7f847015dd90 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f8477c50700 1 -- 172.21.15.7:0/4093099101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015dd90 msgr2=0x7f84700aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f8477c50700 1 --2- 172.21.15.7:0/4093099101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015dd90 0x7f84700aafd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8468009e60 tx=0x7f8468009780 comp rx=0 tx=0).stop 2024-01-11T07:10:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f8477c50700 1 -- 172.21.15.7:0/4093099101 shutdown_connections 2024-01-11T07:10:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f8477c50700 1 --2- 172.21.15.7:0/4093099101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015dd90 0x7f84700aafd0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f8477c50700 1 -- 172.21.15.7:0/4093099101 >> 172.21.15.7:0/4093099101 conn(0x7f84700cd090 msgr2=0x7f84700cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f8477c50700 1 -- 172.21.15.7:0/4093099101 shutdown_connections 2024-01-11T07:10:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.466+0000 7f8477c50700 1 -- 172.21.15.7:0/4093099101 wait complete. 2024-01-11T07:10:48.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.467+0000 7f8477c50700 1 Processor -- start 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f8477c50700 1 -- start start 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f8477c50700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84700ac3a0 0x7f847013aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f8477c50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015d920 0x7f8470157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f8477c50700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f847015dd90 0x7f8470158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f8477c50700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700b21a0 con 0x7f847015d920 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f8477c50700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700b1590 con 0x7f84700ac3a0 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f8477c50700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700b1870 con 0x7f847015dd90 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f84751eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015d920 0x7f8470157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f84751eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015d920 0x7f8470157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41476/0 (socket says 172.21.15.7:41476) 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.468+0000 7f84751eb700 1 -- 172.21.15.7:0/497927379 learned_addr learned my addr 172.21.15.7:0/497927379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84759ec700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84700ac3a0 0x7f847013aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84761ed700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f847015dd90 0x7f8470158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84751eb700 1 -- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f847015dd90 msgr2=0x7f8470158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:48.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84751eb700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f847015dd90 0x7f8470158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84751eb700 1 -- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84700ac3a0 msgr2=0x7f847013aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84751eb700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84700ac3a0 0x7f847013aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84751eb700 1 -- 172.21.15.7:0/497927379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468009a20 con 0x7f847015d920 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84761ed700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f847015dd90 0x7f8470158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f84751eb700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015d920 0x7f8470157f40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f846000c7e0 tx=0x7f8460007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f8465ffb700 1 -- 172.21.15.7:0/497927379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460005b30 con 0x7f847015d920 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f8465ffb700 1 -- 172.21.15.7:0/497927379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8460005c90 con 0x7f847015d920 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84700b1b50 con 0x7f847015d920 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f8465ffb700 1 -- 172.21.15.7:0/497927379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460007300 con 0x7f847015d920 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.469+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847013af40 con 0x7f847015d920 2024-01-11T07:10:48.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.470+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84700b2310 con 0x7f847015d920 2024-01-11T07:10:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.471+0000 7f8465ffb700 1 -- 172.21.15.7:0/497927379 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f846001b400 con 0x7f847015d920 2024-01-11T07:10:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.471+0000 7f8465ffb700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.471+0000 7f8465ffb700 1 -- 172.21.15.7:0/497927379 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8460083d10 con 0x7f847015d920 2024-01-11T07:10:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.472+0000 7f84759ec700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.472+0000 7f84759ec700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f846c003860 tx=0x7f846c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.476+0000 7f8465ffb700 1 -- 172.21.15.7:0/497927379 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8460051300 con 0x7f847015d920 2024-01-11T07:10:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:48.762+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f847004c3c0 con 0x7f847015d920 2024-01-11T07:10:49.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.311+0000 7f8465ffb700 1 -- 172.21.15.7:0/497927379 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v200) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8460054920 con 0x7f847015d920 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 msgr2=0x7f84540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f846c003860 tx=0x7f846c009040 comp rx=0 tx=0).stop 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015d920 msgr2=0x7f8470157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015d920 0x7f8470157f40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f846000c7e0 tx=0x7f8460007df0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 shutdown_connections 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8454066f30 0x7f84540693e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f847015dd90 0x7f8470158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84700ac3a0 0x7f847013aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 --2- 172.21.15.7:0/497927379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847015d920 0x7f8470157f40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 >> 172.21.15.7:0/497927379 conn(0x7f84700cd090 msgr2=0x7f847013c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:49.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 shutdown_connections 2024-01-11T07:10:49.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.314+0000 7f8477c50700 1 -- 172.21.15.7:0/497927379 wait complete. 2024-01-11T07:10:49.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-11T07:10:49.482 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-11T07:10:49.483 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-11T07:10:49.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:10:49.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.618+0000 7fa94ce8c700 1 Processor -- start 2024-01-11T07:10:49.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.618+0000 7fa94ce8c700 1 -- start start 2024-01-11T07:10:49.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa94ce8c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9480cd500 0x7fa948157e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:49.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa94ce8c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9480c8f60 0x7fa9480c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:49.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa94ce8c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9480ca3d0 con 0x7fa948157a90 2024-01-11T07:10:49.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa94ce8c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9480c9940 con 0x7fa9480cd500 2024-01-11T07:10:49.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa94ce8c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480cf8c0 con 0x7fa9480c8f60 2024-01-11T07:10:49.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa945d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9480cd500 0x7fa948157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:49.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa94659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa948157a90 0x7fa9480cf1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40312/0 (socket says 172.21.15.7:40312) 2024-01-11T07:10:49.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa94659c700 1 -- 172.21.15.7:0/2143422852 learned_addr learned my addr 172.21.15.7:0/2143422852 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:49.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa946d9d700 1 --2- 172.21.15.7:0/2143422852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9480c8f60 0x7fa9480c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:49.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa945d9b700 1 -- 172.21.15.7:0/2143422852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9480c8f60 msgr2=0x7fa9480c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:49.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa945d9b700 1 --2- 172.21.15.7:0/2143422852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9480c8f60 0x7fa9480c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa945d9b700 1 -- 172.21.15.7:0/2143422852 >> v1:172.21.15.7:6789/0 conn(0x7fa948157a90 legacy=0x7fa9480cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:49.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.619+0000 7fa945d9b700 1 -- 172.21.15.7:0/2143422852 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480b1790 con 0x7fa9480cd500 2024-01-11T07:10:49.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.620+0000 7fa945d9b700 1 --2- 172.21.15.7:0/2143422852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480cd500 0x7fa948157e60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9300098e0 tx=0x7fa9300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882e4fb1902d350a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:49.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.620+0000 7fa94559a700 1 -- 172.21.15.7:0/2143422852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930009e80 con 0x7fa9480cd500 2024-01-11T07:10:49.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.620+0000 7fa94559a700 1 -- 172.21.15.7:0/2143422852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93000b860 con 0x7fa9480cd500 2024-01-11T07:10:49.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.620+0000 7fa94559a700 1 -- 172.21.15.7:0/2143422852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000bbe0 con 0x7fa9480cd500 2024-01-11T07:10:49.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.620+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2143422852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480cd500 msgr2=0x7fa948157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:49.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.620+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2143422852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480cd500 0x7fa948157e60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9300098e0 tx=0x7fa9300091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.621+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2143422852 shutdown_connections 2024-01-11T07:10:49.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.621+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2143422852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9480c8f60 0x7fa9480c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.621+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2143422852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480cd500 0x7fa948157e60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.621+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2143422852 >> 172.21.15.7:0/2143422852 conn(0x7fa948097a50 msgr2=0x7fa9480ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.621+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2143422852 shutdown_connections 2024-01-11T07:10:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.621+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2143422852 wait complete. 2024-01-11T07:10:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.622+0000 7fa94ce8c700 1 Processor -- start 2024-01-11T07:10:49.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.622+0000 7fa94ce8c700 1 -- start start 2024-01-11T07:10:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94ce8c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480c8f60 0x7fa948154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94ce8c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9480cd500 0x7fa9480b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94ce8c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa948157a90 0x7fa94815b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94ce8c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480c4040 con 0x7fa9480c8f60 2024-01-11T07:10:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94ce8c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480c3430 con 0x7fa9480cd500 2024-01-11T07:10:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94ce8c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480c3710 con 0x7fa948157a90 2024-01-11T07:10:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480c8f60 0x7fa948154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480c8f60 0x7fa948154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41490/0 (socket says 172.21.15.7:41490) 2024-01-11T07:10:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 -- 172.21.15.7:0/2377656343 learned_addr learned my addr 172.21.15.7:0/2377656343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa945d9b700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9480cd500 0x7fa9480b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa946d9d700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa948157a90 0x7fa94815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 -- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa948157a90 msgr2=0x7fa94815b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa948157a90 0x7fa94815b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 -- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9480cd500 msgr2=0x7fa9480b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9480cd500 0x7fa9480b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 -- 172.21.15.7:0/2377656343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa930009420 con 0x7fa9480c8f60 2024-01-11T07:10:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa946d9d700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa948157a90 0x7fa94815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa945d9b700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9480cd500 0x7fa9480b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa94659c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480c8f60 0x7fa948154c90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c7c0 tx=0x7fa938007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.623+0000 7fa93e7fc700 1 -- 172.21.15.7:0/2377656343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9380073d0 con 0x7fa9480c8f60 2024-01-11T07:10:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.624+0000 7fa93e7fc700 1 -- 172.21.15.7:0/2377656343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938007530 con 0x7fa9480c8f60 2024-01-11T07:10:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.624+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9480c39f0 con 0x7fa9480c8f60 2024-01-11T07:10:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.624+0000 7fa93e7fc700 1 -- 172.21.15.7:0/2377656343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938005a40 con 0x7fa9480c8f60 2024-01-11T07:10:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.624+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9480c3050 con 0x7fa9480c8f60 2024-01-11T07:10:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.625+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94804c3c0 con 0x7fa9480c8f60 2024-01-11T07:10:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.625+0000 7fa93e7fc700 1 -- 172.21.15.7:0/2377656343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa93801b400 con 0x7fa9480c8f60 2024-01-11T07:10:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.625+0000 7fa93e7fc700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa928066f30 0x7fa9280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.626+0000 7fa93e7fc700 1 -- 172.21.15.7:0/2377656343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa938083ad0 con 0x7fa9480c8f60 2024-01-11T07:10:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.626+0000 7fa945d9b700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa928066f30 0x7fa9280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.626+0000 7fa945d9b700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa928066f30 0x7fa9280693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa94809c710 tx=0x7fa930017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.630+0000 7fa93e7fc700 1 -- 172.21.15.7:0/2377656343 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9380510c0 con 0x7fa9480c8f60 2024-01-11T07:10:49.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.832+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa9480ca3b0 con 0x7fa9480c8f60 2024-01-11T07:10:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.836+0000 7fa93e7fc700 1 -- 172.21.15.7:0/2377656343 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa9380546e0 con 0x7fa9480c8f60 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa928066f30 msgr2=0x7fa9280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa928066f30 0x7fa9280693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa94809c710 tx=0x7fa930017000 comp rx=0 tx=0).stop 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480c8f60 msgr2=0x7fa948154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480c8f60 0x7fa948154c90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa93800c7c0 tx=0x7fa938007de0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 shutdown_connections 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa928066f30 0x7fa9280693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa948157a90 0x7fa94815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480c8f60 0x7fa948154c90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 --2- 172.21.15.7:0/2377656343 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9480cd500 0x7fa9480b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 >> 172.21.15.7:0/2377656343 conn(0x7fa948097a50 msgr2=0x7fa94806a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 shutdown_connections 2024-01-11T07:10:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:49.838+0000 7fa94ce8c700 1 -- 172.21.15.7:0/2377656343 wait complete. 2024-01-11T07:10:49.851 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:10:49.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:10:49.901 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:10:49.951 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:10:49.951 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:10:49.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:10:50.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.089+0000 7f4e50e64700 1 Processor -- start 2024-01-11T07:10:50.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.089+0000 7f4e50e64700 1 -- start start 2024-01-11T07:10:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.090+0000 7f4e50e64700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e4c06c360 0x7f4e4c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e50e64700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c15e010 con 0x7f4e4c159e20 2024-01-11T07:10:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e50e64700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0acf80 con 0x7f4e4c159a50 2024-01-11T07:10:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e50e64700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c15d9c0 con 0x7f4e4c06c360 2024-01-11T07:10:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e49d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4e4c159e20 0x7f4e4c06bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40320/0 (socket says 172.21.15.7:40320) 2024-01-11T07:10:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e49d9b700 1 -- 172.21.15.7:0/3339580678 learned_addr learned my addr 172.21.15.7:0/3339580678 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e4ad9d700 1 --2- 172.21.15.7:0/3339580678 >> v2:172.21.15.113:3300/0 conn(0x7f4e4c06c360 0x7f4e4c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e4959a700 1 -- 172.21.15.7:0/3339580678 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069628566 0 0) 0x7f4e4c15e010 con 0x7f4e4c159e20 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e4959a700 1 -- 172.21.15.7:0/3339580678 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e34003570 con 0x7f4e4c159e20 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e4ad9d700 1 -- 172.21.15.7:0/3339580678 >> v1:172.21.15.113:6789/0 conn(0x7f4e4c159a50 legacy=0x7f4e4c0ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e4ad9d700 1 -- 172.21.15.7:0/3339580678 >> v1:172.21.15.7:6789/0 conn(0x7f4e4c159e20 legacy=0x7f4e4c06bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e4959a700 1 -- 172.21.15.7:0/3339580678 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111891142 0 0) 0x7f4e4c0acf80 con 0x7f4e4c159a50 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.091+0000 7f4e4ad9d700 1 -- 172.21.15.7:0/3339580678 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c15dca0 con 0x7f4e4c06c360 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e4ad9d700 1 --2- 172.21.15.7:0/3339580678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 0x7f4e4c15e4a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009e60 tx=0x7f4e44009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf0d39b25e93eee server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e4959a700 1 -- 172.21.15.7:0/3339580678 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400e070 con 0x7f4e4c06c360 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e4959a700 1 -- 172.21.15.7:0/3339580678 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4400b850 con 0x7f4e4c06c360 2024-01-11T07:10:50.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e4959a700 1 -- 172.21.15.7:0/3339580678 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40001470 con 0x7f4e4c06c360 2024-01-11T07:10:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e50e64700 1 -- 172.21.15.7:0/3339580678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 msgr2=0x7f4e4c15e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e50e64700 1 --2- 172.21.15.7:0/3339580678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 0x7f4e4c15e4a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009e60 tx=0x7f4e44009780 comp rx=0 tx=0).stop 2024-01-11T07:10:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e50e64700 1 -- 172.21.15.7:0/3339580678 shutdown_connections 2024-01-11T07:10:50.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e50e64700 1 --2- 172.21.15.7:0/3339580678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 0x7f4e4c15e4a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e50e64700 1 -- 172.21.15.7:0/3339580678 >> 172.21.15.7:0/3339580678 conn(0x7f4e4c0cd910 msgr2=0x7f4e4c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e50e64700 1 -- 172.21.15.7:0/3339580678 shutdown_connections 2024-01-11T07:10:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.092+0000 7f4e50e64700 1 -- 172.21.15.7:0/3339580678 wait complete. 2024-01-11T07:10:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.094+0000 7f4e50e64700 1 Processor -- start 2024-01-11T07:10:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.094+0000 7f4e50e64700 1 -- start start 2024-01-11T07:10:50.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.094+0000 7f4e50e64700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 0x7f4e4c0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.094+0000 7f4e50e64700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c159a50 0x7f4e4c15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.094+0000 7f4e50e64700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e4c159e20 0x7f4e4c16d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.094+0000 7f4e50e64700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c16de80 con 0x7f4e4c159a50 2024-01-11T07:10:50.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.094+0000 7f4e50e64700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c15a450 con 0x7f4e4c06c360 2024-01-11T07:10:50.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.094+0000 7f4e50e64700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c15d1a0 con 0x7f4e4c159e20 2024-01-11T07:10:50.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c159a50 0x7f4e4c15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c159a50 0x7f4e4c15c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41498/0 (socket says 172.21.15.7:41498) 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 -- 172.21.15.7:0/2013193852 learned_addr learned my addr 172.21.15.7:0/2013193852 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e4ad9d700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e4c159e20 0x7f4e4c16d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e4a59c700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 0x7f4e4c0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 -- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e4c159e20 msgr2=0x7f4e4c16d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e4c159e20 0x7f4e4c16d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 -- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 msgr2=0x7f4e4c0ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 0x7f4e4c0ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 -- 172.21.15.7:0/2013193852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44009a20 con 0x7f4e4c159a50 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e4ad9d700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e4c159e20 0x7f4e4c16d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e4a59c700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 0x7f4e4c0ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e49d9b700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c159a50 0x7f4e4c15c200 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4e400015a0 tx=0x7f4e40001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e3a7fc700 1 -- 172.21.15.7:0/2013193852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40007360 con 0x7f4e4c159a50 2024-01-11T07:10:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e3a7fc700 1 -- 172.21.15.7:0/2013193852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40007520 con 0x7f4e4c159a50 2024-01-11T07:10:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c15c900 con 0x7f4e4c159a50 2024-01-11T07:10:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e3a7fc700 1 -- 172.21.15.7:0/2013193852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40004a80 con 0x7f4e4c159a50 2024-01-11T07:10:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.095+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c15cec0 con 0x7f4e4c159a50 2024-01-11T07:10:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.096+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c04c3c0 con 0x7f4e4c159a50 2024-01-11T07:10:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.097+0000 7f4e3a7fc700 1 -- 172.21.15.7:0/2013193852 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4e4000a570 con 0x7f4e4c159a50 2024-01-11T07:10:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.097+0000 7f4e3a7fc700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4e2c066f30 0x7f4e2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.097+0000 7f4e3a7fc700 1 -- 172.21.15.7:0/2013193852 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4e400837e0 con 0x7f4e4c159a50 2024-01-11T07:10:50.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.098+0000 7f4e4a59c700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4e2c066f30 0x7f4e2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.098+0000 7f4e4a59c700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4e2c066f30 0x7f4e2c0693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c145bc0 tx=0x7f4e3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:50.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.102+0000 7f4e3a7fc700 1 -- 172.21.15.7:0/2013193852 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e40054740 con 0x7f4e4c159a50 2024-01-11T07:10:50.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.354+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e4c0ad260 con 0x7f4e4c159a50 2024-01-11T07:10:50.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.354+0000 7f4e3a7fc700 1 -- 172.21.15.7:0/2013193852 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f4e40018090 con 0x7f4e4c159a50 2024-01-11T07:10:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:50.354 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5213,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1734166534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1734166534},{"type":"v1","addr":"172.21.15.113:6836","nonce":1734166534}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3195231529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3195231529},{"type":"v1","addr":"172.21.15.113:6837","nonce":3195231529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5709,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/338342475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":338342475},{"type":"v1","addr":"172.21.15.7:6834","nonce":338342475}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":5937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2970910885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2970910885},{"type":"v1","addr":"172.21.15.7:6835","nonce":2970910885}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[]} 2024-01-11T07:10:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4e2c066f30 msgr2=0x7f4e2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4e2c066f30 0x7f4e2c0693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c145bc0 tx=0x7f4e3c009040 comp rx=0 tx=0).stop 2024-01-11T07:10:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c159a50 msgr2=0x7f4e4c15c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c159a50 0x7f4e4c15c200 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4e400015a0 tx=0x7f4e40001b50 comp rx=0 tx=0).stop 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 shutdown_connections 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4e2c066f30 0x7f4e2c0693e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4e4c159e20 0x7f4e4c16d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e4c06c360 0x7f4e4c0ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 --2- 172.21.15.7:0/2013193852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4c159a50 0x7f4e4c15c200 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 >> 172.21.15.7:0/2013193852 conn(0x7f4e4c0cd910 msgr2=0x7f4e4c0b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.356+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 shutdown_connections 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.357+0000 7f4e50e64700 1 -- 172.21.15.7:0/2013193852 wait complete. 2024-01-11T07:10:50.357 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-01-11T07:10:50.369 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:10:50.369 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:10:50.370 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:10:50.370 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:10:50.370 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:10:50.370 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:10:50.370 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:10:50.370 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:10:50.371 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:10:50.371 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:10:50.371 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:10:50.371 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:10:50.371 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:10:50.371 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:10:50.371 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:10:50.371 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:10:50.380 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:10:50.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:10:50.381 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:10:50.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:10:50.382 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:10:50.382 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:10:50.383 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:10:50.383 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:10:50.423 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:10:50.423 DEBUG:teuthology.parallel:result is None 2024-01-11T07:10:50.424 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:10:50.424 DEBUG:teuthology.parallel:result is None 2024-01-11T07:10:50.425 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:10:50.425 DEBUG:teuthology.parallel:result is None 2024-01-11T07:10:50.426 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:10:50.426 DEBUG:teuthology.parallel:result is None 2024-01-11T07:10:50.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:10:50.510 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:10:50.510 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:10:50.511 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:10:50.512 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:10:50.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.566+0000 7f0a1d06c700 1 Processor -- start 2024-01-11T07:10:50.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.566+0000 7f0a1d06c700 1 -- start start 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1d06c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a1806afc0 0x7f0a1815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1d06c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1815b770 con 0x7f0a1806e0d0 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1d06c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1815abf0 con 0x7f0a1806dd00 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1d06c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1815aed0 con 0x7f0a1806afc0 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0a1806e0d0 0x7f0a18159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40328/0 (socket says 172.21.15.7:40328) 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1659c700 1 -- 172.21.15.7:0/1592456043 learned_addr learned my addr 172.21.15.7:0/1592456043 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1759e700 1 --2- 172.21.15.7:0/1592456043 >> v2:172.21.15.113:3300/0 conn(0x7f0a1806afc0 0x7f0a1815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a15d9b700 1 -- 172.21.15.7:0/1592456043 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750674735 0 0) 0x7f0a1815b770 con 0x7f0a1806e0d0 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a15d9b700 1 -- 172.21.15.7:0/1592456043 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a00003570 con 0x7f0a1806e0d0 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a15d9b700 1 -- 172.21.15.7:0/1592456043 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873351051 0 0) 0x7f0a1815abf0 con 0x7f0a1806dd00 2024-01-11T07:10:50.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a15d9b700 1 -- 172.21.15.7:0/1592456043 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a1815b770 con 0x7f0a1806dd00 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1759e700 1 -- 172.21.15.7:0/1592456043 >> v1:172.21.15.113:6789/0 conn(0x7f0a1806dd00 legacy=0x7f0a180c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1759e700 1 -- 172.21.15.7:0/1592456043 >> v1:172.21.15.7:6789/0 conn(0x7f0a1806e0d0 legacy=0x7f0a18159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1759e700 1 -- 172.21.15.7:0/1592456043 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1815b1b0 con 0x7f0a1806afc0 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.567+0000 7f0a1759e700 1 --2- 172.21.15.7:0/1592456043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806afc0 0x7f0a1815a1c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009e60 tx=0x7f0a10009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44ea818fec4bc14 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a15d9b700 1 -- 172.21.15.7:0/1592456043 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000e070 con 0x7f0a1806afc0 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a15d9b700 1 -- 172.21.15.7:0/1592456043 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1000bab0 con 0x7f0a1806afc0 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a15d9b700 1 -- 172.21.15.7:0/1592456043 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000be70 con 0x7f0a1806afc0 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a1d06c700 1 -- 172.21.15.7:0/1592456043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806afc0 msgr2=0x7f0a1815a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a1d06c700 1 --2- 172.21.15.7:0/1592456043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806afc0 0x7f0a1815a1c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009e60 tx=0x7f0a10009780 comp rx=0 tx=0).stop 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a1d06c700 1 -- 172.21.15.7:0/1592456043 shutdown_connections 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a1d06c700 1 --2- 172.21.15.7:0/1592456043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806afc0 0x7f0a1815a1c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a1d06c700 1 -- 172.21.15.7:0/1592456043 >> 172.21.15.7:0/1592456043 conn(0x7f0a180680e0 msgr2=0x7f0a180cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a1d06c700 1 -- 172.21.15.7:0/1592456043 shutdown_connections 2024-01-11T07:10:50.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.568+0000 7f0a1d06c700 1 -- 172.21.15.7:0/1592456043 wait complete. 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.570+0000 7f0a1d06c700 1 Processor -- start 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.570+0000 7f0a1d06c700 1 -- start start 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.570+0000 7f0a1d06c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a1806afc0 0x7f0a180c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.570+0000 7f0a1d06c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806dd00 0x7f0a1815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.570+0000 7f0a1d06c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1806e0d0 0x7f0a1815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.570+0000 7f0a1d06c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180c87b0 con 0x7f0a1806e0d0 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.570+0000 7f0a1d06c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180c4e20 con 0x7f0a1806dd00 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.570+0000 7f0a1d06c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180c5050 con 0x7f0a1806afc0 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1806e0d0 0x7f0a1815dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1806e0d0 0x7f0a1815dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41528/0 (socket says 172.21.15.7:41528) 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 -- 172.21.15.7:0/302331647 learned_addr learned my addr 172.21.15.7:0/302331647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1659c700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806dd00 0x7f0a1815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a16d9d700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a1806afc0 0x7f0a180c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 -- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a1806afc0 msgr2=0x7f0a180c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a1806afc0 0x7f0a180c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 -- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806dd00 msgr2=0x7f0a1815da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806dd00 0x7f0a1815da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 -- 172.21.15.7:0/302331647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a10009a20 con 0x7f0a1806e0d0 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1659c700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806dd00 0x7f0a1815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1759e700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1806e0d0 0x7f0a1815dfb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0a1806d820 tx=0x7f0a10019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f09feffd700 1 -- 172.21.15.7:0/302331647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10012e60 con 0x7f0a1806e0d0 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f09feffd700 1 -- 172.21.15.7:0/302331647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10026480 con 0x7f0a1806e0d0 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f09feffd700 1 -- 172.21.15.7:0/302331647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10026820 con 0x7f0a1806e0d0 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a180c5330 con 0x7f0a1806e0d0 2024-01-11T07:10:50.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.571+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a180c58f0 con 0x7f0a1806e0d0 2024-01-11T07:10:50.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.573+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1804c3c0 con 0x7f0a1806e0d0 2024-01-11T07:10:50.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.573+0000 7f09feffd700 1 -- 172.21.15.7:0/302331647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0a1001c400 con 0x7f0a1806e0d0 2024-01-11T07:10:50.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.574+0000 7f09feffd700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0a04066f30 0x7f0a040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:50.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.574+0000 7f09feffd700 1 -- 172.21.15.7:0/302331647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0a100861b0 con 0x7f0a1806e0d0 2024-01-11T07:10:50.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.574+0000 7f0a16d9d700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0a04066f30 0x7f0a040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:50.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.574+0000 7f0a16d9d700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0a04066f30 0x7f0a040693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a08007fd0 tx=0x7f0a08007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.578+0000 7f09feffd700 1 -- 172.21.15.7:0/302331647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a10053720 con 0x7f0a1806e0d0 2024-01-11T07:10:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:50.798+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0a1806eb20 con 0x7f0a1806e0d0 2024-01-11T07:10:51.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.859+0000 7f09feffd700 1 -- 172.21.15.7:0/302331647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0a10056d40 con 0x7f0a1806e0d0 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0a04066f30 msgr2=0x7f0a040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0a04066f30 0x7f0a040693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a08007fd0 tx=0x7f0a08007d50 comp rx=0 tx=0).stop 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1806e0d0 msgr2=0x7f0a1815dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1806e0d0 0x7f0a1815dfb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0a1806d820 tx=0x7f0a10019000 comp rx=0 tx=0).stop 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 shutdown_connections 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0a04066f30 0x7f0a040693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a1806e0d0 0x7f0a1815dfb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0a1806afc0 0x7f0a180c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 --2- 172.21.15.7:0/302331647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a1806dd00 0x7f0a1815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 >> 172.21.15.7:0/302331647 conn(0x7f0a180680e0 msgr2=0x7f0a180cc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 shutdown_connections 2024-01-11T07:10:51.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:51.862+0000 7f0a1d06c700 1 -- 172.21.15.7:0/302331647 wait complete. 2024-01-11T07:10:51.863 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:10:51.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:10:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.013+0000 7f2b007e2700 1 Processor -- start 2024-01-11T07:10:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.013+0000 7f2b007e2700 1 -- start start 2024-01-11T07:10:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.014+0000 7f2b007e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2af8152030 0x7f2af806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2b007e2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af8159130 con 0x7f2af8152400 2024-01-11T07:10:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2b007e2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af8158890 con 0x7f2af8158150 2024-01-11T07:10:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2b007e2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af8070a50 con 0x7f2af8152030 2024-01-11T07:10:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afe57e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2af8152030 0x7f2af806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afdd7d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2af8152400 0x7f2af8069610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40344/0 (socket says 172.21.15.7:40344) 2024-01-11T07:10:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afdd7d700 1 -- 172.21.15.7:0/1194821773 learned_addr learned my addr 172.21.15.7:0/1194821773 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afe57e700 1 -- 172.21.15.7:0/1194821773 >> v1:172.21.15.113:6789/0 conn(0x7f2af8158150 legacy=0x7f2af815bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afe57e700 1 -- 172.21.15.7:0/1194821773 >> v1:172.21.15.7:6789/0 conn(0x7f2af8152400 legacy=0x7f2af8069610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afe57e700 1 -- 172.21.15.7:0/1194821773 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af8158b70 con 0x7f2af8152030 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afd57c700 1 -- 172.21.15.7:0/1194821773 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504325417 0 0) 0x7f2ae8001110 con 0x7f2af8152400 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afe57e700 1 --2- 172.21.15.7:0/1194821773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8152030 0x7f2af806d400 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2af40098e0 tx=0x7f2af40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5263164e87f557 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afd57c700 1 -- 172.21.15.7:0/1194821773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4009e80 con 0x7f2af8152030 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.015+0000 7f2afd57c700 1 -- 172.21.15.7:0/1194821773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af400b860 con 0x7f2af8152030 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.016+0000 7f2afd57c700 1 -- 172.21.15.7:0/1194821773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400bbe0 con 0x7f2af8152030 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.016+0000 7f2b007e2700 1 -- 172.21.15.7:0/1194821773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8152030 msgr2=0x7f2af806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.016+0000 7f2b007e2700 1 --2- 172.21.15.7:0/1194821773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8152030 0x7f2af806d400 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2af40098e0 tx=0x7f2af40091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.016+0000 7f2b007e2700 1 -- 172.21.15.7:0/1194821773 shutdown_connections 2024-01-11T07:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.016+0000 7f2b007e2700 1 --2- 172.21.15.7:0/1194821773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8152030 0x7f2af806d400 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.016+0000 7f2b007e2700 1 -- 172.21.15.7:0/1194821773 >> 172.21.15.7:0/1194821773 conn(0x7f2af80cd090 msgr2=0x7f2af80cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.016+0000 7f2b007e2700 1 -- 172.21.15.7:0/1194821773 shutdown_connections 2024-01-11T07:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.016+0000 7f2b007e2700 1 -- 172.21.15.7:0/1194821773 wait complete. 2024-01-11T07:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.017+0000 7f2b007e2700 1 Processor -- start 2024-01-11T07:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.017+0000 7f2b007e2700 1 -- start start 2024-01-11T07:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2b007e2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2af8152030 0x7f2af814c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2b007e2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2af8152400 0x7f2af816abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2b007e2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8158150 0x7f2af816b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2b007e2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af815cae0 con 0x7f2af8158150 2024-01-11T07:10:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2b007e2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af816a020 con 0x7f2af8152400 2024-01-11T07:10:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2b007e2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80cb140 con 0x7f2af8152030 2024-01-11T07:10:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2afed7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8158150 0x7f2af816b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2afed7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8158150 0x7f2af816b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41552/0 (socket says 172.21.15.7:41552) 2024-01-11T07:10:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.018+0000 7f2afed7f700 1 -- 172.21.15.7:0/519895506 learned_addr learned my addr 172.21.15.7:0/519895506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afe57e700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2af8152030 0x7f2af814c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afdd7d700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2af8152400 0x7f2af816abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afed7f700 1 -- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2af8152030 msgr2=0x7f2af814c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afed7f700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2af8152030 0x7f2af814c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afed7f700 1 -- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2af8152400 msgr2=0x7f2af816abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afed7f700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2af8152400 0x7f2af816abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afed7f700 1 -- 172.21.15.7:0/519895506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af4009420 con 0x7f2af8158150 2024-01-11T07:10:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afdd7d700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2af8152400 0x7f2af816abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2afed7f700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8158150 0x7f2af816b390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c8e0 tx=0x7f2af0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2aee7fc700 1 -- 172.21.15.7:0/519895506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000ce80 con 0x7f2af8158150 2024-01-11T07:10:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2aee7fc700 1 -- 172.21.15.7:0/519895506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000a7e0 con 0x7f2af8158150 2024-01-11T07:10:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2aee7fc700 1 -- 172.21.15.7:0/519895506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000ab80 con 0x7f2af8158150 2024-01-11T07:10:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af80cb420 con 0x7f2af8158150 2024-01-11T07:10:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.019+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af80cb9e0 con 0x7f2af8158150 2024-01-11T07:10:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.020+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af804c3c0 con 0x7f2af8158150 2024-01-11T07:10:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.021+0000 7f2aee7fc700 1 -- 172.21.15.7:0/519895506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2af0012420 con 0x7f2af8158150 2024-01-11T07:10:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.021+0000 7f2aee7fc700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.021+0000 7f2aee7fc700 1 -- 172.21.15.7:0/519895506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2af00844d0 con 0x7f2af8158150 2024-01-11T07:10:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.021+0000 7f2afe57e700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.022+0000 7f2afe57e700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2adc066f30 0x7f2adc0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2af40093f0 tx=0x7f2af4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.025+0000 7f2aee7fc700 1 -- 172.21.15.7:0/519895506 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2af0051a90 con 0x7f2af8158150 2024-01-11T07:10:52.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.310+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2af8158e50 con 0x7f2af8158150 2024-01-11T07:10:52.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.311+0000 7f2aee7fc700 1 -- 172.21.15.7:0/519895506 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2af00550b0 con 0x7f2af8158150 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:10:52.311 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:10:52.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.314 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:10:52.315 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:10:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.318 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:10:52.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.320 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:10:52.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:10:52.323 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:10:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:52.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:10:52.324 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2adc066f30 msgr2=0x7f2adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2adc066f30 0x7f2adc0693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2af40093f0 tx=0x7f2af4017000 comp rx=0 tx=0).stop 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8158150 msgr2=0x7f2af816b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8158150 0x7f2af816b390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c8e0 tx=0x7f2af0001710 comp rx=0 tx=0).stop 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 shutdown_connections 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af8158150 0x7f2af816b390 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2af8152030 0x7f2af814c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 --2- 172.21.15.7:0/519895506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2af8152400 0x7f2af816abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.313+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 >> 172.21.15.7:0/519895506 conn(0x7f2af80cd090 msgr2=0x7f2af80c35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.314+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 shutdown_connections 2024-01-11T07:10:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.314+0000 7f2b007e2700 1 -- 172.21.15.7:0/519895506 wait complete. 2024-01-11T07:10:52.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:10:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.459+0000 7fb5549fc700 1 Processor -- start 2024-01-11T07:10:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.459+0000 7fb5549fc700 1 -- start start 2024-01-11T07:10:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.460+0000 7fb5549fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb55006dd00 0x7fb5500c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb5549fc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55015b770 con 0x7fb55006e0d0 2024-01-11T07:10:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb5549fc700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55015a620 con 0x7fb55015a1c0 2024-01-11T07:10:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb5549fc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55015abf0 con 0x7fb55006dd00 2024-01-11T07:10:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb55006dd00 0x7fb5500c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb55006e0d0 0x7fb550159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40354/0 (socket says 172.21.15.7:40354) 2024-01-11T07:10:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54dd9b700 1 -- 172.21.15.7:0/3706670407 learned_addr learned my addr 172.21.15.7:0/3706670407 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54e59c700 1 -- 172.21.15.7:0/3706670407 >> v1:172.21.15.113:6789/0 conn(0x7fb55015a1c0 legacy=0x7fb550069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54e59c700 1 -- 172.21.15.7:0/3706670407 >> v1:172.21.15.7:6789/0 conn(0x7fb55006e0d0 legacy=0x7fb550159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54e59c700 1 -- 172.21.15.7:0/3706670407 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55015aed0 con 0x7fb55006dd00 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54d59a700 1 -- 172.21.15.7:0/3706670407 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691663553 0 0) 0x7fb55015b770 con 0x7fb55006e0d0 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54e59c700 1 --2- 172.21.15.7:0/3706670407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55006dd00 0x7fb5500c98d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb5400098e0 tx=0x7fb5400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d75401ea3b5dcd6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54d59a700 1 -- 172.21.15.7:0/3706670407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540009e80 con 0x7fb55006dd00 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54d59a700 1 -- 172.21.15.7:0/3706670407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54000b820 con 0x7fb55006dd00 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.461+0000 7fb54d59a700 1 -- 172.21.15.7:0/3706670407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000bba0 con 0x7fb55006dd00 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.462+0000 7fb5549fc700 1 -- 172.21.15.7:0/3706670407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55006dd00 msgr2=0x7fb5500c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.462+0000 7fb5549fc700 1 --2- 172.21.15.7:0/3706670407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55006dd00 0x7fb5500c98d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb5400098e0 tx=0x7fb5400091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.462+0000 7fb5549fc700 1 -- 172.21.15.7:0/3706670407 shutdown_connections 2024-01-11T07:10:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.462+0000 7fb5549fc700 1 --2- 172.21.15.7:0/3706670407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55006dd00 0x7fb5500c98d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.462+0000 7fb5549fc700 1 -- 172.21.15.7:0/3706670407 >> 172.21.15.7:0/3706670407 conn(0x7fb5500680e0 msgr2=0x7fb5500cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.462+0000 7fb5549fc700 1 -- 172.21.15.7:0/3706670407 shutdown_connections 2024-01-11T07:10:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.462+0000 7fb5549fc700 1 -- 172.21.15.7:0/3706670407 wait complete. 2024-01-11T07:10:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.463+0000 7fb5549fc700 1 Processor -- start 2024-01-11T07:10:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.463+0000 7fb5549fc700 1 -- start start 2024-01-11T07:10:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb5549fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb55006dd00 0x7fb55014ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb5549fc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb55006e0d0 0x7fb5500cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb5549fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55015a1c0 0x7fb5500cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb5549fc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55016c4a0 con 0x7fb55015a1c0 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb5549fc700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55016b890 con 0x7fb55006dd00 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb5549fc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55016bb70 con 0x7fb55006e0d0 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb54ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55015a1c0 0x7fb5500cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb54ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55015a1c0 0x7fb5500cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41572/0 (socket says 172.21.15.7:41572) 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.464+0000 7fb54ed9d700 1 -- 172.21.15.7:0/133884428 learned_addr learned my addr 172.21.15.7:0/133884428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54dd9b700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb55006e0d0 0x7fb5500cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54e59c700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb55006dd00 0x7fb55014ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54ed9d700 1 -- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb55006e0d0 msgr2=0x7fb5500cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54ed9d700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb55006e0d0 0x7fb5500cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54ed9d700 1 -- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb55006dd00 msgr2=0x7fb55014ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54ed9d700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb55006dd00 0x7fb55014ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54ed9d700 1 -- 172.21.15.7:0/133884428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540009420 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54dd9b700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb55006e0d0 0x7fb5500cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54e59c700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb55006dd00 0x7fb55014ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb54ed9d700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55015a1c0 0x7fb5500cc8a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb54800bc30 tx=0x7fb548009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb53e7fc700 1 -- 172.21.15.7:0/133884428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54800c9b0 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb53e7fc700 1 -- 172.21.15.7:0/133884428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54800cb30 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55016be50 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb53e7fc700 1 -- 172.21.15.7:0/133884428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548014410 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.465+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55014b160 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.466+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb550064850 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.467+0000 7fb53e7fc700 1 -- 172.21.15.7:0/133884428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb548014840 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.467+0000 7fb53e7fc700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb530066f30 0x7fb5300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.467+0000 7fb53e7fc700 1 -- 172.21.15.7:0/133884428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb548084200 con 0x7fb55015a1c0 2024-01-11T07:10:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.467+0000 7fb54e59c700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb530066f30 0x7fb5300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.468+0000 7fb54e59c700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb530066f30 0x7fb5300693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb5400030c0 tx=0x7fb540016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.472+0000 7fb53e7fc700 1 -- 172.21.15.7:0/133884428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb54804c5a0 con 0x7fb55015a1c0 2024-01-11T07:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.676+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb55004c3c0 con 0x7fb55015a1c0 2024-01-11T07:10:52.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.677+0000 7fb53e7fc700 1 -- 172.21.15.7:0/133884428 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+12086 (secure 0 0 0) 0x7fb5480517c0 con 0x7fb55015a1c0 2024-01-11T07:10:52.676 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:52.677 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:10:51.856238+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:10:52.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb530066f30 msgr2=0x7fb5300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb530066f30 0x7fb5300693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb5400030c0 tx=0x7fb540016000 comp rx=0 tx=0).stop 2024-01-11T07:10:52.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55015a1c0 msgr2=0x7fb5500cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55015a1c0 0x7fb5500cc8a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb54800bc30 tx=0x7fb548009820 comp rx=0 tx=0).stop 2024-01-11T07:10:52.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 shutdown_connections 2024-01-11T07:10:52.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb530066f30 0x7fb5300693e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb55015a1c0 0x7fb5500cc8a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb55006dd00 0x7fb55014ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 --2- 172.21.15.7:0/133884428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb55006e0d0 0x7fb5500cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 >> 172.21.15.7:0/133884428 conn(0x7fb5500680e0 msgr2=0x7fb5500b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:52.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 shutdown_connections 2024-01-11T07:10:52.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.679+0000 7fb5549fc700 1 -- 172.21.15.7:0/133884428 wait complete. 2024-01-11T07:10:52.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:10:52.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:10:52.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.828+0000 7fdd4c323700 1 Processor -- start 2024-01-11T07:10:52.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.828+0000 7fdd4c323700 1 -- start start 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.829+0000 7fdd4c323700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd44063940 0x7fdd440cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.829+0000 7fdd4c323700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd440cd6c0 con 0x7fdd44159dc0 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.829+0000 7fdd4c323700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd440cb4c0 con 0x7fdd4415d900 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.829+0000 7fdd4c323700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4415a190 con 0x7fdd44063940 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.829+0000 7fdd4a0bf700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdd44159dc0 0x7fdd440c80c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40368/0 (socket says 172.21.15.7:40368) 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.829+0000 7fdd4a0bf700 1 -- 172.21.15.7:0/1856338575 learned_addr learned my addr 172.21.15.7:0/1856338575 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd4a8c0700 1 --2- 172.21.15.7:0/1856338575 >> v2:172.21.15.113:3300/0 conn(0x7fdd44063940 0x7fdd440cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd490bd700 1 -- 172.21.15.7:0/1856338575 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133767897 0 0) 0x7fdd40001110 con 0x7fdd44159dc0 2024-01-11T07:10:52.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd490bd700 1 -- 172.21.15.7:0/1856338575 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd30003570 con 0x7fdd44159dc0 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd490bd700 1 -- 172.21.15.7:0/1856338575 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645377122 0 0) 0x7fdd440cb4c0 con 0x7fdd4415d900 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd490bd700 1 -- 172.21.15.7:0/1856338575 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd40001110 con 0x7fdd4415d900 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd4a8c0700 1 -- 172.21.15.7:0/1856338575 >> v1:172.21.15.113:6789/0 conn(0x7fdd4415d900 legacy=0x7fdd44063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd4a8c0700 1 -- 172.21.15.7:0/1856338575 >> v1:172.21.15.7:6789/0 conn(0x7fdd44159dc0 legacy=0x7fdd440c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd4a8c0700 1 -- 172.21.15.7:0/1856338575 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd440c87d0 con 0x7fdd44063940 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd4a8c0700 1 --2- 172.21.15.7:0/1856338575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 0x7fdd440cb950 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009e60 tx=0x7fdd3c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc0c7fc401064a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd490bd700 1 -- 172.21.15.7:0/1856338575 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00e070 con 0x7fdd44063940 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd490bd700 1 -- 172.21.15.7:0/1856338575 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c00bab0 con 0x7fdd44063940 2024-01-11T07:10:52.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.830+0000 7fdd490bd700 1 -- 172.21.15.7:0/1856338575 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00be70 con 0x7fdd44063940 2024-01-11T07:10:52.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.831+0000 7fdd4c323700 1 -- 172.21.15.7:0/1856338575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 msgr2=0x7fdd440cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.831+0000 7fdd4c323700 1 --2- 172.21.15.7:0/1856338575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 0x7fdd440cb950 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009e60 tx=0x7fdd3c009780 comp rx=0 tx=0).stop 2024-01-11T07:10:52.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.831+0000 7fdd4c323700 1 -- 172.21.15.7:0/1856338575 shutdown_connections 2024-01-11T07:10:52.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.831+0000 7fdd4c323700 1 --2- 172.21.15.7:0/1856338575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 0x7fdd440cb950 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.831+0000 7fdd4c323700 1 -- 172.21.15.7:0/1856338575 >> 172.21.15.7:0/1856338575 conn(0x7fdd440cd860 msgr2=0x7fdd440cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:52.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.831+0000 7fdd4c323700 1 -- 172.21.15.7:0/1856338575 shutdown_connections 2024-01-11T07:10:52.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.831+0000 7fdd4c323700 1 -- 172.21.15.7:0/1856338575 wait complete. 2024-01-11T07:10:52.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.833+0000 7fdd4c323700 1 Processor -- start 2024-01-11T07:10:52.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.833+0000 7fdd4c323700 1 -- start start 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4c323700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 0x7fdd44141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4c323700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd44159dc0 0x7fdd44154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4c323700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd4415d900 0x7fdd440b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4c323700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd441564d0 con 0x7fdd4415d900 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4c323700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4416d6a0 con 0x7fdd44063940 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4c323700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44155850 con 0x7fdd44159dc0 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd4415d900 0x7fdd440b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd4415d900 0x7fdd440b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41586/0 (socket says 172.21.15.7:41586) 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 -- 172.21.15.7:0/3194316098 learned_addr learned my addr 172.21.15.7:0/3194316098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a0bf700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 0x7fdd44141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd498be700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd44159dc0 0x7fdd44154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 -- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd44159dc0 msgr2=0x7fdd44154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd44159dc0 0x7fdd44154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 -- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 msgr2=0x7fdd44141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 0x7fdd44141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 -- 172.21.15.7:0/3194316098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009a20 con 0x7fdd4415d900 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.834+0000 7fdd4a8c0700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd4415d900 0x7fdd440b0600 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c013000 tx=0x7fdd3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.835+0000 7fdd3a7fc700 1 -- 172.21.15.7:0/3194316098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c026480 con 0x7fdd4415d900 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.835+0000 7fdd3a7fc700 1 -- 172.21.15.7:0/3194316098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c026600 con 0x7fdd4415d900 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.835+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd44155b30 con 0x7fdd4415d900 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.835+0000 7fdd3a7fc700 1 -- 172.21.15.7:0/3194316098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0269c0 con 0x7fdd4415d900 2024-01-11T07:10:52.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.835+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd441423b0 con 0x7fdd4415d900 2024-01-11T07:10:52.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.836+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44155d20 con 0x7fdd4415d900 2024-01-11T07:10:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.836+0000 7fdd3a7fc700 1 -- 172.21.15.7:0/3194316098 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdd3c01c400 con 0x7fdd4415d900 2024-01-11T07:10:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.837+0000 7fdd3a7fc700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd28066f30 0x7fdd280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.837+0000 7fdd3a7fc700 1 -- 172.21.15.7:0/3194316098 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdd3c086600 con 0x7fdd4415d900 2024-01-11T07:10:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.837+0000 7fdd4a0bf700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd28066f30 0x7fdd280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.837+0000 7fdd4a0bf700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd28066f30 0x7fdd280693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdd440c7e00 tx=0x7fdd40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:52.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:52.841+0000 7fdd3a7fc700 1 -- 172.21.15.7:0/3194316098 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd3c053bc0 con 0x7fdd4415d900 2024-01-11T07:10:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.043+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd44155f20 con 0x7fdd4415d900 2024-01-11T07:10:53.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.043+0000 7fdd3a7fc700 1 -- 172.21.15.7:0/3194316098 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd3c02b0a0 con 0x7fdd4415d900 2024-01-11T07:10:53.043 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:10:53.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.045+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd28066f30 msgr2=0x7fdd280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.045+0000 7fdd4c323700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd28066f30 0x7fdd280693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdd440c7e00 tx=0x7fdd40009040 comp rx=0 tx=0).stop 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.045+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd4415d900 msgr2=0x7fdd440b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.045+0000 7fdd4c323700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd4415d900 0x7fdd440b0600 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c013000 tx=0x7fdd3c019000 comp rx=0 tx=0).stop 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.046+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 shutdown_connections 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.046+0000 7fdd4c323700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd28066f30 0x7fdd280693e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.046+0000 7fdd4c323700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd4415d900 0x7fdd440b0600 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.046+0000 7fdd4c323700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd44063940 0x7fdd44141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.046+0000 7fdd4c323700 1 --2- 172.21.15.7:0/3194316098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd44159dc0 0x7fdd44154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.046+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 >> 172.21.15.7:0/3194316098 conn(0x7fdd440cd860 msgr2=0x7fdd44154560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.046+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 shutdown_connections 2024-01-11T07:10:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.046+0000 7fdd4c323700 1 -- 172.21.15.7:0/3194316098 wait complete. 2024-01-11T07:10:53.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:10:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.192+0000 7f787d6c5700 1 Processor -- start 2024-01-11T07:10:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.192+0000 7f787d6c5700 1 -- start start 2024-01-11T07:10:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.193+0000 7f787d6c5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7878159d10 0x7f7878068c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.193+0000 7f787d6c5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787815dbb0 con 0x7f787806f3a0 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.193+0000 7f787d6c5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787815de90 con 0x7f78780691d0 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.193+0000 7f787d6c5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787815d9a0 con 0x7f7878159d10 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.193+0000 7f78777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f787806f3a0 0x7f78780b3b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40380/0 (socket says 172.21.15.7:40380) 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.193+0000 7f78777fe700 1 -- 172.21.15.7:0/385653376 learned_addr learned my addr 172.21.15.7:0/385653376 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.193+0000 7f7876ffd700 1 --2- 172.21.15.7:0/385653376 >> v2:172.21.15.113:3300/0 conn(0x7f7878159d10 0x7f7878068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7875ffb700 1 -- 172.21.15.7:0/385653376 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207310628 0 0) 0x7f787815dbb0 con 0x7f787806f3a0 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7875ffb700 1 -- 172.21.15.7:0/385653376 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7860003570 con 0x7f787806f3a0 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7875ffb700 1 -- 172.21.15.7:0/385653376 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427749343 0 0) 0x7f787815de90 con 0x7f78780691d0 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7875ffb700 1 -- 172.21.15.7:0/385653376 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f787815dbb0 con 0x7f78780691d0 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7876ffd700 1 -- 172.21.15.7:0/385653376 >> v1:172.21.15.113:6789/0 conn(0x7f78780691d0 legacy=0x7f787806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7876ffd700 1 -- 172.21.15.7:0/385653376 >> v1:172.21.15.7:6789/0 conn(0x7f787806f3a0 legacy=0x7f78780b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7876ffd700 1 -- 172.21.15.7:0/385653376 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787815e170 con 0x7f7878159d10 2024-01-11T07:10:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7876ffd700 1 --2- 172.21.15.7:0/385653376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7878159d10 0x7f7878068c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7868009910 tx=0x7f78680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1627110c6504ab7d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7875ffb700 1 -- 172.21.15.7:0/385653376 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868011630 con 0x7f7878159d10 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7875ffb700 1 -- 172.21.15.7:0/385653376 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7868011790 con 0x7f7878159d10 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.194+0000 7f7875ffb700 1 -- 172.21.15.7:0/385653376 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868011b30 con 0x7f7878159d10 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.195+0000 7f787d6c5700 1 -- 172.21.15.7:0/385653376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7878159d10 msgr2=0x7f7878068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.195+0000 7f787d6c5700 1 --2- 172.21.15.7:0/385653376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7878159d10 0x7f7878068c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7868009910 tx=0x7f78680091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.195+0000 7f787d6c5700 1 -- 172.21.15.7:0/385653376 shutdown_connections 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.195+0000 7f787d6c5700 1 --2- 172.21.15.7:0/385653376 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7878159d10 0x7f7878068c90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.195+0000 7f787d6c5700 1 -- 172.21.15.7:0/385653376 >> 172.21.15.7:0/385653376 conn(0x7f78780ce9f0 msgr2=0x7f78780cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.195+0000 7f787d6c5700 1 -- 172.21.15.7:0/385653376 shutdown_connections 2024-01-11T07:10:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.195+0000 7f787d6c5700 1 -- 172.21.15.7:0/385653376 wait complete. 2024-01-11T07:10:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.196+0000 7f787d6c5700 1 Processor -- start 2024-01-11T07:10:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.196+0000 7f787d6c5700 1 -- start start 2024-01-11T07:10:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f787d6c5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78780691d0 0x7f78780c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f787d6c5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f787806f3a0 0x7f7878158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f787d6c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878159d10 0x7f787815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f787d6c5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780c8640 con 0x7f7878159d10 2024-01-11T07:10:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f787d6c5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780c7a30 con 0x7f787806f3a0 2024-01-11T07:10:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f787d6c5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780c7d10 con 0x7f78780691d0 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f78777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878159d10 0x7f787815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f78777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878159d10 0x7f787815c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41590/0 (socket says 172.21.15.7:41590) 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f78777fe700 1 -- 172.21.15.7:0/3543467605 learned_addr learned my addr 172.21.15.7:0/3543467605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f7876ffd700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78780691d0 0x7f78780c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.197+0000 7f78767fc700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f787806f3a0 0x7f7878158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f78777fe700 1 -- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78780691d0 msgr2=0x7f78780c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f78777fe700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78780691d0 0x7f78780c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f78777fe700 1 -- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f787806f3a0 msgr2=0x7f7878158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f78777fe700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f787806f3a0 0x7f7878158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f78777fe700 1 -- 172.21.15.7:0/3543467605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7868009420 con 0x7f7878159d10 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f78777fe700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878159d10 0x7f787815c230 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f787000cc00 tx=0x7f7870001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f785effd700 1 -- 172.21.15.7:0/3543467605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000a8b0 con 0x7f7878159d10 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f785effd700 1 -- 172.21.15.7:0/3543467605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787000aa30 con 0x7f7878159d10 2024-01-11T07:10:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f785effd700 1 -- 172.21.15.7:0/3543467605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000adf0 con 0x7f7878159d10 2024-01-11T07:10:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780c7ff0 con 0x7f7878159d10 2024-01-11T07:10:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.198+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78780c7740 con 0x7f7878159d10 2024-01-11T07:10:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.199+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78780c87f0 con 0x7f7878159d10 2024-01-11T07:10:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.200+0000 7f785effd700 1 -- 172.21.15.7:0/3543467605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7870007300 con 0x7f7878159d10 2024-01-11T07:10:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.200+0000 7f785effd700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7864066f30 0x7f78640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.200+0000 7f785effd700 1 -- 172.21.15.7:0/3543467605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f78700844b0 con 0x7f7878159d10 2024-01-11T07:10:53.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.200+0000 7f7876ffd700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7864066f30 0x7f78640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.201+0000 7f7876ffd700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7864066f30 0x7f78640693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f78680093f0 tx=0x7f7868017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:53.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.204+0000 7f785effd700 1 -- 172.21.15.7:0/3543467605 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78780c87f0 con 0x7f7878159d10 2024-01-11T07:10:53.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.414+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f787804c3c0 con 0x7f7878159d10 2024-01-11T07:10:53.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.414+0000 7f785effd700 1 -- 172.21.15.7:0/3543467605 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7870051ba0 con 0x7f7878159d10 2024-01-11T07:10:53.414 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:53.414 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:10:53.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:10:53.414 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:10:53.414 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:10:53.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:10:53.414 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:10:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7864066f30 msgr2=0x7f78640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7864066f30 0x7f78640693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f78680093f0 tx=0x7f7868017000 comp rx=0 tx=0).stop 2024-01-11T07:10:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878159d10 msgr2=0x7f787815c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878159d10 0x7f787815c230 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f787000cc00 tx=0x7f7870001a80 comp rx=0 tx=0).stop 2024-01-11T07:10:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 shutdown_connections 2024-01-11T07:10:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7864066f30 0x7f78640693e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878159d10 0x7f787815c230 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78780691d0 0x7f78780c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 --2- 172.21.15.7:0/3543467605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f787806f3a0 0x7f7878158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 >> 172.21.15.7:0/3543467605 conn(0x7f78780ce9f0 msgr2=0x7f78780b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 shutdown_connections 2024-01-11T07:10:53.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.416+0000 7f787d6c5700 1 -- 172.21.15.7:0/3543467605 wait complete. 2024-01-11T07:10:53.429 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:10:53.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:10:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.566+0000 7ff6ddffd700 1 Processor -- start 2024-01-11T07:10:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.566+0000 7ff6ddffd700 1 -- start start 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.566+0000 7ff6ddffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6d815d100 0x7ff6d8069540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6ddffd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d4d0 0x7ff6d815d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6ddffd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d8069f90 con 0x7ff6d8069a80 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6ddffd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d806a270 con 0x7ff6d815d100 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6ddffd700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d806a550 con 0x7ff6d815d4d0 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6d815d100 0x7ff6d8069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff6d8069a80 0x7ff6d815a340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40384/0 (socket says 172.21.15.7:40384) 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d7fff700 1 -- 172.21.15.7:0/2164130284 learned_addr learned my addr 172.21.15.7:0/2164130284 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d6ffd700 1 --2- 172.21.15.7:0/2164130284 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d4d0 0x7ff6d815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d77fe700 1 -- 172.21.15.7:0/2164130284 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d4d0 msgr2=0x7ff6d815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d77fe700 1 --2- 172.21.15.7:0/2164130284 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d4d0 0x7ff6d815d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d77fe700 1 -- 172.21.15.7:0/2164130284 >> v1:172.21.15.7:6789/0 conn(0x7ff6d8069a80 legacy=0x7ff6d815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d77fe700 1 -- 172.21.15.7:0/2164130284 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d806a830 con 0x7ff6d815d100 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d6ffd700 1 --2- 172.21.15.7:0/2164130284 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d4d0 0x7ff6d815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d77fe700 1 --2- 172.21.15.7:0/2164130284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d100 0x7ff6d8069540 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff6d806d0f0 tx=0x7ff6c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df862b8945136804 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d67fc700 1 -- 172.21.15.7:0/2164130284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800b6b0 con 0x7ff6d815d100 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.567+0000 7ff6d67fc700 1 -- 172.21.15.7:0/2164130284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c800b810 con 0x7ff6d815d100 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6d67fc700 1 -- 172.21.15.7:0/2164130284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800bbb0 con 0x7ff6d815d100 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6ddffd700 1 -- 172.21.15.7:0/2164130284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d100 msgr2=0x7ff6d8069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/2164130284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d100 0x7ff6d8069540 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff6d806d0f0 tx=0x7ff6c80091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6ddffd700 1 -- 172.21.15.7:0/2164130284 shutdown_connections 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/2164130284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d100 0x7ff6d8069540 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/2164130284 >> v2:172.21.15.113:3300/0 conn(0x7ff6d815d4d0 0x7ff6d815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6ddffd700 1 -- 172.21.15.7:0/2164130284 >> 172.21.15.7:0/2164130284 conn(0x7ff6d80680b0 msgr2=0x7ff6d80cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6ddffd700 1 -- 172.21.15.7:0/2164130284 shutdown_connections 2024-01-11T07:10:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.568+0000 7ff6ddffd700 1 -- 172.21.15.7:0/2164130284 wait complete. 2024-01-11T07:10:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.569+0000 7ff6ddffd700 1 Processor -- start 2024-01-11T07:10:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6ddffd700 1 -- start start 2024-01-11T07:10:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6ddffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069a80 0x7ff6d816ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6ddffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d815d100 0x7ff6d80b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6ddffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d4d0 0x7ff6d8139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6ddffd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d815b330 con 0x7ff6d815d4d0 2024-01-11T07:10:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6ddffd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d816b3b0 con 0x7ff6d815d100 2024-01-11T07:10:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6ddffd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d815a720 con 0x7ff6d8069a80 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d4d0 0x7ff6d8139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d4d0 0x7ff6d8139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41606/0 (socket says 172.21.15.7:41606) 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.570+0000 7ff6d7fff700 1 -- 172.21.15.7:0/4156184557 learned_addr learned my addr 172.21.15.7:0/4156184557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6d6ffd700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d815d100 0x7ff6d80b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6d77fe700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069a80 0x7ff6d816ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6d7fff700 1 -- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069a80 msgr2=0x7ff6d816ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6d7fff700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069a80 0x7ff6d816ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6d7fff700 1 -- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d815d100 msgr2=0x7ff6d80b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6d7fff700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d815d100 0x7ff6d80b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6d7fff700 1 -- 172.21.15.7:0/4156184557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c8009420 con 0x7ff6d815d4d0 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6d7fff700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d4d0 0x7ff6d8139bb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000cc40 tx=0x7ff6d0001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6bf7fe700 1 -- 172.21.15.7:0/4156184557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000a8b0 con 0x7ff6d815d4d0 2024-01-11T07:10:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6bf7fe700 1 -- 172.21.15.7:0/4156184557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d000aa10 con 0x7ff6d815d4d0 2024-01-11T07:10:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d815aa00 con 0x7ff6d815d4d0 2024-01-11T07:10:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6bf7fe700 1 -- 172.21.15.7:0/4156184557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000add0 con 0x7ff6d815d4d0 2024-01-11T07:10:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.571+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d815afc0 con 0x7ff6d815d4d0 2024-01-11T07:10:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.572+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d813e370 con 0x7ff6d815d4d0 2024-01-11T07:10:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.573+0000 7ff6bf7fe700 1 -- 172.21.15.7:0/4156184557 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff6d0007300 con 0x7ff6d815d4d0 2024-01-11T07:10:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.573+0000 7ff6bf7fe700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6c4066f30 0x7ff6c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.573+0000 7ff6bf7fe700 1 -- 172.21.15.7:0/4156184557 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff6d0084450 con 0x7ff6d815d4d0 2024-01-11T07:10:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.574+0000 7ff6d77fe700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6c4066f30 0x7ff6c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.574+0000 7ff6d77fe700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6c4066f30 0x7ff6c40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80093f0 tx=0x7ff6c8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.578+0000 7ff6bf7fe700 1 -- 172.21.15.7:0/4156184557 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6d0051a90 con 0x7ff6d815d4d0 2024-01-11T07:10:53.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:53.780+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff6d804c3c0 con 0x7ff6d815d4d0 2024-01-11T07:10:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.330+0000 7ff6bf7fe700 1 -- 172.21.15.7:0/4156184557 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff6d00550b0 con 0x7ff6d815d4d0 2024-01-11T07:10:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6c4066f30 msgr2=0x7ff6c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6c4066f30 0x7ff6c40693e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80093f0 tx=0x7ff6c8017000 comp rx=0 tx=0).stop 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d4d0 msgr2=0x7ff6d8139bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d4d0 0x7ff6d8139bb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000cc40 tx=0x7ff6d0001a50 comp rx=0 tx=0).stop 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 shutdown_connections 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6c4066f30 0x7ff6c40693e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d815d4d0 0x7ff6d8139bb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6d8069a80 0x7ff6d816ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 --2- 172.21.15.7:0/4156184557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6d815d100 0x7ff6d80b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 >> 172.21.15.7:0/4156184557 conn(0x7ff6d80680b0 msgr2=0x7ff6d815c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 shutdown_connections 2024-01-11T07:10:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.335+0000 7ff6ddffd700 1 -- 172.21.15.7:0/4156184557 wait complete. 2024-01-11T07:10:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:10:54.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:10:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.494+0000 7f34bac6e700 1 Processor -- start 2024-01-11T07:10:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.494+0000 7f34bac6e700 1 -- start start 2024-01-11T07:10:54.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34bac6e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b415a280 0x7f34b40aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:54.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34bac6e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34b40ac5b0 0x7f34b40aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34bac6e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b415d880 con 0x7f34b4063230 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34bac6e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b406c5b0 con 0x7f34b40ac5b0 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34bac6e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b415a7f0 con 0x7f34b415a280 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34b920b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34b40ac5b0 0x7f34b40aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34b920b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34b40ac5b0 0x7f34b40aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41614/0 (socket says 172.21.15.7:41614) 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34b920b700 1 -- 172.21.15.7:0/2848826473 learned_addr learned my addr 172.21.15.7:0/2848826473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.495+0000 7f34b3fff700 1 --2- 172.21.15.7:0/2848826473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b415a280 0x7f34b40aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b920b700 1 -- 172.21.15.7:0/2848826473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b415a280 msgr2=0x7f34b40aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b920b700 1 --2- 172.21.15.7:0/2848826473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b415a280 0x7f34b40aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b920b700 1 -- 172.21.15.7:0/2848826473 >> v1:172.21.15.113:6789/0 conn(0x7f34b4063230 legacy=0x7f34b4063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b37fe700 1 -- 172.21.15.7:0/2848826473 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682023701 0 0) 0x7f34b415d880 con 0x7f34b4063230 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b920b700 1 -- 172.21.15.7:0/2848826473 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b415aad0 con 0x7f34b40ac5b0 2024-01-11T07:10:54.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b920b700 1 --2- 172.21.15.7:0/2848826473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b40ac5b0 0x7f34b40aca30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009f20 tx=0x7f34ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d60a11d90626377 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:54.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b37fe700 1 -- 172.21.15.7:0/2848826473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00e070 con 0x7f34b40ac5b0 2024-01-11T07:10:54.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b37fe700 1 -- 172.21.15.7:0/2848826473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac00b9d0 con 0x7f34b40ac5b0 2024-01-11T07:10:54.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.496+0000 7f34b37fe700 1 -- 172.21.15.7:0/2848826473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0092e0 con 0x7f34b40ac5b0 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.497+0000 7f34bac6e700 1 -- 172.21.15.7:0/2848826473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b40ac5b0 msgr2=0x7f34b40aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.497+0000 7f34bac6e700 1 --2- 172.21.15.7:0/2848826473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b40ac5b0 0x7f34b40aca30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009f20 tx=0x7f34ac009840 comp rx=0 tx=0).stop 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.497+0000 7f34bac6e700 1 -- 172.21.15.7:0/2848826473 shutdown_connections 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.497+0000 7f34bac6e700 1 --2- 172.21.15.7:0/2848826473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b40ac5b0 0x7f34b40aca30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.497+0000 7f34bac6e700 1 --2- 172.21.15.7:0/2848826473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b415a280 0x7f34b40aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.497+0000 7f34bac6e700 1 -- 172.21.15.7:0/2848826473 >> 172.21.15.7:0/2848826473 conn(0x7f34b40680e0 msgr2=0x7f34b40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.497+0000 7f34bac6e700 1 -- 172.21.15.7:0/2848826473 shutdown_connections 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.497+0000 7f34bac6e700 1 -- 172.21.15.7:0/2848826473 wait complete. 2024-01-11T07:10:54.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.498+0000 7f34bac6e700 1 Processor -- start 2024-01-11T07:10:54.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.498+0000 7f34bac6e700 1 -- start start 2024-01-11T07:10:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34bac6e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b4063230 0x7f34b4153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34bac6e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b40ac5b0 0x7f34b40afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34bac6e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b415a280 0x7f34b40ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34bac6e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4169fc0 con 0x7f34b415a280 2024-01-11T07:10:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34bac6e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40b3450 con 0x7f34b40ac5b0 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34bac6e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40b2840 con 0x7f34b4063230 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34b920b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b415a280 0x7f34b40ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34b920b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b415a280 0x7f34b40ad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41624/0 (socket says 172.21.15.7:41624) 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34b920b700 1 -- 172.21.15.7:0/4070244144 learned_addr learned my addr 172.21.15.7:0/4070244144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34b3fff700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b40ac5b0 0x7f34b40afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.499+0000 7f34b8a0a700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b4063230 0x7f34b4153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b920b700 1 -- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b4063230 msgr2=0x7f34b4153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b920b700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b4063230 0x7f34b4153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b920b700 1 -- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b40ac5b0 msgr2=0x7f34b40afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b920b700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b40ac5b0 0x7f34b40afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b920b700 1 -- 172.21.15.7:0/4070244144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac009ae0 con 0x7f34b415a280 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b8a0a700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b4063230 0x7f34b4153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b3fff700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b40ac5b0 0x7f34b40afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b920b700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b415a280 0x7f34b40ad820 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b610 tx=0x7f34ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:54.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b0ff9700 1 -- 172.21.15.7:0/4070244144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0094e0 con 0x7f34b415a280 2024-01-11T07:10:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b0ff9700 1 -- 172.21.15.7:0/4070244144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac008c40 con 0x7f34b415a280 2024-01-11T07:10:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b40b2b20 con 0x7f34b415a280 2024-01-11T07:10:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34b0ff9700 1 -- 172.21.15.7:0/4070244144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac026460 con 0x7f34b415a280 2024-01-11T07:10:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.500+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b40b3160 con 0x7f34b415a280 2024-01-11T07:10:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.501+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b416a100 con 0x7f34b415a280 2024-01-11T07:10:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.502+0000 7f34b0ff9700 1 -- 172.21.15.7:0/4070244144 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f34ac003040 con 0x7f34b415a280 2024-01-11T07:10:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.502+0000 7f34b0ff9700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a0066f30 0x7f34a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.502+0000 7f34b0ff9700 1 -- 172.21.15.7:0/4070244144 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f34ac086340 con 0x7f34b415a280 2024-01-11T07:10:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.502+0000 7f34b8a0a700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a0066f30 0x7f34a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.503+0000 7f34b8a0a700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a0066f30 0x7f34a00693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f34a4007fd0 tx=0x7f34a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.507+0000 7f34b0ff9700 1 -- 172.21.15.7:0/4070244144 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34b416a100 con 0x7f34b415a280 2024-01-11T07:10:54.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:54.712+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f34b416a100 con 0x7f34b415a280 2024-01-11T07:10:55.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.335+0000 7f34b0ff9700 1 -- 172.21.15.7:0/4070244144 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7f34ac084e60 con 0x7f34b415a280 2024-01-11T07:10:55.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a0066f30 msgr2=0x7f34a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a0066f30 0x7f34a00693e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f34a4007fd0 tx=0x7f34a4009040 comp rx=0 tx=0).stop 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b415a280 msgr2=0x7f34b40ad820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b415a280 0x7f34b40ad820 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b610 tx=0x7f34ac018000 comp rx=0 tx=0).stop 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 shutdown_connections 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a0066f30 0x7f34a00693e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34b415a280 0x7f34b40ad820 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34b4063230 0x7f34b4153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 --2- 172.21.15.7:0/4070244144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34b40ac5b0 0x7f34b40afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 >> 172.21.15.7:0/4070244144 conn(0x7f34b40680e0 msgr2=0x7f34b415b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 shutdown_connections 2024-01-11T07:10:55.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.346+0000 7f34bac6e700 1 -- 172.21.15.7:0/4070244144 wait complete. 2024-01-11T07:10:55.347 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:10:55.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:10:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.501+0000 7f72e902d700 1 Processor -- start 2024-01-11T07:10:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.501+0000 7f72e902d700 1 -- start start 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e902d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 0x7f72e40b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e902d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72e406dbe0 0x7f72e406e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e902d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e415b940 con 0x7f72e40afd90 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e902d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e406e5c0 con 0x7f72e406dbe0 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e902d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4069c70 con 0x7f72e40b0160 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72e406dbe0 0x7f72e406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72e406dbe0 0x7f72e406e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41632/0 (socket says 172.21.15.7:41632) 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72e40afd90 0x7f72e40c8c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40400/0 (socket says 172.21.15.7:40400) 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e359e700 1 -- 172.21.15.7:0/1107543887 learned_addr learned my addr 172.21.15.7:0/1107543887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e259c700 1 --2- 172.21.15.7:0/1107543887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 0x7f72e40b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e359e700 1 -- 172.21.15.7:0/1107543887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 msgr2=0x7f72e40b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e359e700 1 --2- 172.21.15.7:0/1107543887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 0x7f72e40b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e359e700 1 -- 172.21.15.7:0/1107543887 >> v1:172.21.15.7:6789/0 conn(0x7f72e40afd90 legacy=0x7f72e40c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e359e700 1 -- 172.21.15.7:0/1107543887 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4069f50 con 0x7f72e406dbe0 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e1d9b700 1 -- 172.21.15.7:0/1107543887 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954252905 0 0) 0x7f72e415b940 con 0x7f72e40afd90 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.502+0000 7f72e359e700 1 --2- 172.21.15.7:0/1107543887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e406e080 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009f20 tx=0x7f72dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28d5703d065ac08 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e1d9b700 1 -- 172.21.15.7:0/1107543887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00e070 con 0x7f72e406dbe0 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e1d9b700 1 -- 172.21.15.7:0/1107543887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc00bb10 con 0x7f72e406dbe0 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e1d9b700 1 -- 172.21.15.7:0/1107543887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0092e0 con 0x7f72e406dbe0 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e902d700 1 -- 172.21.15.7:0/1107543887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 msgr2=0x7f72e406e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e902d700 1 --2- 172.21.15.7:0/1107543887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e406e080 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009f20 tx=0x7f72dc009840 comp rx=0 tx=0).stop 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e902d700 1 -- 172.21.15.7:0/1107543887 shutdown_connections 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e902d700 1 --2- 172.21.15.7:0/1107543887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e406e080 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e902d700 1 --2- 172.21.15.7:0/1107543887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 0x7f72e40b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e902d700 1 -- 172.21.15.7:0/1107543887 >> 172.21.15.7:0/1107543887 conn(0x7f72e40680e0 msgr2=0x7f72e40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:55.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e902d700 1 -- 172.21.15.7:0/1107543887 shutdown_connections 2024-01-11T07:10:55.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.503+0000 7f72e902d700 1 -- 172.21.15.7:0/1107543887 wait complete. 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.505+0000 7f72e902d700 1 Processor -- start 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.505+0000 7f72e902d700 1 -- start start 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.505+0000 7f72e902d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e416aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.505+0000 7f72e902d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e40afd90 0x7f72e40cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.505+0000 7f72e902d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 0x7f72e414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.505+0000 7f72e902d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4153ca0 con 0x7f72e406dbe0 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.505+0000 7f72e902d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4153f40 con 0x7f72e40afd90 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.505+0000 7f72e902d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e416a410 con 0x7f72e40b0160 2024-01-11T07:10:55.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e416aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e416aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41638/0 (socket says 172.21.15.7:41638) 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 -- 172.21.15.7:0/2250868973 learned_addr learned my addr 172.21.15.7:0/2250868973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e359e700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 0x7f72e414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e259c700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e40afd90 0x7f72e40cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 -- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 msgr2=0x7f72e414cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 0x7f72e414cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 -- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e40afd90 msgr2=0x7f72e40cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e40afd90 0x7f72e40cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 -- 172.21.15.7:0/2250868973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc009ae0 con 0x7f72e406dbe0 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e2d9d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e416aef0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f72d40011e0 tx=0x7f72d4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72caffd700 1 -- 172.21.15.7:0/2250868973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4005a70 con 0x7f72e406dbe0 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72caffd700 1 -- 172.21.15.7:0/2250868973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d4005bd0 con 0x7f72e406dbe0 2024-01-11T07:10:55.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e416a6f0 con 0x7f72e406dbe0 2024-01-11T07:10:55.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72caffd700 1 -- 172.21.15.7:0/2250868973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d40070d0 con 0x7f72e406dbe0 2024-01-11T07:10:55.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.506+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e414e000 con 0x7f72e406dbe0 2024-01-11T07:10:55.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.507+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e4064a40 con 0x7f72e406dbe0 2024-01-11T07:10:55.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.508+0000 7f72caffd700 1 -- 172.21.15.7:0/2250868973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f72d4007480 con 0x7f72e406dbe0 2024-01-11T07:10:55.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.508+0000 7f72caffd700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d0066f30 0x7f72d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:55.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.508+0000 7f72caffd700 1 -- 172.21.15.7:0/2250868973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f72d4083980 con 0x7f72e406dbe0 2024-01-11T07:10:55.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.509+0000 7f72e259c700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d0066f30 0x7f72d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:55.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.509+0000 7f72e259c700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d0066f30 0x7f72d00693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72e406e9b0 tx=0x7f72d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:55.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.513+0000 7f72caffd700 1 -- 172.21.15.7:0/2250868973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72d4050c50 con 0x7f72e406dbe0 2024-01-11T07:10:55.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:55.763+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f72e404c3c0 con 0x7f72e406dbe0 2024-01-11T07:10:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.366+0000 7f72caffd700 1 -- 172.21.15.7:0/2250868973 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7f72d4017070 con 0x7f72e406dbe0 2024-01-11T07:10:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d0066f30 msgr2=0x7f72d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d0066f30 0x7f72d00693e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72e406e9b0 tx=0x7f72d8008040 comp rx=0 tx=0).stop 2024-01-11T07:10:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 msgr2=0x7f72e416aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e416aef0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f72d40011e0 tx=0x7f72d4007e10 comp rx=0 tx=0).stop 2024-01-11T07:10:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 shutdown_connections 2024-01-11T07:10:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d0066f30 0x7f72d00693e0 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72e406e9b0 tx=0x7f72d8008040 comp rx=0 tx=0).stop 2024-01-11T07:10:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e40b0160 0x7f72e414cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e406dbe0 0x7f72e416aef0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 --2- 172.21.15.7:0/2250868973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e40afd90 0x7f72e40cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 >> 172.21.15.7:0/2250868973 conn(0x7f72e40680e0 msgr2=0x7f72e40c8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.371+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 shutdown_connections 2024-01-11T07:10:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.372+0000 7f72e902d700 1 -- 172.21.15.7:0/2250868973 wait complete. 2024-01-11T07:10:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 10 and data pool 11 2024-01-11T07:10:56.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:10:56.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.525+0000 7f2899886700 1 Processor -- start 2024-01-11T07:10:56.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.525+0000 7f2899886700 1 -- start start 2024-01-11T07:10:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f2899886700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2894159ca0 0x7f289406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f2899886700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f289415a070 0x7f289406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f2899886700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28940d18c0 con 0x7f28940ce7f0 2024-01-11T07:10:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f2899886700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28940d0010 con 0x7f2894159ca0 2024-01-11T07:10:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f2899886700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28940cf400 con 0x7f289415a070 2024-01-11T07:10:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f2892ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2894159ca0 0x7f289406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f28937fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f28940ce7f0 0x7f28940cec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40404/0 (socket says 172.21.15.7:40404) 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f28937fe700 1 -- 172.21.15.7:0/437748266 learned_addr learned my addr 172.21.15.7:0/437748266 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.526+0000 7f28927fc700 1 --2- 172.21.15.7:0/437748266 >> v2:172.21.15.113:3300/0 conn(0x7f289415a070 0x7f289406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2892ffd700 1 -- 172.21.15.7:0/437748266 >> v2:172.21.15.113:3300/0 conn(0x7f289415a070 msgr2=0x7f289406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2892ffd700 1 --2- 172.21.15.7:0/437748266 >> v2:172.21.15.113:3300/0 conn(0x7f289415a070 0x7f289406e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2892ffd700 1 -- 172.21.15.7:0/437748266 >> v1:172.21.15.7:6789/0 conn(0x7f28940ce7f0 legacy=0x7f28940cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2892ffd700 1 -- 172.21.15.7:0/437748266 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940cf6e0 con 0x7f2894159ca0 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2891ffb700 1 -- 172.21.15.7:0/437748266 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779665527 0 0) 0x7f28940d18c0 con 0x7f28940ce7f0 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2892ffd700 1 --2- 172.21.15.7:0/437748266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894159ca0 0x7f289406dc30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f289415a510 tx=0x7f28840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efde7ecfb1389ea9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2891ffb700 1 -- 172.21.15.7:0/437748266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288400b6b0 con 0x7f2894159ca0 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2891ffb700 1 -- 172.21.15.7:0/437748266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288400b810 con 0x7f2894159ca0 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2891ffb700 1 -- 172.21.15.7:0/437748266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288400bbb0 con 0x7f2894159ca0 2024-01-11T07:10:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2899886700 1 -- 172.21.15.7:0/437748266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894159ca0 msgr2=0x7f289406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.527+0000 7f2899886700 1 --2- 172.21.15.7:0/437748266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894159ca0 0x7f289406dc30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f289415a510 tx=0x7f28840091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.528+0000 7f2899886700 1 -- 172.21.15.7:0/437748266 shutdown_connections 2024-01-11T07:10:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.528+0000 7f2899886700 1 --2- 172.21.15.7:0/437748266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2894159ca0 0x7f289406dc30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.528+0000 7f2899886700 1 --2- 172.21.15.7:0/437748266 >> v2:172.21.15.113:3300/0 conn(0x7f289415a070 0x7f289406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.528+0000 7f2899886700 1 -- 172.21.15.7:0/437748266 >> 172.21.15.7:0/437748266 conn(0x7f2894063cc0 msgr2=0x7f28940d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.528+0000 7f2899886700 1 -- 172.21.15.7:0/437748266 shutdown_connections 2024-01-11T07:10:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.528+0000 7f2899886700 1 -- 172.21.15.7:0/437748266 wait complete. 2024-01-11T07:10:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.529+0000 7f2899886700 1 Processor -- start 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.530+0000 7f2899886700 1 -- start start 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.530+0000 7f2899886700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28940ce7f0 0x7f28940b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.530+0000 7f2899886700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2894159ca0 0x7f28940b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2899886700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f289415a070 0x7f28940c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2899886700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b2610 con 0x7f28940ce7f0 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2899886700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b1720 con 0x7f289415a070 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2899886700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b1a00 con 0x7f2894159ca0 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28940ce7f0 0x7f28940b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28940ce7f0 0x7f28940b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41652/0 (socket says 172.21.15.7:41652) 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 -- 172.21.15.7:0/629755883 learned_addr learned my addr 172.21.15.7:0/629755883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f28927fc700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2894159ca0 0x7f28940b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f28937fe700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f289415a070 0x7f28940c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 -- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2894159ca0 msgr2=0x7f28940b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2894159ca0 0x7f28940b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 -- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f289415a070 msgr2=0x7f28940c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f289415a070 0x7f28940c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 -- 172.21.15.7:0/629755883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884009420 con 0x7f28940ce7f0 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f28927fc700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2894159ca0 0x7f28940b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f28937fe700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f289415a070 0x7f28940c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2892ffd700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28940ce7f0 0x7f28940b2aa0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2894068ad0 tx=0x7f2884017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f288affd700 1 -- 172.21.15.7:0/629755883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884011420 con 0x7f28940ce7f0 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f288affd700 1 -- 172.21.15.7:0/629755883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2884011580 con 0x7f28940ce7f0 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28940b1ce0 con 0x7f28940ce7f0 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.531+0000 7f288affd700 1 -- 172.21.15.7:0/629755883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884011940 con 0x7f28940ce7f0 2024-01-11T07:10:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.532+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28940b22a0 con 0x7f28940ce7f0 2024-01-11T07:10:56.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.533+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289404c3c0 con 0x7f28940ce7f0 2024-01-11T07:10:56.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.533+0000 7f288affd700 1 -- 172.21.15.7:0/629755883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2884011d30 con 0x7f28940ce7f0 2024-01-11T07:10:56.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.533+0000 7f288affd700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2874066f30 0x7f28740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.533+0000 7f288affd700 1 -- 172.21.15.7:0/629755883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2884085fe0 con 0x7f28940ce7f0 2024-01-11T07:10:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.534+0000 7f28927fc700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2874066f30 0x7f28740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.534+0000 7f28927fc700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2874066f30 0x7f28740693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f287c006fd0 tx=0x7f287c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.538+0000 7f288affd700 1 -- 172.21.15.7:0/629755883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2884053270 con 0x7f28940ce7f0 2024-01-11T07:10:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.742+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28940cf9c0 con 0x7f28940ce7f0 2024-01-11T07:10:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.743+0000 7f288affd700 1 -- 172.21.15.7:0/629755883 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+15318 (secure 0 0 0) 0x7f2884056890 con 0x7f28940ce7f0 2024-01-11T07:10:56.742 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:56.742 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":83,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:10:55.764976+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:10:53.813087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-11T07:10:54.744603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:10:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2874066f30 msgr2=0x7f28740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2874066f30 0x7f28740693e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f287c006fd0 tx=0x7f287c008040 comp rx=0 tx=0).stop 2024-01-11T07:10:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28940ce7f0 msgr2=0x7f28940b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28940ce7f0 0x7f28940b2aa0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2894068ad0 tx=0x7f2884017000 comp rx=0 tx=0).stop 2024-01-11T07:10:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 shutdown_connections 2024-01-11T07:10:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2874066f30 0x7f28740693e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f289415a070 0x7f28940c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28940ce7f0 0x7f28940b2aa0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 --2- 172.21.15.7:0/629755883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2894159ca0 0x7f28940b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 >> 172.21.15.7:0/629755883 conn(0x7f2894063cc0 msgr2=0x7f289414f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 shutdown_connections 2024-01-11T07:10:56.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.745+0000 7f2899886700 1 -- 172.21.15.7:0/629755883 wait complete. 2024-01-11T07:10:56.758 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:10:56.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.891+0000 7fabfe100700 1 Processor -- start 2024-01-11T07:10:56.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.891+0000 7fabfe100700 1 -- start start 2024-01-11T07:10:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.892+0000 7fabfe100700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 0x7fabf80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.892+0000 7fabfe100700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fabf8158280 0x7fabf80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.892+0000 7fabfe100700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf8159300 con 0x7fabf80b0970 2024-01-11T07:10:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.892+0000 7fabfe100700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf80b0df0 con 0x7fabf8158280 2024-01-11T07:10:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.892+0000 7fabfe100700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80cf260 con 0x7fabf80cafb0 2024-01-11T07:10:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fabf80b0970 0x7fabf8159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40416/0 (socket says 172.21.15.7:40416) 2024-01-11T07:10:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf7fff700 1 -- 172.21.15.7:0/427320335 learned_addr learned my addr 172.21.15.7:0/427320335 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/427320335 >> v2:172.21.15.113:3300/0 conn(0x7fabf8158280 0x7fabf80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf77fe700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 0x7fabf80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344600498 0 0) 0x7fabf8159300 con 0x7fabf80b0970 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe0003570 con 0x7fabf80b0970 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1496812468 0 0) 0x7fabe0003570 con 0x7fabf80b0970 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 msgr2=0x7fabf80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 0x7fabf80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 >> v2:172.21.15.113:3300/0 conn(0x7fabf8158280 msgr2=0x7fabf80b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 --2- 172.21.15.7:0/427320335 >> v2:172.21.15.113:3300/0 conn(0x7fabf8158280 0x7fabf80b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf80cf540 con 0x7fabf80b0970 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fabf00017f0 con 0x7fabf80b0970 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf77fe700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 0x7fabf80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 >> v1:172.21.15.7:6789/0 conn(0x7fabf80b0970 legacy=0x7fabf8159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0003a60 0x7fabe0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe0004810 0x7fabe0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 0x7fabe00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80b0df0 con 0x7fabe0004810 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80cf260 con 0x7fabe0003a60 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe000ba60 con 0x7fabf80cafb0 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/427320335 >> v2:172.21.15.113:3300/0 conn(0x7fabf8158280 0x7fabf80b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.893+0000 7fabf77fe700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0003a60 0x7fabe0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe0004810 0x7fabe0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf7fff700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 0x7fabe00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf6ffd700 1 -- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 msgr2=0x7fabe00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80cafb0 0x7fabe00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf6ffd700 1 -- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0003a60 msgr2=0x7fabe0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0003a60 0x7fabe0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf6ffd700 1 -- 172.21.15.7:0/427320335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe8009420 con 0x7fabe0004810 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf77fe700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0003a60 0x7fabe0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe0004810 0x7fabe0004c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fabf8158650 tx=0x7fabec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247f23f61c27efd8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec009740 con 0x7fabe0004810 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec00e920 con 0x7fabe0004810 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.894+0000 7fabf67fc700 1 -- 172.21.15.7:0/427320335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00ecc0 con 0x7fabe0004810 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.895+0000 7fabfe100700 1 -- 172.21.15.7:0/427320335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe0004810 msgr2=0x7fabe0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.895+0000 7fabfe100700 1 --2- 172.21.15.7:0/427320335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe0004810 0x7fabe0004c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fabf8158650 tx=0x7fabec0091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.895+0000 7fabf7fff700 1 -- 172.21.15.7:0/427320335 reap_dead start 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.895+0000 7fabfe100700 1 -- 172.21.15.7:0/427320335 shutdown_connections 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.895+0000 7fabfe100700 1 -- 172.21.15.7:0/427320335 >> 172.21.15.7:0/427320335 conn(0x7fabf8071100 msgr2=0x7fabf806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.895+0000 7fabfe100700 1 -- 172.21.15.7:0/427320335 shutdown_connections 2024-01-11T07:10:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.895+0000 7fabfe100700 1 -- 172.21.15.7:0/427320335 wait complete. 2024-01-11T07:10:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.896+0000 7fabfe100700 1 Processor -- start 2024-01-11T07:10:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.896+0000 7fabfe100700 1 -- start start 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabfe100700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0004810 0x7fabf80c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabfe100700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf8167fd0 0x7fabf80af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabfe100700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80af800 0x7fabf815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabfe100700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80afc80 con 0x7fabf8167fd0 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabfe100700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80c3930 con 0x7fabe0004810 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabfe100700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80c3c10 con 0x7fabf80af800 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf8167fd0 0x7fabf80af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf8167fd0 0x7fabf80af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41676/0 (socket says 172.21.15.7:41676) 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf6ffd700 1 -- 172.21.15.7:0/2515539287 learned_addr learned my addr 172.21.15.7:0/2515539287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf77fe700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0004810 0x7fabf80c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf7fff700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80af800 0x7fabf815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf6ffd700 1 -- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80af800 msgr2=0x7fabf815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80af800 0x7fabf815bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf6ffd700 1 -- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0004810 msgr2=0x7fabf80c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0004810 0x7fabf80c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf6ffd700 1 -- 172.21.15.7:0/2515539287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf80c3ef0 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf77fe700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0004810 0x7fabf80c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.897+0000 7fabf7fff700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80af800 0x7fabf815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.898+0000 7fabf6ffd700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf8167fd0 0x7fabf80af2c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fabec00e790 tx=0x7fabec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.898+0000 7fabdf7fe700 1 -- 172.21.15.7:0/2515539287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec01b6d0 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.898+0000 7fabdf7fe700 1 -- 172.21.15.7:0/2515539287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec021040 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.898+0000 7fabdf7fe700 1 -- 172.21.15.7:0/2515539287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec01baa0 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.898+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf80c41d0 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.898+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf8168440 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.899+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf80c4670 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.899+0000 7fabdf7fe700 1 -- 172.21.15.7:0/2515539287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fabec02e410 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.900+0000 7fabdf7fe700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.900+0000 7fabdf7fe700 1 -- 172.21.15.7:0/2515539287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fabec0861f0 con 0x7fabf8167fd0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.900+0000 7fabf77fe700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.901+0000 7fabf77fe700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabe4066f30 0x7fabe40693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009910 tx=0x7fabe8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:56.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:56.904+0000 7fabdf7fe700 1 -- 172.21.15.7:0/2515539287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabec0533c0 con 0x7fabf8167fd0 2024-01-11T07:10:57.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.111+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabf80afc40 con 0x7fabf8167fd0 2024-01-11T07:10:57.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.112+0000 7fabdf7fe700 1 -- 172.21.15.7:0/2515539287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+15318 (secure 0 0 0) 0x7fabec0569e0 con 0x7fabf8167fd0 2024-01-11T07:10:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:57.112 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":83,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:10:55.764976+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:10:53.813087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-11T07:10:54.744603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:10:57.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabe4066f30 msgr2=0x7fabe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:57.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabe4066f30 0x7fabe40693e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009910 tx=0x7fabe8008040 comp rx=0 tx=0).stop 2024-01-11T07:10:57.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf8167fd0 msgr2=0x7fabf80af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:57.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf8167fd0 0x7fabf80af2c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fabec00e790 tx=0x7fabec019000 comp rx=0 tx=0).stop 2024-01-11T07:10:57.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 shutdown_connections 2024-01-11T07:10:57.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabe4066f30 0x7fabe40693e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf80af800 0x7fabf815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabe0004810 0x7fabf80c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 --2- 172.21.15.7:0/2515539287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf8167fd0 0x7fabf80af2c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 >> 172.21.15.7:0/2515539287 conn(0x7fabf8071100 msgr2=0x7fabf80ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:57.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 shutdown_connections 2024-01-11T07:10:57.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.114+0000 7fabfe100700 1 -- 172.21.15.7:0/2515539287 wait complete. 2024-01-11T07:10:57.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:10:57.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.263+0000 7f9a77310700 1 Processor -- start 2024-01-11T07:10:57.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.264+0000 7f9a77310700 1 -- start start 2024-01-11T07:10:57.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.264+0000 7f9a77310700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a7015d870 0x7f9a700cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:57.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.264+0000 7f9a77310700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700cca30 0x7f9a70068570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:57.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.264+0000 7f9a77310700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a700680e0 con 0x7f9a7015d4a0 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.264+0000 7f9a77310700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a700cced0 con 0x7f9a7015d870 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.264+0000 7f9a77310700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700b1ea0 con 0x7f9a700cca30 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a748ab700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a7015d870 0x7f9a700cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a748ab700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a7015d870 0x7f9a700cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41690/0 (socket says 172.21.15.7:41690) 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a748ab700 1 -- 172.21.15.7:0/4182630904 learned_addr learned my addr 172.21.15.7:0/4182630904 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a758ad700 1 --2- 172.21.15.7:0/4182630904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700cca30 0x7f9a70068570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a748ab700 1 -- 172.21.15.7:0/4182630904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700cca30 msgr2=0x7f9a70068570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a748ab700 1 --2- 172.21.15.7:0/4182630904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700cca30 0x7f9a70068570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a748ab700 1 -- 172.21.15.7:0/4182630904 >> v1:172.21.15.113:6789/0 conn(0x7f9a7015d4a0 legacy=0x7f9a700b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a748ab700 1 -- 172.21.15.7:0/4182630904 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a700677b0 con 0x7f9a7015d870 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a748ab700 1 --2- 172.21.15.7:0/4182630904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a7015d870 0x7f9a700cc4f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9a600099e0 tx=0x7f9a60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f7817600ce3a13 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a67fff700 1 -- 172.21.15.7:0/4182630904 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000b7e0 con 0x7f9a7015d870 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a67fff700 1 -- 172.21.15.7:0/4182630904 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6000b940 con 0x7f9a7015d870 2024-01-11T07:10:57.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.265+0000 7f9a67fff700 1 -- 172.21.15.7:0/4182630904 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000bce0 con 0x7f9a7015d870 2024-01-11T07:10:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.266+0000 7f9a77310700 1 -- 172.21.15.7:0/4182630904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a7015d870 msgr2=0x7f9a700cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.266+0000 7f9a77310700 1 --2- 172.21.15.7:0/4182630904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a7015d870 0x7f9a700cc4f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9a600099e0 tx=0x7f9a60009250 comp rx=0 tx=0).stop 2024-01-11T07:10:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.266+0000 7f9a77310700 1 -- 172.21.15.7:0/4182630904 shutdown_connections 2024-01-11T07:10:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.266+0000 7f9a77310700 1 --2- 172.21.15.7:0/4182630904 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700cca30 0x7f9a70068570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.266+0000 7f9a77310700 1 --2- 172.21.15.7:0/4182630904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a7015d870 0x7f9a700cc4f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.266+0000 7f9a77310700 1 -- 172.21.15.7:0/4182630904 >> 172.21.15.7:0/4182630904 conn(0x7f9a70097a50 msgr2=0x7f9a700ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.266+0000 7f9a77310700 1 -- 172.21.15.7:0/4182630904 shutdown_connections 2024-01-11T07:10:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.266+0000 7f9a77310700 1 -- 172.21.15.7:0/4182630904 wait complete. 2024-01-11T07:10:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.267+0000 7f9a77310700 1 Processor -- start 2024-01-11T07:10:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.267+0000 7f9a77310700 1 -- start start 2024-01-11T07:10:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a77310700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a700cca30 0x7f9a70169f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a77310700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a7015d4a0 0x7f9a700c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a77310700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700b3220 0x7f9a700b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a77310700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7016aea0 con 0x7f9a700cca30 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a77310700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700b3fb0 con 0x7f9a7015d4a0 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a77310700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7016d750 con 0x7f9a700b3220 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a750ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a700cca30 0x7f9a70169f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a750ac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a700cca30 0x7f9a70169f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41694/0 (socket says 172.21.15.7:41694) 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a750ac700 1 -- 172.21.15.7:0/1707207664 learned_addr learned my addr 172.21.15.7:0/1707207664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a758ad700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700b3220 0x7f9a700b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.268+0000 7f9a748ab700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a7015d4a0 0x7f9a700c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a750ac700 1 -- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700b3220 msgr2=0x7f9a700b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a750ac700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700b3220 0x7f9a700b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a750ac700 1 -- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a7015d4a0 msgr2=0x7f9a700c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a750ac700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a7015d4a0 0x7f9a700c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a750ac700 1 -- 172.21.15.7:0/1707207664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a600094f0 con 0x7f9a700cca30 2024-01-11T07:10:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a750ac700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a700cca30 0x7f9a70169f20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b5e0 tx=0x7f9a6c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a657fa700 1 -- 172.21.15.7:0/1707207664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00f510 con 0x7f9a700cca30 2024-01-11T07:10:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a657fa700 1 -- 172.21.15.7:0/1707207664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00f670 con 0x7f9a700cca30 2024-01-11T07:10:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7016a600 con 0x7f9a700cca30 2024-01-11T07:10:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a657fa700 1 -- 172.21.15.7:0/1707207664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00fa30 con 0x7f9a700cca30 2024-01-11T07:10:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.269+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7016abc0 con 0x7f9a700cca30 2024-01-11T07:10:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.270+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7016d520 con 0x7f9a700cca30 2024-01-11T07:10:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.271+0000 7f9a657fa700 1 -- 172.21.15.7:0/1707207664 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9a6c024430 con 0x7f9a700cca30 2024-01-11T07:10:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.271+0000 7f9a657fa700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a54066f30 0x7f9a540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.272+0000 7f9a657fa700 1 -- 172.21.15.7:0/1707207664 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a6c084720 con 0x7f9a700cca30 2024-01-11T07:10:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.272+0000 7f9a748ab700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a54066f30 0x7f9a540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.272+0000 7f9a748ab700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a54066f30 0x7f9a540693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9a600099e0 tx=0x7f9a60016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:57.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.275+0000 7f9a657fa700 1 -- 172.21.15.7:0/1707207664 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a6c051ba0 con 0x7f9a700cca30 2024-01-11T07:10:57.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:57.527+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9a7016d520 con 0x7f9a700cca30 2024-01-11T07:10:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.371+0000 7f9a657fa700 1 -- 172.21.15.7:0/1707207664 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9a7016d520 con 0x7f9a700cca30 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a54066f30 msgr2=0x7f9a540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a54066f30 0x7f9a540693e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9a600099e0 tx=0x7f9a60016000 comp rx=0 tx=0).stop 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a700cca30 msgr2=0x7f9a70169f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a700cca30 0x7f9a70169f20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b5e0 tx=0x7f9a6c0090f0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 shutdown_connections 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a54066f30 0x7f9a540693e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a700b3220 0x7f9a700b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a700cca30 0x7f9a70169f20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 --2- 172.21.15.7:0/1707207664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a7015d4a0 0x7f9a700c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 >> 172.21.15.7:0/1707207664 conn(0x7f9a70097a50 msgr2=0x7f9a7015b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:58.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 shutdown_connections 2024-01-11T07:10:58.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.374+0000 7f9a77310700 1 -- 172.21.15.7:0/1707207664 wait complete. 2024-01-11T07:10:58.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:10:58.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.529+0000 7f70efeb0700 1 Processor -- start 2024-01-11T07:10:58.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.530+0000 7f70efeb0700 1 -- start start 2024-01-11T07:10:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.530+0000 7f70efeb0700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70e815d4a0 0x7f70e80b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.530+0000 7f70efeb0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d870 0x7f70e80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70efeb0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e80680e0 con 0x7f70e80cca30 2024-01-11T07:10:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70efeb0700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e80cced0 con 0x7f70e815d4a0 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70efeb0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80677b0 con 0x7f70e815d870 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ee44d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f70e80cca30 0x7f70e8068570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44420/0 (socket says 172.21.15.7:44420) 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ee44d700 1 -- 172.21.15.7:0/2167111434 learned_addr learned my addr 172.21.15.7:0/2167111434 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70edc4c700 1 --2- 172.21.15.7:0/2167111434 >> v2:172.21.15.113:3300/0 conn(0x7f70e815d4a0 0x7f70e80b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ed44b700 1 --2- 172.21.15.7:0/2167111434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d870 0x7f70e80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ecc4a700 1 -- 172.21.15.7:0/2167111434 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005906275 0 0) 0x7f70e80680e0 con 0x7f70e80cca30 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ecc4a700 1 -- 172.21.15.7:0/2167111434 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70d4003570 con 0x7f70e80cca30 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ecc4a700 1 -- 172.21.15.7:0/2167111434 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897067864 0 0) 0x7f70d4003570 con 0x7f70e80cca30 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ed44b700 1 -- 172.21.15.7:0/2167111434 >> v2:172.21.15.113:3300/0 conn(0x7f70e815d4a0 msgr2=0x7f70e80b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ed44b700 1 --2- 172.21.15.7:0/2167111434 >> v2:172.21.15.113:3300/0 conn(0x7f70e815d4a0 0x7f70e80b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ed44b700 1 -- 172.21.15.7:0/2167111434 >> v1:172.21.15.7:6789/0 conn(0x7f70e80cca30 legacy=0x7f70e8068570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70ed44b700 1 -- 172.21.15.7:0/2167111434 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e8067a90 con 0x7f70e815d870 2024-01-11T07:10:58.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.531+0000 7f70edc4c700 1 --2- 172.21.15.7:0/2167111434 >> v2:172.21.15.113:3300/0 conn(0x7f70e815d4a0 0x7f70e80b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70ed44b700 1 --2- 172.21.15.7:0/2167111434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d870 0x7f70e80cc4f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f70d80098e0 tx=0x7f70d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a3a6f8f5c06845 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70ecc4a700 1 -- 172.21.15.7:0/2167111434 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e00017f0 con 0x7f70e815d870 2024-01-11T07:10:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70ecc4a700 1 -- 172.21.15.7:0/2167111434 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8011420 con 0x7f70e815d870 2024-01-11T07:10:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70ecc4a700 1 -- 172.21.15.7:0/2167111434 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80117a0 con 0x7f70e815d870 2024-01-11T07:10:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70efeb0700 1 -- 172.21.15.7:0/2167111434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d870 msgr2=0x7f70e80cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2167111434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d870 0x7f70e80cc4f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f70d80098e0 tx=0x7f70d80091a0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70efeb0700 1 -- 172.21.15.7:0/2167111434 shutdown_connections 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2167111434 >> v2:172.21.15.113:3300/0 conn(0x7f70e815d4a0 0x7f70e80b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2167111434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d870 0x7f70e80cc4f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.532+0000 7f70efeb0700 1 -- 172.21.15.7:0/2167111434 >> 172.21.15.7:0/2167111434 conn(0x7f70e8097a50 msgr2=0x7f70e80ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.533+0000 7f70efeb0700 1 -- 172.21.15.7:0/2167111434 shutdown_connections 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.533+0000 7f70efeb0700 1 -- 172.21.15.7:0/2167111434 wait complete. 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.534+0000 7f70efeb0700 1 Processor -- start 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.534+0000 7f70efeb0700 1 -- start start 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70efeb0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e80cca30 0x7f70e8147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70efeb0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d4a0 0x7f70e80b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70efeb0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e80b3270 0x7f70e815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70efeb0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80b4400 con 0x7f70e80cca30 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70efeb0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80b37f0 con 0x7f70e80b3270 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70efeb0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80b3ad0 con 0x7f70e815d4a0 2024-01-11T07:10:58.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e80cca30 0x7f70e8147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e80cca30 0x7f70e8147cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33004/0 (socket says 172.21.15.7:33004) 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 -- 172.21.15.7:0/2213783413 learned_addr learned my addr 172.21.15.7:0/2213783413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70ed44b700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d4a0 0x7f70e80b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70ee44d700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e80b3270 0x7f70e815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 -- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d4a0 msgr2=0x7f70e80b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d4a0 0x7f70e80b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 -- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e80b3270 msgr2=0x7f70e815a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e80b3270 0x7f70e815a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 -- 172.21.15.7:0/2213783413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40093c0 con 0x7f70e80cca30 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70ee44d700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e80b3270 0x7f70e815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70edc4c700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e80cca30 0x7f70e8147cd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f70e400ec80 tx=0x7f70e4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70ddffb700 1 -- 172.21.15.7:0/2213783413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400ca60 con 0x7f70e80cca30 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70ddffb700 1 -- 172.21.15.7:0/2213783413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e400cbc0 con 0x7f70e80cca30 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d8009420 con 0x7f70e80cca30 2024-01-11T07:10:58.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.535+0000 7f70ddffb700 1 -- 172.21.15.7:0/2213783413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4005a00 con 0x7f70e80cca30 2024-01-11T07:10:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.536+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e80b4090 con 0x7f70e80cca30 2024-01-11T07:10:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.537+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e8160bd0 con 0x7f70e80cca30 2024-01-11T07:10:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.537+0000 7f70ddffb700 1 -- 172.21.15.7:0/2213783413 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f70e4006030 con 0x7f70e80cca30 2024-01-11T07:10:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.538+0000 7f70ddffb700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70cc066f30 0x7f70cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.538+0000 7f70ddffb700 1 -- 172.21.15.7:0/2213783413 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70e4084b00 con 0x7f70e80cca30 2024-01-11T07:10:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.538+0000 7f70ed44b700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70cc066f30 0x7f70cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.538+0000 7f70ed44b700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70cc066f30 0x7f70cc0693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70d80091e0 tx=0x7f70d8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.542+0000 7f70ddffb700 1 -- 172.21.15.7:0/2213783413 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70e4051d70 con 0x7f70e80cca30 2024-01-11T07:10:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:58.799+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f70e804c3c0 con 0x7f70e80cca30 2024-01-11T07:10:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.372+0000 7f70ddffb700 1 -- 172.21.15.7:0/2213783413 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f70e4055390 con 0x7f70e80cca30 2024-01-11T07:10:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70cc066f30 msgr2=0x7f70cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70cc066f30 0x7f70cc0693e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70d80091e0 tx=0x7f70d8016000 comp rx=0 tx=0).stop 2024-01-11T07:10:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e80cca30 msgr2=0x7f70e8147cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e80cca30 0x7f70e8147cd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f70e400ec80 tx=0x7f70e4009120 comp rx=0 tx=0).stop 2024-01-11T07:10:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 shutdown_connections 2024-01-11T07:10:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70cc066f30 0x7f70cc0693e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e80b3270 0x7f70e815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e80cca30 0x7f70e8147cd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 --2- 172.21.15.7:0/2213783413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e815d4a0 0x7f70e80b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 >> 172.21.15.7:0/2213783413 conn(0x7f70e8097a50 msgr2=0x7f70e8068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 shutdown_connections 2024-01-11T07:10:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.375+0000 7f70efeb0700 1 -- 172.21.15.7:0/2213783413 wait complete. 2024-01-11T07:10:59.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:10:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.527+0000 7fbca8cee700 1 Processor -- start 2024-01-11T07:10:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.527+0000 7fbca8cee700 1 -- start start 2024-01-11T07:10:59.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca8cee700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbca40d0170 0x7fbca40cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca8cee700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca415acf0 con 0x7fbca40cd6d0 2024-01-11T07:10:59.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca8cee700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca415afd0 con 0x7fbca415dfb0 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca8cee700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca40ce980 con 0x7fbca40d0170 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbca40d0170 0x7fbca40cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbca40d0170 0x7fbca40cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33010/0 (socket says 172.21.15.7:33010) 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbca40cd6d0 0x7fbca415d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44428/0 (socket says 172.21.15.7:44428) 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca1d9b700 1 -- 172.21.15.7:0/2050230101 learned_addr learned my addr 172.21.15.7:0/2050230101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.528+0000 7fbca1d9b700 1 -- 172.21.15.7:0/2050230101 >> v1:172.21.15.113:6789/0 conn(0x7fbca415dfb0 legacy=0x7fbca4159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca1d9b700 1 -- 172.21.15.7:0/2050230101 >> v1:172.21.15.7:6789/0 conn(0x7fbca40cd6d0 legacy=0x7fbca415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca159a700 1 -- 172.21.15.7:0/2050230101 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183613079 0 0) 0x7fbca415acf0 con 0x7fbca40cd6d0 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca1d9b700 1 -- 172.21.15.7:0/2050230101 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40cec60 con 0x7fbca40d0170 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca1d9b700 1 --2- 172.21.15.7:0/2050230101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca40cdaa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0099b0 tx=0x7fbc8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77127310dc705f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca159a700 1 -- 172.21.15.7:0/2050230101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00b860 con 0x7fbca40d0170 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca159a700 1 -- 172.21.15.7:0/2050230101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8c00b9c0 con 0x7fbca40d0170 2024-01-11T07:10:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca159a700 1 -- 172.21.15.7:0/2050230101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00bd60 con 0x7fbca40d0170 2024-01-11T07:10:59.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca8cee700 1 -- 172.21.15.7:0/2050230101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 msgr2=0x7fbca40cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.529+0000 7fbca8cee700 1 --2- 172.21.15.7:0/2050230101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca40cdaa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0099b0 tx=0x7fbc8c009250 comp rx=0 tx=0).stop 2024-01-11T07:10:59.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.530+0000 7fbca8cee700 1 -- 172.21.15.7:0/2050230101 shutdown_connections 2024-01-11T07:10:59.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.530+0000 7fbca8cee700 1 --2- 172.21.15.7:0/2050230101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca40cdaa0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.530+0000 7fbca8cee700 1 -- 172.21.15.7:0/2050230101 >> 172.21.15.7:0/2050230101 conn(0x7fbca4071120 msgr2=0x7fbca406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.530+0000 7fbca8cee700 1 -- 172.21.15.7:0/2050230101 shutdown_connections 2024-01-11T07:10:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.530+0000 7fbca8cee700 1 -- 172.21.15.7:0/2050230101 wait complete. 2024-01-11T07:10:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.531+0000 7fbca8cee700 1 Processor -- start 2024-01-11T07:10:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.531+0000 7fbca8cee700 1 -- start start 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca8cee700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbca40cd6d0 0x7fbca40b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca8cee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca414e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca8cee700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbca415dfb0 0x7fbca40b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca8cee700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca40b41b0 con 0x7fbca40d0170 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca8cee700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca40b67e0 con 0x7fbca415dfb0 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca8cee700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca4150ff0 con 0x7fbca40cd6d0 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca414e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca414e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33018/0 (socket says 172.21.15.7:33018) 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 -- 172.21.15.7:0/1039397749 learned_addr learned my addr 172.21.15.7:0/1039397749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca2d9d700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbca415dfb0 0x7fbca40b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca259c700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbca40cd6d0 0x7fbca40b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 -- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbca40cd6d0 msgr2=0x7fbca40b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbca40cd6d0 0x7fbca40b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 -- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbca415dfb0 msgr2=0x7fbca40b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbca415dfb0 0x7fbca40b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 -- 172.21.15.7:0/1039397749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c0094f0 con 0x7fbca40d0170 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca259c700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbca40cd6d0 0x7fbca40b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbca1d9b700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca414e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c006f50 tx=0x7fbc8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.532+0000 7fbc9a7fc700 1 -- 172.21.15.7:0/1039397749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c01f070 con 0x7fbca40d0170 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.533+0000 7fbc9a7fc700 1 -- 172.21.15.7:0/1039397749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8c011c60 con 0x7fbca40d0170 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.533+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca41512d0 con 0x7fbca40d0170 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.533+0000 7fbc9a7fc700 1 -- 172.21.15.7:0/1039397749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c025420 con 0x7fbca40d0170 2024-01-11T07:10:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.533+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca414e830 con 0x7fbca40d0170 2024-01-11T07:10:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.534+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca40b4000 con 0x7fbca40d0170 2024-01-11T07:10:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.534+0000 7fbc9a7fc700 1 -- 172.21.15.7:0/1039397749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbc8c023070 con 0x7fbca40d0170 2024-01-11T07:10:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.534+0000 7fbc9a7fc700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc84066f30 0x7fbc840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.534+0000 7fbc9a7fc700 1 -- 172.21.15.7:0/1039397749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc8c085b40 con 0x7fbca40d0170 2024-01-11T07:10:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.535+0000 7fbca259c700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc84066f30 0x7fbc840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.535+0000 7fbca259c700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc84066f30 0x7fbc840693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbc940011e0 tx=0x7fbc94007c70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.539+0000 7fbc9a7fc700 1 -- 172.21.15.7:0/1039397749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc8c052db0 con 0x7fbca40d0170 2024-01-11T07:10:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.788+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbca404c3c0 con 0x7fbca40d0170 2024-01-11T07:10:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.789+0000 7fbc9a7fc700 1 -- 172.21.15.7:0/1039397749 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fbc8c02a090 con 0x7fbca40d0170 2024-01-11T07:10:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:10:59.789 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2882608277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2882608277},{"type":"v1","addr":"172.21.15.7:6833","nonce":2882608277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3510440823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3510440823},{"type":"v1","addr":"172.21.15.7:6835","nonce":3510440823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3108557963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3108557963},{"type":"v1","addr":"172.21.15.113:6836","nonce":3108557963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:10:55.765305+0000","modified":"2024-01-11T07:10:59.370619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1888585107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1888585107},{"type":"v1","addr":"172.21.15.113:6837","nonce":1888585107}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-11T07:10:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc84066f30 msgr2=0x7fbc840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc84066f30 0x7fbc840693e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbc940011e0 tx=0x7fbc94007c70 comp rx=0 tx=0).stop 2024-01-11T07:10:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 msgr2=0x7fbca414e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca414e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c006f50 tx=0x7fbc8c017000 comp rx=0 tx=0).stop 2024-01-11T07:10:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 shutdown_connections 2024-01-11T07:10:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc84066f30 0x7fbc840693e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbca415dfb0 0x7fbca40b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbca40cd6d0 0x7fbca40b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 --2- 172.21.15.7:0/1039397749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca40d0170 0x7fbca414e2f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 >> 172.21.15.7:0/1039397749 conn(0x7fbca4071120 msgr2=0x7fbca40c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 shutdown_connections 2024-01-11T07:10:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.791+0000 7fbca8cee700 1 -- 172.21.15.7:0/1039397749 wait complete. 2024-01-11T07:10:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 41 2024-01-11T07:10:59.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:10:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.936+0000 7f13b8eba700 1 Processor -- start 2024-01-11T07:10:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.936+0000 7f13b8eba700 1 -- start start 2024-01-11T07:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b8eba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13b415a060 0x7f13b40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b8eba700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b4069320 0x7f13b40b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b8eba700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b40ce9b0 con 0x7f13b4159c90 2024-01-11T07:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b8eba700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40b1f30 con 0x7f13b415a060 2024-01-11T07:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b8eba700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40cf530 con 0x7f13b4069320 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13b415a060 0x7f13b40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13b415a060 0x7f13b40b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33026/0 (socket says 172.21.15.7:33026) 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b1d9b700 1 -- 172.21.15.7:0/479941342 learned_addr learned my addr 172.21.15.7:0/479941342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b2d9d700 1 --2- 172.21.15.7:0/479941342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b4069320 0x7f13b40b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b1d9b700 1 -- 172.21.15.7:0/479941342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b4069320 msgr2=0x7f13b40b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b1d9b700 1 --2- 172.21.15.7:0/479941342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b4069320 0x7f13b40b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b1d9b700 1 -- 172.21.15.7:0/479941342 >> v1:172.21.15.113:6789/0 conn(0x7f13b4159c90 legacy=0x7f13b40b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.937+0000 7f13b1d9b700 1 -- 172.21.15.7:0/479941342 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b40cf9c0 con 0x7f13b415a060 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b1d9b700 1 --2- 172.21.15.7:0/479941342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b415a060 0x7f13b40b0430 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f13a80099b0 tx=0x7f13a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f8569e9a1ff1b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b159a700 1 -- 172.21.15.7:0/479941342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8009ea0 con 0x7f13b415a060 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b159a700 1 -- 172.21.15.7:0/479941342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a800b860 con 0x7f13b415a060 2024-01-11T07:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b159a700 1 -- 172.21.15.7:0/479941342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a800bc00 con 0x7f13b415a060 2024-01-11T07:10:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b8eba700 1 -- 172.21.15.7:0/479941342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b415a060 msgr2=0x7f13b40b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b8eba700 1 --2- 172.21.15.7:0/479941342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b415a060 0x7f13b40b0430 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f13a80099b0 tx=0x7f13a8009250 comp rx=0 tx=0).stop 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b8eba700 1 -- 172.21.15.7:0/479941342 shutdown_connections 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b8eba700 1 --2- 172.21.15.7:0/479941342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b4069320 0x7f13b40b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b8eba700 1 --2- 172.21.15.7:0/479941342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b415a060 0x7f13b40b0430 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.938+0000 7f13b8eba700 1 -- 172.21.15.7:0/479941342 >> 172.21.15.7:0/479941342 conn(0x7f13b4071100 msgr2=0x7f13b406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.939+0000 7f13b8eba700 1 -- 172.21.15.7:0/479941342 shutdown_connections 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.939+0000 7f13b8eba700 1 -- 172.21.15.7:0/479941342 wait complete. 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.940+0000 7f13b8eba700 1 Processor -- start 2024-01-11T07:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.940+0000 7f13b8eba700 1 -- start start 2024-01-11T07:10:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b8eba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4069320 0x7f13b4141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b8eba700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13b4159c90 0x7f13b415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b8eba700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b415a060 0x7f13b40b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b8eba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b416b380 con 0x7f13b4069320 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b8eba700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b41602a0 con 0x7f13b4159c90 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b8eba700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4160580 con 0x7f13b415a060 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4069320 0x7f13b4141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4069320 0x7f13b4141190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33042/0 (socket says 172.21.15.7:33042) 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.941+0000 7f13b259c700 1 -- 172.21.15.7:0/3293719032 learned_addr learned my addr 172.21.15.7:0/3293719032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b2d9d700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b415a060 0x7f13b40b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b1d9b700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13b4159c90 0x7f13b415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b259c700 1 -- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b415a060 msgr2=0x7f13b40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b259c700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b415a060 0x7f13b40b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b259c700 1 -- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13b4159c90 msgr2=0x7f13b415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b259c700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13b4159c90 0x7f13b415f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b259c700 1 -- 172.21.15.7:0/3293719032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a80094f0 con 0x7f13b4069320 2024-01-11T07:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b1d9b700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13b4159c90 0x7f13b415f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b259c700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4069320 0x7f13b4141190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f13a400c590 tx=0x7f13a400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13a27fc700 1 -- 172.21.15.7:0/3293719032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4011590 con 0x7f13b4069320 2024-01-11T07:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13a27fc700 1 -- 172.21.15.7:0/3293719032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a4011710 con 0x7f13b4069320 2024-01-11T07:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13a27fc700 1 -- 172.21.15.7:0/3293719032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4011ad0 con 0x7f13b4069320 2024-01-11T07:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b416b5d0 con 0x7f13b4069320 2024-01-11T07:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.942+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b4154bc0 con 0x7f13b4069320 2024-01-11T07:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.943+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b404c3c0 con 0x7f13b4069320 2024-01-11T07:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.944+0000 7f13a27fc700 1 -- 172.21.15.7:0/3293719032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f13a401b3b0 con 0x7f13b4069320 2024-01-11T07:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.944+0000 7f13a27fc700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1394066f30 0x7f13940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.945+0000 7f13a27fc700 1 -- 172.21.15.7:0/3293719032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f13a40839a0 con 0x7f13b4069320 2024-01-11T07:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.945+0000 7f13b1d9b700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1394066f30 0x7f13940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.945+0000 7f13b1d9b700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1394066f30 0x7f13940693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f13a80099b0 tx=0x7f13a8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:10:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:10:59.949+0000 7f13a27fc700 1 -- 172.21.15.7:0/3293719032 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13a4050c10 con 0x7f13b4069320 2024-01-11T07:11:00.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.156+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13b416baf0 con 0x7f13b4069320 2024-01-11T07:11:00.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.157+0000 7f13a27fc700 1 -- 172.21.15.7:0/3293719032 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+15305 (secure 0 0 0) 0x7f13b416baf0 con 0x7f13b4069320 2024-01-11T07:11:00.157 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:00.157 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:10:57.348891+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:10:53.813087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-11T07:10:54.744603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:11:00.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1394066f30 msgr2=0x7f13940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1394066f30 0x7f13940693e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f13a80099b0 tx=0x7f13a8017000 comp rx=0 tx=0).stop 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4069320 msgr2=0x7f13b4141190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4069320 0x7f13b4141190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f13a400c590 tx=0x7f13a400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 shutdown_connections 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1394066f30 0x7f13940693e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13b415a060 0x7f13b40b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4069320 0x7f13b4141190 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.159+0000 7f13b8eba700 1 --2- 172.21.15.7:0/3293719032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13b4159c90 0x7f13b415f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.160+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 >> 172.21.15.7:0/3293719032 conn(0x7f13b4071100 msgr2=0x7f13b415c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.160+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 shutdown_connections 2024-01-11T07:11:00.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.160+0000 7f13b8eba700 1 -- 172.21.15.7:0/3293719032 wait complete. 2024-01-11T07:11:00.173 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:11:00.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:11:00.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.305+0000 7fa61d8c6700 1 Processor -- start 2024-01-11T07:11:00.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.306+0000 7fa61d8c6700 1 -- start start 2024-01-11T07:11:00.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.306+0000 7fa61d8c6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa618157d80 0x7fa618063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.306+0000 7fa61d8c6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa618063770 0x7fa618069c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.306+0000 7fa61d8c6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61815c660 con 0x7fa6181579b0 2024-01-11T07:11:00.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.306+0000 7fa61d8c6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa618063b90 con 0x7fa618063770 2024-01-11T07:11:00.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.306+0000 7fa61d8c6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6180aeab0 con 0x7fa618157d80 2024-01-11T07:11:00.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa6177fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa618063770 0x7fa618069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa616ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa6181579b0 0x7fa61815bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44444/0 (socket says 172.21.15.7:44444) 2024-01-11T07:11:00.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa616ffd700 1 -- 172.21.15.7:0/4079038570 learned_addr learned my addr 172.21.15.7:0/4079038570 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa6167fc700 1 --2- 172.21.15.7:0/4079038570 >> v2:172.21.15.113:3300/0 conn(0x7fa618157d80 0x7fa618063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa6177fe700 1 -- 172.21.15.7:0/4079038570 >> v2:172.21.15.113:3300/0 conn(0x7fa618157d80 msgr2=0x7fa618063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa6177fe700 1 --2- 172.21.15.7:0/4079038570 >> v2:172.21.15.113:3300/0 conn(0x7fa618157d80 0x7fa618063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa6177fe700 1 -- 172.21.15.7:0/4079038570 >> v1:172.21.15.7:6789/0 conn(0x7fa6181579b0 legacy=0x7fa61815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa615ffb700 1 -- 172.21.15.7:0/4079038570 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557553952 0 0) 0x7fa61815c660 con 0x7fa6181579b0 2024-01-11T07:11:00.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa6177fe700 1 -- 172.21.15.7:0/4079038570 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6180aed90 con 0x7fa618063770 2024-01-11T07:11:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa6177fe700 1 --2- 172.21.15.7:0/4079038570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618069c40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa6180cd830 tx=0x7fa610009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ddf0e8267008b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa615ffb700 1 -- 172.21.15.7:0/4079038570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61000e070 con 0x7fa618063770 2024-01-11T07:11:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa615ffb700 1 -- 172.21.15.7:0/4079038570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61000bb60 con 0x7fa618063770 2024-01-11T07:11:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.307+0000 7fa615ffb700 1 -- 172.21.15.7:0/4079038570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610012430 con 0x7fa618063770 2024-01-11T07:11:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.308+0000 7fa61d8c6700 1 -- 172.21.15.7:0/4079038570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 msgr2=0x7fa618069c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.308+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/4079038570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618069c40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa6180cd830 tx=0x7fa610009780 comp rx=0 tx=0).stop 2024-01-11T07:11:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.308+0000 7fa61d8c6700 1 -- 172.21.15.7:0/4079038570 shutdown_connections 2024-01-11T07:11:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.308+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/4079038570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618069c40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.308+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/4079038570 >> v2:172.21.15.113:3300/0 conn(0x7fa618157d80 0x7fa618063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.308+0000 7fa61d8c6700 1 -- 172.21.15.7:0/4079038570 >> 172.21.15.7:0/4079038570 conn(0x7fa6180680e0 msgr2=0x7fa6180cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.308+0000 7fa61d8c6700 1 -- 172.21.15.7:0/4079038570 shutdown_connections 2024-01-11T07:11:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.308+0000 7fa61d8c6700 1 -- 172.21.15.7:0/4079038570 wait complete. 2024-01-11T07:11:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.309+0000 7fa61d8c6700 1 Processor -- start 2024-01-11T07:11:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa61d8c6700 1 -- start start 2024-01-11T07:11:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa61d8c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618169580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa61d8c6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6181579b0 0x7fa6180c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa61d8c6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa618157d80 0x7fa61806b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa61d8c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6180c88f0 con 0x7fa618063770 2024-01-11T07:11:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa61d8c6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6180c8bd0 con 0x7fa618157d80 2024-01-11T07:11:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa61d8c6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6180c8eb0 con 0x7fa6181579b0 2024-01-11T07:11:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa616ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618169580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa616ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618169580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33064/0 (socket says 172.21.15.7:33064) 2024-01-11T07:11:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.310+0000 7fa616ffd700 1 -- 172.21.15.7:0/3428968243 learned_addr learned my addr 172.21.15.7:0/3428968243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa6177fe700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa618157d80 0x7fa61806b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa616ffd700 1 -- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6181579b0 msgr2=0x7fa6180c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa616ffd700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6181579b0 0x7fa6180c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa616ffd700 1 -- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa618157d80 msgr2=0x7fa61806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa616ffd700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa618157d80 0x7fa61806b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa616ffd700 1 -- 172.21.15.7:0/3428968243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610009a20 con 0x7fa618063770 2024-01-11T07:11:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa616ffd700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618169580 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa60800c5b0 tx=0x7fa608001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa5feffd700 1 -- 172.21.15.7:0/3428968243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608005b30 con 0x7fa618063770 2024-01-11T07:11:00.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa5feffd700 1 -- 172.21.15.7:0/3428968243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa608005c90 con 0x7fa618063770 2024-01-11T07:11:00.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6180c9190 con 0x7fa618063770 2024-01-11T07:11:00.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa5feffd700 1 -- 172.21.15.7:0/3428968243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608007320 con 0x7fa618063770 2024-01-11T07:11:00.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.311+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa618169fa0 con 0x7fa618063770 2024-01-11T07:11:00.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.312+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61804c3c0 con 0x7fa618063770 2024-01-11T07:11:00.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.313+0000 7fa5feffd700 1 -- 172.21.15.7:0/3428968243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa60801b3b0 con 0x7fa618063770 2024-01-11T07:11:00.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.313+0000 7fa5feffd700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa604066f30 0x7fa6040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.313+0000 7fa5feffd700 1 -- 172.21.15.7:0/3428968243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa608083a10 con 0x7fa618063770 2024-01-11T07:11:00.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.313+0000 7fa6167fc700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa604066f30 0x7fa6040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.314+0000 7fa6167fc700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa604066f30 0x7fa6040693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa618156f40 tx=0x7fa60c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:00.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.317+0000 7fa5feffd700 1 -- 172.21.15.7:0/3428968243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa608050c80 con 0x7fa618063770 2024-01-11T07:11:00.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.518+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6180af070 con 0x7fa604066f30 2024-01-11T07:11:00.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.527+0000 7fa5feffd700 1 -- 172.21.15.7:0/3428968243 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285234 (secure 0 0 0) 0x7fa6180af070 con 0x7fa604066f30 2024-01-11T07:11:00.531 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa604066f30 msgr2=0x7fa6040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa604066f30 0x7fa6040693e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa618156f40 tx=0x7fa60c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 msgr2=0x7fa618169580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618169580 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa60800c5b0 tx=0x7fa608001070 comp rx=0 tx=0).stop 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 shutdown_connections 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa604066f30 0x7fa6040693e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa618157d80 0x7fa61806b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa618063770 0x7fa618169580 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 --2- 172.21.15.7:0/3428968243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa6181579b0 0x7fa6180c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 >> 172.21.15.7:0/3428968243 conn(0x7fa6180680e0 msgr2=0x7fa61806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:00.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.530+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 shutdown_connections 2024-01-11T07:11:00.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.531+0000 7fa61d8c6700 1 -- 172.21.15.7:0/3428968243 wait complete. 2024-01-11T07:11:00.534 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:11:00.544 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-01-11T07:10:59.896425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557874,"statfs":{"total":1600000000,"available":1595263852,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.566351"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.368152+0000","last_change":"2024-01-11T07:10:56.368054+0000","last_active":"2024-01-11T07:10:56.368152+0000","last_peered":"2024-01-11T07:10:56.368152+0000","last_clean":"2024-01-11T07:10:56.368152+0000","last_became_active":"2024-01-11T07:10:56.367677+0000","last_became_peered":"2024-01-11T07:10:56.367677+0000","last_unstale":"2024-01-11T07:10:56.368152+0000","last_undegraded":"2024-01-11T07:10:56.368152+0000","last_fullsized":"2024-01-11T07:10:56.368152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:11:23.743216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365273+0000","last_change":"2024-01-11T07:10:56.362832+0000","last_active":"2024-01-11T07:10:56.365273+0000","last_peered":"2024-01-11T07:10:56.365273+0000","last_clean":"2024-01-11T07:10:56.365273+0000","last_became_active":"2024-01-11T07:10:56.362245+0000","last_became_peered":"2024-01-11T07:10:56.362245+0000","last_unstale":"2024-01-11T07:10:56.365273+0000","last_undegraded":"2024-01-11T07:10:56.365273+0000","last_fullsized":"2024-01-11T07:10:56.365273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:26:38.870456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357646+0000","last_change":"2024-01-11T07:10:56.356999+0000","last_active":"2024-01-11T07:10:56.357646+0000","last_peered":"2024-01-11T07:10:56.357646+0000","last_clean":"2024-01-11T07:10:56.357646+0000","last_became_active":"2024-01-11T07:10:56.356226+0000","last_became_peered":"2024-01-11T07:10:56.356226+0000","last_unstale":"2024-01-11T07:10:56.357646+0000","last_undegraded":"2024-01-11T07:10:56.357646+0000","last_fullsized":"2024-01-11T07:10:56.357646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:08:56.391329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369089+0000","last_change":"2024-01-11T07:10:56.369009+0000","last_active":"2024-01-11T07:10:56.369089+0000","last_peered":"2024-01-11T07:10:56.369089+0000","last_clean":"2024-01-11T07:10:56.369089+0000","last_became_active":"2024-01-11T07:10:56.368603+0000","last_became_peered":"2024-01-11T07:10:56.368603+0000","last_unstale":"2024-01-11T07:10:56.369089+0000","last_undegraded":"2024-01-11T07:10:56.369089+0000","last_fullsized":"2024-01-11T07:10:56.369089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:18.567247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365911+0000","last_change":"2024-01-11T07:10:56.365825+0000","last_active":"2024-01-11T07:10:56.365911+0000","last_peered":"2024-01-11T07:10:56.365911+0000","last_clean":"2024-01-11T07:10:56.365911+0000","last_became_active":"2024-01-11T07:10:56.365612+0000","last_became_peered":"2024-01-11T07:10:56.365612+0000","last_unstale":"2024-01-11T07:10:56.365911+0000","last_undegraded":"2024-01-11T07:10:56.365911+0000","last_fullsized":"2024-01-11T07:10:56.365911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:23:02.941606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369454+0000","last_change":"2024-01-11T07:10:56.369379+0000","last_active":"2024-01-11T07:10:56.369454+0000","last_peered":"2024-01-11T07:10:56.369454+0000","last_clean":"2024-01-11T07:10:56.369454+0000","last_became_active":"2024-01-11T07:10:56.369146+0000","last_became_peered":"2024-01-11T07:10:56.369146+0000","last_unstale":"2024-01-11T07:10:56.369454+0000","last_undegraded":"2024-01-11T07:10:56.369454+0000","last_fullsized":"2024-01-11T07:10:56.369454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:00:19.372481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.361151+0000","last_change":"2024-01-11T07:10:56.360908+0000","last_active":"2024-01-11T07:10:56.361151+0000","last_peered":"2024-01-11T07:10:56.361151+0000","last_clean":"2024-01-11T07:10:56.361151+0000","last_became_active":"2024-01-11T07:10:56.360334+0000","last_became_peered":"2024-01-11T07:10:56.360334+0000","last_unstale":"2024-01-11T07:10:56.361151+0000","last_undegraded":"2024-01-11T07:10:56.361151+0000","last_fullsized":"2024-01-11T07:10:56.361151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:33.873432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365350+0000","last_change":"2024-01-11T07:10:56.364324+0000","last_active":"2024-01-11T07:10:56.365350+0000","last_peered":"2024-01-11T07:10:56.365350+0000","last_clean":"2024-01-11T07:10:56.365350+0000","last_became_active":"2024-01-11T07:10:56.364024+0000","last_became_peered":"2024-01-11T07:10:56.364024+0000","last_unstale":"2024-01-11T07:10:56.365350+0000","last_undegraded":"2024-01-11T07:10:56.365350+0000","last_fullsized":"2024-01-11T07:10:56.365350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:09.106227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363315+0000","last_change":"2024-01-11T07:10:56.363227+0000","last_active":"2024-01-11T07:10:56.363315+0000","last_peered":"2024-01-11T07:10:56.363315+0000","last_clean":"2024-01-11T07:10:56.363315+0000","last_became_active":"2024-01-11T07:10:56.362962+0000","last_became_peered":"2024-01-11T07:10:56.362962+0000","last_unstale":"2024-01-11T07:10:56.363315+0000","last_undegraded":"2024-01-11T07:10:56.363315+0000","last_fullsized":"2024-01-11T07:10:56.363315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:06.164432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364549+0000","last_change":"2024-01-11T07:10:56.364452+0000","last_active":"2024-01-11T07:10:56.364549+0000","last_peered":"2024-01-11T07:10:56.364549+0000","last_clean":"2024-01-11T07:10:56.364549+0000","last_became_active":"2024-01-11T07:10:56.364045+0000","last_became_peered":"2024-01-11T07:10:56.364045+0000","last_unstale":"2024-01-11T07:10:56.364549+0000","last_undegraded":"2024-01-11T07:10:56.364549+0000","last_fullsized":"2024-01-11T07:10:56.364549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:02.816098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.356560+0000","last_change":"2024-01-11T07:10:56.356330+0000","last_active":"2024-01-11T07:10:56.356560+0000","last_peered":"2024-01-11T07:10:56.356560+0000","last_clean":"2024-01-11T07:10:56.356560+0000","last_became_active":"2024-01-11T07:10:56.356095+0000","last_became_peered":"2024-01-11T07:10:56.356095+0000","last_unstale":"2024-01-11T07:10:56.356560+0000","last_undegraded":"2024-01-11T07:10:56.356560+0000","last_fullsized":"2024-01-11T07:10:56.356560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:37.600667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364817+0000","last_change":"2024-01-11T07:10:56.364706+0000","last_active":"2024-01-11T07:10:56.364817+0000","last_peered":"2024-01-11T07:10:56.364817+0000","last_clean":"2024-01-11T07:10:56.364817+0000","last_became_active":"2024-01-11T07:10:56.364117+0000","last_became_peered":"2024-01-11T07:10:56.364117+0000","last_unstale":"2024-01-11T07:10:56.364817+0000","last_undegraded":"2024-01-11T07:10:56.364817+0000","last_fullsized":"2024-01-11T07:10:56.364817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:41.703753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367450+0000","last_change":"2024-01-11T07:10:56.367382+0000","last_active":"2024-01-11T07:10:56.367450+0000","last_peered":"2024-01-11T07:10:56.367450+0000","last_clean":"2024-01-11T07:10:56.367450+0000","last_became_active":"2024-01-11T07:10:56.366924+0000","last_became_peered":"2024-01-11T07:10:56.366924+0000","last_unstale":"2024-01-11T07:10:56.367450+0000","last_undegraded":"2024-01-11T07:10:56.367450+0000","last_fullsized":"2024-01-11T07:10:56.367450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:37.600667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363054+0000","last_change":"2024-01-11T07:10:56.361408+0000","last_active":"2024-01-11T07:10:56.363054+0000","last_peered":"2024-01-11T07:10:56.363054+0000","last_clean":"2024-01-11T07:10:56.363054+0000","last_became_active":"2024-01-11T07:10:56.361150+0000","last_became_peered":"2024-01-11T07:10:56.361150+0000","last_unstale":"2024-01-11T07:10:56.363054+0000","last_undegraded":"2024-01-11T07:10:56.363054+0000","last_fullsized":"2024-01-11T07:10:56.363054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:18:00.531797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.366276+0000","last_change":"2024-01-11T07:10:56.366186+0000","last_active":"2024-01-11T07:10:56.366276+0000","last_peered":"2024-01-11T07:10:56.366276+0000","last_clean":"2024-01-11T07:10:56.366276+0000","last_became_active":"2024-01-11T07:10:56.365938+0000","last_became_peered":"2024-01-11T07:10:56.365938+0000","last_unstale":"2024-01-11T07:10:56.366276+0000","last_undegraded":"2024-01-11T07:10:56.366276+0000","last_fullsized":"2024-01-11T07:10:56.366276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:24:42.859593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369729+0000","last_change":"2024-01-11T07:10:56.369635+0000","last_active":"2024-01-11T07:10:56.369729+0000","last_peered":"2024-01-11T07:10:56.369729+0000","last_clean":"2024-01-11T07:10:56.369729+0000","last_became_active":"2024-01-11T07:10:56.369407+0000","last_became_peered":"2024-01-11T07:10:56.369407+0000","last_unstale":"2024-01-11T07:10:56.369729+0000","last_undegraded":"2024-01-11T07:10:56.369729+0000","last_fullsized":"2024-01-11T07:10:56.369729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:42:24.630686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.366476+0000","last_change":"2024-01-11T07:10:56.366289+0000","last_active":"2024-01-11T07:10:56.366476+0000","last_peered":"2024-01-11T07:10:56.366476+0000","last_clean":"2024-01-11T07:10:56.366476+0000","last_became_active":"2024-01-11T07:10:56.364958+0000","last_became_peered":"2024-01-11T07:10:56.364958+0000","last_unstale":"2024-01-11T07:10:56.366476+0000","last_undegraded":"2024-01-11T07:10:56.366476+0000","last_fullsized":"2024-01-11T07:10:56.366476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:27:06.758736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.362558+0000","last_change":"2024-01-11T07:10:56.362479+0000","last_active":"2024-01-11T07:10:56.362558+0000","last_peered":"2024-01-11T07:10:56.362558+0000","last_clean":"2024-01-11T07:10:56.362558+0000","last_became_active":"2024-01-11T07:10:56.362175+0000","last_became_peered":"2024-01-11T07:10:56.362175+0000","last_unstale":"2024-01-11T07:10:56.362558+0000","last_undegraded":"2024-01-11T07:10:56.362558+0000","last_fullsized":"2024-01-11T07:10:56.362558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:22:28.764740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.368534+0000","last_change":"2024-01-11T07:10:56.368474+0000","last_active":"2024-01-11T07:10:56.368534+0000","last_peered":"2024-01-11T07:10:56.368534+0000","last_clean":"2024-01-11T07:10:56.368534+0000","last_became_active":"2024-01-11T07:10:56.368080+0000","last_became_peered":"2024-01-11T07:10:56.368080+0000","last_unstale":"2024-01-11T07:10:56.368534+0000","last_undegraded":"2024-01-11T07:10:56.368534+0000","last_fullsized":"2024-01-11T07:10:56.368534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:57:53.821988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358875+0000","last_change":"2024-01-11T07:10:56.358786+0000","last_active":"2024-01-11T07:10:56.358875+0000","last_peered":"2024-01-11T07:10:56.358875+0000","last_clean":"2024-01-11T07:10:56.358875+0000","last_became_active":"2024-01-11T07:10:56.358509+0000","last_became_peered":"2024-01-11T07:10:56.358509+0000","last_unstale":"2024-01-11T07:10:56.358875+0000","last_undegraded":"2024-01-11T07:10:56.358875+0000","last_fullsized":"2024-01-11T07:10:56.358875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:43:09.466609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.354667+0000","last_change":"2024-01-11T07:10:56.354518+0000","last_active":"2024-01-11T07:10:56.354667+0000","last_peered":"2024-01-11T07:10:56.354667+0000","last_clean":"2024-01-11T07:10:56.354667+0000","last_became_active":"2024-01-11T07:10:56.354312+0000","last_became_peered":"2024-01-11T07:10:56.354312+0000","last_unstale":"2024-01-11T07:10:56.354667+0000","last_undegraded":"2024-01-11T07:10:56.354667+0000","last_fullsized":"2024-01-11T07:10:56.354667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:20:51.793778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364517+0000","last_change":"2024-01-11T07:10:56.364171+0000","last_active":"2024-01-11T07:10:56.364517+0000","last_peered":"2024-01-11T07:10:56.364517+0000","last_clean":"2024-01-11T07:10:56.364517+0000","last_became_active":"2024-01-11T07:10:56.363918+0000","last_became_peered":"2024-01-11T07:10:56.363918+0000","last_unstale":"2024-01-11T07:10:56.364517+0000","last_undegraded":"2024-01-11T07:10:56.364517+0000","last_fullsized":"2024-01-11T07:10:56.364517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:54:19.083774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358425+0000","last_change":"2024-01-11T07:10:56.358340+0000","last_active":"2024-01-11T07:10:56.358425+0000","last_peered":"2024-01-11T07:10:56.358425+0000","last_clean":"2024-01-11T07:10:56.358425+0000","last_became_active":"2024-01-11T07:10:56.358100+0000","last_became_peered":"2024-01-11T07:10:56.358100+0000","last_unstale":"2024-01-11T07:10:56.358425+0000","last_undegraded":"2024-01-11T07:10:56.358425+0000","last_fullsized":"2024-01-11T07:10:56.358425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:24:42.859593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369405+0000","last_change":"2024-01-11T07:10:56.369332+0000","last_active":"2024-01-11T07:10:56.369405+0000","last_peered":"2024-01-11T07:10:56.369405+0000","last_clean":"2024-01-11T07:10:56.369405+0000","last_became_active":"2024-01-11T07:10:56.368944+0000","last_became_peered":"2024-01-11T07:10:56.368944+0000","last_unstale":"2024-01-11T07:10:56.369405+0000","last_undegraded":"2024-01-11T07:10:56.369405+0000","last_fullsized":"2024-01-11T07:10:56.369405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:01:11.073664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.355359+0000","last_change":"2024-01-11T07:10:56.355284+0000","last_active":"2024-01-11T07:10:56.355359+0000","last_peered":"2024-01-11T07:10:56.355359+0000","last_clean":"2024-01-11T07:10:56.355359+0000","last_became_active":"2024-01-11T07:10:56.355050+0000","last_became_peered":"2024-01-11T07:10:56.355050+0000","last_unstale":"2024-01-11T07:10:56.355359+0000","last_undegraded":"2024-01-11T07:10:56.355359+0000","last_fullsized":"2024-01-11T07:10:56.355359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:14:36.368636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367059+0000","last_change":"2024-01-11T07:10:56.366917+0000","last_active":"2024-01-11T07:10:56.367059+0000","last_peered":"2024-01-11T07:10:56.367059+0000","last_clean":"2024-01-11T07:10:56.367059+0000","last_became_active":"2024-01-11T07:10:56.366697+0000","last_became_peered":"2024-01-11T07:10:56.366697+0000","last_unstale":"2024-01-11T07:10:56.367059+0000","last_undegraded":"2024-01-11T07:10:56.367059+0000","last_fullsized":"2024-01-11T07:10:56.367059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:26:45.713038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.356605+0000","last_change":"2024-01-11T07:10:56.356377+0000","last_active":"2024-01-11T07:10:56.356605+0000","last_peered":"2024-01-11T07:10:56.356605+0000","last_clean":"2024-01-11T07:10:56.356605+0000","last_became_active":"2024-01-11T07:10:56.356098+0000","last_became_peered":"2024-01-11T07:10:56.356098+0000","last_unstale":"2024-01-11T07:10:56.356605+0000","last_undegraded":"2024-01-11T07:10:56.356605+0000","last_fullsized":"2024-01-11T07:10:56.356605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:51.984688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365069+0000","last_change":"2024-01-11T07:10:56.364909+0000","last_active":"2024-01-11T07:10:56.365069+0000","last_peered":"2024-01-11T07:10:56.365069+0000","last_clean":"2024-01-11T07:10:56.365069+0000","last_became_active":"2024-01-11T07:10:56.364750+0000","last_became_peered":"2024-01-11T07:10:56.364750+0000","last_unstale":"2024-01-11T07:10:56.365069+0000","last_undegraded":"2024-01-11T07:10:56.365069+0000","last_fullsized":"2024-01-11T07:10:56.365069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:30:01.409443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358605+0000","last_change":"2024-01-11T07:10:56.358531+0000","last_active":"2024-01-11T07:10:56.358605+0000","last_peered":"2024-01-11T07:10:56.358605+0000","last_clean":"2024-01-11T07:10:56.358605+0000","last_became_active":"2024-01-11T07:10:56.358085+0000","last_became_peered":"2024-01-11T07:10:56.358085+0000","last_unstale":"2024-01-11T07:10:56.358605+0000","last_undegraded":"2024-01-11T07:10:56.358605+0000","last_fullsized":"2024-01-11T07:10:56.358605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:24:15.740179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.360443+0000","last_change":"2024-01-11T07:10:56.360318+0000","last_active":"2024-01-11T07:10:56.360443+0000","last_peered":"2024-01-11T07:10:56.360443+0000","last_clean":"2024-01-11T07:10:56.360443+0000","last_became_active":"2024-01-11T07:10:56.359859+0000","last_became_peered":"2024-01-11T07:10:56.359859+0000","last_unstale":"2024-01-11T07:10:56.360443+0000","last_undegraded":"2024-01-11T07:10:56.360443+0000","last_fullsized":"2024-01-11T07:10:56.360443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:42:24.630686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364165+0000","last_change":"2024-01-11T07:10:56.364089+0000","last_active":"2024-01-11T07:10:56.364165+0000","last_peered":"2024-01-11T07:10:56.364165+0000","last_clean":"2024-01-11T07:10:56.364165+0000","last_became_active":"2024-01-11T07:10:56.363792+0000","last_became_peered":"2024-01-11T07:10:56.363792+0000","last_unstale":"2024-01-11T07:10:56.364165+0000","last_undegraded":"2024-01-11T07:10:56.364165+0000","last_fullsized":"2024-01-11T07:10:56.364165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:04:49.450635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365298+0000","last_change":"2024-01-11T07:10:56.363009+0000","last_active":"2024-01-11T07:10:56.365298+0000","last_peered":"2024-01-11T07:10:56.365298+0000","last_clean":"2024-01-11T07:10:56.365298+0000","last_became_active":"2024-01-11T07:10:56.362589+0000","last_became_peered":"2024-01-11T07:10:56.362589+0000","last_unstale":"2024-01-11T07:10:56.365298+0000","last_undegraded":"2024-01-11T07:10:56.365298+0000","last_fullsized":"2024-01-11T07:10:56.365298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:41.703753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369390+0000","last_change":"2024-01-11T07:10:56.369183+0000","last_active":"2024-01-11T07:10:56.369390+0000","last_peered":"2024-01-11T07:10:56.369390+0000","last_clean":"2024-01-11T07:10:56.369390+0000","last_became_active":"2024-01-11T07:10:56.368582+0000","last_became_peered":"2024-01-11T07:10:56.368582+0000","last_unstale":"2024-01-11T07:10:56.369390+0000","last_undegraded":"2024-01-11T07:10:56.369390+0000","last_fullsized":"2024-01-11T07:10:56.369390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:43:29.078899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"83'4","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.374984+0000","last_change":"2024-01-11T07:10:55.342510+0000","last_active":"2024-01-11T07:10:56.374984+0000","last_peered":"2024-01-11T07:10:56.374984+0000","last_clean":"2024-01-11T07:10:56.374984+0000","last_became_active":"2024-01-11T07:10:55.342203+0000","last_became_peered":"2024-01-11T07:10:55.342203+0000","last_unstale":"2024-01-11T07:10:56.374984+0000","last_undegraded":"2024-01-11T07:10:56.374984+0000","last_fullsized":"2024-01-11T07:10:56.374984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:52:05.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363980+0000","last_change":"2024-01-11T07:10:56.363845+0000","last_active":"2024-01-11T07:10:56.363980+0000","last_peered":"2024-01-11T07:10:56.363980+0000","last_clean":"2024-01-11T07:10:56.363980+0000","last_became_active":"2024-01-11T07:10:56.363334+0000","last_became_peered":"2024-01-11T07:10:56.363334+0000","last_unstale":"2024-01-11T07:10:56.363980+0000","last_undegraded":"2024-01-11T07:10:56.363980+0000","last_fullsized":"2024-01-11T07:10:56.363980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:08:56.485413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.373572+0000","last_change":"2024-01-11T07:10:55.346554+0000","last_active":"2024-01-11T07:10:56.373572+0000","last_peered":"2024-01-11T07:10:56.373572+0000","last_clean":"2024-01-11T07:10:56.373572+0000","last_became_active":"2024-01-11T07:10:55.346238+0000","last_became_peered":"2024-01-11T07:10:55.346238+0000","last_unstale":"2024-01-11T07:10:56.373572+0000","last_undegraded":"2024-01-11T07:10:56.373572+0000","last_fullsized":"2024-01-11T07:10:56.373572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:12.795675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365367+0000","last_change":"2024-01-11T07:10:56.364554+0000","last_active":"2024-01-11T07:10:56.365367+0000","last_peered":"2024-01-11T07:10:56.365367+0000","last_clean":"2024-01-11T07:10:56.365367+0000","last_became_active":"2024-01-11T07:10:56.364243+0000","last_became_peered":"2024-01-11T07:10:56.364243+0000","last_unstale":"2024-01-11T07:10:56.365367+0000","last_undegraded":"2024-01-11T07:10:56.365367+0000","last_fullsized":"2024-01-11T07:10:56.365367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:37.718222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376095+0000","last_change":"2024-01-11T07:10:55.347034+0000","last_active":"2024-01-11T07:10:56.376095+0000","last_peered":"2024-01-11T07:10:56.376095+0000","last_clean":"2024-01-11T07:10:56.376095+0000","last_became_active":"2024-01-11T07:10:55.346667+0000","last_became_peered":"2024-01-11T07:10:55.346667+0000","last_unstale":"2024-01-11T07:10:56.376095+0000","last_undegraded":"2024-01-11T07:10:56.376095+0000","last_fullsized":"2024-01-11T07:10:56.376095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:40.698923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.354382+0000","last_change":"2024-01-11T07:10:56.354265+0000","last_active":"2024-01-11T07:10:56.354382+0000","last_peered":"2024-01-11T07:10:56.354382+0000","last_clean":"2024-01-11T07:10:56.354382+0000","last_became_active":"2024-01-11T07:10:56.353963+0000","last_became_peered":"2024-01-11T07:10:56.353963+0000","last_unstale":"2024-01-11T07:10:56.354382+0000","last_undegraded":"2024-01-11T07:10:56.354382+0000","last_fullsized":"2024-01-11T07:10:56.354382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:34:38.048691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.375155+0000","last_change":"2024-01-11T07:10:55.341202+0000","last_active":"2024-01-11T07:10:56.375155+0000","last_peered":"2024-01-11T07:10:56.375155+0000","last_clean":"2024-01-11T07:10:56.375155+0000","last_became_active":"2024-01-11T07:10:55.340944+0000","last_became_peered":"2024-01-11T07:10:55.340944+0000","last_unstale":"2024-01-11T07:10:56.375155+0000","last_undegraded":"2024-01-11T07:10:56.375155+0000","last_fullsized":"2024-01-11T07:10:56.375155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:41:37.220415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.359517+0000","last_change":"2024-01-11T07:10:56.359436+0000","last_active":"2024-01-11T07:10:56.359517+0000","last_peered":"2024-01-11T07:10:56.359517+0000","last_clean":"2024-01-11T07:10:56.359517+0000","last_became_active":"2024-01-11T07:10:56.359026+0000","last_became_peered":"2024-01-11T07:10:56.359026+0000","last_unstale":"2024-01-11T07:10:56.359517+0000","last_undegraded":"2024-01-11T07:10:56.359517+0000","last_fullsized":"2024-01-11T07:10:56.359517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:02.816098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376102+0000","last_change":"2024-01-11T07:10:55.344983+0000","last_active":"2024-01-11T07:10:56.376102+0000","last_peered":"2024-01-11T07:10:56.376102+0000","last_clean":"2024-01-11T07:10:56.376102+0000","last_became_active":"2024-01-11T07:10:55.344711+0000","last_became_peered":"2024-01-11T07:10:55.344711+0000","last_unstale":"2024-01-11T07:10:56.376102+0000","last_undegraded":"2024-01-11T07:10:56.376102+0000","last_fullsized":"2024-01-11T07:10:56.376102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:18:50.821824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.354637+0000","last_change":"2024-01-11T07:10:56.354014+0000","last_active":"2024-01-11T07:10:56.354637+0000","last_peered":"2024-01-11T07:10:56.354637+0000","last_clean":"2024-01-11T07:10:56.354637+0000","last_became_active":"2024-01-11T07:10:56.353776+0000","last_became_peered":"2024-01-11T07:10:56.353776+0000","last_unstale":"2024-01-11T07:10:56.354637+0000","last_undegraded":"2024-01-11T07:10:56.354637+0000","last_fullsized":"2024-01-11T07:10:56.354637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:52:06.446086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.375410+0000","last_change":"2024-01-11T07:10:55.339454+0000","last_active":"2024-01-11T07:10:56.375410+0000","last_peered":"2024-01-11T07:10:56.375410+0000","last_clean":"2024-01-11T07:10:56.375410+0000","last_became_active":"2024-01-11T07:10:55.339218+0000","last_became_peered":"2024-01-11T07:10:55.339218+0000","last_unstale":"2024-01-11T07:10:56.375410+0000","last_undegraded":"2024-01-11T07:10:56.375410+0000","last_fullsized":"2024-01-11T07:10:56.375410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:38.055778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363111+0000","last_change":"2024-01-11T07:10:56.362950+0000","last_active":"2024-01-11T07:10:56.363111+0000","last_peered":"2024-01-11T07:10:56.363111+0000","last_clean":"2024-01-11T07:10:56.363111+0000","last_became_active":"2024-01-11T07:10:56.362603+0000","last_became_peered":"2024-01-11T07:10:56.362603+0000","last_unstale":"2024-01-11T07:10:56.363111+0000","last_undegraded":"2024-01-11T07:10:56.363111+0000","last_fullsized":"2024-01-11T07:10:56.363111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:40:34.384474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.355624+0000","last_change":"2024-01-11T07:10:55.345176+0000","last_active":"2024-01-11T07:10:56.355624+0000","last_peered":"2024-01-11T07:10:56.355624+0000","last_clean":"2024-01-11T07:10:56.355624+0000","last_became_active":"2024-01-11T07:10:55.344956+0000","last_became_peered":"2024-01-11T07:10:55.344956+0000","last_unstale":"2024-01-11T07:10:56.355624+0000","last_undegraded":"2024-01-11T07:10:56.355624+0000","last_fullsized":"2024-01-11T07:10:56.355624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:41:37.220415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.355135+0000","last_change":"2024-01-11T07:10:56.355015+0000","last_active":"2024-01-11T07:10:56.355135+0000","last_peered":"2024-01-11T07:10:56.355135+0000","last_clean":"2024-01-11T07:10:56.355135+0000","last_became_active":"2024-01-11T07:10:56.354767+0000","last_became_peered":"2024-01-11T07:10:56.354767+0000","last_unstale":"2024-01-11T07:10:56.355135+0000","last_undegraded":"2024-01-11T07:10:56.355135+0000","last_fullsized":"2024-01-11T07:10:56.355135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:53:30.491513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.362099+0000","last_change":"2024-01-11T07:10:55.343271+0000","last_active":"2024-01-11T07:10:56.362099+0000","last_peered":"2024-01-11T07:10:56.362099+0000","last_clean":"2024-01-11T07:10:56.362099+0000","last_became_active":"2024-01-11T07:10:55.342106+0000","last_became_peered":"2024-01-11T07:10:55.342106+0000","last_unstale":"2024-01-11T07:10:56.362099+0000","last_undegraded":"2024-01-11T07:10:56.362099+0000","last_fullsized":"2024-01-11T07:10:56.362099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:44:52.736040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.360926+0000","last_change":"2024-01-11T07:10:56.360588+0000","last_active":"2024-01-11T07:10:56.360926+0000","last_peered":"2024-01-11T07:10:56.360926+0000","last_clean":"2024-01-11T07:10:56.360926+0000","last_became_active":"2024-01-11T07:10:56.359687+0000","last_became_peered":"2024-01-11T07:10:56.359687+0000","last_unstale":"2024-01-11T07:10:56.360926+0000","last_undegraded":"2024-01-11T07:10:56.360926+0000","last_fullsized":"2024-01-11T07:10:56.360926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:14:36.368636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.352321+0000","last_change":"2024-01-11T07:10:55.344254+0000","last_active":"2024-01-11T07:10:56.352321+0000","last_peered":"2024-01-11T07:10:56.352321+0000","last_clean":"2024-01-11T07:10:56.352321+0000","last_became_active":"2024-01-11T07:10:55.343997+0000","last_became_peered":"2024-01-11T07:10:55.343997+0000","last_unstale":"2024-01-11T07:10:56.352321+0000","last_undegraded":"2024-01-11T07:10:56.352321+0000","last_fullsized":"2024-01-11T07:10:56.352321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:17.562418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367062+0000","last_change":"2024-01-11T07:10:56.366924+0000","last_active":"2024-01-11T07:10:56.367062+0000","last_peered":"2024-01-11T07:10:56.367062+0000","last_clean":"2024-01-11T07:10:56.367062+0000","last_became_active":"2024-01-11T07:10:56.366577+0000","last_became_peered":"2024-01-11T07:10:56.366577+0000","last_unstale":"2024-01-11T07:10:56.367062+0000","last_undegraded":"2024-01-11T07:10:56.367062+0000","last_fullsized":"2024-01-11T07:10:56.367062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:11:41.353277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357744+0000","last_change":"2024-01-11T07:10:55.343311+0000","last_active":"2024-01-11T07:10:56.357744+0000","last_peered":"2024-01-11T07:10:56.357744+0000","last_clean":"2024-01-11T07:10:56.357744+0000","last_became_active":"2024-01-11T07:10:55.343063+0000","last_became_peered":"2024-01-11T07:10:55.343063+0000","last_unstale":"2024-01-11T07:10:56.357744+0000","last_undegraded":"2024-01-11T07:10:56.357744+0000","last_fullsized":"2024-01-11T07:10:56.357744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:40:33.379644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.368387+0000","last_change":"2024-01-11T07:10:56.368317+0000","last_active":"2024-01-11T07:10:56.368387+0000","last_peered":"2024-01-11T07:10:56.368387+0000","last_clean":"2024-01-11T07:10:56.368387+0000","last_became_active":"2024-01-11T07:10:56.367731+0000","last_became_peered":"2024-01-11T07:10:56.367731+0000","last_unstale":"2024-01-11T07:10:56.368387+0000","last_undegraded":"2024-01-11T07:10:56.368387+0000","last_fullsized":"2024-01-11T07:10:56.368387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:51.478825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"83'3","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.374248+0000","last_change":"2024-01-11T07:10:55.338940+0000","last_active":"2024-01-11T07:10:56.374248+0000","last_peered":"2024-01-11T07:10:56.374248+0000","last_clean":"2024-01-11T07:10:56.374248+0000","last_became_active":"2024-01-11T07:10:55.338684+0000","last_became_peered":"2024-01-11T07:10:55.338684+0000","last_unstale":"2024-01-11T07:10:56.374248+0000","last_undegraded":"2024-01-11T07:10:56.374248+0000","last_fullsized":"2024-01-11T07:10:56.374248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:11:40.348448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363962+0000","last_change":"2024-01-11T07:10:56.363585+0000","last_active":"2024-01-11T07:10:56.363962+0000","last_peered":"2024-01-11T07:10:56.363962+0000","last_clean":"2024-01-11T07:10:56.363962+0000","last_became_active":"2024-01-11T07:10:56.362959+0000","last_became_peered":"2024-01-11T07:10:56.362959+0000","last_unstale":"2024-01-11T07:10:56.363962+0000","last_undegraded":"2024-01-11T07:10:56.363962+0000","last_fullsized":"2024-01-11T07:10:56.363962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:35:48.029935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"83'6","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.379544+0000","last_change":"2024-01-11T07:10:55.343892+0000","last_active":"2024-01-11T07:10:56.379544+0000","last_peered":"2024-01-11T07:10:56.379544+0000","last_clean":"2024-01-11T07:10:56.379544+0000","last_became_active":"2024-01-11T07:10:55.343631+0000","last_became_peered":"2024-01-11T07:10:55.343631+0000","last_unstale":"2024-01-11T07:10:56.379544+0000","last_undegraded":"2024-01-11T07:10:56.379544+0000","last_fullsized":"2024-01-11T07:10:56.379544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:38:05.153636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367971+0000","last_change":"2024-01-11T07:10:56.367893+0000","last_active":"2024-01-11T07:10:56.367971+0000","last_peered":"2024-01-11T07:10:56.367971+0000","last_clean":"2024-01-11T07:10:56.367971+0000","last_became_active":"2024-01-11T07:10:56.367381+0000","last_became_peered":"2024-01-11T07:10:56.367381+0000","last_unstale":"2024-01-11T07:10:56.367971+0000","last_undegraded":"2024-01-11T07:10:56.367971+0000","last_fullsized":"2024-01-11T07:10:56.367971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:28:14.923144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.377556+0000","last_change":"2024-01-11T07:10:55.338629+0000","last_active":"2024-01-11T07:10:56.377556+0000","last_peered":"2024-01-11T07:10:56.377556+0000","last_clean":"2024-01-11T07:10:56.377556+0000","last_became_active":"2024-01-11T07:10:55.337969+0000","last_became_peered":"2024-01-11T07:10:55.337969+0000","last_unstale":"2024-01-11T07:10:56.377556+0000","last_undegraded":"2024-01-11T07:10:56.377556+0000","last_fullsized":"2024-01-11T07:10:56.377556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:33:24.246025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.362483+0000","last_change":"2024-01-11T07:10:56.362416+0000","last_active":"2024-01-11T07:10:56.362483+0000","last_peered":"2024-01-11T07:10:56.362483+0000","last_clean":"2024-01-11T07:10:56.362483+0000","last_became_active":"2024-01-11T07:10:56.362168+0000","last_became_peered":"2024-01-11T07:10:56.362168+0000","last_unstale":"2024-01-11T07:10:56.362483+0000","last_undegraded":"2024-01-11T07:10:56.362483+0000","last_fullsized":"2024-01-11T07:10:56.362483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:17:46.184174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363478+0000","last_change":"2024-01-11T07:10:55.343545+0000","last_active":"2024-01-11T07:10:56.363478+0000","last_peered":"2024-01-11T07:10:56.363478+0000","last_clean":"2024-01-11T07:10:56.363478+0000","last_became_active":"2024-01-11T07:10:55.342805+0000","last_became_peered":"2024-01-11T07:10:55.342805+0000","last_unstale":"2024-01-11T07:10:56.363478+0000","last_undegraded":"2024-01-11T07:10:56.363478+0000","last_fullsized":"2024-01-11T07:10:56.363478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:33:24.246025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.361139+0000","last_change":"2024-01-11T07:10:56.361064+0000","last_active":"2024-01-11T07:10:56.361139+0000","last_peered":"2024-01-11T07:10:56.361139+0000","last_clean":"2024-01-11T07:10:56.361139+0000","last_became_active":"2024-01-11T07:10:56.360785+0000","last_became_peered":"2024-01-11T07:10:56.360785+0000","last_unstale":"2024-01-11T07:10:56.361139+0000","last_undegraded":"2024-01-11T07:10:56.361139+0000","last_fullsized":"2024-01-11T07:10:56.361139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:29:19.620630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357120+0000","last_change":"2024-01-11T07:10:55.340085+0000","last_active":"2024-01-11T07:10:56.357120+0000","last_peered":"2024-01-11T07:10:56.357120+0000","last_clean":"2024-01-11T07:10:56.357120+0000","last_became_active":"2024-01-11T07:10:55.339180+0000","last_became_peered":"2024-01-11T07:10:55.339180+0000","last_unstale":"2024-01-11T07:10:56.357120+0000","last_undegraded":"2024-01-11T07:10:56.357120+0000","last_fullsized":"2024-01-11T07:10:56.357120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:36.713393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.366073+0000","last_change":"2024-01-11T07:10:56.366015+0000","last_active":"2024-01-11T07:10:56.366073+0000","last_peered":"2024-01-11T07:10:56.366073+0000","last_clean":"2024-01-11T07:10:56.366073+0000","last_became_active":"2024-01-11T07:10:56.364644+0000","last_became_peered":"2024-01-11T07:10:56.364644+0000","last_unstale":"2024-01-11T07:10:56.366073+0000","last_undegraded":"2024-01-11T07:10:56.366073+0000","last_fullsized":"2024-01-11T07:10:56.366073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:26:56.480518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"83'8","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376677+0000","last_change":"2024-01-11T07:10:55.340603+0000","last_active":"2024-01-11T07:10:56.376677+0000","last_peered":"2024-01-11T07:10:56.376677+0000","last_clean":"2024-01-11T07:10:56.376677+0000","last_became_active":"2024-01-11T07:10:55.340183+0000","last_became_peered":"2024-01-11T07:10:55.340183+0000","last_unstale":"2024-01-11T07:10:56.376677+0000","last_undegraded":"2024-01-11T07:10:56.376677+0000","last_fullsized":"2024-01-11T07:10:56.376677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:57:52.817159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.360752+0000","last_change":"2024-01-11T07:10:56.360170+0000","last_active":"2024-01-11T07:10:56.360752+0000","last_peered":"2024-01-11T07:10:56.360752+0000","last_clean":"2024-01-11T07:10:56.360752+0000","last_became_active":"2024-01-11T07:10:56.359934+0000","last_became_peered":"2024-01-11T07:10:56.359934+0000","last_unstale":"2024-01-11T07:10:56.360752+0000","last_undegraded":"2024-01-11T07:10:56.360752+0000","last_fullsized":"2024-01-11T07:10:56.360752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:13.800504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.375274+0000","last_change":"2024-01-11T07:10:55.341762+0000","last_active":"2024-01-11T07:10:56.375274+0000","last_peered":"2024-01-11T07:10:56.375274+0000","last_clean":"2024-01-11T07:10:56.375274+0000","last_became_active":"2024-01-11T07:10:55.341068+0000","last_became_peered":"2024-01-11T07:10:55.341068+0000","last_unstale":"2024-01-11T07:10:56.375274+0000","last_undegraded":"2024-01-11T07:10:56.375274+0000","last_fullsized":"2024-01-11T07:10:56.375274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:56:06.714495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357961+0000","last_change":"2024-01-11T07:10:56.357139+0000","last_active":"2024-01-11T07:10:56.357961+0000","last_peered":"2024-01-11T07:10:56.357961+0000","last_clean":"2024-01-11T07:10:56.357961+0000","last_became_active":"2024-01-11T07:10:56.356898+0000","last_became_peered":"2024-01-11T07:10:56.356898+0000","last_unstale":"2024-01-11T07:10:56.357961+0000","last_undegraded":"2024-01-11T07:10:56.357961+0000","last_fullsized":"2024-01-11T07:10:56.357961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:54:05.034702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376210+0000","last_change":"2024-01-11T07:10:55.346852+0000","last_active":"2024-01-11T07:10:56.376210+0000","last_peered":"2024-01-11T07:10:56.376210+0000","last_clean":"2024-01-11T07:10:56.376210+0000","last_became_active":"2024-01-11T07:10:55.346429+0000","last_became_peered":"2024-01-11T07:10:55.346429+0000","last_unstale":"2024-01-11T07:10:56.376210+0000","last_undegraded":"2024-01-11T07:10:56.376210+0000","last_fullsized":"2024-01-11T07:10:56.376210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:52:01.880653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364695+0000","last_change":"2024-01-11T07:10:56.364354+0000","last_active":"2024-01-11T07:10:56.364695+0000","last_peered":"2024-01-11T07:10:56.364695+0000","last_clean":"2024-01-11T07:10:56.364695+0000","last_became_active":"2024-01-11T07:10:56.363820+0000","last_became_peered":"2024-01-11T07:10:56.363820+0000","last_unstale":"2024-01-11T07:10:56.364695+0000","last_undegraded":"2024-01-11T07:10:56.364695+0000","last_fullsized":"2024-01-11T07:10:56.364695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:51:23.462342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376900+0000","last_change":"2024-01-11T07:10:55.345962+0000","last_active":"2024-01-11T07:10:56.376900+0000","last_peered":"2024-01-11T07:10:56.376900+0000","last_clean":"2024-01-11T07:10:56.376900+0000","last_became_active":"2024-01-11T07:10:55.345711+0000","last_became_peered":"2024-01-11T07:10:55.345711+0000","last_unstale":"2024-01-11T07:10:56.376900+0000","last_undegraded":"2024-01-11T07:10:56.376900+0000","last_fullsized":"2024-01-11T07:10:56.376900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:08.975763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363997+0000","last_change":"2024-01-11T07:10:56.363870+0000","last_active":"2024-01-11T07:10:56.363997+0000","last_peered":"2024-01-11T07:10:56.363997+0000","last_clean":"2024-01-11T07:10:56.363997+0000","last_became_active":"2024-01-11T07:10:56.363600+0000","last_became_peered":"2024-01-11T07:10:56.363600+0000","last_unstale":"2024-01-11T07:10:56.363997+0000","last_undegraded":"2024-01-11T07:10:56.363997+0000","last_fullsized":"2024-01-11T07:10:56.363997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:24:25.934834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.352491+0000","last_change":"2024-01-11T07:10:55.344206+0000","last_active":"2024-01-11T07:10:56.352491+0000","last_peered":"2024-01-11T07:10:56.352491+0000","last_clean":"2024-01-11T07:10:56.352491+0000","last_became_active":"2024-01-11T07:10:55.343982+0000","last_became_peered":"2024-01-11T07:10:55.343982+0000","last_unstale":"2024-01-11T07:10:56.352491+0000","last_undegraded":"2024-01-11T07:10:56.352491+0000","last_fullsized":"2024-01-11T07:10:56.352491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:54:18.078945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357666+0000","last_change":"2024-01-11T07:10:56.357568+0000","last_active":"2024-01-11T07:10:56.357666+0000","last_peered":"2024-01-11T07:10:56.357666+0000","last_clean":"2024-01-11T07:10:56.357666+0000","last_became_active":"2024-01-11T07:10:56.357250+0000","last_became_peered":"2024-01-11T07:10:56.357250+0000","last_unstale":"2024-01-11T07:10:56.357666+0000","last_undegraded":"2024-01-11T07:10:56.357666+0000","last_fullsized":"2024-01-11T07:10:56.357666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:08:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367284+0000","last_change":"2024-01-11T07:10:55.341903+0000","last_active":"2024-01-11T07:10:56.367284+0000","last_peered":"2024-01-11T07:10:56.367284+0000","last_clean":"2024-01-11T07:10:56.367284+0000","last_became_active":"2024-01-11T07:10:55.341386+0000","last_became_peered":"2024-01-11T07:10:55.341386+0000","last_unstale":"2024-01-11T07:10:56.367284+0000","last_undegraded":"2024-01-11T07:10:56.367284+0000","last_fullsized":"2024-01-11T07:10:56.367284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:38:05.153636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369478+0000","last_change":"2024-01-11T07:10:56.369393+0000","last_active":"2024-01-11T07:10:56.369478+0000","last_peered":"2024-01-11T07:10:56.369478+0000","last_clean":"2024-01-11T07:10:56.369478+0000","last_became_active":"2024-01-11T07:10:56.369144+0000","last_became_peered":"2024-01-11T07:10:56.369144+0000","last_unstale":"2024-01-11T07:10:56.369478+0000","last_undegraded":"2024-01-11T07:10:56.369478+0000","last_fullsized":"2024-01-11T07:10:56.369478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:09:23.227992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357224+0000","last_change":"2024-01-11T07:10:55.344607+0000","last_active":"2024-01-11T07:10:56.357224+0000","last_peered":"2024-01-11T07:10:56.357224+0000","last_clean":"2024-01-11T07:10:56.357224+0000","last_became_active":"2024-01-11T07:10:55.344370+0000","last_became_peered":"2024-01-11T07:10:55.344370+0000","last_unstale":"2024-01-11T07:10:56.357224+0000","last_undegraded":"2024-01-11T07:10:56.357224+0000","last_fullsized":"2024-01-11T07:10:56.357224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:00:18.367652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.361246+0000","last_change":"2024-01-11T07:10:56.361140+0000","last_active":"2024-01-11T07:10:56.361246+0000","last_peered":"2024-01-11T07:10:56.361246+0000","last_clean":"2024-01-11T07:10:56.361246+0000","last_became_active":"2024-01-11T07:10:56.360863+0000","last_became_peered":"2024-01-11T07:10:56.360863+0000","last_unstale":"2024-01-11T07:10:56.361246+0000","last_undegraded":"2024-01-11T07:10:56.361246+0000","last_fullsized":"2024-01-11T07:10:56.361246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:50:07.509131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.349964+0000","last_change":"2024-01-11T07:10:55.344198+0000","last_active":"2024-01-11T07:10:56.349964+0000","last_peered":"2024-01-11T07:10:56.349964+0000","last_clean":"2024-01-11T07:10:56.349964+0000","last_became_active":"2024-01-11T07:10:55.343937+0000","last_became_peered":"2024-01-11T07:10:55.343937+0000","last_unstale":"2024-01-11T07:10:56.349964+0000","last_undegraded":"2024-01-11T07:10:56.349964+0000","last_fullsized":"2024-01-11T07:10:56.349964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:52:05.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367272+0000","last_change":"2024-01-11T07:10:56.367194+0000","last_active":"2024-01-11T07:10:56.367272+0000","last_peered":"2024-01-11T07:10:56.367272+0000","last_clean":"2024-01-11T07:10:56.367272+0000","last_became_active":"2024-01-11T07:10:56.366573+0000","last_became_peered":"2024-01-11T07:10:56.366573+0000","last_unstale":"2024-01-11T07:10:56.367272+0000","last_undegraded":"2024-01-11T07:10:56.367272+0000","last_fullsized":"2024-01-11T07:10:56.367272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:34:38.048691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.375322+0000","last_change":"2024-01-11T07:10:55.340334+0000","last_active":"2024-01-11T07:10:56.375322+0000","last_peered":"2024-01-11T07:10:56.375322+0000","last_clean":"2024-01-11T07:10:56.375322+0000","last_became_active":"2024-01-11T07:10:55.339836+0000","last_became_peered":"2024-01-11T07:10:55.339836+0000","last_unstale":"2024-01-11T07:10:56.375322+0000","last_undegraded":"2024-01-11T07:10:56.375322+0000","last_fullsized":"2024-01-11T07:10:56.375322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:24:24.930005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.360953+0000","last_change":"2024-01-11T07:10:56.360743+0000","last_active":"2024-01-11T07:10:56.360953+0000","last_peered":"2024-01-11T07:10:56.360953+0000","last_clean":"2024-01-11T07:10:56.360953+0000","last_became_active":"2024-01-11T07:10:56.360008+0000","last_became_peered":"2024-01-11T07:10:56.360008+0000","last_unstale":"2024-01-11T07:10:56.360953+0000","last_undegraded":"2024-01-11T07:10:56.360953+0000","last_fullsized":"2024-01-11T07:10:56.360953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:09:53.448079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.356681+0000","last_change":"2024-01-11T07:10:55.335172+0000","last_active":"2024-01-11T07:10:56.356681+0000","last_peered":"2024-01-11T07:10:56.356681+0000","last_clean":"2024-01-11T07:10:56.356681+0000","last_became_active":"2024-01-11T07:10:55.334433+0000","last_became_peered":"2024-01-11T07:10:55.334433+0000","last_unstale":"2024-01-11T07:10:56.356681+0000","last_undegraded":"2024-01-11T07:10:56.356681+0000","last_fullsized":"2024-01-11T07:10:56.356681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:50.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.361153+0000","last_change":"2024-01-11T07:10:56.359424+0000","last_active":"2024-01-11T07:10:56.361153+0000","last_peered":"2024-01-11T07:10:56.361153+0000","last_clean":"2024-01-11T07:10:56.361153+0000","last_became_active":"2024-01-11T07:10:56.359192+0000","last_became_peered":"2024-01-11T07:10:56.359192+0000","last_unstale":"2024-01-11T07:10:56.361153+0000","last_undegraded":"2024-01-11T07:10:56.361153+0000","last_fullsized":"2024-01-11T07:10:56.361153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:18:51.826654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358330+0000","last_change":"2024-01-11T07:10:55.345775+0000","last_active":"2024-01-11T07:10:56.358330+0000","last_peered":"2024-01-11T07:10:56.358330+0000","last_clean":"2024-01-11T07:10:56.358330+0000","last_became_active":"2024-01-11T07:10:55.345332+0000","last_became_peered":"2024-01-11T07:10:55.345332+0000","last_unstale":"2024-01-11T07:10:56.358330+0000","last_undegraded":"2024-01-11T07:10:56.358330+0000","last_fullsized":"2024-01-11T07:10:56.358330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:18:15.164132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357980+0000","last_change":"2024-01-11T07:10:56.357834+0000","last_active":"2024-01-11T07:10:56.357980+0000","last_peered":"2024-01-11T07:10:56.357980+0000","last_clean":"2024-01-11T07:10:56.357980+0000","last_became_active":"2024-01-11T07:10:56.357602+0000","last_became_peered":"2024-01-11T07:10:56.357602+0000","last_unstale":"2024-01-11T07:10:56.357980+0000","last_undegraded":"2024-01-11T07:10:56.357980+0000","last_fullsized":"2024-01-11T07:10:56.357980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:23.976115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.350642+0000","last_change":"2024-01-11T07:10:55.346514+0000","last_active":"2024-01-11T07:10:56.350642+0000","last_peered":"2024-01-11T07:10:56.350642+0000","last_clean":"2024-01-11T07:10:56.350642+0000","last_became_active":"2024-01-11T07:10:55.346326+0000","last_became_peered":"2024-01-11T07:10:55.346326+0000","last_unstale":"2024-01-11T07:10:56.350642+0000","last_undegraded":"2024-01-11T07:10:56.350642+0000","last_fullsized":"2024-01-11T07:10:56.350642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:01.811268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365729+0000","last_change":"2024-01-11T07:10:56.365658+0000","last_active":"2024-01-11T07:10:56.365729+0000","last_peered":"2024-01-11T07:10:56.365729+0000","last_clean":"2024-01-11T07:10:56.365729+0000","last_became_active":"2024-01-11T07:10:56.364286+0000","last_became_peered":"2024-01-11T07:10:56.364286+0000","last_unstale":"2024-01-11T07:10:56.365729+0000","last_undegraded":"2024-01-11T07:10:56.365729+0000","last_fullsized":"2024-01-11T07:10:56.365729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:23:39.224518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.359573+0000","last_change":"2024-01-11T07:10:55.343897+0000","last_active":"2024-01-11T07:10:56.359573+0000","last_peered":"2024-01-11T07:10:56.359573+0000","last_clean":"2024-01-11T07:10:56.359573+0000","last_became_active":"2024-01-11T07:10:55.343450+0000","last_became_peered":"2024-01-11T07:10:55.343450+0000","last_unstale":"2024-01-11T07:10:56.359573+0000","last_undegraded":"2024-01-11T07:10:56.359573+0000","last_fullsized":"2024-01-11T07:10:56.359573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:43:41.480347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.366031+0000","last_change":"2024-01-11T07:10:56.365967+0000","last_active":"2024-01-11T07:10:56.366031+0000","last_peered":"2024-01-11T07:10:56.366031+0000","last_clean":"2024-01-11T07:10:56.366031+0000","last_became_active":"2024-01-11T07:10:56.365581+0000","last_became_peered":"2024-01-11T07:10:56.365581+0000","last_unstale":"2024-01-11T07:10:56.366031+0000","last_undegraded":"2024-01-11T07:10:56.366031+0000","last_fullsized":"2024-01-11T07:10:56.366031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:46:08.536333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.355314+0000","last_change":"2024-01-11T07:10:55.344782+0000","last_active":"2024-01-11T07:10:56.355314+0000","last_peered":"2024-01-11T07:10:56.355314+0000","last_clean":"2024-01-11T07:10:56.355314+0000","last_became_active":"2024-01-11T07:10:55.344505+0000","last_became_peered":"2024-01-11T07:10:55.344505+0000","last_unstale":"2024-01-11T07:10:56.355314+0000","last_undegraded":"2024-01-11T07:10:56.355314+0000","last_fullsized":"2024-01-11T07:10:56.355314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:38.055778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358193+0000","last_change":"2024-01-11T07:10:56.357432+0000","last_active":"2024-01-11T07:10:56.358193+0000","last_peered":"2024-01-11T07:10:56.358193+0000","last_clean":"2024-01-11T07:10:56.358193+0000","last_became_active":"2024-01-11T07:10:56.357139+0000","last_became_peered":"2024-01-11T07:10:56.357139+0000","last_unstale":"2024-01-11T07:10:56.358193+0000","last_undegraded":"2024-01-11T07:10:56.358193+0000","last_fullsized":"2024-01-11T07:10:56.358193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:52:02.885482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.351200+0000","last_change":"2024-01-11T07:10:55.340446+0000","last_active":"2024-01-11T07:10:56.351200+0000","last_peered":"2024-01-11T07:10:56.351200+0000","last_clean":"2024-01-11T07:10:56.351200+0000","last_became_active":"2024-01-11T07:10:55.339852+0000","last_became_peered":"2024-01-11T07:10:55.339852+0000","last_unstale":"2024-01-11T07:10:56.351200+0000","last_undegraded":"2024-01-11T07:10:56.351200+0000","last_fullsized":"2024-01-11T07:10:56.351200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:08.101397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365330+0000","last_change":"2024-01-11T07:10:56.363507+0000","last_active":"2024-01-11T07:10:56.365330+0000","last_peered":"2024-01-11T07:10:56.365330+0000","last_clean":"2024-01-11T07:10:56.365330+0000","last_became_active":"2024-01-11T07:10:56.363277+0000","last_became_peered":"2024-01-11T07:10:56.363277+0000","last_unstale":"2024-01-11T07:10:56.365330+0000","last_undegraded":"2024-01-11T07:10:56.365330+0000","last_fullsized":"2024-01-11T07:10:56.365330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:36:46.275312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367365+0000","last_change":"2024-01-11T07:10:55.343995+0000","last_active":"2024-01-11T07:10:56.367365+0000","last_peered":"2024-01-11T07:10:56.367365+0000","last_clean":"2024-01-11T07:10:56.367365+0000","last_became_active":"2024-01-11T07:10:55.342167+0000","last_became_peered":"2024-01-11T07:10:55.342167+0000","last_unstale":"2024-01-11T07:10:56.367365+0000","last_undegraded":"2024-01-11T07:10:56.367365+0000","last_fullsized":"2024-01-11T07:10:56.367365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:23:01.936777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.370358+0000","last_change":"2024-01-11T07:10:56.370254+0000","last_active":"2024-01-11T07:10:56.370358+0000","last_peered":"2024-01-11T07:10:56.370358+0000","last_clean":"2024-01-11T07:10:56.370358+0000","last_became_active":"2024-01-11T07:10:56.369929+0000","last_became_peered":"2024-01-11T07:10:56.369929+0000","last_unstale":"2024-01-11T07:10:56.370358+0000","last_undegraded":"2024-01-11T07:10:56.370358+0000","last_fullsized":"2024-01-11T07:10:56.370358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:12:29.106908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.374458+0000","last_change":"2024-01-11T07:10:55.341727+0000","last_active":"2024-01-11T07:10:56.374458+0000","last_peered":"2024-01-11T07:10:56.374458+0000","last_clean":"2024-01-11T07:10:56.374458+0000","last_became_active":"2024-01-11T07:10:55.341209+0000","last_became_peered":"2024-01-11T07:10:55.341209+0000","last_unstale":"2024-01-11T07:10:56.374458+0000","last_undegraded":"2024-01-11T07:10:56.374458+0000","last_fullsized":"2024-01-11T07:10:56.374458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:27:05.753906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":219,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.354282+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:10:56.354282+0000","last_peered":"2024-01-11T07:10:56.354282+0000","last_clean":"2024-01-11T07:10:56.354282+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:10:56.354282+0000","last_undegraded":"2024-01-11T07:10:56.354282+0000","last_fullsized":"2024-01-11T07:10:56.354282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194301,"statfs":{"total":200000000,"available":198965194,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.34799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.42299999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.754},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56399999999999995,"15min":0.54300000000000004},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.627}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.435,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.48899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.437}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.39300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.436,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76400000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199556058,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":1.157,"15min":1.157},"last":0.5}]},{"osd":1,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.621},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75600000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.47199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.621,"15min":0.623},"min":{"1min":0.44700000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53900000000000003,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54700000000000004,"15min":0.55600000000000005},"min":{"1min":0.29199999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.32700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.34799999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79400000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.71299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":435,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194876,"statfs":{"total":200000000,"available":199554026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.60899999999999999,"15min":0.48299999999999998},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.54600000000000004,"5min":20.41,"15min":20.41},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45300000000000001},"min":{"1min":0.36199999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.57099999999999995,"15min":0.46600000000000003},"min":{"1min":0.313,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56100000000000005,"5min":15.611000000000001,"15min":15.611000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.68799999999999994,"15min":0.54800000000000004},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67800000000000005,"5min":20.199999999999999,"15min":20.199999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.66800000000000004,"15min":0.53900000000000003},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.63400000000000001,"5min":20.369,"15min":20.369},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.309,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70599999999999996,"15min":0.64400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76000000000000001,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.746,"15min":0.66100000000000003},"min":{"1min":0.52600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79300000000000004,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.61199999999999999},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.84099999999999997,"5min":1.413,"15min":1.413},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.72899999999999998,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77600000000000002,"5min":9.891,"15min":9.891},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.48399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88300000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":0.622}]},{"osd":7,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.89600000000000002,"15min":0.77400000000000002},"min":{"1min":0.50600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.89700000000000002,"5min":20.324000000000002,"15min":20.324000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69899999999999995,"15min":0.72199999999999998},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91300000000000003,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1009,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194302,"statfs":{"total":200000000,"available":198965782,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.438,"15min":0.47599999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.628,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.57699999999999996,"15min":0.73599999999999999},"min":{"1min":0.29799999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57099999999999995,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.376},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48399999999999999,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61799999999999999,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.58499999999999996,"15min":0.752},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59699999999999998,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53300000000000003,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70099999999999996,"5min":2.855,"15min":2.855},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.872,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77400000000000002,"15min":0.91400000000000003},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94399999999999995,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68700000000000006,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97099999999999997,"5min":2.819,"15min":2.819},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84499999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.82099999999999995,"15min":0.97499999999999998},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.52500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0249999999999999,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.5},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.82599999999999996,"15min":0.96499999999999997},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0429999999999999,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.61199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.42299999999999999},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.307}]},{"osd":1,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.441,"15min":0.443},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0309999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.38500000000000001},"min":{"1min":0.316,"5min":0.246,"15min":0.246},"max":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.37,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.081,"5min":1.139,"15min":1.139},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65800000000000003,"15min":0.67200000000000004},"min":{"1min":0.50600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0169999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.50800000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90600000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.54800000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.52600000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83999999999999997,"5min":1.052,"15min":1.052},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.55200000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94299999999999995,"5min":1.038,"15min":1.038},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.878,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199556038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.61199999999999999,"15min":0.82199999999999995},"min":{"1min":0.26900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58699999999999997,"5min":11.481,"15min":11.481},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47499999999999998,"15min":0.48399999999999999},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.65100000000000002,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.71399999999999997,"15min":1.0289999999999999},"min":{"1min":0.29499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.54800000000000004,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.435,"15min":0.41899999999999998},"min":{"1min":0.29899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.34300000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70599999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.78000000000000003,"15min":1.0840000000000001},"min":{"1min":0.38100000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63100000000000001,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63},{"interface":"front","average":{"1min":0.69699999999999995,"5min":1.01,"15min":1.3200000000000001},"min":{"1min":0.53400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":17.474,"15min":17.474},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.94499999999999995,"15min":1.2529999999999999},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98799999999999999,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.84899999999999998,"15min":1.046},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.82499999999999996,"5min":11.394,"15min":11.394},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.86299999999999999,"15min":1.0629999999999999},"min":{"1min":0.435,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78500000000000003,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83299999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.98799999999999999,"15min":1.286},"min":{"1min":0.46500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82399999999999995,"5min":17.434000000000001,"15min":17.434000000000001},"last":1.1839999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199556080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.89000000000000001,"15min":1.0940000000000001},"min":{"1min":0.46999999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.77400000000000002,"5min":8.484,"15min":8.484},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.91700000000000004,"15min":1.1539999999999999},"min":{"1min":0.48899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76100000000000001,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.749,"15min":0.89100000000000001},"min":{"1min":0.433,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.84899999999999998,"15min":1.0489999999999999},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.432}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.86099999999999999,"15min":1.0109999999999999},"min":{"1min":0.38800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98699999999999999,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.74199999999999999,"15min":0.90400000000000003},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71899999999999997,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.83999999999999997,"15min":1.091},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.92700000000000005,"15min":1.1659999999999999},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73099999999999998,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.52400000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.72699999999999998,"15min":0.98699999999999999},"min":{"1min":0.313,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69799999999999995,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.70399999999999996,"15min":0.89400000000000002},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75600000000000001,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.74299999999999999,"15min":0.94699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76600000000000001,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":1.119,"15min":1.119},"last":0.75}]},{"osd":7,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.72899999999999998,"15min":0.92200000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.82399999999999995,"15min":1.0229999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81399999999999995,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.81399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199556060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76700000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.5},{"interface":"front","average":{"1min":0.626,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.497,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73499999999999999,"5min":0.87,"15min":0.87},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.438,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76100000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.47899999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.48399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.628},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79600000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.33800000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67200000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.40899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70899999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.433},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.40500000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.621},"min":{"1min":0.38500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61599999999999999}]}]}],"pool_statfs":[]}} 2024-01-11T07:11:00.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:11:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.719+0000 7fd431952700 1 Processor -- start 2024-01-11T07:11:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.719+0000 7fd431952700 1 -- start start 2024-01-11T07:11:00.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd431952700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd42c063230 0x7fd42c063680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd431952700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0c8020 0x7fd42c0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd431952700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42c063bc0 con 0x7fd42c0aff20 2024-01-11T07:11:00.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd431952700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42c0b02f0 con 0x7fd42c063230 2024-01-11T07:11:00.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd431952700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c154860 con 0x7fd42c0c8020 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd42c063230 0x7fd42c063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd42c063230 0x7fd42c063680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33076/0 (socket says 172.21.15.7:33076) 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42a7fc700 1 -- 172.21.15.7:0/832558850 learned_addr learned my addr 172.21.15.7:0/832558850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42b7fe700 1 --2- 172.21.15.7:0/832558850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0c8020 0x7fd42c0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42a7fc700 1 -- 172.21.15.7:0/832558850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0c8020 msgr2=0x7fd42c0c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42a7fc700 1 --2- 172.21.15.7:0/832558850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0c8020 0x7fd42c0c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42a7fc700 1 -- 172.21.15.7:0/832558850 >> v1:172.21.15.113:6789/0 conn(0x7fd42c0aff20 legacy=0x7fd42c0aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42a7fc700 1 -- 172.21.15.7:0/832558850 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c15d910 con 0x7fd42c063230 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42b7fe700 1 --2- 172.21.15.7:0/832558850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0c8020 0x7fd42c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.720+0000 7fd42a7fc700 1 --2- 172.21.15.7:0/832558850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c063230 0x7fd42c063680 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd4200099b0 tx=0x7fd420009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27384af724a2bbed server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd429ffb700 1 -- 172.21.15.7:0/832558850 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42000b860 con 0x7fd42c063230 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd429ffb700 1 -- 172.21.15.7:0/832558850 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42000b9c0 con 0x7fd42c063230 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd429ffb700 1 -- 172.21.15.7:0/832558850 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42000bd60 con 0x7fd42c063230 2024-01-11T07:11:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd431952700 1 -- 172.21.15.7:0/832558850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c063230 msgr2=0x7fd42c063680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd431952700 1 --2- 172.21.15.7:0/832558850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c063230 0x7fd42c063680 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd4200099b0 tx=0x7fd420009250 comp rx=0 tx=0).stop 2024-01-11T07:11:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd431952700 1 -- 172.21.15.7:0/832558850 shutdown_connections 2024-01-11T07:11:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd431952700 1 --2- 172.21.15.7:0/832558850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c0c8020 0x7fd42c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd431952700 1 --2- 172.21.15.7:0/832558850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c063230 0x7fd42c063680 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd431952700 1 -- 172.21.15.7:0/832558850 >> 172.21.15.7:0/832558850 conn(0x7fd42c068090 msgr2=0x7fd42c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd431952700 1 -- 172.21.15.7:0/832558850 shutdown_connections 2024-01-11T07:11:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.721+0000 7fd431952700 1 -- 172.21.15.7:0/832558850 wait complete. 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.723+0000 7fd431952700 1 Processor -- start 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.723+0000 7fd431952700 1 -- start start 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.723+0000 7fd431952700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c063230 0x7fd42c157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd431952700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c0aff20 0x7fd42c0b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd431952700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0c8020 0x7fd42c16a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd431952700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0ac630 con 0x7fd42c0c8020 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd431952700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0b3420 con 0x7fd42c0aff20 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd431952700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0ac880 con 0x7fd42c063230 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0c8020 0x7fd42c16a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0c8020 0x7fd42c16a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33080/0 (socket says 172.21.15.7:33080) 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 -- 172.21.15.7:0/1217503842 learned_addr learned my addr 172.21.15.7:0/1217503842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:00.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42affd700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c063230 0x7fd42c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42a7fc700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c0aff20 0x7fd42c0b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 -- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c063230 msgr2=0x7fd42c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c063230 0x7fd42c157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 -- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c0aff20 msgr2=0x7fd42c0b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c0aff20 0x7fd42c0b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 -- 172.21.15.7:0/1217503842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4200094f0 con 0x7fd42c0c8020 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd42b7fe700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0c8020 0x7fd42c16a650 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d180 tx=0x7fd424009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd412ffd700 1 -- 172.21.15.7:0/1217503842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400dcc0 con 0x7fd42c0c8020 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd412ffd700 1 -- 172.21.15.7:0/1217503842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42400f040 con 0x7fd42c0c8020 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd412ffd700 1 -- 172.21.15.7:0/1217503842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424015690 con 0x7fd42c0c8020 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.724+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c0acb60 con 0x7fd42c0c8020 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.725+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c0ad120 con 0x7fd42c0c8020 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.726+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c0ce0c0 con 0x7fd42c0c8020 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.726+0000 7fd412ffd700 1 -- 172.21.15.7:0/1217503842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd42401e3b0 con 0x7fd42c0c8020 2024-01-11T07:11:00.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.726+0000 7fd412ffd700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd418066f30 0x7fd4180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:00.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.726+0000 7fd412ffd700 1 -- 172.21.15.7:0/1217503842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd424084a90 con 0x7fd42c0c8020 2024-01-11T07:11:00.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.727+0000 7fd42affd700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd418066f30 0x7fd4180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:00.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.727+0000 7fd42affd700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd418066f30 0x7fd4180693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0039f0 tx=0x7fd41c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:00.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.731+0000 7fd412ffd700 1 -- 172.21.15.7:0/1217503842 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd424051c60 con 0x7fd42c0c8020 2024-01-11T07:11:00.809 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:11:00.812 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:11:00.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.931+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd42c04c3c0 con 0x7fd418066f30 2024-01-11T07:11:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.940+0000 7fd412ffd700 1 -- 172.21.15.7:0/1217503842 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285234 (secure 0 0 0) 0x7fd42c04c3c0 con 0x7fd418066f30 2024-01-11T07:11:00.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd418066f30 msgr2=0x7fd4180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd418066f30 0x7fd4180693e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0039f0 tx=0x7fd41c007ec0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0c8020 msgr2=0x7fd42c16a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0c8020 0x7fd42c16a650 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d180 tx=0x7fd424009a50 comp rx=0 tx=0).stop 2024-01-11T07:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 shutdown_connections 2024-01-11T07:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd418066f30 0x7fd4180693e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0c8020 0x7fd42c16a650 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd42c063230 0x7fd42c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 --2- 172.21.15.7:0/1217503842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd42c0aff20 0x7fd42c0b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 >> 172.21.15.7:0/1217503842 conn(0x7fd42c068090 msgr2=0x7fd42c0cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 shutdown_connections 2024-01-11T07:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:00.943+0000 7fd431952700 1 -- 172.21.15.7:0/1217503842 wait complete. 2024-01-11T07:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:11:00.957 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-01-11T07:10:59.896425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557874,"statfs":{"total":1600000000,"available":1595263852,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.566351"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.368152+0000","last_change":"2024-01-11T07:10:56.368054+0000","last_active":"2024-01-11T07:10:56.368152+0000","last_peered":"2024-01-11T07:10:56.368152+0000","last_clean":"2024-01-11T07:10:56.368152+0000","last_became_active":"2024-01-11T07:10:56.367677+0000","last_became_peered":"2024-01-11T07:10:56.367677+0000","last_unstale":"2024-01-11T07:10:56.368152+0000","last_undegraded":"2024-01-11T07:10:56.368152+0000","last_fullsized":"2024-01-11T07:10:56.368152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:11:23.743216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365273+0000","last_change":"2024-01-11T07:10:56.362832+0000","last_active":"2024-01-11T07:10:56.365273+0000","last_peered":"2024-01-11T07:10:56.365273+0000","last_clean":"2024-01-11T07:10:56.365273+0000","last_became_active":"2024-01-11T07:10:56.362245+0000","last_became_peered":"2024-01-11T07:10:56.362245+0000","last_unstale":"2024-01-11T07:10:56.365273+0000","last_undegraded":"2024-01-11T07:10:56.365273+0000","last_fullsized":"2024-01-11T07:10:56.365273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:26:38.870456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357646+0000","last_change":"2024-01-11T07:10:56.356999+0000","last_active":"2024-01-11T07:10:56.357646+0000","last_peered":"2024-01-11T07:10:56.357646+0000","last_clean":"2024-01-11T07:10:56.357646+0000","last_became_active":"2024-01-11T07:10:56.356226+0000","last_became_peered":"2024-01-11T07:10:56.356226+0000","last_unstale":"2024-01-11T07:10:56.357646+0000","last_undegraded":"2024-01-11T07:10:56.357646+0000","last_fullsized":"2024-01-11T07:10:56.357646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:08:56.391329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369089+0000","last_change":"2024-01-11T07:10:56.369009+0000","last_active":"2024-01-11T07:10:56.369089+0000","last_peered":"2024-01-11T07:10:56.369089+0000","last_clean":"2024-01-11T07:10:56.369089+0000","last_became_active":"2024-01-11T07:10:56.368603+0000","last_became_peered":"2024-01-11T07:10:56.368603+0000","last_unstale":"2024-01-11T07:10:56.369089+0000","last_undegraded":"2024-01-11T07:10:56.369089+0000","last_fullsized":"2024-01-11T07:10:56.369089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:18.567247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365911+0000","last_change":"2024-01-11T07:10:56.365825+0000","last_active":"2024-01-11T07:10:56.365911+0000","last_peered":"2024-01-11T07:10:56.365911+0000","last_clean":"2024-01-11T07:10:56.365911+0000","last_became_active":"2024-01-11T07:10:56.365612+0000","last_became_peered":"2024-01-11T07:10:56.365612+0000","last_unstale":"2024-01-11T07:10:56.365911+0000","last_undegraded":"2024-01-11T07:10:56.365911+0000","last_fullsized":"2024-01-11T07:10:56.365911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:23:02.941606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369454+0000","last_change":"2024-01-11T07:10:56.369379+0000","last_active":"2024-01-11T07:10:56.369454+0000","last_peered":"2024-01-11T07:10:56.369454+0000","last_clean":"2024-01-11T07:10:56.369454+0000","last_became_active":"2024-01-11T07:10:56.369146+0000","last_became_peered":"2024-01-11T07:10:56.369146+0000","last_unstale":"2024-01-11T07:10:56.369454+0000","last_undegraded":"2024-01-11T07:10:56.369454+0000","last_fullsized":"2024-01-11T07:10:56.369454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:00:19.372481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.361151+0000","last_change":"2024-01-11T07:10:56.360908+0000","last_active":"2024-01-11T07:10:56.361151+0000","last_peered":"2024-01-11T07:10:56.361151+0000","last_clean":"2024-01-11T07:10:56.361151+0000","last_became_active":"2024-01-11T07:10:56.360334+0000","last_became_peered":"2024-01-11T07:10:56.360334+0000","last_unstale":"2024-01-11T07:10:56.361151+0000","last_undegraded":"2024-01-11T07:10:56.361151+0000","last_fullsized":"2024-01-11T07:10:56.361151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:43:33.873432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365350+0000","last_change":"2024-01-11T07:10:56.364324+0000","last_active":"2024-01-11T07:10:56.365350+0000","last_peered":"2024-01-11T07:10:56.365350+0000","last_clean":"2024-01-11T07:10:56.365350+0000","last_became_active":"2024-01-11T07:10:56.364024+0000","last_became_peered":"2024-01-11T07:10:56.364024+0000","last_unstale":"2024-01-11T07:10:56.365350+0000","last_undegraded":"2024-01-11T07:10:56.365350+0000","last_fullsized":"2024-01-11T07:10:56.365350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:09.106227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363315+0000","last_change":"2024-01-11T07:10:56.363227+0000","last_active":"2024-01-11T07:10:56.363315+0000","last_peered":"2024-01-11T07:10:56.363315+0000","last_clean":"2024-01-11T07:10:56.363315+0000","last_became_active":"2024-01-11T07:10:56.362962+0000","last_became_peered":"2024-01-11T07:10:56.362962+0000","last_unstale":"2024-01-11T07:10:56.363315+0000","last_undegraded":"2024-01-11T07:10:56.363315+0000","last_fullsized":"2024-01-11T07:10:56.363315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:06.164432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364549+0000","last_change":"2024-01-11T07:10:56.364452+0000","last_active":"2024-01-11T07:10:56.364549+0000","last_peered":"2024-01-11T07:10:56.364549+0000","last_clean":"2024-01-11T07:10:56.364549+0000","last_became_active":"2024-01-11T07:10:56.364045+0000","last_became_peered":"2024-01-11T07:10:56.364045+0000","last_unstale":"2024-01-11T07:10:56.364549+0000","last_undegraded":"2024-01-11T07:10:56.364549+0000","last_fullsized":"2024-01-11T07:10:56.364549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:02.816098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.356560+0000","last_change":"2024-01-11T07:10:56.356330+0000","last_active":"2024-01-11T07:10:56.356560+0000","last_peered":"2024-01-11T07:10:56.356560+0000","last_clean":"2024-01-11T07:10:56.356560+0000","last_became_active":"2024-01-11T07:10:56.356095+0000","last_became_peered":"2024-01-11T07:10:56.356095+0000","last_unstale":"2024-01-11T07:10:56.356560+0000","last_undegraded":"2024-01-11T07:10:56.356560+0000","last_fullsized":"2024-01-11T07:10:56.356560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:37.600667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364817+0000","last_change":"2024-01-11T07:10:56.364706+0000","last_active":"2024-01-11T07:10:56.364817+0000","last_peered":"2024-01-11T07:10:56.364817+0000","last_clean":"2024-01-11T07:10:56.364817+0000","last_became_active":"2024-01-11T07:10:56.364117+0000","last_became_peered":"2024-01-11T07:10:56.364117+0000","last_unstale":"2024-01-11T07:10:56.364817+0000","last_undegraded":"2024-01-11T07:10:56.364817+0000","last_fullsized":"2024-01-11T07:10:56.364817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:41.703753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367450+0000","last_change":"2024-01-11T07:10:56.367382+0000","last_active":"2024-01-11T07:10:56.367450+0000","last_peered":"2024-01-11T07:10:56.367450+0000","last_clean":"2024-01-11T07:10:56.367450+0000","last_became_active":"2024-01-11T07:10:56.366924+0000","last_became_peered":"2024-01-11T07:10:56.366924+0000","last_unstale":"2024-01-11T07:10:56.367450+0000","last_undegraded":"2024-01-11T07:10:56.367450+0000","last_fullsized":"2024-01-11T07:10:56.367450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:37.600667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363054+0000","last_change":"2024-01-11T07:10:56.361408+0000","last_active":"2024-01-11T07:10:56.363054+0000","last_peered":"2024-01-11T07:10:56.363054+0000","last_clean":"2024-01-11T07:10:56.363054+0000","last_became_active":"2024-01-11T07:10:56.361150+0000","last_became_peered":"2024-01-11T07:10:56.361150+0000","last_unstale":"2024-01-11T07:10:56.363054+0000","last_undegraded":"2024-01-11T07:10:56.363054+0000","last_fullsized":"2024-01-11T07:10:56.363054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:18:00.531797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.366276+0000","last_change":"2024-01-11T07:10:56.366186+0000","last_active":"2024-01-11T07:10:56.366276+0000","last_peered":"2024-01-11T07:10:56.366276+0000","last_clean":"2024-01-11T07:10:56.366276+0000","last_became_active":"2024-01-11T07:10:56.365938+0000","last_became_peered":"2024-01-11T07:10:56.365938+0000","last_unstale":"2024-01-11T07:10:56.366276+0000","last_undegraded":"2024-01-11T07:10:56.366276+0000","last_fullsized":"2024-01-11T07:10:56.366276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:24:42.859593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369729+0000","last_change":"2024-01-11T07:10:56.369635+0000","last_active":"2024-01-11T07:10:56.369729+0000","last_peered":"2024-01-11T07:10:56.369729+0000","last_clean":"2024-01-11T07:10:56.369729+0000","last_became_active":"2024-01-11T07:10:56.369407+0000","last_became_peered":"2024-01-11T07:10:56.369407+0000","last_unstale":"2024-01-11T07:10:56.369729+0000","last_undegraded":"2024-01-11T07:10:56.369729+0000","last_fullsized":"2024-01-11T07:10:56.369729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:42:24.630686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.366476+0000","last_change":"2024-01-11T07:10:56.366289+0000","last_active":"2024-01-11T07:10:56.366476+0000","last_peered":"2024-01-11T07:10:56.366476+0000","last_clean":"2024-01-11T07:10:56.366476+0000","last_became_active":"2024-01-11T07:10:56.364958+0000","last_became_peered":"2024-01-11T07:10:56.364958+0000","last_unstale":"2024-01-11T07:10:56.366476+0000","last_undegraded":"2024-01-11T07:10:56.366476+0000","last_fullsized":"2024-01-11T07:10:56.366476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:27:06.758736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.362558+0000","last_change":"2024-01-11T07:10:56.362479+0000","last_active":"2024-01-11T07:10:56.362558+0000","last_peered":"2024-01-11T07:10:56.362558+0000","last_clean":"2024-01-11T07:10:56.362558+0000","last_became_active":"2024-01-11T07:10:56.362175+0000","last_became_peered":"2024-01-11T07:10:56.362175+0000","last_unstale":"2024-01-11T07:10:56.362558+0000","last_undegraded":"2024-01-11T07:10:56.362558+0000","last_fullsized":"2024-01-11T07:10:56.362558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:22:28.764740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.368534+0000","last_change":"2024-01-11T07:10:56.368474+0000","last_active":"2024-01-11T07:10:56.368534+0000","last_peered":"2024-01-11T07:10:56.368534+0000","last_clean":"2024-01-11T07:10:56.368534+0000","last_became_active":"2024-01-11T07:10:56.368080+0000","last_became_peered":"2024-01-11T07:10:56.368080+0000","last_unstale":"2024-01-11T07:10:56.368534+0000","last_undegraded":"2024-01-11T07:10:56.368534+0000","last_fullsized":"2024-01-11T07:10:56.368534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:57:53.821988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358875+0000","last_change":"2024-01-11T07:10:56.358786+0000","last_active":"2024-01-11T07:10:56.358875+0000","last_peered":"2024-01-11T07:10:56.358875+0000","last_clean":"2024-01-11T07:10:56.358875+0000","last_became_active":"2024-01-11T07:10:56.358509+0000","last_became_peered":"2024-01-11T07:10:56.358509+0000","last_unstale":"2024-01-11T07:10:56.358875+0000","last_undegraded":"2024-01-11T07:10:56.358875+0000","last_fullsized":"2024-01-11T07:10:56.358875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:43:09.466609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.354667+0000","last_change":"2024-01-11T07:10:56.354518+0000","last_active":"2024-01-11T07:10:56.354667+0000","last_peered":"2024-01-11T07:10:56.354667+0000","last_clean":"2024-01-11T07:10:56.354667+0000","last_became_active":"2024-01-11T07:10:56.354312+0000","last_became_peered":"2024-01-11T07:10:56.354312+0000","last_unstale":"2024-01-11T07:10:56.354667+0000","last_undegraded":"2024-01-11T07:10:56.354667+0000","last_fullsized":"2024-01-11T07:10:56.354667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:20:51.793778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364517+0000","last_change":"2024-01-11T07:10:56.364171+0000","last_active":"2024-01-11T07:10:56.364517+0000","last_peered":"2024-01-11T07:10:56.364517+0000","last_clean":"2024-01-11T07:10:56.364517+0000","last_became_active":"2024-01-11T07:10:56.363918+0000","last_became_peered":"2024-01-11T07:10:56.363918+0000","last_unstale":"2024-01-11T07:10:56.364517+0000","last_undegraded":"2024-01-11T07:10:56.364517+0000","last_fullsized":"2024-01-11T07:10:56.364517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:54:19.083774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358425+0000","last_change":"2024-01-11T07:10:56.358340+0000","last_active":"2024-01-11T07:10:56.358425+0000","last_peered":"2024-01-11T07:10:56.358425+0000","last_clean":"2024-01-11T07:10:56.358425+0000","last_became_active":"2024-01-11T07:10:56.358100+0000","last_became_peered":"2024-01-11T07:10:56.358100+0000","last_unstale":"2024-01-11T07:10:56.358425+0000","last_undegraded":"2024-01-11T07:10:56.358425+0000","last_fullsized":"2024-01-11T07:10:56.358425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:24:42.859593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369405+0000","last_change":"2024-01-11T07:10:56.369332+0000","last_active":"2024-01-11T07:10:56.369405+0000","last_peered":"2024-01-11T07:10:56.369405+0000","last_clean":"2024-01-11T07:10:56.369405+0000","last_became_active":"2024-01-11T07:10:56.368944+0000","last_became_peered":"2024-01-11T07:10:56.368944+0000","last_unstale":"2024-01-11T07:10:56.369405+0000","last_undegraded":"2024-01-11T07:10:56.369405+0000","last_fullsized":"2024-01-11T07:10:56.369405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:01:11.073664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.355359+0000","last_change":"2024-01-11T07:10:56.355284+0000","last_active":"2024-01-11T07:10:56.355359+0000","last_peered":"2024-01-11T07:10:56.355359+0000","last_clean":"2024-01-11T07:10:56.355359+0000","last_became_active":"2024-01-11T07:10:56.355050+0000","last_became_peered":"2024-01-11T07:10:56.355050+0000","last_unstale":"2024-01-11T07:10:56.355359+0000","last_undegraded":"2024-01-11T07:10:56.355359+0000","last_fullsized":"2024-01-11T07:10:56.355359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:14:36.368636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367059+0000","last_change":"2024-01-11T07:10:56.366917+0000","last_active":"2024-01-11T07:10:56.367059+0000","last_peered":"2024-01-11T07:10:56.367059+0000","last_clean":"2024-01-11T07:10:56.367059+0000","last_became_active":"2024-01-11T07:10:56.366697+0000","last_became_peered":"2024-01-11T07:10:56.366697+0000","last_unstale":"2024-01-11T07:10:56.367059+0000","last_undegraded":"2024-01-11T07:10:56.367059+0000","last_fullsized":"2024-01-11T07:10:56.367059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:26:45.713038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.356605+0000","last_change":"2024-01-11T07:10:56.356377+0000","last_active":"2024-01-11T07:10:56.356605+0000","last_peered":"2024-01-11T07:10:56.356605+0000","last_clean":"2024-01-11T07:10:56.356605+0000","last_became_active":"2024-01-11T07:10:56.356098+0000","last_became_peered":"2024-01-11T07:10:56.356098+0000","last_unstale":"2024-01-11T07:10:56.356605+0000","last_undegraded":"2024-01-11T07:10:56.356605+0000","last_fullsized":"2024-01-11T07:10:56.356605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:51.984688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365069+0000","last_change":"2024-01-11T07:10:56.364909+0000","last_active":"2024-01-11T07:10:56.365069+0000","last_peered":"2024-01-11T07:10:56.365069+0000","last_clean":"2024-01-11T07:10:56.365069+0000","last_became_active":"2024-01-11T07:10:56.364750+0000","last_became_peered":"2024-01-11T07:10:56.364750+0000","last_unstale":"2024-01-11T07:10:56.365069+0000","last_undegraded":"2024-01-11T07:10:56.365069+0000","last_fullsized":"2024-01-11T07:10:56.365069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:30:01.409443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358605+0000","last_change":"2024-01-11T07:10:56.358531+0000","last_active":"2024-01-11T07:10:56.358605+0000","last_peered":"2024-01-11T07:10:56.358605+0000","last_clean":"2024-01-11T07:10:56.358605+0000","last_became_active":"2024-01-11T07:10:56.358085+0000","last_became_peered":"2024-01-11T07:10:56.358085+0000","last_unstale":"2024-01-11T07:10:56.358605+0000","last_undegraded":"2024-01-11T07:10:56.358605+0000","last_fullsized":"2024-01-11T07:10:56.358605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:24:15.740179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.360443+0000","last_change":"2024-01-11T07:10:56.360318+0000","last_active":"2024-01-11T07:10:56.360443+0000","last_peered":"2024-01-11T07:10:56.360443+0000","last_clean":"2024-01-11T07:10:56.360443+0000","last_became_active":"2024-01-11T07:10:56.359859+0000","last_became_peered":"2024-01-11T07:10:56.359859+0000","last_unstale":"2024-01-11T07:10:56.360443+0000","last_undegraded":"2024-01-11T07:10:56.360443+0000","last_fullsized":"2024-01-11T07:10:56.360443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:42:24.630686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364165+0000","last_change":"2024-01-11T07:10:56.364089+0000","last_active":"2024-01-11T07:10:56.364165+0000","last_peered":"2024-01-11T07:10:56.364165+0000","last_clean":"2024-01-11T07:10:56.364165+0000","last_became_active":"2024-01-11T07:10:56.363792+0000","last_became_peered":"2024-01-11T07:10:56.363792+0000","last_unstale":"2024-01-11T07:10:56.364165+0000","last_undegraded":"2024-01-11T07:10:56.364165+0000","last_fullsized":"2024-01-11T07:10:56.364165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:04:49.450635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365298+0000","last_change":"2024-01-11T07:10:56.363009+0000","last_active":"2024-01-11T07:10:56.365298+0000","last_peered":"2024-01-11T07:10:56.365298+0000","last_clean":"2024-01-11T07:10:56.365298+0000","last_became_active":"2024-01-11T07:10:56.362589+0000","last_became_peered":"2024-01-11T07:10:56.362589+0000","last_unstale":"2024-01-11T07:10:56.365298+0000","last_undegraded":"2024-01-11T07:10:56.365298+0000","last_fullsized":"2024-01-11T07:10:56.365298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:41.703753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369390+0000","last_change":"2024-01-11T07:10:56.369183+0000","last_active":"2024-01-11T07:10:56.369390+0000","last_peered":"2024-01-11T07:10:56.369390+0000","last_clean":"2024-01-11T07:10:56.369390+0000","last_became_active":"2024-01-11T07:10:56.368582+0000","last_became_peered":"2024-01-11T07:10:56.368582+0000","last_unstale":"2024-01-11T07:10:56.369390+0000","last_undegraded":"2024-01-11T07:10:56.369390+0000","last_fullsized":"2024-01-11T07:10:56.369390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:43:29.078899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"83'4","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.374984+0000","last_change":"2024-01-11T07:10:55.342510+0000","last_active":"2024-01-11T07:10:56.374984+0000","last_peered":"2024-01-11T07:10:56.374984+0000","last_clean":"2024-01-11T07:10:56.374984+0000","last_became_active":"2024-01-11T07:10:55.342203+0000","last_became_peered":"2024-01-11T07:10:55.342203+0000","last_unstale":"2024-01-11T07:10:56.374984+0000","last_undegraded":"2024-01-11T07:10:56.374984+0000","last_fullsized":"2024-01-11T07:10:56.374984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:52:05.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363980+0000","last_change":"2024-01-11T07:10:56.363845+0000","last_active":"2024-01-11T07:10:56.363980+0000","last_peered":"2024-01-11T07:10:56.363980+0000","last_clean":"2024-01-11T07:10:56.363980+0000","last_became_active":"2024-01-11T07:10:56.363334+0000","last_became_peered":"2024-01-11T07:10:56.363334+0000","last_unstale":"2024-01-11T07:10:56.363980+0000","last_undegraded":"2024-01-11T07:10:56.363980+0000","last_fullsized":"2024-01-11T07:10:56.363980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:08:56.485413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.373572+0000","last_change":"2024-01-11T07:10:55.346554+0000","last_active":"2024-01-11T07:10:56.373572+0000","last_peered":"2024-01-11T07:10:56.373572+0000","last_clean":"2024-01-11T07:10:56.373572+0000","last_became_active":"2024-01-11T07:10:55.346238+0000","last_became_peered":"2024-01-11T07:10:55.346238+0000","last_unstale":"2024-01-11T07:10:56.373572+0000","last_undegraded":"2024-01-11T07:10:56.373572+0000","last_fullsized":"2024-01-11T07:10:56.373572+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:12.795675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365367+0000","last_change":"2024-01-11T07:10:56.364554+0000","last_active":"2024-01-11T07:10:56.365367+0000","last_peered":"2024-01-11T07:10:56.365367+0000","last_clean":"2024-01-11T07:10:56.365367+0000","last_became_active":"2024-01-11T07:10:56.364243+0000","last_became_peered":"2024-01-11T07:10:56.364243+0000","last_unstale":"2024-01-11T07:10:56.365367+0000","last_undegraded":"2024-01-11T07:10:56.365367+0000","last_fullsized":"2024-01-11T07:10:56.365367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:37.718222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376095+0000","last_change":"2024-01-11T07:10:55.347034+0000","last_active":"2024-01-11T07:10:56.376095+0000","last_peered":"2024-01-11T07:10:56.376095+0000","last_clean":"2024-01-11T07:10:56.376095+0000","last_became_active":"2024-01-11T07:10:55.346667+0000","last_became_peered":"2024-01-11T07:10:55.346667+0000","last_unstale":"2024-01-11T07:10:56.376095+0000","last_undegraded":"2024-01-11T07:10:56.376095+0000","last_fullsized":"2024-01-11T07:10:56.376095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:40.698923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.354382+0000","last_change":"2024-01-11T07:10:56.354265+0000","last_active":"2024-01-11T07:10:56.354382+0000","last_peered":"2024-01-11T07:10:56.354382+0000","last_clean":"2024-01-11T07:10:56.354382+0000","last_became_active":"2024-01-11T07:10:56.353963+0000","last_became_peered":"2024-01-11T07:10:56.353963+0000","last_unstale":"2024-01-11T07:10:56.354382+0000","last_undegraded":"2024-01-11T07:10:56.354382+0000","last_fullsized":"2024-01-11T07:10:56.354382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:34:38.048691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.375155+0000","last_change":"2024-01-11T07:10:55.341202+0000","last_active":"2024-01-11T07:10:56.375155+0000","last_peered":"2024-01-11T07:10:56.375155+0000","last_clean":"2024-01-11T07:10:56.375155+0000","last_became_active":"2024-01-11T07:10:55.340944+0000","last_became_peered":"2024-01-11T07:10:55.340944+0000","last_unstale":"2024-01-11T07:10:56.375155+0000","last_undegraded":"2024-01-11T07:10:56.375155+0000","last_fullsized":"2024-01-11T07:10:56.375155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:41:37.220415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.359517+0000","last_change":"2024-01-11T07:10:56.359436+0000","last_active":"2024-01-11T07:10:56.359517+0000","last_peered":"2024-01-11T07:10:56.359517+0000","last_clean":"2024-01-11T07:10:56.359517+0000","last_became_active":"2024-01-11T07:10:56.359026+0000","last_became_peered":"2024-01-11T07:10:56.359026+0000","last_unstale":"2024-01-11T07:10:56.359517+0000","last_undegraded":"2024-01-11T07:10:56.359517+0000","last_fullsized":"2024-01-11T07:10:56.359517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:02.816098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376102+0000","last_change":"2024-01-11T07:10:55.344983+0000","last_active":"2024-01-11T07:10:56.376102+0000","last_peered":"2024-01-11T07:10:56.376102+0000","last_clean":"2024-01-11T07:10:56.376102+0000","last_became_active":"2024-01-11T07:10:55.344711+0000","last_became_peered":"2024-01-11T07:10:55.344711+0000","last_unstale":"2024-01-11T07:10:56.376102+0000","last_undegraded":"2024-01-11T07:10:56.376102+0000","last_fullsized":"2024-01-11T07:10:56.376102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:18:50.821824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.354637+0000","last_change":"2024-01-11T07:10:56.354014+0000","last_active":"2024-01-11T07:10:56.354637+0000","last_peered":"2024-01-11T07:10:56.354637+0000","last_clean":"2024-01-11T07:10:56.354637+0000","last_became_active":"2024-01-11T07:10:56.353776+0000","last_became_peered":"2024-01-11T07:10:56.353776+0000","last_unstale":"2024-01-11T07:10:56.354637+0000","last_undegraded":"2024-01-11T07:10:56.354637+0000","last_fullsized":"2024-01-11T07:10:56.354637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:52:06.446086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.375410+0000","last_change":"2024-01-11T07:10:55.339454+0000","last_active":"2024-01-11T07:10:56.375410+0000","last_peered":"2024-01-11T07:10:56.375410+0000","last_clean":"2024-01-11T07:10:56.375410+0000","last_became_active":"2024-01-11T07:10:55.339218+0000","last_became_peered":"2024-01-11T07:10:55.339218+0000","last_unstale":"2024-01-11T07:10:56.375410+0000","last_undegraded":"2024-01-11T07:10:56.375410+0000","last_fullsized":"2024-01-11T07:10:56.375410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:38.055778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363111+0000","last_change":"2024-01-11T07:10:56.362950+0000","last_active":"2024-01-11T07:10:56.363111+0000","last_peered":"2024-01-11T07:10:56.363111+0000","last_clean":"2024-01-11T07:10:56.363111+0000","last_became_active":"2024-01-11T07:10:56.362603+0000","last_became_peered":"2024-01-11T07:10:56.362603+0000","last_unstale":"2024-01-11T07:10:56.363111+0000","last_undegraded":"2024-01-11T07:10:56.363111+0000","last_fullsized":"2024-01-11T07:10:56.363111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:40:34.384474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.355624+0000","last_change":"2024-01-11T07:10:55.345176+0000","last_active":"2024-01-11T07:10:56.355624+0000","last_peered":"2024-01-11T07:10:56.355624+0000","last_clean":"2024-01-11T07:10:56.355624+0000","last_became_active":"2024-01-11T07:10:55.344956+0000","last_became_peered":"2024-01-11T07:10:55.344956+0000","last_unstale":"2024-01-11T07:10:56.355624+0000","last_undegraded":"2024-01-11T07:10:56.355624+0000","last_fullsized":"2024-01-11T07:10:56.355624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:41:37.220415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.355135+0000","last_change":"2024-01-11T07:10:56.355015+0000","last_active":"2024-01-11T07:10:56.355135+0000","last_peered":"2024-01-11T07:10:56.355135+0000","last_clean":"2024-01-11T07:10:56.355135+0000","last_became_active":"2024-01-11T07:10:56.354767+0000","last_became_peered":"2024-01-11T07:10:56.354767+0000","last_unstale":"2024-01-11T07:10:56.355135+0000","last_undegraded":"2024-01-11T07:10:56.355135+0000","last_fullsized":"2024-01-11T07:10:56.355135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:53:30.491513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.362099+0000","last_change":"2024-01-11T07:10:55.343271+0000","last_active":"2024-01-11T07:10:56.362099+0000","last_peered":"2024-01-11T07:10:56.362099+0000","last_clean":"2024-01-11T07:10:56.362099+0000","last_became_active":"2024-01-11T07:10:55.342106+0000","last_became_peered":"2024-01-11T07:10:55.342106+0000","last_unstale":"2024-01-11T07:10:56.362099+0000","last_undegraded":"2024-01-11T07:10:56.362099+0000","last_fullsized":"2024-01-11T07:10:56.362099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:44:52.736040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.360926+0000","last_change":"2024-01-11T07:10:56.360588+0000","last_active":"2024-01-11T07:10:56.360926+0000","last_peered":"2024-01-11T07:10:56.360926+0000","last_clean":"2024-01-11T07:10:56.360926+0000","last_became_active":"2024-01-11T07:10:56.359687+0000","last_became_peered":"2024-01-11T07:10:56.359687+0000","last_unstale":"2024-01-11T07:10:56.360926+0000","last_undegraded":"2024-01-11T07:10:56.360926+0000","last_fullsized":"2024-01-11T07:10:56.360926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:14:36.368636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.352321+0000","last_change":"2024-01-11T07:10:55.344254+0000","last_active":"2024-01-11T07:10:56.352321+0000","last_peered":"2024-01-11T07:10:56.352321+0000","last_clean":"2024-01-11T07:10:56.352321+0000","last_became_active":"2024-01-11T07:10:55.343997+0000","last_became_peered":"2024-01-11T07:10:55.343997+0000","last_unstale":"2024-01-11T07:10:56.352321+0000","last_undegraded":"2024-01-11T07:10:56.352321+0000","last_fullsized":"2024-01-11T07:10:56.352321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:17.562418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367062+0000","last_change":"2024-01-11T07:10:56.366924+0000","last_active":"2024-01-11T07:10:56.367062+0000","last_peered":"2024-01-11T07:10:56.367062+0000","last_clean":"2024-01-11T07:10:56.367062+0000","last_became_active":"2024-01-11T07:10:56.366577+0000","last_became_peered":"2024-01-11T07:10:56.366577+0000","last_unstale":"2024-01-11T07:10:56.367062+0000","last_undegraded":"2024-01-11T07:10:56.367062+0000","last_fullsized":"2024-01-11T07:10:56.367062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:11:41.353277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357744+0000","last_change":"2024-01-11T07:10:55.343311+0000","last_active":"2024-01-11T07:10:56.357744+0000","last_peered":"2024-01-11T07:10:56.357744+0000","last_clean":"2024-01-11T07:10:56.357744+0000","last_became_active":"2024-01-11T07:10:55.343063+0000","last_became_peered":"2024-01-11T07:10:55.343063+0000","last_unstale":"2024-01-11T07:10:56.357744+0000","last_undegraded":"2024-01-11T07:10:56.357744+0000","last_fullsized":"2024-01-11T07:10:56.357744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:40:33.379644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.368387+0000","last_change":"2024-01-11T07:10:56.368317+0000","last_active":"2024-01-11T07:10:56.368387+0000","last_peered":"2024-01-11T07:10:56.368387+0000","last_clean":"2024-01-11T07:10:56.368387+0000","last_became_active":"2024-01-11T07:10:56.367731+0000","last_became_peered":"2024-01-11T07:10:56.367731+0000","last_unstale":"2024-01-11T07:10:56.368387+0000","last_undegraded":"2024-01-11T07:10:56.368387+0000","last_fullsized":"2024-01-11T07:10:56.368387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:51.478825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"83'3","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.374248+0000","last_change":"2024-01-11T07:10:55.338940+0000","last_active":"2024-01-11T07:10:56.374248+0000","last_peered":"2024-01-11T07:10:56.374248+0000","last_clean":"2024-01-11T07:10:56.374248+0000","last_became_active":"2024-01-11T07:10:55.338684+0000","last_became_peered":"2024-01-11T07:10:55.338684+0000","last_unstale":"2024-01-11T07:10:56.374248+0000","last_undegraded":"2024-01-11T07:10:56.374248+0000","last_fullsized":"2024-01-11T07:10:56.374248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:11:40.348448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363962+0000","last_change":"2024-01-11T07:10:56.363585+0000","last_active":"2024-01-11T07:10:56.363962+0000","last_peered":"2024-01-11T07:10:56.363962+0000","last_clean":"2024-01-11T07:10:56.363962+0000","last_became_active":"2024-01-11T07:10:56.362959+0000","last_became_peered":"2024-01-11T07:10:56.362959+0000","last_unstale":"2024-01-11T07:10:56.363962+0000","last_undegraded":"2024-01-11T07:10:56.363962+0000","last_fullsized":"2024-01-11T07:10:56.363962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:35:48.029935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"83'6","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.379544+0000","last_change":"2024-01-11T07:10:55.343892+0000","last_active":"2024-01-11T07:10:56.379544+0000","last_peered":"2024-01-11T07:10:56.379544+0000","last_clean":"2024-01-11T07:10:56.379544+0000","last_became_active":"2024-01-11T07:10:55.343631+0000","last_became_peered":"2024-01-11T07:10:55.343631+0000","last_unstale":"2024-01-11T07:10:56.379544+0000","last_undegraded":"2024-01-11T07:10:56.379544+0000","last_fullsized":"2024-01-11T07:10:56.379544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:38:05.153636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367971+0000","last_change":"2024-01-11T07:10:56.367893+0000","last_active":"2024-01-11T07:10:56.367971+0000","last_peered":"2024-01-11T07:10:56.367971+0000","last_clean":"2024-01-11T07:10:56.367971+0000","last_became_active":"2024-01-11T07:10:56.367381+0000","last_became_peered":"2024-01-11T07:10:56.367381+0000","last_unstale":"2024-01-11T07:10:56.367971+0000","last_undegraded":"2024-01-11T07:10:56.367971+0000","last_fullsized":"2024-01-11T07:10:56.367971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:28:14.923144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.377556+0000","last_change":"2024-01-11T07:10:55.338629+0000","last_active":"2024-01-11T07:10:56.377556+0000","last_peered":"2024-01-11T07:10:56.377556+0000","last_clean":"2024-01-11T07:10:56.377556+0000","last_became_active":"2024-01-11T07:10:55.337969+0000","last_became_peered":"2024-01-11T07:10:55.337969+0000","last_unstale":"2024-01-11T07:10:56.377556+0000","last_undegraded":"2024-01-11T07:10:56.377556+0000","last_fullsized":"2024-01-11T07:10:56.377556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:33:24.246025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.362483+0000","last_change":"2024-01-11T07:10:56.362416+0000","last_active":"2024-01-11T07:10:56.362483+0000","last_peered":"2024-01-11T07:10:56.362483+0000","last_clean":"2024-01-11T07:10:56.362483+0000","last_became_active":"2024-01-11T07:10:56.362168+0000","last_became_peered":"2024-01-11T07:10:56.362168+0000","last_unstale":"2024-01-11T07:10:56.362483+0000","last_undegraded":"2024-01-11T07:10:56.362483+0000","last_fullsized":"2024-01-11T07:10:56.362483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:17:46.184174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363478+0000","last_change":"2024-01-11T07:10:55.343545+0000","last_active":"2024-01-11T07:10:56.363478+0000","last_peered":"2024-01-11T07:10:56.363478+0000","last_clean":"2024-01-11T07:10:56.363478+0000","last_became_active":"2024-01-11T07:10:55.342805+0000","last_became_peered":"2024-01-11T07:10:55.342805+0000","last_unstale":"2024-01-11T07:10:56.363478+0000","last_undegraded":"2024-01-11T07:10:56.363478+0000","last_fullsized":"2024-01-11T07:10:56.363478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:33:24.246025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.361139+0000","last_change":"2024-01-11T07:10:56.361064+0000","last_active":"2024-01-11T07:10:56.361139+0000","last_peered":"2024-01-11T07:10:56.361139+0000","last_clean":"2024-01-11T07:10:56.361139+0000","last_became_active":"2024-01-11T07:10:56.360785+0000","last_became_peered":"2024-01-11T07:10:56.360785+0000","last_unstale":"2024-01-11T07:10:56.361139+0000","last_undegraded":"2024-01-11T07:10:56.361139+0000","last_fullsized":"2024-01-11T07:10:56.361139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:29:19.620630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357120+0000","last_change":"2024-01-11T07:10:55.340085+0000","last_active":"2024-01-11T07:10:56.357120+0000","last_peered":"2024-01-11T07:10:56.357120+0000","last_clean":"2024-01-11T07:10:56.357120+0000","last_became_active":"2024-01-11T07:10:55.339180+0000","last_became_peered":"2024-01-11T07:10:55.339180+0000","last_unstale":"2024-01-11T07:10:56.357120+0000","last_undegraded":"2024-01-11T07:10:56.357120+0000","last_fullsized":"2024-01-11T07:10:56.357120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:10:36.713393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.366073+0000","last_change":"2024-01-11T07:10:56.366015+0000","last_active":"2024-01-11T07:10:56.366073+0000","last_peered":"2024-01-11T07:10:56.366073+0000","last_clean":"2024-01-11T07:10:56.366073+0000","last_became_active":"2024-01-11T07:10:56.364644+0000","last_became_peered":"2024-01-11T07:10:56.364644+0000","last_unstale":"2024-01-11T07:10:56.366073+0000","last_undegraded":"2024-01-11T07:10:56.366073+0000","last_fullsized":"2024-01-11T07:10:56.366073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:26:56.480518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"83'8","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376677+0000","last_change":"2024-01-11T07:10:55.340603+0000","last_active":"2024-01-11T07:10:56.376677+0000","last_peered":"2024-01-11T07:10:56.376677+0000","last_clean":"2024-01-11T07:10:56.376677+0000","last_became_active":"2024-01-11T07:10:55.340183+0000","last_became_peered":"2024-01-11T07:10:55.340183+0000","last_unstale":"2024-01-11T07:10:56.376677+0000","last_undegraded":"2024-01-11T07:10:56.376677+0000","last_fullsized":"2024-01-11T07:10:56.376677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:57:52.817159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.360752+0000","last_change":"2024-01-11T07:10:56.360170+0000","last_active":"2024-01-11T07:10:56.360752+0000","last_peered":"2024-01-11T07:10:56.360752+0000","last_clean":"2024-01-11T07:10:56.360752+0000","last_became_active":"2024-01-11T07:10:56.359934+0000","last_became_peered":"2024-01-11T07:10:56.359934+0000","last_unstale":"2024-01-11T07:10:56.360752+0000","last_undegraded":"2024-01-11T07:10:56.360752+0000","last_fullsized":"2024-01-11T07:10:56.360752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:15:13.800504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.375274+0000","last_change":"2024-01-11T07:10:55.341762+0000","last_active":"2024-01-11T07:10:56.375274+0000","last_peered":"2024-01-11T07:10:56.375274+0000","last_clean":"2024-01-11T07:10:56.375274+0000","last_became_active":"2024-01-11T07:10:55.341068+0000","last_became_peered":"2024-01-11T07:10:55.341068+0000","last_unstale":"2024-01-11T07:10:56.375274+0000","last_undegraded":"2024-01-11T07:10:56.375274+0000","last_fullsized":"2024-01-11T07:10:56.375274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:56:06.714495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357961+0000","last_change":"2024-01-11T07:10:56.357139+0000","last_active":"2024-01-11T07:10:56.357961+0000","last_peered":"2024-01-11T07:10:56.357961+0000","last_clean":"2024-01-11T07:10:56.357961+0000","last_became_active":"2024-01-11T07:10:56.356898+0000","last_became_peered":"2024-01-11T07:10:56.356898+0000","last_unstale":"2024-01-11T07:10:56.357961+0000","last_undegraded":"2024-01-11T07:10:56.357961+0000","last_fullsized":"2024-01-11T07:10:56.357961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:54:05.034702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376210+0000","last_change":"2024-01-11T07:10:55.346852+0000","last_active":"2024-01-11T07:10:56.376210+0000","last_peered":"2024-01-11T07:10:56.376210+0000","last_clean":"2024-01-11T07:10:56.376210+0000","last_became_active":"2024-01-11T07:10:55.346429+0000","last_became_peered":"2024-01-11T07:10:55.346429+0000","last_unstale":"2024-01-11T07:10:56.376210+0000","last_undegraded":"2024-01-11T07:10:56.376210+0000","last_fullsized":"2024-01-11T07:10:56.376210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:52:01.880653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.364695+0000","last_change":"2024-01-11T07:10:56.364354+0000","last_active":"2024-01-11T07:10:56.364695+0000","last_peered":"2024-01-11T07:10:56.364695+0000","last_clean":"2024-01-11T07:10:56.364695+0000","last_became_active":"2024-01-11T07:10:56.363820+0000","last_became_peered":"2024-01-11T07:10:56.363820+0000","last_unstale":"2024-01-11T07:10:56.364695+0000","last_undegraded":"2024-01-11T07:10:56.364695+0000","last_fullsized":"2024-01-11T07:10:56.364695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:51:23.462342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"83'1","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.376900+0000","last_change":"2024-01-11T07:10:55.345962+0000","last_active":"2024-01-11T07:10:56.376900+0000","last_peered":"2024-01-11T07:10:56.376900+0000","last_clean":"2024-01-11T07:10:56.376900+0000","last_became_active":"2024-01-11T07:10:55.345711+0000","last_became_peered":"2024-01-11T07:10:55.345711+0000","last_unstale":"2024-01-11T07:10:56.376900+0000","last_undegraded":"2024-01-11T07:10:56.376900+0000","last_fullsized":"2024-01-11T07:10:56.376900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:08.975763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.363997+0000","last_change":"2024-01-11T07:10:56.363870+0000","last_active":"2024-01-11T07:10:56.363997+0000","last_peered":"2024-01-11T07:10:56.363997+0000","last_clean":"2024-01-11T07:10:56.363997+0000","last_became_active":"2024-01-11T07:10:56.363600+0000","last_became_peered":"2024-01-11T07:10:56.363600+0000","last_unstale":"2024-01-11T07:10:56.363997+0000","last_undegraded":"2024-01-11T07:10:56.363997+0000","last_fullsized":"2024-01-11T07:10:56.363997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:24:25.934834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.352491+0000","last_change":"2024-01-11T07:10:55.344206+0000","last_active":"2024-01-11T07:10:56.352491+0000","last_peered":"2024-01-11T07:10:56.352491+0000","last_clean":"2024-01-11T07:10:56.352491+0000","last_became_active":"2024-01-11T07:10:55.343982+0000","last_became_peered":"2024-01-11T07:10:55.343982+0000","last_unstale":"2024-01-11T07:10:56.352491+0000","last_undegraded":"2024-01-11T07:10:56.352491+0000","last_fullsized":"2024-01-11T07:10:56.352491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:54:18.078945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357666+0000","last_change":"2024-01-11T07:10:56.357568+0000","last_active":"2024-01-11T07:10:56.357666+0000","last_peered":"2024-01-11T07:10:56.357666+0000","last_clean":"2024-01-11T07:10:56.357666+0000","last_became_active":"2024-01-11T07:10:56.357250+0000","last_became_peered":"2024-01-11T07:10:56.357250+0000","last_unstale":"2024-01-11T07:10:56.357666+0000","last_undegraded":"2024-01-11T07:10:56.357666+0000","last_fullsized":"2024-01-11T07:10:56.357666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:08:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367284+0000","last_change":"2024-01-11T07:10:55.341903+0000","last_active":"2024-01-11T07:10:56.367284+0000","last_peered":"2024-01-11T07:10:56.367284+0000","last_clean":"2024-01-11T07:10:56.367284+0000","last_became_active":"2024-01-11T07:10:55.341386+0000","last_became_peered":"2024-01-11T07:10:55.341386+0000","last_unstale":"2024-01-11T07:10:56.367284+0000","last_undegraded":"2024-01-11T07:10:56.367284+0000","last_fullsized":"2024-01-11T07:10:56.367284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:38:05.153636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.369478+0000","last_change":"2024-01-11T07:10:56.369393+0000","last_active":"2024-01-11T07:10:56.369478+0000","last_peered":"2024-01-11T07:10:56.369478+0000","last_clean":"2024-01-11T07:10:56.369478+0000","last_became_active":"2024-01-11T07:10:56.369144+0000","last_became_peered":"2024-01-11T07:10:56.369144+0000","last_unstale":"2024-01-11T07:10:56.369478+0000","last_undegraded":"2024-01-11T07:10:56.369478+0000","last_fullsized":"2024-01-11T07:10:56.369478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:09:23.227992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357224+0000","last_change":"2024-01-11T07:10:55.344607+0000","last_active":"2024-01-11T07:10:56.357224+0000","last_peered":"2024-01-11T07:10:56.357224+0000","last_clean":"2024-01-11T07:10:56.357224+0000","last_became_active":"2024-01-11T07:10:55.344370+0000","last_became_peered":"2024-01-11T07:10:55.344370+0000","last_unstale":"2024-01-11T07:10:56.357224+0000","last_undegraded":"2024-01-11T07:10:56.357224+0000","last_fullsized":"2024-01-11T07:10:56.357224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:00:18.367652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.361246+0000","last_change":"2024-01-11T07:10:56.361140+0000","last_active":"2024-01-11T07:10:56.361246+0000","last_peered":"2024-01-11T07:10:56.361246+0000","last_clean":"2024-01-11T07:10:56.361246+0000","last_became_active":"2024-01-11T07:10:56.360863+0000","last_became_peered":"2024-01-11T07:10:56.360863+0000","last_unstale":"2024-01-11T07:10:56.361246+0000","last_undegraded":"2024-01-11T07:10:56.361246+0000","last_fullsized":"2024-01-11T07:10:56.361246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:50:07.509131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.349964+0000","last_change":"2024-01-11T07:10:55.344198+0000","last_active":"2024-01-11T07:10:56.349964+0000","last_peered":"2024-01-11T07:10:56.349964+0000","last_clean":"2024-01-11T07:10:56.349964+0000","last_became_active":"2024-01-11T07:10:55.343937+0000","last_became_peered":"2024-01-11T07:10:55.343937+0000","last_unstale":"2024-01-11T07:10:56.349964+0000","last_undegraded":"2024-01-11T07:10:56.349964+0000","last_fullsized":"2024-01-11T07:10:56.349964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:52:05.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367272+0000","last_change":"2024-01-11T07:10:56.367194+0000","last_active":"2024-01-11T07:10:56.367272+0000","last_peered":"2024-01-11T07:10:56.367272+0000","last_clean":"2024-01-11T07:10:56.367272+0000","last_became_active":"2024-01-11T07:10:56.366573+0000","last_became_peered":"2024-01-11T07:10:56.366573+0000","last_unstale":"2024-01-11T07:10:56.367272+0000","last_undegraded":"2024-01-11T07:10:56.367272+0000","last_fullsized":"2024-01-11T07:10:56.367272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:34:38.048691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.375322+0000","last_change":"2024-01-11T07:10:55.340334+0000","last_active":"2024-01-11T07:10:56.375322+0000","last_peered":"2024-01-11T07:10:56.375322+0000","last_clean":"2024-01-11T07:10:56.375322+0000","last_became_active":"2024-01-11T07:10:55.339836+0000","last_became_peered":"2024-01-11T07:10:55.339836+0000","last_unstale":"2024-01-11T07:10:56.375322+0000","last_undegraded":"2024-01-11T07:10:56.375322+0000","last_fullsized":"2024-01-11T07:10:56.375322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:24:24.930005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.360953+0000","last_change":"2024-01-11T07:10:56.360743+0000","last_active":"2024-01-11T07:10:56.360953+0000","last_peered":"2024-01-11T07:10:56.360953+0000","last_clean":"2024-01-11T07:10:56.360953+0000","last_became_active":"2024-01-11T07:10:56.360008+0000","last_became_peered":"2024-01-11T07:10:56.360008+0000","last_unstale":"2024-01-11T07:10:56.360953+0000","last_undegraded":"2024-01-11T07:10:56.360953+0000","last_fullsized":"2024-01-11T07:10:56.360953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:09:53.448079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.356681+0000","last_change":"2024-01-11T07:10:55.335172+0000","last_active":"2024-01-11T07:10:56.356681+0000","last_peered":"2024-01-11T07:10:56.356681+0000","last_clean":"2024-01-11T07:10:56.356681+0000","last_became_active":"2024-01-11T07:10:55.334433+0000","last_became_peered":"2024-01-11T07:10:55.334433+0000","last_unstale":"2024-01-11T07:10:56.356681+0000","last_undegraded":"2024-01-11T07:10:56.356681+0000","last_fullsized":"2024-01-11T07:10:56.356681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:50.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.361153+0000","last_change":"2024-01-11T07:10:56.359424+0000","last_active":"2024-01-11T07:10:56.361153+0000","last_peered":"2024-01-11T07:10:56.361153+0000","last_clean":"2024-01-11T07:10:56.361153+0000","last_became_active":"2024-01-11T07:10:56.359192+0000","last_became_peered":"2024-01-11T07:10:56.359192+0000","last_unstale":"2024-01-11T07:10:56.361153+0000","last_undegraded":"2024-01-11T07:10:56.361153+0000","last_fullsized":"2024-01-11T07:10:56.361153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:18:51.826654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358330+0000","last_change":"2024-01-11T07:10:55.345775+0000","last_active":"2024-01-11T07:10:56.358330+0000","last_peered":"2024-01-11T07:10:56.358330+0000","last_clean":"2024-01-11T07:10:56.358330+0000","last_became_active":"2024-01-11T07:10:55.345332+0000","last_became_peered":"2024-01-11T07:10:55.345332+0000","last_unstale":"2024-01-11T07:10:56.358330+0000","last_undegraded":"2024-01-11T07:10:56.358330+0000","last_fullsized":"2024-01-11T07:10:56.358330+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:18:15.164132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.357980+0000","last_change":"2024-01-11T07:10:56.357834+0000","last_active":"2024-01-11T07:10:56.357980+0000","last_peered":"2024-01-11T07:10:56.357980+0000","last_clean":"2024-01-11T07:10:56.357980+0000","last_became_active":"2024-01-11T07:10:56.357602+0000","last_became_peered":"2024-01-11T07:10:56.357602+0000","last_unstale":"2024-01-11T07:10:56.357980+0000","last_undegraded":"2024-01-11T07:10:56.357980+0000","last_fullsized":"2024-01-11T07:10:56.357980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:23.976115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.350642+0000","last_change":"2024-01-11T07:10:55.346514+0000","last_active":"2024-01-11T07:10:56.350642+0000","last_peered":"2024-01-11T07:10:56.350642+0000","last_clean":"2024-01-11T07:10:56.350642+0000","last_became_active":"2024-01-11T07:10:55.346326+0000","last_became_peered":"2024-01-11T07:10:55.346326+0000","last_unstale":"2024-01-11T07:10:56.350642+0000","last_undegraded":"2024-01-11T07:10:56.350642+0000","last_fullsized":"2024-01-11T07:10:56.350642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:01.811268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365729+0000","last_change":"2024-01-11T07:10:56.365658+0000","last_active":"2024-01-11T07:10:56.365729+0000","last_peered":"2024-01-11T07:10:56.365729+0000","last_clean":"2024-01-11T07:10:56.365729+0000","last_became_active":"2024-01-11T07:10:56.364286+0000","last_became_peered":"2024-01-11T07:10:56.364286+0000","last_unstale":"2024-01-11T07:10:56.365729+0000","last_undegraded":"2024-01-11T07:10:56.365729+0000","last_fullsized":"2024-01-11T07:10:56.365729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:23:39.224518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.359573+0000","last_change":"2024-01-11T07:10:55.343897+0000","last_active":"2024-01-11T07:10:56.359573+0000","last_peered":"2024-01-11T07:10:56.359573+0000","last_clean":"2024-01-11T07:10:56.359573+0000","last_became_active":"2024-01-11T07:10:55.343450+0000","last_became_peered":"2024-01-11T07:10:55.343450+0000","last_unstale":"2024-01-11T07:10:56.359573+0000","last_undegraded":"2024-01-11T07:10:56.359573+0000","last_fullsized":"2024-01-11T07:10:56.359573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:43:41.480347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.366031+0000","last_change":"2024-01-11T07:10:56.365967+0000","last_active":"2024-01-11T07:10:56.366031+0000","last_peered":"2024-01-11T07:10:56.366031+0000","last_clean":"2024-01-11T07:10:56.366031+0000","last_became_active":"2024-01-11T07:10:56.365581+0000","last_became_peered":"2024-01-11T07:10:56.365581+0000","last_unstale":"2024-01-11T07:10:56.366031+0000","last_undegraded":"2024-01-11T07:10:56.366031+0000","last_fullsized":"2024-01-11T07:10:56.366031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:46:08.536333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.355314+0000","last_change":"2024-01-11T07:10:55.344782+0000","last_active":"2024-01-11T07:10:56.355314+0000","last_peered":"2024-01-11T07:10:56.355314+0000","last_clean":"2024-01-11T07:10:56.355314+0000","last_became_active":"2024-01-11T07:10:55.344505+0000","last_became_peered":"2024-01-11T07:10:55.344505+0000","last_unstale":"2024-01-11T07:10:56.355314+0000","last_undegraded":"2024-01-11T07:10:56.355314+0000","last_fullsized":"2024-01-11T07:10:56.355314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:38.055778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.358193+0000","last_change":"2024-01-11T07:10:56.357432+0000","last_active":"2024-01-11T07:10:56.358193+0000","last_peered":"2024-01-11T07:10:56.358193+0000","last_clean":"2024-01-11T07:10:56.358193+0000","last_became_active":"2024-01-11T07:10:56.357139+0000","last_became_peered":"2024-01-11T07:10:56.357139+0000","last_unstale":"2024-01-11T07:10:56.358193+0000","last_undegraded":"2024-01-11T07:10:56.358193+0000","last_fullsized":"2024-01-11T07:10:56.358193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:52:02.885482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.351200+0000","last_change":"2024-01-11T07:10:55.340446+0000","last_active":"2024-01-11T07:10:56.351200+0000","last_peered":"2024-01-11T07:10:56.351200+0000","last_clean":"2024-01-11T07:10:56.351200+0000","last_became_active":"2024-01-11T07:10:55.339852+0000","last_became_peered":"2024-01-11T07:10:55.339852+0000","last_unstale":"2024-01-11T07:10:56.351200+0000","last_undegraded":"2024-01-11T07:10:56.351200+0000","last_fullsized":"2024-01-11T07:10:56.351200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:55:08.101397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.365330+0000","last_change":"2024-01-11T07:10:56.363507+0000","last_active":"2024-01-11T07:10:56.365330+0000","last_peered":"2024-01-11T07:10:56.365330+0000","last_clean":"2024-01-11T07:10:56.365330+0000","last_became_active":"2024-01-11T07:10:56.363277+0000","last_became_peered":"2024-01-11T07:10:56.363277+0000","last_unstale":"2024-01-11T07:10:56.365330+0000","last_undegraded":"2024-01-11T07:10:56.365330+0000","last_fullsized":"2024-01-11T07:10:56.365330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:36:46.275312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.367365+0000","last_change":"2024-01-11T07:10:55.343995+0000","last_active":"2024-01-11T07:10:56.367365+0000","last_peered":"2024-01-11T07:10:56.367365+0000","last_clean":"2024-01-11T07:10:56.367365+0000","last_became_active":"2024-01-11T07:10:55.342167+0000","last_became_peered":"2024-01-11T07:10:55.342167+0000","last_unstale":"2024-01-11T07:10:56.367365+0000","last_undegraded":"2024-01-11T07:10:56.367365+0000","last_fullsized":"2024-01-11T07:10:56.367365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:23:01.936777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.370358+0000","last_change":"2024-01-11T07:10:56.370254+0000","last_active":"2024-01-11T07:10:56.370358+0000","last_peered":"2024-01-11T07:10:56.370358+0000","last_clean":"2024-01-11T07:10:56.370358+0000","last_became_active":"2024-01-11T07:10:56.369929+0000","last_became_peered":"2024-01-11T07:10:56.369929+0000","last_unstale":"2024-01-11T07:10:56.370358+0000","last_undegraded":"2024-01-11T07:10:56.370358+0000","last_fullsized":"2024-01-11T07:10:56.370358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:55.328284+0000","last_clean_scrub_stamp":"2024-01-11T07:10:55.328284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:12:29.106908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"83'2","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.374458+0000","last_change":"2024-01-11T07:10:55.341727+0000","last_active":"2024-01-11T07:10:56.374458+0000","last_peered":"2024-01-11T07:10:56.374458+0000","last_clean":"2024-01-11T07:10:56.374458+0000","last_became_active":"2024-01-11T07:10:55.341209+0000","last_became_peered":"2024-01-11T07:10:55.341209+0000","last_unstale":"2024-01-11T07:10:56.374458+0000","last_undegraded":"2024-01-11T07:10:56.374458+0000","last_fullsized":"2024-01-11T07:10:56.374458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:10:54.323455+0000","last_clean_scrub_stamp":"2024-01-11T07:10:54.323455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:27:05.753906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":219,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-01-11T07:10:56.354282+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:10:56.354282+0000","last_peered":"2024-01-11T07:10:56.354282+0000","last_clean":"2024-01-11T07:10:56.354282+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:10:56.354282+0000","last_undegraded":"2024-01-11T07:10:56.354282+0000","last_fullsized":"2024-01-11T07:10:56.354282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1010,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194301,"statfs":{"total":200000000,"available":198965194,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.34799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.52600000000000002},"min":{"1min":0.42299999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54300000000000004,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.754},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56399999999999995,"15min":0.54300000000000004},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.627}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.435,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.48899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.69199999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.437}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.39300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76500000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.56999999999999995},"min":{"1min":0.436,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76400000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199556058,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81200000000000006,"5min":1.157,"15min":1.157},"last":0.5}]},{"osd":1,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.621},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75600000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.47199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.621,"15min":0.623},"min":{"1min":0.44700000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53900000000000003,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54700000000000004,"15min":0.55600000000000005},"min":{"1min":0.29199999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72099999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.32700000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83399999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:10:51 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.34799999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58199999999999996},"min":{"1min":0.34499999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79400000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.71299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":435,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194876,"statfs":{"total":200000000,"available":199554026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.60899999999999999,"15min":0.48299999999999998},"min":{"1min":0.27000000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.54600000000000004,"5min":20.41,"15min":20.41},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45300000000000001},"min":{"1min":0.36199999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.57099999999999995,"15min":0.46600000000000003},"min":{"1min":0.313,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.56100000000000005,"5min":15.611000000000001,"15min":15.611000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.68799999999999994,"15min":0.54800000000000004},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67800000000000005,"5min":20.199999999999999,"15min":20.199999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.66800000000000004,"15min":0.53900000000000003},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.63400000000000001,"5min":20.369,"15min":20.369},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.43099999999999999,"15min":0.438},"min":{"1min":0.309,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66700000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70599999999999996,"15min":0.64400000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76000000000000001,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.746,"15min":0.66100000000000003},"min":{"1min":0.52600000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.79300000000000004,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.61199999999999999},"min":{"1min":0.442,"5min":0.376,"15min":0.376},"max":{"1min":0.84099999999999997,"5min":1.413,"15min":1.413},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.72899999999999998,"15min":0.67500000000000004},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77600000000000002,"5min":9.891,"15min":9.891},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.48399999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88300000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.754,"15min":0.71999999999999997},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":0.622}]},{"osd":7,"last update":"Thu Jan 11 07:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.89600000000000002,"15min":0.77400000000000002},"min":{"1min":0.50600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.89700000000000002,"5min":20.324000000000002,"15min":20.324000000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69899999999999995,"15min":0.72199999999999998},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91300000000000003,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.64000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1009,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194302,"statfs":{"total":200000000,"available":198965782,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.438,"15min":0.47599999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.628,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.57699999999999996,"15min":0.73599999999999999},"min":{"1min":0.29799999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57099999999999995,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.376},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48399999999999999,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61799999999999999,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.58499999999999996,"15min":0.752},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59699999999999998,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53300000000000003,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70099999999999996,"5min":2.855,"15min":2.855},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.872,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77400000000000002,"15min":0.91400000000000003},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94399999999999995,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68700000000000006,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97099999999999997,"5min":2.819,"15min":2.819},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84499999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.82099999999999995,"15min":0.97499999999999998},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.52500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0249999999999999,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.5},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.82599999999999996,"15min":0.96499999999999997},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0429999999999999,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.61199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.41899999999999998,"15min":0.42299999999999999},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.307}]},{"osd":1,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.441,"15min":0.443},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.0309999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.39900000000000002,"15min":0.38500000000000001},"min":{"1min":0.316,"5min":0.246,"15min":0.246},"max":{"1min":0.58899999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.37,"5min":0.313,"15min":0.313},"max":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.081,"5min":1.139,"15min":1.139},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65800000000000003,"15min":0.67200000000000004},"min":{"1min":0.50600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0169999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.50800000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90600000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.56999999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.145,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.54100000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.54800000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.52600000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83999999999999997,"5min":1.052,"15min":1.052},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.55200000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94299999999999995,"5min":1.038,"15min":1.038},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.878,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199556038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.61199999999999999,"15min":0.82199999999999995},"min":{"1min":0.26900000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58699999999999997,"5min":11.481,"15min":11.481},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47499999999999998,"15min":0.48399999999999999},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.65100000000000002,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.35999999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.71399999999999997,"15min":1.0289999999999999},"min":{"1min":0.29499999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.54800000000000004,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.435,"15min":0.41899999999999998},"min":{"1min":0.29899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.34300000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70599999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.78000000000000003,"15min":1.0840000000000001},"min":{"1min":0.38100000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63100000000000001,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63},{"interface":"front","average":{"1min":0.69699999999999995,"5min":1.01,"15min":1.3200000000000001},"min":{"1min":0.53400000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":17.474,"15min":17.474},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.94499999999999995,"15min":1.2529999999999999},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98799999999999999,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.84899999999999998,"15min":1.046},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.82499999999999996,"5min":11.394,"15min":11.394},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.86299999999999999,"15min":1.0629999999999999},"min":{"1min":0.435,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78500000000000003,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83299999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:10:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.98799999999999999,"15min":1.286},"min":{"1min":0.46500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82399999999999995,"5min":17.434000000000001,"15min":17.434000000000001},"last":1.1839999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199556080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.89000000000000001,"15min":1.0940000000000001},"min":{"1min":0.46999999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.77400000000000002,"5min":8.484,"15min":8.484},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.91700000000000004,"15min":1.1539999999999999},"min":{"1min":0.48899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76100000000000001,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.749,"15min":0.89100000000000001},"min":{"1min":0.433,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.84899999999999998,"15min":1.0489999999999999},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.432}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.86099999999999999,"15min":1.0109999999999999},"min":{"1min":0.38800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98699999999999999,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.74199999999999999,"15min":0.90400000000000003},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71899999999999997,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.83999999999999997,"15min":1.091},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.92700000000000005,"15min":1.1659999999999999},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73099999999999998,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.52400000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.72699999999999998,"15min":0.98699999999999999},"min":{"1min":0.313,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69799999999999995,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.70399999999999996,"15min":0.89400000000000002},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75600000000000001,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.68899999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.74299999999999999,"15min":0.94699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76600000000000001,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":1.119,"15min":1.119},"last":0.75}]},{"osd":7,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.72899999999999998,"15min":0.92200000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.82399999999999995,"15min":1.0229999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81399999999999995,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.81399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":433,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194878,"statfs":{"total":200000000,"available":199556060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.76700000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.5},{"interface":"front","average":{"1min":0.626,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.497,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73499999999999999,"5min":0.87,"15min":0.87},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.438,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56699999999999995,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76100000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.47899999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.48399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.628},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.46700000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79600000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.33800000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67200000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.40899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70899999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.433},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.40500000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.621},"min":{"1min":0.38500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61599999999999999}]}]}],"pool_statfs":[]}} 2024-01-11T07:11:00.962 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:11:00.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:11:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.140+0000 7f1ba4bec700 1 Processor -- start 2024-01-11T07:11:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.140+0000 7f1ba4bec700 1 -- start start 2024-01-11T07:11:01.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1ba4bec700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ba015a060 0x7f1ba00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1ba4bec700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba00ce9b0 con 0x7f1ba0159c90 2024-01-11T07:11:01.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1ba4bec700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba00cf530 con 0x7f1ba0069320 2024-01-11T07:11:01.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1ba4bec700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba00b1f30 con 0x7f1ba015a060 2024-01-11T07:11:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1b9dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ba015a060 0x7f1ba00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1b9e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ba0159c90 0x7f1ba00b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44454/0 (socket says 172.21.15.7:44454) 2024-01-11T07:11:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1b9e59c700 1 -- 172.21.15.7:0/2931493831 learned_addr learned my addr 172.21.15.7:0/2931493831 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1b9dd9b700 1 -- 172.21.15.7:0/2931493831 >> v1:172.21.15.113:6789/0 conn(0x7f1ba0069320 legacy=0x7f1ba00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1b9dd9b700 1 -- 172.21.15.7:0/2931493831 >> v1:172.21.15.7:6789/0 conn(0x7f1ba0159c90 legacy=0x7f1ba00b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1b9dd9b700 1 -- 172.21.15.7:0/2931493831 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00cf9c0 con 0x7f1ba015a060 2024-01-11T07:11:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1b9d59a700 1 -- 172.21.15.7:0/2931493831 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282112032 0 0) 0x7f1ba00ce9b0 con 0x7f1ba0159c90 2024-01-11T07:11:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.141+0000 7f1b9dd9b700 1 --2- 172.21.15.7:0/2931493831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba00b0430 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1b940098e0 tx=0x7f1b940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3b77d61785d34e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1b9d59a700 1 -- 172.21.15.7:0/2931493831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94009e80 con 0x7f1ba015a060 2024-01-11T07:11:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1b9d59a700 1 -- 172.21.15.7:0/2931493831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9400b860 con 0x7f1ba015a060 2024-01-11T07:11:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1b9d59a700 1 -- 172.21.15.7:0/2931493831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9400bbe0 con 0x7f1ba015a060 2024-01-11T07:11:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1ba4bec700 1 -- 172.21.15.7:0/2931493831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 msgr2=0x7f1ba00b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1ba4bec700 1 --2- 172.21.15.7:0/2931493831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba00b0430 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1b940098e0 tx=0x7f1b940091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1ba4bec700 1 -- 172.21.15.7:0/2931493831 shutdown_connections 2024-01-11T07:11:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1ba4bec700 1 --2- 172.21.15.7:0/2931493831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba00b0430 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1ba4bec700 1 -- 172.21.15.7:0/2931493831 >> 172.21.15.7:0/2931493831 conn(0x7f1ba0071100 msgr2=0x7f1ba006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1ba4bec700 1 -- 172.21.15.7:0/2931493831 shutdown_connections 2024-01-11T07:11:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.142+0000 7f1ba4bec700 1 -- 172.21.15.7:0/2931493831 wait complete. 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.144+0000 7f1ba4bec700 1 Processor -- start 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.144+0000 7f1ba4bec700 1 -- start start 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.144+0000 7f1ba4bec700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ba0069320 0x7f1ba01555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.144+0000 7f1ba4bec700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ba0159c90 0x7f1ba015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.144+0000 7f1ba4bec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.144+0000 7f1ba4bec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba016af30 con 0x7f1ba015a060 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.144+0000 7f1ba4bec700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba016ac50 con 0x7f1ba0159c90 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.144+0000 7f1ba4bec700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00ade70 con 0x7f1ba0069320 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba015be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33100/0 (socket says 172.21.15.7:33100) 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 -- 172.21.15.7:0/802834070 learned_addr learned my addr 172.21.15.7:0/802834070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9dd9b700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ba0159c90 0x7f1ba015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9e59c700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ba0069320 0x7f1ba01555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 -- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ba0069320 msgr2=0x7f1ba01555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ba0069320 0x7f1ba01555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 -- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ba0159c90 msgr2=0x7f1ba015b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ba0159c90 0x7f1ba015b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 -- 172.21.15.7:0/802834070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b94009420 con 0x7f1ba015a060 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b9ed9d700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba015be50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c840 tx=0x7f1b98001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b8e7fc700 1 -- 172.21.15.7:0/802834070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98012630 con 0x7f1ba015a060 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b8e7fc700 1 -- 172.21.15.7:0/802834070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b980127b0 con 0x7f1ba015a060 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba00ae150 con 0x7f1ba015a060 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1b8e7fc700 1 -- 172.21.15.7:0/802834070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98012b70 con 0x7f1ba015a060 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.145+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba00ae950 con 0x7f1ba015a060 2024-01-11T07:11:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.146+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba00adcb0 con 0x7f1ba015a060 2024-01-11T07:11:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.147+0000 7f1b8e7fc700 1 -- 172.21.15.7:0/802834070 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1b9801a3a0 con 0x7f1ba015a060 2024-01-11T07:11:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.147+0000 7f1b8e7fc700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b80066f30 0x7f1b800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.147+0000 7f1b8e7fc700 1 -- 172.21.15.7:0/802834070 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1b98084700 con 0x7f1ba015a060 2024-01-11T07:11:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.148+0000 7f1b9e59c700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b80066f30 0x7f1b800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.148+0000 7f1b9e59c700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b80066f30 0x7f1b800693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00ce770 tx=0x7f1b90007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.152+0000 7f1b8e7fc700 1 -- 172.21.15.7:0/802834070 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b980518d0 con 0x7f1ba015a060 2024-01-11T07:11:01.318 INFO:tasks.ceph.mds.b.smithi113.stderr:2024-01-11T07:11:01.318+0000 7f8be988f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:11:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.439+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1ba00ae670 con 0x7f1ba015a060 2024-01-11T07:11:01.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.444+0000 7f1b8e7fc700 1 -- 172.21.15.7:0/802834070 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1ba00ae670 con 0x7f1ba015a060 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b80066f30 msgr2=0x7f1b800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b80066f30 0x7f1b800693e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00ce770 tx=0x7f1b90007d10 comp rx=0 tx=0).stop 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 msgr2=0x7f1ba015be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba015be50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800c840 tx=0x7f1b98001670 comp rx=0 tx=0).stop 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 shutdown_connections 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b80066f30 0x7f1b800693e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba015a060 0x7f1ba015be50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ba0069320 0x7f1ba01555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 --2- 172.21.15.7:0/802834070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ba0159c90 0x7f1ba015b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 >> 172.21.15.7:0/802834070 conn(0x7f1ba0071100 msgr2=0x7f1ba0158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 shutdown_connections 2024-01-11T07:11:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.446+0000 7f1ba4bec700 1 -- 172.21.15.7:0/802834070 wait complete. 2024-01-11T07:11:01.447 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:11:01.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:11:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.591+0000 7f8d67265700 1 Processor -- start 2024-01-11T07:11:01.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.591+0000 7f8d67265700 1 -- start start 2024-01-11T07:11:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.592+0000 7f8d67265700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d60159ff0 0x7f8d600b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.592+0000 7f8d67265700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d60067de0 con 0x7f8d60159c20 2024-01-11T07:11:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.592+0000 7f8d67265700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d600b20e0 con 0x7f8d6006f4d0 2024-01-11T07:11:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.592+0000 7f8d67265700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d60069570 con 0x7f8d60159ff0 2024-01-11T07:11:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d65001700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8d60159c20 0x7f8d60068270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44466/0 (socket says 172.21.15.7:44466) 2024-01-11T07:11:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d65001700 1 -- 172.21.15.7:0/1579904282 learned_addr learned my addr 172.21.15.7:0/1579904282 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d64800700 1 --2- 172.21.15.7:0/1579904282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d60159ff0 0x7f8d600b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d57fff700 1 -- 172.21.15.7:0/1579904282 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285009557 0 0) 0x7f8d60067de0 con 0x7f8d60159c20 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d57fff700 1 -- 172.21.15.7:0/1579904282 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d4c003570 con 0x7f8d60159c20 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d57fff700 1 -- 172.21.15.7:0/1579904282 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715263059 0 0) 0x7f8d600b20e0 con 0x7f8d6006f4d0 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d57fff700 1 -- 172.21.15.7:0/1579904282 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d60067de0 con 0x7f8d6006f4d0 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d64800700 1 -- 172.21.15.7:0/1579904282 >> v1:172.21.15.113:6789/0 conn(0x7f8d6006f4d0 legacy=0x7f8d600b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d64800700 1 -- 172.21.15.7:0/1579904282 >> v1:172.21.15.7:6789/0 conn(0x7f8d60159c20 legacy=0x7f8d60068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d64800700 1 -- 172.21.15.7:0/1579904282 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6006f970 con 0x7f8d60159ff0 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.593+0000 7f8d64800700 1 --2- 172.21.15.7:0/1579904282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d60159ff0 0x7f8d600b1820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009910 tx=0x7f8d500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824adbd9f827b48 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.594+0000 7f8d57fff700 1 -- 172.21.15.7:0/1579904282 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50011630 con 0x7f8d60159ff0 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.594+0000 7f8d57fff700 1 -- 172.21.15.7:0/1579904282 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d50011790 con 0x7f8d60159ff0 2024-01-11T07:11:01.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.594+0000 7f8d57fff700 1 -- 172.21.15.7:0/1579904282 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50011b10 con 0x7f8d60159ff0 2024-01-11T07:11:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.594+0000 7f8d67265700 1 -- 172.21.15.7:0/1579904282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d60159ff0 msgr2=0x7f8d600b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.594+0000 7f8d67265700 1 --2- 172.21.15.7:0/1579904282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d60159ff0 0x7f8d600b1820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009910 tx=0x7f8d500091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.594+0000 7f8d67265700 1 -- 172.21.15.7:0/1579904282 shutdown_connections 2024-01-11T07:11:01.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.594+0000 7f8d67265700 1 --2- 172.21.15.7:0/1579904282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d60159ff0 0x7f8d600b1820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.594+0000 7f8d67265700 1 -- 172.21.15.7:0/1579904282 >> 172.21.15.7:0/1579904282 conn(0x7f8d60071160 msgr2=0x7f8d6006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.595+0000 7f8d67265700 1 -- 172.21.15.7:0/1579904282 shutdown_connections 2024-01-11T07:11:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.595+0000 7f8d67265700 1 -- 172.21.15.7:0/1579904282 wait complete. 2024-01-11T07:11:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.596+0000 7f8d67265700 1 Processor -- start 2024-01-11T07:11:01.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.596+0000 7f8d67265700 1 -- start start 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d67265700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d6006f4d0 0x7f8d60146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d67265700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d60159c20 0x7f8d60156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d67265700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d60159ff0 0x7f8d601567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d67265700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d600cad60 con 0x7f8d60159ff0 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d67265700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d600ca430 con 0x7f8d60159c20 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d67265700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d600ca710 con 0x7f8d6006f4d0 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d60159ff0 0x7f8d601567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d60159ff0 0x7f8d601567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33112/0 (socket says 172.21.15.7:33112) 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 -- 172.21.15.7:0/3182290334 learned_addr learned my addr 172.21.15.7:0/3182290334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65001700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d6006f4d0 0x7f8d60146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d64800700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d60159c20 0x7f8d60156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 -- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d6006f4d0 msgr2=0x7f8d60146400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d6006f4d0 0x7f8d60146400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 -- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d60159c20 msgr2=0x7f8d60156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d60159c20 0x7f8d60156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 -- 172.21.15.7:0/3182290334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50009420 con 0x7f8d60159ff0 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.597+0000 7f8d65802700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d60159ff0 0x7f8d601567a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c910 tx=0x7f8d580027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.598+0000 7f8d557fa700 1 -- 172.21.15.7:0/3182290334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58012630 con 0x7f8d60159ff0 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.598+0000 7f8d557fa700 1 -- 172.21.15.7:0/3182290334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d580127b0 con 0x7f8d60159ff0 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.598+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d600ca9f0 con 0x7f8d60159ff0 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.598+0000 7f8d557fa700 1 -- 172.21.15.7:0/3182290334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58012b70 con 0x7f8d60159ff0 2024-01-11T07:11:01.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.598+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6016d660 con 0x7f8d60159ff0 2024-01-11T07:11:01.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.599+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d600caed0 con 0x7f8d60159ff0 2024-01-11T07:11:01.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.599+0000 7f8d557fa700 1 -- 172.21.15.7:0/3182290334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8d5801b400 con 0x7f8d60159ff0 2024-01-11T07:11:01.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.600+0000 7f8d557fa700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d44066f30 0x7f8d440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:01.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.600+0000 7f8d557fa700 1 -- 172.21.15.7:0/3182290334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d58084800 con 0x7f8d60159ff0 2024-01-11T07:11:01.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.600+0000 7f8d65001700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d44066f30 0x7f8d440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:01.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.600+0000 7f8d65001700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d44066f30 0x7f8d440693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c007fd0 tx=0x7f8d5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:01.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.604+0000 7f8d557fa700 1 -- 172.21.15.7:0/3182290334 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d58051a70 con 0x7f8d60159ff0 2024-01-11T07:11:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.855+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d6004c3c0 con 0x7f8d60159ff0 2024-01-11T07:11:01.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.856+0000 7f8d557fa700 1 -- 172.21.15.7:0/3182290334 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f8d58016020 con 0x7f8d60159ff0 2024-01-11T07:11:01.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:01.856 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2882608277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2882608277},{"type":"v1","addr":"172.21.15.7:6833","nonce":2882608277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3510440823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3510440823},{"type":"v1","addr":"172.21.15.7:6835","nonce":3510440823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3108557963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3108557963},{"type":"v1","addr":"172.21.15.113:6836","nonce":3108557963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:10:55.765305+0000","modified":"2024-01-11T07:10:59.370619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1888585107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1888585107},{"type":"v1","addr":"172.21.15.113:6837","nonce":1888585107}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-11T07:11:01.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d44066f30 msgr2=0x7f8d440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d44066f30 0x7f8d440693e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c007fd0 tx=0x7f8d5c009040 comp rx=0 tx=0).stop 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d60159ff0 msgr2=0x7f8d601567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d60159ff0 0x7f8d601567a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c910 tx=0x7f8d580027a0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 shutdown_connections 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d44066f30 0x7f8d440693e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d60159ff0 0x7f8d601567a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d6006f4d0 0x7f8d60146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 --2- 172.21.15.7:0/3182290334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d60159c20 0x7f8d60156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 >> 172.21.15.7:0/3182290334 conn(0x7f8d60071160 msgr2=0x7f8d60158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:01.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 shutdown_connections 2024-01-11T07:11:01.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:01.858+0000 7f8d67265700 1 -- 172.21.15.7:0/3182290334 wait complete. 2024-01-11T07:11:01.860 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 41 2024-01-11T07:11:02.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:10:55.765305+0000', 'modified': '2024-01-11T07:10:59.370619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6015}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6015': {'gid': 6015, 'name': 'b', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6837/1888585107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 1888585107}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 1888585107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:11:02.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:11:02.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-11T07:11:02.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.419+0000 7f619a958700 1 Processor -- start 2024-01-11T07:11:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.419+0000 7f619a958700 1 -- start start 2024-01-11T07:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f619a958700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6194158280 0x7f61940b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f619a958700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6194159300 con 0x7f61940cafb0 2024-01-11T07:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f619a958700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61940cf000 con 0x7f619415aa80 2024-01-11T07:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f619a958700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61940cf260 con 0x7f6194158280 2024-01-11T07:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f6193fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f61940cafb0 0x7f61940ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44474/0 (socket says 172.21.15.7:44474) 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f6193fff700 1 -- 172.21.15.7:0/3566246165 learned_addr learned my addr 172.21.15.7:0/3566246165 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f61937fe700 1 --2- 172.21.15.7:0/3566246165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6194158280 0x7f61940b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f6192ffd700 1 -- 172.21.15.7:0/3566246165 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263366839 0 0) 0x7f6194159300 con 0x7f61940cafb0 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.420+0000 7f6192ffd700 1 -- 172.21.15.7:0/3566246165 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6180003570 con 0x7f61940cafb0 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f6192ffd700 1 -- 172.21.15.7:0/3566246165 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839256723 0 0) 0x7f61940cf000 con 0x7f619415aa80 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f6192ffd700 1 -- 172.21.15.7:0/3566246165 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6194159300 con 0x7f619415aa80 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f6192ffd700 1 -- 172.21.15.7:0/3566246165 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690895477 0 0) 0x7f6180003570 con 0x7f61940cafb0 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f61937fe700 1 -- 172.21.15.7:0/3566246165 >> v1:172.21.15.113:6789/0 conn(0x7f619415aa80 legacy=0x7f6194161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f61937fe700 1 -- 172.21.15.7:0/3566246165 >> v1:172.21.15.7:6789/0 conn(0x7f61940cafb0 legacy=0x7f61940ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f61937fe700 1 -- 172.21.15.7:0/3566246165 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61940cf540 con 0x7f6194158280 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f61937fe700 1 --2- 172.21.15.7:0/3566246165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6194158280 0x7f61940b0430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6184009910 tx=0x7f61840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca62d32f12ce807 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f6192ffd700 1 -- 172.21.15.7:0/3566246165 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c001470 con 0x7f6194158280 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f6192ffd700 1 -- 172.21.15.7:0/3566246165 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184011420 con 0x7f6194158280 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.421+0000 7f6192ffd700 1 -- 172.21.15.7:0/3566246165 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840117c0 con 0x7f6194158280 2024-01-11T07:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.422+0000 7f619a958700 1 -- 172.21.15.7:0/3566246165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6194158280 msgr2=0x7f61940b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.422+0000 7f619a958700 1 --2- 172.21.15.7:0/3566246165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6194158280 0x7f61940b0430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6184009910 tx=0x7f61840091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.422+0000 7f619a958700 1 -- 172.21.15.7:0/3566246165 shutdown_connections 2024-01-11T07:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.422+0000 7f619a958700 1 --2- 172.21.15.7:0/3566246165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6194158280 0x7f61940b0430 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.422+0000 7f619a958700 1 -- 172.21.15.7:0/3566246165 >> 172.21.15.7:0/3566246165 conn(0x7f6194071100 msgr2=0x7f619406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.422+0000 7f619a958700 1 -- 172.21.15.7:0/3566246165 shutdown_connections 2024-01-11T07:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.422+0000 7f619a958700 1 -- 172.21.15.7:0/3566246165 wait complete. 2024-01-11T07:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.423+0000 7f619a958700 1 Processor -- start 2024-01-11T07:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.424+0000 7f619a958700 1 -- start start 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f619a958700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61940cafb0 0x7f61940c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f619a958700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6194158280 0x7f619416b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f619a958700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619415aa80 0x7f619415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f619a958700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6194142fe0 con 0x7f6194158280 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f619a958700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61940c3d10 con 0x7f619415aa80 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f619a958700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619416bcf0 con 0x7f61940cafb0 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6194158280 0x7f619416b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6194158280 0x7f619416b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33120/0 (socket says 172.21.15.7:33120) 2024-01-11T07:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 -- 172.21.15.7:0/2668010868 learned_addr learned my addr 172.21.15.7:0/2668010868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f6193fff700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61940cafb0 0x7f61940c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f6198ef5700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619415aa80 0x7f619415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 -- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61940cafb0 msgr2=0x7f61940c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61940cafb0 0x7f61940c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 -- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619415aa80 msgr2=0x7f619415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619415aa80 0x7f619415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 -- 172.21.15.7:0/2668010868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6184009420 con 0x7f6194158280 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.425+0000 7f61937fe700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6194158280 0x7f619416b2c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f61840091e0 tx=0x7f6184016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.426+0000 7f618bfff700 1 -- 172.21.15.7:0/2668010868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618401cb30 con 0x7f6194158280 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.426+0000 7f618bfff700 1 -- 172.21.15.7:0/2668010868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618401cc90 con 0x7f6194158280 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.426+0000 7f619a958700 1 -- 172.21.15.7:0/2668010868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619416bfd0 con 0x7f6194158280 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.426+0000 7f618bfff700 1 -- 172.21.15.7:0/2668010868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618401a6a0 con 0x7f6194158280 2024-01-11T07:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.426+0000 7f619a958700 1 -- 172.21.15.7:0/2668010868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61940c6960 con 0x7f6194158280 2024-01-11T07:11:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.427+0000 7f618bfff700 1 -- 172.21.15.7:0/2668010868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f618401aad0 con 0x7f6194158280 2024-01-11T07:11:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.428+0000 7f618bfff700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6174066f30 0x7f61740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.428+0000 7f618bfff700 1 -- 172.21.15.7:0/2668010868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61840863b0 con 0x7f6194158280 2024-01-11T07:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.428+0000 7f6193fff700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6174066f30 0x7f61740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.428+0000 7f6193fff700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6174066f30 0x7f61740693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f617c0015a0 tx=0x7f617c000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.430+0000 7f61a3c58b80 1 Processor -- start 2024-01-11T07:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.430+0000 7f61a3c58b80 1 -- start start 2024-01-11T07:11:02.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d704360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7d6163e0 con 0x562d7d6e7fa0 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7d613be0 con 0x562d7d6e88b0 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d7d60f9d0 con 0x562d7d6e4170 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f6193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f6193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6e8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33136/0 (socket says 172.21.15.7:33136) 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f6193fff700 1 -- 172.21.15.7:0/2183240601 learned_addr learned my addr 172.21.15.7:0/2183240601 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f61937fe700 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f6198ef5700 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d704360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f6193fff700 1 -- 172.21.15.7:0/2183240601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 msgr2=0x562d7d704360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f6193fff700 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d704360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f6193fff700 1 -- 172.21.15.7:0/2183240601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 msgr2=0x562d7d6e3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.431+0000 7f6193fff700 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6e3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.432+0000 7f6193fff700 1 -- 172.21.15.7:0/2183240601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d7d4a3360 con 0x562d7d6e7fa0 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.432+0000 7f6193fff700 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6e8370 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f617c00fdc0 tx=0x7f617c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9c4c5cc0451caa server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.432+0000 7f61897fa700 1 -- 172.21.15.7:0/2183240601 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00fe50 con 0x562d7d6e7fa0 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.432+0000 7f61897fa700 1 -- 172.21.15.7:0/2183240601 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c013a90 con 0x562d7d6e7fa0 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.432+0000 7f61897fa700 1 -- 172.21.15.7:0/2183240601 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c013e10 con 0x562d7d6e7fa0 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 -- 172.21.15.7:0/2183240601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 msgr2=0x562d7d6e8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6e8370 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f617c00fdc0 tx=0x7f617c014000 comp rx=0 tx=0).stop 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 -- 172.21.15.7:0/2183240601 shutdown_connections 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d704360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6e8370 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/2183240601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6e3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 -- 172.21.15.7:0/2183240601 >> 172.21.15.7:0/2183240601 conn(0x562d7d6be440 msgr2=0x562d7d709dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 -- 172.21.15.7:0/2183240601 shutdown_connections 2024-01-11T07:11:02.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 -- 172.21.15.7:0/2183240601 wait complete. 2024-01-11T07:11:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.433+0000 7f61a3c58b80 1 -- start start 2024-01-11T07:11:02.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.434+0000 7f61a3c58b80 1 Processor -- start 2024-01-11T07:11:02.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e4170 0x562d7d700dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e88b0 0x562d7d6ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7d5bc1b0 con 0x562d7d6e7fa0 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7d5bbe50 con 0x562d7d6e4170 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d7d508e40 con 0x562d7d6e88b0 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61a3c58b80 10 client.0 _ll_register_callbacks cb 0x562d7d50b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33148/0 (socket says 172.21.15.7:33148) 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61937fe700 1 -- 172.21.15.7:0/984934212 learned_addr learned my addr 172.21.15.7:0/984934212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f6198ef5700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e88b0 0x562d7d6ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f6193fff700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e4170 0x562d7d700dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61937fe700 1 -- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e88b0 msgr2=0x562d7d6ff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61937fe700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e88b0 0x562d7d6ff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61937fe700 1 -- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e4170 msgr2=0x562d7d700dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61937fe700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e4170 0x562d7d700dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f61937fe700 1 -- 172.21.15.7:0/984934212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c00fab0 con 0x562d7d6e7fa0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f6193fff700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e4170 0x562d7d700dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.435+0000 7f6198ef5700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e88b0 0x562d7d6ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61937fe700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fee80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f61840848c0 tx=0x7f6184008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741728ece8a4dad5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61727fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61727fc700 1 -- 172.21.15.7:0/984934212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618405e260 con 0x562d7d6e7fa0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61727fc700 1 -- 172.21.15.7:0/984934212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f618401ce00 con 0x562d7d6e7fa0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 1 -- 172.21.15.7:0/984934212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d7d690eb0 con 0x562d7d6e7fa0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61727fc700 1 -- 172.21.15.7:0/984934212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f618401ad80 con 0x562d7d6e7fa0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 10 client.6159 fetch_fsmap learned FSMap version 41 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 1 -- 172.21.15.7:0/984934212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x562d7d44fb10 con 0x562d7d6e7fa0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61727fc700 1 -- 172.21.15.7:0/984934212 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184062080 con 0x562d7d6e7fa0 2024-01-11T07:11:02.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61727fc700 1 -- 172.21.15.7:0/984934212 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f61840860b0 con 0x562d7d6e7fa0 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 10 client.6159 fetch_fsmap finished waiting for FSMap version 41 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 10 client.6159 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 20 client.6159 populate_metadata read hostname 'smithi007' 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] conn(0x562d7d6e73b0 0x562d7d723200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 4 client.6159 mds_command: new command op to 6015 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.436+0000 7f61a3c58b80 1 -- 172.21.15.7:0/984934212 --> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d7d5e6880 con 0x562d7d6e73b0 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.437+0000 7f6193fff700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] conn(0x562d7d6e73b0 0x562d7d723200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.437+0000 7f6193fff700 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] conn(0x562d7d6e73b0 0x562d7d723200 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48e61b3ad2a727e server_cookie=e84f8fffddb31ec8 in_seq=0 out_seq=0 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.437+0000 7f61727fc700 10 client.6159 ms_handle_connect on v2:172.21.15.113:6835/1888585107 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61727fc700 1 -- 172.21.15.7:0/984934212 <== mds.0 v2:172.21.15.113:6835/1888585107 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562d7d5e6880 con 0x562d7d6e73b0 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61727fc700 10 client.6159 handle_command_reply: tid=0 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 client.6159 shutdown 2024-01-11T07:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 -- 172.21.15.7:0/984934212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 msgr2=0x562d7d6fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fee80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f61840848c0 tx=0x7f6184008040 comp rx=0 tx=0).stop 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 -- 172.21.15.7:0/984934212 shutdown_connections 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] conn(0x562d7d6e73b0 0x562d7d723200 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e88b0 0x562d7d6ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e4170 0x562d7d700dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61727fc700 0 client.6159 ms_handle_reset on v2:172.21.15.113:6835/1888585107 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/984934212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fee80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 -- 172.21.15.7:0/984934212 >> 172.21.15.7:0/984934212 conn(0x562d7d6be440 msgr2=0x562d7d6be810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 -- 172.21.15.7:0/984934212 shutdown_connections 2024-01-11T07:11:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 1 -- 172.21.15.7:0/984934212 wait complete. 2024-01-11T07:11:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.438+0000 7f61a3c58b80 20 client.6159 trim_cache size 0 max 16384 2024-01-11T07:11:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.456+0000 7f61a3c58b80 1 Processor -- start 2024-01-11T07:11:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.456+0000 7f61a3c58b80 1 -- start start 2024-01-11T07:11:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e4170 0x562d7d6fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e7fa0 0x562d7d6bba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7d6bf800 con 0x562d7d6e4170 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7d6c25a0 con 0x562d7d6e88b0 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d7d6bbf70 con 0x562d7d6e7fa0 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f6193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e4170 0x562d7d6fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f6193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e4170 0x562d7d6fe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33162/0 (socket says 172.21.15.7:33162) 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f6193fff700 1 -- 172.21.15.7:0/44445730 learned_addr learned my addr 172.21.15.7:0/44445730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.457+0000 7f61937fe700 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e7fa0 0x562d7d6bba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6198ef5700 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6193fff700 1 -- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e7fa0 msgr2=0x562d7d6bba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6193fff700 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e7fa0 0x562d7d6bba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6193fff700 1 -- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 msgr2=0x562d7d6c1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6193fff700 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6193fff700 1 -- 172.21.15.7:0/44445730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d7d5de290 con 0x562d7d6e4170 2024-01-11T07:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6198ef5700 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6193fff700 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e4170 0x562d7d6fe540 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f617c023040 tx=0x7f617c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712e51476afcd182 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6189ffb700 1 -- 172.21.15.7:0/44445730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c0016e0 con 0x562d7d6e4170 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6189ffb700 1 -- 172.21.15.7:0/44445730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c013a90 con 0x562d7d6e4170 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.458+0000 7f6189ffb700 1 -- 172.21.15.7:0/44445730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c013c00 con 0x562d7d6e4170 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 -- 172.21.15.7:0/44445730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e4170 msgr2=0x562d7d6fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e4170 0x562d7d6fe540 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f617c023040 tx=0x7f617c015040 comp rx=0 tx=0).stop 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 -- 172.21.15.7:0/44445730 shutdown_connections 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e4170 0x562d7d6fe540 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/44445730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e7fa0 0x562d7d6bba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 -- 172.21.15.7:0/44445730 >> 172.21.15.7:0/44445730 conn(0x562d7d6be440 msgr2=0x562d7d6be810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 -- 172.21.15.7:0/44445730 shutdown_connections 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 -- 172.21.15.7:0/44445730 wait complete. 2024-01-11T07:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.459+0000 7f61a3c58b80 1 -- start start 2024-01-11T07:11:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61a3c58b80 1 Processor -- start 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d76f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61a3c58b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7d702490 con 0x562d7d6e7fa0 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d7d6fcd70 con 0x562d7d6e88b0 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61a3c58b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d7d76f210 con 0x562d7d6e4170 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61a3c58b80 10 client.0 _ll_register_callbacks cb 0x562d7d50b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33178/0 (socket says 172.21.15.7:33178) 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.460+0000 7f61937fe700 1 -- 172.21.15.7:0/3394721175 learned_addr learned my addr 172.21.15.7:0/3394721175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f6193fff700 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d76f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f6198ef5700 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61937fe700 1 -- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 msgr2=0x562d7d76f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61937fe700 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d76f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61937fe700 1 -- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 msgr2=0x562d7d6c1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61937fe700 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61937fe700 1 -- 172.21.15.7:0/3394721175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d7d4a3360 con 0x562d7d6e7fa0 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f6193fff700 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d76f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61937fe700 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fc160 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6184000c30 tx=0x7f618406a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b57abe5245dd2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61727fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:11:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61727fc700 1 -- 172.21.15.7:0/3394721175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618401ad80 con 0x562d7d6e7fa0 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61727fc700 1 -- 172.21.15.7:0/3394721175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f618405e1e0 con 0x562d7d6e7fa0 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61a3c58b80 1 -- 172.21.15.7:0/3394721175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d7d6b4490 con 0x562d7d6e7fa0 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61727fc700 1 -- 172.21.15.7:0/3394721175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f618401ce00 con 0x562d7d6e7fa0 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61a3c58b80 10 client.6165 fetch_fsmap learned FSMap version 41 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61a3c58b80 1 -- 172.21.15.7:0/3394721175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x562d7d4a7c80 con 0x562d7d6e7fa0 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.461+0000 7f61727fc700 1 -- 172.21.15.7:0/3394721175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184062070 con 0x562d7d6e7fa0 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61727fc700 1 -- 172.21.15.7:0/3394721175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6184057540 con 0x562d7d6e7fa0 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61a3c58b80 10 client.6165 fetch_fsmap finished waiting for FSMap version 41 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61a3c58b80 10 client.6165 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61a3c58b80 20 client.6165 populate_metadata read hostname 'smithi007' 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] conn(0x562d7d765940 0x562d7d709dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61a3c58b80 4 client.6165 mds_command: new command op to 6015 tid=0[{"prefix": "status"}] 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61a3c58b80 1 -- 172.21.15.7:0/3394721175 --> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d7d5b2910 con 0x562d7d765940 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f6193fff700 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] conn(0x562d7d765940 0x562d7d709dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f6193fff700 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] conn(0x562d7d765940 0x562d7d709dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04bc5ba5c18ab80 server_cookie=235077b5e285227c in_seq=0 out_seq=0 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61727fc700 10 client.6165 ms_handle_connect on v2:172.21.15.113:6835/1888585107 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61727fc700 1 -- 172.21.15.7:0/3394721175 <== mds.0 v2:172.21.15.113:6835/1888585107 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562d7d5b2910 con 0x562d7d765940 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61727fc700 10 client.6165 handle_command_reply: tid=0 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.462+0000 7f61a3c58b80 1 client.6165 shutdown 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 -- 172.21.15.7:0/3394721175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 msgr2=0x562d7d6fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fc160 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6184000c30 tx=0x7f618406a000 comp rx=0 tx=0).stop 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 -- 172.21.15.7:0/3394721175 shutdown_connections 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:6835/1888585107,v1:172.21.15.113:6837/1888585107] conn(0x562d7d765940 0x562d7d709dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562d7d6e88b0 0x562d7d6c1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562d7d6e4170 0x562d7d76f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61727fc700 0 client.6165 ms_handle_reset on v2:172.21.15.113:6835/1888585107 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 --2- 172.21.15.7:0/3394721175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d7d6e7fa0 0x562d7d6fc160 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 -- 172.21.15.7:0/3394721175 >> 172.21.15.7:0/3394721175 conn(0x562d7d6be440 msgr2=0x562d7d6b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 -- 172.21.15.7:0/3394721175 shutdown_connections 2024-01-11T07:11:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 1 -- 172.21.15.7:0/3394721175 wait complete. 2024-01-11T07:11:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f61a3c58b80 20 client.6165 trim_cache size 0 max 16384 2024-01-11T07:11:02.472 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6015, 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.0999685369999996, 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 41, 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 84, 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 84, 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.948677858 2024-01-11T07:11:02.473 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 -- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6174066f30 msgr2=0x7f61740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6174066f30 0x7f61740693e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f617c0015a0 tx=0x7f617c000bc0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 -- 172.21.15.7:0/2668010868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6194158280 msgr2=0x7f619416b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6194158280 0x7f619416b2c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f61840091e0 tx=0x7f6184016000 comp rx=0 tx=0).stop 2024-01-11T07:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 -- 172.21.15.7:0/2668010868 shutdown_connections 2024-01-11T07:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6174066f30 0x7f61740693e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f619415aa80 0x7f619415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f61940cafb0 0x7f61940c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 --2- 172.21.15.7:0/2668010868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6194158280 0x7f619416b2c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.463+0000 7f619a958700 1 -- 172.21.15.7:0/2668010868 >> 172.21.15.7:0/2668010868 conn(0x7f6194071100 msgr2=0x7f6194156080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.464+0000 7f619a958700 1 -- 172.21.15.7:0/2668010868 shutdown_connections 2024-01-11T07:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.464+0000 7f619a958700 1 -- 172.21.15.7:0/2668010868 wait complete. 2024-01-11T07:11:02.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:11:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.609+0000 7f237f245700 1 Processor -- start 2024-01-11T07:11:02.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.609+0000 7f237f245700 1 -- start start 2024-01-11T07:11:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f237f245700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23780cafb0 0x7f23780ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f237f245700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f23780b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f237f245700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2378159300 con 0x7f23780b0970 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f237f245700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23780b0df0 con 0x7f23780cafb0 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f237f245700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23780cf260 con 0x7f2378158280 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f237cfe1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23780cafb0 0x7f23780ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f237cfe1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23780cafb0 0x7f23780ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33188/0 (socket says 172.21.15.7:33188) 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f237cfe1700 1 -- 172.21.15.7:0/2993366909 learned_addr learned my addr 172.21.15.7:0/2993366909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.610+0000 7f2377fff700 1 --2- 172.21.15.7:0/2993366909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f23780b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f237cfe1700 1 -- 172.21.15.7:0/2993366909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 msgr2=0x7f23780b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f237cfe1700 1 --2- 172.21.15.7:0/2993366909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f23780b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f237cfe1700 1 -- 172.21.15.7:0/2993366909 >> v1:172.21.15.113:6789/0 conn(0x7f23780b0970 legacy=0x7f2378159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f237cfe1700 1 -- 172.21.15.7:0/2993366909 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23780cf540 con 0x7f23780cafb0 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f23777fe700 1 -- 172.21.15.7:0/2993366909 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498932473 0 0) 0x7f2378159300 con 0x7f23780b0970 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f237cfe1700 1 --2- 172.21.15.7:0/2993366909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f23780ce8f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f236c0099b0 tx=0x7f236c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c199860405f8203b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f23777fe700 1 -- 172.21.15.7:0/2993366909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c009ea0 con 0x7f23780cafb0 2024-01-11T07:11:02.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f23777fe700 1 -- 172.21.15.7:0/2993366909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c00b860 con 0x7f23780cafb0 2024-01-11T07:11:02.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.611+0000 7f23777fe700 1 -- 172.21.15.7:0/2993366909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00bc00 con 0x7f23780cafb0 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.612+0000 7f237f245700 1 -- 172.21.15.7:0/2993366909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 msgr2=0x7f23780ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.612+0000 7f237f245700 1 --2- 172.21.15.7:0/2993366909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f23780ce8f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f236c0099b0 tx=0x7f236c009250 comp rx=0 tx=0).stop 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.612+0000 7f237f245700 1 -- 172.21.15.7:0/2993366909 shutdown_connections 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.612+0000 7f237f245700 1 --2- 172.21.15.7:0/2993366909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f23780ce8f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.612+0000 7f237f245700 1 --2- 172.21.15.7:0/2993366909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f23780b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.612+0000 7f237f245700 1 -- 172.21.15.7:0/2993366909 >> 172.21.15.7:0/2993366909 conn(0x7f2378071100 msgr2=0x7f237806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.612+0000 7f237f245700 1 -- 172.21.15.7:0/2993366909 shutdown_connections 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.612+0000 7f237f245700 1 -- 172.21.15.7:0/2993366909 wait complete. 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.613+0000 7f237f245700 1 Processor -- start 2024-01-11T07:11:02.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.613+0000 7f237f245700 1 -- start start 2024-01-11T07:11:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237f245700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780b0970 0x7f23780c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237f245700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f237814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237f245700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f237814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237f245700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23780ca7e0 con 0x7f23780cafb0 2024-01-11T07:11:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237f245700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378069420 con 0x7f23780b0970 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237f245700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23780ca290 con 0x7f2378158280 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f2377fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f237814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f2377fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f237814f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33202/0 (socket says 172.21.15.7:33202) 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f2377fff700 1 -- 172.21.15.7:0/2602073863 learned_addr learned my addr 172.21.15.7:0/2602073863 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237cfe1700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780b0970 0x7f23780c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237d7e2700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f237814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f2377fff700 1 -- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 msgr2=0x7f237814f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f2377fff700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f237814f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f2377fff700 1 -- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780b0970 msgr2=0x7f23780c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f2377fff700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780b0970 0x7f23780c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f2377fff700 1 -- 172.21.15.7:0/2602073863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0094f0 con 0x7f23780cafb0 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.614+0000 7f237d7e2700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f237814f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.615+0000 7f2377fff700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f237814f3e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f236400e930 tx=0x7f23640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.615+0000 7f2374ff9700 1 -- 172.21.15.7:0/2602073863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400eed0 con 0x7f23780cafb0 2024-01-11T07:11:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.615+0000 7f2374ff9700 1 -- 172.21.15.7:0/2602073863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400c850 con 0x7f23780cafb0 2024-01-11T07:11:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.615+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23780ca500 con 0x7f23780cafb0 2024-01-11T07:11:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.615+0000 7f2374ff9700 1 -- 172.21.15.7:0/2602073863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400cc10 con 0x7f23780cafb0 2024-01-11T07:11:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.615+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23780c8150 con 0x7f23780cafb0 2024-01-11T07:11:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.616+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2378069790 con 0x7f23780cafb0 2024-01-11T07:11:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.616+0000 7f2374ff9700 1 -- 172.21.15.7:0/2602073863 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2364013470 con 0x7f23780cafb0 2024-01-11T07:11:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.617+0000 7f2374ff9700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f235c066f30 0x7f235c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.617+0000 7f2374ff9700 1 -- 172.21.15.7:0/2602073863 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f23640846f0 con 0x7f23780cafb0 2024-01-11T07:11:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.617+0000 7f237cfe1700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f235c066f30 0x7f235c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.618+0000 7f237cfe1700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f235c066f30 0x7f235c0693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f236c0099b0 tx=0x7f236c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.621+0000 7f2374ff9700 1 -- 172.21.15.7:0/2602073863 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2364051960 con 0x7f23780cafb0 2024-01-11T07:11:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.872+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2378069590 con 0x7f23780cafb0 2024-01-11T07:11:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.873+0000 7f2374ff9700 1 -- 172.21.15.7:0/2602073863 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2364054f80 con 0x7f23780cafb0 2024-01-11T07:11:02.872 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:11:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f235c066f30 msgr2=0x7f235c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f235c066f30 0x7f235c0693e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f236c0099b0 tx=0x7f236c017000 comp rx=0 tx=0).stop 2024-01-11T07:11:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 msgr2=0x7f237814f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f237814f3e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f236400e930 tx=0x7f23640091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 shutdown_connections 2024-01-11T07:11:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f235c066f30 0x7f235c0693e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2378158280 0x7f237814f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23780b0970 0x7f23780c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 --2- 172.21.15.7:0/2602073863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23780cafb0 0x7f237814f3e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 >> 172.21.15.7:0/2602073863 conn(0x7f2378071100 msgr2=0x7f23780ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 shutdown_connections 2024-01-11T07:11:02.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:02.875+0000 7f237f245700 1 -- 172.21.15.7:0/2602073863 wait complete. 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:11:02.888 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:11:02.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:02.889 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42155sec preferred_lft 42155sec 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:11:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:02.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:02.910 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:11:02.978 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:11:02.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:02.980 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:11:03.044 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:11:03.044 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:11:03.044 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:11:03.044 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:11:03.044 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:11:03.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:03.044 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:11:03.045 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:11:03.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:11:03.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:11:03.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:11:03.045 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:11:03.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:03.169 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:11:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:11:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:11:03.190 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:11:03.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:03.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:11:03.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:11:03.265 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:11:03.326 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:11:03.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:11:03.327 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:11:03.388 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:11:03.388 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:11:03.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:11:03.482 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:11:03.482 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:11:03.482 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:11:03.492 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:11:03.492 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:11:03.492 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:11:03.492 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:11:03.492 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:11:03.511 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:11:03.511 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:11:03.511 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:11:03.511 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:11:03.511 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:11:03.512 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:11:03.512 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:11:03.512 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:11:03.514 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:11:03.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:03.514 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:11:03.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:11:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.727+0000 7f4fbcfb6700 1 Processor -- start 2024-01-11T07:11:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.727+0000 7f4fbcfb6700 1 -- start start 2024-01-11T07:11:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.728+0000 7f4fbcfb6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4fb8159ca0 0x7f4fb806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fbcfb6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb815a070 0x7f4fb806e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fbcfb6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb80d18c0 con 0x7f4fb80ce7f0 2024-01-11T07:11:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fbcfb6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb80d0010 con 0x7f4fb8159ca0 2024-01-11T07:11:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fbcfb6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80cf400 con 0x7f4fb815a070 2024-01-11T07:11:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4fb8159ca0 0x7f4fb806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb6d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4fb80ce7f0 0x7f4fb80cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44484/0 (socket says 172.21.15.7:44484) 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb6d9d700 1 -- 172.21.15.7:0/2225504175 learned_addr learned my addr 172.21.15.7:0/2225504175 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb5d9b700 1 --2- 172.21.15.7:0/2225504175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb815a070 0x7f4fb806e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb659c700 1 -- 172.21.15.7:0/2225504175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb815a070 msgr2=0x7f4fb806e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb659c700 1 --2- 172.21.15.7:0/2225504175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb815a070 0x7f4fb806e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb659c700 1 -- 172.21.15.7:0/2225504175 >> v1:172.21.15.7:6789/0 conn(0x7f4fb80ce7f0 legacy=0x7f4fb80cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb659c700 1 -- 172.21.15.7:0/2225504175 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb80cf6e0 con 0x7f4fb8159ca0 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb5d9b700 1 --2- 172.21.15.7:0/2225504175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb815a070 0x7f4fb806e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb659c700 1 --2- 172.21.15.7:0/2225504175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb8159ca0 0x7f4fb806dc30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80098e0 tx=0x7f4fa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46dfa2eced3d963 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.729+0000 7f4fb559a700 1 -- 172.21.15.7:0/2225504175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8009e80 con 0x7f4fb8159ca0 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fb559a700 1 -- 172.21.15.7:0/2225504175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa800b860 con 0x7f4fb8159ca0 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fb559a700 1 -- 172.21.15.7:0/2225504175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa800bbe0 con 0x7f4fb8159ca0 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/2225504175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb8159ca0 msgr2=0x7f4fb806dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/2225504175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb8159ca0 0x7f4fb806dc30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80098e0 tx=0x7f4fa80091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/2225504175 shutdown_connections 2024-01-11T07:11:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/2225504175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb8159ca0 0x7f4fb806dc30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/2225504175 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb815a070 0x7f4fb806e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/2225504175 >> 172.21.15.7:0/2225504175 conn(0x7f4fb8063cc0 msgr2=0x7f4fb80d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/2225504175 shutdown_connections 2024-01-11T07:11:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.730+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/2225504175 wait complete. 2024-01-11T07:11:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.732+0000 7f4fbcfb6700 1 Processor -- start 2024-01-11T07:11:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.732+0000 7f4fbcfb6700 1 -- start start 2024-01-11T07:11:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.732+0000 7f4fbcfb6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb80ce7f0 0x7f4fb80b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.732+0000 7f4fbcfb6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb8159ca0 0x7f4fb816add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.732+0000 7f4fbcfb6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb815a070 0x7f4fb815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.732+0000 7f4fbcfb6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80b68c0 con 0x7f4fb815a070 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.732+0000 7f4fbcfb6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb815c9b0 con 0x7f4fb8159ca0 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.732+0000 7f4fbcfb6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb816b310 con 0x7f4fb80ce7f0 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb815a070 0x7f4fb815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb815a070 0x7f4fb815ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33210/0 (socket says 172.21.15.7:33210) 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 -- 172.21.15.7:0/4145611202 learned_addr learned my addr 172.21.15.7:0/4145611202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb659c700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb80ce7f0 0x7f4fb80b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb5d9b700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb8159ca0 0x7f4fb816add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 -- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb80ce7f0 msgr2=0x7f4fb80b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb80ce7f0 0x7f4fb80b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 -- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb8159ca0 msgr2=0x7f4fb816add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb8159ca0 0x7f4fb816add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 -- 172.21.15.7:0/4145611202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa8009420 con 0x7f4fb815a070 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb5d9b700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb8159ca0 0x7f4fb816add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fb6d9d700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb815a070 0x7f4fb815ce40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000cc40 tx=0x7f4fb0001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fa67fc700 1 -- 172.21.15.7:0/4145611202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000a9c0 con 0x7f4fb815a070 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fa67fc700 1 -- 172.21.15.7:0/4145611202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb000ab20 con 0x7f4fb815a070 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb80b4140 con 0x7f4fb815a070 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fa67fc700 1 -- 172.21.15.7:0/4145611202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00070f0 con 0x7f4fb815a070 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.733+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb815c640 con 0x7f4fb815a070 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.734+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb815cb40 con 0x7f4fb815a070 2024-01-11T07:11:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.735+0000 7f4fa67fc700 1 -- 172.21.15.7:0/4145611202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4fb00074a0 con 0x7f4fb815a070 2024-01-11T07:11:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.735+0000 7f4fa67fc700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f98066f30 0x7f4f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.736+0000 7f4fa67fc700 1 -- 172.21.15.7:0/4145611202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4fb0084940 con 0x7f4fb815a070 2024-01-11T07:11:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.736+0000 7f4fb659c700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f98066f30 0x7f4f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.736+0000 7f4fb659c700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f98066f30 0x7f4f980693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80091e0 tx=0x7f4fa8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.740+0000 7f4fa67fc700 1 -- 172.21.15.7:0/4145611202 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fb0051c30 con 0x7f4fb815a070 2024-01-11T07:11:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.995+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4fb804c3c0 con 0x7f4fb815a070 2024-01-11T07:11:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.995+0000 7f4fa67fc700 1 -- 172.21.15.7:0/4145611202 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4fb0055250 con 0x7f4fb815a070 2024-01-11T07:11:03.995 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:11:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f98066f30 msgr2=0x7f4f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f98066f30 0x7f4f980693e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80091e0 tx=0x7f4fa8017000 comp rx=0 tx=0).stop 2024-01-11T07:11:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb815a070 msgr2=0x7f4fb815ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb815a070 0x7f4fb815ce40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000cc40 tx=0x7f4fb0001a50 comp rx=0 tx=0).stop 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 shutdown_connections 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f98066f30 0x7f4f980693e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb815a070 0x7f4fb815ce40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4fb80ce7f0 0x7f4fb80b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.997+0000 7f4fbcfb6700 1 --2- 172.21.15.7:0/4145611202 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fb8159ca0 0x7f4fb816add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.998+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 >> 172.21.15.7:0/4145611202 conn(0x7f4fb8063cc0 msgr2=0x7f4fb80c6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.998+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 shutdown_connections 2024-01-11T07:11:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:03.998+0000 7f4fbcfb6700 1 -- 172.21.15.7:0/4145611202 wait complete. 2024-01-11T07:11:04.010 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:11:04.010 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:11:04.011 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:11:04.011 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:11:04.011 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:11:04.011 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:11:04.011 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:11:04.011 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:11:04.011 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:11:04.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:04.011 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:11:04.033 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:11:04.033 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:11:04.033 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:11:04.034 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42196sec preferred_lft 42196sec 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:04.035 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:11:04.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:04.037 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:11:04.104 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:11:04.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:04.105 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:11:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:11:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:11:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:11:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:11:04.167 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:11:04.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:04.168 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:11:04.168 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:11:04.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:11:04.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:11:04.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:11:04.168 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:11:04.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:04.278 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:11:04.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:11:04.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:11:04.296 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:11:04.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:04.296 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:11:04.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:11:04.365 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:11:04.422 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:11:04.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:11:04.423 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:11:04.478 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:11:04.478 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:11:04.479 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:11:04.560 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:11:04.560 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:11:04.560 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:11:04.569 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:11:04.569 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:11:04.569 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:11:04.569 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:11:04.569 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:11:04.586 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:11:04.586 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:11:04.586 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:11:04.586 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:11:04.586 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:11:04.586 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:11:04.587 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:11:04.587 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:11:04.588 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:11:04.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:04.589 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:11:04.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:11:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.797+0000 7ff7674f0700 1 Processor -- start 2024-01-11T07:11:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.797+0000 7ff7674f0700 1 -- start start 2024-01-11T07:11:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.798+0000 7ff7674f0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff76006afc0 0x7ff76015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.798+0000 7ff7674f0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76015b770 con 0x7ff76006e0d0 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.798+0000 7ff7674f0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76015abf0 con 0x7ff76006dd00 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.798+0000 7ff7674f0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76015aed0 con 0x7ff76006afc0 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.798+0000 7ff765a8d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff76006afc0 0x7ff76015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.798+0000 7ff764a8b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff76006e0d0 0x7ff760159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44490/0 (socket says 172.21.15.7:44490) 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.798+0000 7ff765a8d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff76006afc0 0x7ff76015a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33224/0 (socket says 172.21.15.7:33224) 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.798+0000 7ff764a8b700 1 -- 172.21.15.7:0/2650317059 learned_addr learned my addr 172.21.15.7:0/2650317059 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.799+0000 7ff765a8d700 1 -- 172.21.15.7:0/2650317059 >> v1:172.21.15.113:6789/0 conn(0x7ff76006dd00 legacy=0x7ff7600c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.799+0000 7ff765a8d700 1 -- 172.21.15.7:0/2650317059 >> v1:172.21.15.7:6789/0 conn(0x7ff76006e0d0 legacy=0x7ff760159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.799+0000 7ff765a8d700 1 -- 172.21.15.7:0/2650317059 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76015b1b0 con 0x7ff76006afc0 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.799+0000 7ff765a8d700 1 --2- 172.21.15.7:0/2650317059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff76015a1c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff758009e60 tx=0x7ff758009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a1608beb10a501 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.799+0000 7ff757fff700 1 -- 172.21.15.7:0/2650317059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75800e070 con 0x7ff76006afc0 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.799+0000 7ff757fff700 1 -- 172.21.15.7:0/2650317059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75800bc30 con 0x7ff76006afc0 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.799+0000 7ff757fff700 1 -- 172.21.15.7:0/2650317059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758012430 con 0x7ff76006afc0 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.800+0000 7ff7674f0700 1 -- 172.21.15.7:0/2650317059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 msgr2=0x7ff76015a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.800+0000 7ff7674f0700 1 --2- 172.21.15.7:0/2650317059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff76015a1c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff758009e60 tx=0x7ff758009780 comp rx=0 tx=0).stop 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.800+0000 7ff7674f0700 1 -- 172.21.15.7:0/2650317059 shutdown_connections 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.800+0000 7ff7674f0700 1 --2- 172.21.15.7:0/2650317059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff76015a1c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.800+0000 7ff7674f0700 1 -- 172.21.15.7:0/2650317059 >> 172.21.15.7:0/2650317059 conn(0x7ff7600680e0 msgr2=0x7ff7600cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.800+0000 7ff7674f0700 1 -- 172.21.15.7:0/2650317059 shutdown_connections 2024-01-11T07:11:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.800+0000 7ff7674f0700 1 -- 172.21.15.7:0/2650317059 wait complete. 2024-01-11T07:11:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.801+0000 7ff7674f0700 1 Processor -- start 2024-01-11T07:11:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.801+0000 7ff7674f0700 1 -- start start 2024-01-11T07:11:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff7674f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff7600ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff7674f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff76006dd00 0x7ff7600a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff7674f0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff76006e0d0 0x7ff7600aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff7674f0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600ae920 con 0x7ff76006afc0 2024-01-11T07:11:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff7674f0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600a6820 con 0x7ff76006e0d0 2024-01-11T07:11:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff7674f0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600adca0 con 0x7ff76006dd00 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff76528c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff7600ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff76528c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff7600ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33236/0 (socket says 172.21.15.7:33236) 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff76528c700 1 -- 172.21.15.7:0/4285197607 learned_addr learned my addr 172.21.15.7:0/4285197607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff765a8d700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff76006e0d0 0x7ff7600aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff764a8b700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff76006dd00 0x7ff7600a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff76528c700 1 -- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff76006dd00 msgr2=0x7ff7600a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff76528c700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff76006dd00 0x7ff7600a73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff76528c700 1 -- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff76006e0d0 msgr2=0x7ff7600aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.802+0000 7ff76528c700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff76006e0d0 0x7ff7600aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff76528c700 1 -- 172.21.15.7:0/4285197607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758009a20 con 0x7ff76006afc0 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff765a8d700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff76006e0d0 0x7ff7600aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff764a8b700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff76006dd00 0x7ff7600a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff76528c700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff7600ad680 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c590 tx=0x7ff75c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff7557fa700 1 -- 172.21.15.7:0/4285197607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c011590 con 0x7ff76006afc0 2024-01-11T07:11:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff7557fa700 1 -- 172.21.15.7:0/4285197607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c0116f0 con 0x7ff76006afc0 2024-01-11T07:11:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff7557fa700 1 -- 172.21.15.7:0/4285197607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c011ab0 con 0x7ff76006afc0 2024-01-11T07:11:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7600adf80 con 0x7ff76006afc0 2024-01-11T07:11:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.803+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7600ae540 con 0x7ff76006afc0 2024-01-11T07:11:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.804+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76004c3c0 con 0x7ff76006afc0 2024-01-11T07:11:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.805+0000 7ff7557fa700 1 -- 172.21.15.7:0/4285197607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff75c01b3b0 con 0x7ff76006afc0 2024-01-11T07:11:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.805+0000 7ff7557fa700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.805+0000 7ff7557fa700 1 -- 172.21.15.7:0/4285197607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff75c083a70 con 0x7ff76006afc0 2024-01-11T07:11:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.805+0000 7ff764a8b700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.806+0000 7ff764a8b700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff744066f30 0x7ff7440693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7600a7c40 tx=0x7ff750009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:04.809+0000 7ff7557fa700 1 -- 172.21.15.7:0/4285197607 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff75c050ce0 con 0x7ff76006afc0 2024-01-11T07:11:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.059+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff76015b490 con 0x7ff76006afc0 2024-01-11T07:11:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.059+0000 7ff7557fa700 1 -- 172.21.15.7:0/4285197607 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4599 (secure 0 0 0) 0x7ff75c017070 con 0x7ff76006afc0 2024-01-11T07:11:05.059 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:05.059 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2882608277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2882608277},{"type":"v1","addr":"172.21.15.7:6833","nonce":2882608277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3510440823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3510440823},{"type":"v1","addr":"172.21.15.7:6835","nonce":3510440823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3108557963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3108557963},{"type":"v1","addr":"172.21.15.113:6836","nonce":3108557963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:10:55.765305+0000","modified":"2024-01-11T07:10:59.370619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1888585107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1888585107},{"type":"v1","addr":"172.21.15.113:6837","nonce":1888585107}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-11T07:11:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff744066f30 msgr2=0x7ff7440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff744066f30 0x7ff7440693e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7600a7c40 tx=0x7ff750009040 comp rx=0 tx=0).stop 2024-01-11T07:11:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 msgr2=0x7ff7600ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff7600ad680 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c590 tx=0x7ff75c00c950 comp rx=0 tx=0).stop 2024-01-11T07:11:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 shutdown_connections 2024-01-11T07:11:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff744066f30 0x7ff7440693e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff76006e0d0 0x7ff7600aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff76006afc0 0x7ff7600ad680 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 --2- 172.21.15.7:0/4285197607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff76006dd00 0x7ff7600a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 >> 172.21.15.7:0/4285197607 conn(0x7ff7600680e0 msgr2=0x7ff760063230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.061+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 shutdown_connections 2024-01-11T07:11:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:05.062+0000 7ff7674f0700 1 -- 172.21.15.7:0/4285197607 wait complete. 2024-01-11T07:11:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 41 2024-01-11T07:11:05.075 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-01-11T07:11:05.211 INFO:teuthology.orchestra.run.smithi113.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":6015,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8389710400000006,"mdsmap_epoch":41,"osdmap_epoch":84,"osdmap_epoch_barrier":84,"uptime":14.687680358} 2024-01-11T07:11:05.211 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 6015, "mdsmap_epoch": 41, "osdmap_epoch": 84, "osdmap_epoch_barrier": 84, "rank_uptime": 8.83897104, "state": "up:active", "uptime": 14.687680358, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:11:05.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-11T07:11:05.365 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-11T07:11:05.365 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-11T07:11:05.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-01-11T07:11:05.505 INFO:teuthology.orchestra.run.smithi007.stdout:{"mon_tick_interval":"5"} 2024-01-11T07:11:05.506 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-01-11T07:11:05.506 INFO:tasks.cephfs.test_full:pool capacity 557041216, 531MB should be enough to fill it 2024-01-11T07:11:05.506 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:11:05.506 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-01-11T07:11:05.506 DEBUG:teuthology.orchestra.run.smithi007:> import datetime 2024-01-11T07:11:05.506 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-11T07:11:05.506 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:11:05.506 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> # Write some buffered data through before going full, all should be well 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> print("writing some data through which we expect to succeed") 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> bytes = 0 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> # buffered IO, the error should always only appear from fsync and not 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> # from write 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> full = False 2024-01-11T07:11:05.507 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(531 * 1.1)): 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote bytes via buffered write, moving on to fsync") 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> raise 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> full = True 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-11T07:11:05.508 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed successfully") 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> full = True 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> if n > 531 * 0.9: 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> # Be cautious in the last region where we expect to hit 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-01-11T07:11:05.509 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(15.0) 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> if not full: 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> # close() should not raise an error because we already caught it in 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> # since then because all IOs got cancelled on the full flag. 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> print("calling close") 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> print("close() did not raise error") 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-01-11T07:11:05.510 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:11:10.758 INFO:teuthology.orchestra.run.smithi007.stdout:writing some data through which we expect to succeed 2024-01-11T07:11:10.758 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-01-11T07:11:10.758 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.758 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.758 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 1.00 MB 2024-01-11T07:11:10.758 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.758 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.758 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 2.00 MB 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 3.00 MB 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 4.00 MB 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 5.00 MB 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.759 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 6.00 MB 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 7.00 MB 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 8.00 MB 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 9.00 MB 2024-01-11T07:11:10.760 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 10.00 MB 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 11.00 MB 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 12.00 MB 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 13.00 MB 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 14.00 MB 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 15.00 MB 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 16.00 MB 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 17.00 MB 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 18.00 MB 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 19.00 MB 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 20.00 MB 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 21.00 MB 2024-01-11T07:11:10.763 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 22.00 MB 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 23.00 MB 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 24.00 MB 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.764 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 25.00 MB 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 26.00 MB 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 27.00 MB 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 28.00 MB 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.765 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 29.00 MB 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 30.00 MB 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 31.00 MB 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 32.00 MB 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 33.00 MB 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.766 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 34.00 MB 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 35.00 MB 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 36.00 MB 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 37.00 MB 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 38.00 MB 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.767 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 39.00 MB 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 40.00 MB 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 41.00 MB 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 42.00 MB 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.768 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 43.00 MB 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 44.00 MB 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 45.00 MB 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 46.00 MB 2024-01-11T07:11:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 47.00 MB 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 48.00 MB 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 49.00 MB 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 50.00 MB 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.770 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 51.00 MB 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 52.00 MB 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 53.00 MB 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 54.00 MB 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.771 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 55.00 MB 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 56.00 MB 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 57.00 MB 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 58.00 MB 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 59.00 MB 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 60.00 MB 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 61.00 MB 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 62.00 MB 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 63.00 MB 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.773 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 64.00 MB 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 65.00 MB 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 66.00 MB 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 67.00 MB 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 68.00 MB 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.774 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 69.00 MB 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 70.00 MB 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 71.00 MB 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 72.00 MB 2024-01-11T07:11:10.775 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 73.00 MB 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 74.00 MB 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 75.00 MB 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 76.00 MB 2024-01-11T07:11:10.776 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 77.00 MB 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 78.00 MB 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 79.00 MB 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.777 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 80.00 MB 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 81.00 MB 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 82.00 MB 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 83.00 MB 2024-01-11T07:11:10.979 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 84.00 MB 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 85.00 MB 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 86.00 MB 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 87.00 MB 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.980 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 88.00 MB 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 89.00 MB 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 90.00 MB 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 91.00 MB 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 92.00 MB 2024-01-11T07:11:10.981 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 93.00 MB 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 94.00 MB 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 95.00 MB 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 96.00 MB 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.982 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 97.00 MB 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 98.00 MB 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 99.00 MB 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 100.00 MB 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 101.00 MB 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.983 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 102.00 MB 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 103.00 MB 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 104.00 MB 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 105.00 MB 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 106.00 MB 2024-01-11T07:11:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 107.00 MB 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 108.00 MB 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 109.00 MB 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 110.00 MB 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 111.00 MB 2024-01-11T07:11:10.985 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 112.00 MB 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 113.00 MB 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 114.00 MB 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 115.00 MB 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 116.00 MB 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 117.00 MB 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 118.00 MB 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 119.00 MB 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 120.00 MB 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.987 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 121.00 MB 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 122.00 MB 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 123.00 MB 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 124.00 MB 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 125.00 MB 2024-01-11T07:11:10.988 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 126.00 MB 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 127.00 MB 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 128.00 MB 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 129.00 MB 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 130.00 MB 2024-01-11T07:11:10.989 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 131.00 MB 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 132.00 MB 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 133.00 MB 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 134.00 MB 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.990 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 135.00 MB 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 136.00 MB 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 137.00 MB 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 138.00 MB 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 139.00 MB 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.991 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 140.00 MB 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 141.00 MB 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 142.00 MB 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 143.00 MB 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 144.00 MB 2024-01-11T07:11:10.992 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 145.00 MB 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 146.00 MB 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 147.00 MB 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 148.00 MB 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 149.00 MB 2024-01-11T07:11:10.993 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 150.00 MB 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 151.00 MB 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 152.00 MB 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 153.00 MB 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.994 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 154.00 MB 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 155.00 MB 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 156.00 MB 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 157.00 MB 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 158.00 MB 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.995 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 159.00 MB 2024-01-11T07:11:10.996 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.996 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.996 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 160.00 MB 2024-01-11T07:11:10.996 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:10.996 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:10.996 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 161.00 MB 2024-01-11T07:11:11.192 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.192 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.192 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 162.00 MB 2024-01-11T07:11:11.192 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 163.00 MB 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 164.00 MB 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 165.00 MB 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 166.00 MB 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 167.00 MB 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 168.00 MB 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 169.00 MB 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 170.00 MB 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 171.00 MB 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 172.00 MB 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 173.00 MB 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 174.00 MB 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 175.00 MB 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 176.00 MB 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 177.00 MB 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 178.00 MB 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 179.00 MB 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 180.00 MB 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 181.00 MB 2024-01-11T07:11:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 182.00 MB 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 183.00 MB 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 184.00 MB 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 185.00 MB 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 186.00 MB 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 187.00 MB 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 188.00 MB 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 189.00 MB 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 190.00 MB 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.198 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 191.00 MB 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 192.00 MB 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 193.00 MB 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 194.00 MB 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 195.00 MB 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.199 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 196.00 MB 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 197.00 MB 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 198.00 MB 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 199.00 MB 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.200 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 200.00 MB 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 201.00 MB 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 202.00 MB 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 203.00 MB 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 204.00 MB 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 205.00 MB 2024-01-11T07:11:11.201 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 206.00 MB 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 207.00 MB 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 208.00 MB 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 209.00 MB 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 210.00 MB 2024-01-11T07:11:11.202 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 211.00 MB 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 212.00 MB 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 213.00 MB 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 214.00 MB 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.203 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 215.00 MB 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 216.00 MB 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 217.00 MB 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 218.00 MB 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 219.00 MB 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.204 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 220.00 MB 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 221.00 MB 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 222.00 MB 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 223.00 MB 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 224.00 MB 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.205 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 225.00 MB 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 226.00 MB 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 227.00 MB 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 228.00 MB 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 229.00 MB 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.206 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 230.00 MB 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 231.00 MB 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 232.00 MB 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 233.00 MB 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 234.00 MB 2024-01-11T07:11:11.207 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 235.00 MB 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 236.00 MB 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 237.00 MB 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 238.00 MB 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 239.00 MB 2024-01-11T07:11:11.209 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.209 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.209 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 240.00 MB 2024-01-11T07:11:11.209 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.209 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.209 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 241.00 MB 2024-01-11T07:11:11.401 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.401 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.401 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 242.00 MB 2024-01-11T07:11:11.401 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.401 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 243.00 MB 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 244.00 MB 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 245.00 MB 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 246.00 MB 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 247.00 MB 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 248.00 MB 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 249.00 MB 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 250.00 MB 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 251.00 MB 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.403 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 252.00 MB 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 253.00 MB 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 254.00 MB 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 255.00 MB 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 256.00 MB 2024-01-11T07:11:11.404 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 257.00 MB 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 258.00 MB 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 259.00 MB 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 260.00 MB 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 261.00 MB 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 262.00 MB 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 263.00 MB 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 264.00 MB 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 265.00 MB 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.406 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 266.00 MB 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 267.00 MB 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 268.00 MB 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 269.00 MB 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 270.00 MB 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.407 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 271.00 MB 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 272.00 MB 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 273.00 MB 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 274.00 MB 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.408 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 275.00 MB 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 276.00 MB 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 277.00 MB 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 278.00 MB 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 279.00 MB 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.409 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 280.00 MB 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 281.00 MB 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 282.00 MB 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 283.00 MB 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 284.00 MB 2024-01-11T07:11:11.410 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 285.00 MB 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 286.00 MB 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 287.00 MB 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 288.00 MB 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.411 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 289.00 MB 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 290.00 MB 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 291.00 MB 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 292.00 MB 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 293.00 MB 2024-01-11T07:11:11.412 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 294.00 MB 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 295.00 MB 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 296.00 MB 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 297.00 MB 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.413 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 298.00 MB 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 299.00 MB 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 300.00 MB 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 301.00 MB 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 302.00 MB 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.414 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 303.00 MB 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 304.00 MB 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 305.00 MB 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 306.00 MB 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 307.00 MB 2024-01-11T07:11:11.415 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 308.00 MB 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 309.00 MB 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 310.00 MB 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 311.00 MB 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.416 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 312.00 MB 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 313.00 MB 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 314.00 MB 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 315.00 MB 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 316.00 MB 2024-01-11T07:11:11.417 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 317.00 MB 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 318.00 MB 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 319.00 MB 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 320.00 MB 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.418 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 321.00 MB 2024-01-11T07:11:11.419 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.419 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 322.00 MB 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 323.00 MB 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 324.00 MB 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 325.00 MB 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 326.00 MB 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 327.00 MB 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 328.00 MB 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 329.00 MB 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 330.00 MB 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 331.00 MB 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 332.00 MB 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 333.00 MB 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 334.00 MB 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.617 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 335.00 MB 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 336.00 MB 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 337.00 MB 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 338.00 MB 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 339.00 MB 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.618 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 340.00 MB 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 341.00 MB 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 342.00 MB 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 343.00 MB 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 344.00 MB 2024-01-11T07:11:11.619 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 345.00 MB 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 346.00 MB 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 347.00 MB 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 348.00 MB 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.620 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 349.00 MB 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 350.00 MB 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 351.00 MB 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 352.00 MB 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 353.00 MB 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 354.00 MB 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 355.00 MB 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 356.00 MB 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 357.00 MB 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 358.00 MB 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.622 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 359.00 MB 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 360.00 MB 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 361.00 MB 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 362.00 MB 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 363.00 MB 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.623 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 364.00 MB 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 365.00 MB 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 366.00 MB 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 367.00 MB 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 368.00 MB 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.624 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 369.00 MB 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 370.00 MB 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 371.00 MB 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 372.00 MB 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 373.00 MB 2024-01-11T07:11:11.625 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 374.00 MB 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 375.00 MB 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 376.00 MB 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 377.00 MB 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 378.00 MB 2024-01-11T07:11:11.626 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 379.00 MB 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 380.00 MB 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 381.00 MB 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 382.00 MB 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.627 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 383.00 MB 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 384.00 MB 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 385.00 MB 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 386.00 MB 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 387.00 MB 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.628 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 388.00 MB 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 389.00 MB 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 390.00 MB 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 391.00 MB 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 392.00 MB 2024-01-11T07:11:11.629 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 393.00 MB 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 394.00 MB 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 395.00 MB 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 396.00 MB 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 397.00 MB 2024-01-11T07:11:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 398.00 MB 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 399.00 MB 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 400.00 MB 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 401.00 MB 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:11.631 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 402.00 MB 2024-01-11T07:11:13.212 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:11:13.212+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2024-01-11T07:11:22.207 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:11:22.207+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-01-11T07:11:30.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:11:30.889 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:11:41.885 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 403.00 MB 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 404.00 MB 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 405.00 MB 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 406.00 MB 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.886 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 407.00 MB 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 408.00 MB 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 409.00 MB 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 410.00 MB 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 411.00 MB 2024-01-11T07:11:41.887 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 412.00 MB 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 413.00 MB 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 414.00 MB 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 415.00 MB 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 416.00 MB 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 417.00 MB 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 418.00 MB 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 419.00 MB 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 420.00 MB 2024-01-11T07:11:41.889 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 421.00 MB 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 422.00 MB 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 423.00 MB 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 424.00 MB 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.890 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 425.00 MB 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 426.00 MB 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 427.00 MB 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 428.00 MB 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 429.00 MB 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.891 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 430.00 MB 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 431.00 MB 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 432.00 MB 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 433.00 MB 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 434.00 MB 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.892 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 435.00 MB 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 436.00 MB 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 437.00 MB 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 438.00 MB 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 439.00 MB 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.893 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 440.00 MB 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 441.00 MB 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 442.00 MB 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 443.00 MB 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 444.00 MB 2024-01-11T07:11:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 445.00 MB 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 446.00 MB 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 447.00 MB 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 448.00 MB 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.895 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 449.00 MB 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 450.00 MB 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 451.00 MB 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 452.00 MB 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.896 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 453.00 MB 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 454.00 MB 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 455.00 MB 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 456.00 MB 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 457.00 MB 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.897 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 458.00 MB 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 459.00 MB 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 460.00 MB 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 461.00 MB 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 462.00 MB 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.898 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 463.00 MB 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 464.00 MB 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 465.00 MB 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 466.00 MB 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 467.00 MB 2024-01-11T07:11:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 468.00 MB 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 469.00 MB 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 470.00 MB 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 471.00 MB 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.900 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 472.00 MB 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 473.00 MB 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 474.00 MB 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 475.00 MB 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 476.00 MB 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.901 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 477.00 MB 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 478.00 MB 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 479.00 MB 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 480.00 MB 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:Reached fullness after 480.00 MB 2024-01-11T07:11:41.902 INFO:teuthology.orchestra.run.smithi007.stdout:calling close 2024-01-11T07:11:41.903 INFO:teuthology.orchestra.run.smithi007.stdout:close() did not raise error 2024-01-11T07:11:42.027 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:11:42.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:11:42.080 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:11:42.124 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:11:42.124 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:11:42.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:11:42.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:11:42.179 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:11:42.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:42.180 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:11:42.202 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:11:42.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:11:42.202 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:11:42.245 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:11:42.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:11:42.245 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:11:42.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:11:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.477+0000 7f863d691700 1 Processor -- start 2024-01-11T07:11:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.478+0000 7f863d691700 1 -- start start 2024-01-11T07:11:42.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f863d691700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86380cd6d0 0x7f86380cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f863d691700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f863815e450 0x7f8638159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f863d691700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863815acf0 con 0x7f863815d8a0 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f863d691700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863815afd0 con 0x7f86380cd6d0 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f863d691700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86380ce980 con 0x7f863815e450 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f8636ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86380cd6d0 0x7f86380cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f8636ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86380cd6d0 0x7f86380cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43914/0 (socket says 172.21.15.7:43914) 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f8636ffd700 1 -- 172.21.15.7:0/4011350118 learned_addr learned my addr 172.21.15.7:0/4011350118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.479+0000 7f86367fc700 1 --1- 172.21.15.7:0/4011350118 >> v1:172.21.15.113:6789/0 conn(0x7f863815d8a0 0x7f863815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:42798/0 (socket says 172.21.15.7:42798) 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.480+0000 7f8636ffd700 1 -- 172.21.15.7:0/4011350118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f863815e450 msgr2=0x7f8638159600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.480+0000 7f8636ffd700 1 --2- 172.21.15.7:0/4011350118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f863815e450 0x7f8638159600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.480+0000 7f8636ffd700 1 -- 172.21.15.7:0/4011350118 >> v1:172.21.15.113:6789/0 conn(0x7f863815d8a0 legacy=0x7f863815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.480+0000 7f8636ffd700 1 -- 172.21.15.7:0/4011350118 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86380cec60 con 0x7f86380cd6d0 2024-01-11T07:11:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.480+0000 7f8636ffd700 1 --2- 172.21.15.7:0/4011350118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f86380cdaa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f86300099b0 tx=0x7f8630009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4700f102019b0002 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.480+0000 7f8635ffb700 1 -- 172.21.15.7:0/4011350118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000b820 con 0x7f86380cd6d0 2024-01-11T07:11:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.480+0000 7f8635ffb700 1 -- 172.21.15.7:0/4011350118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863000b980 con 0x7f86380cd6d0 2024-01-11T07:11:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.480+0000 7f8635ffb700 1 -- 172.21.15.7:0/4011350118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000bd20 con 0x7f86380cd6d0 2024-01-11T07:11:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.481+0000 7f863d691700 1 -- 172.21.15.7:0/4011350118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 msgr2=0x7f86380cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.481+0000 7f863d691700 1 --2- 172.21.15.7:0/4011350118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f86380cdaa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f86300099b0 tx=0x7f8630009250 comp rx=0 tx=0).stop 2024-01-11T07:11:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.481+0000 7f863d691700 1 -- 172.21.15.7:0/4011350118 shutdown_connections 2024-01-11T07:11:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.481+0000 7f863d691700 1 --2- 172.21.15.7:0/4011350118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f863815e450 0x7f8638159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.481+0000 7f863d691700 1 --2- 172.21.15.7:0/4011350118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f86380cdaa0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.481+0000 7f863d691700 1 -- 172.21.15.7:0/4011350118 >> 172.21.15.7:0/4011350118 conn(0x7f8638071120 msgr2=0x7f863806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.482+0000 7f863d691700 1 -- 172.21.15.7:0/4011350118 shutdown_connections 2024-01-11T07:11:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.482+0000 7f863d691700 1 -- 172.21.15.7:0/4011350118 wait complete. 2024-01-11T07:11:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.484+0000 7f863d691700 1 Processor -- start 2024-01-11T07:11:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.485+0000 7f863d691700 1 -- start start 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f863d691700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f863814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f863d691700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863815d8a0 0x7f86380cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f863d691700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f863815e450 0x7f863816fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f863d691700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86380c7f70 con 0x7f86380cd6d0 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f863d691700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863816f930 con 0x7f863815d8a0 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f863d691700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863816f650 con 0x7f863815e450 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f8636ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f863814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f8636ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f863814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43918/0 (socket says 172.21.15.7:43918) 2024-01-11T07:11:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f8636ffd700 1 -- 172.21.15.7:0/2322844141 learned_addr learned my addr 172.21.15.7:0/2322844141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f8636ffd700 1 -- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f863815e450 msgr2=0x7f863816fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f8636ffd700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f863815e450 0x7f863816fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f8636ffd700 1 -- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863815d8a0 msgr2=0x7f86380cb3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-11T07:11:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f8636ffd700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863815d8a0 0x7f86380cb3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.486+0000 7f8636ffd700 1 -- 172.21.15.7:0/2322844141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86300094f0 con 0x7f86380cd6d0 2024-01-11T07:11:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.487+0000 7f8636ffd700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f863814f720 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8630009290 tx=0x7f8630016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.487+0000 7f862effd700 1 -- 172.21.15.7:0/2322844141 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630011d50 con 0x7f86380cd6d0 2024-01-11T07:11:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.487+0000 7f862effd700 1 -- 172.21.15.7:0/2322844141 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630020070 con 0x7f86380cd6d0 2024-01-11T07:11:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.487+0000 7f862effd700 1 -- 172.21.15.7:0/2322844141 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863001b6a0 con 0x7f86380cd6d0 2024-01-11T07:11:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.487+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86380ca8f0 con 0x7f86380cd6d0 2024-01-11T07:11:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.487+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86380caeb0 con 0x7f86380cd6d0 2024-01-11T07:11:42.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.490+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863804c3c0 con 0x7f86380cd6d0 2024-01-11T07:11:42.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.490+0000 7f862effd700 1 -- 172.21.15.7:0/2322844141 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8630022030 con 0x7f86380cd6d0 2024-01-11T07:11:42.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.491+0000 7f862effd700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618067080 0x7f8618069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.491+0000 7f862effd700 1 -- 172.21.15.7:0/2322844141 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f86300864f0 con 0x7f86380cd6d0 2024-01-11T07:11:42.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.491+0000 7f862effd700 1 -- 172.21.15.7:0/2322844141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f861806eba0 con 0x7f86380cd6d0 2024-01-11T07:11:42.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.491+0000 7f86367fc700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618067080 0x7f8618069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.492+0000 7f86367fc700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618067080 0x7f8618069530 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8638156b20 tx=0x7f8628004fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.499+0000 7f862effd700 1 -- 172.21.15.7:0/2322844141 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8630053740 con 0x7f86380cd6d0 2024-01-11T07:11:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.760+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86380cb0d0 con 0x7f86380cd6d0 2024-01-11T07:11:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.761+0000 7f862effd700 1 -- 172.21.15.7:0/2322844141 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f8630006d70 con 0x7f86380cd6d0 2024-01-11T07:11:42.762 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:42.762 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2882608277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2882608277},{"type":"v1","addr":"172.21.15.7:6833","nonce":2882608277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3510440823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3510440823},{"type":"v1","addr":"172.21.15.7:6835","nonce":3510440823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3108557963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3108557963},{"type":"v1","addr":"172.21.15.113:6836","nonce":3108557963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:10:55.765305+0000","modified":"2024-01-11T07:10:59.370619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1888585107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1888585107},{"type":"v1","addr":"172.21.15.113:6837","nonce":1888585107}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-11T07:11:42.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618067080 msgr2=0x7f8618069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:42.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618067080 0x7f8618069530 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8638156b20 tx=0x7f8628004fc0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 msgr2=0x7f863814f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f863814f720 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8630009290 tx=0x7f8630016000 comp rx=0 tx=0).stop 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 shutdown_connections 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8618067080 0x7f8618069530 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f863815e450 0x7f863816fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86380cd6d0 0x7f863814f720 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 --2- 172.21.15.7:0/2322844141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f863815d8a0 0x7f86380cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 >> 172.21.15.7:0/2322844141 conn(0x7f8638071120 msgr2=0x7f86380afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 shutdown_connections 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.763+0000 7f863d691700 1 -- 172.21.15.7:0/2322844141 wait complete. 2024-01-11T07:11:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 41 2024-01-11T07:11:42.778 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:11:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.911+0000 7f670fb31700 1 Processor -- start 2024-01-11T07:11:42.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.911+0000 7f670fb31700 1 -- start start 2024-01-11T07:11:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670fb31700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f670815a060 0x7f67080b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670fb31700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 0x7f67080b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670fb31700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67080ce9b0 con 0x7f6708159c90 2024-01-11T07:11:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670fb31700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080b1f30 con 0x7f670815a060 2024-01-11T07:11:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670fb31700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080cf530 con 0x7f6708069320 2024-01-11T07:11:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670d8cd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6708159c90 0x7f67080b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35594/0 (socket says 172.21.15.7:35594) 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670d8cd700 1 -- 172.21.15.7:0/1814202298 learned_addr learned my addr 172.21.15.7:0/1814202298 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670e0ce700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 0x7f67080b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.912+0000 7f670d0cc700 1 --2- 172.21.15.7:0/1814202298 >> v2:172.21.15.113:3300/0 conn(0x7f670815a060 0x7f67080b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158992524 0 0) 0x7f67080ce9b0 con 0x7f6708159c90 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f4003570 con 0x7f6708159c90 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1495316073 0 0) 0x7f66f4003570 con 0x7f6708159c90 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 msgr2=0x7f67080b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 0x7f67080b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 >> v2:172.21.15.113:3300/0 conn(0x7f670815a060 msgr2=0x7f67080b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 --2- 172.21.15.7:0/1814202298 >> v2:172.21.15.113:3300/0 conn(0x7f670815a060 0x7f67080b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080cf9c0 con 0x7f6708159c90 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f6704001470 con 0x7f6708159c90 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670d0cc700 1 --2- 172.21.15.7:0/1814202298 >> v2:172.21.15.113:3300/0 conn(0x7f670815a060 0x7f67080b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670e0ce700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 0x7f67080b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:11:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 >> v1:172.21.15.7:6789/0 conn(0x7f6708159c90 legacy=0x7f67080b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 0x7f66f4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f66f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f4004d50 0x7f66f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080cf530 con 0x7f66f4003fa0 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f400ba60 con 0x7f66f4004d50 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f400bba0 con 0x7f6708069320 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670d0cc700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f66f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670d8cd700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 0x7f66f4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670e0ce700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f4004d50 0x7f66f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670d0cc700 1 -- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 msgr2=0x7f66f4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670d0cc700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 0x7f66f4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670d0cc700 1 -- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f4004d50 msgr2=0x7f66f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.913+0000 7f670d0cc700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f4004d50 0x7f66f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670d0cc700 1 -- 172.21.15.7:0/1814202298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700009a20 con 0x7f66f4003fa0 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670d8cd700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6708069320 0x7f66f4003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670e0ce700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66f4004d50 0x7f66f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670d0cc700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f66f4004810 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009df0 tx=0x7f66f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa09c96b491ac77 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800e920 con 0x7f66f4003fa0 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f800ea80 con 0x7f66f4003fa0 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670c8cb700 1 -- 172.21.15.7:0/1814202298 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800ee20 con 0x7f66f4003fa0 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670fb31700 1 -- 172.21.15.7:0/1814202298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 msgr2=0x7f66f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670fb31700 1 --2- 172.21.15.7:0/1814202298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f66f4004810 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009df0 tx=0x7f66f80091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670e0ce700 1 -- 172.21.15.7:0/1814202298 reap_dead start 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670fb31700 1 -- 172.21.15.7:0/1814202298 shutdown_connections 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.914+0000 7f670fb31700 1 -- 172.21.15.7:0/1814202298 >> 172.21.15.7:0/1814202298 conn(0x7f6708071100 msgr2=0x7f670806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.915+0000 7f670fb31700 1 -- 172.21.15.7:0/1814202298 shutdown_connections 2024-01-11T07:11:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.915+0000 7f670fb31700 1 -- 172.21.15.7:0/1814202298 wait complete. 2024-01-11T07:11:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.916+0000 7f670fb31700 1 Processor -- start 2024-01-11T07:11:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.916+0000 7f670fb31700 1 -- start start 2024-01-11T07:11:42.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.916+0000 7f670fb31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f67080cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.916+0000 7f670fb31700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67080ae7b0 0x7f670816da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670fb31700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f670816df90 0x7f670815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670fb31700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708150090 con 0x7f66f4003fa0 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670fb31700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080cd950 con 0x7f670816df90 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670fb31700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080ae320 con 0x7f67080ae7b0 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f67080cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f67080cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43938/0 (socket says 172.21.15.7:43938) 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 -- 172.21.15.7:0/3474676720 learned_addr learned my addr 172.21.15.7:0/3474676720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d0cc700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67080ae7b0 0x7f670816da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670e0ce700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f670816df90 0x7f670815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 -- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67080ae7b0 msgr2=0x7f670816da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67080ae7b0 0x7f670816da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 -- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f670816df90 msgr2=0x7f670815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f670816df90 0x7f670815c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 -- 172.21.15.7:0/3474676720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080afaf0 con 0x7f66f4003fa0 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670d8cd700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f67080cd410 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6704007fd0 tx=0x7f6704001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f66fdffb700 1 -- 172.21.15.7:0/3474676720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704018030 con 0x7f66f4003fa0 2024-01-11T07:11:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f66fdffb700 1 -- 172.21.15.7:0/3474676720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670400f920 con 0x7f66f4003fa0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670816e5a0 con 0x7f66f4003fa0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.917+0000 7f66fdffb700 1 -- 172.21.15.7:0/3474676720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400fce0 con 0x7f66f4003fa0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.918+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67080adcd0 con 0x7f66f4003fa0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.919+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6708159520 con 0x7f66f4003fa0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.919+0000 7f66fdffb700 1 -- 172.21.15.7:0/3474676720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f670401b070 con 0x7f66f4003fa0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.919+0000 7f66fdffb700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66ec066f30 0x7f66ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.919+0000 7f66fdffb700 1 -- 172.21.15.7:0/3474676720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6704019070 con 0x7f66f4003fa0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.919+0000 7f66fdffb700 1 -- 172.21.15.7:0/3474676720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f66ec06ea50 con 0x7f66f4003fa0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.919+0000 7f670d0cc700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66ec066f30 0x7f66ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.920+0000 7f670d0cc700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66ec066f30 0x7f66ec0693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f67080af570 tx=0x7f66f801a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:42.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:42.924+0000 7f66fdffb700 1 -- 172.21.15.7:0/3474676720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6704051660 con 0x7f66f4003fa0 2024-01-11T07:11:43.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.128+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f670804c3c0 con 0x7f66f4003fa0 2024-01-11T07:11:43.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.129+0000 7f66fdffb700 1 -- 172.21.15.7:0/3474676720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+15348 (secure 0 0 0) 0x7f6704054c80 con 0x7f66f4003fa0 2024-01-11T07:11:43.130 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:43.130 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:11:16.228396+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:10:53.813087+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-11T07:10:54.744603+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:11:43.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66ec066f30 msgr2=0x7f66ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66ec066f30 0x7f66ec0693e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f67080af570 tx=0x7f66f801a000 comp rx=0 tx=0).stop 2024-01-11T07:11:43.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 msgr2=0x7f67080cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f67080cd410 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6704007fd0 tx=0x7f6704001e90 comp rx=0 tx=0).stop 2024-01-11T07:11:43.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 shutdown_connections 2024-01-11T07:11:43.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66ec066f30 0x7f66ec0693e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f670816df90 0x7f670815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66f4003fa0 0x7f67080cd410 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 --2- 172.21.15.7:0/3474676720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67080ae7b0 0x7f670816da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 >> 172.21.15.7:0/3474676720 conn(0x7f6708071100 msgr2=0x7f67080b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:43.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 shutdown_connections 2024-01-11T07:11:43.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.131+0000 7f670fb31700 1 -- 172.21.15.7:0/3474676720 wait complete. 2024-01-11T07:11:43.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:10:53.813087+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:10:54.744603+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:11:43.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:11:43.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.280+0000 7fd2998dc700 1 Processor -- start 2024-01-11T07:11:43.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.280+0000 7fd2998dc700 1 -- start start 2024-01-11T07:11:43.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2998dc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd294159ea0 0x7fd2940af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2998dc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2940cc0d0 0x7fd2940cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2998dc700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940cee70 con 0x7fd2940cdf10 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2998dc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940ce860 con 0x7fd2940cc0d0 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2998dc700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940b0100 con 0x7fd294159ea0 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2940cc0d0 0x7fd2940cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2940cc0d0 0x7fd2940cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43954/0 (socket says 172.21.15.7:43954) 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2937fe700 1 -- 172.21.15.7:0/2233488653 learned_addr learned my addr 172.21.15.7:0/2233488653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd292ffd700 1 --2- 172.21.15.7:0/2233488653 >> v2:172.21.15.113:3300/0 conn(0x7fd294159ea0 0x7fd2940af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2937fe700 1 -- 172.21.15.7:0/2233488653 >> v2:172.21.15.113:3300/0 conn(0x7fd294159ea0 msgr2=0x7fd2940af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2937fe700 1 --2- 172.21.15.7:0/2233488653 >> v2:172.21.15.113:3300/0 conn(0x7fd294159ea0 0x7fd2940af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2937fe700 1 -- 172.21.15.7:0/2233488653 >> v1:172.21.15.113:6789/0 conn(0x7fd2940cdf10 legacy=0x7fd2940cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.281+0000 7fd2937fe700 1 -- 172.21.15.7:0/2233488653 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940ceb00 con 0x7fd2940cc0d0 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2937fe700 1 --2- 172.21.15.7:0/2233488653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940cf300 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd2940678d0 tx=0x7fd28c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dfd7f8723afea9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd291ffb700 1 -- 172.21.15.7:0/2233488653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00e070 con 0x7fd2940cc0d0 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd291ffb700 1 -- 172.21.15.7:0/2233488653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c00ba10 con 0x7fd2940cc0d0 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd291ffb700 1 -- 172.21.15.7:0/2233488653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c0092e0 con 0x7fd2940cc0d0 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2998dc700 1 -- 172.21.15.7:0/2233488653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 msgr2=0x7fd2940cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2998dc700 1 --2- 172.21.15.7:0/2233488653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940cf300 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd2940678d0 tx=0x7fd28c009840 comp rx=0 tx=0).stop 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2998dc700 1 -- 172.21.15.7:0/2233488653 shutdown_connections 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2998dc700 1 --2- 172.21.15.7:0/2233488653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940cf300 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2998dc700 1 --2- 172.21.15.7:0/2233488653 >> v2:172.21.15.113:3300/0 conn(0x7fd294159ea0 0x7fd2940af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2998dc700 1 -- 172.21.15.7:0/2233488653 >> 172.21.15.7:0/2233488653 conn(0x7fd294071120 msgr2=0x7fd29406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2998dc700 1 -- 172.21.15.7:0/2233488653 shutdown_connections 2024-01-11T07:11:43.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.282+0000 7fd2998dc700 1 -- 172.21.15.7:0/2233488653 wait complete. 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.284+0000 7fd2998dc700 1 Processor -- start 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.284+0000 7fd2998dc700 1 -- start start 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2998dc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2998dc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2940cdf10 0x7fd2941565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2998dc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd294159ea0 0x7fd2940af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2998dc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940b3050 con 0x7fd2940cc0d0 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2998dc700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294156af0 con 0x7fd2940cdf10 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2998dc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940b6730 con 0x7fd294159ea0 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43970/0 (socket says 172.21.15.7:43970) 2024-01-11T07:11:43.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 -- 172.21.15.7:0/4268114735 learned_addr learned my addr 172.21.15.7:0/4268114735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2937fe700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd294159ea0 0x7fd2940af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2927fc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2940cdf10 0x7fd2941565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 -- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd294159ea0 msgr2=0x7fd2940af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd294159ea0 0x7fd2940af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 -- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2940cdf10 msgr2=0x7fd2941565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2940cdf10 0x7fd2941565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 -- 172.21.15.7:0/4268114735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c009ae0 con 0x7fd2940cc0d0 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd2927fc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2940cdf10 0x7fd2941565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.285+0000 7fd292ffd700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940b5e70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00d890 tx=0x7fd27c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.286+0000 7fd28affd700 1 -- 172.21.15.7:0/4268114735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c009610 con 0x7fd2940cc0d0 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.286+0000 7fd28affd700 1 -- 172.21.15.7:0/4268114735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c009770 con 0x7fd2940cc0d0 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.286+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2940b6450 con 0x7fd2940cc0d0 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.286+0000 7fd28affd700 1 -- 172.21.15.7:0/4268114735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c009b30 con 0x7fd2940cc0d0 2024-01-11T07:11:43.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.286+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2940b2ce0 con 0x7fd2940cc0d0 2024-01-11T07:11:43.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.287+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2940695e0 con 0x7fd2940cc0d0 2024-01-11T07:11:43.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.287+0000 7fd28affd700 1 -- 172.21.15.7:0/4268114735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd27c01e460 con 0x7fd2940cc0d0 2024-01-11T07:11:43.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.288+0000 7fd28affd700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd274066f30 0x7fd2740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.288+0000 7fd28affd700 1 -- 172.21.15.7:0/4268114735 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd27c084b80 con 0x7fd2940cc0d0 2024-01-11T07:11:43.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.288+0000 7fd2927fc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd274066f30 0x7fd2740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.288+0000 7fd28affd700 1 -- 172.21.15.7:0/4268114735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7fd27406ea50 con 0x7fd2940cc0d0 2024-01-11T07:11:43.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.288+0000 7fd2927fc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd274066f30 0x7fd2740693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd2940cd980 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.292+0000 7fd28affd700 1 -- 172.21.15.7:0/4268114735 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd27c084e60 con 0x7fd2940cc0d0 2024-01-11T07:11:43.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.499+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd29404c3c0 con 0x7fd2940cc0d0 2024-01-11T07:11:43.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.499+0000 7fd28affd700 1 -- 172.21.15.7:0/4268114735 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd27c051d50 con 0x7fd2940cc0d0 2024-01-11T07:11:43.500 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:11:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.501+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd274066f30 msgr2=0x7fd2740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.501+0000 7fd2998dc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd274066f30 0x7fd2740693e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd2940cd980 tx=0x7fd284009040 comp rx=0 tx=0).stop 2024-01-11T07:11:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.501+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 msgr2=0x7fd2940b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.501+0000 7fd2998dc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940b5e70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00d890 tx=0x7fd27c008040 comp rx=0 tx=0).stop 2024-01-11T07:11:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.502+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 shutdown_connections 2024-01-11T07:11:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.502+0000 7fd2998dc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd274066f30 0x7fd2740693e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.502+0000 7fd2998dc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd294159ea0 0x7fd2940af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.502+0000 7fd2998dc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2940cc0d0 0x7fd2940b5e70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.502+0000 7fd2998dc700 1 --2- 172.21.15.7:0/4268114735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2940cdf10 0x7fd2941565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.502+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 >> 172.21.15.7:0/4268114735 conn(0x7fd294071120 msgr2=0x7fd29415ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.502+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 shutdown_connections 2024-01-11T07:11:43.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.502+0000 7fd2998dc700 1 -- 172.21.15.7:0/4268114735 wait complete. 2024-01-11T07:11:43.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:11:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.652+0000 7fc4770e5700 1 Processor -- start 2024-01-11T07:11:43.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.652+0000 7fc4770e5700 1 -- start start 2024-01-11T07:11:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc4770e5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc47015a1e0 0x7fc47015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc4770e5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47006e110 con 0x7fc470159e10 2024-01-11T07:11:43.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc4770e5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4700b1c30 con 0x7fc4700b14c0 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc4770e5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700b2e40 con 0x7fc47015a1e0 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc474e81700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc470159e10 0x7fc4700ca2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35598/0 (socket says 172.21.15.7:35598) 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc474e81700 1 -- 172.21.15.7:0/3573249587 learned_addr learned my addr 172.21.15.7:0/3573249587 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46ffff700 1 --2- 172.21.15.7:0/3573249587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc47015a1e0 0x7fc47015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46f7fe700 1 -- 172.21.15.7:0/3573249587 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980641174 0 0) 0x7fc47006e110 con 0x7fc470159e10 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46f7fe700 1 -- 172.21.15.7:0/3573249587 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc460003570 con 0x7fc470159e10 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46f7fe700 1 -- 172.21.15.7:0/3573249587 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695880584 0 0) 0x7fc4700b1c30 con 0x7fc4700b14c0 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46f7fe700 1 -- 172.21.15.7:0/3573249587 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc47006e110 con 0x7fc4700b14c0 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46f7fe700 1 -- 172.21.15.7:0/3573249587 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1394497595 0 0) 0x7fc460003570 con 0x7fc470159e10 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46ffff700 1 -- 172.21.15.7:0/3573249587 >> v1:172.21.15.113:6789/0 conn(0x7fc4700b14c0 legacy=0x7fc47006e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46ffff700 1 -- 172.21.15.7:0/3573249587 >> v1:172.21.15.7:6789/0 conn(0x7fc470159e10 legacy=0x7fc4700ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.653+0000 7fc46ffff700 1 -- 172.21.15.7:0/3573249587 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700b1f10 con 0x7fc47015a1e0 2024-01-11T07:11:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc46ffff700 1 --2- 172.21.15.7:0/3573249587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc47015a1e0 0x7fc47015b1d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009910 tx=0x7fc45c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93353035c50df427 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc46f7fe700 1 -- 172.21.15.7:0/3573249587 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc464001470 con 0x7fc47015a1e0 2024-01-11T07:11:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc46f7fe700 1 -- 172.21.15.7:0/3573249587 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c011420 con 0x7fc47015a1e0 2024-01-11T07:11:43.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc46f7fe700 1 -- 172.21.15.7:0/3573249587 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0117c0 con 0x7fc47015a1e0 2024-01-11T07:11:43.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc4770e5700 1 -- 172.21.15.7:0/3573249587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc47015a1e0 msgr2=0x7fc47015b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc4770e5700 1 --2- 172.21.15.7:0/3573249587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc47015a1e0 0x7fc47015b1d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009910 tx=0x7fc45c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc4770e5700 1 -- 172.21.15.7:0/3573249587 shutdown_connections 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc4770e5700 1 --2- 172.21.15.7:0/3573249587 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc47015a1e0 0x7fc47015b1d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.654+0000 7fc4770e5700 1 -- 172.21.15.7:0/3573249587 >> 172.21.15.7:0/3573249587 conn(0x7fc4700cd910 msgr2=0x7fc4700cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.655+0000 7fc4770e5700 1 -- 172.21.15.7:0/3573249587 shutdown_connections 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.655+0000 7fc4770e5700 1 -- 172.21.15.7:0/3573249587 wait complete. 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.656+0000 7fc4770e5700 1 Processor -- start 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.656+0000 7fc4770e5700 1 -- start start 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc4770e5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4700b14c0 0x7fc4700cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc4770e5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc470159e10 0x7fc470154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc4770e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47015a1e0 0x7fc470156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc4770e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470155ae0 con 0x7fc47015a1e0 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc4770e5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700ae0a0 con 0x7fc4700b14c0 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc4770e5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4701552d0 con 0x7fc470159e10 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47015a1e0 0x7fc470156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47015a1e0 0x7fc470156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43978/0 (socket says 172.21.15.7:43978) 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc474e81700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4700b14c0 0x7fc4700cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 -- 172.21.15.7:0/3105225118 learned_addr learned my addr 172.21.15.7:0/3105225118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc46ffff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc470159e10 0x7fc470154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 -- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc470159e10 msgr2=0x7fc470154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc470159e10 0x7fc470154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 -- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4700b14c0 msgr2=0x7fc4700cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4700b14c0 0x7fc4700cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 -- 172.21.15.7:0/3105225118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c009420 con 0x7fc47015a1e0 2024-01-11T07:11:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.657+0000 7fc475682700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47015a1e0 0x7fc470156940 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc46800c910 tx=0x7fc4680027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.658+0000 7fc46cff9700 1 -- 172.21.15.7:0/3105225118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468012630 con 0x7fc47015a1e0 2024-01-11T07:11:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.658+0000 7fc46cff9700 1 -- 172.21.15.7:0/3105225118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4680127b0 con 0x7fc47015a1e0 2024-01-11T07:11:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.658+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4700ae360 con 0x7fc47015a1e0 2024-01-11T07:11:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.658+0000 7fc46cff9700 1 -- 172.21.15.7:0/3105225118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468012b70 con 0x7fc47015a1e0 2024-01-11T07:11:43.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.658+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc470155520 con 0x7fc47015a1e0 2024-01-11T07:11:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.659+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4700b1210 con 0x7fc47015a1e0 2024-01-11T07:11:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.660+0000 7fc46cff9700 1 -- 172.21.15.7:0/3105225118 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc46801b400 con 0x7fc47015a1e0 2024-01-11T07:11:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.660+0000 7fc46cff9700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc454066f30 0x7fc4540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.660+0000 7fc46cff9700 1 -- 172.21.15.7:0/3105225118 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc468084890 con 0x7fc47015a1e0 2024-01-11T07:11:43.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.660+0000 7fc46cff9700 1 -- 172.21.15.7:0/3105225118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7fc45406ea50 con 0x7fc47015a1e0 2024-01-11T07:11:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.660+0000 7fc474e81700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc454066f30 0x7fc4540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:43.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.661+0000 7fc474e81700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc454066f30 0x7fc4540693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc4701587b0 tx=0x7fc464000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:43.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.664+0000 7fc46cff9700 1 -- 172.21.15.7:0/3105225118 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc468016020 con 0x7fc47015a1e0 2024-01-11T07:11:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.865+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc47015b480 con 0x7fc47015a1e0 2024-01-11T07:11:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.866+0000 7fc46cff9700 1 -- 172.21.15.7:0/3105225118 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc468051b00 con 0x7fc47015a1e0 2024-01-11T07:11:43.867 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:11:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc454066f30 msgr2=0x7fc4540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc454066f30 0x7fc4540693e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc4701587b0 tx=0x7fc464000bc0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47015a1e0 msgr2=0x7fc470156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47015a1e0 0x7fc470156940 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc46800c910 tx=0x7fc4680027a0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 shutdown_connections 2024-01-11T07:11:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc454066f30 0x7fc4540693e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47015a1e0 0x7fc470156940 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4700b14c0 0x7fc4700cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 --2- 172.21.15.7:0/3105225118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc470159e10 0x7fc470154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 >> 172.21.15.7:0/3105225118 conn(0x7fc4700cd910 msgr2=0x7fc4700ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 shutdown_connections 2024-01-11T07:11:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:43.868+0000 7fc4770e5700 1 -- 172.21.15.7:0/3105225118 wait complete. 2024-01-11T07:11:43.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.017+0000 7f82ed54b700 1 Processor -- start 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.017+0000 7f82ed54b700 1 -- start start 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.017+0000 7f82ed54b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82e813c410 0x7f82e80c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.017+0000 7f82ed54b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82e80c75f0 0x7f82e80ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82ed54b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e80abab0 con 0x7f82e8157170 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82ed54b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80c1630 con 0x7f82e80c75f0 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82ed54b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80c2490 con 0x7f82e813c410 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82e80c75f0 0x7f82e80ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82e80c75f0 0x7f82e80ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43990/0 (socket says 172.21.15.7:43990) 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e77fe700 1 -- 172.21.15.7:0/2068127723 learned_addr learned my addr 172.21.15.7:0/2068127723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e6ffd700 1 --2- 172.21.15.7:0/2068127723 >> v2:172.21.15.113:3300/0 conn(0x7f82e813c410 0x7f82e80c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e77fe700 1 -- 172.21.15.7:0/2068127723 >> v2:172.21.15.113:3300/0 conn(0x7f82e813c410 msgr2=0x7f82e80c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e77fe700 1 --2- 172.21.15.7:0/2068127723 >> v2:172.21.15.113:3300/0 conn(0x7f82e813c410 0x7f82e80c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e77fe700 1 -- 172.21.15.7:0/2068127723 >> v1:172.21.15.113:6789/0 conn(0x7f82e8157170 legacy=0x7f82e80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e77fe700 1 -- 172.21.15.7:0/2068127723 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e813c7e0 con 0x7f82e80c75f0 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e6ffd700 1 --2- 172.21.15.7:0/2068127723 >> v2:172.21.15.113:3300/0 conn(0x7f82e813c410 0x7f82e80c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e77fe700 1 --2- 172.21.15.7:0/2068127723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e80c75f0 0x7f82e80ca390 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f82e80716f0 tx=0x7f82e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61036748c8d7a036 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e5ffb700 1 -- 172.21.15.7:0/2068127723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000e070 con 0x7f82e80c75f0 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e5ffb700 1 -- 172.21.15.7:0/2068127723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e000ba50 con 0x7f82e80c75f0 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.018+0000 7f82e5ffb700 1 -- 172.21.15.7:0/2068127723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e00092e0 con 0x7f82e80c75f0 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.019+0000 7f82ed54b700 1 -- 172.21.15.7:0/2068127723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e80c75f0 msgr2=0x7f82e80ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.019+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2068127723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e80c75f0 0x7f82e80ca390 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f82e80716f0 tx=0x7f82e0009840 comp rx=0 tx=0).stop 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.019+0000 7f82ed54b700 1 -- 172.21.15.7:0/2068127723 shutdown_connections 2024-01-11T07:11:44.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.019+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2068127723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e80c75f0 0x7f82e80ca390 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.019+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2068127723 >> v2:172.21.15.113:3300/0 conn(0x7f82e813c410 0x7f82e80c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.019+0000 7f82ed54b700 1 -- 172.21.15.7:0/2068127723 >> 172.21.15.7:0/2068127723 conn(0x7f82e8063230 msgr2=0x7f82e8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.019+0000 7f82ed54b700 1 -- 172.21.15.7:0/2068127723 shutdown_connections 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.019+0000 7f82ed54b700 1 -- 172.21.15.7:0/2068127723 wait complete. 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.021+0000 7f82ed54b700 1 Processor -- start 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.021+0000 7f82ed54b700 1 -- start start 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.021+0000 7f82ed54b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82e80c75f0 0x7f82e8154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.021+0000 7f82ed54b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82e813c410 0x7f82e8165b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.021+0000 7f82ed54b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e8157170 0x7f82e806dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.021+0000 7f82ed54b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e806eba0 con 0x7f82e8157170 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.021+0000 7f82ed54b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e806e110 con 0x7f82e80c75f0 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.021+0000 7f82ed54b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e81692f0 con 0x7f82e813c410 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e8157170 0x7f82e806dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e8157170 0x7f82e806dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44006/0 (socket says 172.21.15.7:44006) 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 -- 172.21.15.7:0/2075939779 learned_addr learned my addr 172.21.15.7:0/2075939779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e6ffd700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82e80c75f0 0x7f82e8154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e67fc700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82e813c410 0x7f82e8165b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 -- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82e813c410 msgr2=0x7f82e8165b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82e813c410 0x7f82e8165b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 -- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82e80c75f0 msgr2=0x7f82e8154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82e80c75f0 0x7f82e8154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 -- 172.21.15.7:0/2075939779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e0009ae0 con 0x7f82e8157170 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e67fc700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82e813c410 0x7f82e8165b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82e77fe700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e8157170 0x7f82e806dbd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f82e806af80 tx=0x7f82e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82deffd700 1 -- 172.21.15.7:0/2075939779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0008e40 con 0x7f82e8157170 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82deffd700 1 -- 172.21.15.7:0/2075939779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e000be60 con 0x7f82e8157170 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82deffd700 1 -- 172.21.15.7:0/2075939779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e00266b0 con 0x7f82e8157170 2024-01-11T07:11:44.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e81661f0 con 0x7f82e8157170 2024-01-11T07:11:44.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.022+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e81667b0 con 0x7f82e8157170 2024-01-11T07:11:44.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.023+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e804c3c0 con 0x7f82e8157170 2024-01-11T07:11:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.024+0000 7f82deffd700 1 -- 172.21.15.7:0/2075939779 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f82e0003040 con 0x7f82e8157170 2024-01-11T07:11:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.024+0000 7f82deffd700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82c8066f30 0x7f82c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.024+0000 7f82deffd700 1 -- 172.21.15.7:0/2075939779 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82e0085f50 con 0x7f82e8157170 2024-01-11T07:11:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.024+0000 7f82deffd700 1 -- 172.21.15.7:0/2075939779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f82c806ea50 con 0x7f82e8157170 2024-01-11T07:11:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.024+0000 7f82e6ffd700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82c8066f30 0x7f82c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.025+0000 7f82e6ffd700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82c8066f30 0x7f82c80693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f82e806a7a0 tx=0x7f82d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.028+0000 7f82deffd700 1 -- 172.21.15.7:0/2075939779 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82e00531c0 con 0x7f82e8157170 2024-01-11T07:11:44.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.232+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82e8169430 con 0x7f82e8157170 2024-01-11T07:11:44.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.233+0000 7f82deffd700 1 -- 172.21.15.7:0/2075939779 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82e0021070 con 0x7f82e8157170 2024-01-11T07:11:44.234 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82c8066f30 msgr2=0x7f82c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82c8066f30 0x7f82c80693e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f82e806a7a0 tx=0x7f82d8008040 comp rx=0 tx=0).stop 2024-01-11T07:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e8157170 msgr2=0x7f82e806dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e8157170 0x7f82e806dbd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f82e806af80 tx=0x7f82e0018000 comp rx=0 tx=0).stop 2024-01-11T07:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 shutdown_connections 2024-01-11T07:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82c8066f30 0x7f82c80693e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e8157170 0x7f82e806dbd0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82e80c75f0 0x7f82e8154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 --2- 172.21.15.7:0/2075939779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82e813c410 0x7f82e8165b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 >> 172.21.15.7:0/2075939779 conn(0x7f82e8063230 msgr2=0x7f82e80cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 shutdown_connections 2024-01-11T07:11:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.235+0000 7f82ed54b700 1 -- 172.21.15.7:0/2075939779 wait complete. 2024-01-11T07:11:44.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:11:44.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.383+0000 7fc3f26c6700 1 Processor -- start 2024-01-11T07:11:44.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.383+0000 7fc3f26c6700 1 -- start start 2024-01-11T07:11:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.384+0000 7fc3f26c6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.384+0000 7fc3f26c6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3ec157d80 0x7fc3ec063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.384+0000 7fc3f26c6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec063bb0 con 0x7fc3ec063770 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.384+0000 7fc3f26c6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec0aeab0 con 0x7fc3ec157d80 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.384+0000 7fc3f26c6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0aed90 con 0x7fc3ec1579b0 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.384+0000 7fc3f0c63700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc3ec063770 0x7fc3ec06dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35614/0 (socket says 172.21.15.7:35614) 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.384+0000 7fc3f0c63700 1 -- 172.21.15.7:0/1857519749 learned_addr learned my addr 172.21.15.7:0/1857519749 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3ebfff700 1 --2- 172.21.15.7:0/1857519749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3eb7fe700 1 --2- 172.21.15.7:0/1857519749 >> v2:172.21.15.113:3300/0 conn(0x7fc3ec157d80 0x7fc3ec063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3eaffd700 1 -- 172.21.15.7:0/1857519749 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928220905 0 0) 0x7fc3ec063bb0 con 0x7fc3ec063770 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3eaffd700 1 -- 172.21.15.7:0/1857519749 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d8003570 con 0x7fc3ec063770 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3ebfff700 1 -- 172.21.15.7:0/1857519749 >> v2:172.21.15.113:3300/0 conn(0x7fc3ec157d80 msgr2=0x7fc3ec063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3ebfff700 1 --2- 172.21.15.7:0/1857519749 >> v2:172.21.15.113:3300/0 conn(0x7fc3ec157d80 0x7fc3ec063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3ebfff700 1 -- 172.21.15.7:0/1857519749 >> v1:172.21.15.7:6789/0 conn(0x7fc3ec063770 legacy=0x7fc3ec06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3eaffd700 1 -- 172.21.15.7:0/1857519749 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2038646697 0 0) 0x7fc3d8003570 con 0x7fc3ec063770 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3ebfff700 1 -- 172.21.15.7:0/1857519749 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0af070 con 0x7fc3ec1579b0 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3eb7fe700 1 --2- 172.21.15.7:0/1857519749 >> v2:172.21.15.113:3300/0 conn(0x7fc3ec157d80 0x7fc3ec063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3ebfff700 1 --2- 172.21.15.7:0/1857519749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15bf50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0098e0 tx=0x7fc3dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=539dfb8383605ae1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3eaffd700 1 -- 172.21.15.7:0/1857519749 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40017f0 con 0x7fc3ec1579b0 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3eaffd700 1 -- 172.21.15.7:0/1857519749 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc011420 con 0x7fc3ec1579b0 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.385+0000 7fc3eaffd700 1 -- 172.21.15.7:0/1857519749 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0117c0 con 0x7fc3ec1579b0 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.386+0000 7fc3f26c6700 1 -- 172.21.15.7:0/1857519749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 msgr2=0x7fc3ec15bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.386+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/1857519749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15bf50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0098e0 tx=0x7fc3dc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.386+0000 7fc3f26c6700 1 -- 172.21.15.7:0/1857519749 shutdown_connections 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.386+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/1857519749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15bf50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.386+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/1857519749 >> v2:172.21.15.113:3300/0 conn(0x7fc3ec157d80 0x7fc3ec063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.386+0000 7fc3f26c6700 1 -- 172.21.15.7:0/1857519749 >> 172.21.15.7:0/1857519749 conn(0x7fc3ec0680e0 msgr2=0x7fc3ec0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.386+0000 7fc3f26c6700 1 -- 172.21.15.7:0/1857519749 shutdown_connections 2024-01-11T07:11:44.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.386+0000 7fc3f26c6700 1 -- 172.21.15.7:0/1857519749 wait complete. 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.387+0000 7fc3f26c6700 1 Processor -- start 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.387+0000 7fc3f26c6700 1 -- start start 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3f26c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ec063770 0x7fc3ec16bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3f26c6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3f26c6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3ec157d80 0x7fc3ec15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3f26c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec15b0e0 con 0x7fc3ec063770 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3f26c6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec1680c0 con 0x7fc3ec157d80 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3f26c6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec168c00 con 0x7fc3ec1579b0 2024-01-11T07:11:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ec063770 0x7fc3ec16bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ec063770 0x7fc3ec16bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44016/0 (socket says 172.21.15.7:44016) 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3ebfff700 1 -- 172.21.15.7:0/199076937 learned_addr learned my addr 172.21.15.7:0/199076937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3eb7fe700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.388+0000 7fc3f0c63700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3ec157d80 0x7fc3ec15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3ebfff700 1 -- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 msgr2=0x7fc3ec15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3ebfff700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3ebfff700 1 -- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3ec157d80 msgr2=0x7fc3ec15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3ebfff700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3ec157d80 0x7fc3ec15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3ebfff700 1 -- 172.21.15.7:0/199076937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d4009420 con 0x7fc3ec063770 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3ebfff700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ec063770 0x7fc3ec16bd00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0030c0 tx=0x7fc3dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3e3fff700 1 -- 172.21.15.7:0/199076937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc01cb30 con 0x7fc3ec063770 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3e3fff700 1 -- 172.21.15.7:0/199076937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc01cc90 con 0x7fc3ec063770 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3dc009420 con 0x7fc3ec063770 2024-01-11T07:11:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3e3fff700 1 -- 172.21.15.7:0/199076937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc01a6a0 con 0x7fc3ec063770 2024-01-11T07:11:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.389+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec1685b0 con 0x7fc3ec063770 2024-01-11T07:11:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.390+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec04c3c0 con 0x7fc3ec063770 2024-01-11T07:11:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.391+0000 7fc3e3fff700 1 -- 172.21.15.7:0/199076937 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc3dc01aad0 con 0x7fc3ec063770 2024-01-11T07:11:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.391+0000 7fc3e3fff700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3cc066f30 0x7fc3cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.391+0000 7fc3e3fff700 1 -- 172.21.15.7:0/199076937 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc3dc085e40 con 0x7fc3ec063770 2024-01-11T07:11:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.391+0000 7fc3e3fff700 1 -- 172.21.15.7:0/199076937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7fc3cc06ea50 con 0x7fc3ec063770 2024-01-11T07:11:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.391+0000 7fc3eb7fe700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3cc066f30 0x7fc3cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.392+0000 7fc3eb7fe700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3cc066f30 0x7fc3cc0693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec16b620 tx=0x7fc3d40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.396+0000 7fc3e3fff700 1 -- 172.21.15.7:0/199076937 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3dc0530b0 con 0x7fc3ec063770 2024-01-11T07:11:44.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.645+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3ec168950 con 0x7fc3ec063770 2024-01-11T07:11:44.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.645+0000 7fc3e3fff700 1 -- 172.21.15.7:0/199076937 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fc3dc029090 con 0x7fc3ec063770 2024-01-11T07:11:44.646 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:44.646 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2882608277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2882608277},{"type":"v1","addr":"172.21.15.7:6833","nonce":2882608277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3510440823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3510440823},{"type":"v1","addr":"172.21.15.7:6835","nonce":3510440823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3108557963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3108557963},{"type":"v1","addr":"172.21.15.113:6836","nonce":3108557963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:10:55.765305+0000","modified":"2024-01-11T07:10:59.370619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1888585107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1888585107},{"type":"v1","addr":"172.21.15.113:6837","nonce":1888585107}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-11T07:11:44.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3cc066f30 msgr2=0x7fc3cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3cc066f30 0x7fc3cc0693e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec16b620 tx=0x7fc3d40091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ec063770 msgr2=0x7fc3ec16bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ec063770 0x7fc3ec16bd00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0030c0 tx=0x7fc3dc016000 comp rx=0 tx=0).stop 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 shutdown_connections 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc3cc066f30 0x7fc3cc0693e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3ec157d80 0x7fc3ec15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ec063770 0x7fc3ec16bd00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 --2- 172.21.15.7:0/199076937 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3ec1579b0 0x7fc3ec15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 >> 172.21.15.7:0/199076937 conn(0x7fc3ec0680e0 msgr2=0x7fc3ec155940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 shutdown_connections 2024-01-11T07:11:44.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.648+0000 7fc3f26c6700 1 -- 172.21.15.7:0/199076937 wait complete. 2024-01-11T07:11:44.650 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 41 2024-01-11T07:11:44.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:11:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.793+0000 7f1f59854700 1 Processor -- start 2024-01-11T07:11:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.793+0000 7f1f59854700 1 -- start start 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f59854700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f540b0b40 0x7f1f540b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f59854700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f54067de0 con 0x7f1f54159c20 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f59854700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f540b20e0 con 0x7f1f54159ff0 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f59854700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f54069570 con 0x7f1f540b0b40 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f540b0b40 0x7f1f540b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f52ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f54159c20 0x7f1f54068270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35616/0 (socket says 172.21.15.7:35616) 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f52ffd700 1 -- 172.21.15.7:0/2394830971 learned_addr learned my addr 172.21.15.7:0/2394830971 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f537fe700 1 --2- 172.21.15.7:0/2394830971 >> v2:172.21.15.7:3300/0 conn(0x7f1f540b0b40 0x7f1f540b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44028/0 (socket says 172.21.15.7:44028) 2024-01-11T07:11:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f537fe700 1 -- 172.21.15.7:0/2394830971 >> v1:172.21.15.113:6789/0 conn(0x7f1f54159ff0 legacy=0x7f1f540b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f537fe700 1 -- 172.21.15.7:0/2394830971 >> v1:172.21.15.7:6789/0 conn(0x7f1f54159c20 legacy=0x7f1f54068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f537fe700 1 -- 172.21.15.7:0/2394830971 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f540b1e00 con 0x7f1f540b0b40 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.794+0000 7f1f537fe700 1 --2- 172.21.15.7:0/2394830971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f540b0b40 0x7f1f540b1820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009e60 tx=0x7f1f4c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf91d4e4a01bdcf4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f51ffb700 1 -- 172.21.15.7:0/2394830971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00e070 con 0x7f1f540b0b40 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f51ffb700 1 -- 172.21.15.7:0/2394830971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00bc30 con 0x7f1f540b0b40 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f51ffb700 1 -- 172.21.15.7:0/2394830971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c012430 con 0x7f1f540b0b40 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f59854700 1 -- 172.21.15.7:0/2394830971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f540b0b40 msgr2=0x7f1f540b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f59854700 1 --2- 172.21.15.7:0/2394830971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f540b0b40 0x7f1f540b1820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009e60 tx=0x7f1f4c009780 comp rx=0 tx=0).stop 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f59854700 1 -- 172.21.15.7:0/2394830971 shutdown_connections 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f59854700 1 --2- 172.21.15.7:0/2394830971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f540b0b40 0x7f1f540b1820 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f59854700 1 -- 172.21.15.7:0/2394830971 >> 172.21.15.7:0/2394830971 conn(0x7f1f54071160 msgr2=0x7f1f5406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f59854700 1 -- 172.21.15.7:0/2394830971 shutdown_connections 2024-01-11T07:11:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.795+0000 7f1f59854700 1 -- 172.21.15.7:0/2394830971 wait complete. 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.797+0000 7f1f59854700 1 Processor -- start 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.797+0000 7f1f59854700 1 -- start start 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.797+0000 7f1f59854700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f540b0b40 0x7f1f540b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.797+0000 7f1f59854700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f54159c20 0x7f1f540b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.797+0000 7f1f59854700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f54159ff0 0x7f1f540cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.797+0000 7f1f59854700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f540a64b0 con 0x7f1f54159ff0 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.797+0000 7f1f59854700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f540a72d0 con 0x7f1f540b0b40 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.797+0000 7f1f59854700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f540a66c0 con 0x7f1f54159c20 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f54159ff0 0x7f1f540cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f54159ff0 0x7f1f540cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44038/0 (socket says 172.21.15.7:44038) 2024-01-11T07:11:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 -- 172.21.15.7:0/286561794 learned_addr learned my addr 172.21.15.7:0/286561794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f52ffd700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f540b0b40 0x7f1f540b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f527fc700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f54159c20 0x7f1f540b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 -- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f54159c20 msgr2=0x7f1f540b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f54159c20 0x7f1f540b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 -- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f540b0b40 msgr2=0x7f1f540b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f540b0b40 0x7f1f540b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 -- 172.21.15.7:0/286561794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c009a20 con 0x7f1f54159ff0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f537fe700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f54159ff0 0x7f1f540cd980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0097c0 tx=0x7f1f4c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f3affd700 1 -- 172.21.15.7:0/286561794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c012e10 con 0x7f1f54159ff0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f3affd700 1 -- 172.21.15.7:0/286561794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c01e630 con 0x7f1f54159ff0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f540a69a0 con 0x7f1f54159ff0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f3affd700 1 -- 172.21.15.7:0/286561794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c01e9f0 con 0x7f1f54159ff0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.798+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f540a6f60 con 0x7f1f54159ff0 2024-01-11T07:11:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.799+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5404c3c0 con 0x7f1f54159ff0 2024-01-11T07:11:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.800+0000 7f1f3affd700 1 -- 172.21.15.7:0/286561794 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f4c024070 con 0x7f1f54159ff0 2024-01-11T07:11:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.800+0000 7f1f3affd700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f40066f30 0x7f1f400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.800+0000 7f1f3affd700 1 -- 172.21.15.7:0/286561794 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f4c086d20 con 0x7f1f54159ff0 2024-01-11T07:11:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.800+0000 7f1f3affd700 1 -- 172.21.15.7:0/286561794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7f1f4006ea50 con 0x7f1f54159ff0 2024-01-11T07:11:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.800+0000 7f1f52ffd700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f40066f30 0x7f1f400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.801+0000 7f1f52ffd700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f40066f30 0x7f1f400693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1f540cd650 tx=0x7f1f44007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:44.805+0000 7f1f3affd700 1 -- 172.21.15.7:0/286561794 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f4c02c060 con 0x7f1f54159ff0 2024-01-11T07:11:45.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.009+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f54067790 con 0x7f1f54159ff0 2024-01-11T07:11:45.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.010+0000 7f1f3affd700 1 -- 172.21.15.7:0/286561794 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+15348 (secure 0 0 0) 0x7f1f4c053f90 con 0x7f1f54159ff0 2024-01-11T07:11:45.011 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:45.011 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:11:16.228396+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:10:53.813087+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-11T07:10:54.744603+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:11:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f40066f30 msgr2=0x7f1f400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f40066f30 0x7f1f400693e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1f540cd650 tx=0x7f1f44007eb0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f54159ff0 msgr2=0x7f1f540cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f54159ff0 0x7f1f540cd980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0097c0 tx=0x7f1f4c019000 comp rx=0 tx=0).stop 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 shutdown_connections 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f40066f30 0x7f1f400693e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f54159ff0 0x7f1f540cd980 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f540b0b40 0x7f1f540b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.012+0000 7f1f59854700 1 --2- 172.21.15.7:0/286561794 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f54159c20 0x7f1f540b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.013+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 >> 172.21.15.7:0/286561794 conn(0x7f1f54071160 msgr2=0x7f1f5415cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.013+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 shutdown_connections 2024-01-11T07:11:45.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.013+0000 7f1f59854700 1 -- 172.21.15.7:0/286561794 wait complete. 2024-01-11T07:11:45.027 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:11:45.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:11:45.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.158+0000 7fd13d33a700 1 Processor -- start 2024-01-11T07:11:45.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.158+0000 7fd13d33a700 1 -- start start 2024-01-11T07:11:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd13d33a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1380aed00 0x7fd13815aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd13d33a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd138156e90 0x7fd138069d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd13d33a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13806a6a0 con 0x7fd13806a240 2024-01-11T07:11:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd13d33a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd138158010 con 0x7fd138156e90 2024-01-11T07:11:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd13d33a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13815b530 con 0x7fd1380aed00 2024-01-11T07:11:45.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd1367fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd138156e90 0x7fd138069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd1367fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd138156e90 0x7fd138069d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44054/0 (socket says 172.21.15.7:44054) 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd1367fc700 1 -- 172.21.15.7:0/3797797884 learned_addr learned my addr 172.21.15.7:0/3797797884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd136ffd700 1 --2- 172.21.15.7:0/3797797884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1380aed00 0x7fd13815aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd1367fc700 1 -- 172.21.15.7:0/3797797884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1380aed00 msgr2=0x7fd13815aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd1367fc700 1 --2- 172.21.15.7:0/3797797884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1380aed00 0x7fd13815aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd1367fc700 1 -- 172.21.15.7:0/3797797884 >> v1:172.21.15.113:6789/0 conn(0x7fd13806a240 legacy=0x7fd1380c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.159+0000 7fd1367fc700 1 -- 172.21.15.7:0/3797797884 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138157400 con 0x7fd138156e90 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd1367fc700 1 --2- 172.21.15.7:0/3797797884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd138069d00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd1280099b0 tx=0x7fd128009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a1f8f9253aadec server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd135ffb700 1 -- 172.21.15.7:0/3797797884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128009ea0 con 0x7fd138156e90 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd135ffb700 1 -- 172.21.15.7:0/3797797884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12800b860 con 0x7fd138156e90 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd135ffb700 1 -- 172.21.15.7:0/3797797884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12800bc00 con 0x7fd138156e90 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd13d33a700 1 -- 172.21.15.7:0/3797797884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 msgr2=0x7fd138069d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd13d33a700 1 --2- 172.21.15.7:0/3797797884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd138069d00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd1280099b0 tx=0x7fd128009250 comp rx=0 tx=0).stop 2024-01-11T07:11:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd13d33a700 1 -- 172.21.15.7:0/3797797884 shutdown_connections 2024-01-11T07:11:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd13d33a700 1 --2- 172.21.15.7:0/3797797884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1380aed00 0x7fd13815aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd13d33a700 1 --2- 172.21.15.7:0/3797797884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd138069d00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd13d33a700 1 -- 172.21.15.7:0/3797797884 >> 172.21.15.7:0/3797797884 conn(0x7fd138063230 msgr2=0x7fd138063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd13d33a700 1 -- 172.21.15.7:0/3797797884 shutdown_connections 2024-01-11T07:11:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.160+0000 7fd13d33a700 1 -- 172.21.15.7:0/3797797884 wait complete. 2024-01-11T07:11:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.162+0000 7fd13d33a700 1 Processor -- start 2024-01-11T07:11:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.162+0000 7fd13d33a700 1 -- start start 2024-01-11T07:11:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd13d33a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd13806a240 0x7fd1380affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd13d33a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1380aed00 0x7fd13814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd13d33a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd1380c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd13d33a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1380c5520 con 0x7fd138156e90 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd13d33a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1380af5d0 con 0x7fd1380aed00 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd13d33a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1380af8b0 con 0x7fd13806a240 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd1380c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd1380c48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44058/0 (socket says 172.21.15.7:44058) 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 -- 172.21.15.7:0/2610013422 learned_addr learned my addr 172.21.15.7:0/2610013422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd136ffd700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd13806a240 0x7fd1380affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1367fc700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1380aed00 0x7fd13814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 -- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd13806a240 msgr2=0x7fd1380affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd13806a240 0x7fd1380affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 -- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1380aed00 msgr2=0x7fd13814d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1380aed00 0x7fd13814d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 -- 172.21.15.7:0/2610013422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1280094f0 con 0x7fd138156e90 2024-01-11T07:11:45.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.163+0000 7fd1377fe700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd1380c48c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd13000c840 tx=0x7fd130001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.164+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130012630 con 0x7fd138156e90 2024-01-11T07:11:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.164+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1300127b0 con 0x7fd138156e90 2024-01-11T07:11:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.164+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd130012b70 con 0x7fd138156e90 2024-01-11T07:11:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.164+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1380afb90 con 0x7fd138156e90 2024-01-11T07:11:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.164+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13814d890 con 0x7fd138156e90 2024-01-11T07:11:45.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.165+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1380c5690 con 0x7fd138156e90 2024-01-11T07:11:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.165+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd13001a3a0 con 0x7fd138156e90 2024-01-11T07:11:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.166+0000 7fd12effd700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.166+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd1300844a0 con 0x7fd138156e90 2024-01-11T07:11:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.166+0000 7fd136ffd700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.166+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=88}) v3 -- 0x7fd11806ea50 con 0x7fd138156e90 2024-01-11T07:11:45.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.166+0000 7fd136ffd700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd118066f30 0x7fd1180693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd120009d10 tx=0x7fd1200091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:45.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.170+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd130051710 con 0x7fd138156e90 2024-01-11T07:11:45.279 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:11:45.278+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-01-11T07:11:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.280+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(88..88 src has 1..88) v4 ==== 296+0+0 (secure 0 0 0) 0x7fd130002260 con 0x7fd138156e90 2024-01-11T07:11:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.280+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fd11806f340 con 0x7fd138156e90 2024-01-11T07:11:45.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.428+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd13806a6a0 con 0x7fd138156e90 2024-01-11T07:11:45.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.428+0000 7fd12effd700 1 -- 172.21.15.7:0/2610013422 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fd13806a6a0 con 0x7fd138156e90 2024-01-11T07:11:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:45.429 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2882608277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2882608277},{"type":"v1","addr":"172.21.15.7:6833","nonce":2882608277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3510440823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3510440823},{"type":"v1","addr":"172.21.15.7:6835","nonce":3510440823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3108557963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3108557963},{"type":"v1","addr":"172.21.15.113:6836","nonce":3108557963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:10:55.765305+0000","modified":"2024-01-11T07:10:59.370619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1888585107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1888585107},{"type":"v1","addr":"172.21.15.113:6837","nonce":1888585107}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-11T07:11:45.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd118066f30 msgr2=0x7fd1180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd118066f30 0x7fd1180693e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd120009d10 tx=0x7fd1200091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 msgr2=0x7fd1380c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd1380c48c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd13000c840 tx=0x7fd130001670 comp rx=0 tx=0).stop 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 shutdown_connections 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd138156e90 0x7fd1380c48c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd13806a240 0x7fd1380affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 --2- 172.21.15.7:0/2610013422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1380aed00 0x7fd13814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 >> 172.21.15.7:0/2610013422 conn(0x7fd138063230 msgr2=0x7fd1380cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 shutdown_connections 2024-01-11T07:11:45.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.431+0000 7fd13d33a700 1 -- 172.21.15.7:0/2610013422 wait complete. 2024-01-11T07:11:45.433 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 41 2024-01-11T07:11:45.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:11:45.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.577+0000 7fe437165700 1 Processor -- start 2024-01-11T07:11:45.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.577+0000 7fe437165700 1 -- start start 2024-01-11T07:11:45.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe437165700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe430069c80 0x7fe43006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe437165700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43006e3f0 con 0x7fe43015b260 2024-01-11T07:11:45.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe437165700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe430157b40 con 0x7fe430156760 2024-01-11T07:11:45.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe437165700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43006ef90 con 0x7fe430069c80 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe42ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe43015b260 0x7fe4300b1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35632/0 (socket says 172.21.15.7:35632) 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe42ffff700 1 -- 172.21.15.7:0/3220592882 learned_addr learned my addr 172.21.15.7:0/3220592882 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe435702700 1 --2- 172.21.15.7:0/3220592882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe430069c80 0x7fe43006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe42f7fe700 1 -- 172.21.15.7:0/3220592882 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371495090 0 0) 0x7fe43006e3f0 con 0x7fe43015b260 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe42f7fe700 1 -- 172.21.15.7:0/3220592882 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe420003570 con 0x7fe43015b260 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe42f7fe700 1 -- 172.21.15.7:0/3220592882 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903286309 0 0) 0x7fe430157b40 con 0x7fe430156760 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe42f7fe700 1 -- 172.21.15.7:0/3220592882 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe43006e3f0 con 0x7fe430156760 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe435702700 1 -- 172.21.15.7:0/3220592882 >> v1:172.21.15.113:6789/0 conn(0x7fe430156760 legacy=0x7fe43006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe435702700 1 -- 172.21.15.7:0/3220592882 >> v1:172.21.15.7:6789/0 conn(0x7fe43015b260 legacy=0x7fe4300b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe42f7fe700 1 -- 172.21.15.7:0/3220592882 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3457925387 0 0) 0x7fe420003570 con 0x7fe43015b260 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.578+0000 7fe435702700 1 -- 172.21.15.7:0/3220592882 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4300b1f10 con 0x7fe430069c80 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe435702700 1 --2- 172.21.15.7:0/3220592882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe430069c80 0x7fe43006a120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe428009e90 tx=0x7fe428009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270f8f3f40cd5093 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:45.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe42f7fe700 1 -- 172.21.15.7:0/3220592882 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42800e070 con 0x7fe430069c80 2024-01-11T07:11:45.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe42f7fe700 1 -- 172.21.15.7:0/3220592882 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42800b870 con 0x7fe430069c80 2024-01-11T07:11:45.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe42f7fe700 1 -- 172.21.15.7:0/3220592882 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c001470 con 0x7fe430069c80 2024-01-11T07:11:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe437165700 1 -- 172.21.15.7:0/3220592882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe430069c80 msgr2=0x7fe43006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe437165700 1 --2- 172.21.15.7:0/3220592882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe430069c80 0x7fe43006a120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe428009e90 tx=0x7fe428009780 comp rx=0 tx=0).stop 2024-01-11T07:11:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe437165700 1 -- 172.21.15.7:0/3220592882 shutdown_connections 2024-01-11T07:11:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe437165700 1 --2- 172.21.15.7:0/3220592882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe430069c80 0x7fe43006a120 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe437165700 1 -- 172.21.15.7:0/3220592882 >> 172.21.15.7:0/3220592882 conn(0x7fe4300cd910 msgr2=0x7fe4300cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe437165700 1 -- 172.21.15.7:0/3220592882 shutdown_connections 2024-01-11T07:11:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.579+0000 7fe437165700 1 -- 172.21.15.7:0/3220592882 wait complete. 2024-01-11T07:11:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.581+0000 7fe437165700 1 Processor -- start 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.581+0000 7fe437165700 1 -- start start 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe437165700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe430069c80 0x7fe4300c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe437165700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe430156760 0x7fe43016ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe437165700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43015b260 0x7fe43014e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe437165700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43016bcb0 con 0x7fe430069c80 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe437165700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300cbe70 con 0x7fe430156760 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe437165700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43016b410 con 0x7fe43015b260 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe430069c80 0x7fe4300c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe430069c80 0x7fe4300c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44064/0 (socket says 172.21.15.7:44064) 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 -- 172.21.15.7:0/2042218478 learned_addr learned my addr 172.21.15.7:0/2042218478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe435702700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43015b260 0x7fe43014e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe42ffff700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe430156760 0x7fe43016ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 -- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43015b260 msgr2=0x7fe43014e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43015b260 0x7fe43014e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 -- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe430156760 msgr2=0x7fe43016ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe430156760 0x7fe43016ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 -- 172.21.15.7:0/2042218478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe428009a20 con 0x7fe430069c80 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe434f01700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe430069c80 0x7fe4300c9490 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c7c0 tx=0x7fe424007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe42cff9700 1 -- 172.21.15.7:0/2042218478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240015f0 con 0x7fe430069c80 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe42cff9700 1 -- 172.21.15.7:0/2042218478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe424001790 con 0x7fe430069c80 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43016b6f0 con 0x7fe430069c80 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.582+0000 7fe42cff9700 1 -- 172.21.15.7:0/2042218478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424001b50 con 0x7fe430069c80 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.583+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43015b8c0 con 0x7fe430069c80 2024-01-11T07:11:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.583+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43015b720 con 0x7fe430069c80 2024-01-11T07:11:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.584+0000 7fe42cff9700 1 -- 172.21.15.7:0/2042218478 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe42401b400 con 0x7fe430069c80 2024-01-11T07:11:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.584+0000 7fe42cff9700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe414066f30 0x7fe4140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.585+0000 7fe42cff9700 1 -- 172.21.15.7:0/2042218478 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe424084330 con 0x7fe430069c80 2024-01-11T07:11:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.585+0000 7fe42ffff700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe414066f30 0x7fe4140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.585+0000 7fe42cff9700 1 -- 172.21.15.7:0/2042218478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fe41406ea50 con 0x7fe430069c80 2024-01-11T07:11:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.585+0000 7fe42ffff700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe414066f30 0x7fe4140693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe41c001080 tx=0x7fe41c001930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.589+0000 7fe42cff9700 1 -- 172.21.15.7:0/2042218478 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe43015b720 con 0x7fe430069c80 2024-01-11T07:11:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.840+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe43015b720 con 0x7fe430069c80 2024-01-11T07:11:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.841+0000 7fe42cff9700 1 -- 172.21.15.7:0/2042218478 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fe43015b720 con 0x7fe430069c80 2024-01-11T07:11:45.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:45.841 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2882608277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2882608277},{"type":"v1","addr":"172.21.15.7:6833","nonce":2882608277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3510440823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3510440823},{"type":"v1","addr":"172.21.15.7:6835","nonce":3510440823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3108557963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3108557963},{"type":"v1","addr":"172.21.15.113:6836","nonce":3108557963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:10:55.765305+0000","modified":"2024-01-11T07:10:59.370619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6015},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6015":{"gid":6015,"name":"b","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1888585107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1888585107},{"type":"v1","addr":"172.21.15.113:6837","nonce":1888585107}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe414066f30 msgr2=0x7fe4140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe414066f30 0x7fe4140693e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe41c001080 tx=0x7fe41c001930 comp rx=0 tx=0).stop 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe430069c80 msgr2=0x7fe4300c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe430069c80 0x7fe4300c9490 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c7c0 tx=0x7fe424007dd0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 shutdown_connections 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe414066f30 0x7fe4140693e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe43015b260 0x7fe43014e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe430069c80 0x7fe4300c9490 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 --2- 172.21.15.7:0/2042218478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe430156760 0x7fe43016ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 >> 172.21.15.7:0/2042218478 conn(0x7fe4300cd910 msgr2=0x7fe430157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 shutdown_connections 2024-01-11T07:11:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.843+0000 7fe437165700 1 -- 172.21.15.7:0/2042218478 wait complete. 2024-01-11T07:11:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 41 2024-01-11T07:11:45.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:11:45.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.989+0000 7f977dfe0700 1 Processor -- start 2024-01-11T07:11:45.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.989+0000 7f977dfe0700 1 -- start start 2024-01-11T07:11:45.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.989+0000 7f977dfe0700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97780cafb0 0x7f97780ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.989+0000 7f977dfe0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 0x7f97780b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f977dfe0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9778159300 con 0x7f97780b0970 2024-01-11T07:11:45.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f977dfe0700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97780b0df0 con 0x7f97780cafb0 2024-01-11T07:11:45.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f977dfe0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97780cf260 con 0x7f9778158280 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f9777fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f97780b0970 0x7f9778159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35644/0 (socket says 172.21.15.7:35644) 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f9777fff700 1 -- 172.21.15.7:0/1520422291 learned_addr learned my addr 172.21.15.7:0/1520422291 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f9776ffd700 1 --2- 172.21.15.7:0/1520422291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 0x7f97780b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f97777fe700 1 --2- 172.21.15.7:0/1520422291 >> v2:172.21.15.113:3300/0 conn(0x7f97780cafb0 0x7f97780ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f97767fc700 1 -- 172.21.15.7:0/1520422291 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328199568 0 0) 0x7f9778159300 con 0x7f97780b0970 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f97767fc700 1 -- 172.21.15.7:0/1520422291 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9760003570 con 0x7f97780b0970 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f97777fe700 1 -- 172.21.15.7:0/1520422291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 msgr2=0x7f97780b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f97777fe700 1 --2- 172.21.15.7:0/1520422291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 0x7f97780b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f97777fe700 1 -- 172.21.15.7:0/1520422291 >> v1:172.21.15.7:6789/0 conn(0x7f97780b0970 legacy=0x7f9778159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.990+0000 7f97777fe700 1 -- 172.21.15.7:0/1520422291 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780cf540 con 0x7f97780cafb0 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f97777fe700 1 --2- 172.21.15.7:0/1520422291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 0x7f97780ce8f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97680098e0 tx=0x7f97680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833ce4f618c66345 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f97767fc700 1 -- 172.21.15.7:0/1520422291 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768011630 con 0x7f97780cafb0 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f97767fc700 1 -- 172.21.15.7:0/1520422291 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768011790 con 0x7f97780cafb0 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f97767fc700 1 -- 172.21.15.7:0/1520422291 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768011b30 con 0x7f97780cafb0 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f977dfe0700 1 -- 172.21.15.7:0/1520422291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 msgr2=0x7f97780ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1520422291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 0x7f97780ce8f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97680098e0 tx=0x7f97680091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f977dfe0700 1 -- 172.21.15.7:0/1520422291 shutdown_connections 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1520422291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 0x7f97780ce8f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1520422291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 0x7f97780b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f977dfe0700 1 -- 172.21.15.7:0/1520422291 >> 172.21.15.7:0/1520422291 conn(0x7f9778071100 msgr2=0x7f977806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f977dfe0700 1 -- 172.21.15.7:0/1520422291 shutdown_connections 2024-01-11T07:11:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.991+0000 7f977dfe0700 1 -- 172.21.15.7:0/1520422291 wait complete. 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.993+0000 7f977dfe0700 1 Processor -- start 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.993+0000 7f977dfe0700 1 -- start start 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f977dfe0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97780b0970 0x7f97780ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f977dfe0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 0x7f977816a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f977dfe0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 0x7f97780c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f977dfe0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977816ab80 con 0x7f97780b0970 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f977dfe0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977816ae60 con 0x7f97780cafb0 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f977dfe0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977816b140 con 0x7f9778158280 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97780b0970 0x7f97780ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97780b0970 0x7f97780ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44070/0 (socket says 172.21.15.7:44070) 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 -- 172.21.15.7:0/1041725118 learned_addr learned my addr 172.21.15.7:0/1041725118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f9777fff700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 0x7f97780c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f9776ffd700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 0x7f977816a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 -- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 msgr2=0x7f97780c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 0x7f97780c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 -- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 msgr2=0x7f977816a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 0x7f977816a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 -- 172.21.15.7:0/1041725118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9768009420 con 0x7f97780b0970 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.994+0000 7f97777fe700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97780b0970 0x7f97780ca480 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f977806a1d0 tx=0x7f9768017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.995+0000 7f975f7fe700 1 -- 172.21.15.7:0/1041725118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768011900 con 0x7f97780b0970 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.995+0000 7f975f7fe700 1 -- 172.21.15.7:0/1041725118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768011a60 con 0x7f97780b0970 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.995+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977816b490 con 0x7f97780b0970 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.995+0000 7f975f7fe700 1 -- 172.21.15.7:0/1041725118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768025640 con 0x7f97780b0970 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.995+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977816a840 con 0x7f97780b0970 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.996+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977816b930 con 0x7f97780b0970 2024-01-11T07:11:45.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.996+0000 7f975f7fe700 1 -- 172.21.15.7:0/1041725118 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9768025a30 con 0x7f97780b0970 2024-01-11T07:11:46.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.997+0000 7f975f7fe700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9764066f60 0x7f9764069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.997+0000 7f975f7fe700 1 -- 172.21.15.7:0/1041725118 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9768085c00 con 0x7f97780b0970 2024-01-11T07:11:46.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.997+0000 7f975f7fe700 1 -- 172.21.15.7:0/1041725118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f976406ea80 con 0x7f97780b0970 2024-01-11T07:11:46.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.997+0000 7f9776ffd700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9764066f60 0x7f9764069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:45.997+0000 7f9776ffd700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9764066f60 0x7f9764069410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97781535d0 tx=0x7f976c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.001+0000 7f975f7fe700 1 -- 172.21.15.7:0/1041725118 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f976808c080 con 0x7f97780b0970 2024-01-11T07:11:46.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.207+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f977804c3c0 con 0x7f97780b0970 2024-01-11T07:11:46.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.208+0000 7f975f7fe700 1 -- 172.21.15.7:0/1041725118 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15334 (secure 0 0 0) 0x7f9768015030 con 0x7f97780b0970 2024-01-11T07:11:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:46.209 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:11:45.280339+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:10:53.813087+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-11T07:10:54.744603+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.210+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9764066f60 msgr2=0x7f9764069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.210+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9764066f60 0x7f9764069410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97781535d0 tx=0x7f976c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97780b0970 msgr2=0x7f97780ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97780b0970 0x7f97780ca480 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f977806a1d0 tx=0x7f9768017000 comp rx=0 tx=0).stop 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 shutdown_connections 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9764066f60 0x7f9764069410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9778158280 0x7f97780c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97780b0970 0x7f97780ca480 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 --2- 172.21.15.7:0/1041725118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97780cafb0 0x7f977816a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 >> 172.21.15.7:0/1041725118 conn(0x7f9778071100 msgr2=0x7f97780cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 shutdown_connections 2024-01-11T07:11:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.211+0000 7f977dfe0700 1 -- 172.21.15.7:0/1041725118 wait complete. 2024-01-11T07:11:46.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:11:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.359+0000 7f60dcc3f700 1 Processor -- start 2024-01-11T07:11:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.359+0000 7f60dcc3f700 1 -- start start 2024-01-11T07:11:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.360+0000 7f60dcc3f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60d80cafb0 0x7f60d80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60dcc3f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80b0b40 0x7f60d8161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60dcc3f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d8159300 con 0x7f60d8158280 2024-01-11T07:11:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60dcc3f700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d80cf260 con 0x7f60d80cafb0 2024-01-11T07:11:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60dcc3f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80cf540 con 0x7f60d80b0b40 2024-01-11T07:11:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d659c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60d80cafb0 0x7f60d80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d5d9b700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f60d8158280 0x7f60d80b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:42882/0 (socket says 172.21.15.7:42882) 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d5d9b700 1 -- 172.21.15.7:0/1654409348 learned_addr learned my addr 172.21.15.7:0/1654409348 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d6d9d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80b0b40 0x7f60d8161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d659c700 1 -- 172.21.15.7:0/1654409348 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80b0b40 msgr2=0x7f60d8161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d659c700 1 --2- 172.21.15.7:0/1654409348 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80b0b40 0x7f60d8161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d659c700 1 -- 172.21.15.7:0/1654409348 >> v1:172.21.15.113:6789/0 conn(0x7f60d8158280 legacy=0x7f60d80b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d659c700 1 -- 172.21.15.7:0/1654409348 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d80cf820 con 0x7f60d80cafb0 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d559a700 1 -- 172.21.15.7:0/1654409348 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523047341 0 0) 0x7f60d8159300 con 0x7f60d8158280 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d6d9d700 1 --2- 172.21.15.7:0/1654409348 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80b0b40 0x7f60d8161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.361+0000 7f60d659c700 1 --2- 172.21.15.7:0/1654409348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d80cafb0 0x7f60d80ce8f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f60c80098e0 tx=0x7f60c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67fbec7678d4b0a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60d559a700 1 -- 172.21.15.7:0/1654409348 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800b640 con 0x7f60d80cafb0 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60d559a700 1 -- 172.21.15.7:0/1654409348 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c800b7a0 con 0x7f60d80cafb0 2024-01-11T07:11:46.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60d559a700 1 -- 172.21.15.7:0/1654409348 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800bb40 con 0x7f60d80cafb0 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1654409348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d80cafb0 msgr2=0x7f60d80ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1654409348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d80cafb0 0x7f60d80ce8f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f60c80098e0 tx=0x7f60c80091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1654409348 shutdown_connections 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1654409348 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80b0b40 0x7f60d8161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1654409348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d80cafb0 0x7f60d80ce8f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1654409348 >> 172.21.15.7:0/1654409348 conn(0x7f60d8071100 msgr2=0x7f60d806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1654409348 shutdown_connections 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.362+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1654409348 wait complete. 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.364+0000 7f60dcc3f700 1 Processor -- start 2024-01-11T07:11:46.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.364+0000 7f60dcc3f700 1 -- start start 2024-01-11T07:11:46.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60dcc3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d80b0b40 0x7f60d80c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60dcc3f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80cafb0 0x7f60d8148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60dcc3f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d8158280 0x7f60d80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60dcc3f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8148ad0 con 0x7f60d80b0b40 2024-01-11T07:11:46.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60dcc3f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d816b2b0 con 0x7f60d8158280 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60dcc3f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8154a10 con 0x7f60d80cafb0 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60d659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d80b0b40 0x7f60d80c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60d659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d80b0b40 0x7f60d80c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44084/0 (socket says 172.21.15.7:44084) 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60d659c700 1 -- 172.21.15.7:0/1942147528 learned_addr learned my addr 172.21.15.7:0/1942147528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.365+0000 7f60d5d9b700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80cafb0 0x7f60d8148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60d6d9d700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d8158280 0x7f60d80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60d659c700 1 -- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80cafb0 msgr2=0x7f60d8148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60d659c700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80cafb0 0x7f60d8148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60d659c700 1 -- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d8158280 msgr2=0x7f60d80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60d659c700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d8158280 0x7f60d80b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60d659c700 1 -- 172.21.15.7:0/1942147528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d0009a20 con 0x7f60d80b0b40 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60d6d9d700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d8158280 0x7f60d80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60d659c700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d80b0b40 0x7f60d80c3800 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f60d806a1d0 tx=0x7f60c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:46.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c800bb20 con 0x7f60d80b0b40 2024-01-11T07:11:46.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c8011420 con 0x7f60d80b0b40 2024-01-11T07:11:46.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c8009420 con 0x7f60d80b0b40 2024-01-11T07:11:46.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c80117a0 con 0x7f60d80b0b40 2024-01-11T07:11:46.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.366+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d816b830 con 0x7f60d80b0b40 2024-01-11T07:11:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.367+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8159260 con 0x7f60d80b0b40 2024-01-11T07:11:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.367+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f60c8011b50 con 0x7f60d80b0b40 2024-01-11T07:11:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.368+0000 7f60ce7fc700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f60b8066f30 0x7f60b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.368+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f60c8085c30 con 0x7f60d80b0b40 2024-01-11T07:11:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.368+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f60b806ea50 con 0x7f60d80b0b40 2024-01-11T07:11:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.368+0000 7f60d5d9b700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f60b8066f30 0x7f60b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.369+0000 7f60d5d9b700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f60b8066f30 0x7f60b80693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f60d8156900 tx=0x7f60c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.373+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60c8052ea0 con 0x7f60d80b0b40 2024-01-11T07:11:46.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.625+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f60d804c3c0 con 0x7f60d80b0b40 2024-01-11T07:11:46.626 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:11:46.625+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:11:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.627+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(90..90 src has 1..90) v4 ==== 366+0+0 (secure 0 0 0) 0x7f60c8057260 con 0x7f60d80b0b40 2024-01-11T07:11:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.627+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f60b806f330 con 0x7f60d80b0b40 2024-01-11T07:11:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.629+0000 7f60ce7fc700 1 -- 172.21.15.7:0/1942147528 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f60c8029030 con 0x7f60d80b0b40 2024-01-11T07:11:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.631+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f60b8066f30 msgr2=0x7f60b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.631+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f60b8066f30 0x7f60b80693e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f60d8156900 tx=0x7f60c0009040 comp rx=0 tx=0).stop 2024-01-11T07:11:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.631+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d80b0b40 msgr2=0x7f60d80c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.631+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d80b0b40 0x7f60d80c3800 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f60d806a1d0 tx=0x7f60c8017000 comp rx=0 tx=0).stop 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.634+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 shutdown_connections 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.634+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f60b8066f30 0x7f60b80693e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.634+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d8158280 0x7f60d80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.634+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60d80b0b40 0x7f60d80c3800 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.634+0000 7f60dcc3f700 1 --2- 172.21.15.7:0/1942147528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d80cafb0 0x7f60d8148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.634+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 >> 172.21.15.7:0/1942147528 conn(0x7f60d8071100 msgr2=0x7f60d806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.634+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 shutdown_connections 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.634+0000 7f60dcc3f700 1 -- 172.21.15.7:0/1942147528 wait complete. 2024-01-11T07:11:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:11:46.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:11:46.667 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:11:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.781+0000 7fb5ed942700 1 Processor -- start 2024-01-11T07:11:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.781+0000 7fb5ed942700 1 -- start start 2024-01-11T07:11:46.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5ed942700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 0x7fb5e80af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5ed942700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5e806e340 0x7fb5e815c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5ed942700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e815c080 con 0x7fb5e80d16e0 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5ed942700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e80b00f0 con 0x7fb5e806e340 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5ed942700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e815ba30 con 0x7fb5e8159e30 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5e77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5e806e340 0x7fb5e815c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5e77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5e806e340 0x7fb5e815c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44098/0 (socket says 172.21.15.7:44098) 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5e77fe700 1 -- 172.21.15.7:0/1934806865 learned_addr learned my addr 172.21.15.7:0/1934806865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.782+0000 7fb5e6ffd700 1 --2- 172.21.15.7:0/1934806865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 0x7fb5e80af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e77fe700 1 -- 172.21.15.7:0/1934806865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 msgr2=0x7fb5e80af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e77fe700 1 --2- 172.21.15.7:0/1934806865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 0x7fb5e80af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e77fe700 1 -- 172.21.15.7:0/1934806865 >> v1:172.21.15.113:6789/0 conn(0x7fb5e80d16e0 legacy=0x7fb5e806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e5ffb700 1 -- 172.21.15.7:0/1934806865 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804099244 0 0) 0x7fb5e815c080 con 0x7fb5e80d16e0 2024-01-11T07:11:46.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e77fe700 1 -- 172.21.15.7:0/1934806865 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e815bd10 con 0x7fb5e806e340 2024-01-11T07:11:46.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e77fe700 1 --2- 172.21.15.7:0/1934806865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e815c510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009f20 tx=0x7fb5e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d603cd55cf48ea7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:46.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e5ffb700 1 -- 172.21.15.7:0/1934806865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000e070 con 0x7fb5e806e340 2024-01-11T07:11:46.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e5ffb700 1 -- 172.21.15.7:0/1934806865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e000b9d0 con 0x7fb5e806e340 2024-01-11T07:11:46.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.783+0000 7fb5e5ffb700 1 -- 172.21.15.7:0/1934806865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00092e0 con 0x7fb5e806e340 2024-01-11T07:11:46.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.784+0000 7fb5ed942700 1 -- 172.21.15.7:0/1934806865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 msgr2=0x7fb5e815c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.784+0000 7fb5ed942700 1 --2- 172.21.15.7:0/1934806865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e815c510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009f20 tx=0x7fb5e0009840 comp rx=0 tx=0).stop 2024-01-11T07:11:46.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.784+0000 7fb5ed942700 1 -- 172.21.15.7:0/1934806865 shutdown_connections 2024-01-11T07:11:46.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.784+0000 7fb5ed942700 1 --2- 172.21.15.7:0/1934806865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e815c510 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.784+0000 7fb5ed942700 1 --2- 172.21.15.7:0/1934806865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 0x7fb5e80af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.784+0000 7fb5ed942700 1 -- 172.21.15.7:0/1934806865 >> 172.21.15.7:0/1934806865 conn(0x7fb5e8063d00 msgr2=0x7fb5e80d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:46.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.784+0000 7fb5ed942700 1 -- 172.21.15.7:0/1934806865 shutdown_connections 2024-01-11T07:11:46.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.784+0000 7fb5ed942700 1 -- 172.21.15.7:0/1934806865 wait complete. 2024-01-11T07:11:46.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.786+0000 7fb5ed942700 1 Processor -- start 2024-01-11T07:11:46.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.786+0000 7fb5ed942700 1 -- start start 2024-01-11T07:11:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5ed942700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e80a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5ed942700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e80d16e0 0x7fb5e8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5ed942700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 0x7fb5e8157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5ed942700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8158e50 con 0x7fb5e806e340 2024-01-11T07:11:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5ed942700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e81583f0 con 0x7fb5e80d16e0 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5ed942700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e816af00 con 0x7fb5e8159e30 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e80a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e80a85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44114/0 (socket says 172.21.15.7:44114) 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 -- 172.21.15.7:0/2222672743 learned_addr learned my addr 172.21.15.7:0/2222672743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e77fe700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 0x7fb5e8157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e67fc700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e80d16e0 0x7fb5e8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 -- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 msgr2=0x7fb5e8157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 0x7fb5e8157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 -- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e80d16e0 msgr2=0x7fb5e8157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e80d16e0 0x7fb5e8157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 -- 172.21.15.7:0/2222672743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e0009ae0 con 0x7fb5e806e340 2024-01-11T07:11:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e67fc700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e80d16e0 0x7fb5e8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.787+0000 7fb5e6ffd700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e80a85c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800e960 tx=0x7fb5d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.788+0000 7fb5deffd700 1 -- 172.21.15.7:0/2222672743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800c850 con 0x7fb5e806e340 2024-01-11T07:11:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.788+0000 7fb5deffd700 1 -- 172.21.15.7:0/2222672743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d800c9b0 con 0x7fb5e806e340 2024-01-11T07:11:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.788+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e816a8b0 con 0x7fb5e806e340 2024-01-11T07:11:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.788+0000 7fb5deffd700 1 -- 172.21.15.7:0/2222672743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800cd50 con 0x7fb5e806e340 2024-01-11T07:11:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.788+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e816b450 con 0x7fb5e806e340 2024-01-11T07:11:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.789+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e80d1ad0 con 0x7fb5e806e340 2024-01-11T07:11:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.789+0000 7fb5deffd700 1 -- 172.21.15.7:0/2222672743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb5d8013680 con 0x7fb5e806e340 2024-01-11T07:11:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.790+0000 7fb5deffd700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb5c8066f30 0x7fb5c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.790+0000 7fb5deffd700 1 -- 172.21.15.7:0/2222672743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fb5d8084760 con 0x7fb5e806e340 2024-01-11T07:11:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.790+0000 7fb5e67fc700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb5c8066f30 0x7fb5c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.790+0000 7fb5deffd700 1 -- 172.21.15.7:0/2222672743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7fb5c806eaf0 con 0x7fb5e806e340 2024-01-11T07:11:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.790+0000 7fb5e67fc700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb5c8066f30 0x7fb5c80693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80b2d90 tx=0x7fb5d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:46.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:46.794+0000 7fb5deffd700 1 -- 172.21.15.7:0/2222672743 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5d8017060 con 0x7fb5e806e340 2024-01-11T07:11:47.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.046+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb5e804c3c0 con 0x7fb5e806e340 2024-01-11T07:11:47.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.630+0000 7fb5deffd700 1 -- 172.21.15.7:0/2222672743 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5d8051980 con 0x7fb5e806e340 2024-01-11T07:11:47.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb5c8066f30 msgr2=0x7fb5c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:47.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb5c8066f30 0x7fb5c80693e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80b2d90 tx=0x7fb5d0009040 comp rx=0 tx=0).stop 2024-01-11T07:11:47.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 msgr2=0x7fb5e80a85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:47.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e80a85c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800e960 tx=0x7fb5d80091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 shutdown_connections 2024-01-11T07:11:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb5c8066f30 0x7fb5c80693e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb5e8159e30 0x7fb5e8157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5e806e340 0x7fb5e80a85c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 --2- 172.21.15.7:0/2222672743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5e80d16e0 0x7fb5e8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.632+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 >> 172.21.15.7:0/2222672743 conn(0x7fb5e8063d00 msgr2=0x7fb5e80c73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.633+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 shutdown_connections 2024-01-11T07:11:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.633+0000 7fb5ed942700 1 -- 172.21.15.7:0/2222672743 wait complete. 2024-01-11T07:11:47.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:11:47.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.788+0000 7f924d0da700 1 Processor -- start 2024-01-11T07:11:47.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.788+0000 7f924d0da700 1 -- start start 2024-01-11T07:11:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924d0da700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f924815d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924d0da700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f924815de80 0x7f924815b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924d0da700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924815e2c0 con 0x7f924815d100 2024-01-11T07:11:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924d0da700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9248069f90 con 0x7f924815de80 2024-01-11T07:11:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924d0da700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924806a270 con 0x7f924815d4d0 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f924815de80 0x7f924815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f924815de80 0x7f924815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59762/0 (socket says 172.21.15.7:59762) 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924759e700 1 -- 172.21.15.7:0/804309921 learned_addr learned my addr 172.21.15.7:0/804309921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.789+0000 7f924659c700 1 --2- 172.21.15.7:0/804309921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f924815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.790+0000 7f924759e700 1 -- 172.21.15.7:0/804309921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 msgr2=0x7f924815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.790+0000 7f924759e700 1 --2- 172.21.15.7:0/804309921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f924815d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.790+0000 7f924759e700 1 -- 172.21.15.7:0/804309921 >> v1:172.21.15.113:6789/0 conn(0x7f924815d100 legacy=0x7f9248069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.790+0000 7f924759e700 1 -- 172.21.15.7:0/804309921 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924806a550 con 0x7f924815de80 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.790+0000 7f924759e700 1 --2- 172.21.15.7:0/804309921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815de80 0x7f924815b510 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9240009f20 tx=0x7f9240009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8d43e794992f48 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.790+0000 7f9245d9b700 1 -- 172.21.15.7:0/804309921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924000e070 con 0x7f924815de80 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.790+0000 7f9245d9b700 1 -- 172.21.15.7:0/804309921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924000ba80 con 0x7f924815de80 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.790+0000 7f9245d9b700 1 -- 172.21.15.7:0/804309921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92400092e0 con 0x7f924815de80 2024-01-11T07:11:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.791+0000 7f924d0da700 1 -- 172.21.15.7:0/804309921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815de80 msgr2=0x7f924815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.791+0000 7f924d0da700 1 --2- 172.21.15.7:0/804309921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815de80 0x7f924815b510 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9240009f20 tx=0x7f9240009840 comp rx=0 tx=0).stop 2024-01-11T07:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.791+0000 7f924d0da700 1 -- 172.21.15.7:0/804309921 shutdown_connections 2024-01-11T07:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.791+0000 7f924d0da700 1 --2- 172.21.15.7:0/804309921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815de80 0x7f924815b510 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.791+0000 7f924d0da700 1 --2- 172.21.15.7:0/804309921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f924815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.791+0000 7f924d0da700 1 -- 172.21.15.7:0/804309921 >> 172.21.15.7:0/804309921 conn(0x7f92480680b0 msgr2=0x7f92480cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.791+0000 7f924d0da700 1 -- 172.21.15.7:0/804309921 shutdown_connections 2024-01-11T07:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.791+0000 7f924d0da700 1 -- 172.21.15.7:0/804309921 wait complete. 2024-01-11T07:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.792+0000 7f924d0da700 1 Processor -- start 2024-01-11T07:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.792+0000 7f924d0da700 1 -- start start 2024-01-11T07:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924d0da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815d100 0x7f924813c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924d0da700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f92480c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924d0da700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f924815de80 0x7f92480c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924d0da700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92480c6e10 con 0x7f924815d100 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924d0da700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92480c89e0 con 0x7f924815de80 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924d0da700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924815c000 con 0x7f924815d4d0 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f9246d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815d100 0x7f924813c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f9246d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815d100 0x7f924813c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59764/0 (socket says 172.21.15.7:59764) 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f9246d9d700 1 -- 172.21.15.7:0/3201159556 learned_addr learned my addr 172.21.15.7:0/3201159556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924759e700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f924815de80 0x7f92480c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924659c700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f92480c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f9246d9d700 1 -- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 msgr2=0x7f92480c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f9246d9d700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f92480c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f9246d9d700 1 -- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f924815de80 msgr2=0x7f92480c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f9246d9d700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f924815de80 0x7f92480c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f9246d9d700 1 -- 172.21.15.7:0/3201159556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9240009ae0 con 0x7f924815d100 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924759e700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f924815de80 0x7f92480c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.793+0000 7f924659c700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f92480c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.794+0000 7f9246d9d700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815d100 0x7f924813c990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f923800c590 tx=0x7f923800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.794+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238011590 con 0x7f924815d100 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.794+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92380116f0 con 0x7f924815d100 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.794+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924815c2e0 con 0x7f924815d100 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.794+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238011ab0 con 0x7f924815d100 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.794+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924815bcd0 con 0x7f924815d100 2024-01-11T07:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.795+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924815c780 con 0x7f924815d100 2024-01-11T07:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.795+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f923801b3b0 con 0x7f924815d100 2024-01-11T07:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.796+0000 7f923effd700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9228066f30 0x7f92280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.796+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f9238083b30 con 0x7f924815d100 2024-01-11T07:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.796+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f922806eaf0 con 0x7f924815d100 2024-01-11T07:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.796+0000 7f924659c700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9228066f30 0x7f92280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.796+0000 7f924659c700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9228066f30 0x7f92280693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f92480c84e0 tx=0x7f9230008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:47.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:47.800+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9238017070 con 0x7f924815d100 2024-01-11T07:11:48.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.004+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f924804c3c0 con 0x7f924815d100 2024-01-11T07:11:48.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.637+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(91..91 src has 1..91) v4 ==== 1115+0+0 (secure 0 0 0) 0x7f9238055110 con 0x7f924815d100 2024-01-11T07:11:48.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.637+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7f922806cf20 con 0x7f924815d100 2024-01-11T07:11:48.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.639+0000 7f923effd700 1 -- 172.21.15.7:0/3201159556 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9238050d50 con 0x7f924815d100 2024-01-11T07:11:48.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.643+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9228066f30 msgr2=0x7f92280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:48.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.643+0000 7f924d0da700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9228066f30 0x7f92280693e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f92480c84e0 tx=0x7f9230008040 comp rx=0 tx=0).stop 2024-01-11T07:11:48.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.643+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815d100 msgr2=0x7f924813c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:48.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.643+0000 7f924d0da700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815d100 0x7f924813c990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f923800c590 tx=0x7f923800c950 comp rx=0 tx=0).stop 2024-01-11T07:11:48.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.645+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 shutdown_connections 2024-01-11T07:11:48.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.645+0000 7f924d0da700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9228066f30 0x7f92280693e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:48.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.645+0000 7f924d0da700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f924815de80 0x7f92480c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:48.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.645+0000 7f924d0da700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924815d100 0x7f924813c990 secure :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f923800c590 tx=0x7f923800c950 comp rx=0 tx=0).stop 2024-01-11T07:11:48.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.645+0000 7f924d0da700 1 --2- 172.21.15.7:0/3201159556 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f924815d4d0 0x7f92480c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:48.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.645+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 >> 172.21.15.7:0/3201159556 conn(0x7f92480680b0 msgr2=0x7f924806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:48.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.649+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 shutdown_connections 2024-01-11T07:11:48.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.649+0000 7f924d0da700 1 -- 172.21.15.7:0/3201159556 wait complete. 2024-01-11T07:11:48.651 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:11:48.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:11:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.806+0000 7f5b8c782700 1 Processor -- start 2024-01-11T07:11:48.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.806+0000 7f5b8c782700 1 -- start start 2024-01-11T07:11:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8c782700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063480 0x7f5b840acc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:48.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8c782700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b84069c50 0x7f5b84159820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8c782700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b840c7e70 con 0x7f5b84063850 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8c782700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b840c8150 con 0x7f5b84069c50 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8c782700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8415e180 con 0x7f5b84063480 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8ad1f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b84069c50 0x7f5b84159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b89d1d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5b84063850 0x7f5b8415d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45266/0 (socket says 172.21.15.7:45266) 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b89d1d700 1 -- 172.21.15.7:0/1990884970 learned_addr learned my addr 172.21.15.7:0/1990884970 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8a51e700 1 --2- 172.21.15.7:0/1990884970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063480 0x7f5b840acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8ad1f700 1 -- 172.21.15.7:0/1990884970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063480 msgr2=0x7f5b840acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8ad1f700 1 --2- 172.21.15.7:0/1990884970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063480 0x7f5b840acc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.807+0000 7f5b8ad1f700 1 -- 172.21.15.7:0/1990884970 >> v1:172.21.15.7:6789/0 conn(0x7f5b84063850 legacy=0x7f5b8415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.808+0000 7f5b8ad1f700 1 -- 172.21.15.7:0/1990884970 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8415e460 con 0x7f5b84069c50 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.808+0000 7f5b8951c700 1 -- 172.21.15.7:0/1990884970 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787896947 0 0) 0x7f5b840c7e70 con 0x7f5b84063850 2024-01-11T07:11:48.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.808+0000 7f5b8ad1f700 1 --2- 172.21.15.7:0/1990884970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84069c50 0x7f5b84159820 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009e60 tx=0x7f5b7c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09006461cda541f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.808+0000 7f5b8951c700 1 -- 172.21.15.7:0/1990884970 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00e070 con 0x7f5b84069c50 2024-01-11T07:11:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.808+0000 7f5b8951c700 1 -- 172.21.15.7:0/1990884970 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c00bc30 con 0x7f5b84069c50 2024-01-11T07:11:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.808+0000 7f5b8951c700 1 -- 172.21.15.7:0/1990884970 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c012430 con 0x7f5b84069c50 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.808+0000 7f5b8c782700 1 -- 172.21.15.7:0/1990884970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84069c50 msgr2=0x7f5b84159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.808+0000 7f5b8c782700 1 --2- 172.21.15.7:0/1990884970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84069c50 0x7f5b84159820 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009e60 tx=0x7f5b7c009780 comp rx=0 tx=0).stop 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.809+0000 7f5b8c782700 1 -- 172.21.15.7:0/1990884970 shutdown_connections 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.809+0000 7f5b8c782700 1 --2- 172.21.15.7:0/1990884970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84069c50 0x7f5b84159820 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.809+0000 7f5b8c782700 1 --2- 172.21.15.7:0/1990884970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063480 0x7f5b840acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.809+0000 7f5b8c782700 1 -- 172.21.15.7:0/1990884970 >> 172.21.15.7:0/1990884970 conn(0x7f5b840680a0 msgr2=0x7f5b840cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.809+0000 7f5b8c782700 1 -- 172.21.15.7:0/1990884970 shutdown_connections 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.809+0000 7f5b8c782700 1 -- 172.21.15.7:0/1990884970 wait complete. 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.810+0000 7f5b8c782700 1 Processor -- start 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.810+0000 7f5b8c782700 1 -- start start 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8c782700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84063480 0x7f5b840b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8c782700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063850 0x7f5b840b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8c782700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b84069c50 0x7f5b840b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8c782700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840c5990 con 0x7f5b84063480 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8c782700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840c5c70 con 0x7f5b84069c50 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8c782700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840c5f50 con 0x7f5b84063850 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84063480 0x7f5b840b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84063480 0x7f5b840b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59784/0 (socket says 172.21.15.7:59784) 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 -- 172.21.15.7:0/3552036401 learned_addr learned my addr 172.21.15.7:0/3552036401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b89d1d700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063850 0x7f5b840b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8ad1f700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b84069c50 0x7f5b840b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 -- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063850 msgr2=0x7f5b840b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063850 0x7f5b840b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 -- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b84069c50 msgr2=0x7f5b840b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b84069c50 0x7f5b840b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 -- 172.21.15.7:0/3552036401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c009a20 con 0x7f5b84063480 2024-01-11T07:11:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b8a51e700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84063480 0x7f5b840b2ea0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000d890 tx=0x7f5b80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.811+0000 7f5b767fc700 1 -- 172.21.15.7:0/3552036401 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800125f0 con 0x7f5b84063480 2024-01-11T07:11:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.812+0000 7f5b767fc700 1 -- 172.21.15.7:0/3552036401 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b80012750 con 0x7f5b84063480 2024-01-11T07:11:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.812+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b840c6230 con 0x7f5b84063480 2024-01-11T07:11:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.812+0000 7f5b767fc700 1 -- 172.21.15.7:0/3552036401 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80012af0 con 0x7f5b84063480 2024-01-11T07:11:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.812+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b840b1580 con 0x7f5b84063480 2024-01-11T07:11:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.813+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8404c3c0 con 0x7f5b84063480 2024-01-11T07:11:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.813+0000 7f5b767fc700 1 -- 172.21.15.7:0/3552036401 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5b8001e460 con 0x7f5b84063480 2024-01-11T07:11:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.814+0000 7f5b767fc700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b68066f30 0x7f5b680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.814+0000 7f5b767fc700 1 -- 172.21.15.7:0/3552036401 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5411+0+0 (secure 0 0 0) 0x7f5b80084a20 con 0x7f5b84063480 2024-01-11T07:11:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.814+0000 7f5b89d1d700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b68066f30 0x7f5b680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.814+0000 7f5b89d1d700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b68066f30 0x7f5b680693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5b78007fd0 tx=0x7f5b78007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:48.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:48.818+0000 7f5b767fc700 1 -- 172.21.15.7:0/3552036401 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b80051d10 con 0x7f5b84063480 2024-01-11T07:11:49.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.020+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5b8415e740 con 0x7f5b84063480 2024-01-11T07:11:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.645+0000 7f5b767fc700 1 -- 172.21.15.7:0/3552036401 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5b80055330 con 0x7f5b84063480 2024-01-11T07:11:49.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.654+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b68066f30 msgr2=0x7f5b680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.654+0000 7f5b8c782700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b68066f30 0x7f5b680693e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5b78007fd0 tx=0x7f5b78007d30 comp rx=0 tx=0).stop 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.654+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84063480 msgr2=0x7f5b840b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.654+0000 7f5b8c782700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84063480 0x7f5b840b2ea0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000d890 tx=0x7f5b80008040 comp rx=0 tx=0).stop 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.655+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 shutdown_connections 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.655+0000 7f5b8c782700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b68066f30 0x7f5b680693e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.655+0000 7f5b8c782700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b84069c50 0x7f5b840b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.655+0000 7f5b8c782700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84063480 0x7f5b840b2ea0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.655+0000 7f5b8c782700 1 --2- 172.21.15.7:0/3552036401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b84063850 0x7f5b840b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.655+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 >> 172.21.15.7:0/3552036401 conn(0x7f5b840680a0 msgr2=0x7f5b840cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.655+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 shutdown_connections 2024-01-11T07:11:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.655+0000 7f5b8c782700 1 -- 172.21.15.7:0/3552036401 wait complete. 2024-01-11T07:11:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:11:49.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:11:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.807+0000 7f08fb583700 1 Processor -- start 2024-01-11T07:11:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.807+0000 7f08fb583700 1 -- start start 2024-01-11T07:11:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08fb583700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08f406dd00 0x7f08f40c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08fb583700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08f406e0d0 0x7f08f4159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08fb583700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f415b770 con 0x7f08f4159ff0 2024-01-11T07:11:49.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08fb583700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f415abf0 con 0x7f08f406e0d0 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08fb583700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f415aed0 con 0x7f08f406dd00 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08f8b1e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08f406e0d0 0x7f08f4159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08f9b20700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f08f4159ff0 0x7f08f40c7e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45270/0 (socket says 172.21.15.7:45270) 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08f9b20700 1 -- 172.21.15.7:0/1637194571 learned_addr learned my addr 172.21.15.7:0/1637194571 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.808+0000 7f08f931f700 1 --2- 172.21.15.7:0/1637194571 >> v2:172.21.15.113:3300/0 conn(0x7f08f406dd00 0x7f08f40c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08f8b1e700 1 -- 172.21.15.7:0/1637194571 >> v2:172.21.15.113:3300/0 conn(0x7f08f406dd00 msgr2=0x7f08f40c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08f8b1e700 1 --2- 172.21.15.7:0/1637194571 >> v2:172.21.15.113:3300/0 conn(0x7f08f406dd00 0x7f08f40c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08f8b1e700 1 -- 172.21.15.7:0/1637194571 >> v1:172.21.15.7:6789/0 conn(0x7f08f4159ff0 legacy=0x7f08f40c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08e7fff700 1 -- 172.21.15.7:0/1637194571 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130405323 0 0) 0x7f08f415b770 con 0x7f08f4159ff0 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08f8b1e700 1 -- 172.21.15.7:0/1637194571 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f415b1b0 con 0x7f08f406e0d0 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08f931f700 1 --2- 172.21.15.7:0/1637194571 >> v2:172.21.15.113:3300/0 conn(0x7f08f406dd00 0x7f08f40c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08f8b1e700 1 --2- 172.21.15.7:0/1637194571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f406e0d0 0x7f08f4159ab0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f08f406d7f0 tx=0x7f08e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172e7830faf02a98 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08e7fff700 1 -- 172.21.15.7:0/1637194571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800b6b0 con 0x7f08f406e0d0 2024-01-11T07:11:49.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08e7fff700 1 -- 172.21.15.7:0/1637194571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e800b810 con 0x7f08f406e0d0 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.809+0000 7f08e7fff700 1 -- 172.21.15.7:0/1637194571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800bbb0 con 0x7f08f406e0d0 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.810+0000 7f08fb583700 1 -- 172.21.15.7:0/1637194571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f406e0d0 msgr2=0x7f08f4159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.810+0000 7f08fb583700 1 --2- 172.21.15.7:0/1637194571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f406e0d0 0x7f08f4159ab0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f08f406d7f0 tx=0x7f08e80091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.810+0000 7f08fb583700 1 -- 172.21.15.7:0/1637194571 shutdown_connections 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.810+0000 7f08fb583700 1 --2- 172.21.15.7:0/1637194571 >> v2:172.21.15.113:3300/0 conn(0x7f08f406dd00 0x7f08f40c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.810+0000 7f08fb583700 1 --2- 172.21.15.7:0/1637194571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f406e0d0 0x7f08f4159ab0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.810+0000 7f08fb583700 1 -- 172.21.15.7:0/1637194571 >> 172.21.15.7:0/1637194571 conn(0x7f08f40680e0 msgr2=0x7f08f40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.810+0000 7f08fb583700 1 -- 172.21.15.7:0/1637194571 shutdown_connections 2024-01-11T07:11:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.810+0000 7f08fb583700 1 -- 172.21.15.7:0/1637194571 wait complete. 2024-01-11T07:11:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.811+0000 7f08fb583700 1 Processor -- start 2024-01-11T07:11:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.811+0000 7f08fb583700 1 -- start start 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08fb583700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f406dd00 0x7f08f40b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08fb583700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08f406e0d0 0x7f08f40c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08fb583700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4159ff0 0x7f08f40c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08fb583700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40c5b90 con 0x7f08f4159ff0 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08fb583700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40c5e40 con 0x7f08f406dd00 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08fb583700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40c6120 con 0x7f08f406e0d0 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4159ff0 0x7f08f40c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4159ff0 0x7f08f40c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59806/0 (socket says 172.21.15.7:59806) 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 -- 172.21.15.7:0/106511410 learned_addr learned my addr 172.21.15.7:0/106511410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f931f700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f406dd00 0x7f08f40b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f8b1e700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08f406e0d0 0x7f08f40c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 -- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08f406e0d0 msgr2=0x7f08f40c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08f406e0d0 0x7f08f40c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 -- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f406dd00 msgr2=0x7f08f40b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f406dd00 0x7f08f40b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 -- 172.21.15.7:0/106511410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e8009420 con 0x7f08f4159ff0 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08f9b20700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4159ff0 0x7f08f40c5650 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00cc10 tx=0x7f08ec001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08e57fa700 1 -- 172.21.15.7:0/106511410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00a850 con 0x7f08f4159ff0 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08e57fa700 1 -- 172.21.15.7:0/106511410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec00a9d0 con 0x7f08f4159ff0 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f40c6400 con 0x7f08f4159ff0 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08e57fa700 1 -- 172.21.15.7:0/106511410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00ad90 con 0x7f08f4159ff0 2024-01-11T07:11:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.814+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40c69c0 con 0x7f08f4159ff0 2024-01-11T07:11:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.815+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f404c3c0 con 0x7f08f4159ff0 2024-01-11T07:11:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.815+0000 7f08e57fa700 1 -- 172.21.15.7:0/106511410 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f08ec007300 con 0x7f08f4159ff0 2024-01-11T07:11:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.815+0000 7f08e57fa700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.816+0000 7f08e57fa700 1 -- 172.21.15.7:0/106511410 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f08ec084450 con 0x7f08f4159ff0 2024-01-11T07:11:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.816+0000 7f08f931f700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:49.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.816+0000 7f08f931f700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006f00 tx=0x7f08f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:49.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:49.820+0000 7f08e57fa700 1 -- 172.21.15.7:0/106511410 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08ec051a40 con 0x7f08f4159ff0 2024-01-11T07:11:50.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.026+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f08f415b490 con 0x7f08f4159ff0 2024-01-11T07:11:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.030+0000 7f08e57fa700 1 -- 172.21.15.7:0/106511410 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f08ec055060 con 0x7f08f4159ff0 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f08d8066f30 msgr2=0x7f08d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006f00 tx=0x7f08f0008040 comp rx=0 tx=0).stop 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4159ff0 msgr2=0x7f08f40c5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4159ff0 0x7f08f40c5650 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00cc10 tx=0x7f08ec001970 comp rx=0 tx=0).stop 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 shutdown_connections 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08f4159ff0 0x7f08f40c5650 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08f406dd00 0x7f08f40b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 --2- 172.21.15.7:0/106511410 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f08f406e0d0 0x7f08f40c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 >> 172.21.15.7:0/106511410 conn(0x7f08f40680e0 msgr2=0x7f08f40c96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 shutdown_connections 2024-01-11T07:11:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.032+0000 7f08fb583700 1 -- 172.21.15.7:0/106511410 wait complete. 2024-01-11T07:11:50.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-01-11T07:11:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.178+0000 7f0343612700 1 Processor -- start 2024-01-11T07:11:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.178+0000 7f0343612700 1 -- start start 2024-01-11T07:11:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f0343612700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f033c0cafb0 0x7f033c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f0343612700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f0343612700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c159300 con 0x7f033c158280 2024-01-11T07:11:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f0343612700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0cf260 con 0x7f033c0cafb0 2024-01-11T07:11:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f0343612700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0cf540 con 0x7f033c0b0b40 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f03413ae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f033c0cafb0 0x7f033c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f03413ae700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f033c0cafb0 0x7f033c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59814/0 (socket says 172.21.15.7:59814) 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f03413ae700 1 -- 172.21.15.7:0/1938116734 learned_addr learned my addr 172.21.15.7:0/1938116734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.179+0000 7f0341baf700 1 --2- 172.21.15.7:0/1938116734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.180+0000 7f03413ae700 1 -- 172.21.15.7:0/1938116734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 msgr2=0x7f033c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.180+0000 7f03413ae700 1 --2- 172.21.15.7:0/1938116734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.180+0000 7f03413ae700 1 -- 172.21.15.7:0/1938116734 >> v1:172.21.15.113:6789/0 conn(0x7f033c158280 legacy=0x7f033c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.180+0000 7f03413ae700 1 -- 172.21.15.7:0/1938116734 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c0cf820 con 0x7f033c0cafb0 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.180+0000 7f03413ae700 1 --2- 172.21.15.7:0/1938116734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0ce8f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f03380099b0 tx=0x7f0338009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3b57c8db13fe3c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.180+0000 7f032ffff700 1 -- 172.21.15.7:0/1938116734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033800b860 con 0x7f033c0cafb0 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.180+0000 7f032ffff700 1 -- 172.21.15.7:0/1938116734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033800b9c0 con 0x7f033c0cafb0 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.180+0000 7f032ffff700 1 -- 172.21.15.7:0/1938116734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033800bd60 con 0x7f033c0cafb0 2024-01-11T07:11:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.181+0000 7f0343612700 1 -- 172.21.15.7:0/1938116734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 msgr2=0x7f033c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.181+0000 7f0343612700 1 --2- 172.21.15.7:0/1938116734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0ce8f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f03380099b0 tx=0x7f0338009250 comp rx=0 tx=0).stop 2024-01-11T07:11:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.181+0000 7f0343612700 1 -- 172.21.15.7:0/1938116734 shutdown_connections 2024-01-11T07:11:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.181+0000 7f0343612700 1 --2- 172.21.15.7:0/1938116734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.181+0000 7f0343612700 1 --2- 172.21.15.7:0/1938116734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0ce8f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.181+0000 7f0343612700 1 -- 172.21.15.7:0/1938116734 >> 172.21.15.7:0/1938116734 conn(0x7f033c071100 msgr2=0x7f033c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.181+0000 7f0343612700 1 -- 172.21.15.7:0/1938116734 shutdown_connections 2024-01-11T07:11:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.181+0000 7f0343612700 1 -- 172.21.15.7:0/1938116734 wait complete. 2024-01-11T07:11:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.182+0000 7f0343612700 1 Processor -- start 2024-01-11T07:11:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.182+0000 7f0343612700 1 -- start start 2024-01-11T07:11:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0343612700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0343612700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0343612700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033c158280 0x7f033c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0343612700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0aed00 con 0x7f033c0cafb0 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0343612700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0aefe0 con 0x7f033c158280 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0343612700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0af2c0 con 0x7f033c0b0b40 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0340bad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0340bad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59818/0 (socket says 172.21.15.7:59818) 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0340bad700 1 -- 172.21.15.7:0/2102041921 learned_addr learned my addr 172.21.15.7:0/2102041921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f03413ae700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0341baf700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033c158280 0x7f033c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0340bad700 1 -- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 msgr2=0x7f033c153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0340bad700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0340bad700 1 -- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033c158280 msgr2=0x7f033c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0340bad700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033c158280 0x7f033c0b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f0340bad700 1 -- 172.21.15.7:0/2102041921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380094f0 con 0x7f033c0cafb0 2024-01-11T07:11:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.183+0000 7f03413ae700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.184+0000 7f0340bad700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0b22d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f033000c590 tx=0x7f0330007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.184+0000 7f032d7fa700 1 -- 172.21.15.7:0/2102041921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330011590 con 0x7f033c0cafb0 2024-01-11T07:11:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.184+0000 7f032d7fa700 1 -- 172.21.15.7:0/2102041921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0330011710 con 0x7f033c0cafb0 2024-01-11T07:11:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.184+0000 7f032d7fa700 1 -- 172.21.15.7:0/2102041921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330011ad0 con 0x7f033c0cafb0 2024-01-11T07:11:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.184+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c0af5a0 con 0x7f033c0cafb0 2024-01-11T07:11:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.184+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c0afb60 con 0x7f033c0cafb0 2024-01-11T07:11:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.185+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c04c3c0 con 0x7f033c0cafb0 2024-01-11T07:11:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.185+0000 7f032d7fa700 1 -- 172.21.15.7:0/2102041921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f033001a360 con 0x7f033c0cafb0 2024-01-11T07:11:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.186+0000 7f032d7fa700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0320066f30 0x7f03200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.186+0000 7f032d7fa700 1 -- 172.21.15.7:0/2102041921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f03300831f0 con 0x7f033c0cafb0 2024-01-11T07:11:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.186+0000 7f03413ae700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0320066f30 0x7f03200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.187+0000 7f03413ae700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0320066f30 0x7f03200693e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f03380099b0 tx=0x7f0338017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:50.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.190+0000 7f032d7fa700 1 -- 172.21.15.7:0/2102041921 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0330050760 con 0x7f033c0cafb0 2024-01-11T07:11:50.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.476+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f033c0c7c10 con 0x7f033c0cafb0 2024-01-11T07:11:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.645+0000 7f032d7fa700 1 -- 172.21.15.7:0/2102041921 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v245) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0330053d80 con 0x7f033c0cafb0 2024-01-11T07:11:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0320066f30 msgr2=0x7f03200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0320066f30 0x7f03200693e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f03380099b0 tx=0x7f0338017000 comp rx=0 tx=0).stop 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 msgr2=0x7f033c0b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0b22d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f033000c590 tx=0x7f0330007f50 comp rx=0 tx=0).stop 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 shutdown_connections 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0320066f30 0x7f03200693e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f033c158280 0x7f033c0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f033c0b0b40 0x7f033c153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 --2- 172.21.15.7:0/2102041921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f033c0cafb0 0x7f033c0b22d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 >> 172.21.15.7:0/2102041921 conn(0x7f033c071100 msgr2=0x7f033c0cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 shutdown_connections 2024-01-11T07:11:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.648+0000 7f0343612700 1 -- 172.21.15.7:0/2102041921 wait complete. 2024-01-11T07:11:50.663 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-01-11T07:11:50.663 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2024-01-11T07:11:50.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-01-11T07:11:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.800+0000 7f6284261700 1 Processor -- start 2024-01-11T07:11:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.800+0000 7f6284261700 1 -- start start 2024-01-11T07:11:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.801+0000 7f6284261700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0c1dc0 0x7f627c06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.801+0000 7f6284261700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f627c0cad50 0x7f627c067610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.801+0000 7f6284261700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c06f640 con 0x7f627c06f1d0 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.801+0000 7f6284261700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c15e850 con 0x7f627c0cad50 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.801+0000 7f6284261700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c161eb0 con 0x7f627c0c1dc0 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.801+0000 7f62827fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f627c0cad50 0x7f627c067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f62827fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f627c0cad50 0x7f627c067610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59826/0 (socket says 172.21.15.7:59826) 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f62827fe700 1 -- 172.21.15.7:0/4273937764 learned_addr learned my addr 172.21.15.7:0/4273937764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f6281ffd700 1 --2- 172.21.15.7:0/4273937764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0c1dc0 0x7f627c06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f62827fe700 1 -- 172.21.15.7:0/4273937764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0c1dc0 msgr2=0x7f627c06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f62827fe700 1 --2- 172.21.15.7:0/4273937764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0c1dc0 0x7f627c06ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f62827fe700 1 -- 172.21.15.7:0/4273937764 >> v1:172.21.15.113:6789/0 conn(0x7f627c06f1d0 legacy=0x7f627c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f62827fe700 1 -- 172.21.15.7:0/4273937764 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c162a30 con 0x7f627c0cad50 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f62827fe700 1 --2- 172.21.15.7:0/4273937764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c0cad50 0x7f627c067610 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6274009f20 tx=0x7f6274009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2dea68e23ce25b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f6280ffb700 1 -- 172.21.15.7:0/4273937764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627400e070 con 0x7f627c0cad50 2024-01-11T07:11:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f6280ffb700 1 -- 172.21.15.7:0/4273937764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627400b9d0 con 0x7f627c0cad50 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.802+0000 7f6280ffb700 1 -- 172.21.15.7:0/4273937764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62740092e0 con 0x7f627c0cad50 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.803+0000 7f6284261700 1 -- 172.21.15.7:0/4273937764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c0cad50 msgr2=0x7f627c067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.803+0000 7f6284261700 1 --2- 172.21.15.7:0/4273937764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c0cad50 0x7f627c067610 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6274009f20 tx=0x7f6274009840 comp rx=0 tx=0).stop 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.803+0000 7f6284261700 1 -- 172.21.15.7:0/4273937764 shutdown_connections 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.803+0000 7f6284261700 1 --2- 172.21.15.7:0/4273937764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c0cad50 0x7f627c067610 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.803+0000 7f6284261700 1 --2- 172.21.15.7:0/4273937764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0c1dc0 0x7f627c06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.803+0000 7f6284261700 1 -- 172.21.15.7:0/4273937764 >> 172.21.15.7:0/4273937764 conn(0x7f627c0ce9f0 msgr2=0x7f627c0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.803+0000 7f6284261700 1 -- 172.21.15.7:0/4273937764 shutdown_connections 2024-01-11T07:11:50.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.803+0000 7f6284261700 1 -- 172.21.15.7:0/4273937764 wait complete. 2024-01-11T07:11:50.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.804+0000 7f6284261700 1 Processor -- start 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.804+0000 7f6284261700 1 -- start start 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6284261700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c06f1d0 0x7f627c15e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6284261700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c1dc0 0x7f627c1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6284261700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0cad50 0x7f627c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6284261700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1615d0 con 0x7f627c06f1d0 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6284261700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c16ea80 con 0x7f627c0c1dc0 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6284261700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c0c14f0 con 0x7f627c0cad50 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6281ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c06f1d0 0x7f627c15e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6281ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c06f1d0 0x7f627c15e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59830/0 (socket says 172.21.15.7:59830) 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6281ffd700 1 -- 172.21.15.7:0/2022766221 learned_addr learned my addr 172.21.15.7:0/2022766221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:50.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f62827fe700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0cad50 0x7f627c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f62817fc700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c1dc0 0x7f627c1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6281ffd700 1 -- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0cad50 msgr2=0x7f627c0c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6281ffd700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0cad50 0x7f627c0c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6281ffd700 1 -- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c1dc0 msgr2=0x7f627c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6281ffd700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c1dc0 0x7f627c1608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.805+0000 7f6281ffd700 1 -- 172.21.15.7:0/2022766221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009ae0 con 0x7f627c06f1d0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.806+0000 7f6281ffd700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c06f1d0 0x7f627c15e120 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f627800e930 tx=0x7f62780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.806+0000 7f6271ffb700 1 -- 172.21.15.7:0/2022766221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278013680 con 0x7f627c06f1d0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.806+0000 7f6271ffb700 1 -- 172.21.15.7:0/2022766221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62780137e0 con 0x7f627c06f1d0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.806+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c16f2a0 con 0x7f627c06f1d0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.806+0000 7f6271ffb700 1 -- 172.21.15.7:0/2022766221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278013b80 con 0x7f627c06f1d0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.806+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c16ece0 con 0x7f627c06f1d0 2024-01-11T07:11:50.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.807+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c16f580 con 0x7f627c06f1d0 2024-01-11T07:11:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.808+0000 7f6271ffb700 1 -- 172.21.15.7:0/2022766221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f62780123d0 con 0x7f627c06f1d0 2024-01-11T07:11:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.808+0000 7f6271ffb700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6260066f30 0x7f62600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:50.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.808+0000 7f6271ffb700 1 -- 172.21.15.7:0/2022766221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f62780842b0 con 0x7f627c06f1d0 2024-01-11T07:11:50.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.808+0000 7f62817fc700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6260066f30 0x7f62600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:50.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.809+0000 7f62817fc700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6260066f30 0x7f62600693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f627c0c8930 tx=0x7f626c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:50.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:50.812+0000 7f6271ffb700 1 -- 172.21.15.7:0/2022766221 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6278051820 con 0x7f627c06f1d0 2024-01-11T07:11:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.100+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f627c04c3c0 con 0x7f627c06f1d0 2024-01-11T07:11:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.647+0000 7f6271ffb700 1 -- 172.21.15.7:0/2022766221 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v246) v1 ==== 140+0+0 (secure 0 0 0) 0x7f6278054e40 con 0x7f627c06f1d0 2024-01-11T07:11:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6260066f30 msgr2=0x7f62600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6260066f30 0x7f62600693e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f627c0c8930 tx=0x7f626c007ec0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c06f1d0 msgr2=0x7f627c15e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c06f1d0 0x7f627c15e120 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f627800e930 tx=0x7f62780091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 shutdown_connections 2024-01-11T07:11:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6260066f30 0x7f62600693e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f627c0cad50 0x7f627c0c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f627c06f1d0 0x7f627c15e120 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 --2- 172.21.15.7:0/2022766221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c1dc0 0x7f627c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 >> 172.21.15.7:0/2022766221 conn(0x7f627c0ce9f0 msgr2=0x7f627c0cd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 shutdown_connections 2024-01-11T07:11:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.650+0000 7f6284261700 1 -- 172.21.15.7:0/2022766221 wait complete. 2024-01-11T07:11:51.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-01-11T07:11:51.817 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_objectstore":"memstore"} 2024-01-11T07:11:51.818 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-01-11T07:11:51.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:11:51.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.954+0000 7ff554770700 1 Processor -- start 2024-01-11T07:11:51.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.954+0000 7ff554770700 1 -- start start 2024-01-11T07:11:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.955+0000 7ff554770700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff54c0ccc00 0x7ff54c068570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.955+0000 7ff554770700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c0680e0 con 0x7ff54c15d4a0 2024-01-11T07:11:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.955+0000 7ff554770700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c0b1ea0 con 0x7ff54c15d870 2024-01-11T07:11:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.955+0000 7ff554770700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c0677b0 con 0x7ff54c0ccc00 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff55250c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff54c15d4a0 0x7ff54c0b1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45276/0 (socket says 172.21.15.7:45276) 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff55250c700 1 -- 172.21.15.7:0/1135728880 learned_addr learned my addr 172.21.15.7:0/1135728880 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff552d0d700 1 --2- 172.21.15.7:0/1135728880 >> v2:172.21.15.113:3300/0 conn(0x7ff54c0ccc00 0x7ff54c068570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff55150a700 1 -- 172.21.15.7:0/1135728880 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266445162 0 0) 0x7ff54c0680e0 con 0x7ff54c15d4a0 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff55150a700 1 -- 172.21.15.7:0/1135728880 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff538003570 con 0x7ff54c15d4a0 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff55150a700 1 -- 172.21.15.7:0/1135728880 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131938452 0 0) 0x7ff54c0b1ea0 con 0x7ff54c15d870 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff55150a700 1 -- 172.21.15.7:0/1135728880 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff54c0680e0 con 0x7ff54c15d870 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff55150a700 1 -- 172.21.15.7:0/1135728880 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445990637 0 0) 0x7ff538003570 con 0x7ff54c15d4a0 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff552d0d700 1 -- 172.21.15.7:0/1135728880 >> v1:172.21.15.113:6789/0 conn(0x7ff54c15d870 legacy=0x7ff54c0cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff552d0d700 1 -- 172.21.15.7:0/1135728880 >> v1:172.21.15.7:6789/0 conn(0x7ff54c15d4a0 legacy=0x7ff54c0b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff552d0d700 1 -- 172.21.15.7:0/1135728880 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c067a90 con 0x7ff54c0ccc00 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.956+0000 7ff552d0d700 1 --2- 172.21.15.7:0/1135728880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c0ccc00 0x7ff54c068570 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff544009e90 tx=0x7ff544009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a53e6b10b4fac02 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff55150a700 1 -- 172.21.15.7:0/1135728880 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400e070 con 0x7ff54c0ccc00 2024-01-11T07:11:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff55150a700 1 -- 172.21.15.7:0/1135728880 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54400b870 con 0x7ff54c0ccc00 2024-01-11T07:11:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff55150a700 1 -- 172.21.15.7:0/1135728880 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480015f0 con 0x7ff54c0ccc00 2024-01-11T07:11:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff554770700 1 -- 172.21.15.7:0/1135728880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c0ccc00 msgr2=0x7ff54c068570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff554770700 1 --2- 172.21.15.7:0/1135728880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c0ccc00 0x7ff54c068570 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff544009e90 tx=0x7ff544009780 comp rx=0 tx=0).stop 2024-01-11T07:11:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff554770700 1 -- 172.21.15.7:0/1135728880 shutdown_connections 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff554770700 1 --2- 172.21.15.7:0/1135728880 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c0ccc00 0x7ff54c068570 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff554770700 1 -- 172.21.15.7:0/1135728880 >> 172.21.15.7:0/1135728880 conn(0x7ff54c097a50 msgr2=0x7ff54c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff554770700 1 -- 172.21.15.7:0/1135728880 shutdown_connections 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.957+0000 7ff554770700 1 -- 172.21.15.7:0/1135728880 wait complete. 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff554770700 1 Processor -- start 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff554770700 1 -- start start 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff554770700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff54c0ccc00 0x7ff54c0c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff554770700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c15d4a0 0x7ff54c0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff554770700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff54c155e60 0x7ff54c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff554770700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c156ad0 con 0x7ff54c0ccc00 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff554770700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c157370 con 0x7ff54c15d4a0 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff554770700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0c6aa0 con 0x7ff54c155e60 2024-01-11T07:11:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.959+0000 7ff55250c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff54c0ccc00 0x7ff54c0c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff55250c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff54c0ccc00 0x7ff54c0c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59832/0 (socket says 172.21.15.7:59832) 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff55250c700 1 -- 172.21.15.7:0/111479981 learned_addr learned my addr 172.21.15.7:0/111479981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff552d0d700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff54c155e60 0x7ff54c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff551d0b700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c15d4a0 0x7ff54c0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff55250c700 1 -- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff54c155e60 msgr2=0x7ff54c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff55250c700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff54c155e60 0x7ff54c157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff55250c700 1 -- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c15d4a0 msgr2=0x7ff54c0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff55250c700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c15d4a0 0x7ff54c0c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff55250c700 1 -- 172.21.15.7:0/111479981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544009a20 con 0x7ff54c0ccc00 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff55250c700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff54c0ccc00 0x7ff54c0c63e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff5480010e0 tx=0x7ff548002b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff5427fc700 1 -- 172.21.15.7:0/111479981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548007e40 con 0x7ff54c0ccc00 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff5427fc700 1 -- 172.21.15.7:0/111479981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff548007300 con 0x7ff54c0ccc00 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff5427fc700 1 -- 172.21.15.7:0/111479981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480057f0 con 0x7ff54c0ccc00 2024-01-11T07:11:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c156d20 con 0x7ff54c0ccc00 2024-01-11T07:11:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.960+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c0c7ba0 con 0x7ff54c0ccc00 2024-01-11T07:11:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.961+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c04c3c0 con 0x7ff54c0ccc00 2024-01-11T07:11:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.962+0000 7ff5427fc700 1 -- 172.21.15.7:0/111479981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff548005ba0 con 0x7ff54c0ccc00 2024-01-11T07:11:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.962+0000 7ff5427fc700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff530066f30 0x7ff5300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.962+0000 7ff5427fc700 1 -- 172.21.15.7:0/111479981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff548083fe0 con 0x7ff54c0ccc00 2024-01-11T07:11:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.963+0000 7ff551d0b700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff530066f30 0x7ff5300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.963+0000 7ff551d0b700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff530066f30 0x7ff5300693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff54c1586c0 tx=0x7ff53c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:51.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:51.967+0000 7ff5427fc700 1 -- 172.21.15.7:0/111479981 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff548051550 con 0x7ff54c0ccc00 2024-01-11T07:11:52.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.173+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff54c067d70 con 0x7ff54c0ccc00 2024-01-11T07:11:52.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.177+0000 7ff5427fc700 1 -- 172.21.15.7:0/111479981 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff548054b70 con 0x7ff54c0ccc00 2024-01-11T07:11:52.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff530066f30 msgr2=0x7ff5300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff530066f30 0x7ff5300693e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff54c1586c0 tx=0x7ff53c009040 comp rx=0 tx=0).stop 2024-01-11T07:11:52.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff54c0ccc00 msgr2=0x7ff54c0c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff54c0ccc00 0x7ff54c0c63e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff5480010e0 tx=0x7ff548002b10 comp rx=0 tx=0).stop 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 shutdown_connections 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff530066f30 0x7ff5300693e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff54c155e60 0x7ff54c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff54c0ccc00 0x7ff54c0c63e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 --2- 172.21.15.7:0/111479981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff54c15d4a0 0x7ff54c0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 >> 172.21.15.7:0/111479981 conn(0x7ff54c097a50 msgr2=0x7ff54c159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 shutdown_connections 2024-01-11T07:11:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.179+0000 7ff554770700 1 -- 172.21.15.7:0/111479981 wait complete. 2024-01-11T07:11:52.193 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:11:52.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:11:52.234 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:11:52.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:11:52.294 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:11:52.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:11:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.432+0000 7f7cf50c1700 1 Processor -- start 2024-01-11T07:11:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.432+0000 7f7cf50c1700 1 -- start start 2024-01-11T07:11:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cf50c1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cf00cdaa0 0x7f7cf015a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cf50c1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf015e6c0 con 0x7f7cf00d0170 2024-01-11T07:11:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cf50c1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf015acf0 con 0x7f7cf00cd6d0 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cf50c1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf00cdf00 con 0x7f7cf00cdaa0 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cef59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cf00cdaa0 0x7f7cf015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cee59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7cf00d0170 0x7f7cf015dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45280/0 (socket says 172.21.15.7:45280) 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cee59c700 1 -- 172.21.15.7:0/302296618 learned_addr learned my addr 172.21.15.7:0/302296618 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cef59e700 1 -- 172.21.15.7:0/302296618 >> v1:172.21.15.113:6789/0 conn(0x7f7cf00cd6d0 legacy=0x7f7cf015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cef59e700 1 -- 172.21.15.7:0/302296618 >> v1:172.21.15.7:6789/0 conn(0x7f7cf00d0170 legacy=0x7f7cf015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cedd9b700 1 -- 172.21.15.7:0/302296618 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781550415 0 0) 0x7f7cf015e6c0 con 0x7f7cf00d0170 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.433+0000 7f7cef59e700 1 -- 172.21.15.7:0/302296618 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf015afd0 con 0x7f7cf00cdaa0 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cef59e700 1 --2- 172.21.15.7:0/302296618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00cdaa0 0x7f7cf015a610 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009e90 tx=0x7f7ce8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8de3a450810e15a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cedd9b700 1 -- 172.21.15.7:0/302296618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800e070 con 0x7f7cf00cdaa0 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cedd9b700 1 -- 172.21.15.7:0/302296618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce800bbf0 con 0x7f7cf00cdaa0 2024-01-11T07:11:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cedd9b700 1 -- 172.21.15.7:0/302296618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8012430 con 0x7f7cf00cdaa0 2024-01-11T07:11:52.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cf50c1700 1 -- 172.21.15.7:0/302296618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00cdaa0 msgr2=0x7f7cf015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cf50c1700 1 --2- 172.21.15.7:0/302296618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00cdaa0 0x7f7cf015a610 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009e90 tx=0x7f7ce8009780 comp rx=0 tx=0).stop 2024-01-11T07:11:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cf50c1700 1 -- 172.21.15.7:0/302296618 shutdown_connections 2024-01-11T07:11:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cf50c1700 1 --2- 172.21.15.7:0/302296618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00cdaa0 0x7f7cf015a610 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cf50c1700 1 -- 172.21.15.7:0/302296618 >> 172.21.15.7:0/302296618 conn(0x7f7cf0071120 msgr2=0x7f7cf006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.434+0000 7f7cf50c1700 1 -- 172.21.15.7:0/302296618 shutdown_connections 2024-01-11T07:11:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.435+0000 7f7cf50c1700 1 -- 172.21.15.7:0/302296618 wait complete. 2024-01-11T07:11:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.436+0000 7f7cf50c1700 1 Processor -- start 2024-01-11T07:11:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.436+0000 7f7cf50c1700 1 -- start start 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cf50c1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cf00cd6d0 0x7f7cf015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cf50c1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf00cdaa0 0x7f7cf0151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cf50c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00d0170 0x7f7cf0154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cf50c1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf015bad0 con 0x7f7cf00d0170 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cf50c1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf016e7f0 con 0x7f7cf00cdaa0 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cf50c1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf016e360 con 0x7f7cf00cd6d0 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00d0170 0x7f7cf0154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00d0170 0x7f7cf0154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59854/0 (socket says 172.21.15.7:59854) 2024-01-11T07:11:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 -- 172.21.15.7:0/376975539 learned_addr learned my addr 172.21.15.7:0/376975539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7ceed9d700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cf00cd6d0 0x7f7cf015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cee59c700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf00cdaa0 0x7f7cf0151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 -- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cf00cd6d0 msgr2=0x7f7cf015b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cf00cd6d0 0x7f7cf015b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 -- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf00cdaa0 msgr2=0x7f7cf0151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf00cdaa0 0x7f7cf0151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 -- 172.21.15.7:0/376975539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8009a20 con 0x7f7cf00d0170 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cee59c700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf00cdaa0 0x7f7cf0151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7cef59e700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00d0170 0x7f7cf0154800 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0063db0 tx=0x7f7ce8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.437+0000 7f7ce6ffd700 1 -- 172.21.15.7:0/376975539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8012e50 con 0x7f7cf00d0170 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.438+0000 7f7ce6ffd700 1 -- 172.21.15.7:0/376975539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce801b630 con 0x7f7cf00d0170 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.438+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf0157810 con 0x7f7cf00d0170 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.438+0000 7f7ce6ffd700 1 -- 172.21.15.7:0/376975539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce801b9f0 con 0x7f7cf00d0170 2024-01-11T07:11:52.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.438+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf016dd10 con 0x7f7cf00d0170 2024-01-11T07:11:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.439+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf016e4d0 con 0x7f7cf00d0170 2024-01-11T07:11:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.439+0000 7f7ce6ffd700 1 -- 172.21.15.7:0/376975539 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7ce801c400 con 0x7f7cf00d0170 2024-01-11T07:11:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.439+0000 7f7ce6ffd700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7cd0066f30 0x7f7cd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.439+0000 7f7ce6ffd700 1 -- 172.21.15.7:0/376975539 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7ce80860e0 con 0x7f7cf00d0170 2024-01-11T07:11:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.440+0000 7f7ceed9d700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7cd0066f30 0x7f7cd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.440+0000 7f7ceed9d700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7cd0066f30 0x7f7cd00693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00039e0 tx=0x7f7ce0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:52.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.444+0000 7f7ce6ffd700 1 -- 172.21.15.7:0/376975539 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ce8053650 con 0x7f7cf00d0170 2024-01-11T07:11:52.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.697+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cf004c3c0 con 0x7f7cf00d0170 2024-01-11T07:11:52.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.697+0000 7f7ce6ffd700 1 -- 172.21.15.7:0/376975539 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f7ce802ade0 con 0x7f7cf00d0170 2024-01-11T07:11:52.697 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:52.697 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/459576658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":459576658},{"type":"v1","addr":"172.21.15.113:6837","nonce":459576658}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":5905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2882608277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2882608277},{"type":"v1","addr":"172.21.15.7:6833","nonce":2882608277}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3510440823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3510440823},{"type":"v1","addr":"172.21.15.7:6835","nonce":3510440823}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3108557963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3108557963},{"type":"v1","addr":"172.21.15.113:6836","nonce":3108557963}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[]} 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7cd0066f30 msgr2=0x7f7cd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7cd0066f30 0x7f7cd00693e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00039e0 tx=0x7f7ce0007ec0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00d0170 msgr2=0x7f7cf0154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00d0170 0x7f7cf0154800 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0063db0 tx=0x7f7ce8019000 comp rx=0 tx=0).stop 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 shutdown_connections 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7cd0066f30 0x7f7cd00693e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cf00d0170 0x7f7cf0154800 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cf00cd6d0 0x7f7cf015b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 --2- 172.21.15.7:0/376975539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cf00cdaa0 0x7f7cf0151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.699+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 >> 172.21.15.7:0/376975539 conn(0x7f7cf0071120 msgr2=0x7f7cf00c62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.700+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 shutdown_connections 2024-01-11T07:11:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.700+0000 7f7cf50c1700 1 -- 172.21.15.7:0/376975539 wait complete. 2024-01-11T07:11:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-01-11T07:11:52.713 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:11:52.713 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:11:52.713 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:11:52.713 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:11:52.714 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:11:52.714 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:11:52.714 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:11:52.714 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:11:52.714 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:11:52.714 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:11:52.714 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:11:52.714 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:11:52.715 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:11:52.715 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:11:52.715 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:11:52.715 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:11:52.723 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:11:52.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:11:52.724 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:11:52.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:11:52.725 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:11:52.725 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:11:52.726 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:11:52.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:11:52.766 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:11:52.766 DEBUG:teuthology.parallel:result is None 2024-01-11T07:11:52.767 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:11:52.768 DEBUG:teuthology.parallel:result is None 2024-01-11T07:11:52.768 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:11:52.769 DEBUG:teuthology.parallel:result is None 2024-01-11T07:11:52.769 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:11:52.769 DEBUG:teuthology.parallel:result is None 2024-01-11T07:11:52.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:11:52.851 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:11:52.851 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:11:52.853 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:11:52.854 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:11:52.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.910+0000 7f4828649700 1 Processor -- start 2024-01-11T07:11:52.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.910+0000 7f4828649700 1 -- start start 2024-01-11T07:11:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.911+0000 7f4828649700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48200cd6d0 0x7f48200cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.911+0000 7f4828649700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f482015e420 0x7f4820159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.911+0000 7f4828649700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482015acf0 con 0x7f482015d8a0 2024-01-11T07:11:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4828649700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482015afd0 con 0x7f482015e420 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4828649700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200ce980 con 0x7f48200cd6d0 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4825be4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f482015d8a0 0x7f482015dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45298/0 (socket says 172.21.15.7:45298) 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4825be4700 1 -- 172.21.15.7:0/1221566584 learned_addr learned my addr 172.21.15.7:0/1221566584 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4826be6700 1 --2- 172.21.15.7:0/1221566584 >> v2:172.21.15.113:3300/0 conn(0x7f482015e420 0x7f4820159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f48263e5700 1 --2- 172.21.15.7:0/1221566584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48200cd6d0 0x7f48200cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f48253e3700 1 -- 172.21.15.7:0/1221566584 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807227541 0 0) 0x7f482015acf0 con 0x7f482015d8a0 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f48253e3700 1 -- 172.21.15.7:0/1221566584 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f480c003570 con 0x7f482015d8a0 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4826be6700 1 -- 172.21.15.7:0/1221566584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48200cd6d0 msgr2=0x7f48200cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4826be6700 1 --2- 172.21.15.7:0/1221566584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48200cd6d0 0x7f48200cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4826be6700 1 -- 172.21.15.7:0/1221566584 >> v1:172.21.15.7:6789/0 conn(0x7f482015d8a0 legacy=0x7f482015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4826be6700 1 -- 172.21.15.7:0/1221566584 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200cec60 con 0x7f482015e420 2024-01-11T07:11:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.912+0000 7f4826be6700 1 --2- 172.21.15.7:0/1221566584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f482015e420 0x7f4820159600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4818009e60 tx=0x7f4818009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af639b76ab5b54d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f48253e3700 1 -- 172.21.15.7:0/1221566584 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800e070 con 0x7f482015e420 2024-01-11T07:11:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f48253e3700 1 -- 172.21.15.7:0/1221566584 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481800bab0 con 0x7f482015e420 2024-01-11T07:11:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f48253e3700 1 -- 172.21.15.7:0/1221566584 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800be30 con 0x7f482015e420 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f4828649700 1 -- 172.21.15.7:0/1221566584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f482015e420 msgr2=0x7f4820159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f4828649700 1 --2- 172.21.15.7:0/1221566584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f482015e420 0x7f4820159600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4818009e60 tx=0x7f4818009780 comp rx=0 tx=0).stop 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f4828649700 1 -- 172.21.15.7:0/1221566584 shutdown_connections 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f4828649700 1 --2- 172.21.15.7:0/1221566584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f482015e420 0x7f4820159600 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f4828649700 1 --2- 172.21.15.7:0/1221566584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48200cd6d0 0x7f48200cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f4828649700 1 -- 172.21.15.7:0/1221566584 >> 172.21.15.7:0/1221566584 conn(0x7f4820071120 msgr2=0x7f482006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f4828649700 1 -- 172.21.15.7:0/1221566584 shutdown_connections 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.913+0000 7f4828649700 1 -- 172.21.15.7:0/1221566584 wait complete. 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.915+0000 7f4828649700 1 Processor -- start 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.915+0000 7f4828649700 1 -- start start 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.915+0000 7f4828649700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200cd6d0 0x7f4820143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.915+0000 7f4828649700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482015d8a0 0x7f48200c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.915+0000 7f4828649700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015e420 0x7f48200c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.915+0000 7f4828649700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200ca3a0 con 0x7f482015d8a0 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.915+0000 7f4828649700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200c9dc0 con 0x7f48200cd6d0 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.915+0000 7f4828649700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48201601f0 con 0x7f482015e420 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482015d8a0 0x7f48200c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482015d8a0 0x7f48200c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59886/0 (socket says 172.21.15.7:59886) 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 -- 172.21.15.7:0/1166792063 learned_addr learned my addr 172.21.15.7:0/1166792063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4826be6700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015e420 0x7f48200c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f48263e5700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200cd6d0 0x7f4820143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 -- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015e420 msgr2=0x7f48200c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015e420 0x7f48200c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 -- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200cd6d0 msgr2=0x7f4820143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200cd6d0 0x7f4820143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 -- 172.21.15.7:0/1166792063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818009a20 con 0x7f482015d8a0 2024-01-11T07:11:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4825be4700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482015d8a0 0x7f48200c7cc0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f481000c790 tx=0x7f4810007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f48167fc700 1 -- 172.21.15.7:0/1166792063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810005b30 con 0x7f482015d8a0 2024-01-11T07:11:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f48167fc700 1 -- 172.21.15.7:0/1166792063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4810005c90 con 0x7f482015d8a0 2024-01-11T07:11:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48200ca030 con 0x7f482015d8a0 2024-01-11T07:11:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f48167fc700 1 -- 172.21.15.7:0/1166792063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810007320 con 0x7f482015d8a0 2024-01-11T07:11:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.916+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4820143a70 con 0x7f482015d8a0 2024-01-11T07:11:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.917+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482004c3c0 con 0x7f482015d8a0 2024-01-11T07:11:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.918+0000 7f48167fc700 1 -- 172.21.15.7:0/1166792063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4810007690 con 0x7f482015d8a0 2024-01-11T07:11:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.918+0000 7f48167fc700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.918+0000 7f48167fc700 1 -- 172.21.15.7:0/1166792063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f48100836f0 con 0x7f482015d8a0 2024-01-11T07:11:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.918+0000 7f48263e5700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.919+0000 7f48263e5700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4804066f30 0x7f48040693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f481c009db0 tx=0x7f481c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:52.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:52.923+0000 7f48167fc700 1 -- 172.21.15.7:0/1166792063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4810050bc0 con 0x7f482015d8a0 2024-01-11T07:11:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.142+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4820158cc0 con 0x7f482015d8a0 2024-01-11T07:11:53.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.663+0000 7f48167fc700 1 -- 172.21.15.7:0/1166792063 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4810017080 con 0x7f482015d8a0 2024-01-11T07:11:53.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.666+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4804066f30 msgr2=0x7f48040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:53.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.666+0000 7f4828649700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4804066f30 0x7f48040693e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f481c009db0 tx=0x7f481c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.666+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482015d8a0 msgr2=0x7f48200c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:53.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.666+0000 7f4828649700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482015d8a0 0x7f48200c7cc0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f481000c790 tx=0x7f4810007df0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.667+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 shutdown_connections 2024-01-11T07:11:53.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.667+0000 7f4828649700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4804066f30 0x7f48040693e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.667+0000 7f4828649700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f482015e420 0x7f48200c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.667+0000 7f4828649700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48200cd6d0 0x7f4820143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.667+0000 7f4828649700 1 --2- 172.21.15.7:0/1166792063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f482015d8a0 0x7f48200c7cc0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.667+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 >> 172.21.15.7:0/1166792063 conn(0x7f4820071120 msgr2=0x7f4820159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:53.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.667+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 shutdown_connections 2024-01-11T07:11:53.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.667+0000 7f4828649700 1 -- 172.21.15.7:0/1166792063 wait complete. 2024-01-11T07:11:53.667 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:11:53.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:11:53.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.819+0000 7f3ca800b700 1 Processor -- start 2024-01-11T07:11:53.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.819+0000 7f3ca800b700 1 -- start start 2024-01-11T07:11:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca800b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ca015a060 0x7f3ca00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca800b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ca0069320 0x7f3ca00b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca800b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca00ce9b0 con 0x7f3ca0159c90 2024-01-11T07:11:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca800b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca00b1f30 con 0x7f3ca0069320 2024-01-11T07:11:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca800b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca00cf530 con 0x7f3ca015a060 2024-01-11T07:11:53.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca65a8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ca0069320 0x7f3ca00b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca5da7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3ca0159c90 0x7f3ca00b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45308/0 (socket says 172.21.15.7:45308) 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca5da7700 1 -- 172.21.15.7:0/3523619814 learned_addr learned my addr 172.21.15.7:0/3523619814 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca55a6700 1 --2- 172.21.15.7:0/3523619814 >> v2:172.21.15.113:3300/0 conn(0x7f3ca015a060 0x7f3ca00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca65a8700 1 -- 172.21.15.7:0/3523619814 >> v2:172.21.15.113:3300/0 conn(0x7f3ca015a060 msgr2=0x7f3ca00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca65a8700 1 --2- 172.21.15.7:0/3523619814 >> v2:172.21.15.113:3300/0 conn(0x7f3ca015a060 0x7f3ca00b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca65a8700 1 -- 172.21.15.7:0/3523619814 >> v1:172.21.15.7:6789/0 conn(0x7f3ca0159c90 legacy=0x7f3ca00b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca65a8700 1 -- 172.21.15.7:0/3523619814 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00cf9c0 con 0x7f3ca0069320 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca4da5700 1 -- 172.21.15.7:0/3523619814 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338699767 0 0) 0x7f3ca00ce9b0 con 0x7f3ca0159c90 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.820+0000 7f3ca55a6700 1 --2- 172.21.15.7:0/3523619814 >> v2:172.21.15.113:3300/0 conn(0x7f3ca015a060 0x7f3ca00b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca65a8700 1 --2- 172.21.15.7:0/3523619814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00b0970 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00699e0 tx=0x7f3c98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8609ad75b291997f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca4da5700 1 -- 172.21.15.7:0/3523619814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800e070 con 0x7f3ca0069320 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca4da5700 1 -- 172.21.15.7:0/3523619814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9800bc30 con 0x7f3ca0069320 2024-01-11T07:11:53.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca4da5700 1 -- 172.21.15.7:0/3523619814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98012430 con 0x7f3ca0069320 2024-01-11T07:11:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca800b700 1 -- 172.21.15.7:0/3523619814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 msgr2=0x7f3ca00b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca800b700 1 --2- 172.21.15.7:0/3523619814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00b0970 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00699e0 tx=0x7f3c98009780 comp rx=0 tx=0).stop 2024-01-11T07:11:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca800b700 1 -- 172.21.15.7:0/3523619814 shutdown_connections 2024-01-11T07:11:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca800b700 1 --2- 172.21.15.7:0/3523619814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00b0970 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca800b700 1 --2- 172.21.15.7:0/3523619814 >> v2:172.21.15.113:3300/0 conn(0x7f3ca015a060 0x7f3ca00b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca800b700 1 -- 172.21.15.7:0/3523619814 >> 172.21.15.7:0/3523619814 conn(0x7f3ca0071100 msgr2=0x7f3ca006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca800b700 1 -- 172.21.15.7:0/3523619814 shutdown_connections 2024-01-11T07:11:53.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.821+0000 7f3ca800b700 1 -- 172.21.15.7:0/3523619814 wait complete. 2024-01-11T07:11:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.823+0000 7f3ca800b700 1 Processor -- start 2024-01-11T07:11:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.823+0000 7f3ca800b700 1 -- start start 2024-01-11T07:11:53.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.823+0000 7f3ca800b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.823+0000 7f3ca800b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ca0159c90 0x7f3ca0155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.823+0000 7f3ca800b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ca015a060 0x7f3ca00cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.823+0000 7f3ca800b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca01699e0 con 0x7f3ca0069320 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.823+0000 7f3ca800b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0169cc0 con 0x7f3ca015a060 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.823+0000 7f3ca800b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0169fa0 con 0x7f3ca0159c90 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59900/0 (socket says 172.21.15.7:59900) 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 -- 172.21.15.7:0/2875614633 learned_addr learned my addr 172.21.15.7:0/2875614633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca65a8700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ca015a060 0x7f3ca00cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 -- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ca0159c90 msgr2=0x7f3ca0155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ca0159c90 0x7f3ca0155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 -- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ca015a060 msgr2=0x7f3ca00cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:53.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ca015a060 0x7f3ca00cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 -- 172.21.15.7:0/2875614633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c98009a20 con 0x7f3ca0069320 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca65a8700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ca015a060 0x7f3ca00cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca5da7700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00af140 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00c7e0 tx=0x7f3c9c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3c95ffb700 1 -- 172.21.15.7:0/2875614633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c005b30 con 0x7f3ca0069320 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3c95ffb700 1 -- 172.21.15.7:0/2875614633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c005c90 con 0x7f3ca0069320 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca016a280 con 0x7f3ca0069320 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3c95ffb700 1 -- 172.21.15.7:0/2875614633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c007300 con 0x7f3ca0069320 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.824+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca0156490 con 0x7f3ca0069320 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.825+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca00b1400 con 0x7f3ca0069320 2024-01-11T07:11:53.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.826+0000 7f3c95ffb700 1 -- 172.21.15.7:0/2875614633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3c9c01b400 con 0x7f3ca0069320 2024-01-11T07:11:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.826+0000 7f3c95ffb700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3c84066f30 0x7f3c840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.826+0000 7f3c95ffb700 1 -- 172.21.15.7:0/2875614633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3c9c083d60 con 0x7f3ca0069320 2024-01-11T07:11:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.827+0000 7f3ca55a6700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3c84066f30 0x7f3c840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:53.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.827+0000 7f3ca55a6700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3c84066f30 0x7f3c840693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0163bb0 tx=0x7f3c90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:53.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:53.831+0000 7f3c95ffb700 1 -- 172.21.15.7:0/2875614633 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c9c0513a0 con 0x7f3ca0069320 2024-01-11T07:11:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.118+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3ca004c3c0 con 0x7f3ca0069320 2024-01-11T07:11:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.119+0000 7f3c95ffb700 1 -- 172.21.15.7:0/2875614633 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3c9c0549c0 con 0x7f3ca0069320 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:11:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:11:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:11:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:11:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:11:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:11:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:11:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:11:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:11:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:11:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:11:54.132 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3c84066f30 msgr2=0x7f3c840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3c84066f30 0x7f3c840693e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0163bb0 tx=0x7f3c90008040 comp rx=0 tx=0).stop 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 msgr2=0x7f3ca00af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00af140 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00c7e0 tx=0x7f3c9c007df0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 shutdown_connections 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3c84066f30 0x7f3c840693e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ca015a060 0x7f3ca00cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ca0069320 0x7f3ca00af140 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 --2- 172.21.15.7:0/2875614633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ca0159c90 0x7f3ca0155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 >> 172.21.15.7:0/2875614633 conn(0x7f3ca0071100 msgr2=0x7f3ca00b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 shutdown_connections 2024-01-11T07:11:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.121+0000 7f3ca800b700 1 -- 172.21.15.7:0/2875614633 wait complete. 2024-01-11T07:11:54.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:11:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.267+0000 7fcd21e33700 1 Processor -- start 2024-01-11T07:11:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.267+0000 7fcd21e33700 1 -- start start 2024-01-11T07:11:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd21e33700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 0x7fcd1c063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd21e33700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcd1c15a280 0x7fcd1c0aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd21e33700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd1c0ac5b0 0x7fcd1c0aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd21e33700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c15d880 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd21e33700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c06c5b0 con 0x7fcd1c15a280 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd21e33700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c063b40 con 0x7fcd1c063230 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1affd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcd1c15a280 0x7fcd1c0aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd1c0ac5b0 0x7fcd1c0aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1affd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcd1c15a280 0x7fcd1c0aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:38914/0 (socket says 172.21.15.7:38914) 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1affd700 1 -- 172.21.15.7:0/4270183160 learned_addr learned my addr 172.21.15.7:0/4270183160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1b7fe700 1 --2- 172.21.15.7:0/4270183160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 0x7fcd1c063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1bfff700 1 -- 172.21.15.7:0/4270183160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 msgr2=0x7fcd1c063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1bfff700 1 --2- 172.21.15.7:0/4270183160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 0x7fcd1c063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1bfff700 1 -- 172.21.15.7:0/4270183160 >> v2:172.21.15.113:3300/0 conn(0x7fcd1c15a280 msgr2=0x7fcd1c0aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.268+0000 7fcd1bfff700 1 --2- 172.21.15.7:0/4270183160 >> v2:172.21.15.113:3300/0 conn(0x7fcd1c15a280 0x7fcd1c0aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.269+0000 7fcd1bfff700 1 -- 172.21.15.7:0/4270183160 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c15a7f0 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.269+0000 7fcd1affd700 1 --2- 172.21.15.7:0/4270183160 >> v2:172.21.15.113:3300/0 conn(0x7fcd1c15a280 0x7fcd1c0aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:11:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.269+0000 7fcd1bfff700 1 --2- 172.21.15.7:0/4270183160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0aca30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c06d500 tx=0x7fcd14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed08a2696f9eb445 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.269+0000 7fcd1a7fc700 1 -- 172.21.15.7:0/4270183160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400e070 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.269+0000 7fcd1a7fc700 1 -- 172.21.15.7:0/4270183160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1400bab0 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.269+0000 7fcd1a7fc700 1 -- 172.21.15.7:0/4270183160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400be70 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.269+0000 7fcd21e33700 1 -- 172.21.15.7:0/4270183160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 msgr2=0x7fcd1c0aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.269+0000 7fcd21e33700 1 --2- 172.21.15.7:0/4270183160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0aca30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c06d500 tx=0x7fcd14009780 comp rx=0 tx=0).stop 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.270+0000 7fcd21e33700 1 -- 172.21.15.7:0/4270183160 shutdown_connections 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.270+0000 7fcd21e33700 1 --2- 172.21.15.7:0/4270183160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0aca30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.270+0000 7fcd21e33700 1 --2- 172.21.15.7:0/4270183160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 0x7fcd1c063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.270+0000 7fcd21e33700 1 --2- 172.21.15.7:0/4270183160 >> v2:172.21.15.113:3300/0 conn(0x7fcd1c15a280 0x7fcd1c0aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.270+0000 7fcd21e33700 1 -- 172.21.15.7:0/4270183160 >> 172.21.15.7:0/4270183160 conn(0x7fcd1c0680e0 msgr2=0x7fcd1c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.270+0000 7fcd21e33700 1 -- 172.21.15.7:0/4270183160 shutdown_connections 2024-01-11T07:11:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.270+0000 7fcd21e33700 1 -- 172.21.15.7:0/4270183160 wait complete. 2024-01-11T07:11:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.271+0000 7fcd21e33700 1 Processor -- start 2024-01-11T07:11:54.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.271+0000 7fcd21e33700 1 -- start start 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd21e33700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 0x7fcd1c153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd21e33700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd21e33700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1c15a280 0x7fcd1c0cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd21e33700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c16a940 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd21e33700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0c2250 con 0x7fcd1c15a280 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd21e33700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0c2530 con 0x7fcd1c063230 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd1affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd1affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59922/0 (socket says 172.21.15.7:59922) 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd1affd700 1 -- 172.21.15.7:0/3003819698 learned_addr learned my addr 172.21.15.7:0/3003819698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd1bfff700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1c15a280 0x7fcd1c0cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.272+0000 7fcd1b7fe700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 0x7fcd1c153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd1affd700 1 -- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 msgr2=0x7fcd1c153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd1affd700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 0x7fcd1c153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd1affd700 1 -- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1c15a280 msgr2=0x7fcd1c0cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd1affd700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1c15a280 0x7fcd1c0cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd1affd700 1 -- 172.21.15.7:0/3003819698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100094f0 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd1affd700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0cb400 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ec50 tx=0x7fcd10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd037fe700 1 -- 172.21.15.7:0/3003819698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000c920 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd037fe700 1 -- 172.21.15.7:0/3003819698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000ca80 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd037fe700 1 -- 172.21.15.7:0/3003819698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000ce00 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd14009a20 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.273+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c16ab90 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.274+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c04c3c0 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.274+0000 7fcd037fe700 1 -- 172.21.15.7:0/3003819698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcd10013680 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.275+0000 7fcd037fe700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd08066f30 0x7fcd080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.275+0000 7fcd037fe700 1 -- 172.21.15.7:0/3003819698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcd10084530 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.275+0000 7fcd1b7fe700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd08066f30 0x7fcd080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.276+0000 7fcd1b7fe700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd08066f30 0x7fcd080693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009ca0 tx=0x7fcd0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:54.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.279+0000 7fcd037fe700 1 -- 172.21.15.7:0/3003819698 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd10051af0 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.485+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd1c15aad0 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.486+0000 7fcd037fe700 1 -- 172.21.15.7:0/3003819698 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+12087 (secure 0 0 0) 0x7fcd10055110 con 0x7fcd1c0ac5b0 2024-01-11T07:11:54.486 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:54.486 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:11:53.669736+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:11:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd08066f30 msgr2=0x7fcd080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd08066f30 0x7fcd080693e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009ca0 tx=0x7fcd0c008040 comp rx=0 tx=0).stop 2024-01-11T07:11:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 msgr2=0x7fcd1c0cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0cb400 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ec50 tx=0x7fcd10009250 comp rx=0 tx=0).stop 2024-01-11T07:11:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 shutdown_connections 2024-01-11T07:11:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd08066f30 0x7fcd080693e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd1c15a280 0x7fcd1c0cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd1c063230 0x7fcd1c153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 --2- 172.21.15.7:0/3003819698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1c0ac5b0 0x7fcd1c0cb400 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 >> 172.21.15.7:0/3003819698 conn(0x7fcd1c0680e0 msgr2=0x7fcd1c15cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 shutdown_connections 2024-01-11T07:11:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.488+0000 7fcd21e33700 1 -- 172.21.15.7:0/3003819698 wait complete. 2024-01-11T07:11:54.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:11:54.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:11:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.637+0000 7f45360fa700 1 Processor -- start 2024-01-11T07:11:54.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.637+0000 7f45360fa700 1 -- start start 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f45360fa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 0x7f4530063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f45360fa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f453015a280 0x7f45300aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f45360fa700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45300ac5b0 0x7f45300aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f45360fa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453015d880 con 0x7f453015a280 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f45360fa700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453006c5b0 con 0x7f45300ac5b0 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f45360fa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4530063b40 con 0x7f4530063230 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f452effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f453015a280 0x7f45300aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f452effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f453015a280 0x7f45300aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59934/0 (socket says 172.21.15.7:59934) 2024-01-11T07:11:54.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f452effd700 1 -- 172.21.15.7:0/568724676 learned_addr learned my addr 172.21.15.7:0/568724676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f452ffff700 1 --2- 172.21.15.7:0/568724676 >> v2:172.21.15.113:3300/0 conn(0x7f45300ac5b0 0x7f45300aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.638+0000 7f452f7fe700 1 --2- 172.21.15.7:0/568724676 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 0x7f4530063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452effd700 1 -- 172.21.15.7:0/568724676 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 msgr2=0x7f4530063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452effd700 1 --2- 172.21.15.7:0/568724676 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 0x7f4530063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452effd700 1 -- 172.21.15.7:0/568724676 >> v2:172.21.15.113:3300/0 conn(0x7f45300ac5b0 msgr2=0x7f45300aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452effd700 1 --2- 172.21.15.7:0/568724676 >> v2:172.21.15.113:3300/0 conn(0x7f45300ac5b0 0x7f45300aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452effd700 1 -- 172.21.15.7:0/568724676 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453015a7f0 con 0x7f453015a280 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452effd700 1 --2- 172.21.15.7:0/568724676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453015a280 0x7f45300aa9f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f453006f050 tx=0x7f4524009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a358e837b69dfa4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452e7fc700 1 -- 172.21.15.7:0/568724676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400b640 con 0x7f453015a280 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452e7fc700 1 -- 172.21.15.7:0/568724676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452400b7a0 con 0x7f453015a280 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.639+0000 7f452e7fc700 1 -- 172.21.15.7:0/568724676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400bb60 con 0x7f453015a280 2024-01-11T07:11:54.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 -- 172.21.15.7:0/568724676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453015a280 msgr2=0x7f45300aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 --2- 172.21.15.7:0/568724676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453015a280 0x7f45300aa9f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f453006f050 tx=0x7f4524009250 comp rx=0 tx=0).stop 2024-01-11T07:11:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 -- 172.21.15.7:0/568724676 shutdown_connections 2024-01-11T07:11:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 --2- 172.21.15.7:0/568724676 >> v2:172.21.15.113:3300/0 conn(0x7f45300ac5b0 0x7f45300aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 --2- 172.21.15.7:0/568724676 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 0x7f4530063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 --2- 172.21.15.7:0/568724676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f453015a280 0x7f45300aa9f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 -- 172.21.15.7:0/568724676 >> 172.21.15.7:0/568724676 conn(0x7f45300680e0 msgr2=0x7f45300cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 -- 172.21.15.7:0/568724676 shutdown_connections 2024-01-11T07:11:54.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.640+0000 7f45360fa700 1 -- 172.21.15.7:0/568724676 wait complete. 2024-01-11T07:11:54.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.641+0000 7f45360fa700 1 Processor -- start 2024-01-11T07:11:54.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.641+0000 7f45360fa700 1 -- start start 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f45360fa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 0x7f45300b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f45360fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45300ac5b0 0x7f453016a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f45360fa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f453015a280 0x7f453016a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f45360fa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45300c32f0 con 0x7f45300ac5b0 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f45360fa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453015f650 con 0x7f453015a280 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f45360fa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45300c29c0 con 0x7f4530063230 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f452effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45300ac5b0 0x7f453016a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f452effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45300ac5b0 0x7f453016a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59938/0 (socket says 172.21.15.7:59938) 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f452effd700 1 -- 172.21.15.7:0/3158457468 learned_addr learned my addr 172.21.15.7:0/3158457468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f452f7fe700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 0x7f45300b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.642+0000 7f452ffff700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f453015a280 0x7f453016a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f452effd700 1 -- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 msgr2=0x7f45300b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f452effd700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 0x7f45300b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f452effd700 1 -- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f453015a280 msgr2=0x7f453016a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f452effd700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f453015a280 0x7f453016a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f452effd700 1 -- 172.21.15.7:0/3158457468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45240094f0 con 0x7f45300ac5b0 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f452effd700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45300ac5b0 0x7f453016a110 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f453006d820 tx=0x7f4524017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f45177fe700 1 -- 172.21.15.7:0/3158457468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400b910 con 0x7f45300ac5b0 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f45177fe700 1 -- 172.21.15.7:0/3158457468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452400ba70 con 0x7f45300ac5b0 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45300c2ca0 con 0x7f45300ac5b0 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f45177fe700 1 -- 172.21.15.7:0/3158457468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524011630 con 0x7f45300ac5b0 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.643+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45300b25f0 con 0x7f45300ac5b0 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.644+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45300ac0f0 con 0x7f45300ac5b0 2024-01-11T07:11:54.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.645+0000 7f45177fe700 1 -- 172.21.15.7:0/3158457468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f45240119a0 con 0x7f45300ac5b0 2024-01-11T07:11:54.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.645+0000 7f45177fe700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f451c066f30 0x7f451c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:54.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.645+0000 7f45177fe700 1 -- 172.21.15.7:0/3158457468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4524085880 con 0x7f45300ac5b0 2024-01-11T07:11:54.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.645+0000 7f452f7fe700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f451c066f30 0x7f451c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:54.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.646+0000 7f452f7fe700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f451c066f30 0x7f451c0693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4520009d10 tx=0x7f45200091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:54.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.650+0000 7f45177fe700 1 -- 172.21.15.7:0/3158457468 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4524052e40 con 0x7f45300ac5b0 2024-01-11T07:11:54.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.858+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f453004c3c0 con 0x7f45300ac5b0 2024-01-11T07:11:54.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.859+0000 7f45177fe700 1 -- 172.21.15.7:0/3158457468 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45240150a0 con 0x7f45300ac5b0 2024-01-11T07:11:54.859 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:11:54.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f451c066f30 msgr2=0x7f451c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f451c066f30 0x7f451c0693e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4520009d10 tx=0x7f45200091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45300ac5b0 msgr2=0x7f453016a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:54.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45300ac5b0 0x7f453016a110 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f453006d820 tx=0x7f4524017000 comp rx=0 tx=0).stop 2024-01-11T07:11:54.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 shutdown_connections 2024-01-11T07:11:54.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f451c066f30 0x7f451c0693e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f453015a280 0x7f453016a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4530063230 0x7f45300b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 --2- 172.21.15.7:0/3158457468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45300ac5b0 0x7f453016a110 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:54.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 >> 172.21.15.7:0/3158457468 conn(0x7f45300680e0 msgr2=0x7f453006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:54.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 shutdown_connections 2024-01-11T07:11:54.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:54.861+0000 7f45360fa700 1 -- 172.21.15.7:0/3158457468 wait complete. 2024-01-11T07:11:54.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:11:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.006+0000 7f0ce19e7700 1 Processor -- start 2024-01-11T07:11:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.006+0000 7f0ce19e7700 1 -- start start 2024-01-11T07:11:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0ce19e7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cdc0cd6d0 0x7f0cdc0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0ce19e7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cdc15d8a0 0x7f0cdc15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0ce19e7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc15e280 0x7f0cdc0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0ce19e7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cdc15e700 con 0x7f0cdc0cd6d0 2024-01-11T07:11:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0ce19e7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cdc15acf0 con 0x7f0cdc15d8a0 2024-01-11T07:11:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0ce19e7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc15afd0 con 0x7f0cdc15e280 2024-01-11T07:11:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0cdaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cdc0cd6d0 0x7f0cdc0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0cdaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cdc0cd6d0 0x7f0cdc0cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59950/0 (socket says 172.21.15.7:59950) 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.007+0000 7f0cdaffd700 1 -- 172.21.15.7:0/485707488 learned_addr learned my addr 172.21.15.7:0/485707488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cda7fc700 1 --2- 172.21.15.7:0/485707488 >> v2:172.21.15.113:3300/0 conn(0x7f0cdc15d8a0 0x7f0cdc15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cdb7fe700 1 --2- 172.21.15.7:0/485707488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc15e280 0x7f0cdc0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cdaffd700 1 -- 172.21.15.7:0/485707488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc15e280 msgr2=0x7f0cdc0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cdaffd700 1 --2- 172.21.15.7:0/485707488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc15e280 0x7f0cdc0cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cdaffd700 1 -- 172.21.15.7:0/485707488 >> v2:172.21.15.113:3300/0 conn(0x7f0cdc15d8a0 msgr2=0x7f0cdc15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cdaffd700 1 --2- 172.21.15.7:0/485707488 >> v2:172.21.15.113:3300/0 conn(0x7f0cdc15d8a0 0x7f0cdc15dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cdaffd700 1 -- 172.21.15.7:0/485707488 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc0ce980 con 0x7f0cdc0cd6d0 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cdaffd700 1 --2- 172.21.15.7:0/485707488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc0cd6d0 0x7f0cdc0cdaa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0cc620 tx=0x7f0ccc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ae1e19ebbc5447 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cd9ffb700 1 -- 172.21.15.7:0/485707488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00b860 con 0x7f0cdc0cd6d0 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cd9ffb700 1 -- 172.21.15.7:0/485707488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc00b9c0 con 0x7f0cdc0cd6d0 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.008+0000 7f0cd9ffb700 1 -- 172.21.15.7:0/485707488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00bd60 con 0x7f0cdc0cd6d0 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 -- 172.21.15.7:0/485707488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc0cd6d0 msgr2=0x7f0cdc0cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/485707488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc0cd6d0 0x7f0cdc0cdaa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0cc620 tx=0x7f0ccc009250 comp rx=0 tx=0).stop 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 -- 172.21.15.7:0/485707488 shutdown_connections 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/485707488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc0cd6d0 0x7f0cdc0cdaa0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/485707488 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc15e280 0x7f0cdc0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/485707488 >> v2:172.21.15.113:3300/0 conn(0x7f0cdc15d8a0 0x7f0cdc15dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 -- 172.21.15.7:0/485707488 >> 172.21.15.7:0/485707488 conn(0x7f0cdc071120 msgr2=0x7f0cdc06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 -- 172.21.15.7:0/485707488 shutdown_connections 2024-01-11T07:11:55.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.009+0000 7f0ce19e7700 1 -- 172.21.15.7:0/485707488 wait complete. 2024-01-11T07:11:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.011+0000 7f0ce19e7700 1 Processor -- start 2024-01-11T07:11:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.011+0000 7f0ce19e7700 1 -- start start 2024-01-11T07:11:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0ce19e7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc0cd6d0 0x7f0cdc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0ce19e7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cdc15d8a0 0x7f0cdc15d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0ce19e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc15e280 0x7f0cdc148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0ce19e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc15d550 con 0x7f0cdc15e280 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0ce19e7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc169e90 con 0x7f0cdc15d8a0 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0ce19e7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc0b4020 con 0x7f0cdc0cd6d0 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc15e280 0x7f0cdc148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc15e280 0x7f0cdc148620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59958/0 (socket says 172.21.15.7:59958) 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 -- 172.21.15.7:0/825290263 learned_addr learned my addr 172.21.15.7:0/825290263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cda7fc700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cdc15d8a0 0x7f0cdc15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdaffd700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc0cd6d0 0x7f0cdc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 -- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc0cd6d0 msgr2=0x7f0cdc0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc0cd6d0 0x7f0cdc0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 -- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cdc15d8a0 msgr2=0x7f0cdc15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cdc15d8a0 0x7f0cdc15d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 -- 172.21.15.7:0/825290263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc0094f0 con 0x7f0cdc15e280 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdaffd700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc0cd6d0 0x7f0cdc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cdb7fe700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc15e280 0x7f0cdc148620 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4007f20 tx=0x7f0cd4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.012+0000 7f0cd2ffd700 1 -- 172.21.15.7:0/825290263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400ebf0 con 0x7f0cdc15e280 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.013+0000 7f0cd2ffd700 1 -- 172.21.15.7:0/825290263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd400ed70 con 0x7f0cdc15e280 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.013+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc0b4300 con 0x7f0cdc15e280 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.013+0000 7f0cd2ffd700 1 -- 172.21.15.7:0/825290263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4014670 con 0x7f0cdc15e280 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.013+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc0b48c0 con 0x7f0cdc15e280 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.014+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cdc16a1e0 con 0x7f0cdc15e280 2024-01-11T07:11:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.014+0000 7f0cd2ffd700 1 -- 172.21.15.7:0/825290263 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0cd40149e0 con 0x7f0cdc15e280 2024-01-11T07:11:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.014+0000 7f0cd2ffd700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cbc066f30 0x7f0cbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.015+0000 7f0cd2ffd700 1 -- 172.21.15.7:0/825290263 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0cd4084870 con 0x7f0cdc15e280 2024-01-11T07:11:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.015+0000 7f0cdaffd700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cbc066f30 0x7f0cbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.015+0000 7f0cdaffd700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cbc066f30 0x7f0cbc0693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0cc620 tx=0x7f0ccc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.019+0000 7f0cd2ffd700 1 -- 172.21.15.7:0/825290263 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cd4051d90 con 0x7f0cdc15e280 2024-01-11T07:11:55.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.224+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0cdc0b4ae0 con 0x7f0cdc15e280 2024-01-11T07:11:55.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.224+0000 7f0cd2ffd700 1 -- 172.21.15.7:0/825290263 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0cd40553b0 con 0x7f0cdc15e280 2024-01-11T07:11:55.224 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:55.224 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:11:55.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:11:55.225 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:11:55.225 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:11:55.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:11:55.225 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:11:55.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cbc066f30 msgr2=0x7f0cbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cbc066f30 0x7f0cbc0693e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0cc620 tx=0x7f0ccc017000 comp rx=0 tx=0).stop 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc15e280 msgr2=0x7f0cdc148620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc15e280 0x7f0cdc148620 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4007f20 tx=0x7f0cd4009860 comp rx=0 tx=0).stop 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 shutdown_connections 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cbc066f30 0x7f0cbc0693e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cdc15e280 0x7f0cdc148620 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cdc0cd6d0 0x7f0cdc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 --2- 172.21.15.7:0/825290263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cdc15d8a0 0x7f0cdc15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 >> 172.21.15.7:0/825290263 conn(0x7f0cdc071120 msgr2=0x7f0cdc0cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 shutdown_connections 2024-01-11T07:11:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.227+0000 7f0ce19e7700 1 -- 172.21.15.7:0/825290263 wait complete. 2024-01-11T07:11:55.240 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:11:55.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:11:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.376+0000 7f670a736700 1 Processor -- start 2024-01-11T07:11:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.376+0000 7f670a736700 1 -- start start 2024-01-11T07:11:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f670a736700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6704157990 0x7f670406d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f670a736700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157d60 0x7f67040ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f670a736700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f670415c3b0 0x7f67040caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f670a736700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67040cd260 con 0x7f6704157990 2024-01-11T07:11:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f670a736700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67040caf40 con 0x7f670415c3b0 2024-01-11T07:11:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f670a736700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67040cc690 con 0x7f6704157d60 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f6703fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6704157990 0x7f670406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f6703fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6704157990 0x7f670406d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59968/0 (socket says 172.21.15.7:59968) 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.377+0000 7f6703fff700 1 -- 172.21.15.7:0/2718861604 learned_addr learned my addr 172.21.15.7:0/2718861604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6708cd3700 1 --2- 172.21.15.7:0/2718861604 >> v2:172.21.15.113:3300/0 conn(0x7f670415c3b0 0x7f67040caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f67037fe700 1 --2- 172.21.15.7:0/2718861604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157d60 0x7f67040ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6703fff700 1 -- 172.21.15.7:0/2718861604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157d60 msgr2=0x7f67040ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6703fff700 1 --2- 172.21.15.7:0/2718861604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157d60 0x7f67040ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6703fff700 1 -- 172.21.15.7:0/2718861604 >> v2:172.21.15.113:3300/0 conn(0x7f670415c3b0 msgr2=0x7f67040caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:55.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6703fff700 1 --2- 172.21.15.7:0/2718861604 >> v2:172.21.15.113:3300/0 conn(0x7f670415c3b0 0x7f67040caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6703fff700 1 -- 172.21.15.7:0/2718861604 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67040cc930 con 0x7f6704157990 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f67037fe700 1 --2- 172.21.15.7:0/2718861604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157d60 0x7f67040ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6703fff700 1 --2- 172.21.15.7:0/2718861604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6704157990 0x7f670406d140 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f67040b0a40 tx=0x7f66f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800fe31325ccab65 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6702ffd700 1 -- 172.21.15.7:0/2718861604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4009e70 con 0x7f6704157990 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6702ffd700 1 -- 172.21.15.7:0/2718861604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f400b860 con 0x7f6704157990 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.378+0000 7f6702ffd700 1 -- 172.21.15.7:0/2718861604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400bc00 con 0x7f6704157990 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 -- 172.21.15.7:0/2718861604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6704157990 msgr2=0x7f670406d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 --2- 172.21.15.7:0/2718861604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6704157990 0x7f670406d140 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f67040b0a40 tx=0x7f66f4009250 comp rx=0 tx=0).stop 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 -- 172.21.15.7:0/2718861604 shutdown_connections 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 --2- 172.21.15.7:0/2718861604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6704157990 0x7f670406d140 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 --2- 172.21.15.7:0/2718861604 >> v2:172.21.15.113:3300/0 conn(0x7f670415c3b0 0x7f67040caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 --2- 172.21.15.7:0/2718861604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157d60 0x7f67040ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 -- 172.21.15.7:0/2718861604 >> 172.21.15.7:0/2718861604 conn(0x7f6704063230 msgr2=0x7f6704063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 -- 172.21.15.7:0/2718861604 shutdown_connections 2024-01-11T07:11:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.379+0000 7f670a736700 1 -- 172.21.15.7:0/2718861604 wait complete. 2024-01-11T07:11:55.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.380+0000 7f670a736700 1 Processor -- start 2024-01-11T07:11:55.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.380+0000 7f670a736700 1 -- start start 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f670a736700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157990 0x7f670414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f670a736700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6704157d60 0x7f670416b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f670a736700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f670415c3b0 0x7f6704153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f670a736700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670416c470 con 0x7f670415c3b0 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f670a736700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67040b2440 con 0x7f6704157d60 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f670a736700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670416beb0 con 0x7f6704157990 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f6708cd3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f670415c3b0 0x7f6704153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f6708cd3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f670415c3b0 0x7f6704153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59970/0 (socket says 172.21.15.7:59970) 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.381+0000 7f6708cd3700 1 -- 172.21.15.7:0/2157979660 learned_addr learned my addr 172.21.15.7:0/2157979660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f67037fe700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6704157d60 0x7f670416b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f6703fff700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157990 0x7f670414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f6708cd3700 1 -- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157990 msgr2=0x7f670414fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f6708cd3700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157990 0x7f670414fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f6708cd3700 1 -- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6704157d60 msgr2=0x7f670416b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f6708cd3700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6704157d60 0x7f670416b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f6708cd3700 1 -- 172.21.15.7:0/2157979660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f40094f0 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f6708cd3700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f670415c3b0 0x7f6704153d50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009df0 tx=0x7f66fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f66fbfff700 1 -- 172.21.15.7:0/2157979660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc00cb50 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f66fbfff700 1 -- 172.21.15.7:0/2157979660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc00f040 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670416c190 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f66fbfff700 1 -- 172.21.15.7:0/2157979660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc014480 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.382+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67040b29a0 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.383+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670404c3c0 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.384+0000 7f66fbfff700 1 -- 172.21.15.7:0/2157979660 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f66fc014830 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.384+0000 7f66fbfff700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.384+0000 7f66fbfff700 1 -- 172.21.15.7:0/2157979660 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f66fc084b60 con 0x7f670415c3b0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.384+0000 7f6703fff700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.385+0000 7f6703fff700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f67040b0a40 tx=0x7f66f4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.388+0000 7f66fbfff700 1 -- 172.21.15.7:0/2157979660 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66fc052120 con 0x7f670415c3b0 2024-01-11T07:11:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.594+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f67040b2b90 con 0x7f670415c3b0 2024-01-11T07:11:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.675+0000 7f66fbfff700 1 -- 172.21.15.7:0/2157979660 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) v1 ==== 138+0+0 (secure 0 0 0) 0x7f66fc055740 con 0x7f670415c3b0 2024-01-11T07:11:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.679+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 msgr2=0x7f66e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.679+0000 7f670a736700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f67040b0a40 tx=0x7f66f4017000 comp rx=0 tx=0).stop 2024-01-11T07:11:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.679+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f670415c3b0 msgr2=0x7f6704153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.679+0000 7f670a736700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f670415c3b0 0x7f6704153d50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009df0 tx=0x7f66fc00b040 comp rx=0 tx=0).stop 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.680+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 shutdown_connections 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.680+0000 7f670a736700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f66e4066f30 0x7f66e40693e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.680+0000 7f670a736700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f670415c3b0 0x7f6704153d50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.680+0000 7f670a736700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6704157990 0x7f670414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.680+0000 7f670a736700 1 --2- 172.21.15.7:0/2157979660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6704157d60 0x7f670416b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.680+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 >> 172.21.15.7:0/2157979660 conn(0x7f6704063230 msgr2=0x7f67040cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.680+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 shutdown_connections 2024-01-11T07:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.680+0000 7f670a736700 1 -- 172.21.15.7:0/2157979660 wait complete. 2024-01-11T07:11:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:11:55.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:11:55.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.834+0000 7fbd12385700 1 Processor -- start 2024-01-11T07:11:55.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.834+0000 7fbd12385700 1 -- start start 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd12385700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063230 0x7fbd0c15e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd12385700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd0c15aa40 0x7fbd0c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd12385700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0c15df80 con 0x7fbd0c063600 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd12385700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c15e950 con 0x7fbd0c15aa40 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd12385700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0aab70 con 0x7fbd0c063230 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd10922700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd0c15aa40 0x7fbd0c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd0b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbd0c063600 0x7fbd0c15a330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45312/0 (socket says 172.21.15.7:45312) 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd10922700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd0c15aa40 0x7fbd0c06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59974/0 (socket says 172.21.15.7:59974) 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.835+0000 7fbd0b7fe700 1 -- 172.21.15.7:0/1099827075 learned_addr learned my addr 172.21.15.7:0/1099827075 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd0bfff700 1 --2- 172.21.15.7:0/1099827075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063230 0x7fbd0c15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd10922700 1 -- 172.21.15.7:0/1099827075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063230 msgr2=0x7fbd0c15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd10922700 1 --2- 172.21.15.7:0/1099827075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063230 0x7fbd0c15e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd10922700 1 -- 172.21.15.7:0/1099827075 >> v1:172.21.15.7:6789/0 conn(0x7fbd0c063600 legacy=0x7fbd0c15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd10922700 1 -- 172.21.15.7:0/1099827075 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c15af10 con 0x7fbd0c15aa40 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd0affd700 1 -- 172.21.15.7:0/1099827075 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597932269 0 0) 0x7fbd0c15df80 con 0x7fbd0c063600 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd10922700 1 --2- 172.21.15.7:0/1099827075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c06b530 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009e60 tx=0x7fbd04009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65c5fa7dbaa1b58 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd0affd700 1 -- 172.21.15.7:0/1099827075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400e070 con 0x7fbd0c15aa40 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd0affd700 1 -- 172.21.15.7:0/1099827075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0400bc30 con 0x7fbd0c15aa40 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.836+0000 7fbd0affd700 1 -- 172.21.15.7:0/1099827075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04012430 con 0x7fbd0c15aa40 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.837+0000 7fbd12385700 1 -- 172.21.15.7:0/1099827075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 msgr2=0x7fbd0c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.837+0000 7fbd12385700 1 --2- 172.21.15.7:0/1099827075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c06b530 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009e60 tx=0x7fbd04009780 comp rx=0 tx=0).stop 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.837+0000 7fbd12385700 1 -- 172.21.15.7:0/1099827075 shutdown_connections 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.837+0000 7fbd12385700 1 --2- 172.21.15.7:0/1099827075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c06b530 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.837+0000 7fbd12385700 1 --2- 172.21.15.7:0/1099827075 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063230 0x7fbd0c15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.837+0000 7fbd12385700 1 -- 172.21.15.7:0/1099827075 >> 172.21.15.7:0/1099827075 conn(0x7fbd0c0cd910 msgr2=0x7fbd0c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:55.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.837+0000 7fbd12385700 1 -- 172.21.15.7:0/1099827075 shutdown_connections 2024-01-11T07:11:55.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.837+0000 7fbd12385700 1 -- 172.21.15.7:0/1099827075 wait complete. 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.838+0000 7fbd12385700 1 Processor -- start 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.838+0000 7fbd12385700 1 -- start start 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd12385700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd0c063230 0x7fbd0c14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd12385700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063600 0x7fbd0c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd12385700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd12385700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c14dd00 con 0x7fbd0c15aa40 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd12385700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c14ce10 con 0x7fbd0c063230 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd12385700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c14d0f0 con 0x7fbd0c063600 2024-01-11T07:11:55.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd10922700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd10922700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c0b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59978/0 (socket says 172.21.15.7:59978) 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd10922700 1 -- 172.21.15.7:0/1113292107 learned_addr learned my addr 172.21.15.7:0/1113292107 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd0bfff700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd0c063230 0x7fbd0c14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd0b7fe700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063600 0x7fbd0c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd10922700 1 -- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063600 msgr2=0x7fbd0c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd10922700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063600 0x7fbd0c0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd10922700 1 -- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd0c063230 msgr2=0x7fbd0c14bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd10922700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd0c063230 0x7fbd0c14bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.839+0000 7fbd10922700 1 -- 172.21.15.7:0/1113292107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04009a20 con 0x7fbd0c15aa40 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.840+0000 7fbd10922700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c0b21b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbd04013000 tx=0x7fbd04019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.840+0000 7fbcf3fff700 1 -- 172.21.15.7:0/1113292107 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04012e30 con 0x7fbd0c15aa40 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.840+0000 7fbcf3fff700 1 -- 172.21.15.7:0/1113292107 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0401d630 con 0x7fbd0c15aa40 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.840+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c14d3d0 con 0x7fbd0c15aa40 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.840+0000 7fbcf3fff700 1 -- 172.21.15.7:0/1113292107 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0401d9f0 con 0x7fbd0c15aa40 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.840+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c14d990 con 0x7fbd0c15aa40 2024-01-11T07:11:55.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.841+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c04c3c0 con 0x7fbd0c15aa40 2024-01-11T07:11:55.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.841+0000 7fbcf3fff700 1 -- 172.21.15.7:0/1113292107 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbd04024070 con 0x7fbd0c15aa40 2024-01-11T07:11:55.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.842+0000 7fbcf3fff700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbcf8066f30 0x7fbcf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:55.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.842+0000 7fbcf3fff700 1 -- 172.21.15.7:0/1113292107 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fbd04086c80 con 0x7fbd0c15aa40 2024-01-11T07:11:55.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.842+0000 7fbd0bfff700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbcf8066f30 0x7fbcf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:55.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.843+0000 7fbd0bfff700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbcf8066f30 0x7fbcf80693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc006fd0 tx=0x7fbcfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:55.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:55.847+0000 7fbcf3fff700 1 -- 172.21.15.7:0/1113292107 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd040540d0 con 0x7fbd0c15aa40 2024-01-11T07:11:56.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.050+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbd0c15b1f0 con 0x7fbd0c15aa40 2024-01-11T07:11:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.685+0000 7fbcf3fff700 1 -- 172.21.15.7:0/1113292107 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbd0402c0d0 con 0x7fbd0c15aa40 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbcf8066f30 msgr2=0x7fbcf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbcf8066f30 0x7fbcf80693e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc006fd0 tx=0x7fbcfc008040 comp rx=0 tx=0).stop 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 msgr2=0x7fbd0c0b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c0b21b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbd04013000 tx=0x7fbd04019000 comp rx=0 tx=0).stop 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 shutdown_connections 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbcf8066f30 0x7fbcf80693e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd0c15aa40 0x7fbd0c0b21b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd0c063230 0x7fbd0c14bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 --2- 172.21.15.7:0/1113292107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd0c063600 0x7fbd0c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:56.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 >> 172.21.15.7:0/1113292107 conn(0x7fbd0c0cd910 msgr2=0x7fbd0c15b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:56.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 shutdown_connections 2024-01-11T07:11:56.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.696+0000 7fbd12385700 1 -- 172.21.15.7:0/1113292107 wait complete. 2024-01-11T07:11:56.697 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:11:56.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:11:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.851+0000 7f3f869c9700 1 Processor -- start 2024-01-11T07:11:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.851+0000 7f3f869c9700 1 -- start start 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f869c9700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f800add00 0x7f3f80158c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f869c9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f800cee70 con 0x7f3f800b35e0 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f869c9700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f800ce840 con 0x7f3f800b0430 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f869c9700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800cfa10 con 0x7f3f800add00 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f84f66700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3f800b35e0 0x7f3f800b39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45316/0 (socket says 172.21.15.7:45316) 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f84f66700 1 -- 172.21.15.7:0/189768931 learned_addr learned my addr 172.21.15.7:0/189768931 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f7ffff700 1 --2- 172.21.15.7:0/189768931 >> v2:172.21.15.113:3300/0 conn(0x7f3f800add00 0x7f3f80158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f7effd700 1 -- 172.21.15.7:0/189768931 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489544190 0 0) 0x7f3f800cee70 con 0x7f3f800b35e0 2024-01-11T07:11:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f7effd700 1 -- 172.21.15.7:0/189768931 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f68003570 con 0x7f3f800b35e0 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f7effd700 1 -- 172.21.15.7:0/189768931 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146812150 0 0) 0x7f3f800ce840 con 0x7f3f800b0430 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f7effd700 1 -- 172.21.15.7:0/189768931 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f800cee70 con 0x7f3f800b0430 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f7ffff700 1 -- 172.21.15.7:0/189768931 >> v1:172.21.15.113:6789/0 conn(0x7f3f800b0430 legacy=0x7f3f800cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f7ffff700 1 -- 172.21.15.7:0/189768931 >> v1:172.21.15.7:6789/0 conn(0x7f3f800b35e0 legacy=0x7f3f800b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.852+0000 7f3f7ffff700 1 -- 172.21.15.7:0/189768931 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800b08b0 con 0x7f3f800add00 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f7ffff700 1 --2- 172.21.15.7:0/189768931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800add00 0x7f3f80158c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3f700098e0 tx=0x7f3f700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=117e6f1ec1694f8f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f7effd700 1 -- 172.21.15.7:0/189768931 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70011630 con 0x7f3f800add00 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f7effd700 1 -- 172.21.15.7:0/189768931 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f70011790 con 0x7f3f800add00 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f7effd700 1 -- 172.21.15.7:0/189768931 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70011b10 con 0x7f3f800add00 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f869c9700 1 -- 172.21.15.7:0/189768931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800add00 msgr2=0x7f3f80158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f869c9700 1 --2- 172.21.15.7:0/189768931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800add00 0x7f3f80158c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3f700098e0 tx=0x7f3f700091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f869c9700 1 -- 172.21.15.7:0/189768931 shutdown_connections 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f869c9700 1 --2- 172.21.15.7:0/189768931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800add00 0x7f3f80158c80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f869c9700 1 -- 172.21.15.7:0/189768931 >> 172.21.15.7:0/189768931 conn(0x7f3f80071110 msgr2=0x7f3f8006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f869c9700 1 -- 172.21.15.7:0/189768931 shutdown_connections 2024-01-11T07:11:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.853+0000 7f3f869c9700 1 -- 172.21.15.7:0/189768931 wait complete. 2024-01-11T07:11:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.855+0000 7f3f869c9700 1 Processor -- start 2024-01-11T07:11:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.855+0000 7f3f869c9700 1 -- start start 2024-01-11T07:11:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.856+0000 7f3f869c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f800add00 0x7f3f801574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.856+0000 7f3f869c9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f800b0430 0x7f3f800af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.856+0000 7f3f869c9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800b35e0 0x7f3f800af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.856+0000 7f3f869c9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800ad300 con 0x7f3f800add00 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.856+0000 7f3f869c9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800ac2a0 con 0x7f3f800b35e0 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.856+0000 7f3f869c9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800ac540 con 0x7f3f800b0430 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.856+0000 7f3f7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f800add00 0x7f3f801574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f800add00 0x7f3f801574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59986/0 (socket says 172.21.15.7:59986) 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7ffff700 1 -- 172.21.15.7:0/3591424858 learned_addr learned my addr 172.21.15.7:0/3591424858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f84f66700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800b35e0 0x7f3f800af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7f7fe700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f800b0430 0x7f3f800af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7ffff700 1 -- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f800b0430 msgr2=0x7f3f800af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7ffff700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f800b0430 0x7f3f800af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7ffff700 1 -- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800b35e0 msgr2=0x7f3f800af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7ffff700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800b35e0 0x7f3f800af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7ffff700 1 -- 172.21.15.7:0/3591424858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f70009420 con 0x7f3f800add00 2024-01-11T07:11:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f7ffff700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f800add00 0x7f3f801574f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009ed0 tx=0x7f3f70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f67fff700 1 -- 172.21.15.7:0/3591424858 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70011b10 con 0x7f3f800add00 2024-01-11T07:11:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f67fff700 1 -- 172.21.15.7:0/3591424858 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f70000c90 con 0x7f3f800add00 2024-01-11T07:11:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f67fff700 1 -- 172.21.15.7:0/3591424858 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7001a680 con 0x7f3f800add00 2024-01-11T07:11:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f800ac820 con 0x7f3f800add00 2024-01-11T07:11:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.857+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f800acde0 con 0x7f3f800add00 2024-01-11T07:11:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.858+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8004c3c0 con 0x7f3f800add00 2024-01-11T07:11:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.859+0000 7f3f67fff700 1 -- 172.21.15.7:0/3591424858 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3f7001aa30 con 0x7f3f800add00 2024-01-11T07:11:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.859+0000 7f3f67fff700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3f6c066f30 0x7f3f6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.859+0000 7f3f67fff700 1 -- 172.21.15.7:0/3591424858 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f3f70086090 con 0x7f3f800add00 2024-01-11T07:11:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.859+0000 7f3f7f7fe700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3f6c066f30 0x7f3f6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.860+0000 7f3f7f7fe700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3f6c066f30 0x7f3f6c0693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3f800a9b80 tx=0x7f3f74007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:56.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:56.864+0000 7f3f67fff700 1 -- 172.21.15.7:0/3591424858 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f700532c0 con 0x7f3f800add00 2024-01-11T07:11:57.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.117+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3f800b0b80 con 0x7f3f800add00 2024-01-11T07:11:57.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.683+0000 7f3f67fff700 1 -- 172.21.15.7:0/3591424858 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3f700568e0 con 0x7f3f800add00 2024-01-11T07:11:57.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3f6c066f30 msgr2=0x7f3f6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3f6c066f30 0x7f3f6c0693e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3f800a9b80 tx=0x7f3f74007d30 comp rx=0 tx=0).stop 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f800add00 msgr2=0x7f3f801574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f800add00 0x7f3f801574f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009ed0 tx=0x7f3f70017000 comp rx=0 tx=0).stop 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 shutdown_connections 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3f6c066f30 0x7f3f6c0693e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f800b35e0 0x7f3f800af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f800add00 0x7f3f801574f0 secure :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009ed0 tx=0x7f3f70017000 comp rx=0 tx=0).stop 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.690+0000 7f3f869c9700 1 --2- 172.21.15.7:0/3591424858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f800b0430 0x7f3f800af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.691+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 >> 172.21.15.7:0/3591424858 conn(0x7f3f80071110 msgr2=0x7f3f80159810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.692+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 shutdown_connections 2024-01-11T07:11:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.692+0000 7f3f869c9700 1 -- 172.21.15.7:0/3591424858 wait complete. 2024-01-11T07:11:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 12 and data pool 13 2024-01-11T07:11:57.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:11:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.843+0000 7f6d3e29c700 1 Processor -- start 2024-01-11T07:11:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.843+0000 7f6d3e29c700 1 -- start start 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.844+0000 7f6d3e29c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6d38159c20 0x7f6d3815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.844+0000 7f6d3e29c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d380ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.844+0000 7f6d3e29c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3815aa80 con 0x7f6d380ac450 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.844+0000 7f6d3e29c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d380ace30 con 0x7f6d38159c20 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.844+0000 7f6d3e29c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380ad010 con 0x7f6d38063940 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.844+0000 7f6d3c839700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d380ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.844+0000 7f6d37fff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6d38159c20 0x7f6d3815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d3c839700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d380ac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:48220/0 (socket says 172.21.15.7:48220) 2024-01-11T07:11:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d37fff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6d38159c20 0x7f6d3815af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:44282/0 (socket says 172.21.15.7:44282) 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d3c839700 1 -- 172.21.15.7:0/3632984598 learned_addr learned my addr 172.21.15.7:0/3632984598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d377fe700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6d380ac450 0x7f6d38063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:42398/0 (socket says 172.21.15.7:42398) 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d3c839700 1 -- 172.21.15.7:0/3632984598 >> v2:172.21.15.113:3300/0 conn(0x7f6d38159c20 msgr2=0x7f6d3815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d3c839700 1 --2- 172.21.15.7:0/3632984598 >> v2:172.21.15.113:3300/0 conn(0x7f6d38159c20 0x7f6d3815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d3c839700 1 -- 172.21.15.7:0/3632984598 >> v1:172.21.15.113:6789/0 conn(0x7f6d380ac450 legacy=0x7f6d38063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d3c839700 1 -- 172.21.15.7:0/3632984598 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380ad2f0 con 0x7f6d38063940 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d37fff700 1 --2- 172.21.15.7:0/3632984598 >> v2:172.21.15.113:3300/0 conn(0x7f6d38159c20 0x7f6d3815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d36ffd700 1 -- 172.21.15.7:0/3632984598 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619077682 0 0) 0x7f6d3815aa80 con 0x7f6d380ac450 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d3c839700 1 --2- 172.21.15.7:0/3632984598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d380ac8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009f20 tx=0x7f6d30009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c12a91889edbf36 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d36ffd700 1 -- 172.21.15.7:0/3632984598 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000e070 con 0x7f6d38063940 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d36ffd700 1 -- 172.21.15.7:0/3632984598 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3000b9d0 con 0x7f6d38063940 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.845+0000 7f6d36ffd700 1 -- 172.21.15.7:0/3632984598 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300092e0 con 0x7f6d38063940 2024-01-11T07:11:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.846+0000 7f6d3e29c700 1 -- 172.21.15.7:0/3632984598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 msgr2=0x7f6d380ac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.846+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/3632984598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d380ac8f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009f20 tx=0x7f6d30009840 comp rx=0 tx=0).stop 2024-01-11T07:11:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.846+0000 7f6d3e29c700 1 -- 172.21.15.7:0/3632984598 shutdown_connections 2024-01-11T07:11:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.846+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/3632984598 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d380ac8f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.846+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/3632984598 >> v2:172.21.15.113:3300/0 conn(0x7f6d38159c20 0x7f6d3815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.846+0000 7f6d3e29c700 1 -- 172.21.15.7:0/3632984598 >> 172.21.15.7:0/3632984598 conn(0x7f6d380cd910 msgr2=0x7f6d380cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.846+0000 7f6d3e29c700 1 -- 172.21.15.7:0/3632984598 shutdown_connections 2024-01-11T07:11:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.846+0000 7f6d3e29c700 1 -- 172.21.15.7:0/3632984598 wait complete. 2024-01-11T07:11:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.847+0000 7f6d3e29c700 1 Processor -- start 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.847+0000 7f6d3e29c700 1 -- start start 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3e29c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d38169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3e29c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d380ac450 0x7f6d3815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3e29c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d38159c20 0x7f6d380c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3e29c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3816a220 con 0x7f6d38159c20 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3e29c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3816a500 con 0x7f6d380ac450 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3e29c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3816a7e0 con 0x7f6d38063940 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3c839700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d38159c20 0x7f6d380c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3c839700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d38159c20 0x7f6d380c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56802/0 (socket says 172.21.15.7:56802) 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.848+0000 7f6d3c839700 1 -- 172.21.15.7:0/788778131 learned_addr learned my addr 172.21.15.7:0/788778131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d377fe700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d380ac450 0x7f6d3815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d37fff700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d38169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d3c839700 1 -- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 msgr2=0x7f6d38169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d3c839700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d38169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d3c839700 1 -- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d380ac450 msgr2=0x7f6d3815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d3c839700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d380ac450 0x7f6d3815d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d3c839700 1 -- 172.21.15.7:0/788778131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d20009420 con 0x7f6d38159c20 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d3c839700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d38159c20 0x7f6d380c27e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6d30012000 tx=0x7f6d30018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d2ffff700 1 -- 172.21.15.7:0/788778131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d30008c30 con 0x7f6d38159c20 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d2ffff700 1 -- 172.21.15.7:0/788778131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d30008dd0 con 0x7f6d38159c20 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d2ffff700 1 -- 172.21.15.7:0/788778131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300266b0 con 0x7f6d38159c20 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d30009ae0 con 0x7f6d38159c20 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.849+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3815b850 con 0x7f6d38159c20 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.850+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3804c3c0 con 0x7f6d38159c20 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.850+0000 7f6d2ffff700 1 -- 172.21.15.7:0/788778131 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6d30003040 con 0x7f6d38159c20 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.851+0000 7f6d2ffff700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d18066f30 0x7f6d180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.851+0000 7f6d2ffff700 1 -- 172.21.15.7:0/788778131 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6d30086590 con 0x7f6d38159c20 2024-01-11T07:11:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.851+0000 7f6d37fff700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d18066f30 0x7f6d180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.852+0000 7f6d37fff700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d18066f30 0x7f6d180693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d200093f0 tx=0x7f6d20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:57.855+0000 7f6d2ffff700 1 -- 172.21.15.7:0/788778131 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d30053800 con 0x7f6d38159c20 2024-01-11T07:11:58.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.060+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d3815ba40 con 0x7f6d38159c20 2024-01-11T07:11:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.061+0000 7f6d2ffff700 1 -- 172.21.15.7:0/788778131 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+15318 (secure 0 0 0) 0x7f6d30056e20 con 0x7f6d38159c20 2024-01-11T07:11:58.060 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:58.061 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:11:57.118927+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:11:55.626974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-11T07:11:56.083170+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d18066f30 msgr2=0x7f6d180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d18066f30 0x7f6d180693e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d200093f0 tx=0x7f6d20008040 comp rx=0 tx=0).stop 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d38159c20 msgr2=0x7f6d380c27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d38159c20 0x7f6d380c27e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6d30012000 tx=0x7f6d30018000 comp rx=0 tx=0).stop 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 shutdown_connections 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6d18066f30 0x7f6d180693e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d38159c20 0x7f6d380c27e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6d38063940 0x7f6d38169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 --2- 172.21.15.7:0/788778131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d380ac450 0x7f6d3815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 >> 172.21.15.7:0/788778131 conn(0x7f6d380cd910 msgr2=0x7f6d380b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:58.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 shutdown_connections 2024-01-11T07:11:58.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.063+0000 7f6d3e29c700 1 -- 172.21.15.7:0/788778131 wait complete. 2024-01-11T07:11:58.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.209+0000 7fd346e1b700 1 Processor -- start 2024-01-11T07:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.209+0000 7fd346e1b700 1 -- start start 2024-01-11T07:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.210+0000 7fd346e1b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd340159c90 0x7fd3400b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.210+0000 7fd346e1b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd340069320 0x7fd3400b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.210+0000 7fd346e1b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400ce9b0 con 0x7fd34015a060 2024-01-11T07:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.210+0000 7fd346e1b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3400cf530 con 0x7fd340069320 2024-01-11T07:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.210+0000 7fd346e1b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3400cf9c0 con 0x7fd340159c90 2024-01-11T07:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd3453b8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd340069320 0x7fd3400b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd3453b8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd340069320 0x7fd3400b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56806/0 (socket says 172.21.15.7:56806) 2024-01-11T07:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd3453b8700 1 -- 172.21.15.7:0/3735955737 learned_addr learned my addr 172.21.15.7:0/3735955737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd344bb7700 1 --2- 172.21.15.7:0/3735955737 >> v2:172.21.15.113:3300/0 conn(0x7fd340159c90 0x7fd3400b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd3453b8700 1 -- 172.21.15.7:0/3735955737 >> v2:172.21.15.113:3300/0 conn(0x7fd340159c90 msgr2=0x7fd3400b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd3453b8700 1 --2- 172.21.15.7:0/3735955737 >> v2:172.21.15.113:3300/0 conn(0x7fd340159c90 0x7fd3400b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd3453b8700 1 -- 172.21.15.7:0/3735955737 >> v1:172.21.15.113:6789/0 conn(0x7fd34015a060 legacy=0x7fd3400b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd3453b8700 1 -- 172.21.15.7:0/3735955737 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400cec00 con 0x7fd340069320 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd3453b8700 1 --2- 172.21.15.7:0/3735955737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340069320 0x7fd3400b3520 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd34005f2e0 tx=0x7fd338009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f1878cdc69a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd33f7fe700 1 -- 172.21.15.7:0/3735955737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800e070 con 0x7fd340069320 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd33f7fe700 1 -- 172.21.15.7:0/3735955737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800ba20 con 0x7fd340069320 2024-01-11T07:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.211+0000 7fd33f7fe700 1 -- 172.21.15.7:0/3735955737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3380092e0 con 0x7fd340069320 2024-01-11T07:11:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.212+0000 7fd346e1b700 1 -- 172.21.15.7:0/3735955737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340069320 msgr2=0x7fd3400b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.212+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3735955737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340069320 0x7fd3400b3520 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd34005f2e0 tx=0x7fd338009840 comp rx=0 tx=0).stop 2024-01-11T07:11:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.212+0000 7fd346e1b700 1 -- 172.21.15.7:0/3735955737 shutdown_connections 2024-01-11T07:11:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.212+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3735955737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340069320 0x7fd3400b3520 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.212+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3735955737 >> v2:172.21.15.113:3300/0 conn(0x7fd340159c90 0x7fd3400b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.212+0000 7fd346e1b700 1 -- 172.21.15.7:0/3735955737 >> 172.21.15.7:0/3735955737 conn(0x7fd340071100 msgr2=0x7fd34006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.212+0000 7fd346e1b700 1 -- 172.21.15.7:0/3735955737 shutdown_connections 2024-01-11T07:11:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.212+0000 7fd346e1b700 1 -- 172.21.15.7:0/3735955737 wait complete. 2024-01-11T07:11:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.213+0000 7fd346e1b700 1 Processor -- start 2024-01-11T07:11:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.214+0000 7fd346e1b700 1 -- start start 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.214+0000 7fd346e1b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd340069320 0x7fd340152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.214+0000 7fd346e1b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340159c90 0x7fd34015d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.214+0000 7fd346e1b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd34015a060 0x7fd34015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.214+0000 7fd346e1b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34015ed40 con 0x7fd340159c90 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.214+0000 7fd346e1b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400af2b0 con 0x7fd340069320 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.214+0000 7fd346e1b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34015e130 con 0x7fd34015a060 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.214+0000 7fd33ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340159c90 0x7fd34015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340159c90 0x7fd34015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56822/0 (socket says 172.21.15.7:56822) 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33ffff700 1 -- 172.21.15.7:0/3655935835 learned_addr learned my addr 172.21.15.7:0/3655935835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd344bb7700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd340069320 0x7fd340152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd3453b8700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd34015a060 0x7fd34015fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33ffff700 1 -- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd34015a060 msgr2=0x7fd34015fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33ffff700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd34015a060 0x7fd34015fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33ffff700 1 -- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd340069320 msgr2=0x7fd340152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33ffff700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd340069320 0x7fd340152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33ffff700 1 -- 172.21.15.7:0/3655935835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd338009ae0 con 0x7fd340159c90 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33ffff700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340159c90 0x7fd34015d690 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00c4e0 tx=0x7fd32c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33cff9700 1 -- 172.21.15.7:0/3655935835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c011590 con 0x7fd340159c90 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33cff9700 1 -- 172.21.15.7:0/3655935835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c011710 con 0x7fd340159c90 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd33cff9700 1 -- 172.21.15.7:0/3655935835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c011ad0 con 0x7fd340159c90 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34015e410 con 0x7fd340159c90 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.215+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34015e9d0 con 0x7fd340159c90 2024-01-11T07:11:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.217+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34004c3c0 con 0x7fd340159c90 2024-01-11T07:11:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.217+0000 7fd33cff9700 1 -- 172.21.15.7:0/3655935835 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd32c01b3b0 con 0x7fd340159c90 2024-01-11T07:11:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.218+0000 7fd33cff9700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd324066f30 0x7fd3240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.218+0000 7fd33cff9700 1 -- 172.21.15.7:0/3655935835 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd32c0838f0 con 0x7fd340159c90 2024-01-11T07:11:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.218+0000 7fd344bb7700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd324066f30 0x7fd3240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.219+0000 7fd344bb7700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd324066f30 0x7fd3240693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd334009db0 tx=0x7fd3340091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:58.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.224+0000 7fd33cff9700 1 -- 172.21.15.7:0/3655935835 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd32c050b60 con 0x7fd340159c90 2024-01-11T07:11:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.431+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd34015eb60 con 0x7fd340159c90 2024-01-11T07:11:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.431+0000 7fd33cff9700 1 -- 172.21.15.7:0/3655935835 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15307 (secure 0 0 0) 0x7fd32c017070 con 0x7fd340159c90 2024-01-11T07:11:58.431 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:11:58.431 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":100,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:11:58.216644+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:11:55.626974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-11T07:11:56.083170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:11:58.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd324066f30 msgr2=0x7fd3240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd324066f30 0x7fd3240693e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd334009db0 tx=0x7fd3340091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340159c90 msgr2=0x7fd34015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340159c90 0x7fd34015d690 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00c4e0 tx=0x7fd32c00c7f0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 shutdown_connections 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd324066f30 0x7fd3240693e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd34015a060 0x7fd34015fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd340069320 0x7fd340152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 --2- 172.21.15.7:0/3655935835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd340159c90 0x7fd34015d690 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 >> 172.21.15.7:0/3655935835 conn(0x7fd340071100 msgr2=0x7fd3401570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 shutdown_connections 2024-01-11T07:11:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.434+0000 7fd346e1b700 1 -- 172.21.15.7:0/3655935835 wait complete. 2024-01-11T07:11:58.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:11:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.583+0000 7f3de807e700 1 Processor -- start 2024-01-11T07:11:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.583+0000 7f3de807e700 1 -- start start 2024-01-11T07:11:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.584+0000 7f3de807e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3de00cbb00 0x7f3de0069280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.584+0000 7f3de807e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de00ce390 con 0x7f3de015f440 2024-01-11T07:11:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.584+0000 7f3de807e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de015efb0 con 0x7f3de006d340 2024-01-11T07:11:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.584+0000 7f3de807e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de00ce8b0 con 0x7f3de00cbb00 2024-01-11T07:11:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.584+0000 7f3de5619700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3de015f440 0x7f3de00cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33270/0 (socket says 172.21.15.7:33270) 2024-01-11T07:11:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.584+0000 7f3de5619700 1 -- 172.21.15.7:0/1516572315 learned_addr learned my addr 172.21.15.7:0/1516572315 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de661b700 1 --2- 172.21.15.7:0/1516572315 >> v2:172.21.15.113:3300/0 conn(0x7f3de00cbb00 0x7f3de0069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de4e18700 1 -- 172.21.15.7:0/1516572315 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292516754 0 0) 0x7f3ddc001110 con 0x7f3de015f440 2024-01-11T07:11:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de4e18700 1 -- 172.21.15.7:0/1516572315 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dcc003570 con 0x7f3de015f440 2024-01-11T07:11:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de4e18700 1 -- 172.21.15.7:0/1516572315 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709439603 0 0) 0x7f3de015efb0 con 0x7f3de006d340 2024-01-11T07:11:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de4e18700 1 -- 172.21.15.7:0/1516572315 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ddc001110 con 0x7f3de006d340 2024-01-11T07:11:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de661b700 1 -- 172.21.15.7:0/1516572315 >> v1:172.21.15.113:6789/0 conn(0x7f3de006d340 legacy=0x7f3de00cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de661b700 1 -- 172.21.15.7:0/1516572315 >> v1:172.21.15.7:6789/0 conn(0x7f3de015f440 legacy=0x7f3de00cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de661b700 1 -- 172.21.15.7:0/1516572315 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de006d710 con 0x7f3de00cbb00 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de661b700 1 --2- 172.21.15.7:0/1516572315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de00cbb00 0x7f3de0069280 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009e60 tx=0x7f3dd8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89bba29ad47468fb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de4e18700 1 -- 172.21.15.7:0/1516572315 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800e070 con 0x7f3de00cbb00 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de4e18700 1 -- 172.21.15.7:0/1516572315 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd800bab0 con 0x7f3de00cbb00 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.585+0000 7f3de4e18700 1 -- 172.21.15.7:0/1516572315 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800be70 con 0x7f3de00cbb00 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.586+0000 7f3de807e700 1 -- 172.21.15.7:0/1516572315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de00cbb00 msgr2=0x7f3de0069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.586+0000 7f3de807e700 1 --2- 172.21.15.7:0/1516572315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de00cbb00 0x7f3de0069280 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009e60 tx=0x7f3dd8009780 comp rx=0 tx=0).stop 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.586+0000 7f3de807e700 1 -- 172.21.15.7:0/1516572315 shutdown_connections 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.586+0000 7f3de807e700 1 --2- 172.21.15.7:0/1516572315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de00cbb00 0x7f3de0069280 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.586+0000 7f3de807e700 1 -- 172.21.15.7:0/1516572315 >> 172.21.15.7:0/1516572315 conn(0x7f3de006ae00 msgr2=0x7f3de006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.586+0000 7f3de807e700 1 -- 172.21.15.7:0/1516572315 shutdown_connections 2024-01-11T07:11:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.586+0000 7f3de807e700 1 -- 172.21.15.7:0/1516572315 wait complete. 2024-01-11T07:11:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.587+0000 7f3de807e700 1 Processor -- start 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.587+0000 7f3de807e700 1 -- start start 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de807e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3de006d340 0x7f3de014fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de807e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de00cbb00 0x7f3de0067e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de807e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de015f440 0x7f3de00683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de807e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0150280 con 0x7f3de00cbb00 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de807e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00ade40 con 0x7f3de015f440 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de807e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00b3cd0 con 0x7f3de006d340 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de5619700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de00cbb00 0x7f3de0067e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de5619700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de00cbb00 0x7f3de0067e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56830/0 (socket says 172.21.15.7:56830) 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de5619700 1 -- 172.21.15.7:0/277385990 learned_addr learned my addr 172.21.15.7:0/277385990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.588+0000 7f3de661b700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de015f440 0x7f3de00683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de5e1a700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3de006d340 0x7f3de014fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de5619700 1 -- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3de006d340 msgr2=0x7f3de014fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de5619700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3de006d340 0x7f3de014fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de5619700 1 -- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de015f440 msgr2=0x7f3de00683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de5619700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de015f440 0x7f3de00683b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de5619700 1 -- 172.21.15.7:0/277385990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8009a20 con 0x7f3de00cbb00 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de661b700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de015f440 0x7f3de00683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de5619700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de00cbb00 0x7f3de0067e70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c7c0 tx=0x7f3ddc007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3dd1ffb700 1 -- 172.21.15.7:0/277385990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc005b30 con 0x7f3de00cbb00 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3dd1ffb700 1 -- 172.21.15.7:0/277385990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc005cb0 con 0x7f3de00cbb00 2024-01-11T07:11:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de00b3fb0 con 0x7f3de00cbb00 2024-01-11T07:11:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3dd1ffb700 1 -- 172.21.15.7:0/277385990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc007340 con 0x7f3de00cbb00 2024-01-11T07:11:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.589+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de00b4570 con 0x7f3de00cbb00 2024-01-11T07:11:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.590+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de00adfd0 con 0x7f3de00cbb00 2024-01-11T07:11:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.591+0000 7f3dd1ffb700 1 -- 172.21.15.7:0/277385990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3ddc01b400 con 0x7f3de00cbb00 2024-01-11T07:11:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.591+0000 7f3dd1ffb700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3dc4066f30 0x7f3dc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.591+0000 7f3dd1ffb700 1 -- 172.21.15.7:0/277385990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3ddc083f40 con 0x7f3de00cbb00 2024-01-11T07:11:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.591+0000 7f3de5e1a700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3dc4066f30 0x7f3dc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.592+0000 7f3de5e1a700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3dc4066f30 0x7f3dc40693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4003870 tx=0x7f3dd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.596+0000 7f3dd1ffb700 1 -- 172.21.15.7:0/277385990 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ddc0511b0 con 0x7f3de00cbb00 2024-01-11T07:11:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:58.853+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3de00ce390 con 0x7f3de00cbb00 2024-01-11T07:11:59.665 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-11T07:11:59.666+0000 7f1966df0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:11:59.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.699+0000 7f3dd1ffb700 1 -- 172.21.15.7:0/277385990 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3de00ce390 con 0x7f3de00cbb00 2024-01-11T07:11:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3dc4066f30 msgr2=0x7f3dc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3dc4066f30 0x7f3dc40693e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4003870 tx=0x7f3dd4009040 comp rx=0 tx=0).stop 2024-01-11T07:11:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de00cbb00 msgr2=0x7f3de0067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de00cbb00 0x7f3de0067e70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c7c0 tx=0x7f3ddc007e40 comp rx=0 tx=0).stop 2024-01-11T07:11:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 shutdown_connections 2024-01-11T07:11:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3dc4066f30 0x7f3dc40693e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3de015f440 0x7f3de00683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3de006d340 0x7f3de014fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 --2- 172.21.15.7:0/277385990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de00cbb00 0x7f3de0067e70 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 >> 172.21.15.7:0/277385990 conn(0x7f3de006ae00 msgr2=0x7f3de00c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:59.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 shutdown_connections 2024-01-11T07:11:59.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.702+0000 7f3de807e700 1 -- 172.21.15.7:0/277385990 wait complete. 2024-01-11T07:11:59.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:11:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.857+0000 7fab09ed6700 1 Processor -- start 2024-01-11T07:11:59.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.857+0000 7fab09ed6700 1 -- start start 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab09ed6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab04159d70 0x7fab0415a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab09ed6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab040cf0d0 0x7fab040c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab09ed6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab04067840 con 0x7fab040cf540 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab09ed6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab040cb0c0 con 0x7fab040cf0d0 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab09ed6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab040ca820 con 0x7fab04159d70 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab02ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab040cf0d0 0x7fab040c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab03fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fab040cf540 0x7fab040c9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33284/0 (socket says 172.21.15.7:33284) 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab03fff700 1 -- 172.21.15.7:0/1610849919 learned_addr learned my addr 172.21.15.7:0/1610849919 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.858+0000 7fab02ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab040cf0d0 0x7fab040c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56832/0 (socket says 172.21.15.7:56832) 2024-01-11T07:11:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab037fe700 1 --2- 172.21.15.7:0/1610849919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab04159d70 0x7fab0415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab02ffd700 1 -- 172.21.15.7:0/1610849919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab04159d70 msgr2=0x7fab0415a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab02ffd700 1 --2- 172.21.15.7:0/1610849919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab04159d70 0x7fab0415a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab02ffd700 1 -- 172.21.15.7:0/1610849919 >> v1:172.21.15.7:6789/0 conn(0x7fab040cf540 legacy=0x7fab040c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab02ffd700 1 -- 172.21.15.7:0/1610849919 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040cab00 con 0x7fab040cf0d0 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab027fc700 1 -- 172.21.15.7:0/1610849919 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462471812 0 0) 0x7fab04067840 con 0x7fab040cf540 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab037fe700 1 --2- 172.21.15.7:0/1610849919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab04159d70 0x7fab0415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab02ffd700 1 --2- 172.21.15.7:0/1610849919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab040c98b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7faaf80098e0 tx=0x7faaf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe88294740b950f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab027fc700 1 -- 172.21.15.7:0/1610849919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8009e80 con 0x7fab040cf0d0 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab027fc700 1 -- 172.21.15.7:0/1610849919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf800b860 con 0x7fab040cf0d0 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.859+0000 7fab027fc700 1 -- 172.21.15.7:0/1610849919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf800bbe0 con 0x7fab040cf0d0 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.860+0000 7fab09ed6700 1 -- 172.21.15.7:0/1610849919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 msgr2=0x7fab040c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.860+0000 7fab09ed6700 1 --2- 172.21.15.7:0/1610849919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab040c98b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7faaf80098e0 tx=0x7faaf80091a0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.860+0000 7fab09ed6700 1 -- 172.21.15.7:0/1610849919 shutdown_connections 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.860+0000 7fab09ed6700 1 --2- 172.21.15.7:0/1610849919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab04159d70 0x7fab0415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.860+0000 7fab09ed6700 1 --2- 172.21.15.7:0/1610849919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab040c98b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.860+0000 7fab09ed6700 1 -- 172.21.15.7:0/1610849919 >> 172.21.15.7:0/1610849919 conn(0x7fab04097a50 msgr2=0x7fab040ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:11:59.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.860+0000 7fab09ed6700 1 -- 172.21.15.7:0/1610849919 shutdown_connections 2024-01-11T07:11:59.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.860+0000 7fab09ed6700 1 -- 172.21.15.7:0/1610849919 wait complete. 2024-01-11T07:11:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.861+0000 7fab09ed6700 1 Processor -- start 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.861+0000 7fab09ed6700 1 -- start start 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab09ed6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab0415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab09ed6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab040cf540 0x7fab040c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab09ed6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab04159d70 0x7fab04157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab09ed6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040b2130 con 0x7fab040cf0d0 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab09ed6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040c7d30 con 0x7fab04159d70 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab09ed6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab040c7b40 con 0x7fab040cf540 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab0415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab0415f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56836/0 (socket says 172.21.15.7:56836) 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab037fe700 1 -- 172.21.15.7:0/3168319352 learned_addr learned my addr 172.21.15.7:0/3168319352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab03fff700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab04159d70 0x7fab04157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab02ffd700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab040cf540 0x7fab040c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab037fe700 1 -- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab040cf540 msgr2=0x7fab040c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab037fe700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab040cf540 0x7fab040c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab037fe700 1 -- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab04159d70 msgr2=0x7fab04157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab037fe700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab04159d70 0x7fab04157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab037fe700 1 -- 172.21.15.7:0/3168319352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8009420 con 0x7fab040cf0d0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.862+0000 7fab03fff700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab04159d70 0x7fab04157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.863+0000 7fab037fe700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab0415f2c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7faaf400c880 tx=0x7faaf4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.863+0000 7faaeb7fe700 1 -- 172.21.15.7:0/3168319352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4005a00 con 0x7fab040cf0d0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.863+0000 7faaeb7fe700 1 -- 172.21.15.7:0/3168319352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf4005b60 con 0x7fab040cf0d0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.863+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab040b2c50 con 0x7fab040cf0d0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.863+0000 7faaeb7fe700 1 -- 172.21.15.7:0/3168319352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4011600 con 0x7fab040cf0d0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.863+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab040b2600 con 0x7fab040cf0d0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.864+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0404c3c0 con 0x7fab040cf0d0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.865+0000 7faaeb7fe700 1 -- 172.21.15.7:0/3168319352 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faaf40119f0 con 0x7fab040cf0d0 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.865+0000 7faaeb7fe700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faaf0066f30 0x7faaf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:11:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.865+0000 7faaeb7fe700 1 -- 172.21.15.7:0/3168319352 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faaf40843c0 con 0x7fab040cf0d0 2024-01-11T07:11:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.865+0000 7fab02ffd700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faaf0066f30 0x7faaf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:11:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.866+0000 7fab02ffd700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faaf0066f30 0x7faaf00693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab0415efe0 tx=0x7faaf8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:11:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:11:59.870+0000 7faaeb7fe700 1 -- 172.21.15.7:0/3168319352 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faaf4051630 con 0x7fab040cf0d0 2024-01-11T07:12:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.124+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fab040cade0 con 0x7fab040cf0d0 2024-01-11T07:12:00.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.705+0000 7faaeb7fe700 1 -- 172.21.15.7:0/3168319352 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7faaf4054c50 con 0x7fab040cf0d0 2024-01-11T07:12:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faaf0066f30 msgr2=0x7faaf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faaf0066f30 0x7faaf00693e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab0415efe0 tx=0x7faaf8017000 comp rx=0 tx=0).stop 2024-01-11T07:12:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 msgr2=0x7fab0415f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab0415f2c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7faaf400c880 tx=0x7faaf4008040 comp rx=0 tx=0).stop 2024-01-11T07:12:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 shutdown_connections 2024-01-11T07:12:00.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faaf0066f30 0x7faaf00693e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab04159d70 0x7fab04157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab040cf0d0 0x7fab0415f2c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 --2- 172.21.15.7:0/3168319352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab040cf540 0x7fab040c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 >> 172.21.15.7:0/3168319352 conn(0x7fab04097a50 msgr2=0x7fab0406bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 shutdown_connections 2024-01-11T07:12:00.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.708+0000 7fab09ed6700 1 -- 172.21.15.7:0/3168319352 wait complete. 2024-01-11T07:12:00.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:00.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.858+0000 7feb0bded700 1 Processor -- start 2024-01-11T07:12:00.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.859+0000 7feb0bded700 1 -- start start 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.859+0000 7feb0bded700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb040ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.859+0000 7feb0bded700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb040b0b40 0x7feb04161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0bded700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb04159300 con 0x7feb04158280 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0bded700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb040cf260 con 0x7feb040b0b40 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0bded700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb040cf540 con 0x7feb040cafb0 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0a38a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb040b0b40 0x7feb04161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0a38a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb040b0b40 0x7feb04161760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56840/0 (socket says 172.21.15.7:56840) 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0a38a700 1 -- 172.21.15.7:0/1823840207 learned_addr learned my addr 172.21.15.7:0/1823840207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb09b89700 1 --2- 172.21.15.7:0/1823840207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb040ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0a38a700 1 -- 172.21.15.7:0/1823840207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 msgr2=0x7feb040ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0a38a700 1 --2- 172.21.15.7:0/1823840207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb040ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0a38a700 1 -- 172.21.15.7:0/1823840207 >> v1:172.21.15.113:6789/0 conn(0x7feb04158280 legacy=0x7feb040b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0a38a700 1 -- 172.21.15.7:0/1823840207 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb040cf820 con 0x7feb040b0b40 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb0a38a700 1 --2- 172.21.15.7:0/1823840207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04161760 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feafc009f20 tx=0x7feafc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d1587aaecd4c3f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb08b87700 1 -- 172.21.15.7:0/1823840207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00e070 con 0x7feb040b0b40 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb08b87700 1 -- 172.21.15.7:0/1823840207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00ba80 con 0x7feb040b0b40 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.860+0000 7feb08b87700 1 -- 172.21.15.7:0/1823840207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0092e0 con 0x7feb040b0b40 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.861+0000 7feb0bded700 1 -- 172.21.15.7:0/1823840207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 msgr2=0x7feb04161760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.861+0000 7feb0bded700 1 --2- 172.21.15.7:0/1823840207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04161760 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feafc009f20 tx=0x7feafc009840 comp rx=0 tx=0).stop 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.861+0000 7feb0bded700 1 -- 172.21.15.7:0/1823840207 shutdown_connections 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.861+0000 7feb0bded700 1 --2- 172.21.15.7:0/1823840207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04161760 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.861+0000 7feb0bded700 1 --2- 172.21.15.7:0/1823840207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb040ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.861+0000 7feb0bded700 1 -- 172.21.15.7:0/1823840207 >> 172.21.15.7:0/1823840207 conn(0x7feb04071100 msgr2=0x7feb0406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.861+0000 7feb0bded700 1 -- 172.21.15.7:0/1823840207 shutdown_connections 2024-01-11T07:12:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.861+0000 7feb0bded700 1 -- 172.21.15.7:0/1823840207 wait complete. 2024-01-11T07:12:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.862+0000 7feb0bded700 1 Processor -- start 2024-01-11T07:12:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.862+0000 7feb0bded700 1 -- start start 2024-01-11T07:12:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb0bded700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04067610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb0bded700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb041544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb0bded700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb04158280 0x7feb041551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb0bded700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04067bf0 con 0x7feb040b0b40 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb0bded700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04067ed0 con 0x7feb04158280 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb0bded700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb040681b0 con 0x7feb040cafb0 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb09b89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb09b89700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04067610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56842/0 (socket says 172.21.15.7:56842) 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb09b89700 1 -- 172.21.15.7:0/3621597759 learned_addr learned my addr 172.21.15.7:0/3621597759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb0a38a700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb04158280 0x7feb041551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb09388700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb041544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.863+0000 7feb09b89700 1 -- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 msgr2=0x7feb041544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb09b89700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb041544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb09b89700 1 -- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb04158280 msgr2=0x7feb041551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb09b89700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb04158280 0x7feb041551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb09b89700 1 -- 172.21.15.7:0/3621597759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc009ae0 con 0x7feb040b0b40 2024-01-11T07:12:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb0a38a700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb04158280 0x7feb041551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb09388700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb041544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb09b89700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04067610 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d890 tx=0x7feb00008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feaf9ffb700 1 -- 172.21.15.7:0/3621597759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0000b600 con 0x7feb040b0b40 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feaf9ffb700 1 -- 172.21.15.7:0/3621597759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0000b760 con 0x7feb040b0b40 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb04068490 con 0x7feb040b0b40 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feaf9ffb700 1 -- 172.21.15.7:0/3621597759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0000bae0 con 0x7feb040b0b40 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.864+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb041556f0 con 0x7feb040b0b40 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.865+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0404c3c0 con 0x7feb040b0b40 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.865+0000 7feaf9ffb700 1 -- 172.21.15.7:0/3621597759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7feb0001e460 con 0x7feb040b0b40 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.866+0000 7feaf9ffb700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feae8066f30 0x7feae80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.866+0000 7feaf9ffb700 1 -- 172.21.15.7:0/3621597759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feb00084be0 con 0x7feb040b0b40 2024-01-11T07:12:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.866+0000 7feb09388700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feae8066f30 0x7feae80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.867+0000 7feb09388700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feae8066f30 0x7feae80693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feb04158650 tx=0x7feaf4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:00.871+0000 7feaf9ffb700 1 -- 172.21.15.7:0/3621597759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb00051e50 con 0x7feb040b0b40 2024-01-11T07:12:00.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:12:00.968 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:12:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.121+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb04159550 con 0x7feb040b0b40 2024-01-11T07:12:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.122+0000 7feaf9ffb700 1 -- 172.21.15.7:0/3621597759 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7feb00055470 con 0x7feb040b0b40 2024-01-11T07:12:01.121 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:01.121 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feae8066f30 msgr2=0x7feae80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feae8066f30 0x7feae80693e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feb04158650 tx=0x7feaf4009040 comp rx=0 tx=0).stop 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 msgr2=0x7feb04067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04067610 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d890 tx=0x7feb00008040 comp rx=0 tx=0).stop 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 shutdown_connections 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7feae8066f30 0x7feae80693e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb04158280 0x7feb041551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb040b0b40 0x7feb04067610 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 --2- 172.21.15.7:0/3621597759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feb040cafb0 0x7feb041544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 >> 172.21.15.7:0/3621597759 conn(0x7feb04071100 msgr2=0x7feb040ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 shutdown_connections 2024-01-11T07:12:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.124+0000 7feb0bded700 1 -- 172.21.15.7:0/3621597759 wait complete. 2024-01-11T07:12:01.125 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:01.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.269+0000 7fb3b45b8700 1 Processor -- start 2024-01-11T07:12:01.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.269+0000 7fb3b45b8700 1 -- start start 2024-01-11T07:12:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b45b8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb3ac159c90 0x7fb3ac0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b45b8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac0ce9b0 con 0x7fb3ac069320 2024-01-11T07:12:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b45b8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac0cf530 con 0x7fb3ac15a060 2024-01-11T07:12:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b45b8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ac0cf9c0 con 0x7fb3ac159c90 2024-01-11T07:12:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b2354700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb3ac159c90 0x7fb3ac0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b2b55700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb3ac069320 0x7fb3ac0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33290/0 (socket says 172.21.15.7:33290) 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b2354700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb3ac159c90 0x7fb3ac0b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56844/0 (socket says 172.21.15.7:56844) 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b2b55700 1 -- 172.21.15.7:0/4056405407 learned_addr learned my addr 172.21.15.7:0/4056405407 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b2354700 1 -- 172.21.15.7:0/4056405407 >> v1:172.21.15.113:6789/0 conn(0x7fb3ac15a060 legacy=0x7fb3ac0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b2354700 1 -- 172.21.15.7:0/4056405407 >> v1:172.21.15.7:6789/0 conn(0x7fb3ac069320 legacy=0x7fb3ac0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b1352700 1 -- 172.21.15.7:0/4056405407 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817515694 0 0) 0x7fb3ac0ce9b0 con 0x7fb3ac069320 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b2354700 1 -- 172.21.15.7:0/4056405407 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0cec00 con 0x7fb3ac159c90 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.270+0000 7fb3b2354700 1 --2- 172.21.15.7:0/4056405407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac159c90 0x7fb3ac0b1820 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00098e0 tx=0x7fb3a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40eec83ef5b542a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b1352700 1 -- 172.21.15.7:0/4056405407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0009e80 con 0x7fb3ac159c90 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b1352700 1 -- 172.21.15.7:0/4056405407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a000b860 con 0x7fb3ac159c90 2024-01-11T07:12:01.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b1352700 1 -- 172.21.15.7:0/4056405407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000bbe0 con 0x7fb3ac159c90 2024-01-11T07:12:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b45b8700 1 -- 172.21.15.7:0/4056405407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac159c90 msgr2=0x7fb3ac0b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b45b8700 1 --2- 172.21.15.7:0/4056405407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac159c90 0x7fb3ac0b1820 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00098e0 tx=0x7fb3a00091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b45b8700 1 -- 172.21.15.7:0/4056405407 shutdown_connections 2024-01-11T07:12:01.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b45b8700 1 --2- 172.21.15.7:0/4056405407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac159c90 0x7fb3ac0b1820 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b45b8700 1 -- 172.21.15.7:0/4056405407 >> 172.21.15.7:0/4056405407 conn(0x7fb3ac071100 msgr2=0x7fb3ac06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b45b8700 1 -- 172.21.15.7:0/4056405407 shutdown_connections 2024-01-11T07:12:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.271+0000 7fb3b45b8700 1 -- 172.21.15.7:0/4056405407 wait complete. 2024-01-11T07:12:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b45b8700 1 Processor -- start 2024-01-11T07:12:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b45b8700 1 -- start start 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b45b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac069320 0x7fb3ac152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b45b8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac159c90 0x7fb3ac0c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b45b8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ac15a060 0x7fb3ac0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b45b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac153480 con 0x7fb3ac069320 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b45b8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac153640 con 0x7fb3ac159c90 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b45b8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac154930 con 0x7fb3ac15a060 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.273+0000 7fb3b2354700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac069320 0x7fb3ac152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2354700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac069320 0x7fb3ac152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56850/0 (socket says 172.21.15.7:56850) 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2354700 1 -- 172.21.15.7:0/2525075836 learned_addr learned my addr 172.21.15.7:0/2525075836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b1b53700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac159c90 0x7fb3ac0c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2b55700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ac15a060 0x7fb3ac0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2354700 1 -- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ac15a060 msgr2=0x7fb3ac0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2354700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ac15a060 0x7fb3ac0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2354700 1 -- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac159c90 msgr2=0x7fb3ac0c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2354700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac159c90 0x7fb3ac0c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2354700 1 -- 172.21.15.7:0/2525075836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a0009420 con 0x7fb3ac069320 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b2354700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac069320 0x7fb3ac152f40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac06a1d0 tx=0x7fb3a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb39e7fc700 1 -- 172.21.15.7:0/2525075836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a001f070 con 0x7fb3ac069320 2024-01-11T07:12:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb39e7fc700 1 -- 172.21.15.7:0/2525075836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a0011420 con 0x7fb3ac069320 2024-01-11T07:12:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac153890 con 0x7fb3ac069320 2024-01-11T07:12:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb39e7fc700 1 -- 172.21.15.7:0/2525075836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000bbe0 con 0x7fb3ac069320 2024-01-11T07:12:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.274+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac153e50 con 0x7fb3ac069320 2024-01-11T07:12:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.275+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac04c3c0 con 0x7fb3ac069320 2024-01-11T07:12:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.276+0000 7fb39e7fc700 1 -- 172.21.15.7:0/2525075836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb3a0011a50 con 0x7fb3ac069320 2024-01-11T07:12:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.276+0000 7fb39e7fc700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb390066f30 0x7fb3900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.276+0000 7fb39e7fc700 1 -- 172.21.15.7:0/2525075836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3a0085a90 con 0x7fb3ac069320 2024-01-11T07:12:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.277+0000 7fb3b1b53700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb390066f30 0x7fb3900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.277+0000 7fb3b1b53700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb390066f30 0x7fb3900693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ce670 tx=0x7fb3a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:01.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.281+0000 7fb39e7fc700 1 -- 172.21.15.7:0/2525075836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3a0052d00 con 0x7fb3ac069320 2024-01-11T07:12:01.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.486+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3ac15e6a0 con 0x7fb3ac069320 2024-01-11T07:12:01.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.487+0000 7fb39e7fc700 1 -- 172.21.15.7:0/2525075836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15307 (secure 0 0 0) 0x7fb3a0015090 con 0x7fb3ac069320 2024-01-11T07:12:01.487 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:01.487 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":100,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:11:58.216644+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:11:55.626974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-11T07:11:56.083170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:01.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb390066f30 msgr2=0x7fb3900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb390066f30 0x7fb3900693e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ce670 tx=0x7fb3a8009040 comp rx=0 tx=0).stop 2024-01-11T07:12:01.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac069320 msgr2=0x7fb3ac152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac069320 0x7fb3ac152f40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac06a1d0 tx=0x7fb3a0017000 comp rx=0 tx=0).stop 2024-01-11T07:12:01.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 shutdown_connections 2024-01-11T07:12:01.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb390066f30 0x7fb3900693e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3ac15a060 0x7fb3ac0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3ac069320 0x7fb3ac152f40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 --2- 172.21.15.7:0/2525075836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac159c90 0x7fb3ac0c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 >> 172.21.15.7:0/2525075836 conn(0x7fb3ac071100 msgr2=0x7fb3ac1552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:01.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 shutdown_connections 2024-01-11T07:12:01.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.489+0000 7fb3b45b8700 1 -- 172.21.15.7:0/2525075836 wait complete. 2024-01-11T07:12:01.502 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:12:01.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:12:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.635+0000 7f00ab830700 1 Processor -- start 2024-01-11T07:12:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.635+0000 7f00ab830700 1 -- start start 2024-01-11T07:12:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00ab830700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00a40c0a70 0x7f00a4161da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00ab830700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a4063940 0x7f00a40c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00ab830700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a40c83f0 con 0x7f00a41622e0 2024-01-11T07:12:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00ab830700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a415e740 con 0x7f00a40c0a70 2024-01-11T07:12:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00ab830700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4162780 con 0x7f00a4063940 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00a95cc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00a40c0a70 0x7f00a4161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00a95cc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00a40c0a70 0x7f00a4161da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56862/0 (socket says 172.21.15.7:56862) 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00a8dcb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f00a41622e0 0x7f00a4063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33294/0 (socket says 172.21.15.7:33294) 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00a95cc700 1 -- 172.21.15.7:0/3041363287 learned_addr learned my addr 172.21.15.7:0/3041363287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.636+0000 7f00a9dcd700 1 --2- 172.21.15.7:0/3041363287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a4063940 0x7f00a40c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f00a95cc700 1 -- 172.21.15.7:0/3041363287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a4063940 msgr2=0x7f00a40c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f00a95cc700 1 --2- 172.21.15.7:0/3041363287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a4063940 0x7f00a40c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f00a95cc700 1 -- 172.21.15.7:0/3041363287 >> v1:172.21.15.7:6789/0 conn(0x7f00a41622e0 legacy=0x7f00a4063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f00a95cc700 1 -- 172.21.15.7:0/3041363287 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a4162a60 con 0x7f00a40c0a70 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f009bfff700 1 -- 172.21.15.7:0/3041363287 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412738731 0 0) 0x7f00a40c83f0 con 0x7f00a41622e0 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f00a95cc700 1 --2- 172.21.15.7:0/3041363287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a4161da0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f00a00099b0 tx=0x7f00a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99a5608e0a6e94c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f009bfff700 1 -- 172.21.15.7:0/3041363287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000b860 con 0x7f00a40c0a70 2024-01-11T07:12:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f009bfff700 1 -- 172.21.15.7:0/3041363287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000b9c0 con 0x7f00a40c0a70 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f009bfff700 1 -- 172.21.15.7:0/3041363287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000bd60 con 0x7f00a40c0a70 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f00ab830700 1 -- 172.21.15.7:0/3041363287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 msgr2=0x7f00a4161da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.637+0000 7f00ab830700 1 --2- 172.21.15.7:0/3041363287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a4161da0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f00a00099b0 tx=0x7f00a0009250 comp rx=0 tx=0).stop 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.638+0000 7f00ab830700 1 -- 172.21.15.7:0/3041363287 shutdown_connections 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.638+0000 7f00ab830700 1 --2- 172.21.15.7:0/3041363287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a4063940 0x7f00a40c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.638+0000 7f00ab830700 1 --2- 172.21.15.7:0/3041363287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a4161da0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.638+0000 7f00ab830700 1 -- 172.21.15.7:0/3041363287 >> 172.21.15.7:0/3041363287 conn(0x7f00a40680e0 msgr2=0x7f00a40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.638+0000 7f00ab830700 1 -- 172.21.15.7:0/3041363287 shutdown_connections 2024-01-11T07:12:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.638+0000 7f00ab830700 1 -- 172.21.15.7:0/3041363287 wait complete. 2024-01-11T07:12:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.639+0000 7f00ab830700 1 Processor -- start 2024-01-11T07:12:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.639+0000 7f00ab830700 1 -- start start 2024-01-11T07:12:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00ab830700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a4063940 0x7f00a414b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00ab830700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a40c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00ab830700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a41622e0 0x7f00a40ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00ab830700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40b1b30 con 0x7f00a40c0a70 2024-01-11T07:12:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00ab830700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40b0f20 con 0x7f00a4063940 2024-01-11T07:12:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00ab830700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40b1200 con 0x7f00a41622e0 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00a8dcb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a40c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00a8dcb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a40c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56876/0 (socket says 172.21.15.7:56876) 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00a8dcb700 1 -- 172.21.15.7:0/2404422463 learned_addr learned my addr 172.21.15.7:0/2404422463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00a9dcd700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a41622e0 0x7f00a40ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.640+0000 7f00a95cc700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a4063940 0x7f00a414b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00a8dcb700 1 -- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a41622e0 msgr2=0x7f00a40ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00a8dcb700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a41622e0 0x7f00a40ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00a8dcb700 1 -- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a4063940 msgr2=0x7f00a414b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00a8dcb700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a4063940 0x7f00a414b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00a8dcb700 1 -- 172.21.15.7:0/2404422463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a00094f0 con 0x7f00a40c0a70 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00a8dcb700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a40c6a80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f009400c5b0 tx=0x7f0094001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00997fa700 1 -- 172.21.15.7:0/2404422463 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094005c10 con 0x7f00a40c0a70 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00997fa700 1 -- 172.21.15.7:0/2404422463 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0094005d70 con 0x7f00a40c0a70 2024-01-11T07:12:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a40b14e0 con 0x7f00a40c0a70 2024-01-11T07:12:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00997fa700 1 -- 172.21.15.7:0/2404422463 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094007300 con 0x7f00a40c0a70 2024-01-11T07:12:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.641+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a40b0b60 con 0x7f00a40c0a70 2024-01-11T07:12:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.642+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a404c3c0 con 0x7f00a40c0a70 2024-01-11T07:12:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.642+0000 7f00997fa700 1 -- 172.21.15.7:0/2404422463 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f009401b3b0 con 0x7f00a40c0a70 2024-01-11T07:12:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.643+0000 7f00997fa700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0088066f30 0x7f00880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.643+0000 7f00997fa700 1 -- 172.21.15.7:0/2404422463 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0094083c30 con 0x7f00a40c0a70 2024-01-11T07:12:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.643+0000 7f00a95cc700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0088066f30 0x7f00880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.644+0000 7f00a95cc700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0088066f30 0x7f00880693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f00a00099b0 tx=0x7f00a0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.647+0000 7f00997fa700 1 -- 172.21.15.7:0/2404422463 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0094050e00 con 0x7f00a40c0a70 2024-01-11T07:12:01.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.846+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00a40c8080 con 0x7f0088066f30 2024-01-11T07:12:01.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.855+0000 7f00997fa700 1 -- 172.21.15.7:0/2404422463 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285271 (secure 0 0 0) 0x7f00a40c8080 con 0x7f0088066f30 2024-01-11T07:12:01.859 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0088066f30 msgr2=0x7f00880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0088066f30 0x7f00880693e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f00a00099b0 tx=0x7f00a0017000 comp rx=0 tx=0).stop 2024-01-11T07:12:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 msgr2=0x7f00a40c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a40c6a80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f009400c5b0 tx=0x7f0094001040 comp rx=0 tx=0).stop 2024-01-11T07:12:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 shutdown_connections 2024-01-11T07:12:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0088066f30 0x7f00880693e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00a41622e0 0x7f00a40ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a4063940 0x7f00a414b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.858+0000 7f00ab830700 1 --2- 172.21.15.7:0/2404422463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a40c0a70 0x7f00a40c6a80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.859+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 >> 172.21.15.7:0/2404422463 conn(0x7f00a40680e0 msgr2=0x7f00a415dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.859+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 shutdown_connections 2024-01-11T07:12:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:01.859+0000 7f00ab830700 1 -- 172.21.15.7:0/2404422463 wait complete. 2024-01-11T07:12:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:12:01.872 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-01-11T07:11:59.910896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5328,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557171,"statfs":{"total":1600000000,"available":1594543916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.235371"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228186+0000","last_change":"2024-01-11T07:11:57.228025+0000","last_active":"2024-01-11T07:11:57.228186+0000","last_peered":"2024-01-11T07:11:57.228186+0000","last_clean":"2024-01-11T07:11:57.228186+0000","last_became_active":"2024-01-11T07:11:57.227653+0000","last_became_peered":"2024-01-11T07:11:57.227653+0000","last_unstale":"2024-01-11T07:11:57.228186+0000","last_undegraded":"2024-01-11T07:11:57.228186+0000","last_fullsized":"2024-01-11T07:11:57.228186+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:56.072710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.235351+0000","last_change":"2024-01-11T07:11:57.235104+0000","last_active":"2024-01-11T07:11:57.235351+0000","last_peered":"2024-01-11T07:11:57.235351+0000","last_clean":"2024-01-11T07:11:57.235351+0000","last_became_active":"2024-01-11T07:11:57.234485+0000","last_became_peered":"2024-01-11T07:11:57.234485+0000","last_unstale":"2024-01-11T07:11:57.235351+0000","last_undegraded":"2024-01-11T07:11:57.235351+0000","last_fullsized":"2024-01-11T07:11:57.235351+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:32:05.990591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231770+0000","last_change":"2024-01-11T07:11:57.231674+0000","last_active":"2024-01-11T07:11:57.231770+0000","last_peered":"2024-01-11T07:11:57.231770+0000","last_clean":"2024-01-11T07:11:57.231770+0000","last_became_active":"2024-01-11T07:11:57.231368+0000","last_became_peered":"2024-01-11T07:11:57.231368+0000","last_unstale":"2024-01-11T07:11:57.231770+0000","last_undegraded":"2024-01-11T07:11:57.231770+0000","last_fullsized":"2024-01-11T07:11:57.231770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:30:45.036920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232696+0000","last_change":"2024-01-11T07:11:57.232599+0000","last_active":"2024-01-11T07:11:57.232696+0000","last_peered":"2024-01-11T07:11:57.232696+0000","last_clean":"2024-01-11T07:11:57.232696+0000","last_became_active":"2024-01-11T07:11:57.232004+0000","last_became_peered":"2024-01-11T07:11:57.232004+0000","last_unstale":"2024-01-11T07:11:57.232696+0000","last_undegraded":"2024-01-11T07:11:57.232696+0000","last_fullsized":"2024-01-11T07:11:57.232696+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:40:23.300549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.230837+0000","last_change":"2024-01-11T07:11:57.230755+0000","last_active":"2024-01-11T07:11:57.230837+0000","last_peered":"2024-01-11T07:11:57.230837+0000","last_clean":"2024-01-11T07:11:57.230837+0000","last_became_active":"2024-01-11T07:11:57.230558+0000","last_became_peered":"2024-01-11T07:11:57.230558+0000","last_unstale":"2024-01-11T07:11:57.230837+0000","last_undegraded":"2024-01-11T07:11:57.230837+0000","last_fullsized":"2024-01-11T07:11:57.230837+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:17.294168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234873+0000","last_change":"2024-01-11T07:11:57.234804+0000","last_active":"2024-01-11T07:11:57.234873+0000","last_peered":"2024-01-11T07:11:57.234873+0000","last_clean":"2024-01-11T07:11:57.234873+0000","last_became_active":"2024-01-11T07:11:57.234563+0000","last_became_peered":"2024-01-11T07:11:57.234563+0000","last_unstale":"2024-01-11T07:11:57.234873+0000","last_undegraded":"2024-01-11T07:11:57.234873+0000","last_fullsized":"2024-01-11T07:11:57.234873+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:17.294168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.236300+0000","last_change":"2024-01-11T07:11:57.236250+0000","last_active":"2024-01-11T07:11:57.236300+0000","last_peered":"2024-01-11T07:11:57.236300+0000","last_clean":"2024-01-11T07:11:57.236300+0000","last_became_active":"2024-01-11T07:11:57.236062+0000","last_became_peered":"2024-01-11T07:11:57.236062+0000","last_unstale":"2024-01-11T07:11:57.236300+0000","last_undegraded":"2024-01-11T07:11:57.236300+0000","last_fullsized":"2024-01-11T07:11:57.236300+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:51:41.103684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.229750+0000","last_change":"2024-01-11T07:11:57.229669+0000","last_active":"2024-01-11T07:11:57.229750+0000","last_peered":"2024-01-11T07:11:57.229750+0000","last_clean":"2024-01-11T07:11:57.229750+0000","last_became_active":"2024-01-11T07:11:57.229257+0000","last_became_peered":"2024-01-11T07:11:57.229257+0000","last_unstale":"2024-01-11T07:11:57.229750+0000","last_undegraded":"2024-01-11T07:11:57.229750+0000","last_fullsized":"2024-01-11T07:11:57.229750+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:27.939787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225204+0000","last_change":"2024-01-11T07:11:57.225129+0000","last_active":"2024-01-11T07:11:57.225204+0000","last_peered":"2024-01-11T07:11:57.225204+0000","last_clean":"2024-01-11T07:11:57.225204+0000","last_became_active":"2024-01-11T07:11:57.224889+0000","last_became_peered":"2024-01-11T07:11:57.224889+0000","last_unstale":"2024-01-11T07:11:57.225204+0000","last_undegraded":"2024-01-11T07:11:57.225204+0000","last_fullsized":"2024-01-11T07:11:57.225204+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:56.072710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225444+0000","last_change":"2024-01-11T07:11:57.225321+0000","last_active":"2024-01-11T07:11:57.225444+0000","last_peered":"2024-01-11T07:11:57.225444+0000","last_clean":"2024-01-11T07:11:57.225444+0000","last_became_active":"2024-01-11T07:11:57.225088+0000","last_became_peered":"2024-01-11T07:11:57.225088+0000","last_unstale":"2024-01-11T07:11:57.225444+0000","last_undegraded":"2024-01-11T07:11:57.225444+0000","last_fullsized":"2024-01-11T07:11:57.225444+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:37:32.425877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234258+0000","last_change":"2024-01-11T07:11:57.234016+0000","last_active":"2024-01-11T07:11:57.234258+0000","last_peered":"2024-01-11T07:11:57.234258+0000","last_clean":"2024-01-11T07:11:57.234258+0000","last_became_active":"2024-01-11T07:11:57.233772+0000","last_became_peered":"2024-01-11T07:11:57.233772+0000","last_unstale":"2024-01-11T07:11:57.234258+0000","last_undegraded":"2024-01-11T07:11:57.234258+0000","last_fullsized":"2024-01-11T07:11:57.234258+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:43:25.351251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226370+0000","last_change":"2024-01-11T07:11:57.226267+0000","last_active":"2024-01-11T07:11:57.226370+0000","last_peered":"2024-01-11T07:11:57.226370+0000","last_clean":"2024-01-11T07:11:57.226370+0000","last_became_active":"2024-01-11T07:11:57.226026+0000","last_became_peered":"2024-01-11T07:11:57.226026+0000","last_unstale":"2024-01-11T07:11:57.226370+0000","last_undegraded":"2024-01-11T07:11:57.226370+0000","last_fullsized":"2024-01-11T07:11:57.226370+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:05:00.875268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232310+0000","last_change":"2024-01-11T07:11:57.232226+0000","last_active":"2024-01-11T07:11:57.232310+0000","last_peered":"2024-01-11T07:11:57.232310+0000","last_clean":"2024-01-11T07:11:57.232310+0000","last_became_active":"2024-01-11T07:11:57.232064+0000","last_became_peered":"2024-01-11T07:11:57.232064+0000","last_unstale":"2024-01-11T07:11:57.232310+0000","last_undegraded":"2024-01-11T07:11:57.232310+0000","last_fullsized":"2024-01-11T07:11:57.232310+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:01:12.447452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232680+0000","last_change":"2024-01-11T07:11:57.232432+0000","last_active":"2024-01-11T07:11:57.232680+0000","last_peered":"2024-01-11T07:11:57.232680+0000","last_clean":"2024-01-11T07:11:57.232680+0000","last_became_active":"2024-01-11T07:11:57.231656+0000","last_became_peered":"2024-01-11T07:11:57.231656+0000","last_unstale":"2024-01-11T07:11:57.232680+0000","last_undegraded":"2024-01-11T07:11:57.232680+0000","last_fullsized":"2024-01-11T07:11:57.232680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:05:49.938435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232458+0000","last_change":"2024-01-11T07:11:57.232393+0000","last_active":"2024-01-11T07:11:57.232458+0000","last_peered":"2024-01-11T07:11:57.232458+0000","last_clean":"2024-01-11T07:11:57.232458+0000","last_became_active":"2024-01-11T07:11:57.232156+0000","last_became_peered":"2024-01-11T07:11:57.232156+0000","last_unstale":"2024-01-11T07:11:57.232458+0000","last_undegraded":"2024-01-11T07:11:57.232458+0000","last_fullsized":"2024-01-11T07:11:57.232458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:48.096243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228322+0000","last_change":"2024-01-11T07:11:57.227100+0000","last_active":"2024-01-11T07:11:57.228322+0000","last_peered":"2024-01-11T07:11:57.228322+0000","last_clean":"2024-01-11T07:11:57.228322+0000","last_became_active":"2024-01-11T07:11:57.226772+0000","last_became_peered":"2024-01-11T07:11:57.226772+0000","last_unstale":"2024-01-11T07:11:57.228322+0000","last_undegraded":"2024-01-11T07:11:57.228322+0000","last_fullsized":"2024-01-11T07:11:57.228322+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:10:40.115835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.237511+0000","last_change":"2024-01-11T07:11:57.237416+0000","last_active":"2024-01-11T07:11:57.237511+0000","last_peered":"2024-01-11T07:11:57.237511+0000","last_clean":"2024-01-11T07:11:57.237511+0000","last_became_active":"2024-01-11T07:11:57.236704+0000","last_became_peered":"2024-01-11T07:11:57.236704+0000","last_unstale":"2024-01-11T07:11:57.237511+0000","last_undegraded":"2024-01-11T07:11:57.237511+0000","last_fullsized":"2024-01-11T07:11:57.237511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:01:12.447452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231088+0000","last_change":"2024-01-11T07:11:57.230826+0000","last_active":"2024-01-11T07:11:57.231088+0000","last_peered":"2024-01-11T07:11:57.231088+0000","last_clean":"2024-01-11T07:11:57.231088+0000","last_became_active":"2024-01-11T07:11:57.230459+0000","last_became_peered":"2024-01-11T07:11:57.230459+0000","last_unstale":"2024-01-11T07:11:57.231088+0000","last_undegraded":"2024-01-11T07:11:57.231088+0000","last_fullsized":"2024-01-11T07:11:57.231088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:43:45.702913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221762+0000","last_change":"2024-01-11T07:11:57.221686+0000","last_active":"2024-01-11T07:11:57.221762+0000","last_peered":"2024-01-11T07:11:57.221762+0000","last_clean":"2024-01-11T07:11:57.221762+0000","last_became_active":"2024-01-11T07:11:57.221369+0000","last_became_peered":"2024-01-11T07:11:57.221369+0000","last_unstale":"2024-01-11T07:11:57.221762+0000","last_undegraded":"2024-01-11T07:11:57.221762+0000","last_fullsized":"2024-01-11T07:11:57.221762+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:16.259881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222107+0000","last_change":"2024-01-11T07:11:57.222052+0000","last_active":"2024-01-11T07:11:57.222107+0000","last_peered":"2024-01-11T07:11:57.222107+0000","last_clean":"2024-01-11T07:11:57.222107+0000","last_became_active":"2024-01-11T07:11:57.221633+0000","last_became_peered":"2024-01-11T07:11:57.221633+0000","last_unstale":"2024-01-11T07:11:57.222107+0000","last_undegraded":"2024-01-11T07:11:57.222107+0000","last_fullsized":"2024-01-11T07:11:57.222107+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:46:20.345152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222089+0000","last_change":"2024-01-11T07:11:57.222003+0000","last_active":"2024-01-11T07:11:57.222089+0000","last_peered":"2024-01-11T07:11:57.222089+0000","last_clean":"2024-01-11T07:11:57.222089+0000","last_became_active":"2024-01-11T07:11:57.221785+0000","last_became_peered":"2024-01-11T07:11:57.221785+0000","last_unstale":"2024-01-11T07:11:57.222089+0000","last_undegraded":"2024-01-11T07:11:57.222089+0000","last_fullsized":"2024-01-11T07:11:57.222089+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:54:32.751965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231448+0000","last_change":"2024-01-11T07:11:57.231387+0000","last_active":"2024-01-11T07:11:57.231448+0000","last_peered":"2024-01-11T07:11:57.231448+0000","last_clean":"2024-01-11T07:11:57.231448+0000","last_became_active":"2024-01-11T07:11:57.230985+0000","last_became_peered":"2024-01-11T07:11:57.230985+0000","last_unstale":"2024-01-11T07:11:57.231448+0000","last_undegraded":"2024-01-11T07:11:57.231448+0000","last_fullsized":"2024-01-11T07:11:57.231448+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:19:13.036740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231512+0000","last_change":"2024-01-11T07:11:57.228276+0000","last_active":"2024-01-11T07:11:57.231512+0000","last_peered":"2024-01-11T07:11:57.231512+0000","last_clean":"2024-01-11T07:11:57.231512+0000","last_became_active":"2024-01-11T07:11:57.227438+0000","last_became_peered":"2024-01-11T07:11:57.227438+0000","last_unstale":"2024-01-11T07:11:57.231512+0000","last_undegraded":"2024-01-11T07:11:57.231512+0000","last_fullsized":"2024-01-11T07:11:57.231512+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:32:01.689019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221862+0000","last_change":"2024-01-11T07:11:57.221595+0000","last_active":"2024-01-11T07:11:57.221862+0000","last_peered":"2024-01-11T07:11:57.221862+0000","last_clean":"2024-01-11T07:11:57.221862+0000","last_became_active":"2024-01-11T07:11:57.221350+0000","last_became_peered":"2024-01-11T07:11:57.221350+0000","last_unstale":"2024-01-11T07:11:57.221862+0000","last_undegraded":"2024-01-11T07:11:57.221862+0000","last_fullsized":"2024-01-11T07:11:57.221862+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:00:08.312302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.230550+0000","last_change":"2024-01-11T07:11:57.229468+0000","last_active":"2024-01-11T07:11:57.230550+0000","last_peered":"2024-01-11T07:11:57.230550+0000","last_clean":"2024-01-11T07:11:57.230550+0000","last_became_active":"2024-01-11T07:11:57.229247+0000","last_became_peered":"2024-01-11T07:11:57.229247+0000","last_unstale":"2024-01-11T07:11:57.230550+0000","last_undegraded":"2024-01-11T07:11:57.230550+0000","last_fullsized":"2024-01-11T07:11:57.230550+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:51:06.947140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234296+0000","last_change":"2024-01-11T07:11:57.234223+0000","last_active":"2024-01-11T07:11:57.234296+0000","last_peered":"2024-01-11T07:11:57.234296+0000","last_clean":"2024-01-11T07:11:57.234296+0000","last_became_active":"2024-01-11T07:11:57.233973+0000","last_became_peered":"2024-01-11T07:11:57.233973+0000","last_unstale":"2024-01-11T07:11:57.234296+0000","last_undegraded":"2024-01-11T07:11:57.234296+0000","last_fullsized":"2024-01-11T07:11:57.234296+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:29:42.362997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.237668+0000","last_change":"2024-01-11T07:11:57.237597+0000","last_active":"2024-01-11T07:11:57.237668+0000","last_peered":"2024-01-11T07:11:57.237668+0000","last_clean":"2024-01-11T07:11:57.237668+0000","last_became_active":"2024-01-11T07:11:57.237030+0000","last_became_peered":"2024-01-11T07:11:57.237030+0000","last_unstale":"2024-01-11T07:11:57.237668+0000","last_undegraded":"2024-01-11T07:11:57.237668+0000","last_fullsized":"2024-01-11T07:11:57.237668+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:29:43.771375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231557+0000","last_change":"2024-01-11T07:11:57.229330+0000","last_active":"2024-01-11T07:11:57.231557+0000","last_peered":"2024-01-11T07:11:57.231557+0000","last_clean":"2024-01-11T07:11:57.231557+0000","last_became_active":"2024-01-11T07:11:57.229085+0000","last_became_peered":"2024-01-11T07:11:57.229085+0000","last_unstale":"2024-01-11T07:11:57.231557+0000","last_undegraded":"2024-01-11T07:11:57.231557+0000","last_fullsized":"2024-01-11T07:11:57.231557+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:09:10.736111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.230527+0000","last_change":"2024-01-11T07:11:57.228700+0000","last_active":"2024-01-11T07:11:57.230527+0000","last_peered":"2024-01-11T07:11:57.230527+0000","last_clean":"2024-01-11T07:11:57.230527+0000","last_became_active":"2024-01-11T07:11:57.228468+0000","last_became_peered":"2024-01-11T07:11:57.228468+0000","last_unstale":"2024-01-11T07:11:57.230527+0000","last_undegraded":"2024-01-11T07:11:57.230527+0000","last_fullsized":"2024-01-11T07:11:57.230527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:38:49.807723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232939+0000","last_change":"2024-01-11T07:11:57.232884+0000","last_active":"2024-01-11T07:11:57.232939+0000","last_peered":"2024-01-11T07:11:57.232939+0000","last_clean":"2024-01-11T07:11:57.232939+0000","last_became_active":"2024-01-11T07:11:57.232455+0000","last_became_peered":"2024-01-11T07:11:57.232455+0000","last_unstale":"2024-01-11T07:11:57.232939+0000","last_undegraded":"2024-01-11T07:11:57.232939+0000","last_fullsized":"2024-01-11T07:11:57.232939+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:00:27.293551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226667+0000","last_change":"2024-01-11T07:11:57.226030+0000","last_active":"2024-01-11T07:11:57.226667+0000","last_peered":"2024-01-11T07:11:57.226667+0000","last_clean":"2024-01-11T07:11:57.226667+0000","last_became_active":"2024-01-11T07:11:57.225734+0000","last_became_peered":"2024-01-11T07:11:57.225734+0000","last_unstale":"2024-01-11T07:11:57.226667+0000","last_undegraded":"2024-01-11T07:11:57.226667+0000","last_fullsized":"2024-01-11T07:11:57.226667+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:50.009991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231598+0000","last_change":"2024-01-11T07:11:57.231472+0000","last_active":"2024-01-11T07:11:57.231598+0000","last_peered":"2024-01-11T07:11:57.231598+0000","last_clean":"2024-01-11T07:11:57.231598+0000","last_became_active":"2024-01-11T07:11:57.231218+0000","last_became_peered":"2024-01-11T07:11:57.231218+0000","last_unstale":"2024-01-11T07:11:57.231598+0000","last_undegraded":"2024-01-11T07:11:57.231598+0000","last_fullsized":"2024-01-11T07:11:57.231598+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:40:17.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228728+0000","last_change":"2024-01-11T07:11:57.228659+0000","last_active":"2024-01-11T07:11:57.228728+0000","last_peered":"2024-01-11T07:11:57.228728+0000","last_clean":"2024-01-11T07:11:57.228728+0000","last_became_active":"2024-01-11T07:11:57.228406+0000","last_became_peered":"2024-01-11T07:11:57.228406+0000","last_unstale":"2024-01-11T07:11:57.228728+0000","last_undegraded":"2024-01-11T07:11:57.228728+0000","last_fullsized":"2024-01-11T07:11:57.228728+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:25:07.957466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"99'4","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.691980+0000","last_change":"2024-01-11T07:11:56.698594+0000","last_active":"2024-01-11T07:11:57.691980+0000","last_peered":"2024-01-11T07:11:57.691980+0000","last_clean":"2024-01-11T07:11:57.691980+0000","last_became_active":"2024-01-11T07:11:56.697387+0000","last_became_peered":"2024-01-11T07:11:56.697387+0000","last_unstale":"2024-01-11T07:11:57.691980+0000","last_undegraded":"2024-01-11T07:11:57.691980+0000","last_fullsized":"2024-01-11T07:11:57.691980+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:30:31.730592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231530+0000","last_change":"2024-01-11T07:11:57.228461+0000","last_active":"2024-01-11T07:11:57.231530+0000","last_peered":"2024-01-11T07:11:57.231530+0000","last_clean":"2024-01-11T07:11:57.231530+0000","last_became_active":"2024-01-11T07:11:57.227789+0000","last_became_peered":"2024-01-11T07:11:57.227789+0000","last_unstale":"2024-01-11T07:11:57.231530+0000","last_undegraded":"2024-01-11T07:11:57.231530+0000","last_fullsized":"2024-01-11T07:11:57.231530+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:30:24.474768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.691856+0000","last_change":"2024-01-11T07:11:56.694501+0000","last_active":"2024-01-11T07:11:57.691856+0000","last_peered":"2024-01-11T07:11:57.691856+0000","last_clean":"2024-01-11T07:11:57.691856+0000","last_became_active":"2024-01-11T07:11:56.694256+0000","last_became_peered":"2024-01-11T07:11:56.694256+0000","last_unstale":"2024-01-11T07:11:57.691856+0000","last_undegraded":"2024-01-11T07:11:57.691856+0000","last_fullsized":"2024-01-11T07:11:57.691856+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:51:05.943963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231580+0000","last_change":"2024-01-11T07:11:57.230605+0000","last_active":"2024-01-11T07:11:57.231580+0000","last_peered":"2024-01-11T07:11:57.231580+0000","last_clean":"2024-01-11T07:11:57.231580+0000","last_became_active":"2024-01-11T07:11:57.230355+0000","last_became_peered":"2024-01-11T07:11:57.230355+0000","last_unstale":"2024-01-11T07:11:57.231580+0000","last_undegraded":"2024-01-11T07:11:57.231580+0000","last_fullsized":"2024-01-11T07:11:57.231580+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:17.294168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.691702+0000","last_change":"2024-01-11T07:11:56.687635+0000","last_active":"2024-01-11T07:11:57.691702+0000","last_peered":"2024-01-11T07:11:57.691702+0000","last_clean":"2024-01-11T07:11:57.691702+0000","last_became_active":"2024-01-11T07:11:56.687229+0000","last_became_peered":"2024-01-11T07:11:56.687229+0000","last_unstale":"2024-01-11T07:11:57.691702+0000","last_undegraded":"2024-01-11T07:11:57.691702+0000","last_fullsized":"2024-01-11T07:11:57.691702+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:29:42.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233125+0000","last_change":"2024-01-11T07:11:57.233040+0000","last_active":"2024-01-11T07:11:57.233125+0000","last_peered":"2024-01-11T07:11:57.233125+0000","last_clean":"2024-01-11T07:11:57.233125+0000","last_became_active":"2024-01-11T07:11:57.232293+0000","last_became_peered":"2024-01-11T07:11:57.232293+0000","last_unstale":"2024-01-11T07:11:57.233125+0000","last_undegraded":"2024-01-11T07:11:57.233125+0000","last_fullsized":"2024-01-11T07:11:57.233125+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:09.306979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.689969+0000","last_change":"2024-01-11T07:11:56.694850+0000","last_active":"2024-01-11T07:11:57.689969+0000","last_peered":"2024-01-11T07:11:57.689969+0000","last_clean":"2024-01-11T07:11:57.689969+0000","last_became_active":"2024-01-11T07:11:56.694598+0000","last_became_peered":"2024-01-11T07:11:56.694598+0000","last_unstale":"2024-01-11T07:11:57.689969+0000","last_undegraded":"2024-01-11T07:11:57.689969+0000","last_fullsized":"2024-01-11T07:11:57.689969+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:25:06.954289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232958+0000","last_change":"2024-01-11T07:11:57.232902+0000","last_active":"2024-01-11T07:11:57.232958+0000","last_peered":"2024-01-11T07:11:57.232958+0000","last_clean":"2024-01-11T07:11:57.232958+0000","last_became_active":"2024-01-11T07:11:57.232490+0000","last_became_peered":"2024-01-11T07:11:57.232490+0000","last_unstale":"2024-01-11T07:11:57.232958+0000","last_undegraded":"2024-01-11T07:11:57.232958+0000","last_fullsized":"2024-01-11T07:11:57.232958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:12.903076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.690569+0000","last_change":"2024-01-11T07:11:56.695426+0000","last_active":"2024-01-11T07:11:57.690569+0000","last_peered":"2024-01-11T07:11:57.690569+0000","last_clean":"2024-01-11T07:11:57.690569+0000","last_became_active":"2024-01-11T07:11:56.695111+0000","last_became_peered":"2024-01-11T07:11:56.695111+0000","last_unstale":"2024-01-11T07:11:57.690569+0000","last_undegraded":"2024-01-11T07:11:57.690569+0000","last_fullsized":"2024-01-11T07:11:57.690569+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:19:12.033563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.237360+0000","last_change":"2024-01-11T07:11:57.237243+0000","last_active":"2024-01-11T07:11:57.237360+0000","last_peered":"2024-01-11T07:11:57.237360+0000","last_clean":"2024-01-11T07:11:57.237360+0000","last_became_active":"2024-01-11T07:11:57.236378+0000","last_became_peered":"2024-01-11T07:11:57.236378+0000","last_unstale":"2024-01-11T07:11:57.237360+0000","last_undegraded":"2024-01-11T07:11:57.237360+0000","last_fullsized":"2024-01-11T07:11:57.237360+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:29:16.174430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692570+0000","last_change":"2024-01-11T07:11:56.692379+0000","last_active":"2024-01-11T07:11:57.692570+0000","last_peered":"2024-01-11T07:11:57.692570+0000","last_clean":"2024-01-11T07:11:57.692570+0000","last_became_active":"2024-01-11T07:11:56.692199+0000","last_became_peered":"2024-01-11T07:11:56.692199+0000","last_unstale":"2024-01-11T07:11:57.692570+0000","last_undegraded":"2024-01-11T07:11:57.692570+0000","last_fullsized":"2024-01-11T07:11:57.692570+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:35:57.387820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233028+0000","last_change":"2024-01-11T07:11:57.232849+0000","last_active":"2024-01-11T07:11:57.233028+0000","last_peered":"2024-01-11T07:11:57.233028+0000","last_clean":"2024-01-11T07:11:57.233028+0000","last_became_active":"2024-01-11T07:11:57.231931+0000","last_became_peered":"2024-01-11T07:11:57.231931+0000","last_unstale":"2024-01-11T07:11:57.233028+0000","last_undegraded":"2024-01-11T07:11:57.233028+0000","last_fullsized":"2024-01-11T07:11:57.233028+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:08:28.809018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.218012+0000","last_change":"2024-01-11T07:11:56.688466+0000","last_active":"2024-01-11T07:11:57.218012+0000","last_peered":"2024-01-11T07:11:57.218012+0000","last_clean":"2024-01-11T07:11:57.218012+0000","last_became_active":"2024-01-11T07:11:56.688187+0000","last_became_peered":"2024-01-11T07:11:56.688187+0000","last_unstale":"2024-01-11T07:11:57.218012+0000","last_undegraded":"2024-01-11T07:11:57.218012+0000","last_fullsized":"2024-01-11T07:11:57.218012+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:01:11.444275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228056+0000","last_change":"2024-01-11T07:11:57.227422+0000","last_active":"2024-01-11T07:11:57.228056+0000","last_peered":"2024-01-11T07:11:57.228056+0000","last_clean":"2024-01-11T07:11:57.228056+0000","last_became_active":"2024-01-11T07:11:57.227196+0000","last_became_peered":"2024-01-11T07:11:57.227196+0000","last_unstale":"2024-01-11T07:11:57.228056+0000","last_undegraded":"2024-01-11T07:11:57.228056+0000","last_fullsized":"2024-01-11T07:11:57.228056+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:36:15.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228867+0000","last_change":"2024-01-11T07:11:56.694343+0000","last_active":"2024-01-11T07:11:57.228867+0000","last_peered":"2024-01-11T07:11:57.228867+0000","last_clean":"2024-01-11T07:11:57.228867+0000","last_became_active":"2024-01-11T07:11:56.693989+0000","last_became_peered":"2024-01-11T07:11:56.693989+0000","last_unstale":"2024-01-11T07:11:57.228867+0000","last_undegraded":"2024-01-11T07:11:57.228867+0000","last_fullsized":"2024-01-11T07:11:57.228867+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:47:28.786603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233611+0000","last_change":"2024-01-11T07:11:57.233400+0000","last_active":"2024-01-11T07:11:57.233611+0000","last_peered":"2024-01-11T07:11:57.233611+0000","last_clean":"2024-01-11T07:11:57.233611+0000","last_became_active":"2024-01-11T07:11:57.233156+0000","last_became_peered":"2024-01-11T07:11:57.233156+0000","last_unstale":"2024-01-11T07:11:57.233611+0000","last_undegraded":"2024-01-11T07:11:57.233611+0000","last_fullsized":"2024-01-11T07:11:57.233611+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:16.259881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222289+0000","last_change":"2024-01-11T07:11:56.693933+0000","last_active":"2024-01-11T07:11:57.222289+0000","last_peered":"2024-01-11T07:11:57.222289+0000","last_clean":"2024-01-11T07:11:57.222289+0000","last_became_active":"2024-01-11T07:11:56.693775+0000","last_became_peered":"2024-01-11T07:11:56.693775+0000","last_unstale":"2024-01-11T07:11:57.222289+0000","last_undegraded":"2024-01-11T07:11:57.222289+0000","last_fullsized":"2024-01-11T07:11:57.222289+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:43:24.348074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234194+0000","last_change":"2024-01-11T07:11:57.234081+0000","last_active":"2024-01-11T07:11:57.234194+0000","last_peered":"2024-01-11T07:11:57.234194+0000","last_clean":"2024-01-11T07:11:57.234194+0000","last_became_active":"2024-01-11T07:11:57.233837+0000","last_became_peered":"2024-01-11T07:11:57.233837+0000","last_unstale":"2024-01-11T07:11:57.234194+0000","last_undegraded":"2024-01-11T07:11:57.234194+0000","last_fullsized":"2024-01-11T07:11:57.234194+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:50.009991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225398+0000","last_change":"2024-01-11T07:11:56.695843+0000","last_active":"2024-01-11T07:11:57.225398+0000","last_peered":"2024-01-11T07:11:57.225398+0000","last_clean":"2024-01-11T07:11:57.225398+0000","last_became_active":"2024-01-11T07:11:56.695572+0000","last_became_peered":"2024-01-11T07:11:56.695572+0000","last_unstale":"2024-01-11T07:11:57.225398+0000","last_undegraded":"2024-01-11T07:11:57.225398+0000","last_fullsized":"2024-01-11T07:11:57.225398+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:05:37.725631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225051+0000","last_change":"2024-01-11T07:11:57.224709+0000","last_active":"2024-01-11T07:11:57.225051+0000","last_peered":"2024-01-11T07:11:57.225051+0000","last_clean":"2024-01-11T07:11:57.225051+0000","last_became_active":"2024-01-11T07:11:57.224498+0000","last_became_peered":"2024-01-11T07:11:57.224498+0000","last_unstale":"2024-01-11T07:11:57.225051+0000","last_undegraded":"2024-01-11T07:11:57.225051+0000","last_fullsized":"2024-01-11T07:11:57.225051+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:50:54.352317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"99'3","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692905+0000","last_change":"2024-01-11T07:11:56.698414+0000","last_active":"2024-01-11T07:11:57.692905+0000","last_peered":"2024-01-11T07:11:57.692905+0000","last_clean":"2024-01-11T07:11:57.692905+0000","last_became_active":"2024-01-11T07:11:56.696842+0000","last_became_peered":"2024-01-11T07:11:56.696842+0000","last_unstale":"2024-01-11T07:11:57.692905+0000","last_undegraded":"2024-01-11T07:11:57.692905+0000","last_fullsized":"2024-01-11T07:11:57.692905+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:03.139204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.230518+0000","last_change":"2024-01-11T07:11:57.230455+0000","last_active":"2024-01-11T07:11:57.230518+0000","last_peered":"2024-01-11T07:11:57.230518+0000","last_clean":"2024-01-11T07:11:57.230518+0000","last_became_active":"2024-01-11T07:11:57.230217+0000","last_became_peered":"2024-01-11T07:11:57.230217+0000","last_unstale":"2024-01-11T07:11:57.230518+0000","last_undegraded":"2024-01-11T07:11:57.230518+0000","last_fullsized":"2024-01-11T07:11:57.230518+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:27.939787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"99'6","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.693798+0000","last_change":"2024-01-11T07:11:56.688525+0000","last_active":"2024-01-11T07:11:57.693798+0000","last_peered":"2024-01-11T07:11:57.693798+0000","last_clean":"2024-01-11T07:11:57.693798+0000","last_became_active":"2024-01-11T07:11:56.688250+0000","last_became_peered":"2024-01-11T07:11:56.688250+0000","last_unstale":"2024-01-11T07:11:57.693798+0000","last_undegraded":"2024-01-11T07:11:57.693798+0000","last_fullsized":"2024-01-11T07:11:57.693798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:11.899899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.229811+0000","last_change":"2024-01-11T07:11:57.229474+0000","last_active":"2024-01-11T07:11:57.229811+0000","last_peered":"2024-01-11T07:11:57.229811+0000","last_clean":"2024-01-11T07:11:57.229811+0000","last_became_active":"2024-01-11T07:11:57.228881+0000","last_became_peered":"2024-01-11T07:11:57.228881+0000","last_unstale":"2024-01-11T07:11:57.229811+0000","last_undegraded":"2024-01-11T07:11:57.229811+0000","last_fullsized":"2024-01-11T07:11:57.229811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:53:15.929249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.690371+0000","last_change":"2024-01-11T07:11:56.685658+0000","last_active":"2024-01-11T07:11:57.690371+0000","last_peered":"2024-01-11T07:11:57.690371+0000","last_clean":"2024-01-11T07:11:57.690371+0000","last_became_active":"2024-01-11T07:11:56.685409+0000","last_became_peered":"2024-01-11T07:11:56.685409+0000","last_unstale":"2024-01-11T07:11:57.690371+0000","last_undegraded":"2024-01-11T07:11:57.690371+0000","last_fullsized":"2024-01-11T07:11:57.690371+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:45:19.318830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233423+0000","last_change":"2024-01-11T07:11:57.233342+0000","last_active":"2024-01-11T07:11:57.233423+0000","last_peered":"2024-01-11T07:11:57.233423+0000","last_clean":"2024-01-11T07:11:57.233423+0000","last_became_active":"2024-01-11T07:11:57.232626+0000","last_became_peered":"2024-01-11T07:11:57.232626+0000","last_unstale":"2024-01-11T07:11:57.233423+0000","last_undegraded":"2024-01-11T07:11:57.233423+0000","last_fullsized":"2024-01-11T07:11:57.233423+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:29:16.174430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221022+0000","last_change":"2024-01-11T07:11:56.685620+0000","last_active":"2024-01-11T07:11:57.221022+0000","last_peered":"2024-01-11T07:11:57.221022+0000","last_clean":"2024-01-11T07:11:57.221022+0000","last_became_active":"2024-01-11T07:11:56.685344+0000","last_became_peered":"2024-01-11T07:11:56.685344+0000","last_unstale":"2024-01-11T07:11:57.221022+0000","last_undegraded":"2024-01-11T07:11:57.221022+0000","last_fullsized":"2024-01-11T07:11:57.221022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:53:43.075643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232813+0000","last_change":"2024-01-11T07:11:57.232740+0000","last_active":"2024-01-11T07:11:57.232813+0000","last_peered":"2024-01-11T07:11:57.232813+0000","last_clean":"2024-01-11T07:11:57.232813+0000","last_became_active":"2024-01-11T07:11:57.232119+0000","last_became_peered":"2024-01-11T07:11:57.232119+0000","last_unstale":"2024-01-11T07:11:57.232813+0000","last_undegraded":"2024-01-11T07:11:57.232813+0000","last_fullsized":"2024-01-11T07:11:57.232813+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:35:58.390997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.224329+0000","last_change":"2024-01-11T07:11:56.694765+0000","last_active":"2024-01-11T07:11:57.224329+0000","last_peered":"2024-01-11T07:11:57.224329+0000","last_clean":"2024-01-11T07:11:57.224329+0000","last_became_active":"2024-01-11T07:11:56.694508+0000","last_became_peered":"2024-01-11T07:11:56.694508+0000","last_unstale":"2024-01-11T07:11:57.224329+0000","last_undegraded":"2024-01-11T07:11:57.224329+0000","last_fullsized":"2024-01-11T07:11:57.224329+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:00:26.290374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.235371+0000","last_change":"2024-01-11T07:11:57.235283+0000","last_active":"2024-01-11T07:11:57.235371+0000","last_peered":"2024-01-11T07:11:57.235371+0000","last_clean":"2024-01-11T07:11:57.235371+0000","last_became_active":"2024-01-11T07:11:57.234817+0000","last_became_peered":"2024-01-11T07:11:57.234817+0000","last_unstale":"2024-01-11T07:11:57.235371+0000","last_undegraded":"2024-01-11T07:11:57.235371+0000","last_fullsized":"2024-01-11T07:11:57.235371+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:32.679088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"99'8","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692997+0000","last_change":"2024-01-11T07:11:56.691703+0000","last_active":"2024-01-11T07:11:57.692997+0000","last_peered":"2024-01-11T07:11:57.692997+0000","last_clean":"2024-01-11T07:11:57.692997+0000","last_became_active":"2024-01-11T07:11:56.691388+0000","last_became_peered":"2024-01-11T07:11:56.691388+0000","last_unstale":"2024-01-11T07:11:57.692997+0000","last_undegraded":"2024-01-11T07:11:57.692997+0000","last_fullsized":"2024-01-11T07:11:57.692997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:30:23.471591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234636+0000","last_change":"2024-01-11T07:11:57.234567+0000","last_active":"2024-01-11T07:11:57.234636+0000","last_peered":"2024-01-11T07:11:57.234636+0000","last_clean":"2024-01-11T07:11:57.234636+0000","last_became_active":"2024-01-11T07:11:57.234333+0000","last_became_peered":"2024-01-11T07:11:57.234333+0000","last_unstale":"2024-01-11T07:11:57.234636+0000","last_undegraded":"2024-01-11T07:11:57.234636+0000","last_fullsized":"2024-01-11T07:11:57.234636+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:19:15.424694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.688601+0000","last_change":"2024-01-11T07:11:56.689612+0000","last_active":"2024-01-11T07:11:57.688601+0000","last_peered":"2024-01-11T07:11:57.688601+0000","last_clean":"2024-01-11T07:11:57.688601+0000","last_became_active":"2024-01-11T07:11:56.689123+0000","last_became_peered":"2024-01-11T07:11:56.689123+0000","last_unstale":"2024-01-11T07:11:57.688601+0000","last_undegraded":"2024-01-11T07:11:57.688601+0000","last_fullsized":"2024-01-11T07:11:57.688601+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:22:22.610321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.236910+0000","last_change":"2024-01-11T07:11:57.236847+0000","last_active":"2024-01-11T07:11:57.236910+0000","last_peered":"2024-01-11T07:11:57.236910+0000","last_clean":"2024-01-11T07:11:57.236910+0000","last_became_active":"2024-01-11T07:11:57.236589+0000","last_became_peered":"2024-01-11T07:11:57.236589+0000","last_unstale":"2024-01-11T07:11:57.236910+0000","last_undegraded":"2024-01-11T07:11:57.236910+0000","last_fullsized":"2024-01-11T07:11:57.236910+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:56.349066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692343+0000","last_change":"2024-01-11T07:11:56.691137+0000","last_active":"2024-01-11T07:11:57.692343+0000","last_peered":"2024-01-11T07:11:57.692343+0000","last_clean":"2024-01-11T07:11:57.692343+0000","last_became_active":"2024-01-11T07:11:56.690819+0000","last_became_peered":"2024-01-11T07:11:56.690819+0000","last_unstale":"2024-01-11T07:11:57.692343+0000","last_undegraded":"2024-01-11T07:11:57.692343+0000","last_fullsized":"2024-01-11T07:11:57.692343+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:19:14.421517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231325+0000","last_change":"2024-01-11T07:11:57.231073+0000","last_active":"2024-01-11T07:11:57.231325+0000","last_peered":"2024-01-11T07:11:57.231325+0000","last_clean":"2024-01-11T07:11:57.231325+0000","last_became_active":"2024-01-11T07:11:57.230354+0000","last_became_peered":"2024-01-11T07:11:57.230354+0000","last_unstale":"2024-01-11T07:11:57.231325+0000","last_undegraded":"2024-01-11T07:11:57.231325+0000","last_fullsized":"2024-01-11T07:11:57.231325+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:30:24.474768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.217497+0000","last_change":"2024-01-11T07:11:56.687904+0000","last_active":"2024-01-11T07:11:57.217497+0000","last_peered":"2024-01-11T07:11:57.217497+0000","last_clean":"2024-01-11T07:11:57.217497+0000","last_became_active":"2024-01-11T07:11:56.687649+0000","last_became_peered":"2024-01-11T07:11:56.687649+0000","last_unstale":"2024-01-11T07:11:57.217497+0000","last_undegraded":"2024-01-11T07:11:57.217497+0000","last_fullsized":"2024-01-11T07:11:57.217497+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:29:15.171253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232661+0000","last_change":"2024-01-11T07:11:57.232260+0000","last_active":"2024-01-11T07:11:57.232661+0000","last_peered":"2024-01-11T07:11:57.232661+0000","last_clean":"2024-01-11T07:11:57.232661+0000","last_became_active":"2024-01-11T07:11:57.231328+0000","last_became_peered":"2024-01-11T07:11:57.231328+0000","last_unstale":"2024-01-11T07:11:57.232661+0000","last_undegraded":"2024-01-11T07:11:57.232661+0000","last_fullsized":"2024-01-11T07:11:57.232661+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:09:49.762710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692001+0000","last_change":"2024-01-11T07:11:56.687885+0000","last_active":"2024-01-11T07:11:57.692001+0000","last_peered":"2024-01-11T07:11:57.692001+0000","last_clean":"2024-01-11T07:11:57.692001+0000","last_became_active":"2024-01-11T07:11:56.687610+0000","last_became_peered":"2024-01-11T07:11:56.687610+0000","last_unstale":"2024-01-11T07:11:57.692001+0000","last_undegraded":"2024-01-11T07:11:57.692001+0000","last_fullsized":"2024-01-11T07:11:57.692001+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:08.303802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226088+0000","last_change":"2024-01-11T07:11:57.226019+0000","last_active":"2024-01-11T07:11:57.226088+0000","last_peered":"2024-01-11T07:11:57.226088+0000","last_clean":"2024-01-11T07:11:57.226088+0000","last_became_active":"2024-01-11T07:11:57.225786+0000","last_became_peered":"2024-01-11T07:11:57.225786+0000","last_unstale":"2024-01-11T07:11:57.226088+0000","last_undegraded":"2024-01-11T07:11:57.226088+0000","last_fullsized":"2024-01-11T07:11:57.226088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:05:00.875268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.220697+0000","last_change":"2024-01-11T07:11:56.687901+0000","last_active":"2024-01-11T07:11:57.220697+0000","last_peered":"2024-01-11T07:11:57.220697+0000","last_clean":"2024-01-11T07:11:57.220697+0000","last_became_active":"2024-01-11T07:11:56.687645+0000","last_became_peered":"2024-01-11T07:11:56.687645+0000","last_unstale":"2024-01-11T07:11:57.220697+0000","last_undegraded":"2024-01-11T07:11:57.220697+0000","last_fullsized":"2024-01-11T07:11:57.220697+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:01:37.696909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233599+0000","last_change":"2024-01-11T07:11:57.233460+0000","last_active":"2024-01-11T07:11:57.233599+0000","last_peered":"2024-01-11T07:11:57.233599+0000","last_clean":"2024-01-11T07:11:57.233599+0000","last_became_active":"2024-01-11T07:11:57.229127+0000","last_became_peered":"2024-01-11T07:11:57.229127+0000","last_unstale":"2024-01-11T07:11:57.233599+0000","last_undegraded":"2024-01-11T07:11:57.233599+0000","last_fullsized":"2024-01-11T07:11:57.233599+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:25:31.452694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.219525+0000","last_change":"2024-01-11T07:11:56.695920+0000","last_active":"2024-01-11T07:11:57.219525+0000","last_peered":"2024-01-11T07:11:57.219525+0000","last_clean":"2024-01-11T07:11:57.219525+0000","last_became_active":"2024-01-11T07:11:56.695671+0000","last_became_peered":"2024-01-11T07:11:56.695671+0000","last_unstale":"2024-01-11T07:11:57.219525+0000","last_undegraded":"2024-01-11T07:11:57.219525+0000","last_fullsized":"2024-01-11T07:11:57.219525+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:34:04.132443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.227680+0000","last_change":"2024-01-11T07:11:57.227515+0000","last_active":"2024-01-11T07:11:57.227680+0000","last_peered":"2024-01-11T07:11:57.227680+0000","last_clean":"2024-01-11T07:11:57.227680+0000","last_became_active":"2024-01-11T07:11:57.227290+0000","last_became_peered":"2024-01-11T07:11:57.227290+0000","last_unstale":"2024-01-11T07:11:57.227680+0000","last_undegraded":"2024-01-11T07:11:57.227680+0000","last_fullsized":"2024-01-11T07:11:57.227680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:37:32.425877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221112+0000","last_change":"2024-01-11T07:11:56.692052+0000","last_active":"2024-01-11T07:11:57.221112+0000","last_peered":"2024-01-11T07:11:57.221112+0000","last_clean":"2024-01-11T07:11:57.221112+0000","last_became_active":"2024-01-11T07:11:56.691786+0000","last_became_peered":"2024-01-11T07:11:56.691786+0000","last_unstale":"2024-01-11T07:11:57.221112+0000","last_undegraded":"2024-01-11T07:11:57.221112+0000","last_fullsized":"2024-01-11T07:11:57.221112+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:08:27.805841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228364+0000","last_change":"2024-01-11T07:11:57.228255+0000","last_active":"2024-01-11T07:11:57.228364+0000","last_peered":"2024-01-11T07:11:57.228364+0000","last_clean":"2024-01-11T07:11:57.228364+0000","last_became_active":"2024-01-11T07:11:57.227998+0000","last_became_peered":"2024-01-11T07:11:57.227998+0000","last_unstale":"2024-01-11T07:11:57.228364+0000","last_undegraded":"2024-01-11T07:11:57.228364+0000","last_fullsized":"2024-01-11T07:11:57.228364+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:56.072710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.690264+0000","last_change":"2024-01-11T07:11:56.690344+0000","last_active":"2024-01-11T07:11:57.690264+0000","last_peered":"2024-01-11T07:11:57.690264+0000","last_clean":"2024-01-11T07:11:57.690264+0000","last_became_active":"2024-01-11T07:11:56.689769+0000","last_became_peered":"2024-01-11T07:11:56.689769+0000","last_unstale":"2024-01-11T07:11:57.690264+0000","last_undegraded":"2024-01-11T07:11:57.690264+0000","last_fullsized":"2024-01-11T07:11:57.690264+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:34:22.736536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.236882+0000","last_change":"2024-01-11T07:11:57.236813+0000","last_active":"2024-01-11T07:11:57.236882+0000","last_peered":"2024-01-11T07:11:57.236882+0000","last_clean":"2024-01-11T07:11:57.236882+0000","last_became_active":"2024-01-11T07:11:57.236512+0000","last_became_peered":"2024-01-11T07:11:57.236512+0000","last_unstale":"2024-01-11T07:11:57.236882+0000","last_undegraded":"2024-01-11T07:11:57.236882+0000","last_fullsized":"2024-01-11T07:11:57.236882+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:44:39.925336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226425+0000","last_change":"2024-01-11T07:11:56.691047+0000","last_active":"2024-01-11T07:11:57.226425+0000","last_peered":"2024-01-11T07:11:57.226425+0000","last_clean":"2024-01-11T07:11:57.226425+0000","last_became_active":"2024-01-11T07:11:56.690095+0000","last_became_peered":"2024-01-11T07:11:56.690095+0000","last_unstale":"2024-01-11T07:11:57.226425+0000","last_undegraded":"2024-01-11T07:11:57.226425+0000","last_fullsized":"2024-01-11T07:11:57.226425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:43:44.699736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225236+0000","last_change":"2024-01-11T07:11:57.224157+0000","last_active":"2024-01-11T07:11:57.225236+0000","last_peered":"2024-01-11T07:11:57.225236+0000","last_clean":"2024-01-11T07:11:57.225236+0000","last_became_active":"2024-01-11T07:11:57.223907+0000","last_became_peered":"2024-01-11T07:11:57.223907+0000","last_unstale":"2024-01-11T07:11:57.225236+0000","last_undegraded":"2024-01-11T07:11:57.225236+0000","last_fullsized":"2024-01-11T07:11:57.225236+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:11.847550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222839+0000","last_change":"2024-01-11T07:11:56.695761+0000","last_active":"2024-01-11T07:11:57.222839+0000","last_peered":"2024-01-11T07:11:57.222839+0000","last_clean":"2024-01-11T07:11:57.222839+0000","last_became_active":"2024-01-11T07:11:56.695509+0000","last_became_peered":"2024-01-11T07:11:56.695509+0000","last_unstale":"2024-01-11T07:11:57.222839+0000","last_undegraded":"2024-01-11T07:11:57.222839+0000","last_fullsized":"2024-01-11T07:11:57.222839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:15:55.700740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226929+0000","last_change":"2024-01-11T07:11:57.226836+0000","last_active":"2024-01-11T07:11:57.226929+0000","last_peered":"2024-01-11T07:11:57.226929+0000","last_clean":"2024-01-11T07:11:57.226929+0000","last_became_active":"2024-01-11T07:11:57.226544+0000","last_became_peered":"2024-01-11T07:11:57.226544+0000","last_unstale":"2024-01-11T07:11:57.226929+0000","last_undegraded":"2024-01-11T07:11:57.226929+0000","last_fullsized":"2024-01-11T07:11:57.226929+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:19:15.424694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222093+0000","last_change":"2024-01-11T07:11:56.692905+0000","last_active":"2024-01-11T07:11:57.222093+0000","last_peered":"2024-01-11T07:11:57.222093+0000","last_clean":"2024-01-11T07:11:57.222093+0000","last_became_active":"2024-01-11T07:11:56.692682+0000","last_became_peered":"2024-01-11T07:11:56.692682+0000","last_unstale":"2024-01-11T07:11:57.222093+0000","last_undegraded":"2024-01-11T07:11:57.222093+0000","last_fullsized":"2024-01-11T07:11:57.222093+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:27:52.893353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225298+0000","last_change":"2024-01-11T07:11:57.225237+0000","last_active":"2024-01-11T07:11:57.225298+0000","last_peered":"2024-01-11T07:11:57.225298+0000","last_clean":"2024-01-11T07:11:57.225298+0000","last_became_active":"2024-01-11T07:11:57.224644+0000","last_became_peered":"2024-01-11T07:11:57.224644+0000","last_unstale":"2024-01-11T07:11:57.225298+0000","last_undegraded":"2024-01-11T07:11:57.225298+0000","last_fullsized":"2024-01-11T07:11:57.225298+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:15:50.159042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.218186+0000","last_change":"2024-01-11T07:11:56.691350+0000","last_active":"2024-01-11T07:11:57.218186+0000","last_peered":"2024-01-11T07:11:57.218186+0000","last_clean":"2024-01-11T07:11:57.218186+0000","last_became_active":"2024-01-11T07:11:56.691069+0000","last_became_peered":"2024-01-11T07:11:56.691069+0000","last_unstale":"2024-01-11T07:11:57.218186+0000","last_undegraded":"2024-01-11T07:11:57.218186+0000","last_fullsized":"2024-01-11T07:11:57.218186+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:36:00.714584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.229792+0000","last_change":"2024-01-11T07:11:57.229330+0000","last_active":"2024-01-11T07:11:57.229792+0000","last_peered":"2024-01-11T07:11:57.229792+0000","last_clean":"2024-01-11T07:11:57.229792+0000","last_became_active":"2024-01-11T07:11:57.228547+0000","last_became_peered":"2024-01-11T07:11:57.228547+0000","last_unstale":"2024-01-11T07:11:57.229792+0000","last_undegraded":"2024-01-11T07:11:57.229792+0000","last_fullsized":"2024-01-11T07:11:57.229792+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:32:54.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.229911+0000","last_change":"2024-01-11T07:11:56.694628+0000","last_active":"2024-01-11T07:11:57.229911+0000","last_peered":"2024-01-11T07:11:57.229911+0000","last_clean":"2024-01-11T07:11:57.229911+0000","last_became_active":"2024-01-11T07:11:56.694311+0000","last_became_peered":"2024-01-11T07:11:56.694311+0000","last_unstale":"2024-01-11T07:11:57.229911+0000","last_undegraded":"2024-01-11T07:11:57.229911+0000","last_fullsized":"2024-01-11T07:11:57.229911+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:11.899899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.236975+0000","last_change":"2024-01-11T07:11:57.236855+0000","last_active":"2024-01-11T07:11:57.236975+0000","last_peered":"2024-01-11T07:11:57.236975+0000","last_clean":"2024-01-11T07:11:57.236975+0000","last_became_active":"2024-01-11T07:11:57.236606+0000","last_became_peered":"2024-01-11T07:11:57.236606+0000","last_unstale":"2024-01-11T07:11:57.236975+0000","last_undegraded":"2024-01-11T07:11:57.236975+0000","last_fullsized":"2024-01-11T07:11:57.236975+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:53:44.078820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.224083+0000","last_change":"2024-01-11T07:11:56.698022+0000","last_active":"2024-01-11T07:11:57.224083+0000","last_peered":"2024-01-11T07:11:57.224083+0000","last_clean":"2024-01-11T07:11:57.224083+0000","last_became_active":"2024-01-11T07:11:56.696485+0000","last_became_peered":"2024-01-11T07:11:56.696485+0000","last_unstale":"2024-01-11T07:11:57.224083+0000","last_undegraded":"2024-01-11T07:11:57.224083+0000","last_fullsized":"2024-01-11T07:11:57.224083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:52:11.313063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228207+0000","last_change":"2024-01-11T07:11:57.228122+0000","last_active":"2024-01-11T07:11:57.228207+0000","last_peered":"2024-01-11T07:11:57.228207+0000","last_clean":"2024-01-11T07:11:57.228207+0000","last_became_active":"2024-01-11T07:11:57.226994+0000","last_became_peered":"2024-01-11T07:11:57.226994+0000","last_unstale":"2024-01-11T07:11:57.228207+0000","last_undegraded":"2024-01-11T07:11:57.228207+0000","last_fullsized":"2024-01-11T07:11:57.228207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:36:01.717761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221950+0000","last_change":"2024-01-11T07:11:56.696114+0000","last_active":"2024-01-11T07:11:57.221950+0000","last_peered":"2024-01-11T07:11:57.221950+0000","last_clean":"2024-01-11T07:11:57.221950+0000","last_became_active":"2024-01-11T07:11:56.695944+0000","last_became_peered":"2024-01-11T07:11:56.695944+0000","last_unstale":"2024-01-11T07:11:57.221950+0000","last_undegraded":"2024-01-11T07:11:57.221950+0000","last_fullsized":"2024-01-11T07:11:57.221950+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:00:26.290374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231605+0000","last_change":"2024-01-11T07:11:57.231554+0000","last_active":"2024-01-11T07:11:57.231605+0000","last_peered":"2024-01-11T07:11:57.231605+0000","last_clean":"2024-01-11T07:11:57.231605+0000","last_became_active":"2024-01-11T07:11:57.231219+0000","last_became_peered":"2024-01-11T07:11:57.231219+0000","last_unstale":"2024-01-11T07:11:57.231605+0000","last_undegraded":"2024-01-11T07:11:57.231605+0000","last_fullsized":"2024-01-11T07:11:57.231605+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:22:17.401294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.690660+0000","last_change":"2024-01-11T07:11:56.696768+0000","last_active":"2024-01-11T07:11:57.690660+0000","last_peered":"2024-01-11T07:11:57.690660+0000","last_clean":"2024-01-11T07:11:57.690660+0000","last_became_active":"2024-01-11T07:11:56.696494+0000","last_became_peered":"2024-01-11T07:11:56.696494+0000","last_unstale":"2024-01-11T07:11:57.690660+0000","last_undegraded":"2024-01-11T07:11:57.690660+0000","last_fullsized":"2024-01-11T07:11:57.690660+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:15:49.155865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.224099+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:11:57.224099+0000","last_peered":"2024-01-11T07:11:57.224099+0000","last_clean":"2024-01-11T07:11:57.224099+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:11:57.224099+0000","last_undegraded":"2024-01-11T07:11:57.224099+0000","last_fullsized":"2024-01-11T07:11:57.224099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1102,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194209,"statfs":{"total":200000000,"available":198871004,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64400000000000002,"5min":0.754,"15min":0.754},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.78000000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70099999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.75900000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.53300000000000003},"min":{"1min":0.46500000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76100000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.747,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.68700000000000006}]},{"osd":4,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48299999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77900000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.623},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79900000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82899999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194792,"statfs":{"total":200000000,"available":199467064,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.42399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68000000000000005,"5min":1.157,"15min":1.157},"last":0.64700000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.622},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70799999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69399999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51800000000000002,"15min":0.54600000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66600000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53400000000000003,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59699999999999998,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.53300000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67800000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71299999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194792,"statfs":{"total":200000000,"available":199467536,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.60699999999999998,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56699999999999995,"5min":20.41,"15min":20.41},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.57199999999999995,"15min":0.46600000000000003},"min":{"1min":0.313,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69199999999999995,"5min":15.611000000000001,"15min":15.611000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.69099999999999995,"15min":0.54900000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65300000000000002,"5min":20.199999999999999,"15min":20.199999999999999},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.66800000000000004,"15min":0.53900000000000003},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":20.369,"15min":20.369},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42699999999999999,"15min":0.437},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61399999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.70499999999999996,"15min":0.64300000000000002},"min":{"1min":0.41399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.83099999999999996,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.748,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85799999999999998,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":0.72899999999999998,"5min":1.413,"15min":1.413},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72699999999999998,"15min":0.67400000000000004},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95299999999999996,"5min":9.891,"15min":9.891},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.502},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.746,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.88700000000000001,"15min":0.77100000000000002},"min":{"1min":0.48199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.91000000000000003,"5min":20.324000000000002,"15min":20.324000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69499999999999995,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92600000000000005,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.46999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1096,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194216,"statfs":{"total":200000000,"available":198877328,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.438,"15min":0.47599999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.628,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.495},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.627}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.57699999999999996,"15min":0.73599999999999999},"min":{"1min":0.29799999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57099999999999995,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48399999999999999,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61799999999999999,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.58499999999999996,"15min":0.752},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59699999999999998,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53300000000000003,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70099999999999996,"5min":2.855,"15min":2.855},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.872,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.753},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77400000000000002,"15min":0.91400000000000003},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94399999999999995,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.67100000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68700000000000006,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97099999999999997,"5min":2.819,"15min":2.819},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84499999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.82099999999999995,"15min":0.97499999999999998},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.52500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0249999999999999,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.82599999999999996,"15min":0.96499999999999997},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0429999999999999,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.83599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194792,"statfs":{"total":200000000,"available":199467176,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66800000000000004,"5min":1.472,"15min":1.472},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.439,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58199999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.59499999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.74399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73499999999999999,"5min":1.139,"15min":1.139},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68999999999999995,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0229999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.52900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98199999999999998,"5min":1.052,"15min":1.052},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67000000000000004,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.624},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":527,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194785,"statfs":{"total":200000000,"available":199459906,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.50600000000000001,"15min":0.78700000000000003},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":11.481,"15min":11.481},"last":0.378},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.48299999999999998},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.55900000000000005,"15min":0.97699999999999998},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60799999999999998,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.30199999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.56799999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.63,"15min":1.034},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.40899999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91300000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.84899999999999998,"15min":1.2669999999999999},"min":{"1min":0.38500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84299999999999997,"5min":17.474,"15min":17.474},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.78900000000000003,"15min":1.2010000000000001},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.754,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.877,"5min":11.394,"15min":11.394},"last":0.81899999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.76500000000000001,"15min":1.0309999999999999},"min":{"1min":0.498,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.55200000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.84399999999999997,"15min":1.238},"min":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1839999999999999,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.65800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194792,"statfs":{"total":200000000,"available":199467730,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.89000000000000001,"15min":1.0940000000000001},"min":{"1min":0.46999999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.77400000000000002,"5min":8.484,"15min":8.484},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.91700000000000004,"15min":1.1539999999999999},"min":{"1min":0.48899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76100000000000001,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.749,"15min":0.89100000000000001},"min":{"1min":0.433,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.84899999999999998,"15min":1.0489999999999999},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.86099999999999999,"15min":1.0109999999999999},"min":{"1min":0.38800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98699999999999999,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.74199999999999999,"15min":0.90400000000000003},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71899999999999997,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.749}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.83999999999999997,"15min":1.091},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.92700000000000005,"15min":1.1659999999999999},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73099999999999998,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.72699999999999998,"15min":0.98699999999999999},"min":{"1min":0.313,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69799999999999995,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.70399999999999996,"15min":0.89400000000000002},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75600000000000001,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.74299999999999999,"15min":0.94699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76600000000000001,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":1.119,"15min":1.119},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.72899999999999998,"15min":0.92200000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.82399999999999995,"15min":1.0229999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81399999999999995,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":521,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72799999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.48999999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76700000000000002,"5min":0.87,"15min":0.87},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.44400000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.621}]},{"osd":2,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40899999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76600000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.505},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.46600000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78100000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.49399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.45000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81699999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.439},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.37}]},{"osd":6,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.754,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.61699999999999999},"min":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54400000000000004}]}]}],"pool_statfs":[]}} 2024-01-11T07:12:01.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:12:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.047+0000 7f1d2c3b2700 1 Processor -- start 2024-01-11T07:12:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.047+0000 7f1d2c3b2700 1 -- start start 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.048+0000 7f1d2c3b2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d2415a060 0x7f1d240b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.048+0000 7f1d2c3b2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d24069320 0x7f1d240b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.048+0000 7f1d2c3b2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d240ce9b0 con 0x7f1d24159c90 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.048+0000 7f1d2c3b2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d240b1f30 con 0x7f1d24069320 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.048+0000 7f1d2c3b2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d240cf530 con 0x7f1d2415a060 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2a94f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d24069320 0x7f1d240b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2a94f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d24069320 0x7f1d240b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56880/0 (socket says 172.21.15.7:56880) 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2a94f700 1 -- 172.21.15.7:0/735731360 learned_addr learned my addr 172.21.15.7:0/735731360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2994d700 1 --2- 172.21.15.7:0/735731360 >> v2:172.21.15.113:3300/0 conn(0x7f1d2415a060 0x7f1d240b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2a94f700 1 -- 172.21.15.7:0/735731360 >> v2:172.21.15.113:3300/0 conn(0x7f1d2415a060 msgr2=0x7f1d240b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:02.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2a94f700 1 --2- 172.21.15.7:0/735731360 >> v2:172.21.15.113:3300/0 conn(0x7f1d2415a060 0x7f1d240b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2a94f700 1 -- 172.21.15.7:0/735731360 >> v1:172.21.15.113:6789/0 conn(0x7f1d24159c90 legacy=0x7f1d240b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:02.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2a94f700 1 -- 172.21.15.7:0/735731360 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240cf9c0 con 0x7f1d24069320 2024-01-11T07:12:02.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2a94f700 1 --2- 172.21.15.7:0/735731360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24069320 0x7f1d240b0970 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1d240699e0 tx=0x7f1d1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965dd4b9fafd118c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2914c700 1 -- 172.21.15.7:0/735731360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00e070 con 0x7f1d24069320 2024-01-11T07:12:02.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2914c700 1 -- 172.21.15.7:0/735731360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c00ba50 con 0x7f1d24069320 2024-01-11T07:12:02.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.049+0000 7f1d2914c700 1 -- 172.21.15.7:0/735731360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0092e0 con 0x7f1d24069320 2024-01-11T07:12:02.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.050+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/735731360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24069320 msgr2=0x7f1d240b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.050+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/735731360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24069320 0x7f1d240b0970 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1d240699e0 tx=0x7f1d1c009840 comp rx=0 tx=0).stop 2024-01-11T07:12:02.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.050+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/735731360 shutdown_connections 2024-01-11T07:12:02.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.050+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/735731360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24069320 0x7f1d240b0970 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.050+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/735731360 >> v2:172.21.15.113:3300/0 conn(0x7f1d2415a060 0x7f1d240b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.050+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/735731360 >> 172.21.15.7:0/735731360 conn(0x7f1d24071100 msgr2=0x7f1d2406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:02.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.050+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/735731360 shutdown_connections 2024-01-11T07:12:02.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.050+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/735731360 wait complete. 2024-01-11T07:12:02.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.052+0000 7f1d2c3b2700 1 Processor -- start 2024-01-11T07:12:02.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.052+0000 7f1d2c3b2700 1 -- start start 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2c3b2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d24069320 0x7f1d2413fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2c3b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24159c90 0x7f1d2415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2c3b2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2415a060 0x7f1d2416d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2c3b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d24155b30 con 0x7f1d24159c90 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2c3b2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2416dea0 con 0x7f1d2415a060 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2c3b2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2416e180 con 0x7f1d24069320 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2994d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24159c90 0x7f1d2415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2994d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24159c90 0x7f1d2415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56882/0 (socket says 172.21.15.7:56882) 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2994d700 1 -- 172.21.15.7:0/1740705877 learned_addr learned my addr 172.21.15.7:0/1740705877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2a94f700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2415a060 0x7f1d2416d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2a14e700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d24069320 0x7f1d2413fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2994d700 1 -- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d24069320 msgr2=0x7f1d2413fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2994d700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d24069320 0x7f1d2413fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2994d700 1 -- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2415a060 msgr2=0x7f1d2416d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2994d700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2415a060 0x7f1d2416d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.053+0000 7f1d2994d700 1 -- 172.21.15.7:0/1740705877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c009ae0 con 0x7f1d24159c90 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.054+0000 7f1d2994d700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24159c90 0x7f1d2415b050 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000d890 tx=0x7f1d20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.054+0000 7f1d167fc700 1 -- 172.21.15.7:0/1740705877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d200125f0 con 0x7f1d24159c90 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.054+0000 7f1d167fc700 1 -- 172.21.15.7:0/1740705877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d20012750 con 0x7f1d24159c90 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.054+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2415a850 con 0x7f1d24159c90 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.054+0000 7f1d167fc700 1 -- 172.21.15.7:0/1740705877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20012af0 con 0x7f1d24159c90 2024-01-11T07:12:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.054+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2415b590 con 0x7f1d24159c90 2024-01-11T07:12:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.055+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2415ad70 con 0x7f1d24159c90 2024-01-11T07:12:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.055+0000 7f1d167fc700 1 -- 172.21.15.7:0/1740705877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1d2001e460 con 0x7f1d24159c90 2024-01-11T07:12:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.056+0000 7f1d167fc700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d08066f30 0x7f1d080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.056+0000 7f1d167fc700 1 -- 172.21.15.7:0/1740705877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d20084b00 con 0x7f1d24159c90 2024-01-11T07:12:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.056+0000 7f1d2a14e700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d08066f30 0x7f1d080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.057+0000 7f1d2a14e700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d08066f30 0x7f1d080693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1d18003a80 tx=0x7f1d18007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.060+0000 7f1d167fc700 1 -- 172.21.15.7:0/1740705877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d2415ad70 con 0x7f1d24159c90 2024-01-11T07:12:02.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.260+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d2415a6d0 con 0x7f1d08066f30 2024-01-11T07:12:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.269+0000 7f1d167fc700 1 -- 172.21.15.7:0/1740705877 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285271 (secure 0 0 0) 0x7f1d18002480 con 0x7f1d08066f30 2024-01-11T07:12:02.272 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d08066f30 msgr2=0x7f1d080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d08066f30 0x7f1d080693e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1d18003a80 tx=0x7f1d18007ec0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24159c90 msgr2=0x7f1d2415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24159c90 0x7f1d2415b050 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000d890 tx=0x7f1d20008040 comp rx=0 tx=0).stop 2024-01-11T07:12:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 shutdown_connections 2024-01-11T07:12:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d08066f30 0x7f1d080693e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d2415a060 0x7f1d2416d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d24069320 0x7f1d2413fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 --2- 172.21.15.7:0/1740705877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d24159c90 0x7f1d2415b050 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 >> 172.21.15.7:0/1740705877 conn(0x7f1d24071100 msgr2=0x7f1d24157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 shutdown_connections 2024-01-11T07:12:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.272+0000 7f1d2c3b2700 1 -- 172.21.15.7:0/1740705877 wait complete. 2024-01-11T07:12:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:12:02.285 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-01-11T07:12:01.911554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5328,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557171,"statfs":{"total":1600000000,"available":1594543916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001456"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228186+0000","last_change":"2024-01-11T07:11:57.228025+0000","last_active":"2024-01-11T07:11:57.228186+0000","last_peered":"2024-01-11T07:11:57.228186+0000","last_clean":"2024-01-11T07:11:57.228186+0000","last_became_active":"2024-01-11T07:11:57.227653+0000","last_became_peered":"2024-01-11T07:11:57.227653+0000","last_unstale":"2024-01-11T07:11:57.228186+0000","last_undegraded":"2024-01-11T07:11:57.228186+0000","last_fullsized":"2024-01-11T07:11:57.228186+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:56.072710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.235351+0000","last_change":"2024-01-11T07:11:57.235104+0000","last_active":"2024-01-11T07:11:57.235351+0000","last_peered":"2024-01-11T07:11:57.235351+0000","last_clean":"2024-01-11T07:11:57.235351+0000","last_became_active":"2024-01-11T07:11:57.234485+0000","last_became_peered":"2024-01-11T07:11:57.234485+0000","last_unstale":"2024-01-11T07:11:57.235351+0000","last_undegraded":"2024-01-11T07:11:57.235351+0000","last_fullsized":"2024-01-11T07:11:57.235351+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:32:05.990591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231770+0000","last_change":"2024-01-11T07:11:57.231674+0000","last_active":"2024-01-11T07:11:57.231770+0000","last_peered":"2024-01-11T07:11:57.231770+0000","last_clean":"2024-01-11T07:11:57.231770+0000","last_became_active":"2024-01-11T07:11:57.231368+0000","last_became_peered":"2024-01-11T07:11:57.231368+0000","last_unstale":"2024-01-11T07:11:57.231770+0000","last_undegraded":"2024-01-11T07:11:57.231770+0000","last_fullsized":"2024-01-11T07:11:57.231770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:30:45.036920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232696+0000","last_change":"2024-01-11T07:11:57.232599+0000","last_active":"2024-01-11T07:11:57.232696+0000","last_peered":"2024-01-11T07:11:57.232696+0000","last_clean":"2024-01-11T07:11:57.232696+0000","last_became_active":"2024-01-11T07:11:57.232004+0000","last_became_peered":"2024-01-11T07:11:57.232004+0000","last_unstale":"2024-01-11T07:11:57.232696+0000","last_undegraded":"2024-01-11T07:11:57.232696+0000","last_fullsized":"2024-01-11T07:11:57.232696+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:40:23.300549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.230837+0000","last_change":"2024-01-11T07:11:57.230755+0000","last_active":"2024-01-11T07:11:57.230837+0000","last_peered":"2024-01-11T07:11:57.230837+0000","last_clean":"2024-01-11T07:11:57.230837+0000","last_became_active":"2024-01-11T07:11:57.230558+0000","last_became_peered":"2024-01-11T07:11:57.230558+0000","last_unstale":"2024-01-11T07:11:57.230837+0000","last_undegraded":"2024-01-11T07:11:57.230837+0000","last_fullsized":"2024-01-11T07:11:57.230837+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:17.294168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234873+0000","last_change":"2024-01-11T07:11:57.234804+0000","last_active":"2024-01-11T07:11:57.234873+0000","last_peered":"2024-01-11T07:11:57.234873+0000","last_clean":"2024-01-11T07:11:57.234873+0000","last_became_active":"2024-01-11T07:11:57.234563+0000","last_became_peered":"2024-01-11T07:11:57.234563+0000","last_unstale":"2024-01-11T07:11:57.234873+0000","last_undegraded":"2024-01-11T07:11:57.234873+0000","last_fullsized":"2024-01-11T07:11:57.234873+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:17.294168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.236300+0000","last_change":"2024-01-11T07:11:57.236250+0000","last_active":"2024-01-11T07:11:57.236300+0000","last_peered":"2024-01-11T07:11:57.236300+0000","last_clean":"2024-01-11T07:11:57.236300+0000","last_became_active":"2024-01-11T07:11:57.236062+0000","last_became_peered":"2024-01-11T07:11:57.236062+0000","last_unstale":"2024-01-11T07:11:57.236300+0000","last_undegraded":"2024-01-11T07:11:57.236300+0000","last_fullsized":"2024-01-11T07:11:57.236300+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:51:41.103684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.229750+0000","last_change":"2024-01-11T07:11:57.229669+0000","last_active":"2024-01-11T07:11:57.229750+0000","last_peered":"2024-01-11T07:11:57.229750+0000","last_clean":"2024-01-11T07:11:57.229750+0000","last_became_active":"2024-01-11T07:11:57.229257+0000","last_became_peered":"2024-01-11T07:11:57.229257+0000","last_unstale":"2024-01-11T07:11:57.229750+0000","last_undegraded":"2024-01-11T07:11:57.229750+0000","last_fullsized":"2024-01-11T07:11:57.229750+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:27.939787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225204+0000","last_change":"2024-01-11T07:11:57.225129+0000","last_active":"2024-01-11T07:11:57.225204+0000","last_peered":"2024-01-11T07:11:57.225204+0000","last_clean":"2024-01-11T07:11:57.225204+0000","last_became_active":"2024-01-11T07:11:57.224889+0000","last_became_peered":"2024-01-11T07:11:57.224889+0000","last_unstale":"2024-01-11T07:11:57.225204+0000","last_undegraded":"2024-01-11T07:11:57.225204+0000","last_fullsized":"2024-01-11T07:11:57.225204+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:56.072710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225444+0000","last_change":"2024-01-11T07:11:57.225321+0000","last_active":"2024-01-11T07:11:57.225444+0000","last_peered":"2024-01-11T07:11:57.225444+0000","last_clean":"2024-01-11T07:11:57.225444+0000","last_became_active":"2024-01-11T07:11:57.225088+0000","last_became_peered":"2024-01-11T07:11:57.225088+0000","last_unstale":"2024-01-11T07:11:57.225444+0000","last_undegraded":"2024-01-11T07:11:57.225444+0000","last_fullsized":"2024-01-11T07:11:57.225444+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:37:32.425877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234258+0000","last_change":"2024-01-11T07:11:57.234016+0000","last_active":"2024-01-11T07:11:57.234258+0000","last_peered":"2024-01-11T07:11:57.234258+0000","last_clean":"2024-01-11T07:11:57.234258+0000","last_became_active":"2024-01-11T07:11:57.233772+0000","last_became_peered":"2024-01-11T07:11:57.233772+0000","last_unstale":"2024-01-11T07:11:57.234258+0000","last_undegraded":"2024-01-11T07:11:57.234258+0000","last_fullsized":"2024-01-11T07:11:57.234258+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:43:25.351251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226370+0000","last_change":"2024-01-11T07:11:57.226267+0000","last_active":"2024-01-11T07:11:57.226370+0000","last_peered":"2024-01-11T07:11:57.226370+0000","last_clean":"2024-01-11T07:11:57.226370+0000","last_became_active":"2024-01-11T07:11:57.226026+0000","last_became_peered":"2024-01-11T07:11:57.226026+0000","last_unstale":"2024-01-11T07:11:57.226370+0000","last_undegraded":"2024-01-11T07:11:57.226370+0000","last_fullsized":"2024-01-11T07:11:57.226370+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:05:00.875268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232310+0000","last_change":"2024-01-11T07:11:57.232226+0000","last_active":"2024-01-11T07:11:57.232310+0000","last_peered":"2024-01-11T07:11:57.232310+0000","last_clean":"2024-01-11T07:11:57.232310+0000","last_became_active":"2024-01-11T07:11:57.232064+0000","last_became_peered":"2024-01-11T07:11:57.232064+0000","last_unstale":"2024-01-11T07:11:57.232310+0000","last_undegraded":"2024-01-11T07:11:57.232310+0000","last_fullsized":"2024-01-11T07:11:57.232310+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:01:12.447452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232680+0000","last_change":"2024-01-11T07:11:57.232432+0000","last_active":"2024-01-11T07:11:57.232680+0000","last_peered":"2024-01-11T07:11:57.232680+0000","last_clean":"2024-01-11T07:11:57.232680+0000","last_became_active":"2024-01-11T07:11:57.231656+0000","last_became_peered":"2024-01-11T07:11:57.231656+0000","last_unstale":"2024-01-11T07:11:57.232680+0000","last_undegraded":"2024-01-11T07:11:57.232680+0000","last_fullsized":"2024-01-11T07:11:57.232680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:05:49.938435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232458+0000","last_change":"2024-01-11T07:11:57.232393+0000","last_active":"2024-01-11T07:11:57.232458+0000","last_peered":"2024-01-11T07:11:57.232458+0000","last_clean":"2024-01-11T07:11:57.232458+0000","last_became_active":"2024-01-11T07:11:57.232156+0000","last_became_peered":"2024-01-11T07:11:57.232156+0000","last_unstale":"2024-01-11T07:11:57.232458+0000","last_undegraded":"2024-01-11T07:11:57.232458+0000","last_fullsized":"2024-01-11T07:11:57.232458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:48.096243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228322+0000","last_change":"2024-01-11T07:11:57.227100+0000","last_active":"2024-01-11T07:11:57.228322+0000","last_peered":"2024-01-11T07:11:57.228322+0000","last_clean":"2024-01-11T07:11:57.228322+0000","last_became_active":"2024-01-11T07:11:57.226772+0000","last_became_peered":"2024-01-11T07:11:57.226772+0000","last_unstale":"2024-01-11T07:11:57.228322+0000","last_undegraded":"2024-01-11T07:11:57.228322+0000","last_fullsized":"2024-01-11T07:11:57.228322+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:10:40.115835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.237511+0000","last_change":"2024-01-11T07:11:57.237416+0000","last_active":"2024-01-11T07:11:57.237511+0000","last_peered":"2024-01-11T07:11:57.237511+0000","last_clean":"2024-01-11T07:11:57.237511+0000","last_became_active":"2024-01-11T07:11:57.236704+0000","last_became_peered":"2024-01-11T07:11:57.236704+0000","last_unstale":"2024-01-11T07:11:57.237511+0000","last_undegraded":"2024-01-11T07:11:57.237511+0000","last_fullsized":"2024-01-11T07:11:57.237511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:01:12.447452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231088+0000","last_change":"2024-01-11T07:11:57.230826+0000","last_active":"2024-01-11T07:11:57.231088+0000","last_peered":"2024-01-11T07:11:57.231088+0000","last_clean":"2024-01-11T07:11:57.231088+0000","last_became_active":"2024-01-11T07:11:57.230459+0000","last_became_peered":"2024-01-11T07:11:57.230459+0000","last_unstale":"2024-01-11T07:11:57.231088+0000","last_undegraded":"2024-01-11T07:11:57.231088+0000","last_fullsized":"2024-01-11T07:11:57.231088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:43:45.702913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221762+0000","last_change":"2024-01-11T07:11:57.221686+0000","last_active":"2024-01-11T07:11:57.221762+0000","last_peered":"2024-01-11T07:11:57.221762+0000","last_clean":"2024-01-11T07:11:57.221762+0000","last_became_active":"2024-01-11T07:11:57.221369+0000","last_became_peered":"2024-01-11T07:11:57.221369+0000","last_unstale":"2024-01-11T07:11:57.221762+0000","last_undegraded":"2024-01-11T07:11:57.221762+0000","last_fullsized":"2024-01-11T07:11:57.221762+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:16.259881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222107+0000","last_change":"2024-01-11T07:11:57.222052+0000","last_active":"2024-01-11T07:11:57.222107+0000","last_peered":"2024-01-11T07:11:57.222107+0000","last_clean":"2024-01-11T07:11:57.222107+0000","last_became_active":"2024-01-11T07:11:57.221633+0000","last_became_peered":"2024-01-11T07:11:57.221633+0000","last_unstale":"2024-01-11T07:11:57.222107+0000","last_undegraded":"2024-01-11T07:11:57.222107+0000","last_fullsized":"2024-01-11T07:11:57.222107+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:46:20.345152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222089+0000","last_change":"2024-01-11T07:11:57.222003+0000","last_active":"2024-01-11T07:11:57.222089+0000","last_peered":"2024-01-11T07:11:57.222089+0000","last_clean":"2024-01-11T07:11:57.222089+0000","last_became_active":"2024-01-11T07:11:57.221785+0000","last_became_peered":"2024-01-11T07:11:57.221785+0000","last_unstale":"2024-01-11T07:11:57.222089+0000","last_undegraded":"2024-01-11T07:11:57.222089+0000","last_fullsized":"2024-01-11T07:11:57.222089+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:54:32.751965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231448+0000","last_change":"2024-01-11T07:11:57.231387+0000","last_active":"2024-01-11T07:11:57.231448+0000","last_peered":"2024-01-11T07:11:57.231448+0000","last_clean":"2024-01-11T07:11:57.231448+0000","last_became_active":"2024-01-11T07:11:57.230985+0000","last_became_peered":"2024-01-11T07:11:57.230985+0000","last_unstale":"2024-01-11T07:11:57.231448+0000","last_undegraded":"2024-01-11T07:11:57.231448+0000","last_fullsized":"2024-01-11T07:11:57.231448+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:19:13.036740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231512+0000","last_change":"2024-01-11T07:11:57.228276+0000","last_active":"2024-01-11T07:11:57.231512+0000","last_peered":"2024-01-11T07:11:57.231512+0000","last_clean":"2024-01-11T07:11:57.231512+0000","last_became_active":"2024-01-11T07:11:57.227438+0000","last_became_peered":"2024-01-11T07:11:57.227438+0000","last_unstale":"2024-01-11T07:11:57.231512+0000","last_undegraded":"2024-01-11T07:11:57.231512+0000","last_fullsized":"2024-01-11T07:11:57.231512+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:32:01.689019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221862+0000","last_change":"2024-01-11T07:11:57.221595+0000","last_active":"2024-01-11T07:11:57.221862+0000","last_peered":"2024-01-11T07:11:57.221862+0000","last_clean":"2024-01-11T07:11:57.221862+0000","last_became_active":"2024-01-11T07:11:57.221350+0000","last_became_peered":"2024-01-11T07:11:57.221350+0000","last_unstale":"2024-01-11T07:11:57.221862+0000","last_undegraded":"2024-01-11T07:11:57.221862+0000","last_fullsized":"2024-01-11T07:11:57.221862+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:00:08.312302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.230550+0000","last_change":"2024-01-11T07:11:57.229468+0000","last_active":"2024-01-11T07:11:57.230550+0000","last_peered":"2024-01-11T07:11:57.230550+0000","last_clean":"2024-01-11T07:11:57.230550+0000","last_became_active":"2024-01-11T07:11:57.229247+0000","last_became_peered":"2024-01-11T07:11:57.229247+0000","last_unstale":"2024-01-11T07:11:57.230550+0000","last_undegraded":"2024-01-11T07:11:57.230550+0000","last_fullsized":"2024-01-11T07:11:57.230550+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:51:06.947140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234296+0000","last_change":"2024-01-11T07:11:57.234223+0000","last_active":"2024-01-11T07:11:57.234296+0000","last_peered":"2024-01-11T07:11:57.234296+0000","last_clean":"2024-01-11T07:11:57.234296+0000","last_became_active":"2024-01-11T07:11:57.233973+0000","last_became_peered":"2024-01-11T07:11:57.233973+0000","last_unstale":"2024-01-11T07:11:57.234296+0000","last_undegraded":"2024-01-11T07:11:57.234296+0000","last_fullsized":"2024-01-11T07:11:57.234296+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:29:42.362997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.237668+0000","last_change":"2024-01-11T07:11:57.237597+0000","last_active":"2024-01-11T07:11:57.237668+0000","last_peered":"2024-01-11T07:11:57.237668+0000","last_clean":"2024-01-11T07:11:57.237668+0000","last_became_active":"2024-01-11T07:11:57.237030+0000","last_became_peered":"2024-01-11T07:11:57.237030+0000","last_unstale":"2024-01-11T07:11:57.237668+0000","last_undegraded":"2024-01-11T07:11:57.237668+0000","last_fullsized":"2024-01-11T07:11:57.237668+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:29:43.771375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231557+0000","last_change":"2024-01-11T07:11:57.229330+0000","last_active":"2024-01-11T07:11:57.231557+0000","last_peered":"2024-01-11T07:11:57.231557+0000","last_clean":"2024-01-11T07:11:57.231557+0000","last_became_active":"2024-01-11T07:11:57.229085+0000","last_became_peered":"2024-01-11T07:11:57.229085+0000","last_unstale":"2024-01-11T07:11:57.231557+0000","last_undegraded":"2024-01-11T07:11:57.231557+0000","last_fullsized":"2024-01-11T07:11:57.231557+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:09:10.736111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.230527+0000","last_change":"2024-01-11T07:11:57.228700+0000","last_active":"2024-01-11T07:11:57.230527+0000","last_peered":"2024-01-11T07:11:57.230527+0000","last_clean":"2024-01-11T07:11:57.230527+0000","last_became_active":"2024-01-11T07:11:57.228468+0000","last_became_peered":"2024-01-11T07:11:57.228468+0000","last_unstale":"2024-01-11T07:11:57.230527+0000","last_undegraded":"2024-01-11T07:11:57.230527+0000","last_fullsized":"2024-01-11T07:11:57.230527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:38:49.807723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232939+0000","last_change":"2024-01-11T07:11:57.232884+0000","last_active":"2024-01-11T07:11:57.232939+0000","last_peered":"2024-01-11T07:11:57.232939+0000","last_clean":"2024-01-11T07:11:57.232939+0000","last_became_active":"2024-01-11T07:11:57.232455+0000","last_became_peered":"2024-01-11T07:11:57.232455+0000","last_unstale":"2024-01-11T07:11:57.232939+0000","last_undegraded":"2024-01-11T07:11:57.232939+0000","last_fullsized":"2024-01-11T07:11:57.232939+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:00:27.293551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226667+0000","last_change":"2024-01-11T07:11:57.226030+0000","last_active":"2024-01-11T07:11:57.226667+0000","last_peered":"2024-01-11T07:11:57.226667+0000","last_clean":"2024-01-11T07:11:57.226667+0000","last_became_active":"2024-01-11T07:11:57.225734+0000","last_became_peered":"2024-01-11T07:11:57.225734+0000","last_unstale":"2024-01-11T07:11:57.226667+0000","last_undegraded":"2024-01-11T07:11:57.226667+0000","last_fullsized":"2024-01-11T07:11:57.226667+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:50.009991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231598+0000","last_change":"2024-01-11T07:11:57.231472+0000","last_active":"2024-01-11T07:11:57.231598+0000","last_peered":"2024-01-11T07:11:57.231598+0000","last_clean":"2024-01-11T07:11:57.231598+0000","last_became_active":"2024-01-11T07:11:57.231218+0000","last_became_peered":"2024-01-11T07:11:57.231218+0000","last_unstale":"2024-01-11T07:11:57.231598+0000","last_undegraded":"2024-01-11T07:11:57.231598+0000","last_fullsized":"2024-01-11T07:11:57.231598+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:40:17.044456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228728+0000","last_change":"2024-01-11T07:11:57.228659+0000","last_active":"2024-01-11T07:11:57.228728+0000","last_peered":"2024-01-11T07:11:57.228728+0000","last_clean":"2024-01-11T07:11:57.228728+0000","last_became_active":"2024-01-11T07:11:57.228406+0000","last_became_peered":"2024-01-11T07:11:57.228406+0000","last_unstale":"2024-01-11T07:11:57.228728+0000","last_undegraded":"2024-01-11T07:11:57.228728+0000","last_fullsized":"2024-01-11T07:11:57.228728+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:25:07.957466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"99'4","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.691980+0000","last_change":"2024-01-11T07:11:56.698594+0000","last_active":"2024-01-11T07:11:57.691980+0000","last_peered":"2024-01-11T07:11:57.691980+0000","last_clean":"2024-01-11T07:11:57.691980+0000","last_became_active":"2024-01-11T07:11:56.697387+0000","last_became_peered":"2024-01-11T07:11:56.697387+0000","last_unstale":"2024-01-11T07:11:57.691980+0000","last_undegraded":"2024-01-11T07:11:57.691980+0000","last_fullsized":"2024-01-11T07:11:57.691980+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:30:31.730592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231530+0000","last_change":"2024-01-11T07:11:57.228461+0000","last_active":"2024-01-11T07:11:57.231530+0000","last_peered":"2024-01-11T07:11:57.231530+0000","last_clean":"2024-01-11T07:11:57.231530+0000","last_became_active":"2024-01-11T07:11:57.227789+0000","last_became_peered":"2024-01-11T07:11:57.227789+0000","last_unstale":"2024-01-11T07:11:57.231530+0000","last_undegraded":"2024-01-11T07:11:57.231530+0000","last_fullsized":"2024-01-11T07:11:57.231530+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:30:24.474768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.691856+0000","last_change":"2024-01-11T07:11:56.694501+0000","last_active":"2024-01-11T07:11:57.691856+0000","last_peered":"2024-01-11T07:11:57.691856+0000","last_clean":"2024-01-11T07:11:57.691856+0000","last_became_active":"2024-01-11T07:11:56.694256+0000","last_became_peered":"2024-01-11T07:11:56.694256+0000","last_unstale":"2024-01-11T07:11:57.691856+0000","last_undegraded":"2024-01-11T07:11:57.691856+0000","last_fullsized":"2024-01-11T07:11:57.691856+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:51:05.943963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231580+0000","last_change":"2024-01-11T07:11:57.230605+0000","last_active":"2024-01-11T07:11:57.231580+0000","last_peered":"2024-01-11T07:11:57.231580+0000","last_clean":"2024-01-11T07:11:57.231580+0000","last_became_active":"2024-01-11T07:11:57.230355+0000","last_became_peered":"2024-01-11T07:11:57.230355+0000","last_unstale":"2024-01-11T07:11:57.231580+0000","last_undegraded":"2024-01-11T07:11:57.231580+0000","last_fullsized":"2024-01-11T07:11:57.231580+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:12:17.294168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.691702+0000","last_change":"2024-01-11T07:11:56.687635+0000","last_active":"2024-01-11T07:11:57.691702+0000","last_peered":"2024-01-11T07:11:57.691702+0000","last_clean":"2024-01-11T07:11:57.691702+0000","last_became_active":"2024-01-11T07:11:56.687229+0000","last_became_peered":"2024-01-11T07:11:56.687229+0000","last_unstale":"2024-01-11T07:11:57.691702+0000","last_undegraded":"2024-01-11T07:11:57.691702+0000","last_fullsized":"2024-01-11T07:11:57.691702+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:29:42.768198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233125+0000","last_change":"2024-01-11T07:11:57.233040+0000","last_active":"2024-01-11T07:11:57.233125+0000","last_peered":"2024-01-11T07:11:57.233125+0000","last_clean":"2024-01-11T07:11:57.233125+0000","last_became_active":"2024-01-11T07:11:57.232293+0000","last_became_peered":"2024-01-11T07:11:57.232293+0000","last_unstale":"2024-01-11T07:11:57.233125+0000","last_undegraded":"2024-01-11T07:11:57.233125+0000","last_fullsized":"2024-01-11T07:11:57.233125+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:09.306979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.689969+0000","last_change":"2024-01-11T07:11:56.694850+0000","last_active":"2024-01-11T07:11:57.689969+0000","last_peered":"2024-01-11T07:11:57.689969+0000","last_clean":"2024-01-11T07:11:57.689969+0000","last_became_active":"2024-01-11T07:11:56.694598+0000","last_became_peered":"2024-01-11T07:11:56.694598+0000","last_unstale":"2024-01-11T07:11:57.689969+0000","last_undegraded":"2024-01-11T07:11:57.689969+0000","last_fullsized":"2024-01-11T07:11:57.689969+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:25:06.954289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232958+0000","last_change":"2024-01-11T07:11:57.232902+0000","last_active":"2024-01-11T07:11:57.232958+0000","last_peered":"2024-01-11T07:11:57.232958+0000","last_clean":"2024-01-11T07:11:57.232958+0000","last_became_active":"2024-01-11T07:11:57.232490+0000","last_became_peered":"2024-01-11T07:11:57.232490+0000","last_unstale":"2024-01-11T07:11:57.232958+0000","last_undegraded":"2024-01-11T07:11:57.232958+0000","last_fullsized":"2024-01-11T07:11:57.232958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:12.903076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.690569+0000","last_change":"2024-01-11T07:11:56.695426+0000","last_active":"2024-01-11T07:11:57.690569+0000","last_peered":"2024-01-11T07:11:57.690569+0000","last_clean":"2024-01-11T07:11:57.690569+0000","last_became_active":"2024-01-11T07:11:56.695111+0000","last_became_peered":"2024-01-11T07:11:56.695111+0000","last_unstale":"2024-01-11T07:11:57.690569+0000","last_undegraded":"2024-01-11T07:11:57.690569+0000","last_fullsized":"2024-01-11T07:11:57.690569+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:19:12.033563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.237360+0000","last_change":"2024-01-11T07:11:57.237243+0000","last_active":"2024-01-11T07:11:57.237360+0000","last_peered":"2024-01-11T07:11:57.237360+0000","last_clean":"2024-01-11T07:11:57.237360+0000","last_became_active":"2024-01-11T07:11:57.236378+0000","last_became_peered":"2024-01-11T07:11:57.236378+0000","last_unstale":"2024-01-11T07:11:57.237360+0000","last_undegraded":"2024-01-11T07:11:57.237360+0000","last_fullsized":"2024-01-11T07:11:57.237360+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:29:16.174430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692570+0000","last_change":"2024-01-11T07:11:56.692379+0000","last_active":"2024-01-11T07:11:57.692570+0000","last_peered":"2024-01-11T07:11:57.692570+0000","last_clean":"2024-01-11T07:11:57.692570+0000","last_became_active":"2024-01-11T07:11:56.692199+0000","last_became_peered":"2024-01-11T07:11:56.692199+0000","last_unstale":"2024-01-11T07:11:57.692570+0000","last_undegraded":"2024-01-11T07:11:57.692570+0000","last_fullsized":"2024-01-11T07:11:57.692570+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:35:57.387820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233028+0000","last_change":"2024-01-11T07:11:57.232849+0000","last_active":"2024-01-11T07:11:57.233028+0000","last_peered":"2024-01-11T07:11:57.233028+0000","last_clean":"2024-01-11T07:11:57.233028+0000","last_became_active":"2024-01-11T07:11:57.231931+0000","last_became_peered":"2024-01-11T07:11:57.231931+0000","last_unstale":"2024-01-11T07:11:57.233028+0000","last_undegraded":"2024-01-11T07:11:57.233028+0000","last_fullsized":"2024-01-11T07:11:57.233028+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:08:28.809018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.218012+0000","last_change":"2024-01-11T07:11:56.688466+0000","last_active":"2024-01-11T07:11:57.218012+0000","last_peered":"2024-01-11T07:11:57.218012+0000","last_clean":"2024-01-11T07:11:57.218012+0000","last_became_active":"2024-01-11T07:11:56.688187+0000","last_became_peered":"2024-01-11T07:11:56.688187+0000","last_unstale":"2024-01-11T07:11:57.218012+0000","last_undegraded":"2024-01-11T07:11:57.218012+0000","last_fullsized":"2024-01-11T07:11:57.218012+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:01:11.444275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228056+0000","last_change":"2024-01-11T07:11:57.227422+0000","last_active":"2024-01-11T07:11:57.228056+0000","last_peered":"2024-01-11T07:11:57.228056+0000","last_clean":"2024-01-11T07:11:57.228056+0000","last_became_active":"2024-01-11T07:11:57.227196+0000","last_became_peered":"2024-01-11T07:11:57.227196+0000","last_unstale":"2024-01-11T07:11:57.228056+0000","last_undegraded":"2024-01-11T07:11:57.228056+0000","last_fullsized":"2024-01-11T07:11:57.228056+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:36:15.603141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228867+0000","last_change":"2024-01-11T07:11:56.694343+0000","last_active":"2024-01-11T07:11:57.228867+0000","last_peered":"2024-01-11T07:11:57.228867+0000","last_clean":"2024-01-11T07:11:57.228867+0000","last_became_active":"2024-01-11T07:11:56.693989+0000","last_became_peered":"2024-01-11T07:11:56.693989+0000","last_unstale":"2024-01-11T07:11:57.228867+0000","last_undegraded":"2024-01-11T07:11:57.228867+0000","last_fullsized":"2024-01-11T07:11:57.228867+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:47:28.786603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233611+0000","last_change":"2024-01-11T07:11:57.233400+0000","last_active":"2024-01-11T07:11:57.233611+0000","last_peered":"2024-01-11T07:11:57.233611+0000","last_clean":"2024-01-11T07:11:57.233611+0000","last_became_active":"2024-01-11T07:11:57.233156+0000","last_became_peered":"2024-01-11T07:11:57.233156+0000","last_unstale":"2024-01-11T07:11:57.233611+0000","last_undegraded":"2024-01-11T07:11:57.233611+0000","last_fullsized":"2024-01-11T07:11:57.233611+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:16.259881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222289+0000","last_change":"2024-01-11T07:11:56.693933+0000","last_active":"2024-01-11T07:11:57.222289+0000","last_peered":"2024-01-11T07:11:57.222289+0000","last_clean":"2024-01-11T07:11:57.222289+0000","last_became_active":"2024-01-11T07:11:56.693775+0000","last_became_peered":"2024-01-11T07:11:56.693775+0000","last_unstale":"2024-01-11T07:11:57.222289+0000","last_undegraded":"2024-01-11T07:11:57.222289+0000","last_fullsized":"2024-01-11T07:11:57.222289+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:43:24.348074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234194+0000","last_change":"2024-01-11T07:11:57.234081+0000","last_active":"2024-01-11T07:11:57.234194+0000","last_peered":"2024-01-11T07:11:57.234194+0000","last_clean":"2024-01-11T07:11:57.234194+0000","last_became_active":"2024-01-11T07:11:57.233837+0000","last_became_peered":"2024-01-11T07:11:57.233837+0000","last_unstale":"2024-01-11T07:11:57.234194+0000","last_undegraded":"2024-01-11T07:11:57.234194+0000","last_fullsized":"2024-01-11T07:11:57.234194+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:50.009991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225398+0000","last_change":"2024-01-11T07:11:56.695843+0000","last_active":"2024-01-11T07:11:57.225398+0000","last_peered":"2024-01-11T07:11:57.225398+0000","last_clean":"2024-01-11T07:11:57.225398+0000","last_became_active":"2024-01-11T07:11:56.695572+0000","last_became_peered":"2024-01-11T07:11:56.695572+0000","last_unstale":"2024-01-11T07:11:57.225398+0000","last_undegraded":"2024-01-11T07:11:57.225398+0000","last_fullsized":"2024-01-11T07:11:57.225398+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:05:37.725631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225051+0000","last_change":"2024-01-11T07:11:57.224709+0000","last_active":"2024-01-11T07:11:57.225051+0000","last_peered":"2024-01-11T07:11:57.225051+0000","last_clean":"2024-01-11T07:11:57.225051+0000","last_became_active":"2024-01-11T07:11:57.224498+0000","last_became_peered":"2024-01-11T07:11:57.224498+0000","last_unstale":"2024-01-11T07:11:57.225051+0000","last_undegraded":"2024-01-11T07:11:57.225051+0000","last_fullsized":"2024-01-11T07:11:57.225051+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:50:54.352317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"99'3","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692905+0000","last_change":"2024-01-11T07:11:56.698414+0000","last_active":"2024-01-11T07:11:57.692905+0000","last_peered":"2024-01-11T07:11:57.692905+0000","last_clean":"2024-01-11T07:11:57.692905+0000","last_became_active":"2024-01-11T07:11:56.696842+0000","last_became_peered":"2024-01-11T07:11:56.696842+0000","last_unstale":"2024-01-11T07:11:57.692905+0000","last_undegraded":"2024-01-11T07:11:57.692905+0000","last_fullsized":"2024-01-11T07:11:57.692905+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:03.139204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.230518+0000","last_change":"2024-01-11T07:11:57.230455+0000","last_active":"2024-01-11T07:11:57.230518+0000","last_peered":"2024-01-11T07:11:57.230518+0000","last_clean":"2024-01-11T07:11:57.230518+0000","last_became_active":"2024-01-11T07:11:57.230217+0000","last_became_peered":"2024-01-11T07:11:57.230217+0000","last_unstale":"2024-01-11T07:11:57.230518+0000","last_undegraded":"2024-01-11T07:11:57.230518+0000","last_fullsized":"2024-01-11T07:11:57.230518+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:27:27.939787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"99'6","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.693798+0000","last_change":"2024-01-11T07:11:56.688525+0000","last_active":"2024-01-11T07:11:57.693798+0000","last_peered":"2024-01-11T07:11:57.693798+0000","last_clean":"2024-01-11T07:11:57.693798+0000","last_became_active":"2024-01-11T07:11:56.688250+0000","last_became_peered":"2024-01-11T07:11:56.688250+0000","last_unstale":"2024-01-11T07:11:57.693798+0000","last_undegraded":"2024-01-11T07:11:57.693798+0000","last_fullsized":"2024-01-11T07:11:57.693798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:11.899899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.229811+0000","last_change":"2024-01-11T07:11:57.229474+0000","last_active":"2024-01-11T07:11:57.229811+0000","last_peered":"2024-01-11T07:11:57.229811+0000","last_clean":"2024-01-11T07:11:57.229811+0000","last_became_active":"2024-01-11T07:11:57.228881+0000","last_became_peered":"2024-01-11T07:11:57.228881+0000","last_unstale":"2024-01-11T07:11:57.229811+0000","last_undegraded":"2024-01-11T07:11:57.229811+0000","last_fullsized":"2024-01-11T07:11:57.229811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:53:15.929249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.690371+0000","last_change":"2024-01-11T07:11:56.685658+0000","last_active":"2024-01-11T07:11:57.690371+0000","last_peered":"2024-01-11T07:11:57.690371+0000","last_clean":"2024-01-11T07:11:57.690371+0000","last_became_active":"2024-01-11T07:11:56.685409+0000","last_became_peered":"2024-01-11T07:11:56.685409+0000","last_unstale":"2024-01-11T07:11:57.690371+0000","last_undegraded":"2024-01-11T07:11:57.690371+0000","last_fullsized":"2024-01-11T07:11:57.690371+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:45:19.318830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233423+0000","last_change":"2024-01-11T07:11:57.233342+0000","last_active":"2024-01-11T07:11:57.233423+0000","last_peered":"2024-01-11T07:11:57.233423+0000","last_clean":"2024-01-11T07:11:57.233423+0000","last_became_active":"2024-01-11T07:11:57.232626+0000","last_became_peered":"2024-01-11T07:11:57.232626+0000","last_unstale":"2024-01-11T07:11:57.233423+0000","last_undegraded":"2024-01-11T07:11:57.233423+0000","last_fullsized":"2024-01-11T07:11:57.233423+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:29:16.174430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221022+0000","last_change":"2024-01-11T07:11:56.685620+0000","last_active":"2024-01-11T07:11:57.221022+0000","last_peered":"2024-01-11T07:11:57.221022+0000","last_clean":"2024-01-11T07:11:57.221022+0000","last_became_active":"2024-01-11T07:11:56.685344+0000","last_became_peered":"2024-01-11T07:11:56.685344+0000","last_unstale":"2024-01-11T07:11:57.221022+0000","last_undegraded":"2024-01-11T07:11:57.221022+0000","last_fullsized":"2024-01-11T07:11:57.221022+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:53:43.075643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232813+0000","last_change":"2024-01-11T07:11:57.232740+0000","last_active":"2024-01-11T07:11:57.232813+0000","last_peered":"2024-01-11T07:11:57.232813+0000","last_clean":"2024-01-11T07:11:57.232813+0000","last_became_active":"2024-01-11T07:11:57.232119+0000","last_became_peered":"2024-01-11T07:11:57.232119+0000","last_unstale":"2024-01-11T07:11:57.232813+0000","last_undegraded":"2024-01-11T07:11:57.232813+0000","last_fullsized":"2024-01-11T07:11:57.232813+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:35:58.390997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.224329+0000","last_change":"2024-01-11T07:11:56.694765+0000","last_active":"2024-01-11T07:11:57.224329+0000","last_peered":"2024-01-11T07:11:57.224329+0000","last_clean":"2024-01-11T07:11:57.224329+0000","last_became_active":"2024-01-11T07:11:56.694508+0000","last_became_peered":"2024-01-11T07:11:56.694508+0000","last_unstale":"2024-01-11T07:11:57.224329+0000","last_undegraded":"2024-01-11T07:11:57.224329+0000","last_fullsized":"2024-01-11T07:11:57.224329+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:00:26.290374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.235371+0000","last_change":"2024-01-11T07:11:57.235283+0000","last_active":"2024-01-11T07:11:57.235371+0000","last_peered":"2024-01-11T07:11:57.235371+0000","last_clean":"2024-01-11T07:11:57.235371+0000","last_became_active":"2024-01-11T07:11:57.234817+0000","last_became_peered":"2024-01-11T07:11:57.234817+0000","last_unstale":"2024-01-11T07:11:57.235371+0000","last_undegraded":"2024-01-11T07:11:57.235371+0000","last_fullsized":"2024-01-11T07:11:57.235371+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:32.679088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"99'8","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692997+0000","last_change":"2024-01-11T07:11:56.691703+0000","last_active":"2024-01-11T07:11:57.692997+0000","last_peered":"2024-01-11T07:11:57.692997+0000","last_clean":"2024-01-11T07:11:57.692997+0000","last_became_active":"2024-01-11T07:11:56.691388+0000","last_became_peered":"2024-01-11T07:11:56.691388+0000","last_unstale":"2024-01-11T07:11:57.692997+0000","last_undegraded":"2024-01-11T07:11:57.692997+0000","last_fullsized":"2024-01-11T07:11:57.692997+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:30:23.471591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.234636+0000","last_change":"2024-01-11T07:11:57.234567+0000","last_active":"2024-01-11T07:11:57.234636+0000","last_peered":"2024-01-11T07:11:57.234636+0000","last_clean":"2024-01-11T07:11:57.234636+0000","last_became_active":"2024-01-11T07:11:57.234333+0000","last_became_peered":"2024-01-11T07:11:57.234333+0000","last_unstale":"2024-01-11T07:11:57.234636+0000","last_undegraded":"2024-01-11T07:11:57.234636+0000","last_fullsized":"2024-01-11T07:11:57.234636+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:19:15.424694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.688601+0000","last_change":"2024-01-11T07:11:56.689612+0000","last_active":"2024-01-11T07:11:57.688601+0000","last_peered":"2024-01-11T07:11:57.688601+0000","last_clean":"2024-01-11T07:11:57.688601+0000","last_became_active":"2024-01-11T07:11:56.689123+0000","last_became_peered":"2024-01-11T07:11:56.689123+0000","last_unstale":"2024-01-11T07:11:57.688601+0000","last_undegraded":"2024-01-11T07:11:57.688601+0000","last_fullsized":"2024-01-11T07:11:57.688601+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:22:22.610321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.236910+0000","last_change":"2024-01-11T07:11:57.236847+0000","last_active":"2024-01-11T07:11:57.236910+0000","last_peered":"2024-01-11T07:11:57.236910+0000","last_clean":"2024-01-11T07:11:57.236910+0000","last_became_active":"2024-01-11T07:11:57.236589+0000","last_became_peered":"2024-01-11T07:11:57.236589+0000","last_unstale":"2024-01-11T07:11:57.236910+0000","last_undegraded":"2024-01-11T07:11:57.236910+0000","last_fullsized":"2024-01-11T07:11:57.236910+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:56.349066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692343+0000","last_change":"2024-01-11T07:11:56.691137+0000","last_active":"2024-01-11T07:11:57.692343+0000","last_peered":"2024-01-11T07:11:57.692343+0000","last_clean":"2024-01-11T07:11:57.692343+0000","last_became_active":"2024-01-11T07:11:56.690819+0000","last_became_peered":"2024-01-11T07:11:56.690819+0000","last_unstale":"2024-01-11T07:11:57.692343+0000","last_undegraded":"2024-01-11T07:11:57.692343+0000","last_fullsized":"2024-01-11T07:11:57.692343+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:19:14.421517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231325+0000","last_change":"2024-01-11T07:11:57.231073+0000","last_active":"2024-01-11T07:11:57.231325+0000","last_peered":"2024-01-11T07:11:57.231325+0000","last_clean":"2024-01-11T07:11:57.231325+0000","last_became_active":"2024-01-11T07:11:57.230354+0000","last_became_peered":"2024-01-11T07:11:57.230354+0000","last_unstale":"2024-01-11T07:11:57.231325+0000","last_undegraded":"2024-01-11T07:11:57.231325+0000","last_fullsized":"2024-01-11T07:11:57.231325+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:30:24.474768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.217497+0000","last_change":"2024-01-11T07:11:56.687904+0000","last_active":"2024-01-11T07:11:57.217497+0000","last_peered":"2024-01-11T07:11:57.217497+0000","last_clean":"2024-01-11T07:11:57.217497+0000","last_became_active":"2024-01-11T07:11:56.687649+0000","last_became_peered":"2024-01-11T07:11:56.687649+0000","last_unstale":"2024-01-11T07:11:57.217497+0000","last_undegraded":"2024-01-11T07:11:57.217497+0000","last_fullsized":"2024-01-11T07:11:57.217497+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:29:15.171253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.232661+0000","last_change":"2024-01-11T07:11:57.232260+0000","last_active":"2024-01-11T07:11:57.232661+0000","last_peered":"2024-01-11T07:11:57.232661+0000","last_clean":"2024-01-11T07:11:57.232661+0000","last_became_active":"2024-01-11T07:11:57.231328+0000","last_became_peered":"2024-01-11T07:11:57.231328+0000","last_unstale":"2024-01-11T07:11:57.232661+0000","last_undegraded":"2024-01-11T07:11:57.232661+0000","last_fullsized":"2024-01-11T07:11:57.232661+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:09:49.762710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.692001+0000","last_change":"2024-01-11T07:11:56.687885+0000","last_active":"2024-01-11T07:11:57.692001+0000","last_peered":"2024-01-11T07:11:57.692001+0000","last_clean":"2024-01-11T07:11:57.692001+0000","last_became_active":"2024-01-11T07:11:56.687610+0000","last_became_peered":"2024-01-11T07:11:56.687610+0000","last_unstale":"2024-01-11T07:11:57.692001+0000","last_undegraded":"2024-01-11T07:11:57.692001+0000","last_fullsized":"2024-01-11T07:11:57.692001+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:08.303802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226088+0000","last_change":"2024-01-11T07:11:57.226019+0000","last_active":"2024-01-11T07:11:57.226088+0000","last_peered":"2024-01-11T07:11:57.226088+0000","last_clean":"2024-01-11T07:11:57.226088+0000","last_became_active":"2024-01-11T07:11:57.225786+0000","last_became_peered":"2024-01-11T07:11:57.225786+0000","last_unstale":"2024-01-11T07:11:57.226088+0000","last_undegraded":"2024-01-11T07:11:57.226088+0000","last_fullsized":"2024-01-11T07:11:57.226088+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:05:00.875268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.220697+0000","last_change":"2024-01-11T07:11:56.687901+0000","last_active":"2024-01-11T07:11:57.220697+0000","last_peered":"2024-01-11T07:11:57.220697+0000","last_clean":"2024-01-11T07:11:57.220697+0000","last_became_active":"2024-01-11T07:11:56.687645+0000","last_became_peered":"2024-01-11T07:11:56.687645+0000","last_unstale":"2024-01-11T07:11:57.220697+0000","last_undegraded":"2024-01-11T07:11:57.220697+0000","last_fullsized":"2024-01-11T07:11:57.220697+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:01:37.696909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.233599+0000","last_change":"2024-01-11T07:11:57.233460+0000","last_active":"2024-01-11T07:11:57.233599+0000","last_peered":"2024-01-11T07:11:57.233599+0000","last_clean":"2024-01-11T07:11:57.233599+0000","last_became_active":"2024-01-11T07:11:57.229127+0000","last_became_peered":"2024-01-11T07:11:57.229127+0000","last_unstale":"2024-01-11T07:11:57.233599+0000","last_undegraded":"2024-01-11T07:11:57.233599+0000","last_fullsized":"2024-01-11T07:11:57.233599+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:25:31.452694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.219525+0000","last_change":"2024-01-11T07:11:56.695920+0000","last_active":"2024-01-11T07:11:57.219525+0000","last_peered":"2024-01-11T07:11:57.219525+0000","last_clean":"2024-01-11T07:11:57.219525+0000","last_became_active":"2024-01-11T07:11:56.695671+0000","last_became_peered":"2024-01-11T07:11:56.695671+0000","last_unstale":"2024-01-11T07:11:57.219525+0000","last_undegraded":"2024-01-11T07:11:57.219525+0000","last_fullsized":"2024-01-11T07:11:57.219525+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:34:04.132443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.227680+0000","last_change":"2024-01-11T07:11:57.227515+0000","last_active":"2024-01-11T07:11:57.227680+0000","last_peered":"2024-01-11T07:11:57.227680+0000","last_clean":"2024-01-11T07:11:57.227680+0000","last_became_active":"2024-01-11T07:11:57.227290+0000","last_became_peered":"2024-01-11T07:11:57.227290+0000","last_unstale":"2024-01-11T07:11:57.227680+0000","last_undegraded":"2024-01-11T07:11:57.227680+0000","last_fullsized":"2024-01-11T07:11:57.227680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:37:32.425877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221112+0000","last_change":"2024-01-11T07:11:56.692052+0000","last_active":"2024-01-11T07:11:57.221112+0000","last_peered":"2024-01-11T07:11:57.221112+0000","last_clean":"2024-01-11T07:11:57.221112+0000","last_became_active":"2024-01-11T07:11:56.691786+0000","last_became_peered":"2024-01-11T07:11:56.691786+0000","last_unstale":"2024-01-11T07:11:57.221112+0000","last_undegraded":"2024-01-11T07:11:57.221112+0000","last_fullsized":"2024-01-11T07:11:57.221112+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:08:27.805841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228364+0000","last_change":"2024-01-11T07:11:57.228255+0000","last_active":"2024-01-11T07:11:57.228364+0000","last_peered":"2024-01-11T07:11:57.228364+0000","last_clean":"2024-01-11T07:11:57.228364+0000","last_became_active":"2024-01-11T07:11:57.227998+0000","last_became_peered":"2024-01-11T07:11:57.227998+0000","last_unstale":"2024-01-11T07:11:57.228364+0000","last_undegraded":"2024-01-11T07:11:57.228364+0000","last_fullsized":"2024-01-11T07:11:57.228364+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:10:56.072710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"99'2","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.690264+0000","last_change":"2024-01-11T07:11:56.690344+0000","last_active":"2024-01-11T07:11:57.690264+0000","last_peered":"2024-01-11T07:11:57.690264+0000","last_clean":"2024-01-11T07:11:57.690264+0000","last_became_active":"2024-01-11T07:11:56.689769+0000","last_became_peered":"2024-01-11T07:11:56.689769+0000","last_unstale":"2024-01-11T07:11:57.690264+0000","last_undegraded":"2024-01-11T07:11:57.690264+0000","last_fullsized":"2024-01-11T07:11:57.690264+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:34:22.736536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.236882+0000","last_change":"2024-01-11T07:11:57.236813+0000","last_active":"2024-01-11T07:11:57.236882+0000","last_peered":"2024-01-11T07:11:57.236882+0000","last_clean":"2024-01-11T07:11:57.236882+0000","last_became_active":"2024-01-11T07:11:57.236512+0000","last_became_peered":"2024-01-11T07:11:57.236512+0000","last_unstale":"2024-01-11T07:11:57.236882+0000","last_undegraded":"2024-01-11T07:11:57.236882+0000","last_fullsized":"2024-01-11T07:11:57.236882+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:44:39.925336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226425+0000","last_change":"2024-01-11T07:11:56.691047+0000","last_active":"2024-01-11T07:11:57.226425+0000","last_peered":"2024-01-11T07:11:57.226425+0000","last_clean":"2024-01-11T07:11:57.226425+0000","last_became_active":"2024-01-11T07:11:56.690095+0000","last_became_peered":"2024-01-11T07:11:56.690095+0000","last_unstale":"2024-01-11T07:11:57.226425+0000","last_undegraded":"2024-01-11T07:11:57.226425+0000","last_fullsized":"2024-01-11T07:11:57.226425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:43:44.699736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225236+0000","last_change":"2024-01-11T07:11:57.224157+0000","last_active":"2024-01-11T07:11:57.225236+0000","last_peered":"2024-01-11T07:11:57.225236+0000","last_clean":"2024-01-11T07:11:57.225236+0000","last_became_active":"2024-01-11T07:11:57.223907+0000","last_became_peered":"2024-01-11T07:11:57.223907+0000","last_unstale":"2024-01-11T07:11:57.225236+0000","last_undegraded":"2024-01-11T07:11:57.225236+0000","last_fullsized":"2024-01-11T07:11:57.225236+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:33:11.847550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222839+0000","last_change":"2024-01-11T07:11:56.695761+0000","last_active":"2024-01-11T07:11:57.222839+0000","last_peered":"2024-01-11T07:11:57.222839+0000","last_clean":"2024-01-11T07:11:57.222839+0000","last_became_active":"2024-01-11T07:11:56.695509+0000","last_became_peered":"2024-01-11T07:11:56.695509+0000","last_unstale":"2024-01-11T07:11:57.222839+0000","last_undegraded":"2024-01-11T07:11:57.222839+0000","last_fullsized":"2024-01-11T07:11:57.222839+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:15:55.700740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.226929+0000","last_change":"2024-01-11T07:11:57.226836+0000","last_active":"2024-01-11T07:11:57.226929+0000","last_peered":"2024-01-11T07:11:57.226929+0000","last_clean":"2024-01-11T07:11:57.226929+0000","last_became_active":"2024-01-11T07:11:57.226544+0000","last_became_peered":"2024-01-11T07:11:57.226544+0000","last_unstale":"2024-01-11T07:11:57.226929+0000","last_undegraded":"2024-01-11T07:11:57.226929+0000","last_fullsized":"2024-01-11T07:11:57.226929+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:19:15.424694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.222093+0000","last_change":"2024-01-11T07:11:56.692905+0000","last_active":"2024-01-11T07:11:57.222093+0000","last_peered":"2024-01-11T07:11:57.222093+0000","last_clean":"2024-01-11T07:11:57.222093+0000","last_became_active":"2024-01-11T07:11:56.692682+0000","last_became_peered":"2024-01-11T07:11:56.692682+0000","last_unstale":"2024-01-11T07:11:57.222093+0000","last_undegraded":"2024-01-11T07:11:57.222093+0000","last_fullsized":"2024-01-11T07:11:57.222093+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:27:52.893353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.225298+0000","last_change":"2024-01-11T07:11:57.225237+0000","last_active":"2024-01-11T07:11:57.225298+0000","last_peered":"2024-01-11T07:11:57.225298+0000","last_clean":"2024-01-11T07:11:57.225298+0000","last_became_active":"2024-01-11T07:11:57.224644+0000","last_became_peered":"2024-01-11T07:11:57.224644+0000","last_unstale":"2024-01-11T07:11:57.225298+0000","last_undegraded":"2024-01-11T07:11:57.225298+0000","last_fullsized":"2024-01-11T07:11:57.225298+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:15:50.159042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.218186+0000","last_change":"2024-01-11T07:11:56.691350+0000","last_active":"2024-01-11T07:11:57.218186+0000","last_peered":"2024-01-11T07:11:57.218186+0000","last_clean":"2024-01-11T07:11:57.218186+0000","last_became_active":"2024-01-11T07:11:56.691069+0000","last_became_peered":"2024-01-11T07:11:56.691069+0000","last_unstale":"2024-01-11T07:11:57.218186+0000","last_undegraded":"2024-01-11T07:11:57.218186+0000","last_fullsized":"2024-01-11T07:11:57.218186+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:36:00.714584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.229792+0000","last_change":"2024-01-11T07:11:57.229330+0000","last_active":"2024-01-11T07:11:57.229792+0000","last_peered":"2024-01-11T07:11:57.229792+0000","last_clean":"2024-01-11T07:11:57.229792+0000","last_became_active":"2024-01-11T07:11:57.228547+0000","last_became_peered":"2024-01-11T07:11:57.228547+0000","last_unstale":"2024-01-11T07:11:57.229792+0000","last_undegraded":"2024-01-11T07:11:57.229792+0000","last_fullsized":"2024-01-11T07:11:57.229792+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:32:54.350786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.229911+0000","last_change":"2024-01-11T07:11:56.694628+0000","last_active":"2024-01-11T07:11:57.229911+0000","last_peered":"2024-01-11T07:11:57.229911+0000","last_clean":"2024-01-11T07:11:57.229911+0000","last_became_active":"2024-01-11T07:11:56.694311+0000","last_became_peered":"2024-01-11T07:11:56.694311+0000","last_unstale":"2024-01-11T07:11:57.229911+0000","last_undegraded":"2024-01-11T07:11:57.229911+0000","last_fullsized":"2024-01-11T07:11:57.229911+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:11.899899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.236975+0000","last_change":"2024-01-11T07:11:57.236855+0000","last_active":"2024-01-11T07:11:57.236975+0000","last_peered":"2024-01-11T07:11:57.236975+0000","last_clean":"2024-01-11T07:11:57.236975+0000","last_became_active":"2024-01-11T07:11:57.236606+0000","last_became_peered":"2024-01-11T07:11:57.236606+0000","last_unstale":"2024-01-11T07:11:57.236975+0000","last_undegraded":"2024-01-11T07:11:57.236975+0000","last_fullsized":"2024-01-11T07:11:57.236975+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:53:44.078820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.224083+0000","last_change":"2024-01-11T07:11:56.698022+0000","last_active":"2024-01-11T07:11:57.224083+0000","last_peered":"2024-01-11T07:11:57.224083+0000","last_clean":"2024-01-11T07:11:57.224083+0000","last_became_active":"2024-01-11T07:11:56.696485+0000","last_became_peered":"2024-01-11T07:11:56.696485+0000","last_unstale":"2024-01-11T07:11:57.224083+0000","last_undegraded":"2024-01-11T07:11:57.224083+0000","last_fullsized":"2024-01-11T07:11:57.224083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:52:11.313063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.228207+0000","last_change":"2024-01-11T07:11:57.228122+0000","last_active":"2024-01-11T07:11:57.228207+0000","last_peered":"2024-01-11T07:11:57.228207+0000","last_clean":"2024-01-11T07:11:57.228207+0000","last_became_active":"2024-01-11T07:11:57.226994+0000","last_became_peered":"2024-01-11T07:11:57.226994+0000","last_unstale":"2024-01-11T07:11:57.228207+0000","last_undegraded":"2024-01-11T07:11:57.228207+0000","last_fullsized":"2024-01-11T07:11:57.228207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:36:01.717761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.221950+0000","last_change":"2024-01-11T07:11:56.696114+0000","last_active":"2024-01-11T07:11:57.221950+0000","last_peered":"2024-01-11T07:11:57.221950+0000","last_clean":"2024-01-11T07:11:57.221950+0000","last_became_active":"2024-01-11T07:11:56.695944+0000","last_became_peered":"2024-01-11T07:11:56.695944+0000","last_unstale":"2024-01-11T07:11:57.221950+0000","last_undegraded":"2024-01-11T07:11:57.221950+0000","last_fullsized":"2024-01-11T07:11:57.221950+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:00:26.290374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.231605+0000","last_change":"2024-01-11T07:11:57.231554+0000","last_active":"2024-01-11T07:11:57.231605+0000","last_peered":"2024-01-11T07:11:57.231605+0000","last_clean":"2024-01-11T07:11:57.231605+0000","last_became_active":"2024-01-11T07:11:57.231219+0000","last_became_peered":"2024-01-11T07:11:57.231219+0000","last_unstale":"2024-01-11T07:11:57.231605+0000","last_undegraded":"2024-01-11T07:11:57.231605+0000","last_fullsized":"2024-01-11T07:11:57.231605+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:56.675174+0000","last_clean_scrub_stamp":"2024-01-11T07:11:56.675174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:22:17.401294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"99'1","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.690660+0000","last_change":"2024-01-11T07:11:56.696768+0000","last_active":"2024-01-11T07:11:57.690660+0000","last_peered":"2024-01-11T07:11:57.690660+0000","last_clean":"2024-01-11T07:11:57.690660+0000","last_became_active":"2024-01-11T07:11:56.696494+0000","last_became_peered":"2024-01-11T07:11:56.696494+0000","last_unstale":"2024-01-11T07:11:57.690660+0000","last_undegraded":"2024-01-11T07:11:57.690660+0000","last_fullsized":"2024-01-11T07:11:57.690660+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:11:55.671997+0000","last_clean_scrub_stamp":"2024-01-11T07:11:55.671997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:15:49.155865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-11T07:11:57.224099+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:11:57.224099+0000","last_peered":"2024-01-11T07:11:57.224099+0000","last_clean":"2024-01-11T07:11:57.224099+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:11:57.224099+0000","last_undegraded":"2024-01-11T07:11:57.224099+0000","last_fullsized":"2024-01-11T07:11:57.224099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1102,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194209,"statfs":{"total":200000000,"available":198871004,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64400000000000002,"5min":0.754,"15min":0.754},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.78000000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70099999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.75900000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.53300000000000003},"min":{"1min":0.46500000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76100000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.747,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.68700000000000006}]},{"osd":4,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48299999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77900000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.623},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79900000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82899999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194792,"statfs":{"total":200000000,"available":199467064,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.42399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68000000000000005,"5min":1.157,"15min":1.157},"last":0.64700000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.622},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70799999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69399999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51800000000000002,"15min":0.54600000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66600000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53400000000000003,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59699999999999998,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.53300000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67800000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71299999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194792,"statfs":{"total":200000000,"available":199467536,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.60699999999999998,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56699999999999995,"5min":20.41,"15min":20.41},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.57199999999999995,"15min":0.46600000000000003},"min":{"1min":0.313,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69199999999999995,"5min":15.611000000000001,"15min":15.611000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.69099999999999995,"15min":0.54900000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65300000000000002,"5min":20.199999999999999,"15min":20.199999999999999},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.66800000000000004,"15min":0.53900000000000003},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":20.369,"15min":20.369},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42699999999999999,"15min":0.437},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61399999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.70499999999999996,"15min":0.64300000000000002},"min":{"1min":0.41399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.83099999999999996,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.748,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85799999999999998,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":0.72899999999999998,"5min":1.413,"15min":1.413},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72699999999999998,"15min":0.67400000000000004},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95299999999999996,"5min":9.891,"15min":9.891},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.502},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.746,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.88700000000000001,"15min":0.77100000000000002},"min":{"1min":0.48199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.91000000000000003,"5min":20.324000000000002,"15min":20.324000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69499999999999995,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92600000000000005,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.46999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1096,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194216,"statfs":{"total":200000000,"available":198877328,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.438,"15min":0.47599999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.628,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.495},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59999999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.627}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.57699999999999996,"15min":0.73599999999999999},"min":{"1min":0.29799999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57099999999999995,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48399999999999999,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.61799999999999999,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.58499999999999996,"15min":0.752},"min":{"1min":0.32000000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59699999999999998,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53300000000000003,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70099999999999996,"5min":2.855,"15min":2.855},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.872,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.753},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.77400000000000002,"15min":0.91400000000000003},"min":{"1min":0.46600000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94399999999999995,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.67100000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68700000000000006,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97099999999999997,"5min":2.819,"15min":2.819},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84499999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.82099999999999995,"15min":0.97499999999999998},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.79000000000000004,"15min":0.77900000000000003},"min":{"1min":0.52500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0249999999999999,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.82599999999999996,"15min":0.96499999999999997},"min":{"1min":0.45900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0429999999999999,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.83599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194792,"statfs":{"total":200000000,"available":199467176,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66800000000000004,"5min":1.472,"15min":1.472},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.439,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58199999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.59499999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.74399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73499999999999999,"5min":1.139,"15min":1.139},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68999999999999995,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0229999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.52900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98199999999999998,"5min":1.052,"15min":1.052},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67000000000000004,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.624},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":527,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194785,"statfs":{"total":200000000,"available":199459906,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.50600000000000001,"15min":0.78700000000000003},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":11.481,"15min":11.481},"last":0.378},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.48299999999999998},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.55900000000000005,"15min":0.97699999999999998},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60799999999999998,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.30199999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.56799999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.63,"15min":1.034},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.40899999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91300000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.84899999999999998,"15min":1.2669999999999999},"min":{"1min":0.38500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84299999999999997,"5min":17.474,"15min":17.474},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.78900000000000003,"15min":1.2010000000000001},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.754,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.877,"5min":11.394,"15min":11.394},"last":0.81899999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.76500000000000001,"15min":1.0309999999999999},"min":{"1min":0.498,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.55200000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.84399999999999997,"15min":1.238},"min":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1839999999999999,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.65800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":519,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194792,"statfs":{"total":200000000,"available":199467730,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.89000000000000001,"15min":1.0940000000000001},"min":{"1min":0.46999999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.77400000000000002,"5min":8.484,"15min":8.484},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.91700000000000004,"15min":1.1539999999999999},"min":{"1min":0.48899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76100000000000001,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.749,"15min":0.89100000000000001},"min":{"1min":0.433,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93300000000000005,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.84899999999999998,"15min":1.0489999999999999},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.86099999999999999,"15min":1.0109999999999999},"min":{"1min":0.38800000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98699999999999999,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.74199999999999999,"15min":0.90400000000000003},"min":{"1min":0.45500000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71899999999999997,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.749}]},{"osd":3,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.83999999999999997,"15min":1.091},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70199999999999996,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.92700000000000005,"15min":1.1659999999999999},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73099999999999998,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.72699999999999998,"15min":0.98699999999999999},"min":{"1min":0.313,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69799999999999995,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.70399999999999996,"15min":0.89400000000000002},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75600000000000001,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.74299999999999999,"15min":0.94699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76600000000000001,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":1.119,"15min":1.119},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.72899999999999998,"15min":0.92200000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.82399999999999995,"15min":1.0229999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81399999999999995,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":521,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194791,"statfs":{"total":200000000,"available":199466172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72799999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.48999999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76700000000000002,"5min":0.87,"15min":0.87},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.44400000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.621}]},{"osd":2,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40899999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76600000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.505},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.46600000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78100000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.49399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.45000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81699999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.439},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.37}]},{"osd":6,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.754,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.61699999999999999},"min":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54400000000000004}]}]}],"pool_statfs":[]}} 2024-01-11T07:12:02.290 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:12:02.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:12:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.468+0000 7f35c2ca5700 1 Processor -- start 2024-01-11T07:12:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.468+0000 7f35c2ca5700 1 -- start start 2024-01-11T07:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c2ca5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 0x7f35bc0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c2ca5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35bc0ab8d0 0x7f35bc15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c2ca5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35bc15e2d0 con 0x7f35bc15d920 2024-01-11T07:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c2ca5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35bc06ac30 con 0x7f35bc0ab8d0 2024-01-11T07:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c2ca5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0ac770 con 0x7f35bc0ac3a0 2024-01-11T07:12:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35bbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f35bc15d920 0x7f35bc15a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33302/0 (socket says 172.21.15.7:33302) 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c1242700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35bc0ab8d0 0x7f35bc15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35bbfff700 1 -- 172.21.15.7:0/1617927931 learned_addr learned my addr 172.21.15.7:0/1617927931 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c0a41700 1 --2- 172.21.15.7:0/1617927931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 0x7f35bc0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c1242700 1 -- 172.21.15.7:0/1617927931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 msgr2=0x7f35bc0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c1242700 1 --2- 172.21.15.7:0/1617927931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 0x7f35bc0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c1242700 1 -- 172.21.15.7:0/1617927931 >> v1:172.21.15.7:6789/0 conn(0x7f35bc15d920 legacy=0x7f35bc15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35c1242700 1 -- 172.21.15.7:0/1617927931 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc06a300 con 0x7f35bc0ab8d0 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.469+0000 7f35bb7fe700 1 -- 172.21.15.7:0/1617927931 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808482387 0 0) 0x7f35bc15e2d0 con 0x7f35bc15d920 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35c1242700 1 --2- 172.21.15.7:0/1617927931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc0ab8d0 0x7f35bc15dd90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009e60 tx=0x7f35b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16e554b7f8db1a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35bb7fe700 1 -- 172.21.15.7:0/1617927931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400e070 con 0x7f35bc0ab8d0 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35bb7fe700 1 -- 172.21.15.7:0/1617927931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b400bb60 con 0x7f35bc0ab8d0 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35bb7fe700 1 -- 172.21.15.7:0/1617927931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4012430 con 0x7f35bc0ab8d0 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35c2ca5700 1 -- 172.21.15.7:0/1617927931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc0ab8d0 msgr2=0x7f35bc15dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/1617927931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc0ab8d0 0x7f35bc15dd90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009e60 tx=0x7f35b4009780 comp rx=0 tx=0).stop 2024-01-11T07:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35c2ca5700 1 -- 172.21.15.7:0/1617927931 shutdown_connections 2024-01-11T07:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/1617927931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc0ab8d0 0x7f35bc15dd90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/1617927931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 0x7f35bc0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.470+0000 7f35c2ca5700 1 -- 172.21.15.7:0/1617927931 >> 172.21.15.7:0/1617927931 conn(0x7f35bc0cd090 msgr2=0x7f35bc0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.471+0000 7f35c2ca5700 1 -- 172.21.15.7:0/1617927931 shutdown_connections 2024-01-11T07:12:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.471+0000 7f35c2ca5700 1 -- 172.21.15.7:0/1617927931 wait complete. 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.472+0000 7f35c2ca5700 1 Processor -- start 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.472+0000 7f35c2ca5700 1 -- start start 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c2ca5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35bc0ab8d0 0x7f35bc16f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c2ca5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 0x7f35bc16e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c2ca5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc15d920 0x7f35bc06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c2ca5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc15bb20 con 0x7f35bc15d920 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c2ca5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc15a670 con 0x7f35bc0ab8d0 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c2ca5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc15a950 con 0x7f35bc0ac3a0 2024-01-11T07:12:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc15d920 0x7f35bc06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc15d920 0x7f35bc06e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56906/0 (socket says 172.21.15.7:56906) 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 -- 172.21.15.7:0/2223816535 learned_addr learned my addr 172.21.15.7:0/2223816535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c0a41700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35bc0ab8d0 0x7f35bc16f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35bbfff700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 0x7f35bc16e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 -- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 msgr2=0x7f35bc16e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 0x7f35bc16e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 -- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35bc0ab8d0 msgr2=0x7f35bc16f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35bc0ab8d0 0x7f35bc16f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 -- 172.21.15.7:0/2223816535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b4009a20 con 0x7f35bc15d920 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.473+0000 7f35c1242700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc15d920 0x7f35bc06e840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f35b4013000 tx=0x7f35b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.474+0000 7f35b8ff9700 1 -- 172.21.15.7:0/2223816535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4027470 con 0x7f35bc15d920 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.474+0000 7f35b8ff9700 1 -- 172.21.15.7:0/2223816535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b40275d0 con 0x7f35bc15d920 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.474+0000 7f35b8ff9700 1 -- 172.21.15.7:0/2223816535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4027990 con 0x7f35bc15d920 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.474+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc15ac30 con 0x7f35bc15d920 2024-01-11T07:12:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.474+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc16fba0 con 0x7f35bc15d920 2024-01-11T07:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.475+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc0b1d40 con 0x7f35bc15d920 2024-01-11T07:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.475+0000 7f35b8ff9700 1 -- 172.21.15.7:0/2223816535 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f35b4024070 con 0x7f35bc15d920 2024-01-11T07:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.476+0000 7f35b8ff9700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f35a0066f30 0x7f35a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.476+0000 7f35b8ff9700 1 -- 172.21.15.7:0/2223816535 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f35b4086f30 con 0x7f35bc15d920 2024-01-11T07:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.476+0000 7f35c0a41700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f35a0066f30 0x7f35a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.477+0000 7f35c0a41700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f35a0066f30 0x7f35a00693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f35bc159630 tx=0x7f35b00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.481+0000 7f35b8ff9700 1 -- 172.21.15.7:0/2223816535 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35b4054100 con 0x7f35bc15d920 2024-01-11T07:12:02.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.767+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f35bc04c3c0 con 0x7f35bc15d920 2024-01-11T07:12:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.770+0000 7f35b8ff9700 1 -- 172.21.15.7:0/2223816535 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f35b4057720 con 0x7f35bc15d920 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f35a0066f30 msgr2=0x7f35a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f35a0066f30 0x7f35a00693e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f35bc159630 tx=0x7f35b00091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc15d920 msgr2=0x7f35bc06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc15d920 0x7f35bc06e840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f35b4013000 tx=0x7f35b4019000 comp rx=0 tx=0).stop 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 shutdown_connections 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f35a0066f30 0x7f35a00693e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35bc15d920 0x7f35bc06e840 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f35bc0ab8d0 0x7f35bc16f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 --2- 172.21.15.7:0/2223816535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f35bc0ac3a0 0x7f35bc16e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 >> 172.21.15.7:0/2223816535 conn(0x7f35bc0cd090 msgr2=0x7f35bc13b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 shutdown_connections 2024-01-11T07:12:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.773+0000 7f35c2ca5700 1 -- 172.21.15.7:0/2223816535 wait complete. 2024-01-11T07:12:02.774 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:12:02.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:02.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.917+0000 7fd42491c700 1 Processor -- start 2024-01-11T07:12:02.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.917+0000 7fd42491c700 1 -- start start 2024-01-11T07:12:02.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.918+0000 7fd42491c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd42006d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd42491c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd420067c60 con 0x7fd4200680f0 2024-01-11T07:12:02.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd42491c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd420069230 con 0x7fd420159f70 2024-01-11T07:12:02.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd42491c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd420069510 con 0x7fd420159ba0 2024-01-11T07:12:02.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd4200680f0 0x7fd4200cdc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33306/0 (socket says 172.21.15.7:33306) 2024-01-11T07:12:02.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41ed9d700 1 -- 172.21.15.7:0/2866924207 learned_addr learned my addr 172.21.15.7:0/2866924207 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:02.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41e59c700 1 --2- 172.21.15.7:0/2866924207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd42006d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41d59a700 1 -- 172.21.15.7:0/2866924207 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705993745 0 0) 0x7fd420067c60 con 0x7fd4200680f0 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41d59a700 1 -- 172.21.15.7:0/2866924207 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd408003570 con 0x7fd4200680f0 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41d59a700 1 -- 172.21.15.7:0/2866924207 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398567685 0 0) 0x7fd420069230 con 0x7fd420159f70 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41d59a700 1 -- 172.21.15.7:0/2866924207 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd420067c60 con 0x7fd420159f70 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41e59c700 1 -- 172.21.15.7:0/2866924207 >> v1:172.21.15.113:6789/0 conn(0x7fd420159f70 legacy=0x7fd4200c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41e59c700 1 -- 172.21.15.7:0/2866924207 >> v1:172.21.15.7:6789/0 conn(0x7fd4200680f0 legacy=0x7fd4200cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41e59c700 1 -- 172.21.15.7:0/2866924207 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420067610 con 0x7fd420159ba0 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.919+0000 7fd41e59c700 1 --2- 172.21.15.7:0/2866924207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd42006d020 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd410009910 tx=0x7fd4100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8db687a55e5ffc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd41d59a700 1 -- 172.21.15.7:0/2866924207 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410011630 con 0x7fd420159ba0 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd41d59a700 1 -- 172.21.15.7:0/2866924207 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd410011790 con 0x7fd420159ba0 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd41d59a700 1 -- 172.21.15.7:0/2866924207 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410011b30 con 0x7fd420159ba0 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd42491c700 1 -- 172.21.15.7:0/2866924207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 msgr2=0x7fd42006d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd42491c700 1 --2- 172.21.15.7:0/2866924207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd42006d020 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd410009910 tx=0x7fd4100091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd42491c700 1 -- 172.21.15.7:0/2866924207 shutdown_connections 2024-01-11T07:12:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd42491c700 1 --2- 172.21.15.7:0/2866924207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd42006d020 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd42491c700 1 -- 172.21.15.7:0/2866924207 >> 172.21.15.7:0/2866924207 conn(0x7fd42006ae00 msgr2=0x7fd42006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:02.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd42491c700 1 -- 172.21.15.7:0/2866924207 shutdown_connections 2024-01-11T07:12:02.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.920+0000 7fd42491c700 1 -- 172.21.15.7:0/2866924207 wait complete. 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd42491c700 1 Processor -- start 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd42491c700 1 -- start start 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd42491c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4200680f0 0x7fd42014a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd42491c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd4200c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd42491c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd420159f70 0x7fd4200c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd42491c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200c8ff0 con 0x7fd420159f70 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd42491c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200ca000 con 0x7fd4200680f0 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd42491c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4200c9240 con 0x7fd420159ba0 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.922+0000 7fd41ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd420159f70 0x7fd4200c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd420159f70 0x7fd4200c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56916/0 (socket says 172.21.15.7:56916) 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41ed9d700 1 -- 172.21.15.7:0/3495597635 learned_addr learned my addr 172.21.15.7:0/3495597635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41e59c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4200680f0 0x7fd42014a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41dd9b700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd4200c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41ed9d700 1 -- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 msgr2=0x7fd4200c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41ed9d700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd4200c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41ed9d700 1 -- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4200680f0 msgr2=0x7fd42014a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41ed9d700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4200680f0 0x7fd42014a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41ed9d700 1 -- 172.21.15.7:0/3495597635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410009420 con 0x7fd420159f70 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41dd9b700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd4200c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd41ed9d700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd420159f70 0x7fd4200c3cd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd41800cc20 tx=0x7fd418001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd40e7fc700 1 -- 172.21.15.7:0/3495597635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800a8b0 con 0x7fd420159f70 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd40e7fc700 1 -- 172.21.15.7:0/3495597635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41800aa10 con 0x7fd420159f70 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4200c9520 con 0x7fd420159f70 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd40e7fc700 1 -- 172.21.15.7:0/3495597635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800add0 con 0x7fd420159f70 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.923+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4200c9ae0 con 0x7fd420159f70 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.924+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42004c3c0 con 0x7fd420159f70 2024-01-11T07:12:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.925+0000 7fd40e7fc700 1 -- 172.21.15.7:0/3495597635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd418007300 con 0x7fd420159f70 2024-01-11T07:12:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.925+0000 7fd40e7fc700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd400066f30 0x7fd4000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.925+0000 7fd40e7fc700 1 -- 172.21.15.7:0/3495597635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd4180848a0 con 0x7fd420159f70 2024-01-11T07:12:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.925+0000 7fd41e59c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd400066f30 0x7fd4000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.926+0000 7fd41e59c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd400066f30 0x7fd4000693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd410009860 tx=0x7fd410017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:02.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:02.930+0000 7fd40e7fc700 1 -- 172.21.15.7:0/3495597635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd418051a70 con 0x7fd420159f70 2024-01-11T07:12:03.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.184+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4200c9d00 con 0x7fd420159f70 2024-01-11T07:12:03.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.185+0000 7fd40e7fc700 1 -- 172.21.15.7:0/3495597635 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fd418055090 con 0x7fd420159f70 2024-01-11T07:12:03.184 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:03.185 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd400066f30 msgr2=0x7fd4000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd400066f30 0x7fd4000693e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd410009860 tx=0x7fd410017000 comp rx=0 tx=0).stop 2024-01-11T07:12:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd420159f70 msgr2=0x7fd4200c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd420159f70 0x7fd4200c3cd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd41800cc20 tx=0x7fd418001a40 comp rx=0 tx=0).stop 2024-01-11T07:12:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 shutdown_connections 2024-01-11T07:12:03.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd400066f30 0x7fd4000693e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd420159f70 0x7fd4200c3cd0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4200680f0 0x7fd42014a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 --2- 172.21.15.7:0/3495597635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd420159ba0 0x7fd4200c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 >> 172.21.15.7:0/3495597635 conn(0x7fd42006ae00 msgr2=0x7fd42006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 shutdown_connections 2024-01-11T07:12:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.187+0000 7fd42491c700 1 -- 172.21.15.7:0/3495597635 wait complete. 2024-01-11T07:12:03.188 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:03.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:11:57.119254+0000', 'modified': '2024-01-11T07:12:00.699363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6175': {'gid': 6175, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/401484670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 401484670}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 401484670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:12:03.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:12:03.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-11T07:12:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.746+0000 7ff28d64e700 1 Processor -- start 2024-01-11T07:12:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.746+0000 7ff28d64e700 1 -- start start 2024-01-11T07:12:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.747+0000 7ff28d64e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff2880b0b40 0x7ff288162790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.747+0000 7ff28d64e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2880cc2a0 con 0x7ff2880adf10 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.747+0000 7ff28d64e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2880b3b50 con 0x7ff2880adb40 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.747+0000 7ff28d64e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2880b3e30 con 0x7ff2880b0b40 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.747+0000 7ff2877fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff2880b0b40 0x7ff288162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.747+0000 7ff2877fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff2880b0b40 0x7ff288162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56926/0 (socket says 172.21.15.7:56926) 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.747+0000 7ff27ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff2880adf10 0x7ff2880b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33312/0 (socket says 172.21.15.7:33312) 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.747+0000 7ff2877fe700 1 -- 172.21.15.7:0/1367970237 learned_addr learned my addr 172.21.15.7:0/1367970237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.748+0000 7ff2877fe700 1 -- 172.21.15.7:0/1367970237 >> v1:172.21.15.113:6789/0 conn(0x7ff2880adb40 legacy=0x7ff2880b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.748+0000 7ff2877fe700 1 -- 172.21.15.7:0/1367970237 >> v1:172.21.15.7:6789/0 conn(0x7ff2880adf10 legacy=0x7ff2880b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.748+0000 7ff2877fe700 1 -- 172.21.15.7:0/1367970237 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880cc540 con 0x7ff2880b0b40 2024-01-11T07:12:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.748+0000 7ff2867fc700 1 -- 172.21.15.7:0/1367970237 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764867116 0 0) 0x7ff2880cc2a0 con 0x7ff2880adf10 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.748+0000 7ff2877fe700 1 --2- 172.21.15.7:0/1367970237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff288162790 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff280009f20 tx=0x7ff280009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3552b446bc55471 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.748+0000 7ff2867fc700 1 -- 172.21.15.7:0/1367970237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000e070 con 0x7ff2880b0b40 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.748+0000 7ff2867fc700 1 -- 172.21.15.7:0/1367970237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28000ba80 con 0x7ff2880b0b40 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.748+0000 7ff2867fc700 1 -- 172.21.15.7:0/1367970237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800092e0 con 0x7ff2880b0b40 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.749+0000 7ff28d64e700 1 -- 172.21.15.7:0/1367970237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 msgr2=0x7ff288162790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.749+0000 7ff28d64e700 1 --2- 172.21.15.7:0/1367970237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff288162790 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff280009f20 tx=0x7ff280009840 comp rx=0 tx=0).stop 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.749+0000 7ff28d64e700 1 -- 172.21.15.7:0/1367970237 shutdown_connections 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.749+0000 7ff28d64e700 1 --2- 172.21.15.7:0/1367970237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff288162790 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.749+0000 7ff28d64e700 1 -- 172.21.15.7:0/1367970237 >> 172.21.15.7:0/1367970237 conn(0x7ff288071160 msgr2=0x7ff28806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.749+0000 7ff28d64e700 1 -- 172.21.15.7:0/1367970237 shutdown_connections 2024-01-11T07:12:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.749+0000 7ff28d64e700 1 -- 172.21.15.7:0/1367970237 wait complete. 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.750+0000 7ff28d64e700 1 Processor -- start 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.750+0000 7ff28d64e700 1 -- start start 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff28d64e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2880adb40 0x7ff2881545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff28d64e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2880adf10 0x7ff28815b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff28d64e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff28815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff28d64e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff288154b10 con 0x7ff2880b0b40 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff28d64e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28815c280 con 0x7ff2880adf10 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff28d64e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28815c560 con 0x7ff2880adb40 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff2877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff28815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff2877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff28815bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56928/0 (socket says 172.21.15.7:56928) 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff2877fe700 1 -- 172.21.15.7:0/2893779603 learned_addr learned my addr 172.21.15.7:0/2893779603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff27ffff700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2880adf10 0x7ff28815b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff286ffd700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2880adb40 0x7ff2881545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff2877fe700 1 -- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2880adb40 msgr2=0x7ff2881545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff2877fe700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2880adb40 0x7ff2881545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff2877fe700 1 -- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2880adf10 msgr2=0x7ff28815b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff2877fe700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2880adf10 0x7ff28815b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff2877fe700 1 -- 172.21.15.7:0/2893779603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280009ae0 con 0x7ff2880b0b40 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff286ffd700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2880adb40 0x7ff2881545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.751+0000 7ff27ffff700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2880adf10 0x7ff28815b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.752+0000 7ff2877fe700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff28815bbb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff280012000 tx=0x7ff280018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.752+0000 7ff27effd700 1 -- 172.21.15.7:0/2893779603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280007e30 con 0x7ff2880b0b40 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.752+0000 7ff27effd700 1 -- 172.21.15.7:0/2893779603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28001c410 con 0x7ff2880b0b40 2024-01-11T07:12:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.752+0000 7ff28d64e700 1 -- 172.21.15.7:0/2893779603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28815c840 con 0x7ff2880b0b40 2024-01-11T07:12:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.752+0000 7ff27effd700 1 -- 172.21.15.7:0/2893779603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28001c7d0 con 0x7ff2880b0b40 2024-01-11T07:12:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.752+0000 7ff28d64e700 1 -- 172.21.15.7:0/2893779603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28815ce00 con 0x7ff2880b0b40 2024-01-11T07:12:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.753+0000 7ff27effd700 1 -- 172.21.15.7:0/2893779603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff28001cb80 con 0x7ff2880b0b40 2024-01-11T07:12:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.754+0000 7ff27effd700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff268066f30 0x7ff2680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.754+0000 7ff27effd700 1 -- 172.21.15.7:0/2893779603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2800863b0 con 0x7ff2880b0b40 2024-01-11T07:12:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.754+0000 7ff286ffd700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff268066f30 0x7ff2680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.755+0000 7ff286ffd700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff268066f30 0x7ff2680693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2880caff0 tx=0x7ff278009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.756+0000 7ff29694eb80 1 Processor -- start 2024-01-11T07:12:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.756+0000 7ff29694eb80 1 -- start start 2024-01-11T07:12:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c84b020 0x55ce1c84b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c8694d0 0x55ce1c863360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c8638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff29694eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c7753e0 con 0x55ce1c8694d0 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff29694eb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c772be0 con 0x55ce1c869940 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff29694eb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c76e9d0 con 0x55ce1c84b020 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff27ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c8694d0 0x55ce1c863360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff27ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c8694d0 0x55ce1c863360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56940/0 (socket says 172.21.15.7:56940) 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.757+0000 7ff27ffff700 1 -- 172.21.15.7:0/1063882679 learned_addr learned my addr 172.21.15.7:0/1063882679 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff2877fe700 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c8638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff286ffd700 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c84b020 0x55ce1c84b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff27ffff700 1 -- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c84b020 msgr2=0x55ce1c84b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff27ffff700 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c84b020 0x55ce1c84b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff27ffff700 1 -- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 msgr2=0x55ce1c8638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff27ffff700 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c8638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff27ffff700 1 -- 172.21.15.7:0/1063882679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce1c602360 con 0x55ce1c8694d0 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff2877fe700 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c8638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff27ffff700 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c8694d0 0x55ce1c863360 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff2700018d0 tx=0x7ff270009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1762ac94c0d67bb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff267fff700 1 -- 172.21.15.7:0/1063882679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000b070 con 0x55ce1c8694d0 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff267fff700 1 -- 172.21.15.7:0/1063882679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff270005a70 con 0x55ce1c8694d0 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.758+0000 7ff267fff700 1 -- 172.21.15.7:0/1063882679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270005df0 con 0x55ce1c8694d0 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 -- 172.21.15.7:0/1063882679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c8694d0 msgr2=0x55ce1c863360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c8694d0 0x55ce1c863360 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff2700018d0 tx=0x7ff270009040 comp rx=0 tx=0).stop 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 -- 172.21.15.7:0/1063882679 shutdown_connections 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c8638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c84b020 0x55ce1c84b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 --2- 172.21.15.7:0/1063882679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c8694d0 0x55ce1c863360 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 -- 172.21.15.7:0/1063882679 >> 172.21.15.7:0/1063882679 conn(0x55ce1c81d440 msgr2=0x55ce1c868dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 -- 172.21.15.7:0/1063882679 shutdown_connections 2024-01-11T07:12:03.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.759+0000 7ff29694eb80 1 -- 172.21.15.7:0/1063882679 wait complete. 2024-01-11T07:12:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.760+0000 7ff29694eb80 1 -- start start 2024-01-11T07:12:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.760+0000 7ff29694eb80 1 Processor -- start 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c84b020 0x55ce1c8635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c879b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c869940 0x55ce1c861180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff29694eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c71b1b0 con 0x55ce1c869940 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff29694eb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c71ae50 con 0x55ce1c84b020 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff29694eb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c667e40 con 0x55ce1c8694d0 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff29694eb80 10 client.0 _ll_register_callbacks cb 0x55ce1c66a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff2877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c869940 0x55ce1c861180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff2877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c869940 0x55ce1c861180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56942/0 (socket says 172.21.15.7:56942) 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff2877fe700 1 -- 172.21.15.7:0/4101432160 learned_addr learned my addr 172.21.15.7:0/4101432160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff286ffd700 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c84b020 0x55ce1c8635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff27ffff700 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c879b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff2877fe700 1 -- 172.21.15.7:0/4101432160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 msgr2=0x55ce1c879b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff2877fe700 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c879b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff2877fe700 1 -- 172.21.15.7:0/4101432160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c84b020 msgr2=0x55ce1c8635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff2877fe700 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c84b020 0x55ce1c8635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.761+0000 7ff2877fe700 1 -- 172.21.15.7:0/4101432160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff270001480 con 0x55ce1c869940 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff2877fe700 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c869940 0x55ce1c861180 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff280003440 tx=0x7ff280009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c602066648a4b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff2657fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff2657fa700 1 -- 172.21.15.7:0/4101432160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000e040 con 0x55ce1c869940 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff2657fa700 1 -- 172.21.15.7:0/4101432160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff28005a490 con 0x55ce1c869940 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 1 -- 172.21.15.7:0/4101432160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce1c7efeb0 con 0x55ce1c869940 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff2657fa700 1 -- 172.21.15.7:0/4101432160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff28005e1e0 con 0x55ce1c869940 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 10 client.6483 fetch_fsmap learned FSMap version 50 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 1 -- 172.21.15.7:0/4101432160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55ce1c5aeb10 con 0x55ce1c869940 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff2657fa700 1 -- 172.21.15.7:0/4101432160 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce1c7efeb0 con 0x55ce1c869940 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff2657fa700 1 -- 172.21.15.7:0/4101432160 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff2800574a0 con 0x55ce1c869940 2024-01-11T07:12:03.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 10 client.6483 fetch_fsmap finished waiting for FSMap version 50 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 10 client.6483 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 20 client.6483 populate_metadata read hostname 'smithi007' 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] conn(0x55ce1c847b20 0x55ce1c847ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 4 client.6483 mds_command: new command op to 6175 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff29694eb80 1 -- 172.21.15.7:0/4101432160 --> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce1c745880 con 0x55ce1c847b20 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.762+0000 7ff286ffd700 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] conn(0x55ce1c847b20 0x55ce1c847ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.763+0000 7ff286ffd700 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] conn(0x55ce1c847b20 0x55ce1c847ef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32101d98b7ce8fe1 server_cookie=a92542d175f93fc0 in_seq=0 out_seq=0 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.763+0000 7ff2657fa700 10 client.6483 ms_handle_connect on v2:172.21.15.7:6834/401484670 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.763+0000 7ff2657fa700 1 -- 172.21.15.7:0/4101432160 <== mds.0 v2:172.21.15.7:6834/401484670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ce1c745880 con 0x55ce1c847b20 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.763+0000 7ff2657fa700 10 client.6483 handle_command_reply: tid=0 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.763+0000 7ff29694eb80 1 client.6483 shutdown 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 -- 172.21.15.7:0/4101432160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c869940 msgr2=0x55ce1c861180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c869940 0x55ce1c861180 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff280003440 tx=0x7ff280009040 comp rx=0 tx=0).stop 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 -- 172.21.15.7:0/4101432160 shutdown_connections 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] conn(0x55ce1c847b20 0x55ce1c847ef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c869940 0x55ce1c861180 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c84b020 0x55ce1c8635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff2657fa700 0 client.6483 ms_handle_reset on v2:172.21.15.7:6834/401484670 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 --2- 172.21.15.7:0/4101432160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c879b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 -- 172.21.15.7:0/4101432160 >> 172.21.15.7:0/4101432160 conn(0x55ce1c81d440 msgr2=0x55ce1c820f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 -- 172.21.15.7:0/4101432160 shutdown_connections 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 1 -- 172.21.15.7:0/4101432160 wait complete. 2024-01-11T07:12:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.764+0000 7ff29694eb80 20 client.6483 trim_cache size 0 max 16384 2024-01-11T07:12:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.782+0000 7ff29694eb80 1 Processor -- start 2024-01-11T07:12:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.782+0000 7ff29694eb80 1 -- start start 2024-01-11T07:12:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c879a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c8786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff29694eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c81dca0 con 0x55ce1c84b020 2024-01-11T07:12:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff29694eb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c84d240 con 0x55ce1c869940 2024-01-11T07:12:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff29694eb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c81d230 con 0x55ce1c8694d0 2024-01-11T07:12:03.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c879a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c879a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56956/0 (socket says 172.21.15.7:56956) 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 -- 172.21.15.7:0/3067967769 learned_addr learned my addr 172.21.15.7:0/3067967769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff2877fe700 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff27ffff700 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c8786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 -- 172.21.15.7:0/3067967769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 msgr2=0x55ce1c8786c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c8786c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 -- 172.21.15.7:0/3067967769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 msgr2=0x55ce1c879110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 -- 172.21.15.7:0/3067967769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce1c81c990 con 0x55ce1c84b020 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff286ffd700 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c879a30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff278007fd0 tx=0x7ff278013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5e79179748c33d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.783+0000 7ff27cff9700 1 -- 172.21.15.7:0/3067967769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278025680 con 0x55ce1c84b020 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff27cff9700 1 -- 172.21.15.7:0/3067967769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2780257e0 con 0x55ce1c84b020 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff27cff9700 1 -- 172.21.15.7:0/3067967769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278025950 con 0x55ce1c84b020 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 -- 172.21.15.7:0/3067967769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 msgr2=0x55ce1c879a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c879a30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff278007fd0 tx=0x7ff278013000 comp rx=0 tx=0).stop 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 -- 172.21.15.7:0/3067967769 shutdown_connections 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c879a30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 --2- 172.21.15.7:0/3067967769 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c8786c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 -- 172.21.15.7:0/3067967769 >> 172.21.15.7:0/3067967769 conn(0x55ce1c81d440 msgr2=0x55ce1c81ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 -- 172.21.15.7:0/3067967769 shutdown_connections 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.784+0000 7ff29694eb80 1 -- 172.21.15.7:0/3067967769 wait complete. 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.785+0000 7ff29694eb80 1 -- start start 2024-01-11T07:12:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.785+0000 7ff29694eb80 1 Processor -- start 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c877bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c879030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff29694eb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff29694eb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c869f20 con 0x55ce1c84b020 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff29694eb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c8cf950 con 0x55ce1c869940 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff29694eb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce1c86da30 con 0x55ce1c8694d0 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff29694eb80 10 client.0 _ll_register_callbacks cb 0x55ce1c66a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c877bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c877bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56958/0 (socket says 172.21.15.7:56958) 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 -- 172.21.15.7:0/2149089457 learned_addr learned my addr 172.21.15.7:0/2149089457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff2877fe700 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff27ffff700 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c879030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 -- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 msgr2=0x55ce1c879030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c879030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 -- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 msgr2=0x55ce1c879570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 -- 172.21.15.7:0/2149089457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27801c040 con 0x55ce1c84b020 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff2877fe700 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff286ffd700 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c877bb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff2780011b0 tx=0x7ff278016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae2d296b7563d02 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.786+0000 7ff2657fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff2657fa700 1 -- 172.21.15.7:0/2149089457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2780137d0 con 0x55ce1c84b020 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff2657fa700 1 -- 172.21.15.7:0/2149089457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff278013930 con 0x55ce1c84b020 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 1 -- 172.21.15.7:0/2149089457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce1c813490 con 0x55ce1c84b020 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff2657fa700 1 -- 172.21.15.7:0/2149089457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff278013d00 con 0x55ce1c84b020 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 10 client.6489 fetch_fsmap learned FSMap version 50 2024-01-11T07:12:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 1 -- 172.21.15.7:0/2149089457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55ce1c73d290 con 0x55ce1c84b020 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff2657fa700 1 -- 172.21.15.7:0/2149089457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce1c813490 con 0x55ce1c84b020 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff2657fa700 1 -- 172.21.15.7:0/2149089457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) v1 ==== 3100+0+0 (secure 0 0 0) 0x7ff278037480 con 0x55ce1c84b020 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 10 client.6489 fetch_fsmap finished waiting for FSMap version 50 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 10 client.6489 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 20 client.6489 populate_metadata read hostname 'smithi007' 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] conn(0x55ce1c8ceaa0 0x55ce1c8cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 4 client.6489 mds_command: new command op to 6175 tid=0[{"prefix": "status"}] 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff29694eb80 1 -- 172.21.15.7:0/2149089457 --> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ce1c711910 con 0x55ce1c8ceaa0 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.787+0000 7ff27ffff700 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] conn(0x55ce1c8ceaa0 0x55ce1c8cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff27ffff700 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] conn(0x55ce1c8ceaa0 0x55ce1c8cee70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56942222d44b2686 server_cookie=e2ad8d28d56a41db in_seq=0 out_seq=0 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff2657fa700 10 client.6489 ms_handle_connect on v2:172.21.15.7:6834/401484670 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff2657fa700 1 -- 172.21.15.7:0/2149089457 <== mds.0 v2:172.21.15.7:6834/401484670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ce1c711910 con 0x55ce1c8ceaa0 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff2657fa700 10 client.6489 handle_command_reply: tid=0 2024-01-11T07:12:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 client.6489 shutdown 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 -- 172.21.15.7:0/2149089457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 msgr2=0x55ce1c877bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c877bb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff2780011b0 tx=0x7ff278016070 comp rx=0 tx=0).stop 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 -- 172.21.15.7:0/2149089457 shutdown_connections 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.7:6834/401484670,v1:172.21.15.7:6835/401484670] conn(0x55ce1c8ceaa0 0x55ce1c8cee70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ce1c869940 0x55ce1c879570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ce1c84b020 0x55ce1c877bb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 --2- 172.21.15.7:0/2149089457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ce1c8694d0 0x55ce1c879030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff2657fa700 0 client.6489 ms_handle_reset on v2:172.21.15.7:6834/401484670 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 -- 172.21.15.7:0/2149089457 >> 172.21.15.7:0/2149089457 conn(0x55ce1c81d440 msgr2=0x55ce1c81ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 -- 172.21.15.7:0/2149089457 shutdown_connections 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 1 -- 172.21.15.7:0/2149089457 wait complete. 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.788+0000 7ff29694eb80 20 client.6489 trim_cache size 0 max 16384 2024-01-11T07:12:03.793 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6175, 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1064704900000004, 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 50, 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 100, 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 100, 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 10.930515047 2024-01-11T07:12:03.794 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 -- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff268066f30 msgr2=0x7ff2680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff268066f30 0x7ff2680693e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2880caff0 tx=0x7ff278009040 comp rx=0 tx=0).stop 2024-01-11T07:12:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 -- 172.21.15.7:0/2893779603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 msgr2=0x7ff28815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff28815bbb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff280012000 tx=0x7ff280018000 comp rx=0 tx=0).stop 2024-01-11T07:12:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 -- 172.21.15.7:0/2893779603 shutdown_connections 2024-01-11T07:12:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff268066f30 0x7ff2680693e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2880b0b40 0x7ff28815bbb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2880adb40 0x7ff2881545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 --2- 172.21.15.7:0/2893779603 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2880adf10 0x7ff28815b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 -- 172.21.15.7:0/2893779603 >> 172.21.15.7:0/2893779603 conn(0x7ff288071160 msgr2=0x7ff288157680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 -- 172.21.15.7:0/2893779603 shutdown_connections 2024-01-11T07:12:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.789+0000 7ff28d64e700 1 -- 172.21.15.7:0/2893779603 wait complete. 2024-01-11T07:12:03.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:12:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.934+0000 7f5b5fe6a700 1 Processor -- start 2024-01-11T07:12:03.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.934+0000 7f5b5fe6a700 1 -- start start 2024-01-11T07:12:03.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5fe6a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b580adf10 0x7f5b580b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5fe6a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b580cc2a0 con 0x7f5b580b3590 2024-01-11T07:12:03.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5fe6a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b580b3b50 con 0x7f5b580adb40 2024-01-11T07:12:03.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5fe6a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b580b3e30 con 0x7f5b580adf10 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5d405700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b580adf10 0x7f5b580b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5d405700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b580adf10 0x7f5b580b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56960/0 (socket says 172.21.15.7:56960) 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5d405700 1 -- 172.21.15.7:0/2363101465 learned_addr learned my addr 172.21.15.7:0/2363101465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5e407700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5b580b3590 0x7f5b580cca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33316/0 (socket says 172.21.15.7:33316) 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5d405700 1 -- 172.21.15.7:0/2363101465 >> v1:172.21.15.113:6789/0 conn(0x7f5b580adb40 legacy=0x7f5b580b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.935+0000 7f5b5d405700 1 -- 172.21.15.7:0/2363101465 >> v1:172.21.15.7:6789/0 conn(0x7f5b580b3590 legacy=0x7f5b580cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.936+0000 7f5b5d405700 1 -- 172.21.15.7:0/2363101465 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580cc540 con 0x7f5b580adf10 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.936+0000 7f5b5cc04700 1 -- 172.21.15.7:0/2363101465 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947153627 0 0) 0x7f5b580cc2a0 con 0x7f5b580b3590 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.936+0000 7f5b5d405700 1 --2- 172.21.15.7:0/2363101465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580adf10 0x7f5b580b0430 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099b0 tx=0x7f5b48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbc4c04f355ef31 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.936+0000 7f5b5cc04700 1 -- 172.21.15.7:0/2363101465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800b860 con 0x7f5b580adf10 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.936+0000 7f5b5cc04700 1 -- 172.21.15.7:0/2363101465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4800b9c0 con 0x7f5b580adf10 2024-01-11T07:12:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.936+0000 7f5b5cc04700 1 -- 172.21.15.7:0/2363101465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800bd60 con 0x7f5b580adf10 2024-01-11T07:12:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.936+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/2363101465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580adf10 msgr2=0x7f5b580b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.936+0000 7f5b5fe6a700 1 --2- 172.21.15.7:0/2363101465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580adf10 0x7f5b580b0430 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099b0 tx=0x7f5b48009250 comp rx=0 tx=0).stop 2024-01-11T07:12:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.937+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/2363101465 shutdown_connections 2024-01-11T07:12:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.937+0000 7f5b5fe6a700 1 --2- 172.21.15.7:0/2363101465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580adf10 0x7f5b580b0430 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.937+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/2363101465 >> 172.21.15.7:0/2363101465 conn(0x7f5b58071160 msgr2=0x7f5b5806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.937+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/2363101465 shutdown_connections 2024-01-11T07:12:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.937+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/2363101465 wait complete. 2024-01-11T07:12:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.938+0000 7f5b5fe6a700 1 Processor -- start 2024-01-11T07:12:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.938+0000 7f5b5fe6a700 1 -- start start 2024-01-11T07:12:03.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5fe6a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b580adb40 0x7f5b58149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5fe6a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b580adf10 0x7f5b580b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5fe6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580b3590 0x7f5b580b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5fe6a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580a73f0 con 0x7f5b580b3590 2024-01-11T07:12:03.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5fe6a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580a67e0 con 0x7f5b580adf10 2024-01-11T07:12:03.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5fe6a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580a6ac0 con 0x7f5b580adb40 2024-01-11T07:12:03.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580b3590 0x7f5b580b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580b3590 0x7f5b580b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56962/0 (socket says 172.21.15.7:56962) 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 -- 172.21.15.7:0/460736650 learned_addr learned my addr 172.21.15.7:0/460736650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5d405700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b580adf10 0x7f5b580b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5dc06700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b580adb40 0x7f5b58149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 -- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b580adb40 msgr2=0x7f5b58149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b580adb40 0x7f5b58149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 -- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b580adf10 msgr2=0x7f5b580b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b580adf10 0x7f5b580b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 -- 172.21.15.7:0/460736650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480094f0 con 0x7f5b580b3590 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5e407700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580b3590 0x7f5b580b1fb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000ccc0 tx=0x7f5b50001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b4dffb700 1 -- 172.21.15.7:0/460736650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50005df0 con 0x7f5b580b3590 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b4dffb700 1 -- 172.21.15.7:0/460736650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5000a670 con 0x7f5b580b3590 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.939+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b580a6da0 con 0x7f5b580b3590 2024-01-11T07:12:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.940+0000 7f5b4dffb700 1 -- 172.21.15.7:0/460736650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5000aa30 con 0x7f5b580b3590 2024-01-11T07:12:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.940+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b580a64b0 con 0x7f5b580b3590 2024-01-11T07:12:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.941+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b580a75a0 con 0x7f5b580b3590 2024-01-11T07:12:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.941+0000 7f5b4dffb700 1 -- 172.21.15.7:0/460736650 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5b500070f0 con 0x7f5b580b3590 2024-01-11T07:12:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.942+0000 7f5b4dffb700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b3c066f30 0x7f5b3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.942+0000 7f5b4dffb700 1 -- 172.21.15.7:0/460736650 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b50084540 con 0x7f5b580b3590 2024-01-11T07:12:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.942+0000 7f5b5dc06700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b3c066f30 0x7f5b3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.942+0000 7f5b5dc06700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b3c066f30 0x7f5b3c0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b54003a80 tx=0x7f5b54007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:03.946+0000 7f5b4dffb700 1 -- 172.21.15.7:0/460736650 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b580a75a0 con 0x7f5b580b3590 2024-01-11T07:12:04.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.198+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5b580a75a0 con 0x7f5b580b3590 2024-01-11T07:12:04.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.198+0000 7f5b4dffb700 1 -- 172.21.15.7:0/460736650 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5b580a75a0 con 0x7f5b580b3590 2024-01-11T07:12:04.198 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:12:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b3c066f30 msgr2=0x7f5b3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b3c066f30 0x7f5b3c0693e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b54003a80 tx=0x7f5b54007ec0 comp rx=0 tx=0).stop 2024-01-11T07:12:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580b3590 msgr2=0x7f5b580b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580b3590 0x7f5b580b1fb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000ccc0 tx=0x7f5b50001a20 comp rx=0 tx=0).stop 2024-01-11T07:12:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 shutdown_connections 2024-01-11T07:12:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b3c066f30 0x7f5b3c0693e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580b3590 0x7f5b580b1fb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b580adb40 0x7f5b58149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 --2- 172.21.15.7:0/460736650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b580adf10 0x7f5b580b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 >> 172.21.15.7:0/460736650 conn(0x7f5b58071160 msgr2=0x7f5b58157530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.200+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 shutdown_connections 2024-01-11T07:12:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:04.201+0000 7f5b5fe6a700 1 -- 172.21.15.7:0/460736650 wait complete. 2024-01-11T07:12:04.213 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:12:04.214 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:12:04.214 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:12:04.214 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:12:04.214 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:04.214 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:12:04.214 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:12:04.214 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:12:04.214 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:12:04.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:04.214 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:12:04.234 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:12:04.235 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42093sec preferred_lft 42093sec 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:12:04.236 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:04.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:04.238 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:12:04.306 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:12:04.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:04.308 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:12:04.373 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:12:04.373 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:12:04.373 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:12:04.373 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:12:04.373 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:12:04.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:04.374 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:12:04.374 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:12:04.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:12:04.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:12:04.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:12:04.374 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:12:04.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:04.498 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:12:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:12:04.518 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:12:04.519 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:12:04.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:04.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:12:04.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:12:04.594 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:04.655 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:12:04.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:12:04.656 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:04.716 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:12:04.716 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:12:04.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:12:04.810 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:12:04.811 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:12:04.811 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:12:04.820 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:12:04.821 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:12:04.821 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:12:04.821 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:12:04.821 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:12:04.838 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:12:04.838 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:12:04.838 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:12:04.839 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:12:04.839 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:12:04.839 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:12:04.839 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:12:04.839 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:12:04.841 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:12:04.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:04.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:04.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:12:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.050+0000 7fb091d23700 1 Processor -- start 2024-01-11T07:12:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.051+0000 7fb091d23700 1 -- start start 2024-01-11T07:12:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.051+0000 7fb091d23700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c063230 0x7fb08c063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.051+0000 7fb091d23700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb08c0ac5b0 0x7fb08c0aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.051+0000 7fb091d23700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c0ab100 con 0x7fb08c15a280 2024-01-11T07:12:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.051+0000 7fb091d23700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c15d880 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.051+0000 7fb091d23700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c06c5b0 con 0x7fb08c063230 2024-01-11T07:12:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb08c15a280 0x7fb08c0aa9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33330/0 (socket says 172.21.15.7:33330) 2024-01-11T07:12:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08affd700 1 -- 172.21.15.7:0/1058268939 learned_addr learned my addr 172.21.15.7:0/1058268939 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:05.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08bfff700 1 --2- 172.21.15.7:0/1058268939 >> v2:172.21.15.113:3300/0 conn(0x7fb08c0ac5b0 0x7fb08c0aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08b7fe700 1 --2- 172.21.15.7:0/1058268939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c063230 0x7fb08c063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08a7fc700 1 -- 172.21.15.7:0/1058268939 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648051267 0 0) 0x7fb08c0ab100 con 0x7fb08c15a280 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08a7fc700 1 -- 172.21.15.7:0/1058268939 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb074003570 con 0x7fb08c15a280 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08b7fe700 1 -- 172.21.15.7:0/1058268939 >> v2:172.21.15.113:3300/0 conn(0x7fb08c0ac5b0 msgr2=0x7fb08c0aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08b7fe700 1 --2- 172.21.15.7:0/1058268939 >> v2:172.21.15.113:3300/0 conn(0x7fb08c0ac5b0 0x7fb08c0aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08b7fe700 1 -- 172.21.15.7:0/1058268939 >> v1:172.21.15.7:6789/0 conn(0x7fb08c15a280 legacy=0x7fb08c0aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08b7fe700 1 -- 172.21.15.7:0/1058268939 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c063b40 con 0x7fb08c063230 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08bfff700 1 --2- 172.21.15.7:0/1058268939 >> v2:172.21.15.113:3300/0 conn(0x7fb08c0ac5b0 0x7fb08c0aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.052+0000 7fb08b7fe700 1 --2- 172.21.15.7:0/1058268939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c063230 0x7fb08c063600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0098e0 tx=0x7fb07c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed700220f37b463a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb08a7fc700 1 -- 172.21.15.7:0/1058268939 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c011630 con 0x7fb08c063230 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb08a7fc700 1 -- 172.21.15.7:0/1058268939 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c011790 con 0x7fb08c063230 2024-01-11T07:12:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb08a7fc700 1 -- 172.21.15.7:0/1058268939 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c011b30 con 0x7fb08c063230 2024-01-11T07:12:05.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb091d23700 1 -- 172.21.15.7:0/1058268939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c063230 msgr2=0x7fb08c063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:05.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb091d23700 1 --2- 172.21.15.7:0/1058268939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c063230 0x7fb08c063600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0098e0 tx=0x7fb07c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb091d23700 1 -- 172.21.15.7:0/1058268939 shutdown_connections 2024-01-11T07:12:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb091d23700 1 --2- 172.21.15.7:0/1058268939 >> v2:172.21.15.113:3300/0 conn(0x7fb08c0ac5b0 0x7fb08c0aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb091d23700 1 --2- 172.21.15.7:0/1058268939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c063230 0x7fb08c063600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb091d23700 1 -- 172.21.15.7:0/1058268939 >> 172.21.15.7:0/1058268939 conn(0x7fb08c0680e0 msgr2=0x7fb08c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb091d23700 1 -- 172.21.15.7:0/1058268939 shutdown_connections 2024-01-11T07:12:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.053+0000 7fb091d23700 1 -- 172.21.15.7:0/1058268939 wait complete. 2024-01-11T07:12:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.055+0000 7fb091d23700 1 Processor -- start 2024-01-11T07:12:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.055+0000 7fb091d23700 1 -- start start 2024-01-11T07:12:05.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.055+0000 7fb091d23700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08c063230 0x7fb08c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:05.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.055+0000 7fb091d23700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb08c0ac5b0 0x7fb08c15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:05.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.055+0000 7fb091d23700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c15a280 0x7fb08c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.055+0000 7fb091d23700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c15fca0 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.055+0000 7fb091d23700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c15f090 con 0x7fb08c063230 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.055+0000 7fb091d23700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c15f370 con 0x7fb08c15a280 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb08c0ac5b0 0x7fb08c15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb08c0ac5b0 0x7fb08c15e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56978/0 (socket says 172.21.15.7:56978) 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 -- 172.21.15.7:0/2337447976 learned_addr learned my addr 172.21.15.7:0/2337447976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08bfff700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c15a280 0x7fb08c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08b7fe700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08c063230 0x7fb08c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 -- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c15a280 msgr2=0x7fb08c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c15a280 0x7fb08c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 -- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08c063230 msgr2=0x7fb08c153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08c063230 0x7fb08c153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 -- 172.21.15.7:0/2337447976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb084009a20 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08bfff700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c15a280 0x7fb08c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb08affd700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb08c0ac5b0 0x7fb08c15e6d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c7c0 tx=0x7fb080007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb0737fe700 1 -- 172.21.15.7:0/2337447976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080005b30 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb0737fe700 1 -- 172.21.15.7:0/2337447976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080005cb0 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.056+0000 7fb0737fe700 1 -- 172.21.15.7:0/2337447976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080007320 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.057+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c009420 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.057+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c15f930 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.058+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c04c3c0 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.058+0000 7fb0737fe700 1 -- 172.21.15.7:0/2337447976 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb08001b3b0 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.059+0000 7fb0737fe700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb078066f30 0x7fb0780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.059+0000 7fb0737fe700 1 -- 172.21.15.7:0/2337447976 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb0800839a0 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.059+0000 7fb08b7fe700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb078066f30 0x7fb0780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.059+0000 7fb08b7fe700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb078066f30 0x7fb0780693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0091e0 tx=0x7fb07c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.063+0000 7fb0737fe700 1 -- 172.21.15.7:0/2337447976 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb080050cd0 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.321+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb08c0c46a0 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.322+0000 7fb0737fe700 1 -- 172.21.15.7:0/2337447976 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb080017020 con 0x7fb08c0ac5b0 2024-01-11T07:12:05.321 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb078066f30 msgr2=0x7fb0780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb078066f30 0x7fb0780693e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0091e0 tx=0x7fb07c017000 comp rx=0 tx=0).stop 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb08c0ac5b0 msgr2=0x7fb08c15e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb08c0ac5b0 0x7fb08c15e6d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c7c0 tx=0x7fb080007e00 comp rx=0 tx=0).stop 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 shutdown_connections 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb078066f30 0x7fb0780693e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb08c15a280 0x7fb08c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08c063230 0x7fb08c153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 --2- 172.21.15.7:0/2337447976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb08c0ac5b0 0x7fb08c15e6d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 >> 172.21.15.7:0/2337447976 conn(0x7fb08c0680e0 msgr2=0x7fb08c0c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 shutdown_connections 2024-01-11T07:12:05.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:05.324+0000 7fb091d23700 1 -- 172.21.15.7:0/2337447976 wait complete. 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:12:05.337 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:12:05.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:05.338 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:12:05.359 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:12:05.359 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:12:05.359 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:12:05.359 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:05.359 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:12:05.359 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:05.359 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:05.359 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42135sec preferred_lft 42135sec 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:05.360 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:12:05.361 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:05.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:05.362 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:12:05.429 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:12:05.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:05.431 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:12:05.493 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:12:05.494 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:12:05.494 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:12:05.494 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:12:05.494 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:12:05.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:05.494 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:12:05.494 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:12:05.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:12:05.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:12:05.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:12:05.494 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:12:05.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:05.605 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:12:05.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:12:05.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:12:05.622 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:12:05.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:05.622 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:12:05.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:12:05.690 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:05.747 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:12:05.747 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:12:05.748 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:05.804 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:12:05.804 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:12:05.804 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:12:05.886 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:12:05.886 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:12:05.886 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:12:05.895 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:12:05.895 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:12:05.895 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:12:05.895 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:12:05.895 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:12:05.912 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:12:05.912 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:12:05.912 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:12:05.912 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:12:05.913 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:12:05.913 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:12:05.913 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:12:05.913 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:12:05.914 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:12:05.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:05.915 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:05.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.123+0000 7f5f2c986700 1 Processor -- start 2024-01-11T07:12:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.123+0000 7f5f2c986700 1 -- start start 2024-01-11T07:12:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f2c986700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f280ac3a0 0x7f5f280cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f2c986700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f2815d920 0x7f5f2815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f2c986700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2815e2d0 con 0x7f5f2815a2c0 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f2c986700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2806ac30 con 0x7f5f2815d920 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f2c986700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f280ac770 con 0x7f5f280ac3a0 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f25d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f2815d920 0x7f5f2815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f25d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f2815d920 0x7f5f2815dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56980/0 (socket says 172.21.15.7:56980) 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f26d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5f2815a2c0 0x7f5f2815a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33344/0 (socket says 172.21.15.7:33344) 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f25d9b700 1 -- 172.21.15.7:0/3560036893 learned_addr learned my addr 172.21.15.7:0/3560036893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.124+0000 7f5f2659c700 1 --2- 172.21.15.7:0/3560036893 >> v2:172.21.15.113:3300/0 conn(0x7f5f280ac3a0 0x7f5f280cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f25d9b700 1 -- 172.21.15.7:0/3560036893 >> v2:172.21.15.113:3300/0 conn(0x7f5f280ac3a0 msgr2=0x7f5f280cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/3560036893 >> v2:172.21.15.113:3300/0 conn(0x7f5f280ac3a0 0x7f5f280cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f25d9b700 1 -- 172.21.15.7:0/3560036893 >> v1:172.21.15.7:6789/0 conn(0x7f5f2815a2c0 legacy=0x7f5f2815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f25d9b700 1 -- 172.21.15.7:0/3560036893 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2806a300 con 0x7f5f2815d920 2024-01-11T07:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f2559a700 1 -- 172.21.15.7:0/3560036893 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405403462 0 0) 0x7f5f2815e2d0 con 0x7f5f2815a2c0 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/3560036893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f2815d920 0x7f5f2815dd90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5f280ccff0 tx=0x7f5f1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e3452ee21c85ce server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f2559a700 1 -- 172.21.15.7:0/3560036893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00b860 con 0x7f5f2815d920 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f2559a700 1 -- 172.21.15.7:0/3560036893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00b9c0 con 0x7f5f2815d920 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f2559a700 1 -- 172.21.15.7:0/3560036893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00bd60 con 0x7f5f2815d920 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.125+0000 7f5f2c986700 1 -- 172.21.15.7:0/3560036893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f2815d920 msgr2=0x7f5f2815dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.126+0000 7f5f2c986700 1 --2- 172.21.15.7:0/3560036893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f2815d920 0x7f5f2815dd90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5f280ccff0 tx=0x7f5f1c009250 comp rx=0 tx=0).stop 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.126+0000 7f5f2c986700 1 -- 172.21.15.7:0/3560036893 shutdown_connections 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.126+0000 7f5f2c986700 1 --2- 172.21.15.7:0/3560036893 >> v2:172.21.15.113:3300/0 conn(0x7f5f280ac3a0 0x7f5f280cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.126+0000 7f5f2c986700 1 --2- 172.21.15.7:0/3560036893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f2815d920 0x7f5f2815dd90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.126+0000 7f5f2c986700 1 -- 172.21.15.7:0/3560036893 >> 172.21.15.7:0/3560036893 conn(0x7f5f280cd090 msgr2=0x7f5f280cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.126+0000 7f5f2c986700 1 -- 172.21.15.7:0/3560036893 shutdown_connections 2024-01-11T07:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.126+0000 7f5f2c986700 1 -- 172.21.15.7:0/3560036893 wait complete. 2024-01-11T07:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.127+0000 7f5f2c986700 1 Processor -- start 2024-01-11T07:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.127+0000 7f5f2c986700 1 -- start start 2024-01-11T07:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2c986700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f280ac3a0 0x7f5f281405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2c986700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815a2c0 0x7f5f2813ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2c986700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815d920 0x7f5f2815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2c986700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280b2d50 con 0x7f5f280ac3a0 2024-01-11T07:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2c986700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f280ad340 con 0x7f5f2815a2c0 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2c986700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f28140cc0 con 0x7f5f2815d920 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f280ac3a0 0x7f5f281405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f280ac3a0 0x7f5f281405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56994/0 (socket says 172.21.15.7:56994) 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 -- 172.21.15.7:0/584615442 learned_addr learned my addr 172.21.15.7:0/584615442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815a2c0 0x7f5f2813ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f26d9d700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815d920 0x7f5f2815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 -- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815d920 msgr2=0x7f5f2815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815d920 0x7f5f2815a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 -- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815a2c0 msgr2=0x7f5f2813ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815a2c0 0x7f5f2813ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 -- 172.21.15.7:0/584615442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0094f0 con 0x7f5f280ac3a0 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f2659c700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f280ac3a0 0x7f5f281405e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800d860 tx=0x7f5f18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.128+0000 7f5f167fc700 1 -- 172.21.15.7:0/584615442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180125f0 con 0x7f5f280ac3a0 2024-01-11T07:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.129+0000 7f5f167fc700 1 -- 172.21.15.7:0/584615442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18012750 con 0x7f5f280ac3a0 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.129+0000 7f5f167fc700 1 -- 172.21.15.7:0/584615442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18012b10 con 0x7f5f280ac3a0 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.129+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f280aca10 con 0x7f5f280ac3a0 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.129+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f280acfd0 con 0x7f5f280ac3a0 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.129+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2804c3c0 con 0x7f5f280ac3a0 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.130+0000 7f5f167fc700 1 -- 172.21.15.7:0/584615442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5f1801e460 con 0x7f5f280ac3a0 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.130+0000 7f5f167fc700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f08066f30 0x7f5f080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.131+0000 7f5f167fc700 1 -- 172.21.15.7:0/584615442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f18084a60 con 0x7f5f280ac3a0 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.131+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f08066f30 0x7f5f080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.131+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f08066f30 0x7f5f080693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f280ccff0 tx=0x7f5f1c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:06.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.135+0000 7f5f167fc700 1 -- 172.21.15.7:0/584615442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f18055760 con 0x7f5f280ac3a0 2024-01-11T07:12:06.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.389+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f2806a5e0 con 0x7f5f280ac3a0 2024-01-11T07:12:06.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.390+0000 7f5f167fc700 1 -- 172.21.15.7:0/584615442 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f5f180552f0 con 0x7f5f280ac3a0 2024-01-11T07:12:06.389 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:06.389 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f08066f30 msgr2=0x7f5f080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f08066f30 0x7f5f080693e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5f280ccff0 tx=0x7f5f1c017000 comp rx=0 tx=0).stop 2024-01-11T07:12:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f280ac3a0 msgr2=0x7f5f281405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f280ac3a0 0x7f5f281405e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800d860 tx=0x7f5f18008040 comp rx=0 tx=0).stop 2024-01-11T07:12:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 shutdown_connections 2024-01-11T07:12:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f08066f30 0x7f5f080693e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f2815d920 0x7f5f2815a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f280ac3a0 0x7f5f281405e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 --2- 172.21.15.7:0/584615442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f2815a2c0 0x7f5f2813ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:06.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 >> 172.21.15.7:0/584615442 conn(0x7f5f280cd090 msgr2=0x7f5f28159620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:06.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 shutdown_connections 2024-01-11T07:12:06.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:06.392+0000 7f5f2c986700 1 -- 172.21.15.7:0/584615442 wait complete. 2024-01-11T07:12:06.393 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:06.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-11T07:12:06.558 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":6175,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8656359350000002,"mdsmap_epoch":50,"osdmap_epoch":100,"osdmap_epoch_barrier":100,"uptime":13.689682804} 2024-01-11T07:12:06.558 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 6175, "mdsmap_epoch": 50, "osdmap_epoch": 100, "osdmap_epoch_barrier": 100, "rank_uptime": 8.865635935, "state": "up:active", "uptime": 13.689682804, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:12:06.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-11T07:12:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-11T07:12:06.707 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-11T07:12:06.708 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-01-11T07:12:06.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:06.708 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-01-11T07:12:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:265+0 records in 2024-01-11T07:12:08.221 INFO:teuthology.orchestra.run.smithi007.stderr:265+0 records out 2024-01-11T07:12:08.221 INFO:teuthology.orchestra.run.smithi007.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.49085 s, 186 MB/s 2024-01-11T07:12:08.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:08.222 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-01-11T07:12:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:292+0 records in 2024-01-11T07:12:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:292+0 records out 2024-01-11T07:12:09.900 INFO:teuthology.orchestra.run.smithi007.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.65591 s, 185 MB/s 2024-01-11T07:12:09.901 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-01-11T07:12:09.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:12:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.037+0000 7f706eab5700 1 Processor -- start 2024-01-11T07:12:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.037+0000 7f706eab5700 1 -- start start 2024-01-11T07:12:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.038+0000 7f706eab5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f706815a060 0x7f70680b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.038+0000 7f706eab5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70680ce9b0 con 0x7f7068159c90 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.038+0000 7f706eab5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70680cf530 con 0x7f7068069320 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.038+0000 7f706eab5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70680b1f30 con 0x7f706815a060 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.038+0000 7f706c851700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7068159c90 0x7f70680b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40990/0 (socket says 172.21.15.7:40990) 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.038+0000 7f706c851700 1 -- 172.21.15.7:0/1412718031 learned_addr learned my addr 172.21.15.7:0/1412718031 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.038+0000 7f7067fff700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f706815a060 0x7f70680b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.038+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267429044 0 0) 0x7f70680ce9b0 con 0x7f7068159c90 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7058003570 con 0x7f7068159c90 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4231378122 0 0) 0x7f7058003570 con 0x7f7068159c90 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f706815a060 msgr2=0x7f70680b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f706815a060 0x7f70680b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 >> v1:172.21.15.113:6789/0 conn(0x7f7068069320 legacy=0x7f70680b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70680cf9c0 con 0x7f7068159c90 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f705c001470 con 0x7f7068159c90 2024-01-11T07:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 >> v1:172.21.15.7:6789/0 conn(0x7f7068159c90 legacy=0x7f70680b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7058003a90 0x7f7058003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f706815a060 0x7f7058004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7058004d50 0x7f70580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705800bb10 con 0x7f7058004d50 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705800bc50 con 0x7f7058003a90 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705800be10 con 0x7f706815a060 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f706c851700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7058003a90 0x7f7058003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f7067fff700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f706815a060 0x7f7058004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f706d052700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7058004d50 0x7f70580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f706d052700 1 -- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f706815a060 msgr2=0x7f7058004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f706d052700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f706815a060 0x7f7058004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f706d052700 1 -- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7058003a90 msgr2=0x7f7058003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f706d052700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7058003a90 0x7f7058003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.039+0000 7f706d052700 1 -- 172.21.15.7:0/1412718031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7058003570 con 0x7f7058004d50 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f706d052700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7058004d50 0x7f70580051c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7060002070 tx=0x7f70600016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3a40d1b803fa60 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70600019e0 con 0x7f7058004d50 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706000c8b0 con 0x7f7058004d50 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f70677fe700 1 -- 172.21.15.7:0/1412718031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706000cc50 con 0x7f7058004d50 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f706eab5700 1 -- 172.21.15.7:0/1412718031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7058004d50 msgr2=0x7f70580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f706eab5700 1 --2- 172.21.15.7:0/1412718031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7058004d50 0x7f70580051c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7060002070 tx=0x7f70600016d0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f706d052700 1 -- 172.21.15.7:0/1412718031 reap_dead start 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f706eab5700 1 -- 172.21.15.7:0/1412718031 shutdown_connections 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f706eab5700 1 -- 172.21.15.7:0/1412718031 >> 172.21.15.7:0/1412718031 conn(0x7f7068071100 msgr2=0x7f706806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f706eab5700 1 -- 172.21.15.7:0/1412718031 shutdown_connections 2024-01-11T07:12:10.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.040+0000 7f706eab5700 1 -- 172.21.15.7:0/1412718031 wait complete. 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.042+0000 7f706eab5700 1 Processor -- start 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.042+0000 7f706eab5700 1 -- start start 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706eab5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7058004d50 0x7f706814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706eab5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f706816acd0 0x7f706815f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706eab5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706816b140 0x7f706815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706eab5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7068150c10 con 0x7f706816b140 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706eab5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706814fd60 con 0x7f706816acd0 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706eab5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7068150000 con 0x7f7058004d50 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706816b140 0x7f706815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706c851700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7058004d50 0x7f706814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706816b140 0x7f706815f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36576/0 (socket says 172.21.15.7:36576) 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 -- 172.21.15.7:0/920056562 learned_addr learned my addr 172.21.15.7:0/920056562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f7067fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f706816acd0 0x7f706815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 -- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7058004d50 msgr2=0x7f706814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7058004d50 0x7f706814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 -- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f706816acd0 msgr2=0x7f706815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f706816acd0 0x7f706815f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 -- 172.21.15.7:0/920056562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70681502e0 con 0x7f706816b140 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f7067fff700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f706816acd0 0x7f706815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706d052700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706816b140 0x7f706815f670 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7058004770 tx=0x7f7060015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f7064ff9700 1 -- 172.21.15.7:0/920056562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70600019e0 con 0x7f706816b140 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f7064ff9700 1 -- 172.21.15.7:0/920056562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7060012dd0 con 0x7f706816b140 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70681505c0 con 0x7f706816b140 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f7064ff9700 1 -- 172.21.15.7:0/920056562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706001f660 con 0x7f706816b140 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.043+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706814fae0 con 0x7f706816b140 2024-01-11T07:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.044+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706804c3c0 con 0x7f706816b140 2024-01-11T07:12:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.045+0000 7f7064ff9700 1 -- 172.21.15.7:0/920056562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f706001fa50 con 0x7f706816b140 2024-01-11T07:12:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.045+0000 7f7064ff9700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f704c066f30 0x7f704c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.046+0000 7f7064ff9700 1 -- 172.21.15.7:0/920056562 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70600864d0 con 0x7f706816b140 2024-01-11T07:12:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.046+0000 7f706c851700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f704c066f30 0x7f704c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.046+0000 7f706c851700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f704c066f30 0x7f704c0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f705c0010e0 tx=0x7f705c001df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:10.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.050+0000 7f7064ff9700 1 -- 172.21.15.7:0/920056562 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7060053740 con 0x7f706816b140 2024-01-11T07:12:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.257+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f70681578d0 con 0x7f706816b140 2024-01-11T07:12:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.258+0000 7f7064ff9700 1 -- 172.21.15.7:0/920056562 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v100) v1 ==== 81+0+32217 (secure 0 0 0) 0x7f7060056d60 con 0x7f706816b140 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 100, 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:11:58.216644+0000", 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:12:10.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:12:10.259 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:10.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:10.261 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "91", 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:10.262 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:10.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:10.264 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:55.626974+0000", 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:10.266 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:10.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "99", 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:10.268 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:10.269 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:10.270 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:10.271 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:56.083170+0000", 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:10.272 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:10.273 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "100", 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:10.274 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:10.275 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:10.276 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:10.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:12:10.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:12:10.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:10.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:10.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:12:10.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.289 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:10.290 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:12:10.293 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:12:10.295 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:10.296 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:10.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:10.302 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.304 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.305 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:10.307 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:12:10.308 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:12:10.312 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:10.313 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:12:10.314 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.316 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.320 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.321 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:10.322 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:10.323 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:12:10.324 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:10.327 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:12:10.328 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:12:10.329 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:12:10.330 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:10.331 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f704c066f30 msgr2=0x7f704c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f704c066f30 0x7f704c0693e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f705c0010e0 tx=0x7f705c001df0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706816b140 msgr2=0x7f706815f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706816b140 0x7f706815f670 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7058004770 tx=0x7f7060015000 comp rx=0 tx=0).stop 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 shutdown_connections 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f704c066f30 0x7f704c0693e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706816b140 0x7f706815f670 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7058004d50 0x7f706814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 --2- 172.21.15.7:0/920056562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f706816acd0 0x7f706815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 >> 172.21.15.7:0/920056562 conn(0x7f7068071100 msgr2=0x7f706806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 shutdown_connections 2024-01-11T07:12:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:10.260+0000 7f706eab5700 1 -- 172.21.15.7:0/920056562 wait complete. 2024-01-11T07:12:10.334 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:12:12.236 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:12:12.237+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-01-11T07:12:15.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:12:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.475+0000 7fb0ef9be700 1 Processor -- start 2024-01-11T07:12:15.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.475+0000 7fb0ef9be700 1 -- start start 2024-01-11T07:12:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.476+0000 7fb0ef9be700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e80cd6d0 0x7fb0e80cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.476+0000 7fb0ef9be700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0e815d8a0 0x7fb0e815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.476+0000 7fb0ef9be700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0e815e250 0x7fb0e80cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.476+0000 7fb0ef9be700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e815e690 con 0x7fb0e815e250 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.476+0000 7fb0ef9be700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e815acf0 con 0x7fb0e815d8a0 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.476+0000 7fb0ef9be700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e815afd0 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.476+0000 7fb0edf5b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0e815e250 0x7fb0e80cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0edf5b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0e815e250 0x7fb0e80cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36582/0 (socket says 172.21.15.7:36582) 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0edf5b700 1 -- 172.21.15.7:0/235770567 learned_addr learned my addr 172.21.15.7:0/235770567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0ed75a700 1 --2- 172.21.15.7:0/235770567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e80cd6d0 0x7fb0e80cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0ecf59700 1 --2- 172.21.15.7:0/235770567 >> v2:172.21.15.113:3300/0 conn(0x7fb0e815d8a0 0x7fb0e815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0edf5b700 1 -- 172.21.15.7:0/235770567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e80cd6d0 msgr2=0x7fb0e80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0edf5b700 1 --2- 172.21.15.7:0/235770567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e80cd6d0 0x7fb0e80cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0edf5b700 1 -- 172.21.15.7:0/235770567 >> v2:172.21.15.113:3300/0 conn(0x7fb0e815d8a0 msgr2=0x7fb0e815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0edf5b700 1 --2- 172.21.15.7:0/235770567 >> v2:172.21.15.113:3300/0 conn(0x7fb0e815d8a0 0x7fb0e815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0edf5b700 1 -- 172.21.15.7:0/235770567 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80ce980 con 0x7fb0e815e250 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0ecf59700 1 --2- 172.21.15.7:0/235770567 >> v2:172.21.15.113:3300/0 conn(0x7fb0e815d8a0 0x7fb0e815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0edf5b700 1 --2- 172.21.15.7:0/235770567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e815e250 0x7fb0e80cf160 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80677f0 tx=0x7fb0e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425e55b7f522f837 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0dffff700 1 -- 172.21.15.7:0/235770567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000e070 con 0x7fb0e815e250 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0dffff700 1 -- 172.21.15.7:0/235770567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e000ba20 con 0x7fb0e815e250 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.477+0000 7fb0dffff700 1 -- 172.21.15.7:0/235770567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00092e0 con 0x7fb0e815e250 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 -- 172.21.15.7:0/235770567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e815e250 msgr2=0x7fb0e80cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/235770567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e815e250 0x7fb0e80cf160 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80677f0 tx=0x7fb0e0009840 comp rx=0 tx=0).stop 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 -- 172.21.15.7:0/235770567 shutdown_connections 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/235770567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e815e250 0x7fb0e80cf160 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/235770567 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e80cd6d0 0x7fb0e80cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/235770567 >> v2:172.21.15.113:3300/0 conn(0x7fb0e815d8a0 0x7fb0e815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 -- 172.21.15.7:0/235770567 >> 172.21.15.7:0/235770567 conn(0x7fb0e8071120 msgr2=0x7fb0e806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:15.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 -- 172.21.15.7:0/235770567 shutdown_connections 2024-01-11T07:12:15.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.478+0000 7fb0ef9be700 1 -- 172.21.15.7:0/235770567 wait complete. 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.479+0000 7fb0ef9be700 1 Processor -- start 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.479+0000 7fb0ef9be700 1 -- start start 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ef9be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cd6d0 0x7fb0e80b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ef9be700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0e815d8a0 0x7fb0e8146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ef9be700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e815e250 0x7fb0e80cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ef9be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80cafc0 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ef9be700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80ca3b0 con 0x7fb0e815d8a0 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ef9be700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e80ca690 con 0x7fb0e815e250 2024-01-11T07:12:15.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ed75a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cd6d0 0x7fb0e80b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ed75a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cd6d0 0x7fb0e80b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36588/0 (socket says 172.21.15.7:36588) 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.480+0000 7fb0ed75a700 1 -- 172.21.15.7:0/996677358 learned_addr learned my addr 172.21.15.7:0/996677358 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ecf59700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0e815d8a0 0x7fb0e8146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0edf5b700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e815e250 0x7fb0e80cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ed75a700 1 -- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e815e250 msgr2=0x7fb0e80cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ed75a700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e815e250 0x7fb0e80cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ed75a700 1 -- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0e815d8a0 msgr2=0x7fb0e8146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ed75a700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0e815d8a0 0x7fb0e8146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ed75a700 1 -- 172.21.15.7:0/996677358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0009ae0 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ecf59700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0e815d8a0 0x7fb0e8146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0edf5b700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e815e250 0x7fb0e80cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ed75a700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cd6d0 0x7fb0e80b22e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400ec80 tx=0x7fb0e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0dd7fa700 1 -- 172.21.15.7:0/996677358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400c9d0 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0dd7fa700 1 -- 172.21.15.7:0/996677358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400cb30 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e80ca970 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0dd7fa700 1 -- 172.21.15.7:0/996677358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4013470 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.481+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e80b2820 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.482+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e804c3c0 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.483+0000 7fb0dd7fa700 1 -- 172.21.15.7:0/996677358 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb0e4013820 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.483+0000 7fb0dd7fa700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0cc066f30 0x7fb0cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:15.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.483+0000 7fb0dd7fa700 1 -- 172.21.15.7:0/996677358 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb0e40848c0 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.483+0000 7fb0dd7fa700 1 -- 172.21.15.7:0/996677358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=103}) v3 -- 0x7fb0cc06ea50 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.484+0000 7fb0ecf59700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0cc066f30 0x7fb0cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:15.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.484+0000 7fb0ecf59700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0cc066f30 0x7fb0cc0693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8006f00 tx=0x7fb0d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:15.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.488+0000 7fb0dd7fa700 1 -- 172.21.15.7:0/996677358 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0e4017060 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.692+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb0e80ce490 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.693+0000 7fb0dd7fa700 1 -- 172.21.15.7:0/996677358 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v102) v1 ==== 81+0+32362 (secure 0 0 0) 0x7fb0e4021090 con 0x7fb0e80cd6d0 2024-01-11T07:12:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:15.693 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 102, 2024-01-11T07:12:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:12:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:12:13.242582+0000", 2024-01-11T07:12:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:12:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:12:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:12:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:55.626974+0000", 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:12:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:56.083170+0000", 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:15.707 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:15.708 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:15.709 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:15.710 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:15.711 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:15.712 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:15.713 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:12:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:12:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:12:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.737 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:15.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:15.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:12:15.740 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.741 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:12:15.746 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:15.747 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:15.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:15.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:12:15.750 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:12:15.751 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:12:15.752 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:15.753 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:12:15.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:12:15.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:15.758 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:15.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:15.760 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:15.761 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:15.762 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:15.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:15.764 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:15.766 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0cc066f30 msgr2=0x7fb0cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0cc066f30 0x7fb0cc0693e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8006f00 tx=0x7fb0d8008040 comp rx=0 tx=0).stop 2024-01-11T07:12:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cd6d0 msgr2=0x7fb0e80b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cd6d0 0x7fb0e80b22e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400ec80 tx=0x7fb0e40091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 shutdown_connections 2024-01-11T07:12:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb0cc066f30 0x7fb0cc0693e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0e815e250 0x7fb0e80cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0e80cd6d0 0x7fb0e80b22e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 --2- 172.21.15.7:0/996677358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0e815d8a0 0x7fb0e8146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:15.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 >> 172.21.15.7:0/996677358 conn(0x7fb0e8071120 msgr2=0x7fb0e80c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:15.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 shutdown_connections 2024-01-11T07:12:15.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:15.695+0000 7fb0ef9be700 1 -- 172.21.15.7:0/996677358 wait complete. 2024-01-11T07:12:15.769 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-01-11T07:12:15.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:15.770 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-01-11T07:12:16.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:12:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-01-11T07:12:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-11T07:12:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-11T07:12:16.055 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.257712 s, 203 MB/s 2024-01-11T07:12:16.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:12:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.187+0000 7fb72147b700 1 Processor -- start 2024-01-11T07:12:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.188+0000 7fb72147b700 1 -- start start 2024-01-11T07:12:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.188+0000 7fb72147b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb71c0aed00 0x7fb71c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.188+0000 7fb72147b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb71c158010 con 0x7fb71c06a410 2024-01-11T07:12:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.188+0000 7fb72147b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb71c15b530 con 0x7fb71c156e90 2024-01-11T07:12:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.188+0000 7fb72147b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c157400 con 0x7fb71c0aed00 2024-01-11T07:12:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb71b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb71c06a410 0x7fb71c0cafc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40994/0 (socket says 172.21.15.7:40994) 2024-01-11T07:12:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb71b7fe700 1 -- 172.21.15.7:0/4181628861 learned_addr learned my addr 172.21.15.7:0/4181628861 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb71affd700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb71c0aed00 0x7fb71c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685619217 0 0) 0x7fb71c158010 con 0x7fb71c06a410 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb708003570 con 0x7fb71c06a410 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625653460 0 0) 0x7fb71c15b530 con 0x7fb71c156e90 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb71c158010 con 0x7fb71c156e90 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333665289 0 0) 0x7fb708003570 con 0x7fb71c06a410 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb71c0aed00 msgr2=0x7fb71c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb71c0aed00 0x7fb71c15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 >> v1:172.21.15.113:6789/0 conn(0x7fb71c156e90 legacy=0x7fb71c069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c1576e0 con 0x7fb71c06a410 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fb7140017f0 con 0x7fb71c06a410 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 >> v1:172.21.15.7:6789/0 conn(0x7fb71c06a410 legacy=0x7fb71c0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb708003c20 0x7fb708004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb708004810 0x7fb708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb71c0aed00 0x7fb708005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c157400 con 0x7fb708003c20 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70800bbf0 con 0x7fb708004810 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70800bdb0 con 0x7fb71c0aed00 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.189+0000 7fb71affd700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb708003c20 0x7fb708004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb71b7fe700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb71c0aed00 0x7fb708005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb71a7fc700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb708004810 0x7fb708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb71affd700 1 -- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb71c0aed00 msgr2=0x7fb708005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb71affd700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb71c0aed00 0x7fb708005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb71affd700 1 -- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb708004810 msgr2=0x7fb708004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb71affd700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb708004810 0x7fb708004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb71affd700 1 -- 172.21.15.7:0/4181628861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708003570 con 0x7fb708003c20 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb71affd700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb708003c20 0x7fb708004010 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb704009c00 tx=0x7fb7040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3a427b64ded3f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400e710 con 0x7fb708003c20 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70400e870 con 0x7fb708003c20 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.190+0000 7fb719ffb700 1 -- 172.21.15.7:0/4181628861 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400ec30 con 0x7fb708003c20 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.191+0000 7fb72147b700 1 -- 172.21.15.7:0/4181628861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb708003c20 msgr2=0x7fb708004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.191+0000 7fb72147b700 1 --2- 172.21.15.7:0/4181628861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb708003c20 0x7fb708004010 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb704009c00 tx=0x7fb7040091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.191+0000 7fb71b7fe700 1 -- 172.21.15.7:0/4181628861 reap_dead start 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.191+0000 7fb72147b700 1 -- 172.21.15.7:0/4181628861 shutdown_connections 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.191+0000 7fb72147b700 1 -- 172.21.15.7:0/4181628861 >> 172.21.15.7:0/4181628861 conn(0x7fb71c063230 msgr2=0x7fb71c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.191+0000 7fb72147b700 1 -- 172.21.15.7:0/4181628861 shutdown_connections 2024-01-11T07:12:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.191+0000 7fb72147b700 1 -- 172.21.15.7:0/4181628861 wait complete. 2024-01-11T07:12:16.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.193+0000 7fb72147b700 1 Processor -- start 2024-01-11T07:12:16.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.193+0000 7fb72147b700 1 -- start start 2024-01-11T07:12:16.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb72147b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb708003c20 0x7fb71c15bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb72147b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c15a6f0 0x7fb71c0b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb72147b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c155850 0x7fb71c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb72147b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c15b870 con 0x7fb71c155850 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb72147b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c15c240 con 0x7fb71c15a6f0 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb72147b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c0b21e0 con 0x7fb708003c20 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c155850 0x7fb71c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c155850 0x7fb71c068bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36600/0 (socket says 172.21.15.7:36600) 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 -- 172.21.15.7:0/1607801577 learned_addr learned my addr 172.21.15.7:0/1607801577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71a7fc700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c15a6f0 0x7fb71c0b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71affd700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb708003c20 0x7fb71c15bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 -- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb708003c20 msgr2=0x7fb71c15bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb708003c20 0x7fb71c15bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 -- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c15a6f0 msgr2=0x7fb71c0b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c15a6f0 0x7fb71c0b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 -- 172.21.15.7:0/1607801577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c15ac60 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.194+0000 7fb71b7fe700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c155850 0x7fb71c068bf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c6da0 tx=0x7fb7140020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.195+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400ac90 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.195+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71400ae10 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.195+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714013640 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.195+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71c15af40 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.195+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c15b500 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.196+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71c15ba20 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.196+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb714013a30 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.197+0000 7fb712ffd700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6fc066f30 0x7fb6fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.197+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb714084d00 con 0x7fb71c155850 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.197+0000 7fb71affd700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6fc066f30 0x7fb6fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.197+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=103}) v3 -- 0x7fb6fc06ea50 con 0x7fb71c155850 2024-01-11T07:12:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.197+0000 7fb71affd700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6fc066f30 0x7fb6fc0693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb7040091e0 tx=0x7fb70401a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.201+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb71c15ba20 con 0x7fb71c155850 2024-01-11T07:12:16.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.250+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(103..103 src has 1..103) v4 ==== 288+0+0 (secure 0 0 0) 0x7fb7140563f0 con 0x7fb71c155850 2024-01-11T07:12:16.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.250+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=104}) v3 -- 0x7fb6fc06f3a0 con 0x7fb71c155850 2024-01-11T07:12:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.409+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb71c04c3c0 con 0x7fb71c155850 2024-01-11T07:12:16.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.410+0000 7fb712ffd700 1 -- 172.21.15.7:0/1607801577 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+32386 (secure 0 0 0) 0x7fb714052030 con 0x7fb71c155850 2024-01-11T07:12:16.410 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 103, 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:12:16.246827+0000", 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:12:16.411 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:12:16.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:16.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:12:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:16.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:16.416 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:16.417 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:16.418 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-11T07:12:16.419 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:55.626974+0000", 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:16.420 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:16.421 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:16.422 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:16.423 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:16.424 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:16.425 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:56.083170+0000", 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:16.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:12:16.427 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:16.429 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:16.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:16.431 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.432 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:16.433 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:16.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:16.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:12:16.438 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:16.439 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:16.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:12:16.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:12:16.444 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:16.445 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:12:16.449 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:16.451 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:16.452 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:16.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.457 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:16.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:16.459 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:12:16.463 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:16.465 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:16.466 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:12:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.470 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:12:16.471 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:16.472 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:16.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.478 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:16.479 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:12:16.482 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:12:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:12:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:12:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:12:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:16.494 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6fc066f30 msgr2=0x7fb6fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6fc066f30 0x7fb6fc0693e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb7040091e0 tx=0x7fb70401a000 comp rx=0 tx=0).stop 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c155850 msgr2=0x7fb71c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c155850 0x7fb71c068bf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c6da0 tx=0x7fb7140020a0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 shutdown_connections 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6fc066f30 0x7fb6fc0693e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71c155850 0x7fb71c068bf0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb708003c20 0x7fb71c15bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 --2- 172.21.15.7:0/1607801577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb71c15a6f0 0x7fb71c0b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 >> 172.21.15.7:0/1607801577 conn(0x7fb71c063230 msgr2=0x7fb71c0c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 shutdown_connections 2024-01-11T07:12:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.413+0000 7fb72147b700 1 -- 172.21.15.7:0/1607801577 wait complete. 2024-01-11T07:12:16.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.632+0000 7f3a300d5700 1 Processor -- start 2024-01-11T07:12:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.633+0000 7f3a300d5700 1 -- start start 2024-01-11T07:12:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a300d5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3a280adf10 0x7f3a280b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a300d5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a300d5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a280cc2a0 con 0x7f3a280adb40 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a300d5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a280b3b50 con 0x7f3a280adf10 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a300d5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280b3e30 con 0x7f3a280b3590 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2e672700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2d670700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3a280adf10 0x7f3a280b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2e672700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:54718/0 (socket says 172.21.15.7:54718) 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2d670700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3a280adf10 0x7f3a280b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:56450/0 (socket says 172.21.15.7:56450) 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2de71700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3a280adb40 0x7f3a280b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:56566/0 (socket says 172.21.15.7:56566) 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2e672700 1 -- 172.21.15.7:0/1268918871 learned_addr learned my addr 172.21.15.7:0/1268918871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2ce6f700 1 -- 172.21.15.7:0/1268918871 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774280737 0 0) 0x7f3a280cc2a0 con 0x7f3a280adb40 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2ce6f700 1 -- 172.21.15.7:0/1268918871 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a14003570 con 0x7f3a280adb40 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2e672700 1 -- 172.21.15.7:0/1268918871 >> v2:172.21.15.113:3300/0 conn(0x7f3a280adf10 msgr2=0x7f3a280b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2e672700 1 --2- 172.21.15.7:0/1268918871 >> v2:172.21.15.113:3300/0 conn(0x7f3a280adf10 0x7f3a280b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.634+0000 7f3a2e672700 1 -- 172.21.15.7:0/1268918871 >> v1:172.21.15.113:6789/0 conn(0x7f3a280adb40 legacy=0x7f3a280b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.635+0000 7f3a2e672700 1 -- 172.21.15.7:0/1268918871 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a280cc540 con 0x7f3a280b3590 2024-01-11T07:12:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.635+0000 7f3a2d670700 1 --2- 172.21.15.7:0/1268918871 >> v2:172.21.15.113:3300/0 conn(0x7f3a280adf10 0x7f3a280b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.635+0000 7f3a2e672700 1 --2- 172.21.15.7:0/1268918871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0970 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009f20 tx=0x7f3a20009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57bc2a6926199f97 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.635+0000 7f3a2ce6f700 1 -- 172.21.15.7:0/1268918871 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000e070 con 0x7f3a280b3590 2024-01-11T07:12:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.635+0000 7f3a2ce6f700 1 -- 172.21.15.7:0/1268918871 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000b970 con 0x7f3a280b3590 2024-01-11T07:12:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.635+0000 7f3a2ce6f700 1 -- 172.21.15.7:0/1268918871 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a200092e0 con 0x7f3a280b3590 2024-01-11T07:12:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.635+0000 7f3a300d5700 1 -- 172.21.15.7:0/1268918871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 msgr2=0x7f3a280b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.636+0000 7f3a300d5700 1 --2- 172.21.15.7:0/1268918871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0970 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009f20 tx=0x7f3a20009840 comp rx=0 tx=0).stop 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.636+0000 7f3a300d5700 1 -- 172.21.15.7:0/1268918871 shutdown_connections 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.636+0000 7f3a300d5700 1 --2- 172.21.15.7:0/1268918871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0970 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.636+0000 7f3a300d5700 1 --2- 172.21.15.7:0/1268918871 >> v2:172.21.15.113:3300/0 conn(0x7f3a280adf10 0x7f3a280b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.636+0000 7f3a300d5700 1 -- 172.21.15.7:0/1268918871 >> 172.21.15.7:0/1268918871 conn(0x7f3a28071160 msgr2=0x7f3a2806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.636+0000 7f3a300d5700 1 -- 172.21.15.7:0/1268918871 shutdown_connections 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.636+0000 7f3a300d5700 1 -- 172.21.15.7:0/1268918871 wait complete. 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.637+0000 7f3a300d5700 1 Processor -- start 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.637+0000 7f3a300d5700 1 -- start start 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a300d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a280adb40 0x7f3a2814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a300d5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a280adf10 0x7f3a28141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a300d5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a300d5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28141a80 con 0x7f3a280adb40 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a300d5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280afea0 con 0x7f3a280adf10 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a300d5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280af570 con 0x7f3a280b3590 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a280adb40 0x7f3a2814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a280adb40 0x7f3a2814e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36606/0 (socket says 172.21.15.7:36606) 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 -- 172.21.15.7:0/161418519 learned_addr learned my addr 172.21.15.7:0/161418519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2e672700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2d670700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a280adf10 0x7f3a28141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 -- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 msgr2=0x7f3a280b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 -- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a280adf10 msgr2=0x7f3a28141540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a280adf10 0x7f3a28141540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 -- 172.21.15.7:0/161418519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c00d8d0 con 0x7f3a280adb40 2024-01-11T07:12:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.638+0000 7f3a2de71700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a280adb40 0x7f3a2814e5c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c7f0 tx=0x7f3a24007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.639+0000 7f3a19ffb700 1 -- 172.21.15.7:0/161418519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24005c10 con 0x7f3a280adb40 2024-01-11T07:12:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.639+0000 7f3a19ffb700 1 -- 172.21.15.7:0/161418519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a24005d90 con 0x7f3a280adb40 2024-01-11T07:12:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.639+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a20009ae0 con 0x7f3a280adb40 2024-01-11T07:12:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.639+0000 7f3a19ffb700 1 -- 172.21.15.7:0/161418519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24007320 con 0x7f3a280adb40 2024-01-11T07:12:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.639+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a280afb30 con 0x7f3a280adb40 2024-01-11T07:12:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.640+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2804c3c0 con 0x7f3a280adb40 2024-01-11T07:12:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.640+0000 7f3a19ffb700 1 -- 172.21.15.7:0/161418519 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3a2401b3b0 con 0x7f3a280adb40 2024-01-11T07:12:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.641+0000 7f3a19ffb700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3a0c066f30 0x7f3a0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.641+0000 7f3a19ffb700 1 -- 172.21.15.7:0/161418519 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a24083be0 con 0x7f3a280adb40 2024-01-11T07:12:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.641+0000 7f3a19ffb700 1 -- 172.21.15.7:0/161418519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=104}) v3 -- 0x7f3a0c06ea50 con 0x7f3a280adb40 2024-01-11T07:12:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.641+0000 7f3a2d670700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3a0c066f30 0x7f3a0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.642+0000 7f3a2d670700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3a0c066f30 0x7f3a0c0693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00dfd0 tx=0x7f3a1c00f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.645+0000 7f3a19ffb700 1 -- 172.21.15.7:0/161418519 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a24017020 con 0x7f3a280adb40 2024-01-11T07:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.896+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a280b0180 con 0x7f3a280adb40 2024-01-11T07:12:16.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.896+0000 7f3a19ffb700 1 -- 172.21.15.7:0/161418519 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f3a24050e50 con 0x7f3a280adb40 2024-01-11T07:12:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:16.896 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3a0c066f30 msgr2=0x7f3a0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3a0c066f30 0x7f3a0c0693e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00dfd0 tx=0x7f3a1c00f040 comp rx=0 tx=0).stop 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a280adb40 msgr2=0x7f3a2814e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a280adb40 0x7f3a2814e5c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c7f0 tx=0x7f3a24007e00 comp rx=0 tx=0).stop 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 shutdown_connections 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3a0c066f30 0x7f3a0c0693e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3a280b3590 0x7f3a280b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a280adb40 0x7f3a2814e5c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 --2- 172.21.15.7:0/161418519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a280adf10 0x7f3a28141540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 >> 172.21.15.7:0/161418519 conn(0x7f3a28071160 msgr2=0x7f3a28068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:16.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 shutdown_connections 2024-01-11T07:12:16.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:16.899+0000 7f3a300d5700 1 -- 172.21.15.7:0/161418519 wait complete. 2024-01-11T07:12:16.900 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:16.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-11T07:12:17.063 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":6175,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.370853422,"mdsmap_epoch":50,"osdmap_epoch":103,"osdmap_epoch_barrier":102,"uptime":24.194900739000001} 2024-01-11T07:12:17.063 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 6175, "mdsmap_epoch": 50, "osdmap_epoch": 103, "osdmap_epoch_barrier": 102, "rank_uptime": 19.370853422, "state": "up:active", "uptime": 24.194900739, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:12:17.063 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:12:17.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:17.064 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-01-11T07:12:17.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:12:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:dd: failed to open 'small_file_1': Permission denied 2024-01-11T07:12:17.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:17.083 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-01-11T07:12:17.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:17.145 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-01-11T07:12:17.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:12:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.410+0000 7fada436a700 1 Processor -- start 2024-01-11T07:12:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.410+0000 7fada436a700 1 -- start start 2024-01-11T07:12:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.411+0000 7fada436a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad9c159c90 0x7fad9c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.411+0000 7fada436a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c15a060 0x7fad9c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.411+0000 7fada436a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c0ce9b0 con 0x7fad9c069320 2024-01-11T07:12:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.411+0000 7fada436a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c0cf530 con 0x7fad9c159c90 2024-01-11T07:12:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.411+0000 7fada436a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0cf9c0 con 0x7fad9c15a060 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada1905700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c15a060 0x7fad9c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada2106700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad9c159c90 0x7fad9c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada2907700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fad9c069320 0x7fad9c067610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:56976/0 (socket says 172.21.15.7:56976) 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada2907700 1 -- 172.21.15.7:0/3376754079 learned_addr learned my addr 172.21.15.7:0/3376754079 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada1905700 1 -- 172.21.15.7:0/3376754079 >> v2:172.21.15.113:3300/0 conn(0x7fad9c159c90 msgr2=0x7fad9c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada1905700 1 --2- 172.21.15.7:0/3376754079 >> v2:172.21.15.113:3300/0 conn(0x7fad9c159c90 0x7fad9c0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada1905700 1 -- 172.21.15.7:0/3376754079 >> v1:172.21.15.113:6789/0 conn(0x7fad9c069320 legacy=0x7fad9c067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada1905700 1 -- 172.21.15.7:0/3376754079 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c0cec00 con 0x7fad9c15a060 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada1104700 1 -- 172.21.15.7:0/3376754079 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439495436 0 0) 0x7fad9c0ce9b0 con 0x7fad9c069320 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada2106700 1 --2- 172.21.15.7:0/3376754079 >> v2:172.21.15.113:3300/0 conn(0x7fad9c159c90 0x7fad9c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada1905700 1 --2- 172.21.15.7:0/3376754079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c15a060 0x7fad9c0b0430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad900098e0 tx=0x7fad900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6bcf88a85d286c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.412+0000 7fada1104700 1 -- 172.21.15.7:0/3376754079 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9000b640 con 0x7fad9c15a060 2024-01-11T07:12:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada1104700 1 -- 172.21.15.7:0/3376754079 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9000b7a0 con 0x7fad9c15a060 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada1104700 1 -- 172.21.15.7:0/3376754079 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9000bb40 con 0x7fad9c15a060 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada436a700 1 -- 172.21.15.7:0/3376754079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c15a060 msgr2=0x7fad9c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada436a700 1 --2- 172.21.15.7:0/3376754079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c15a060 0x7fad9c0b0430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad900098e0 tx=0x7fad900091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada436a700 1 -- 172.21.15.7:0/3376754079 shutdown_connections 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada436a700 1 --2- 172.21.15.7:0/3376754079 >> v2:172.21.15.113:3300/0 conn(0x7fad9c159c90 0x7fad9c0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada436a700 1 --2- 172.21.15.7:0/3376754079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c15a060 0x7fad9c0b0430 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada436a700 1 -- 172.21.15.7:0/3376754079 >> 172.21.15.7:0/3376754079 conn(0x7fad9c071100 msgr2=0x7fad9c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada436a700 1 -- 172.21.15.7:0/3376754079 shutdown_connections 2024-01-11T07:12:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.413+0000 7fada436a700 1 -- 172.21.15.7:0/3376754079 wait complete. 2024-01-11T07:12:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.414+0000 7fada436a700 1 Processor -- start 2024-01-11T07:12:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.414+0000 7fada436a700 1 -- start start 2024-01-11T07:12:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada436a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c069320 0x7fad9c0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada436a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c159c90 0x7fad9c146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada436a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad9c15a060 0x7fad9c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada436a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0b0c30 con 0x7fad9c15a060 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada436a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0c8b00 con 0x7fad9c069320 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada436a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c146a40 con 0x7fad9c159c90 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada2907700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad9c15a060 0x7fad9c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada2907700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad9c15a060 0x7fad9c0cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53112/0 (socket says 172.21.15.7:53112) 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.415+0000 7fada2907700 1 -- 172.21.15.7:0/50614784 learned_addr learned my addr 172.21.15.7:0/50614784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada1905700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c159c90 0x7fad9c146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada2106700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c069320 0x7fad9c0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada2907700 1 -- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c159c90 msgr2=0x7fad9c146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada2907700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c159c90 0x7fad9c146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada2907700 1 -- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c069320 msgr2=0x7fad9c0b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada2907700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c069320 0x7fad9c0b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada2907700 1 -- 172.21.15.7:0/50614784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98009420 con 0x7fad9c15a060 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada2907700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad9c15a060 0x7fad9c0cbf40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fad9400cbf0 tx=0x7fad94001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fad8e7fc700 1 -- 172.21.15.7:0/50614784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9400a920 con 0x7fad9c15a060 2024-01-11T07:12:17.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fad8e7fc700 1 -- 172.21.15.7:0/50614784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9400aa80 con 0x7fad9c15a060 2024-01-11T07:12:17.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad90009420 con 0x7fad9c15a060 2024-01-11T07:12:17.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fad8e7fc700 1 -- 172.21.15.7:0/50614784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9400ae40 con 0x7fad9c15a060 2024-01-11T07:12:17.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.416+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c0c7ef0 con 0x7fad9c15a060 2024-01-11T07:12:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.417+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c04c3c0 con 0x7fad9c15a060 2024-01-11T07:12:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.418+0000 7fad8e7fc700 1 -- 172.21.15.7:0/50614784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fad94007300 con 0x7fad9c15a060 2024-01-11T07:12:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.418+0000 7fad8e7fc700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fad80066f30 0x7fad800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.418+0000 7fad8e7fc700 1 -- 172.21.15.7:0/50614784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fad94084970 con 0x7fad9c15a060 2024-01-11T07:12:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.418+0000 7fada2106700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fad80066f30 0x7fad800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.418+0000 7fad8e7fc700 1 -- 172.21.15.7:0/50614784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=104}) v3 -- 0x7fad8006ea50 con 0x7fad9c15a060 2024-01-11T07:12:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.419+0000 7fada2106700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fad80066f30 0x7fad800693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fad980093f0 tx=0x7fad98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:17.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.423+0000 7fad8e7fc700 1 -- 172.21.15.7:0/50614784 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad94051bc0 con 0x7fad9c15a060 2024-01-11T07:12:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.629+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fad9c15e800 con 0x7fad9c15a060 2024-01-11T07:12:17.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.630+0000 7fad8e7fc700 1 -- 172.21.15.7:0/50614784 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+32386 (secure 0 0 0) 0x7fad940551e0 con 0x7fad9c15a060 2024-01-11T07:12:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:17.630 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 103, 2024-01-11T07:12:17.630 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:12:16.246827+0000", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:12:17.631 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:12:17.632 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:12:17.633 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:17.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:17.635 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:17.636 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:55.626974+0000", 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:17.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:12:17.639 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:17.640 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:17.642 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:56.083170+0000", 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:17.644 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.645 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.647 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:12:17.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:17.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:17.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:17.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:17.654 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:17.655 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.659 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:17.660 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:17.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:12:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:12:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:12:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:12:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:17.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:17.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:17.693 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:12:17.694 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:17.695 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:17.696 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:17.697 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:12:17.698 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:12:17.699 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:12:17.700 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:12:17.700 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:12:17.700 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:12:17.700 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:12:17.700 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:12:17.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:17.700 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fad80066f30 msgr2=0x7fad800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fad80066f30 0x7fad800693e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fad980093f0 tx=0x7fad98008040 comp rx=0 tx=0).stop 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad9c15a060 msgr2=0x7fad9c0cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad9c15a060 0x7fad9c0cbf40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fad9400cbf0 tx=0x7fad94001980 comp rx=0 tx=0).stop 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 shutdown_connections 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fad80066f30 0x7fad800693e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad9c15a060 0x7fad9c0cbf40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad9c069320 0x7fad9c0b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.632+0000 7fada436a700 1 --2- 172.21.15.7:0/50614784 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fad9c159c90 0x7fad9c146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.633+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 >> 172.21.15.7:0/50614784 conn(0x7fad9c071100 msgr2=0x7fad9c15db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.633+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 shutdown_connections 2024-01-11T07:12:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:17.633+0000 7fada436a700 1 -- 172.21.15.7:0/50614784 wait complete. 2024-01-11T07:12:17.703 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:12:20.258 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:12:20.259+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-01-11T07:12:22.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:12:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.845+0000 7f5d3bc38700 1 Processor -- start 2024-01-11T07:12:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.846+0000 7f5d3bc38700 1 -- start start 2024-01-11T07:12:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.846+0000 7f5d3bc38700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d3406dd00 0x7f5d340c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.846+0000 7f5d3bc38700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3415b770 con 0x7f5d3406e0d0 2024-01-11T07:12:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.846+0000 7f5d3bc38700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3415a620 con 0x7f5d3415a1c0 2024-01-11T07:12:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.846+0000 7f5d3bc38700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3415abf0 con 0x7f5d3406dd00 2024-01-11T07:12:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d399d4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d3406dd00 0x7f5d340c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d391d3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5d3406e0d0 0x7f5d34159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47244/0 (socket says 172.21.15.7:47244) 2024-01-11T07:12:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d391d3700 1 -- 172.21.15.7:0/3648782548 learned_addr learned my addr 172.21.15.7:0/3648782548 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d399d4700 1 -- 172.21.15.7:0/3648782548 >> v1:172.21.15.113:6789/0 conn(0x7f5d3415a1c0 legacy=0x7f5d34069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d399d4700 1 -- 172.21.15.7:0/3648782548 >> v1:172.21.15.7:6789/0 conn(0x7f5d3406e0d0 legacy=0x7f5d34159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d399d4700 1 -- 172.21.15.7:0/3648782548 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3415aed0 con 0x7f5d3406dd00 2024-01-11T07:12:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d389d2700 1 -- 172.21.15.7:0/3648782548 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165759756 0 0) 0x7f5d3415b770 con 0x7f5d3406e0d0 2024-01-11T07:12:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d399d4700 1 --2- 172.21.15.7:0/3648782548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3406dd00 0x7f5d340c98d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5d300098e0 tx=0x7f5d300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15a3aa3b01a04fe server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d389d2700 1 -- 172.21.15.7:0/3648782548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30009e80 con 0x7f5d3406dd00 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d389d2700 1 -- 172.21.15.7:0/3648782548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3000b860 con 0x7f5d3406dd00 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.847+0000 7f5d389d2700 1 -- 172.21.15.7:0/3648782548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000bbe0 con 0x7f5d3406dd00 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.848+0000 7f5d3bc38700 1 -- 172.21.15.7:0/3648782548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3406dd00 msgr2=0x7f5d340c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.848+0000 7f5d3bc38700 1 --2- 172.21.15.7:0/3648782548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3406dd00 0x7f5d340c98d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5d300098e0 tx=0x7f5d300091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.848+0000 7f5d3bc38700 1 -- 172.21.15.7:0/3648782548 shutdown_connections 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.848+0000 7f5d3bc38700 1 --2- 172.21.15.7:0/3648782548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3406dd00 0x7f5d340c98d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.848+0000 7f5d3bc38700 1 -- 172.21.15.7:0/3648782548 >> 172.21.15.7:0/3648782548 conn(0x7f5d340680e0 msgr2=0x7f5d340cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.848+0000 7f5d3bc38700 1 -- 172.21.15.7:0/3648782548 shutdown_connections 2024-01-11T07:12:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.848+0000 7f5d3bc38700 1 -- 172.21.15.7:0/3648782548 wait complete. 2024-01-11T07:12:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.849+0000 7f5d3bc38700 1 Processor -- start 2024-01-11T07:12:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3bc38700 1 -- start start 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3bc38700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d3406dd00 0x7f5d340ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3bc38700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d3406e0d0 0x7f5d340af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3bc38700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3415a1c0 0x7f5d34158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3bc38700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340c63f0 con 0x7f5d3415a1c0 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3bc38700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340af700 con 0x7f5d3406dd00 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3bc38700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340ae5f0 con 0x7f5d3406e0d0 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3a1d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3415a1c0 0x7f5d34158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3a1d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3415a1c0 0x7f5d34158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53138/0 (socket says 172.21.15.7:53138) 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.850+0000 7f5d3a1d5700 1 -- 172.21.15.7:0/1147482817 learned_addr learned my addr 172.21.15.7:0/1147482817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d3a1d5700 1 -- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d3406e0d0 msgr2=0x7f5d340af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d391d3700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d3406e0d0 0x7f5d340af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d3a1d5700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d3406e0d0 0x7f5d340af1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d399d4700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d3406dd00 0x7f5d340ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d3a1d5700 1 -- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d3406dd00 msgr2=0x7f5d340ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d3a1d5700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d3406dd00 0x7f5d340ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d3a1d5700 1 -- 172.21.15.7:0/1147482817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d30009420 con 0x7f5d3415a1c0 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d399d4700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d3406dd00 0x7f5d340ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d3a1d5700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3415a1c0 0x7f5d34158eb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c840 tx=0x7f5d2c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d29ffb700 1 -- 172.21.15.7:0/1147482817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00a650 con 0x7f5d3415a1c0 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d29ffb700 1 -- 172.21.15.7:0/1147482817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c00a7d0 con 0x7f5d3415a1c0 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d340c60c0 con 0x7f5d3415a1c0 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d29ffb700 1 -- 172.21.15.7:0/1147482817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c00ab90 con 0x7f5d3415a1c0 2024-01-11T07:12:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.851+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d340ae030 con 0x7f5d3415a1c0 2024-01-11T07:12:22.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.852+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d340c6560 con 0x7f5d3415a1c0 2024-01-11T07:12:22.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.853+0000 7f5d29ffb700 1 -- 172.21.15.7:0/1147482817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5d2c003030 con 0x7f5d3415a1c0 2024-01-11T07:12:22.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.853+0000 7f5d29ffb700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d18066f30 0x7f5d180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:22.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.853+0000 7f5d29ffb700 1 -- 172.21.15.7:0/1147482817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d2c084590 con 0x7f5d3415a1c0 2024-01-11T07:12:22.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.853+0000 7f5d29ffb700 1 -- 172.21.15.7:0/1147482817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=105}) v3 -- 0x7f5d1806ea50 con 0x7f5d3415a1c0 2024-01-11T07:12:22.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.854+0000 7f5d399d4700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d18066f30 0x7f5d180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:22.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.854+0000 7f5d399d4700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d18066f30 0x7f5d180693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5d300098e0 tx=0x7f5d30017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:22.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:22.858+0000 7f5d29ffb700 1 -- 172.21.15.7:0/1147482817 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d340c6560 con 0x7f5d3415a1c0 2024-01-11T07:12:23.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.063+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5d340c6530 con 0x7f5d3415a1c0 2024-01-11T07:12:23.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.064+0000 7f5d29ffb700 1 -- 172.21.15.7:0/1147482817 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) v1 ==== 81+0+32394 (secure 0 0 0) 0x7f5d2c051800 con 0x7f5d3415a1c0 2024-01-11T07:12:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:23.064 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 104, 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:12:20.259735+0000", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:12:23.065 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.066 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:23.067 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:23.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:23.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.070 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:23.071 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:55.626974+0000", 2024-01-11T07:12:23.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:12:23.073 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:23.074 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:23.075 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:23.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:23.077 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:56.083170+0000", 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 3, 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full", 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:23.078 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:23.079 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "101", 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:23.080 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:23.083 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:23.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:23.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.083 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:23.083 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:23.083 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:23.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:23.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:23.084 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:23.085 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:23.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:23.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:23.090 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:23.091 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:23.097 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:23.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:23.099 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:23.100 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:12:23.101 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:23.102 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:23.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:23.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:23.108 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:23.113 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.114 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:23.117 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:12:23.118 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:23.119 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:23.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "full", 2024-01-11T07:12:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.126 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.129 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull", 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:12:23.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:23.131 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:12:23.132 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:23.133 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:23.134 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:23.135 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:12:23.136 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:12:23.137 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:23.138 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d18066f30 msgr2=0x7f5d180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d18066f30 0x7f5d180693e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5d300098e0 tx=0x7f5d30017000 comp rx=0 tx=0).stop 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3415a1c0 msgr2=0x7f5d34158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3415a1c0 0x7f5d34158eb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c840 tx=0x7f5d2c001670 comp rx=0 tx=0).stop 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 shutdown_connections 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5d18066f30 0x7f5d180693e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d3415a1c0 0x7f5d34158eb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d3406dd00 0x7f5d340ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:23.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 --2- 172.21.15.7:0/1147482817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d3406e0d0 0x7f5d340af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:23.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 >> 172.21.15.7:0/1147482817 conn(0x7f5d340680e0 msgr2=0x7f5d340cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:23.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 shutdown_connections 2024-01-11T07:12:23.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:23.066+0000 7f5d3bc38700 1 -- 172.21.15.7:0/1147482817 wait complete. 2024-01-11T07:12:23.141 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:12:25.316 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:12:25.317+0000 7f12ff4d6700 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-01-11T07:12:28.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:12:28.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.284+0000 7f920e682700 1 Processor -- start 2024-01-11T07:12:28.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.284+0000 7f920e682700 1 -- start start 2024-01-11T07:12:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f920e682700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9208063230 0x7f9208063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f920e682700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f920815a280 0x7f92080aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f920e682700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920815d880 con 0x7f92080ac5b0 2024-01-11T07:12:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f920e682700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920806c5b0 con 0x7f920815a280 2024-01-11T07:12:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f920e682700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9208063b40 con 0x7f9208063230 2024-01-11T07:12:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f92077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f920815a280 0x7f92080aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f92077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f920815a280 0x7f92080aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46478/0 (socket says 172.21.15.7:46478) 2024-01-11T07:12:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f92077fe700 1 -- 172.21.15.7:0/3077724955 learned_addr learned my addr 172.21.15.7:0/3077724955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f9207fff700 1 --2- 172.21.15.7:0/3077724955 >> v2:172.21.15.113:3300/0 conn(0x7f9208063230 0x7f9208063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f92077fe700 1 -- 172.21.15.7:0/3077724955 >> v2:172.21.15.113:3300/0 conn(0x7f9208063230 msgr2=0x7f9208063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f92077fe700 1 --2- 172.21.15.7:0/3077724955 >> v2:172.21.15.113:3300/0 conn(0x7f9208063230 0x7f9208063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f92077fe700 1 -- 172.21.15.7:0/3077724955 >> v1:172.21.15.113:6789/0 conn(0x7f92080ac5b0 legacy=0x7f92080ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f92077fe700 1 -- 172.21.15.7:0/3077724955 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920815a7f0 con 0x7f920815a280 2024-01-11T07:12:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.285+0000 7f92077fe700 1 --2- 172.21.15.7:0/3077724955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080aa9f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f92080ad1f0 tx=0x7f91f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8324dd5645ed4eec server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f9206ffd700 1 -- 172.21.15.7:0/3077724955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f000b820 con 0x7f920815a280 2024-01-11T07:12:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f9206ffd700 1 -- 172.21.15.7:0/3077724955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f000b980 con 0x7f920815a280 2024-01-11T07:12:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f9206ffd700 1 -- 172.21.15.7:0/3077724955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f000bd20 con 0x7f920815a280 2024-01-11T07:12:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f920e682700 1 -- 172.21.15.7:0/3077724955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 msgr2=0x7f92080aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f920e682700 1 --2- 172.21.15.7:0/3077724955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080aa9f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f92080ad1f0 tx=0x7f91f0009250 comp rx=0 tx=0).stop 2024-01-11T07:12:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f920e682700 1 -- 172.21.15.7:0/3077724955 shutdown_connections 2024-01-11T07:12:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f920e682700 1 --2- 172.21.15.7:0/3077724955 >> v2:172.21.15.113:3300/0 conn(0x7f9208063230 0x7f9208063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f920e682700 1 --2- 172.21.15.7:0/3077724955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080aa9f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f920e682700 1 -- 172.21.15.7:0/3077724955 >> 172.21.15.7:0/3077724955 conn(0x7f92080680e0 msgr2=0x7f92080cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f920e682700 1 -- 172.21.15.7:0/3077724955 shutdown_connections 2024-01-11T07:12:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.286+0000 7f920e682700 1 -- 172.21.15.7:0/3077724955 wait complete. 2024-01-11T07:12:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.287+0000 7f920e682700 1 Processor -- start 2024-01-11T07:12:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.288+0000 7f920e682700 1 -- start start 2024-01-11T07:12:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.288+0000 7f920e682700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9208063230 0x7f92080afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.288+0000 7f920e682700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080ac5b0 0x7f9208153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.288+0000 7f920e682700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.288+0000 7f920e682700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080c3530 con 0x7f920815a280 2024-01-11T07:12:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.288+0000 7f920e682700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9208154310 con 0x7f92080ac5b0 2024-01-11T07:12:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.288+0000 7f920e682700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92080c2680 con 0x7f9208063230 2024-01-11T07:12:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46492/0 (socket says 172.21.15.7:46492) 2024-01-11T07:12:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 -- 172.21.15.7:0/711035081 learned_addr learned my addr 172.21.15.7:0/711035081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f92077fe700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080ac5b0 0x7f9208153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f9207fff700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9208063230 0x7f92080afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 -- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9208063230 msgr2=0x7f92080afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9208063230 0x7f92080afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 -- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080ac5b0 msgr2=0x7f9208153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080ac5b0 0x7f9208153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 -- 172.21.15.7:0/711035081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00094f0 con 0x7f920815a280 2024-01-11T07:12:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f9207fff700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9208063230 0x7f92080afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920cc1f700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080c2140 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f920000c930 tx=0x7f92000026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200012630 con 0x7f920815a280 2024-01-11T07:12:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92000127d0 con 0x7f920815a280 2024-01-11T07:12:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92080c2920 con 0x7f920815a280 2024-01-11T07:12:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200012b90 con 0x7f920815a280 2024-01-11T07:12:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.289+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92080c2ee0 con 0x7f920815a280 2024-01-11T07:12:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.290+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920804c3c0 con 0x7f920815a280 2024-01-11T07:12:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.291+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f920000a3b0 con 0x7f920815a280 2024-01-11T07:12:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.291+0000 7f91fffff700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91e8066f30 0x7f91e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.291+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9200084240 con 0x7f920815a280 2024-01-11T07:12:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.291+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=108}) v3 -- 0x7f91e806ea50 con 0x7f920815a280 2024-01-11T07:12:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.292+0000 7f9207fff700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91e8066f30 0x7f91e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.292+0000 7f9207fff700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91e8066f30 0x7f91e80693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006fd0 tx=0x7f91f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.296+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92000514b0 con 0x7f920815a280 2024-01-11T07:12:28.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.336+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(108..108 src has 1..108) v4 ==== 1460+0+0 (secure 0 0 0) 0x7f9200019310 con 0x7f920815a280 2024-01-11T07:12:28.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.336+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=109}) v3 -- 0x7f91e806fc40 con 0x7f920815a280 2024-01-11T07:12:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.507+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f920815aad0 con 0x7f920815a280 2024-01-11T07:12:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.508+0000 7f91fffff700 1 -- 172.21.15.7:0/711035081 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) v1 ==== 81+0+32219 (secure 0 0 0) 0x7f9200054ad0 con 0x7f920815a280 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 108, 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:12:28.332197+0000", 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:12:28.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:12:28.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:28.510 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:28.511 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:28.512 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:28.514 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:55.626974+0000", 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:28.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:12:28.516 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:28.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:28.518 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:28.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.520 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:56.083170+0000", 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:28.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:28.522 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:28.523 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:28.524 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:28.525 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.527 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:12:28.531 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:12:28.532 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:12:28.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:12:28.537 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:12:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.540 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.541 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:12:28.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.543 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.548 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:28.552 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.558 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.563 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.567 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:12:28.568 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.569 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:28.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.572 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:28.574 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:28.575 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:28.577 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:28.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:28.579 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:12:28.580 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:12:28.581 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:12:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:12:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:12:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:12:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:12:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:12:28.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.582 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91e8066f30 msgr2=0x7f91e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91e8066f30 0x7f91e80693e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006fd0 tx=0x7f91f8008040 comp rx=0 tx=0).stop 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 msgr2=0x7f92080c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080c2140 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f920000c930 tx=0x7f92000026e0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 shutdown_connections 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f91e8066f30 0x7f91e80693e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f920815a280 0x7f92080c2140 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9208063230 0x7f92080afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 --2- 172.21.15.7:0/711035081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92080ac5b0 0x7f9208153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.510+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 >> 172.21.15.7:0/711035081 conn(0x7f92080680e0 msgr2=0x7f92080cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:28.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.511+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 shutdown_connections 2024-01-11T07:12:28.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.511+0000 7f920e682700 1 -- 172.21.15.7:0/711035081 wait complete. 2024-01-11T07:12:28.585 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-01-11T07:12:28.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:12:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.723+0000 7fd46c955700 1 Processor -- start 2024-01-11T07:12:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.723+0000 7fd46c955700 1 -- start start 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46c955700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd468063230 0x7fd468063680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46c955700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 0x7fd4680c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46c955700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd468063bc0 con 0x7fd4680aff20 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46c955700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4680b02f0 con 0x7fd468063230 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46c955700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd468154860 con 0x7fd4680c8020 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd4680aff20 0x7fd4680aab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57496/0 (socket says 172.21.15.7:57496) 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46659c700 1 -- 172.21.15.7:0/818556030 learned_addr learned my addr 172.21.15.7:0/818556030 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd466d9d700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 0x7fd4680c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd465d9b700 1 --2- 172.21.15.7:0/818556030 >> v2:172.21.15.113:3300/0 conn(0x7fd468063230 0x7fd468063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610761373 0 0) 0x7fd468063bc0 con 0x7fd4680aff20 2024-01-11T07:12:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd450003570 con 0x7fd4680aff20 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4214628730 0 0) 0x7fd450003570 con 0x7fd4680aff20 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 msgr2=0x7fd4680c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46559a700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 0x7fd4680c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.724+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 >> v2:172.21.15.113:3300/0 conn(0x7fd468063230 msgr2=0x7fd468063680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 --2- 172.21.15.7:0/818556030 >> v2:172.21.15.113:3300/0 conn(0x7fd468063230 0x7fd468063680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46815d910 con 0x7fd4680aff20 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fd458001470 con 0x7fd4680aff20 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd466d9d700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 0x7fd4680c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 >> v1:172.21.15.7:6789/0 conn(0x7fd4680aff20 legacy=0x7fd4680aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 0x7fd450003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd450003fa0 0x7fd450004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd450004d50 0x7fd4500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd468154860 con 0x7fd450003fa0 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45000ba60 con 0x7fd450004d50 2024-01-11T07:12:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45000bba0 con 0x7fd4680c8020 2024-01-11T07:12:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd465d9b700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd450003fa0 0x7fd450004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46659c700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 0x7fd450003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd466d9d700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd450004d50 0x7fd4500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd465d9b700 1 -- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 msgr2=0x7fd450003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd465d9b700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd4680c8020 0x7fd450003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd465d9b700 1 -- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd450004d50 msgr2=0x7fd4500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd465d9b700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd450004d50 0x7fd4500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd465d9b700 1 -- 172.21.15.7:0/818556030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460009a20 con 0x7fd450003fa0 2024-01-11T07:12:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd465d9b700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd450003fa0 0x7fd450004810 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009c00 tx=0x7fd45c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6de1d5de54eba server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00e710 con 0x7fd450003fa0 2024-01-11T07:12:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c00e870 con 0x7fd450003fa0 2024-01-11T07:12:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.725+0000 7fd46559a700 1 -- 172.21.15.7:0/818556030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00ec30 con 0x7fd450003fa0 2024-01-11T07:12:28.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.726+0000 7fd46c955700 1 -- 172.21.15.7:0/818556030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd450003fa0 msgr2=0x7fd450004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.726+0000 7fd46c955700 1 --2- 172.21.15.7:0/818556030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd450003fa0 0x7fd450004810 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009c00 tx=0x7fd45c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.726+0000 7fd466d9d700 1 -- 172.21.15.7:0/818556030 reap_dead start 2024-01-11T07:12:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.726+0000 7fd46c955700 1 -- 172.21.15.7:0/818556030 shutdown_connections 2024-01-11T07:12:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.726+0000 7fd46c955700 1 -- 172.21.15.7:0/818556030 >> 172.21.15.7:0/818556030 conn(0x7fd468068090 msgr2=0x7fd4680cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.726+0000 7fd46c955700 1 -- 172.21.15.7:0/818556030 shutdown_connections 2024-01-11T07:12:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.726+0000 7fd46c955700 1 -- 172.21.15.7:0/818556030 wait complete. 2024-01-11T07:12:28.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.728+0000 7fd46c955700 1 Processor -- start 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.728+0000 7fd46c955700 1 -- start start 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd46c955700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd450003fa0 0x7fd4680ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd46c955700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4680b0df0 0x7fd468153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd46c955700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd46806b680 0x7fd46806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd46c955700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680aee10 con 0x7fd46806b680 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd46c955700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46806c010 con 0x7fd4680b0df0 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd46c955700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680ae200 con 0x7fd450003fa0 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd466d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd46806b680 0x7fd46806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd466d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd46806b680 0x7fd46806bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46524/0 (socket says 172.21.15.7:46524) 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd466d9d700 1 -- 172.21.15.7:0/58480195 learned_addr learned my addr 172.21.15.7:0/58480195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd46659c700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd450003fa0 0x7fd4680ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd465d9b700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4680b0df0 0x7fd468153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd466d9d700 1 -- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd450003fa0 msgr2=0x7fd4680ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd466d9d700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd450003fa0 0x7fd4680ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd466d9d700 1 -- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4680b0df0 msgr2=0x7fd468153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd466d9d700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4680b0df0 0x7fd468153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.729+0000 7fd466d9d700 1 -- 172.21.15.7:0/58480195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680ae4e0 con 0x7fd46806b680 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.730+0000 7fd466d9d700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd46806b680 0x7fd46806bad0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd4600094d0 tx=0x7fd46000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.730+0000 7fd4567fc700 1 -- 172.21.15.7:0/58480195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46000e040 con 0x7fd46806b680 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.730+0000 7fd4567fc700 1 -- 172.21.15.7:0/58480195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46000f040 con 0x7fd46806b680 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.730+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4680ae7c0 con 0x7fd46806b680 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.730+0000 7fd4567fc700 1 -- 172.21.15.7:0/58480195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46000fed0 con 0x7fd46806b680 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.730+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4680adef0 con 0x7fd46806b680 2024-01-11T07:12:28.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.731+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46804c3c0 con 0x7fd46806b680 2024-01-11T07:12:28.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.732+0000 7fd4567fc700 1 -- 172.21.15.7:0/58480195 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd460015640 con 0x7fd46806b680 2024-01-11T07:12:28.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.732+0000 7fd4567fc700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd448066f30 0x7fd4480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:28.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.732+0000 7fd4567fc700 1 -- 172.21.15.7:0/58480195 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd460084d60 con 0x7fd46806b680 2024-01-11T07:12:28.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.732+0000 7fd46659c700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd448066f30 0x7fd4480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:28.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.733+0000 7fd46659c700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd448066f30 0x7fd4480693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd458000e50 tx=0x7fd458001e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:28.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.737+0000 7fd4567fc700 1 -- 172.21.15.7:0/58480195 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd460055a60 con 0x7fd46806b680 2024-01-11T07:12:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.942+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd46815dbf0 con 0x7fd46806b680 2024-01-11T07:12:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.943+0000 7fd4567fc700 1 -- 172.21.15.7:0/58480195 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) v1 ==== 81+0+32219 (secure 0 0 0) 0x7fd4600555f0 con 0x7fd46806b680 2024-01-11T07:12:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:28.943 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 108, 2024-01-11T07:12:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:12:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:12:28.332197+0000", 2024-01-11T07:12:28.943 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:12:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 13, 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:28.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:28.946 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:28.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:28.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:12:28.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 12, 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:55.626974+0000", 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:28.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:28.952 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:28.953 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:28.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:28.955 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 13, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:11:56.083170+0000", 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:12:28.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:12:28.959 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.960 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:12:28.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.962 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.968 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.969 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:12:28.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:12:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:28.973 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:12:28.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.978 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.979 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.983 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.985 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:12:28.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:28.990 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:12:28.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:28.995 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:28.996 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:28.999 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.000 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:12:29.001 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 98, 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:29.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:12:29.003 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:12:29.004 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:29.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:12:29.006 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:29.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:29.008 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:12:29.009 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:29.010 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:12:29.011 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:12:29.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:12:29.013 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:12:29.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:29.015 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.945+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd448066f30 msgr2=0x7fd4480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.945+0000 7fd46c955700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd448066f30 0x7fd4480693e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd458000e50 tx=0x7fd458001e90 comp rx=0 tx=0).stop 2024-01-11T07:12:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd46806b680 msgr2=0x7fd46806bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd46806b680 0x7fd46806bad0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd4600094d0 tx=0x7fd46000b040 comp rx=0 tx=0).stop 2024-01-11T07:12:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 shutdown_connections 2024-01-11T07:12:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd448066f30 0x7fd4480693e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd46806b680 0x7fd46806bad0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd450003fa0 0x7fd4680ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 --2- 172.21.15.7:0/58480195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4680b0df0 0x7fd468153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 >> 172.21.15.7:0/58480195 conn(0x7fd468068090 msgr2=0x7fd4680cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 shutdown_connections 2024-01-11T07:12:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:28.946+0000 7fd46c955700 1 -- 172.21.15.7:0/58480195 wait complete. 2024-01-11T07:12:29.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.152+0000 7fc6524fd700 1 Processor -- start 2024-01-11T07:12:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.152+0000 7fc6524fd700 1 -- start start 2024-01-11T07:12:29.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.153+0000 7fc6524fd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc64c06dd00 0x7fc64c0c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:29.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.153+0000 7fc6524fd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc64c06e0d0 0x7fc64c159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:29.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.153+0000 7fc6524fd700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64c15b770 con 0x7fc64c159ff0 2024-01-11T07:12:29.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.153+0000 7fc6524fd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c15abf0 con 0x7fc64c06e0d0 2024-01-11T07:12:29.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.153+0000 7fc6524fd700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c15aed0 con 0x7fc64c06dd00 2024-01-11T07:12:29.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.153+0000 7fc64b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc64c06e0d0 0x7fc64c159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:29.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc64c06e0d0 0x7fc64c159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46530/0 (socket says 172.21.15.7:46530) 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64b7fe700 1 -- 172.21.15.7:0/1082239911 learned_addr learned my addr 172.21.15.7:0/1082239911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64bfff700 1 --2- 172.21.15.7:0/1082239911 >> v2:172.21.15.113:3300/0 conn(0x7fc64c06dd00 0x7fc64c0c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64b7fe700 1 -- 172.21.15.7:0/1082239911 >> v2:172.21.15.113:3300/0 conn(0x7fc64c06dd00 msgr2=0x7fc64c0c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64b7fe700 1 --2- 172.21.15.7:0/1082239911 >> v2:172.21.15.113:3300/0 conn(0x7fc64c06dd00 0x7fc64c0c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64b7fe700 1 -- 172.21.15.7:0/1082239911 >> v1:172.21.15.113:6789/0 conn(0x7fc64c159ff0 legacy=0x7fc64c0c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64b7fe700 1 -- 172.21.15.7:0/1082239911 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c15b1b0 con 0x7fc64c06e0d0 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64bfff700 1 --2- 172.21.15.7:0/1082239911 >> v2:172.21.15.113:3300/0 conn(0x7fc64c06dd00 0x7fc64c0c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64b7fe700 1 --2- 172.21.15.7:0/1082239911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c06e0d0 0x7fc64c159ab0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc64c06d7f0 tx=0x7fc64000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ff5d0054692e58 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64affd700 1 -- 172.21.15.7:0/1082239911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000dc50 con 0x7fc64c06e0d0 2024-01-11T07:12:29.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64affd700 1 -- 172.21.15.7:0/1082239911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64000ddb0 con 0x7fc64c06e0d0 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.154+0000 7fc64affd700 1 -- 172.21.15.7:0/1082239911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400196d0 con 0x7fc64c06e0d0 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.155+0000 7fc6524fd700 1 -- 172.21.15.7:0/1082239911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c06e0d0 msgr2=0x7fc64c159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.155+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1082239911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c06e0d0 0x7fc64c159ab0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc64c06d7f0 tx=0x7fc64000b4f0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.155+0000 7fc6524fd700 1 -- 172.21.15.7:0/1082239911 shutdown_connections 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.155+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1082239911 >> v2:172.21.15.113:3300/0 conn(0x7fc64c06dd00 0x7fc64c0c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.155+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1082239911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c06e0d0 0x7fc64c159ab0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.155+0000 7fc6524fd700 1 -- 172.21.15.7:0/1082239911 >> 172.21.15.7:0/1082239911 conn(0x7fc64c0680e0 msgr2=0x7fc64c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.155+0000 7fc6524fd700 1 -- 172.21.15.7:0/1082239911 shutdown_connections 2024-01-11T07:12:29.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.155+0000 7fc6524fd700 1 -- 172.21.15.7:0/1082239911 wait complete. 2024-01-11T07:12:29.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.156+0000 7fc6524fd700 1 Processor -- start 2024-01-11T07:12:29.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.157+0000 7fc6524fd700 1 -- start start 2024-01-11T07:12:29.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc6524fd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc64c06dd00 0x7fc64c14acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:29.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc6524fd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc64c06e0d0 0x7fc64c0c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:29.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc6524fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c159ff0 0x7fc64c0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:29.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc6524fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0c57f0 con 0x7fc64c159ff0 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc6524fd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0c5ad0 con 0x7fc64c06e0d0 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc6524fd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0c5db0 con 0x7fc64c06dd00 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c159ff0 0x7fc64c0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c159ff0 0x7fc64c0c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46534/0 (socket says 172.21.15.7:46534) 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 -- 172.21.15.7:0/1810395694 learned_addr learned my addr 172.21.15.7:0/1810395694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc64b7fe700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc64c06e0d0 0x7fc64c0c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc64bfff700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc64c06dd00 0x7fc64c14acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 -- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc64c06dd00 msgr2=0x7fc64c14acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc64c06dd00 0x7fc64c14acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 -- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc64c06e0d0 msgr2=0x7fc64c0c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc64c06e0d0 0x7fc64c0c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 -- 172.21.15.7:0/1810395694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64000b790 con 0x7fc64c159ff0 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.158+0000 7fc650a9a700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c159ff0 0x7fc64c0c6bb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc64400c840 tx=0x7fc644001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:29.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.159+0000 7fc633fff700 1 -- 172.21.15.7:0/1810395694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644012630 con 0x7fc64c159ff0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.159+0000 7fc633fff700 1 -- 172.21.15.7:0/1810395694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440127b0 con 0x7fc64c159ff0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.159+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c0b06c0 con 0x7fc64c159ff0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.159+0000 7fc633fff700 1 -- 172.21.15.7:0/1810395694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644012b70 con 0x7fc64c159ff0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.159+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c14b210 con 0x7fc64c159ff0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.160+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c04c3c0 con 0x7fc64c159ff0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.160+0000 7fc633fff700 1 -- 172.21.15.7:0/1810395694 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc64401a3a0 con 0x7fc64c159ff0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.161+0000 7fc633fff700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc638066f30 0x7fc6380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.161+0000 7fc633fff700 1 -- 172.21.15.7:0/1810395694 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc644084690 con 0x7fc64c159ff0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.161+0000 7fc64bfff700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc638066f30 0x7fc6380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:29.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.162+0000 7fc64bfff700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc638066f30 0x7fc6380693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006f00 tx=0x7fc63c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:29.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.165+0000 7fc633fff700 1 -- 172.21.15.7:0/1810395694 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc644051900 con 0x7fc64c159ff0 2024-01-11T07:12:29.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.420+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc64c0c5930 con 0x7fc64c159ff0 2024-01-11T07:12:29.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.420+0000 7fc633fff700 1 -- 172.21.15.7:0/1810395694 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fc64c0c5930 con 0x7fc64c159ff0 2024-01-11T07:12:29.420 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:29.420 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:29.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc638066f30 msgr2=0x7fc6380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:29.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc638066f30 0x7fc6380693e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006f00 tx=0x7fc63c008040 comp rx=0 tx=0).stop 2024-01-11T07:12:29.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c159ff0 msgr2=0x7fc64c0c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c159ff0 0x7fc64c0c6bb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc64400c840 tx=0x7fc644001670 comp rx=0 tx=0).stop 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 shutdown_connections 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc638066f30 0x7fc6380693e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc64c159ff0 0x7fc64c0c6bb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc64c06dd00 0x7fc64c14acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 --2- 172.21.15.7:0/1810395694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc64c06e0d0 0x7fc64c0c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 >> 172.21.15.7:0/1810395694 conn(0x7fc64c0680e0 msgr2=0x7fc64c06ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 shutdown_connections 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:29.423+0000 7fc6524fd700 1 -- 172.21.15.7:0/1810395694 wait complete. 2024-01-11T07:12:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:29.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-11T07:12:29.587 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":6175,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.895323561000001,"mdsmap_epoch":50,"osdmap_epoch":108,"osdmap_epoch_barrier":102,"uptime":36.719371604000003} 2024-01-11T07:12:29.588 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 6175, "mdsmap_epoch": 50, "osdmap_epoch": 108, "osdmap_epoch_barrier": 102, "rank_uptime": 31.895323561, "state": "up:active", "uptime": 36.719371604, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:12:29.588 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:12:29.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:29.588 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-01-11T07:12:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-11T07:12:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-11T07:12:29.914 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.304295 s, 172 MB/s 2024-01-11T07:12:29.915 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:12:29.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:12:29.966 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:12:30.009 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:12:30.009 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:12:30.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:30.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:30.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:12:30.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:30.095 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:12:30.114 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:12:30.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:30.114 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:30.162 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:12:30.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:30.162 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:12:30.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:30.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.319+0000 7f0e3f052700 1 Processor -- start 2024-01-11T07:12:30.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.319+0000 7f0e3f052700 1 -- start start 2024-01-11T07:12:30.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e3f052700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e380cd6d0 0x7f0e380cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e3f052700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3815acf0 con 0x7f0e3815d8a0 2024-01-11T07:12:30.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e3f052700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3815afd0 con 0x7f0e3815e450 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e3f052700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e380ce980 con 0x7f0e380cd6d0 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e3cdee700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e380cd6d0 0x7f0e380cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e37fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0e3815d8a0 0x7f0e3815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57498/0 (socket says 172.21.15.7:57498) 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e37fff700 1 -- 172.21.15.7:0/2087796978 learned_addr learned my addr 172.21.15.7:0/2087796978 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e3cdee700 1 -- 172.21.15.7:0/2087796978 >> v1:172.21.15.113:6789/0 conn(0x7f0e3815e450 legacy=0x7f0e38159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e3cdee700 1 -- 172.21.15.7:0/2087796978 >> v1:172.21.15.7:6789/0 conn(0x7f0e3815d8a0 legacy=0x7f0e3815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e3cdee700 1 -- 172.21.15.7:0/2087796978 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380cec60 con 0x7f0e380cd6d0 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.320+0000 7f0e377fe700 1 -- 172.21.15.7:0/2087796978 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662886100 0 0) 0x7f0e3815acf0 con 0x7f0e3815d8a0 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e3cdee700 1 --2- 172.21.15.7:0/2087796978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e380cd6d0 0x7f0e380cdaa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0098e0 tx=0x7f0e2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba083c5dfd736e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e377fe700 1 -- 172.21.15.7:0/2087796978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c009e80 con 0x7f0e380cd6d0 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e377fe700 1 -- 172.21.15.7:0/2087796978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c00b860 con 0x7f0e380cd6d0 2024-01-11T07:12:30.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e377fe700 1 -- 172.21.15.7:0/2087796978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00bbe0 con 0x7f0e380cd6d0 2024-01-11T07:12:30.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e3f052700 1 -- 172.21.15.7:0/2087796978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e380cd6d0 msgr2=0x7f0e380cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e3f052700 1 --2- 172.21.15.7:0/2087796978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e380cd6d0 0x7f0e380cdaa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0098e0 tx=0x7f0e2c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e3f052700 1 -- 172.21.15.7:0/2087796978 shutdown_connections 2024-01-11T07:12:30.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e3f052700 1 --2- 172.21.15.7:0/2087796978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e380cd6d0 0x7f0e380cdaa0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.321+0000 7f0e3f052700 1 -- 172.21.15.7:0/2087796978 >> 172.21.15.7:0/2087796978 conn(0x7f0e38071120 msgr2=0x7f0e3806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:30.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.322+0000 7f0e3f052700 1 -- 172.21.15.7:0/2087796978 shutdown_connections 2024-01-11T07:12:30.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.322+0000 7f0e3f052700 1 -- 172.21.15.7:0/2087796978 wait complete. 2024-01-11T07:12:30.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.323+0000 7f0e3f052700 1 Processor -- start 2024-01-11T07:12:30.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.323+0000 7f0e3f052700 1 -- start start 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3f052700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e380cd6d0 0x7f0e38155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3f052700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e3815d8a0 0x7f0e380b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3f052700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3815e450 0x7f0e3815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3f052700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380b1040 con 0x7f0e3815e450 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3f052700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380b2d20 con 0x7f0e380cd6d0 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3f052700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380b2fc0 con 0x7f0e3815d8a0 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3815e450 0x7f0e3815c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3815e450 0x7f0e3815c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46548/0 (socket says 172.21.15.7:46548) 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 -- 172.21.15.7:0/3711487521 learned_addr learned my addr 172.21.15.7:0/3711487521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3cdee700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e380cd6d0 0x7f0e38155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e37fff700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e3815d8a0 0x7f0e380b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 -- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e3815d8a0 msgr2=0x7f0e380b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e3815d8a0 0x7f0e380b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 -- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e380cd6d0 msgr2=0x7f0e38155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e380cd6d0 0x7f0e38155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 -- 172.21.15.7:0/3711487521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c009420 con 0x7f0e3815e450 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3cdee700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e380cd6d0 0x7f0e38155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.324+0000 7f0e3d5ef700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3815e450 0x7f0e3815c7e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c840 tx=0x7f0e30001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.325+0000 7f0e34ff9700 1 -- 172.21.15.7:0/3711487521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30012630 con 0x7f0e3815e450 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.325+0000 7f0e34ff9700 1 -- 172.21.15.7:0/3711487521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e300127b0 con 0x7f0e3815e450 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.325+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e380b32a0 con 0x7f0e3815e450 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.325+0000 7f0e34ff9700 1 -- 172.21.15.7:0/3711487521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30012b70 con 0x7f0e3815e450 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.325+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e380b0cd0 con 0x7f0e3815e450 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.326+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e380b11f0 con 0x7f0e3815e450 2024-01-11T07:12:30.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.326+0000 7f0e34ff9700 1 -- 172.21.15.7:0/3711487521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0e30003030 con 0x7f0e3815e450 2024-01-11T07:12:30.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.327+0000 7f0e34ff9700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e1c066f30 0x7f0e1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.327+0000 7f0e34ff9700 1 -- 172.21.15.7:0/3711487521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e30084540 con 0x7f0e3815e450 2024-01-11T07:12:30.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.327+0000 7f0e3cdee700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e1c066f30 0x7f0e1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.328+0000 7f0e3cdee700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e1c066f30 0x7f0e1c0693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0093f0 tx=0x7f0e2c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:30.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.331+0000 7f0e34ff9700 1 -- 172.21.15.7:0/3711487521 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e380b11f0 con 0x7f0e3815e450 2024-01-11T07:12:30.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.584+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e3804c3c0 con 0x7f0e3815e450 2024-01-11T07:12:30.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.585+0000 7f0e34ff9700 1 -- 172.21.15.7:0/3711487521 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f0e30051710 con 0x7f0e3815e450 2024-01-11T07:12:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:30.585 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e1c066f30 msgr2=0x7f0e1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e1c066f30 0x7f0e1c0693e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0093f0 tx=0x7f0e2c017000 comp rx=0 tx=0).stop 2024-01-11T07:12:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3815e450 msgr2=0x7f0e3815c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3815e450 0x7f0e3815c7e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c840 tx=0x7f0e30001670 comp rx=0 tx=0).stop 2024-01-11T07:12:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 shutdown_connections 2024-01-11T07:12:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e1c066f30 0x7f0e1c0693e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3815e450 0x7f0e3815c7e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e380cd6d0 0x7f0e38155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 --2- 172.21.15.7:0/3711487521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e3815d8a0 0x7f0e380b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 >> 172.21.15.7:0/3711487521 conn(0x7f0e38071120 msgr2=0x7f0e380c7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:30.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 shutdown_connections 2024-01-11T07:12:30.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.587+0000 7f0e3f052700 1 -- 172.21.15.7:0/3711487521 wait complete. 2024-01-11T07:12:30.588 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:30.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:30.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.733+0000 7f6530054700 1 Processor -- start 2024-01-11T07:12:30.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.734+0000 7f6530054700 1 -- start start 2024-01-11T07:12:30.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.734+0000 7f6530054700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f652806e0d0 0x7f6528159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f6530054700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6528159ff0 0x7f6528069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f6530054700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652815b770 con 0x7f652806dd00 2024-01-11T07:12:30.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f6530054700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652815abf0 con 0x7f6528159ff0 2024-01-11T07:12:30.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f6530054700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652815aed0 con 0x7f652806e0d0 2024-01-11T07:12:30.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652e5f1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6528159ff0 0x7f6528069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652e5f1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6528159ff0 0x7f6528069fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46562/0 (socket says 172.21.15.7:46562) 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652e5f1700 1 -- 172.21.15.7:0/1419622417 learned_addr learned my addr 172.21.15.7:0/1419622417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652d5ef700 1 --2- 172.21.15.7:0/1419622417 >> v2:172.21.15.113:3300/0 conn(0x7f652806e0d0 0x7f6528159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652e5f1700 1 -- 172.21.15.7:0/1419622417 >> v2:172.21.15.113:3300/0 conn(0x7f652806e0d0 msgr2=0x7f6528159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652e5f1700 1 --2- 172.21.15.7:0/1419622417 >> v2:172.21.15.113:3300/0 conn(0x7f652806e0d0 0x7f6528159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652e5f1700 1 -- 172.21.15.7:0/1419622417 >> v1:172.21.15.113:6789/0 conn(0x7f652806dd00 legacy=0x7f65280c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652e5f1700 1 -- 172.21.15.7:0/1419622417 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652815b1b0 con 0x7f6528159ff0 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652e5f1700 1 --2- 172.21.15.7:0/1419622417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f6528069fb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f652806d2e0 tx=0x7f6520009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3d9196c14f79f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.735+0000 7f652cdee700 1 -- 172.21.15.7:0/1419622417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000e070 con 0x7f6528159ff0 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f652cdee700 1 -- 172.21.15.7:0/1419622417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652000ba20 con 0x7f6528159ff0 2024-01-11T07:12:30.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f652cdee700 1 -- 172.21.15.7:0/1419622417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200092e0 con 0x7f6528159ff0 2024-01-11T07:12:30.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f6530054700 1 -- 172.21.15.7:0/1419622417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 msgr2=0x7f6528069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f6530054700 1 --2- 172.21.15.7:0/1419622417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f6528069fb0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f652806d2e0 tx=0x7f6520009840 comp rx=0 tx=0).stop 2024-01-11T07:12:30.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f6530054700 1 -- 172.21.15.7:0/1419622417 shutdown_connections 2024-01-11T07:12:30.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f6530054700 1 --2- 172.21.15.7:0/1419622417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f6528069fb0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f6530054700 1 --2- 172.21.15.7:0/1419622417 >> v2:172.21.15.113:3300/0 conn(0x7f652806e0d0 0x7f6528159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f6530054700 1 -- 172.21.15.7:0/1419622417 >> 172.21.15.7:0/1419622417 conn(0x7f65280680e0 msgr2=0x7f65280cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f6530054700 1 -- 172.21.15.7:0/1419622417 shutdown_connections 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.736+0000 7f6530054700 1 -- 172.21.15.7:0/1419622417 wait complete. 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.737+0000 7f6530054700 1 Processor -- start 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f6530054700 1 -- start start 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f6530054700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f652806dd00 0x7f6528158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f6530054700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652806e0d0 0x7f65280aa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f6530054700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f65280a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f6530054700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65281593d0 con 0x7f6528159ff0 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f6530054700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65280c5a50 con 0x7f652806e0d0 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f6530054700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65280c5d30 con 0x7f652806dd00 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f652e5f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f65280a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f652e5f1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f65280a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46568/0 (socket says 172.21.15.7:46568) 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.738+0000 7f652e5f1700 1 -- 172.21.15.7:0/2703563561 learned_addr learned my addr 172.21.15.7:0/2703563561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:30.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652d5ef700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652806e0d0 0x7f65280aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652ddf0700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f652806dd00 0x7f6528158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652e5f1700 1 -- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f652806dd00 msgr2=0x7f6528158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652e5f1700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f652806dd00 0x7f6528158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652e5f1700 1 -- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652806e0d0 msgr2=0x7f65280aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652e5f1700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652806e0d0 0x7f65280aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652e5f1700 1 -- 172.21.15.7:0/2703563561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6520009ae0 con 0x7f6528159ff0 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652ddf0700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f652806dd00 0x7f6528158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f652e5f1700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f65280a6e70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f65200072f0 tx=0x7f6520018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f651dffb700 1 -- 172.21.15.7:0/2703563561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520008bb0 con 0x7f6528159ff0 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f651dffb700 1 -- 172.21.15.7:0/2703563561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520008d10 con 0x7f6528159ff0 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65280c6010 con 0x7f6528159ff0 2024-01-11T07:12:30.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f651dffb700 1 -- 172.21.15.7:0/2703563561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65200256b0 con 0x7f6528159ff0 2024-01-11T07:12:30.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.739+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65280c86e0 con 0x7f6528159ff0 2024-01-11T07:12:30.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.741+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65280c64b0 con 0x7f6528159ff0 2024-01-11T07:12:30.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.741+0000 7f651dffb700 1 -- 172.21.15.7:0/2703563561 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6520003040 con 0x7f6528159ff0 2024-01-11T07:12:30.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.741+0000 7f651dffb700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f650c066f30 0x7f650c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:30.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.741+0000 7f651dffb700 1 -- 172.21.15.7:0/2703563561 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6520085ab0 con 0x7f6528159ff0 2024-01-11T07:12:30.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.741+0000 7f652ddf0700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f650c066f30 0x7f650c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:30.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.742+0000 7f652ddf0700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f650c066f30 0x7f650c0693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6524007fd0 tx=0x7f6524009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:30.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.746+0000 7f651dffb700 1 -- 172.21.15.7:0/2703563561 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6520052d20 con 0x7f6528159ff0 2024-01-11T07:12:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.949+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65280c5b90 con 0x7f6528159ff0 2024-01-11T07:12:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.950+0000 7f651dffb700 1 -- 172.21.15.7:0/2703563561 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+15309 (secure 0 0 0) 0x7f65280c5b90 con 0x7f6528159ff0 2024-01-11T07:12:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:30.950 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":108,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:28.332197+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:11:55.626974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-11T07:11:56.083170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f650c066f30 msgr2=0x7f650c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f650c066f30 0x7f650c0693e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6524007fd0 tx=0x7f6524009040 comp rx=0 tx=0).stop 2024-01-11T07:12:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 msgr2=0x7f65280a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f65280a6e70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f65200072f0 tx=0x7f6520018000 comp rx=0 tx=0).stop 2024-01-11T07:12:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 shutdown_connections 2024-01-11T07:12:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f650c066f30 0x7f650c0693e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6528159ff0 0x7f65280a6e70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f652806dd00 0x7f6528158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 --2- 172.21.15.7:0/2703563561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f652806e0d0 0x7f65280aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 >> 172.21.15.7:0/2703563561 conn(0x7f65280680e0 msgr2=0x7f65280cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 shutdown_connections 2024-01-11T07:12:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:30.952+0000 7f6530054700 1 -- 172.21.15.7:0/2703563561 wait complete. 2024-01-11T07:12:30.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:11:55.626974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:11:56.083170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:12:30.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:12:31.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:12:31.037 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:12:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.102+0000 7ff9113a7700 1 Processor -- start 2024-01-11T07:12:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.102+0000 7ff9113a7700 1 -- start start 2024-01-11T07:12:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff9113a7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff90c0cb790 0x7ff90c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff9113a7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cbb60 0x7ff90c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff9113a7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff90c15e5a0 0x7ff90c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff9113a7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90c15b330 con 0x7ff90c0cb790 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff9113a7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90c0ce700 con 0x7ff90c15e5a0 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff9113a7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c15e110 con 0x7ff90c0cbb60 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff90c0cb790 0x7ff90c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff90c0cb790 0x7ff90c0cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46570/0 (socket says 172.21.15.7:46570) 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90affd700 1 -- 172.21.15.7:0/915602803 learned_addr learned my addr 172.21.15.7:0/915602803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90a7fc700 1 --2- 172.21.15.7:0/915602803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cbb60 0x7ff90c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90b7fe700 1 --2- 172.21.15.7:0/915602803 >> v2:172.21.15.113:3300/0 conn(0x7ff90c15e5a0 0x7ff90c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90affd700 1 -- 172.21.15.7:0/915602803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cbb60 msgr2=0x7ff90c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90affd700 1 --2- 172.21.15.7:0/915602803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cbb60 0x7ff90c0ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90affd700 1 -- 172.21.15.7:0/915602803 >> v2:172.21.15.113:3300/0 conn(0x7ff90c15e5a0 msgr2=0x7ff90c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90affd700 1 --2- 172.21.15.7:0/915602803 >> v2:172.21.15.113:3300/0 conn(0x7ff90c15e5a0 0x7ff90c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.103+0000 7ff90affd700 1 -- 172.21.15.7:0/915602803 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c15b050 con 0x7ff90c0cb790 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff90affd700 1 --2- 172.21.15.7:0/915602803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c0cb790 0x7ff90c0cdc80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff90c157e70 tx=0x7ff8fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f9b61d0a634dee server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff909ffb700 1 -- 172.21.15.7:0/915602803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00b640 con 0x7ff90c0cb790 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff909ffb700 1 -- 172.21.15.7:0/915602803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc00b7a0 con 0x7ff90c0cb790 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff909ffb700 1 -- 172.21.15.7:0/915602803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00bb60 con 0x7ff90c0cb790 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff9113a7700 1 -- 172.21.15.7:0/915602803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c0cb790 msgr2=0x7ff90c0cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff9113a7700 1 --2- 172.21.15.7:0/915602803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c0cb790 0x7ff90c0cdc80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff90c157e70 tx=0x7ff8fc009250 comp rx=0 tx=0).stop 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff9113a7700 1 -- 172.21.15.7:0/915602803 shutdown_connections 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff9113a7700 1 --2- 172.21.15.7:0/915602803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c0cb790 0x7ff90c0cdc80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff9113a7700 1 --2- 172.21.15.7:0/915602803 >> v2:172.21.15.113:3300/0 conn(0x7ff90c15e5a0 0x7ff90c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff9113a7700 1 --2- 172.21.15.7:0/915602803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cbb60 0x7ff90c0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.104+0000 7ff9113a7700 1 -- 172.21.15.7:0/915602803 >> 172.21.15.7:0/915602803 conn(0x7ff90c06ae00 msgr2=0x7ff90c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.105+0000 7ff9113a7700 1 -- 172.21.15.7:0/915602803 shutdown_connections 2024-01-11T07:12:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.105+0000 7ff9113a7700 1 -- 172.21.15.7:0/915602803 wait complete. 2024-01-11T07:12:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.106+0000 7ff9113a7700 1 Processor -- start 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.106+0000 7ff9113a7700 1 -- start start 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff9113a7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cb790 0x7ff90c14c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff9113a7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90c0cbb60 0x7ff90c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff9113a7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c15e5a0 0x7ff90c159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff9113a7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c158f40 con 0x7ff90c15e5a0 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff9113a7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0b23c0 con 0x7ff90c0cbb60 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff9113a7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c158c60 con 0x7ff90c0cb790 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c15e5a0 0x7ff90c159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c15e5a0 0x7ff90c159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46574/0 (socket says 172.21.15.7:46574) 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 -- 172.21.15.7:0/811648995 learned_addr learned my addr 172.21.15.7:0/811648995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90affd700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cb790 0x7ff90c14c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90a7fc700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90c0cbb60 0x7ff90c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 -- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cb790 msgr2=0x7ff90c14c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cb790 0x7ff90c14c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 -- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90c0cbb60 msgr2=0x7ff90c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90c0cbb60 0x7ff90c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 -- 172.21.15.7:0/811648995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0094f0 con 0x7ff90c15e5a0 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.107+0000 7ff90b7fe700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c15e5a0 0x7ff90c159310 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff904009df0 tx=0x7ff90400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.108+0000 7ff8f2ffd700 1 -- 172.21.15.7:0/811648995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90400caa0 con 0x7ff90c15e5a0 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.108+0000 7ff8f2ffd700 1 -- 172.21.15.7:0/811648995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90400f040 con 0x7ff90c15e5a0 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.108+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c0b1d70 con 0x7ff90c15e5a0 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.108+0000 7ff8f2ffd700 1 -- 172.21.15.7:0/811648995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90400ce50 con 0x7ff90c15e5a0 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.108+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90c14ca30 con 0x7ff90c15e5a0 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.109+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90c0cb380 con 0x7ff90c15e5a0 2024-01-11T07:12:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.109+0000 7ff8f2ffd700 1 -- 172.21.15.7:0/811648995 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff904014690 con 0x7ff90c15e5a0 2024-01-11T07:12:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.109+0000 7ff8f2ffd700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8f8066f30 0x7ff8f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.110+0000 7ff8f2ffd700 1 -- 172.21.15.7:0/811648995 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff904085030 con 0x7ff90c15e5a0 2024-01-11T07:12:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.110+0000 7ff90affd700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8f8066f30 0x7ff8f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.110+0000 7ff90affd700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8f8066f30 0x7ff8f80693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff90c157e70 tx=0x7ff8fc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.114+0000 7ff8f2ffd700 1 -- 172.21.15.7:0/811648995 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9040522a0 con 0x7ff90c15e5a0 2024-01-11T07:12:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.316+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff90c04c3c0 con 0x7ff90c15e5a0 2024-01-11T07:12:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.316+0000 7ff8f2ffd700 1 -- 172.21.15.7:0/811648995 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9040558c0 con 0x7ff90c15e5a0 2024-01-11T07:12:31.316 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:12:31.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8f8066f30 msgr2=0x7ff8f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8f8066f30 0x7ff8f80693e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff90c157e70 tx=0x7ff8fc017000 comp rx=0 tx=0).stop 2024-01-11T07:12:31.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c15e5a0 msgr2=0x7ff90c159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c15e5a0 0x7ff90c159310 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff904009df0 tx=0x7ff90400b040 comp rx=0 tx=0).stop 2024-01-11T07:12:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 shutdown_connections 2024-01-11T07:12:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff8f8066f30 0x7ff8f80693e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff90c15e5a0 0x7ff90c159310 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff90c0cb790 0x7ff90c14c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 --2- 172.21.15.7:0/811648995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff90c0cbb60 0x7ff90c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 >> 172.21.15.7:0/811648995 conn(0x7ff90c06ae00 msgr2=0x7ff90c0c4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.318+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 shutdown_connections 2024-01-11T07:12:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.319+0000 7ff9113a7700 1 -- 172.21.15.7:0/811648995 wait complete. 2024-01-11T07:12:31.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:12:31.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.466+0000 7fce839b8700 1 Processor -- start 2024-01-11T07:12:31.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.467+0000 7fce839b8700 1 -- start start 2024-01-11T07:12:31.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.467+0000 7fce839b8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce7c06dd00 0x7fce7c0c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.467+0000 7fce839b8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c15b770 con 0x7fce7c15a1c0 2024-01-11T07:12:31.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce839b8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c15a620 con 0x7fce7c06e0d0 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce839b8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c15abf0 con 0x7fce7c06dd00 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce81f55700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce7c15a1c0 0x7fce7c069fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57508/0 (socket says 172.21.15.7:57508) 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce81f55700 1 -- 172.21.15.7:0/1029279920 learned_addr learned my addr 172.21.15.7:0/1029279920 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce81754700 1 --2- 172.21.15.7:0/1029279920 >> v2:172.21.15.113:3300/0 conn(0x7fce7c06dd00 0x7fce7c0c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706100809 0 0) 0x7fce7c15b770 con 0x7fce7c15a1c0 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce68003570 con 0x7fce7c15a1c0 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2548806105 0 0) 0x7fce68003570 con 0x7fce7c15a1c0 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 >> v2:172.21.15.113:3300/0 conn(0x7fce7c06dd00 msgr2=0x7fce7c0c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 --2- 172.21.15.7:0/1029279920 >> v2:172.21.15.113:3300/0 conn(0x7fce7c06dd00 0x7fce7c0c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 >> v1:172.21.15.113:6789/0 conn(0x7fce7c06e0d0 legacy=0x7fce7c159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c15aed0 con 0x7fce7c15a1c0 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fce740017f0 con 0x7fce7c15a1c0 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 >> v1:172.21.15.7:6789/0 conn(0x7fce7c15a1c0 legacy=0x7fce7c069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce68003a60 0x7fce68003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce680051c0 0x7fce6800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c15abf0 con 0x7fce68004810 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6800c160 con 0x7fce68003a60 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.468+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6800c320 con 0x7fce680051c0 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce80f53700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce81754700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce68003a60 0x7fce68003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce81f55700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce680051c0 0x7fce6800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce80f53700 1 -- 172.21.15.7:0/1029279920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce680051c0 msgr2=0x7fce6800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce80f53700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce680051c0 0x7fce6800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce80f53700 1 -- 172.21.15.7:0/1029279920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce68003a60 msgr2=0x7fce68003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce80f53700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce68003a60 0x7fce68003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce80f53700 1 -- 172.21.15.7:0/1029279920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68003570 con 0x7fce68004810 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce81f55700 1 -- 172.21.15.7:0/1029279920 reap_dead start 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce80f53700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce68004c80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0018b0 tx=0x7fce6c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deecf06faf8a4588 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c012420 con 0x7fce68004810 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c0125a0 con 0x7fce68004810 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.469+0000 7fce73fff700 1 -- 172.21.15.7:0/1029279920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c012960 con 0x7fce68004810 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.470+0000 7fce839b8700 1 -- 172.21.15.7:0/1029279920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 msgr2=0x7fce68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.470+0000 7fce839b8700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce68004c80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0018b0 tx=0x7fce6c007d30 comp rx=0 tx=0).stop 2024-01-11T07:12:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.470+0000 7fce839b8700 1 -- 172.21.15.7:0/1029279920 shutdown_connections 2024-01-11T07:12:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.470+0000 7fce839b8700 1 --2- 172.21.15.7:0/1029279920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce68004c80 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.470+0000 7fce839b8700 1 -- 172.21.15.7:0/1029279920 >> 172.21.15.7:0/1029279920 conn(0x7fce7c0680e0 msgr2=0x7fce7c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.470+0000 7fce839b8700 1 -- 172.21.15.7:0/1029279920 shutdown_connections 2024-01-11T07:12:31.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.470+0000 7fce839b8700 1 -- 172.21.15.7:0/1029279920 wait complete. 2024-01-11T07:12:31.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.471+0000 7fce839b8700 1 Processor -- start 2024-01-11T07:12:31.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.471+0000 7fce839b8700 1 -- start start 2024-01-11T07:12:31.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce839b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce7c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce839b8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7c063480 0x7fce7c0a6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce839b8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce7c0638d0 0x7fce7c0a7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce839b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c159350 con 0x7fce68004810 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce839b8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0a7ca0 con 0x7fce7c0638d0 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce839b8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0b0e40 con 0x7fce7c063480 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce81754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce7c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce81754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce7c0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46590/0 (socket says 172.21.15.7:46590) 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.472+0000 7fce81754700 1 -- 172.21.15.7:0/2296234219 learned_addr learned my addr 172.21.15.7:0/2296234219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce81f55700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce7c0638d0 0x7fce7c0a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce80f53700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7c063480 0x7fce7c0a6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce81754700 1 -- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7c063480 msgr2=0x7fce7c0a6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce81754700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7c063480 0x7fce7c0a6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce81754700 1 -- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce7c0638d0 msgr2=0x7fce7c0a7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce81754700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce7c0638d0 0x7fce7c0a7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce81754700 1 -- 172.21.15.7:0/2296234219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c0a79c0 con 0x7fce68004810 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce81f55700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce7c0638d0 0x7fce7c0a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce80f53700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7c063480 0x7fce7c0a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce81754700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce7c0b12d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fce78011a10 tx=0x7fce780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce717fa700 1 -- 172.21.15.7:0/2296234219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7800f940 con 0x7fce68004810 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce717fa700 1 -- 172.21.15.7:0/2296234219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7800faa0 con 0x7fce68004810 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c0b07f0 con 0x7fce68004810 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce717fa700 1 -- 172.21.15.7:0/2296234219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7800fe20 con 0x7fce68004810 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.473+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c1583c0 con 0x7fce68004810 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.474+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c04c3c0 con 0x7fce68004810 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.474+0000 7fce717fa700 1 -- 172.21.15.7:0/2296234219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fce78014650 con 0x7fce68004810 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.475+0000 7fce717fa700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fce60066f60 0x7fce60069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.475+0000 7fce717fa700 1 -- 172.21.15.7:0/2296234219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce78018030 con 0x7fce68004810 2024-01-11T07:12:31.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.475+0000 7fce80f53700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fce60066f60 0x7fce60069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.476+0000 7fce80f53700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fce60066f60 0x7fce60069410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fce7c063f70 tx=0x7fce6c015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.479+0000 7fce717fa700 1 -- 172.21.15.7:0/2296234219 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce78052160 con 0x7fce68004810 2024-01-11T07:12:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.684+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce7c15b1b0 con 0x7fce68004810 2024-01-11T07:12:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.685+0000 7fce717fa700 1 -- 172.21.15.7:0/2296234219 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce78055780 con 0x7fce68004810 2024-01-11T07:12:31.685 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:12:31.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fce60066f60 msgr2=0x7fce60069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fce60066f60 0x7fce60069410 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fce7c063f70 tx=0x7fce6c015000 comp rx=0 tx=0).stop 2024-01-11T07:12:31.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 msgr2=0x7fce7c0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce7c0b12d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fce78011a10 tx=0x7fce780091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 shutdown_connections 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fce60066f60 0x7fce60069410 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce7c0638d0 0x7fce7c0a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce68004810 0x7fce7c0b12d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 --2- 172.21.15.7:0/2296234219 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce7c063480 0x7fce7c0a6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 >> 172.21.15.7:0/2296234219 conn(0x7fce7c0680e0 msgr2=0x7fce7c0ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 shutdown_connections 2024-01-11T07:12:31.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.687+0000 7fce839b8700 1 -- 172.21.15.7:0/2296234219 wait complete. 2024-01-11T07:12:31.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:12:31.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.835+0000 7f524a276700 1 Processor -- start 2024-01-11T07:12:31.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.835+0000 7f524a276700 1 -- start start 2024-01-11T07:12:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f524a276700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f524406a9c0 0x7f524415af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f524a276700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5244063230 0x7f5244063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f524a276700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524415b4b0 con 0x7f52441579a0 2024-01-11T07:12:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f524a276700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524406bb60 con 0x7f524406a9c0 2024-01-11T07:12:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f524a276700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524406e8b0 con 0x7f5244063230 2024-01-11T07:12:31.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f52437fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f524406a9c0 0x7f524415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f52437fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f524406a9c0 0x7f524415af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46606/0 (socket says 172.21.15.7:46606) 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f52437fe700 1 -- 172.21.15.7:0/1038929612 learned_addr learned my addr 172.21.15.7:0/1038929612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f5248813700 1 --2- 172.21.15.7:0/1038929612 >> v2:172.21.15.113:3300/0 conn(0x7f5244063230 0x7f5244063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f52437fe700 1 -- 172.21.15.7:0/1038929612 >> v2:172.21.15.113:3300/0 conn(0x7f5244063230 msgr2=0x7f5244063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f52437fe700 1 --2- 172.21.15.7:0/1038929612 >> v2:172.21.15.113:3300/0 conn(0x7f5244063230 0x7f5244063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f52437fe700 1 -- 172.21.15.7:0/1038929612 >> v1:172.21.15.113:6789/0 conn(0x7f52441579a0 legacy=0x7f524406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f52437fe700 1 -- 172.21.15.7:0/1038929612 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5244063b40 con 0x7f524406a9c0 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.836+0000 7f52437fe700 1 --2- 172.21.15.7:0/1038929612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f524415af70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f52440b05f0 tx=0x7f5238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab579cb55ca1d70 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f5242ffd700 1 -- 172.21.15.7:0/1038929612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238009e70 con 0x7f524406a9c0 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f5242ffd700 1 -- 172.21.15.7:0/1038929612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523800b860 con 0x7f524406a9c0 2024-01-11T07:12:31.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f5242ffd700 1 -- 172.21.15.7:0/1038929612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523800bc00 con 0x7f524406a9c0 2024-01-11T07:12:31.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f524a276700 1 -- 172.21.15.7:0/1038929612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 msgr2=0x7f524415af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f524a276700 1 --2- 172.21.15.7:0/1038929612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f524415af70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f52440b05f0 tx=0x7f5238009250 comp rx=0 tx=0).stop 2024-01-11T07:12:31.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f524a276700 1 -- 172.21.15.7:0/1038929612 shutdown_connections 2024-01-11T07:12:31.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f524a276700 1 --2- 172.21.15.7:0/1038929612 >> v2:172.21.15.113:3300/0 conn(0x7f5244063230 0x7f5244063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f524a276700 1 --2- 172.21.15.7:0/1038929612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f524415af70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f524a276700 1 -- 172.21.15.7:0/1038929612 >> 172.21.15.7:0/1038929612 conn(0x7f52440cd8f0 msgr2=0x7f52440cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:31.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f524a276700 1 -- 172.21.15.7:0/1038929612 shutdown_connections 2024-01-11T07:12:31.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.837+0000 7f524a276700 1 -- 172.21.15.7:0/1038929612 wait complete. 2024-01-11T07:12:31.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.838+0000 7f524a276700 1 Processor -- start 2024-01-11T07:12:31.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.839+0000 7f524a276700 1 -- start start 2024-01-11T07:12:31.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.839+0000 7f524a276700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5244063230 0x7f524415a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.839+0000 7f524a276700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f52440b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.839+0000 7f524a276700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52441579a0 0x7f52440b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.839+0000 7f524a276700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52440b1760 con 0x7f524406a9c0 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.839+0000 7f524a276700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5244153d40 con 0x7f52441579a0 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.839+0000 7f524a276700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52440aeda0 con 0x7f5244063230 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f52440b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f52440b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46616/0 (socket says 172.21.15.7:46616) 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 -- 172.21.15.7:0/1507555457 learned_addr learned my addr 172.21.15.7:0/1507555457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f5248813700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52441579a0 0x7f52440b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f5243fff700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5244063230 0x7f524415a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 -- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5244063230 msgr2=0x7f524415a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5244063230 0x7f524415a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 -- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52441579a0 msgr2=0x7f52440b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:31.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52441579a0 0x7f52440b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 -- 172.21.15.7:0/1507555457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380094f0 con 0x7f524406a9c0 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f52437fe700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f52440b1220 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5238006fd0 tx=0x7f5238017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f522bfff700 1 -- 172.21.15.7:0/1507555457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523800bbe0 con 0x7f524406a9c0 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f522bfff700 1 -- 172.21.15.7:0/1507555457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238011420 con 0x7f524406a9c0 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52440afbd0 con 0x7f524406a9c0 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f522bfff700 1 -- 172.21.15.7:0/1507555457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380117c0 con 0x7f524406a9c0 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.840+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52440ae470 con 0x7f524406a9c0 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.841+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5244064a40 con 0x7f524406a9c0 2024-01-11T07:12:31.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.842+0000 7f522bfff700 1 -- 172.21.15.7:0/1507555457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5238011bb0 con 0x7f524406a9c0 2024-01-11T07:12:31.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.842+0000 7f522bfff700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5230066f30 0x7f52300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:31.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.842+0000 7f522bfff700 1 -- 172.21.15.7:0/1507555457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52380858f0 con 0x7f524406a9c0 2024-01-11T07:12:31.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.843+0000 7f5243fff700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5230066f30 0x7f52300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:31.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.843+0000 7f5243fff700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5230066f30 0x7f52300693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f52340039e0 tx=0x7f5234007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:31.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:31.847+0000 7f522bfff700 1 -- 172.21.15.7:0/1507555457 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5238052be0 con 0x7f524406a9c0 2024-01-11T07:12:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.050+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52440aeee0 con 0x7f524406a9c0 2024-01-11T07:12:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.050+0000 7f522bfff700 1 -- 172.21.15.7:0/1507555457 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52380150a0 con 0x7f524406a9c0 2024-01-11T07:12:32.050 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:12:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5230066f30 msgr2=0x7f52300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5230066f30 0x7f52300693e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f52340039e0 tx=0x7f5234007ec0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 msgr2=0x7f52440b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f52440b1220 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5238006fd0 tx=0x7f5238017000 comp rx=0 tx=0).stop 2024-01-11T07:12:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 shutdown_connections 2024-01-11T07:12:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5230066f30 0x7f52300693e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52441579a0 0x7f52440b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5244063230 0x7f524415a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 --2- 172.21.15.7:0/1507555457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f524406a9c0 0x7f52440b1220 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 >> 172.21.15.7:0/1507555457 conn(0x7f52440cd8f0 msgr2=0x7f52440cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 shutdown_connections 2024-01-11T07:12:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.052+0000 7f524a276700 1 -- 172.21.15.7:0/1507555457 wait complete. 2024-01-11T07:12:32.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.202+0000 7f4c9a469700 1 Processor -- start 2024-01-11T07:12:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.202+0000 7f4c9a469700 1 -- start start 2024-01-11T07:12:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c9a469700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c94158280 0x7f4c940b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c9a469700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c9415aa80 0x7f4c940b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c9a469700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c94159300 con 0x7f4c940cafb0 2024-01-11T07:12:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c9a469700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940cf000 con 0x7f4c94158280 2024-01-11T07:12:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c9a469700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940cf260 con 0x7f4c9415aa80 2024-01-11T07:12:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c94158280 0x7f4c940b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c937fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c94158280 0x7f4c940b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46622/0 (socket says 172.21.15.7:46622) 2024-01-11T07:12:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c937fe700 1 -- 172.21.15.7:0/2708446957 learned_addr learned my addr 172.21.15.7:0/2708446957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c98a06700 1 --2- 172.21.15.7:0/2708446957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c9415aa80 0x7f4c940b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c937fe700 1 -- 172.21.15.7:0/2708446957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c9415aa80 msgr2=0x7f4c940b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c937fe700 1 --2- 172.21.15.7:0/2708446957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c9415aa80 0x7f4c940b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c937fe700 1 -- 172.21.15.7:0/2708446957 >> v1:172.21.15.113:6789/0 conn(0x7f4c940cafb0 legacy=0x7f4c940ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c937fe700 1 -- 172.21.15.7:0/2708446957 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940cf540 con 0x7f4c94158280 2024-01-11T07:12:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c937fe700 1 --2- 172.21.15.7:0/2708446957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94158280 0x7f4c940b0430 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4c880099b0 tx=0x7f4c88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31143f258c62497 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.203+0000 7f4c92ffd700 1 -- 172.21.15.7:0/2708446957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800b860 con 0x7f4c94158280 2024-01-11T07:12:32.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c92ffd700 1 -- 172.21.15.7:0/2708446957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8800b9c0 con 0x7f4c94158280 2024-01-11T07:12:32.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c92ffd700 1 -- 172.21.15.7:0/2708446957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800bd60 con 0x7f4c94158280 2024-01-11T07:12:32.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c9a469700 1 -- 172.21.15.7:0/2708446957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94158280 msgr2=0x7f4c940b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c9a469700 1 --2- 172.21.15.7:0/2708446957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94158280 0x7f4c940b0430 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4c880099b0 tx=0x7f4c88009250 comp rx=0 tx=0).stop 2024-01-11T07:12:32.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c9a469700 1 -- 172.21.15.7:0/2708446957 shutdown_connections 2024-01-11T07:12:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c9a469700 1 --2- 172.21.15.7:0/2708446957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c9415aa80 0x7f4c940b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c9a469700 1 --2- 172.21.15.7:0/2708446957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94158280 0x7f4c940b0430 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c9a469700 1 -- 172.21.15.7:0/2708446957 >> 172.21.15.7:0/2708446957 conn(0x7f4c94071100 msgr2=0x7f4c9406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c9a469700 1 -- 172.21.15.7:0/2708446957 shutdown_connections 2024-01-11T07:12:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.204+0000 7f4c9a469700 1 -- 172.21.15.7:0/2708446957 wait complete. 2024-01-11T07:12:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.205+0000 7f4c9a469700 1 Processor -- start 2024-01-11T07:12:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.206+0000 7f4c9a469700 1 -- start start 2024-01-11T07:12:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.206+0000 7f4c9a469700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c940cafb0 0x7f4c940a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.206+0000 7f4c9a469700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c94158280 0x7f4c940a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.206+0000 7f4c9a469700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9415aa80 0x7f4c940a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.206+0000 7f4c9a469700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940a73e0 con 0x7f4c9415aa80 2024-01-11T07:12:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.206+0000 7f4c9a469700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94167f30 con 0x7f4c940cafb0 2024-01-11T07:12:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.206+0000 7f4c9a469700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940b3420 con 0x7f4c94158280 2024-01-11T07:12:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9415aa80 0x7f4c940a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9415aa80 0x7f4c940a7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46636/0 (socket says 172.21.15.7:46636) 2024-01-11T07:12:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 -- 172.21.15.7:0/650483647 learned_addr learned my addr 172.21.15.7:0/650483647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c93fff700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c940cafb0 0x7f4c940a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c937fe700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c94158280 0x7f4c940a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 -- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c94158280 msgr2=0x7f4c940a75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c94158280 0x7f4c940a75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 -- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c940cafb0 msgr2=0x7f4c940a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c940cafb0 0x7f4c940a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 -- 172.21.15.7:0/650483647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880094f0 con 0x7f4c9415aa80 2024-01-11T07:12:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c98a06700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9415aa80 0x7f4c940a7b20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c007f50 tx=0x7f4c8c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c7bfff700 1 -- 172.21.15.7:0/650483647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00ecd0 con 0x7f4c9415aa80 2024-01-11T07:12:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c7bfff700 1 -- 172.21.15.7:0/650483647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00ee50 con 0x7f4c9415aa80 2024-01-11T07:12:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c940a6b40 con 0x7f4c9415aa80 2024-01-11T07:12:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c7bfff700 1 -- 172.21.15.7:0/650483647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c014670 con 0x7f4c9415aa80 2024-01-11T07:12:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.207+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c940a7100 con 0x7f4c9415aa80 2024-01-11T07:12:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.208+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94167d00 con 0x7f4c9415aa80 2024-01-11T07:12:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.209+0000 7f4c7bfff700 1 -- 172.21.15.7:0/650483647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4c8c0149e0 con 0x7f4c9415aa80 2024-01-11T07:12:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.209+0000 7f4c7bfff700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.209+0000 7f4c7bfff700 1 -- 172.21.15.7:0/650483647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c8c084cb0 con 0x7f4c9415aa80 2024-01-11T07:12:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.210+0000 7f4c93fff700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.210+0000 7f4c93fff700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c80066f30 0x7f4c800693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c84003860 tx=0x7f4c84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.214+0000 7f4c7bfff700 1 -- 172.21.15.7:0/650483647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c94167d00 con 0x7f4c9415aa80 2024-01-11T07:12:32.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.467+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c94167d00 con 0x7f4c9415aa80 2024-01-11T07:12:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.467+0000 7f4c7bfff700 1 -- 172.21.15.7:0/650483647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f4c94167d00 con 0x7f4c9415aa80 2024-01-11T07:12:32.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:32.467 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c80066f30 msgr2=0x7f4c800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c80066f30 0x7f4c800693e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c84003860 tx=0x7f4c84009040 comp rx=0 tx=0).stop 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9415aa80 msgr2=0x7f4c940a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9415aa80 0x7f4c940a7b20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c007f50 tx=0x7f4c8c009860 comp rx=0 tx=0).stop 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 shutdown_connections 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c80066f30 0x7f4c800693e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9415aa80 0x7f4c940a7b20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c940cafb0 0x7f4c940a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 --2- 172.21.15.7:0/650483647 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c94158280 0x7f4c940a75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 >> 172.21.15.7:0/650483647 conn(0x7f4c94071100 msgr2=0x7f4c940cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 shutdown_connections 2024-01-11T07:12:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.470+0000 7f4c9a469700 1 -- 172.21.15.7:0/650483647 wait complete. 2024-01-11T07:12:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:32.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.615+0000 7f1c46f8a700 1 Processor -- start 2024-01-11T07:12:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.615+0000 7f1c46f8a700 1 -- start start 2024-01-11T07:12:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c46f8a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400b0b40 0x7f1c40162770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c46f8a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c40159300 con 0x7f1c400cafb0 2024-01-11T07:12:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c46f8a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400cf000 con 0x7f1c40158280 2024-01-11T07:12:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c46f8a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400cf260 con 0x7f1c400b0b40 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c44d26700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1c400cafb0 0x7f1c400ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57524/0 (socket says 172.21.15.7:57524) 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c44d26700 1 -- 172.21.15.7:0/934101926 learned_addr learned my addr 172.21.15.7:0/934101926 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c45527700 1 --2- 172.21.15.7:0/934101926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400b0b40 0x7f1c40162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c3f7fe700 1 -- 172.21.15.7:0/934101926 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955590799 0 0) 0x7f1c40159300 con 0x7f1c400cafb0 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c3f7fe700 1 -- 172.21.15.7:0/934101926 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c30003570 con 0x7f1c400cafb0 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c3f7fe700 1 -- 172.21.15.7:0/934101926 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845365163 0 0) 0x7f1c400cf000 con 0x7f1c40158280 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c3f7fe700 1 -- 172.21.15.7:0/934101926 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c40159300 con 0x7f1c40158280 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c45527700 1 -- 172.21.15.7:0/934101926 >> v1:172.21.15.113:6789/0 conn(0x7f1c40158280 legacy=0x7f1c400b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c3f7fe700 1 -- 172.21.15.7:0/934101926 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 400242713 0 0) 0x7f1c30003570 con 0x7f1c400cafb0 2024-01-11T07:12:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c45527700 1 -- 172.21.15.7:0/934101926 >> v1:172.21.15.7:6789/0 conn(0x7f1c400cafb0 legacy=0x7f1c400ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.616+0000 7f1c45527700 1 -- 172.21.15.7:0/934101926 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400cf540 con 0x7f1c400b0b40 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.617+0000 7f1c45527700 1 --2- 172.21.15.7:0/934101926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400b0b40 0x7f1c40162770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009e90 tx=0x7f1c38009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae85fe57041fc58 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.617+0000 7f1c3f7fe700 1 -- 172.21.15.7:0/934101926 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800e070 con 0x7f1c400b0b40 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.617+0000 7f1c3f7fe700 1 -- 172.21.15.7:0/934101926 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800b870 con 0x7f1c400b0b40 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.617+0000 7f1c3f7fe700 1 -- 172.21.15.7:0/934101926 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34001470 con 0x7f1c400b0b40 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.617+0000 7f1c46f8a700 1 -- 172.21.15.7:0/934101926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400b0b40 msgr2=0x7f1c40162770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.617+0000 7f1c46f8a700 1 --2- 172.21.15.7:0/934101926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400b0b40 0x7f1c40162770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009e90 tx=0x7f1c38009780 comp rx=0 tx=0).stop 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.618+0000 7f1c46f8a700 1 -- 172.21.15.7:0/934101926 shutdown_connections 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.618+0000 7f1c46f8a700 1 --2- 172.21.15.7:0/934101926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400b0b40 0x7f1c40162770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.618+0000 7f1c46f8a700 1 -- 172.21.15.7:0/934101926 >> 172.21.15.7:0/934101926 conn(0x7f1c40071100 msgr2=0x7f1c4006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.618+0000 7f1c46f8a700 1 -- 172.21.15.7:0/934101926 shutdown_connections 2024-01-11T07:12:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.618+0000 7f1c46f8a700 1 -- 172.21.15.7:0/934101926 wait complete. 2024-01-11T07:12:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.619+0000 7f1c46f8a700 1 Processor -- start 2024-01-11T07:12:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.620+0000 7f1c46f8a700 1 -- start start 2024-01-11T07:12:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.620+0000 7f1c46f8a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c400b0b40 0x7f1c400c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.620+0000 7f1c46f8a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400cafb0 0x7f1c400cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.620+0000 7f1c46f8a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c40158280 0x7f1c40153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.620+0000 7f1c46f8a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400c5920 con 0x7f1c40158280 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.620+0000 7f1c46f8a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400c4db0 con 0x7f1c400b0b40 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.620+0000 7f1c46f8a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400c5080 con 0x7f1c400cafb0 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c40158280 0x7f1c40153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c40158280 0x7f1c40153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46638/0 (socket says 172.21.15.7:46638) 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 -- 172.21.15.7:0/3244573031 learned_addr learned my addr 172.21.15.7:0/3244573031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c3ffff700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400cafb0 0x7f1c400cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c44d26700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c400b0b40 0x7f1c400c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 -- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400cafb0 msgr2=0x7f1c400cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400cafb0 0x7f1c400cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 -- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c400b0b40 msgr2=0x7f1c400c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c400b0b40 0x7f1c400c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 -- 172.21.15.7:0/3244573031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c38009a20 con 0x7f1c40158280 2024-01-11T07:12:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c44d26700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c400b0b40 0x7f1c400c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c45527700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c40158280 0x7f1c40153250 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009de0 tx=0x7f1c38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c3cff9700 1 -- 172.21.15.7:0/3244573031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38012d80 con 0x7f1c40158280 2024-01-11T07:12:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c3cff9700 1 -- 172.21.15.7:0/3244573031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3801e450 con 0x7f1c40158280 2024-01-11T07:12:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c400c5360 con 0x7f1c40158280 2024-01-11T07:12:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c3cff9700 1 -- 172.21.15.7:0/3244573031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3801e810 con 0x7f1c40158280 2024-01-11T07:12:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.621+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c400c5eb0 con 0x7f1c40158280 2024-01-11T07:12:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.622+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c400c6220 con 0x7f1c40158280 2024-01-11T07:12:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.623+0000 7f1c3cff9700 1 -- 172.21.15.7:0/3244573031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1c3801ec40 con 0x7f1c40158280 2024-01-11T07:12:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.623+0000 7f1c3cff9700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c24066f30 0x7f1c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.624+0000 7f1c3cff9700 1 -- 172.21.15.7:0/3244573031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c380863a0 con 0x7f1c40158280 2024-01-11T07:12:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.624+0000 7f1c44d26700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c24066f30 0x7f1c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.624+0000 7f1c44d26700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c24066f30 0x7f1c240693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c340015a0 tx=0x7f1c34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.628+0000 7f1c3cff9700 1 -- 172.21.15.7:0/3244573031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c38053610 con 0x7f1c40158280 2024-01-11T07:12:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.835+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c4004c3c0 con 0x7f1c40158280 2024-01-11T07:12:32.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.836+0000 7f1c3cff9700 1 -- 172.21.15.7:0/3244573031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+15309 (secure 0 0 0) 0x7f1c38056c30 con 0x7f1c40158280 2024-01-11T07:12:32.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:32.836 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":108,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:28.332197+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:11:55.626974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-11T07:11:56.083170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c24066f30 msgr2=0x7f1c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c24066f30 0x7f1c240693e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c340015a0 tx=0x7f1c34009040 comp rx=0 tx=0).stop 2024-01-11T07:12:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c40158280 msgr2=0x7f1c40153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c40158280 0x7f1c40153250 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009de0 tx=0x7f1c38017000 comp rx=0 tx=0).stop 2024-01-11T07:12:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 shutdown_connections 2024-01-11T07:12:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1c24066f30 0x7f1c240693e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c40158280 0x7f1c40153250 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c400b0b40 0x7f1c400c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 --2- 172.21.15.7:0/3244573031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c400cafb0 0x7f1c400cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 >> 172.21.15.7:0/3244573031 conn(0x7f1c40071100 msgr2=0x7f1c40157a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 shutdown_connections 2024-01-11T07:12:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.838+0000 7f1c46f8a700 1 -- 172.21.15.7:0/3244573031 wait complete. 2024-01-11T07:12:32.851 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:12:32.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:32.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.984+0000 7f50dd8d3700 1 Processor -- start 2024-01-11T07:12:32.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.984+0000 7f50dd8d3700 1 -- start start 2024-01-11T07:12:32.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50dd8d3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d806e340 0x7f50d80ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50dd8d3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d80d18c0 con 0x7f50d8159ca0 2024-01-11T07:12:32.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50dd8d3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d80d0010 con 0x7f50d815a070 2024-01-11T07:12:32.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50dd8d3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80cf400 con 0x7f50d806e340 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d6ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f50d8159ca0 0x7f50d806dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57528/0 (socket says 172.21.15.7:57528) 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d6ffd700 1 -- 172.21.15.7:0/2641372849 learned_addr learned my addr 172.21.15.7:0/2641372849 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d67fc700 1 --2- 172.21.15.7:0/2641372849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d806e340 0x7f50d80ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d5ffb700 1 -- 172.21.15.7:0/2641372849 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851934836 0 0) 0x7f50d80d18c0 con 0x7f50d8159ca0 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d5ffb700 1 -- 172.21.15.7:0/2641372849 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c0003570 con 0x7f50d8159ca0 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d5ffb700 1 -- 172.21.15.7:0/2641372849 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596094037 0 0) 0x7f50d80d0010 con 0x7f50d815a070 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d67fc700 1 -- 172.21.15.7:0/2641372849 >> v1:172.21.15.113:6789/0 conn(0x7f50d815a070 legacy=0x7f50d80b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d67fc700 1 -- 172.21.15.7:0/2641372849 >> v1:172.21.15.7:6789/0 conn(0x7f50d8159ca0 legacy=0x7f50d806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.985+0000 7f50d67fc700 1 -- 172.21.15.7:0/2641372849 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80cf6e0 con 0x7f50d806e340 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50d67fc700 1 --2- 172.21.15.7:0/2641372849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d806e340 0x7f50d80ce7f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009910 tx=0x7f50cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5844e361c4464178 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50d5ffb700 1 -- 172.21.15.7:0/2641372849 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00b680 con 0x7f50d806e340 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50d5ffb700 1 -- 172.21.15.7:0/2641372849 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc00b7e0 con 0x7f50d806e340 2024-01-11T07:12:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50d5ffb700 1 -- 172.21.15.7:0/2641372849 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00bba0 con 0x7f50d806e340 2024-01-11T07:12:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50dd8d3700 1 -- 172.21.15.7:0/2641372849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d806e340 msgr2=0x7f50d80ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50dd8d3700 1 --2- 172.21.15.7:0/2641372849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d806e340 0x7f50d80ce7f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009910 tx=0x7f50cc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50dd8d3700 1 -- 172.21.15.7:0/2641372849 shutdown_connections 2024-01-11T07:12:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50dd8d3700 1 --2- 172.21.15.7:0/2641372849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d806e340 0x7f50d80ce7f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50dd8d3700 1 -- 172.21.15.7:0/2641372849 >> 172.21.15.7:0/2641372849 conn(0x7f50d8063cc0 msgr2=0x7f50d80d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50dd8d3700 1 -- 172.21.15.7:0/2641372849 shutdown_connections 2024-01-11T07:12:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.986+0000 7f50dd8d3700 1 -- 172.21.15.7:0/2641372849 wait complete. 2024-01-11T07:12:32.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.988+0000 7f50dd8d3700 1 Processor -- start 2024-01-11T07:12:32.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.988+0000 7f50dd8d3700 1 -- start start 2024-01-11T07:12:32.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50dd8d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d806e340 0x7f50d80c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50dd8d3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d8159ca0 0x7f50d80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50dd8d3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d815a070 0x7f50d816d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50dd8d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80b5400 con 0x7f50d806e340 2024-01-11T07:12:32.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50dd8d3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80b5890 con 0x7f50d8159ca0 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50dd8d3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80b47f0 con 0x7f50d815a070 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d806e340 0x7f50d80c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d806e340 0x7f50d80c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46646/0 (socket says 172.21.15.7:46646) 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 -- 172.21.15.7:0/679937548 learned_addr learned my addr 172.21.15.7:0/679937548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d77fe700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d815a070 0x7f50d816d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d67fc700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d8159ca0 0x7f50d80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 -- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d815a070 msgr2=0x7f50d816d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d815a070 0x7f50d816d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 -- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d8159ca0 msgr2=0x7f50d80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d8159ca0 0x7f50d80b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 -- 172.21.15.7:0/679937548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc009420 con 0x7f50d806e340 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50d6ffd700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d806e340 0x7f50d80c9820 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50c80011e0 tx=0x7f50c8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.989+0000 7f50beffd700 1 -- 172.21.15.7:0/679937548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8005b30 con 0x7f50d806e340 2024-01-11T07:12:32.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.990+0000 7f50beffd700 1 -- 172.21.15.7:0/679937548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8005c90 con 0x7f50d806e340 2024-01-11T07:12:32.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.990+0000 7f50beffd700 1 -- 172.21.15.7:0/679937548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8007300 con 0x7f50d806e340 2024-01-11T07:12:32.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.990+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d80b4ad0 con 0x7f50d806e340 2024-01-11T07:12:32.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.990+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d80b5090 con 0x7f50d806e340 2024-01-11T07:12:32.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.991+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d804c3c0 con 0x7f50d806e340 2024-01-11T07:12:32.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.991+0000 7f50beffd700 1 -- 172.21.15.7:0/679937548 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f50c8007670 con 0x7f50d806e340 2024-01-11T07:12:32.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.991+0000 7f50beffd700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f50c4066f30 0x7f50c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:32.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.992+0000 7f50beffd700 1 -- 172.21.15.7:0/679937548 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50c8083940 con 0x7f50d806e340 2024-01-11T07:12:32.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.992+0000 7f50d67fc700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f50c4066f30 0x7f50c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:32.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.992+0000 7f50d67fc700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f50c4066f30 0x7f50c40693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0093f0 tx=0x7f50cc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:32.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:32.996+0000 7f50beffd700 1 -- 172.21.15.7:0/679937548 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50c8050bb0 con 0x7f50d806e340 2024-01-11T07:12:33.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.249+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50d80cf9c0 con 0x7f50d806e340 2024-01-11T07:12:33.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.250+0000 7f50beffd700 1 -- 172.21.15.7:0/679937548 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f50c8017020 con 0x7f50d806e340 2024-01-11T07:12:33.249 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:33.250 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:33.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f50c4066f30 msgr2=0x7f50c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f50c4066f30 0x7f50c40693e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0093f0 tx=0x7f50cc017000 comp rx=0 tx=0).stop 2024-01-11T07:12:33.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d806e340 msgr2=0x7f50d80c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d806e340 0x7f50d80c9820 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f50c80011e0 tx=0x7f50c8007df0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 shutdown_connections 2024-01-11T07:12:33.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f50c4066f30 0x7f50c40693e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50d815a070 0x7f50d816d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d806e340 0x7f50d80c9820 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 --2- 172.21.15.7:0/679937548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50d8159ca0 0x7f50d80b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 >> 172.21.15.7:0/679937548 conn(0x7f50d8063cc0 msgr2=0x7f50d80c63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 shutdown_connections 2024-01-11T07:12:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.252+0000 7f50dd8d3700 1 -- 172.21.15.7:0/679937548 wait complete. 2024-01-11T07:12:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:33.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:33.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.398+0000 7f2ba038e700 1 Processor -- start 2024-01-11T07:12:33.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.398+0000 7f2ba038e700 1 -- start start 2024-01-11T07:12:33.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2ba038e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b980cafb0 0x7f2b980ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2ba038e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b980b0b40 0x7f2b98161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2ba038e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b98159300 con 0x7f2b98158280 2024-01-11T07:12:33.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2ba038e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b980cf260 con 0x7f2b980b0b40 2024-01-11T07:12:33.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2ba038e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980cf540 con 0x7f2b980cafb0 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9d929700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2b98158280 0x7f2b980b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57538/0 (socket says 172.21.15.7:57538) 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9d929700 1 -- 172.21.15.7:0/3483719328 learned_addr learned my addr 172.21.15.7:0/3483719328 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9e92b700 1 --2- 172.21.15.7:0/3483719328 >> v2:172.21.15.113:3300/0 conn(0x7f2b980b0b40 0x7f2b98161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9e12a700 1 --2- 172.21.15.7:0/3483719328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b980cafb0 0x7f2b980ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9d128700 1 -- 172.21.15.7:0/3483719328 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48155294 0 0) 0x7f2b98159300 con 0x7f2b98158280 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9d128700 1 -- 172.21.15.7:0/3483719328 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b84003570 con 0x7f2b98158280 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9e12a700 1 -- 172.21.15.7:0/3483719328 >> v2:172.21.15.113:3300/0 conn(0x7f2b980b0b40 msgr2=0x7f2b98161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9e12a700 1 --2- 172.21.15.7:0/3483719328 >> v2:172.21.15.113:3300/0 conn(0x7f2b980b0b40 0x7f2b98161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9e12a700 1 -- 172.21.15.7:0/3483719328 >> v1:172.21.15.7:6789/0 conn(0x7f2b98158280 legacy=0x7f2b980b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9e12a700 1 -- 172.21.15.7:0/3483719328 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b980cf820 con 0x7f2b980cafb0 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.399+0000 7f2b9e92b700 1 --2- 172.21.15.7:0/3483719328 >> v2:172.21.15.113:3300/0 conn(0x7f2b980b0b40 0x7f2b98161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.400+0000 7f2b9e12a700 1 --2- 172.21.15.7:0/3483719328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b980cafb0 0x7f2b980ce8f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b940098e0 tx=0x7f2b940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d946e52095c14c41 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.400+0000 7f2b9d128700 1 -- 172.21.15.7:0/3483719328 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94011630 con 0x7f2b980cafb0 2024-01-11T07:12:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.400+0000 7f2b9d128700 1 -- 172.21.15.7:0/3483719328 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b94011790 con 0x7f2b980cafb0 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.400+0000 7f2b9d128700 1 -- 172.21.15.7:0/3483719328 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94011b30 con 0x7f2b980cafb0 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.400+0000 7f2ba038e700 1 -- 172.21.15.7:0/3483719328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b980cafb0 msgr2=0x7f2b980ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.400+0000 7f2ba038e700 1 --2- 172.21.15.7:0/3483719328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b980cafb0 0x7f2b980ce8f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b940098e0 tx=0x7f2b940091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.401+0000 7f2ba038e700 1 -- 172.21.15.7:0/3483719328 shutdown_connections 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.401+0000 7f2ba038e700 1 --2- 172.21.15.7:0/3483719328 >> v2:172.21.15.113:3300/0 conn(0x7f2b980b0b40 0x7f2b98161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.401+0000 7f2ba038e700 1 --2- 172.21.15.7:0/3483719328 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b980cafb0 0x7f2b980ce8f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.401+0000 7f2ba038e700 1 -- 172.21.15.7:0/3483719328 >> 172.21.15.7:0/3483719328 conn(0x7f2b98071100 msgr2=0x7f2b9806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.401+0000 7f2ba038e700 1 -- 172.21.15.7:0/3483719328 shutdown_connections 2024-01-11T07:12:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.401+0000 7f2ba038e700 1 -- 172.21.15.7:0/3483719328 wait complete. 2024-01-11T07:12:33.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.402+0000 7f2ba038e700 1 Processor -- start 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.402+0000 7f2ba038e700 1 -- start start 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2ba038e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b980b0b40 0x7f2b9816be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2ba038e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b980cafb0 0x7f2b9815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2ba038e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b98158280 0x7f2b98155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2ba038e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9815d6d0 con 0x7f2b980cafb0 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2ba038e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9816c3d0 con 0x7f2b980b0b40 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2ba038e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9815c7d0 con 0x7f2b98158280 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2b9d929700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b980cafb0 0x7f2b9815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2b9d929700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b980cafb0 0x7f2b9815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46662/0 (socket says 172.21.15.7:46662) 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.403+0000 7f2b9d929700 1 -- 172.21.15.7:0/2155405017 learned_addr learned my addr 172.21.15.7:0/2155405017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b9e92b700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b98158280 0x7f2b98155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b9e12a700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b980b0b40 0x7f2b9816be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b9d929700 1 -- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b98158280 msgr2=0x7f2b98155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b9d929700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b98158280 0x7f2b98155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b9d929700 1 -- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b980b0b40 msgr2=0x7f2b9816be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b9d929700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b980b0b40 0x7f2b9816be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b9d929700 1 -- 172.21.15.7:0/2155405017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90009a20 con 0x7f2b980cafb0 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b9d929700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b980cafb0 0x7f2b9815cc60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c590 tx=0x7f2b88007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b8e7fc700 1 -- 172.21.15.7:0/2155405017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88005c00 con 0x7f2b980cafb0 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b8e7fc700 1 -- 172.21.15.7:0/2155405017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b88005d60 con 0x7f2b980cafb0 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b94009420 con 0x7f2b980cafb0 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2b8e7fc700 1 -- 172.21.15.7:0/2155405017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88007300 con 0x7f2b980cafb0 2024-01-11T07:12:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.404+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9815bbc0 con 0x7f2b980cafb0 2024-01-11T07:12:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.405+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9804c3c0 con 0x7f2b980cafb0 2024-01-11T07:12:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.406+0000 7f2b8e7fc700 1 -- 172.21.15.7:0/2155405017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2b8801b3b0 con 0x7f2b980cafb0 2024-01-11T07:12:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.406+0000 7f2b8e7fc700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b7c066f30 0x7f2b7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.406+0000 7f2b8e7fc700 1 -- 172.21.15.7:0/2155405017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b880838f0 con 0x7f2b980cafb0 2024-01-11T07:12:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.406+0000 7f2b9e12a700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b7c066f30 0x7f2b7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.407+0000 7f2b9e12a700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b7c066f30 0x7f2b7c0693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2b940091e0 tx=0x7f2b94017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:33.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.411+0000 7f2b8e7fc700 1 -- 172.21.15.7:0/2155405017 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b88050c20 con 0x7f2b980cafb0 2024-01-11T07:12:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.660+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b9815c910 con 0x7f2b980cafb0 2024-01-11T07:12:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.661+0000 7f2b8e7fc700 1 -- 172.21.15.7:0/2155405017 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f2b88017070 con 0x7f2b980cafb0 2024-01-11T07:12:33.660 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:33.661 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:11:57.119254+0000","modified":"2024-01-11T07:12:00.699363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/401484670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":401484670},{"type":"v1","addr":"172.21.15.7:6835","nonce":401484670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b7c066f30 msgr2=0x7f2b7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b7c066f30 0x7f2b7c0693e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2b940091e0 tx=0x7f2b94017000 comp rx=0 tx=0).stop 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b980cafb0 msgr2=0x7f2b9815cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b980cafb0 0x7f2b9815cc60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c590 tx=0x7f2b88007dd0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 shutdown_connections 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b7c066f30 0x7f2b7c0693e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b98158280 0x7f2b98155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b980b0b40 0x7f2b9816be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 --2- 172.21.15.7:0/2155405017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b980cafb0 0x7f2b9815cc60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 >> 172.21.15.7:0/2155405017 conn(0x7f2b98071100 msgr2=0x7f2b980ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 shutdown_connections 2024-01-11T07:12:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.663+0000 7f2ba038e700 1 -- 172.21.15.7:0/2155405017 wait complete. 2024-01-11T07:12:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2024-01-11T07:12:33.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:33.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.807+0000 7ff942122700 1 Processor -- start 2024-01-11T07:12:33.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.807+0000 7ff942122700 1 -- start start 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.808+0000 7ff942122700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff93c159f70 0x7ff93c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.808+0000 7ff942122700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c0680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.808+0000 7ff942122700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93c067c60 con 0x7ff93c159ba0 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.808+0000 7ff942122700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c06d730 con 0x7ff93c159f70 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.808+0000 7ff942122700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c069230 con 0x7ff93c0c9570 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.808+0000 7ff93affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff93c159f70 0x7ff93c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff93c159f70 0x7ff93c0c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46674/0 (socket says 172.21.15.7:46674) 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93affd700 1 -- 172.21.15.7:0/252513335 learned_addr learned my addr 172.21.15.7:0/252513335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93b7fe700 1 --1- 172.21.15.7:0/252513335 >> v1:172.21.15.7:6789/0 conn(0x7ff93c159ba0 0x7ff93c06d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57544/0 (socket says 172.21.15.7:57544) 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93bfff700 1 --2- 172.21.15.7:0/252513335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c0680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93affd700 1 -- 172.21.15.7:0/252513335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 msgr2=0x7ff93c0680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93affd700 1 --2- 172.21.15.7:0/252513335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c0680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93affd700 1 -- 172.21.15.7:0/252513335 >> v1:172.21.15.7:6789/0 conn(0x7ff93c159ba0 legacy=0x7ff93c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93affd700 1 -- 172.21.15.7:0/252513335 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c069510 con 0x7ff93c159f70 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93affd700 1 --2- 172.21.15.7:0/252513335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c9030 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff9300099b0 tx=0x7ff930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe4abd7b40bb04d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93a7fc700 1 -- 172.21.15.7:0/252513335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000b860 con 0x7ff93c159f70 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93a7fc700 1 -- 172.21.15.7:0/252513335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93000b9c0 con 0x7ff93c159f70 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.809+0000 7ff93a7fc700 1 -- 172.21.15.7:0/252513335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000bd60 con 0x7ff93c159f70 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.810+0000 7ff942122700 1 -- 172.21.15.7:0/252513335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 msgr2=0x7ff93c0c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.810+0000 7ff942122700 1 --2- 172.21.15.7:0/252513335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c9030 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff9300099b0 tx=0x7ff930009250 comp rx=0 tx=0).stop 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.810+0000 7ff942122700 1 -- 172.21.15.7:0/252513335 shutdown_connections 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.810+0000 7ff942122700 1 --2- 172.21.15.7:0/252513335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c0680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.810+0000 7ff942122700 1 --2- 172.21.15.7:0/252513335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c9030 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.810+0000 7ff942122700 1 -- 172.21.15.7:0/252513335 >> 172.21.15.7:0/252513335 conn(0x7ff93c06ae00 msgr2=0x7ff93c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.810+0000 7ff942122700 1 -- 172.21.15.7:0/252513335 shutdown_connections 2024-01-11T07:12:33.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.810+0000 7ff942122700 1 -- 172.21.15.7:0/252513335 wait complete. 2024-01-11T07:12:33.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.811+0000 7ff942122700 1 Processor -- start 2024-01-11T07:12:33.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.811+0000 7ff942122700 1 -- start start 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff942122700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff942122700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c159ba0 0x7ff93c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff942122700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff942122700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c155bf0 con 0x7ff93c159f70 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff942122700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0b27a0 con 0x7ff93c159ba0 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff942122700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c1552c0 con 0x7ff93c0c9570 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff93bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff93bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46676/0 (socket says 172.21.15.7:46676) 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff93bfff700 1 -- 172.21.15.7:0/220715120 learned_addr learned my addr 172.21.15.7:0/220715120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff93b7fe700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.812+0000 7ff93affd700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c159ba0 0x7ff93c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff93bfff700 1 -- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 msgr2=0x7ff93c156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff93bfff700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff93bfff700 1 -- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c159ba0 msgr2=0x7ff93c154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff93bfff700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c159ba0 0x7ff93c154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff93bfff700 1 -- 172.21.15.7:0/220715120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300094f0 con 0x7ff93c159f70 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff93b7fe700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff93affd700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c159ba0 0x7ff93c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff93bfff700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c3790 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff934009fa0 tx=0x7ff934009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff9237fe700 1 -- 172.21.15.7:0/220715120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93400ed20 con 0x7ff93c159f70 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff9237fe700 1 -- 172.21.15.7:0/220715120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93400ee80 con 0x7ff93c159f70 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c1555a0 con 0x7ff93c159f70 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff9237fe700 1 -- 172.21.15.7:0/220715120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff934014670 con 0x7ff93c159f70 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.813+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c154980 con 0x7ff93c159f70 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.814+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c155d60 con 0x7ff93c159f70 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.814+0000 7ff9237fe700 1 -- 172.21.15.7:0/220715120 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff9340149e0 con 0x7ff93c159f70 2024-01-11T07:12:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.815+0000 7ff9237fe700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff928066f30 0x7ff9280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:33.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.815+0000 7ff9237fe700 1 -- 172.21.15.7:0/220715120 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff934084d30 con 0x7ff93c159f70 2024-01-11T07:12:33.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.815+0000 7ff93b7fe700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff928066f30 0x7ff9280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:33.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.816+0000 7ff93b7fe700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff928066f30 0x7ff9280693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff92c007fd0 tx=0x7ff92c007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:33.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:33.819+0000 7ff9237fe700 1 -- 172.21.15.7:0/220715120 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff934051fa0 con 0x7ff93c159f70 2024-01-11T07:12:34.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.025+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff93c04c3c0 con 0x7ff93c159f70 2024-01-11T07:12:34.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.026+0000 7ff9237fe700 1 -- 172.21.15.7:0/220715120 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+15309 (secure 0 0 0) 0x7ff9340555c0 con 0x7ff93c159f70 2024-01-11T07:12:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:34.026 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":108,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:28.332197+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:11:55.626974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-11T07:11:56.083170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff928066f30 msgr2=0x7ff9280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff928066f30 0x7ff9280693e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff92c007fd0 tx=0x7ff92c007e30 comp rx=0 tx=0).stop 2024-01-11T07:12:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 msgr2=0x7ff93c0c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c3790 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff934009fa0 tx=0x7ff934009900 comp rx=0 tx=0).stop 2024-01-11T07:12:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 shutdown_connections 2024-01-11T07:12:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff928066f30 0x7ff9280693e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff93c159f70 0x7ff93c0c3790 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff93c0c9570 0x7ff93c156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 --2- 172.21.15.7:0/220715120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff93c159ba0 0x7ff93c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 >> 172.21.15.7:0/220715120 conn(0x7ff93c06ae00 msgr2=0x7ff93c0c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 shutdown_connections 2024-01-11T07:12:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.028+0000 7ff942122700 1 -- 172.21.15.7:0/220715120 wait complete. 2024-01-11T07:12:34.042 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:12:34.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.176+0000 7fc1b882a700 1 Processor -- start 2024-01-11T07:12:34.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.176+0000 7fc1b882a700 1 -- start start 2024-01-11T07:12:34.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1b882a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 0x7fc1b40cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1b882a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1b415d920 0x7fc1b415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1b882a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b415e2d0 con 0x7fc1b415a2c0 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1b882a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b406ac30 con 0x7fc1b415d920 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1b882a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40ac770 con 0x7fc1b40ac3a0 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1abfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1b415d920 0x7fc1b415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1abfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1b415d920 0x7fc1b415dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46682/0 (socket says 172.21.15.7:46682) 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1abfff700 1 -- 172.21.15.7:0/2491777908 learned_addr learned my addr 172.21.15.7:0/2491777908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1b259c700 1 --2- 172.21.15.7:0/2491777908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 0x7fc1b40cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1abfff700 1 -- 172.21.15.7:0/2491777908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 msgr2=0x7fc1b40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1abfff700 1 --2- 172.21.15.7:0/2491777908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 0x7fc1b40cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1abfff700 1 -- 172.21.15.7:0/2491777908 >> v1:172.21.15.113:6789/0 conn(0x7fc1b415a2c0 legacy=0x7fc1b415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.177+0000 7fc1abfff700 1 -- 172.21.15.7:0/2491777908 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b406a300 con 0x7fc1b415d920 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1abfff700 1 --2- 172.21.15.7:0/2491777908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415d920 0x7fc1b415dd90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0099b0 tx=0x7fc19c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390464b63acf3e0a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b1d9b700 1 -- 172.21.15.7:0/2491777908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c009ea0 con 0x7fc1b415d920 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b1d9b700 1 -- 172.21.15.7:0/2491777908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00b860 con 0x7fc1b415d920 2024-01-11T07:12:34.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b1d9b700 1 -- 172.21.15.7:0/2491777908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00bc00 con 0x7fc1b415d920 2024-01-11T07:12:34.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b882a700 1 -- 172.21.15.7:0/2491777908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415d920 msgr2=0x7fc1b415dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b882a700 1 --2- 172.21.15.7:0/2491777908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415d920 0x7fc1b415dd90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0099b0 tx=0x7fc19c009250 comp rx=0 tx=0).stop 2024-01-11T07:12:34.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b882a700 1 -- 172.21.15.7:0/2491777908 shutdown_connections 2024-01-11T07:12:34.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b882a700 1 --2- 172.21.15.7:0/2491777908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 0x7fc1b40cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b882a700 1 --2- 172.21.15.7:0/2491777908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415d920 0x7fc1b415dd90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.178+0000 7fc1b882a700 1 -- 172.21.15.7:0/2491777908 >> 172.21.15.7:0/2491777908 conn(0x7fc1b40cd090 msgr2=0x7fc1b40cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:34.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.179+0000 7fc1b882a700 1 -- 172.21.15.7:0/2491777908 shutdown_connections 2024-01-11T07:12:34.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.179+0000 7fc1b882a700 1 -- 172.21.15.7:0/2491777908 wait complete. 2024-01-11T07:12:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.180+0000 7fc1b882a700 1 Processor -- start 2024-01-11T07:12:34.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.180+0000 7fc1b882a700 1 -- start start 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.180+0000 7fc1b882a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 0x7fc1b406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.180+0000 7fc1b882a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415a2c0 0x7fc1b4142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.180+0000 7fc1b882a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b415d920 0x7fc1b416d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.180+0000 7fc1b882a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b406edf0 con 0x7fc1b415a2c0 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.180+0000 7fc1b882a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b406e960 con 0x7fc1b415d920 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.180+0000 7fc1b882a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b406dd50 con 0x7fc1b40ac3a0 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415a2c0 0x7fc1b4142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415a2c0 0x7fc1b4142d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46688/0 (socket says 172.21.15.7:46688) 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 -- 172.21.15.7:0/4226168989 learned_addr learned my addr 172.21.15.7:0/4226168989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1b259c700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 0x7fc1b406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1b2d9d700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b415d920 0x7fc1b416d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 -- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 msgr2=0x7fc1b406d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 0x7fc1b406d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 -- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b415d920 msgr2=0x7fc1b416d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b415d920 0x7fc1b416d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 -- 172.21.15.7:0/4226168989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0094f0 con 0x7fc1b415a2c0 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1abfff700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415a2c0 0x7fc1b4142d50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc1b406fdf0 tx=0x7fc19c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1aa7fc700 1 -- 172.21.15.7:0/4226168989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c000d10 con 0x7fc1b415a2c0 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1aa7fc700 1 -- 172.21.15.7:0/4226168989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00b9d0 con 0x7fc1b415a2c0 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b406e030 con 0x7fc1b415a2c0 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1aa7fc700 1 -- 172.21.15.7:0/4226168989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c025420 con 0x7fc1b415a2c0 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.181+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b406e5f0 con 0x7fc1b415a2c0 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.182+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b406ead0 con 0x7fc1b415a2c0 2024-01-11T07:12:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.183+0000 7fc1aa7fc700 1 -- 172.21.15.7:0/4226168989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc19c023070 con 0x7fc1b415a2c0 2024-01-11T07:12:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.183+0000 7fc1aa7fc700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc194066f30 0x7fc1940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.183+0000 7fc1aa7fc700 1 -- 172.21.15.7:0/4226168989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc19c085b50 con 0x7fc1b415a2c0 2024-01-11T07:12:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.184+0000 7fc1b259c700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc194066f30 0x7fc1940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.184+0000 7fc1b259c700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc194066f30 0x7fc1940693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009d10 tx=0x7fc1a40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:34.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.188+0000 7fc1aa7fc700 1 -- 172.21.15.7:0/4226168989 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc19c052d60 con 0x7fc1b415a2c0 2024-01-11T07:12:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.440+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc1b404c3c0 con 0x7fc1b415a2c0 2024-01-11T07:12:34.441 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:12:34.441+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:12:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.446+0000 7fc1aa7fc700 1 -- 172.21.15.7:0/4226168989 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc19c02a090 con 0x7fc1b415a2c0 2024-01-11T07:12:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc194066f30 msgr2=0x7fc1940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc194066f30 0x7fc1940693e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009d10 tx=0x7fc1a40091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415a2c0 msgr2=0x7fc1b4142d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415a2c0 0x7fc1b4142d50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc1b406fdf0 tx=0x7fc19c017000 comp rx=0 tx=0).stop 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 shutdown_connections 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc194066f30 0x7fc1940693e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1b415d920 0x7fc1b416d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1b40ac3a0 0x7fc1b406d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 --2- 172.21.15.7:0/4226168989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1b415a2c0 0x7fc1b4142d50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.449+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 >> 172.21.15.7:0/4226168989 conn(0x7fc1b40cd090 msgr2=0x7fc1b4157f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.450+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 shutdown_connections 2024-01-11T07:12:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.450+0000 7fc1b882a700 1 -- 172.21.15.7:0/4226168989 wait complete. 2024-01-11T07:12:34.451 INFO:tasks.ceph.mds.a.smithi007.stderr: -9263> 2024-01-11T07:11:59.666+0000 7f1966df0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:12:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:12:34.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:12:34.483 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:12:34.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.597+0000 7f583b4a3700 1 Processor -- start 2024-01-11T07:12:34.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.598+0000 7f583b4a3700 1 -- start start 2024-01-11T07:12:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.598+0000 7f583b4a3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58340add00 0x7f5834158c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583b4a3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58340b35e0 0x7f58340b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583b4a3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340cee70 con 0x7f58340b0430 2024-01-11T07:12:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583b4a3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58340cfa10 con 0x7f58340add00 2024-01-11T07:12:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583b4a3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58340ce840 con 0x7f58340b35e0 2024-01-11T07:12:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583923f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58340add00 0x7f5834158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583923f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58340add00 0x7f5834158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46710/0 (socket says 172.21.15.7:46710) 2024-01-11T07:12:34.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f5839a40700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58340b35e0 0x7f58340b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583923f700 1 -- 172.21.15.7:0/203992175 learned_addr learned my addr 172.21.15.7:0/203992175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583923f700 1 -- 172.21.15.7:0/203992175 >> v2:172.21.15.113:3300/0 conn(0x7f58340b35e0 msgr2=0x7f58340b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583923f700 1 --2- 172.21.15.7:0/203992175 >> v2:172.21.15.113:3300/0 conn(0x7f58340b35e0 0x7f58340b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583923f700 1 -- 172.21.15.7:0/203992175 >> v1:172.21.15.113:6789/0 conn(0x7f58340b0430 legacy=0x7f58340cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583923f700 1 -- 172.21.15.7:0/203992175 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340b08b0 con 0x7f58340add00 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f5839a40700 1 --2- 172.21.15.7:0/203992175 >> v2:172.21.15.113:3300/0 conn(0x7f58340b35e0 0x7f58340b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f583923f700 1 --2- 172.21.15.7:0/203992175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340add00 0x7f5834158c80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f583415b610 tx=0x7f5830009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9589ecb5ce01f38b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.599+0000 7f582bfff700 1 -- 172.21.15.7:0/203992175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830009e70 con 0x7f58340add00 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f582bfff700 1 -- 172.21.15.7:0/203992175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583000b860 con 0x7f58340add00 2024-01-11T07:12:34.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f582bfff700 1 -- 172.21.15.7:0/203992175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583000bc00 con 0x7f58340add00 2024-01-11T07:12:34.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f583b4a3700 1 -- 172.21.15.7:0/203992175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340add00 msgr2=0x7f5834158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f583b4a3700 1 --2- 172.21.15.7:0/203992175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340add00 0x7f5834158c80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f583415b610 tx=0x7f5830009250 comp rx=0 tx=0).stop 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f583b4a3700 1 -- 172.21.15.7:0/203992175 shutdown_connections 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f583b4a3700 1 --2- 172.21.15.7:0/203992175 >> v2:172.21.15.113:3300/0 conn(0x7f58340b35e0 0x7f58340b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f583b4a3700 1 --2- 172.21.15.7:0/203992175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340add00 0x7f5834158c80 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f583b4a3700 1 -- 172.21.15.7:0/203992175 >> 172.21.15.7:0/203992175 conn(0x7f5834071110 msgr2=0x7f583406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f583b4a3700 1 -- 172.21.15.7:0/203992175 shutdown_connections 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.600+0000 7f583b4a3700 1 -- 172.21.15.7:0/203992175 wait complete. 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.601+0000 7f583b4a3700 1 Processor -- start 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.601+0000 7f583b4a3700 1 -- start start 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.602+0000 7f583b4a3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58340add00 0x7f583415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.602+0000 7f583b4a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340b0430 0x7f58340cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.602+0000 7f583b4a3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58340b35e0 0x7f583414a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.602+0000 7f583b4a3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58340cc620 con 0x7f58340b0430 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.602+0000 7f583b4a3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58340cd0f0 con 0x7f58340add00 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.602+0000 7f583b4a3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58340cc850 con 0x7f58340b35e0 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.602+0000 7f5838a3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340b0430 0x7f58340cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5838a3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340b0430 0x7f58340cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46720/0 (socket says 172.21.15.7:46720) 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5838a3e700 1 -- 172.21.15.7:0/1250927844 learned_addr learned my addr 172.21.15.7:0/1250927844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f583923f700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58340add00 0x7f583415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5839a40700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58340b35e0 0x7f583414a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5838a3e700 1 -- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58340b35e0 msgr2=0x7f583414a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5838a3e700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58340b35e0 0x7f583414a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5838a3e700 1 -- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58340add00 msgr2=0x7f583415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5838a3e700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58340add00 0x7f583415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5838a3e700 1 -- 172.21.15.7:0/1250927844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c009a20 con 0x7f58340b0430 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f583923f700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58340add00 0x7f583415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5839a40700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58340b35e0 0x7f583414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f5838a3e700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340b0430 0x7f58340cc0e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f582400c590 tx=0x7f582400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f58297fa700 1 -- 172.21.15.7:0/1250927844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824012630 con 0x7f58340b0430 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f58297fa700 1 -- 172.21.15.7:0/1250927844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5824012790 con 0x7f58340b0430 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58300094f0 con 0x7f58340b0430 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f58297fa700 1 -- 172.21.15.7:0/1250927844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824012b50 con 0x7f58340b0430 2024-01-11T07:12:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.603+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58340cce10 con 0x7f58340b0430 2024-01-11T07:12:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.604+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583404c3c0 con 0x7f58340b0430 2024-01-11T07:12:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.605+0000 7f58297fa700 1 -- 172.21.15.7:0/1250927844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f582401b400 con 0x7f58340b0430 2024-01-11T07:12:34.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.605+0000 7f58297fa700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.605+0000 7f58297fa700 1 -- 172.21.15.7:0/1250927844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f58240841d0 con 0x7f58340b0430 2024-01-11T07:12:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.605+0000 7f583923f700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:34.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.606+0000 7f583923f700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5818066f30 0x7f58180693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f583415b610 tx=0x7f5830017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:34.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.610+0000 7f58297fa700 1 -- 172.21.15.7:0/1250927844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58240513f0 con 0x7f58340b0430 2024-01-11T07:12:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:34.860+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f58340b0b80 con 0x7f58340b0430 2024-01-11T07:12:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.446+0000 7f58297fa700 1 -- 172.21.15.7:0/1250927844 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5824054a10 con 0x7f58340b0430 2024-01-11T07:12:35.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5818066f30 msgr2=0x7f58180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:35.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5818066f30 0x7f58180693e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f583415b610 tx=0x7f5830017000 comp rx=0 tx=0).stop 2024-01-11T07:12:35.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340b0430 msgr2=0x7f58340cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340b0430 0x7f58340cc0e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f582400c590 tx=0x7f582400c950 comp rx=0 tx=0).stop 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 shutdown_connections 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58340b35e0 0x7f583414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58340add00 0x7f583415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 --2- 172.21.15.7:0/1250927844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340b0430 0x7f58340cc0e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 >> 172.21.15.7:0/1250927844 conn(0x7f5834071110 msgr2=0x7f58340b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 shutdown_connections 2024-01-11T07:12:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.449+0000 7f583b4a3700 1 -- 172.21.15.7:0/1250927844 wait complete. 2024-01-11T07:12:35.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:12:35.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.648+0000 7f0d7b9d0700 1 Processor -- start 2024-01-11T07:12:35.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.648+0000 7f0d7b9d0700 1 -- start start 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d7b9d0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406e260 0x7f0d74159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d7b9d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d7406a270 0x7f0d74159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d7b9d0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7415b0a0 con 0x7f0d74069ea0 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d7b9d0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d740c9860 con 0x7f0d7406a270 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d7b9d0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740c8e80 con 0x7f0d7406e260 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d79f6d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d7406a270 0x7f0d74159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d79f6d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d7406a270 0x7f0d74159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46728/0 (socket says 172.21.15.7:46728) 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d79f6d700 1 -- 172.21.15.7:0/1660678382 learned_addr learned my addr 172.21.15.7:0/1660678382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d78f6b700 1 --2- 172.21.15.7:0/1660678382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406e260 0x7f0d74159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d79f6d700 1 -- 172.21.15.7:0/1660678382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406e260 msgr2=0x7f0d74159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:35.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d79f6d700 1 --2- 172.21.15.7:0/1660678382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406e260 0x7f0d74159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d79f6d700 1 -- 172.21.15.7:0/1660678382 >> v1:172.21.15.113:6789/0 conn(0x7f0d74069ea0 legacy=0x7f0d7406db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d79f6d700 1 -- 172.21.15.7:0/1660678382 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7406e6f0 con 0x7f0d7406a270 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d79f6d700 1 --2- 172.21.15.7:0/1660678382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406a270 0x7f0d74159ff0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00d1a0 tx=0x7f0d6c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692bb3f043b7877f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.649+0000 7f0d6bfff700 1 -- 172.21.15.7:0/1660678382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c010070 con 0x7f0d7406a270 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d6bfff700 1 -- 172.21.15.7:0/1660678382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c00de10 con 0x7f0d7406a270 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d6bfff700 1 -- 172.21.15.7:0/1660678382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c00b320 con 0x7f0d7406a270 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1660678382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406a270 msgr2=0x7f0d74159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1660678382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406a270 0x7f0d74159ff0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00d1a0 tx=0x7f0d6c00ba50 comp rx=0 tx=0).stop 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1660678382 shutdown_connections 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1660678382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406a270 0x7f0d74159ff0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1660678382 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406e260 0x7f0d74159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1660678382 >> 172.21.15.7:0/1660678382 conn(0x7f0d740680a0 msgr2=0x7f0d740cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:35.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1660678382 shutdown_connections 2024-01-11T07:12:35.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.650+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1660678382 wait complete. 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.651+0000 7f0d7b9d0700 1 Processor -- start 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.652+0000 7f0d7b9d0700 1 -- start start 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.652+0000 7f0d7b9d0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d74069ea0 0x7f0d740ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.652+0000 7f0d7b9d0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406a270 0x7f0d740ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d7b9d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406e260 0x7f0d7414dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d7b9d0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740ae7d0 con 0x7f0d7406e260 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d7b9d0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740aeab0 con 0x7f0d74069ea0 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d7b9d0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740aed90 con 0x7f0d7406a270 2024-01-11T07:12:35.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406e260 0x7f0d7414dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406e260 0x7f0d7414dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46742/0 (socket says 172.21.15.7:46742) 2024-01-11T07:12:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 -- 172.21.15.7:0/1032034849 learned_addr learned my addr 172.21.15.7:0/1032034849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d7976c700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d74069ea0 0x7f0d740ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d78f6b700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406a270 0x7f0d740ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 -- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406a270 msgr2=0x7f0d740ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406a270 0x7f0d740ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 -- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d74069ea0 msgr2=0x7f0d740ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d74069ea0 0x7f0d740ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 -- 172.21.15.7:0/1032034849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c00bcf0 con 0x7f0d7406e260 2024-01-11T07:12:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d79f6d700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406e260 0x7f0d7414dc70 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0d740cd410 tx=0x7f0d6c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d697fa700 1 -- 172.21.15.7:0/1032034849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c009a30 con 0x7f0d7406e260 2024-01-11T07:12:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d697fa700 1 -- 172.21.15.7:0/1032034849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c009b90 con 0x7f0d7406e260 2024-01-11T07:12:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d740af070 con 0x7f0d7406e260 2024-01-11T07:12:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.653+0000 7f0d697fa700 1 -- 172.21.15.7:0/1032034849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c01e440 con 0x7f0d7406e260 2024-01-11T07:12:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.654+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d740abe30 con 0x7f0d7406e260 2024-01-11T07:12:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.654+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d740c18a0 con 0x7f0d7406e260 2024-01-11T07:12:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.655+0000 7f0d697fa700 1 -- 172.21.15.7:0/1032034849 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0d6c003040 con 0x7f0d7406e260 2024-01-11T07:12:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.655+0000 7f0d697fa700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d58066f30 0x7f0d580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.655+0000 7f0d697fa700 1 -- 172.21.15.7:0/1032034849 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f0d6c085de0 con 0x7f0d7406e260 2024-01-11T07:12:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.656+0000 7f0d7976c700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d58066f30 0x7f0d580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.656+0000 7f0d7976c700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d58066f30 0x7f0d580693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0d70003a80 tx=0x7f0d70007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:35.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.660+0000 7f0d697fa700 1 -- 172.21.15.7:0/1032034849 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d6c052fe0 con 0x7f0d7406e260 2024-01-11T07:12:35.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:35.865+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0d740c1710 con 0x7f0d7406e260 2024-01-11T07:12:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.450+0000 7f0d697fa700 1 -- 172.21.15.7:0/1032034849 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0d6c056600 con 0x7f0d7406e260 2024-01-11T07:12:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.453+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d58066f30 msgr2=0x7f0d580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.453+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d58066f30 0x7f0d580693e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0d70003a80 tx=0x7f0d70007ec0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.453+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406e260 msgr2=0x7f0d7414dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.453+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406e260 0x7f0d7414dc70 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0d740cd410 tx=0x7f0d6c01a000 comp rx=0 tx=0).stop 2024-01-11T07:12:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.453+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 shutdown_connections 2024-01-11T07:12:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.453+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0d58066f30 0x7f0d580693e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.453+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d7406e260 0x7f0d7414dc70 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.453+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d74069ea0 0x7f0d740ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.454+0000 7f0d7b9d0700 1 --2- 172.21.15.7:0/1032034849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d7406a270 0x7f0d740ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.454+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 >> 172.21.15.7:0/1032034849 conn(0x7f0d740680a0 msgr2=0x7f0d74157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.454+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 shutdown_connections 2024-01-11T07:12:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.454+0000 7f0d7b9d0700 1 -- 172.21.15.7:0/1032034849 wait complete. 2024-01-11T07:12:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:12:36.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:12:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.609+0000 7fdda2f4c700 1 Processor -- start 2024-01-11T07:12:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.609+0000 7fdda2f4c700 1 -- start start 2024-01-11T07:12:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.610+0000 7fdda2f4c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d100 0x7fdd9c069540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.610+0000 7fdda2f4c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd9c15d4d0 0x7fdd9c15d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.610+0000 7fdda2f4c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9c069f90 con 0x7fdd9c069a80 2024-01-11T07:12:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.610+0000 7fdda2f4c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9c06a270 con 0x7fdd9c15d4d0 2024-01-11T07:12:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.610+0000 7fdda2f4c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c06a550 con 0x7fdd9c15d100 2024-01-11T07:12:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.610+0000 7fdda14e9700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdd9c069a80 0x7fdd9c15a340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57566/0 (socket says 172.21.15.7:57566) 2024-01-11T07:12:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdda14e9700 1 -- 172.21.15.7:0/3462461654 learned_addr learned my addr 172.21.15.7:0/3462461654 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdd9bfff700 1 --2- 172.21.15.7:0/3462461654 >> v2:172.21.15.113:3300/0 conn(0x7fdd9c15d4d0 0x7fdd9c15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdda0ce8700 1 --2- 172.21.15.7:0/3462461654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d100 0x7fdd9c069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdd9b7fe700 1 -- 172.21.15.7:0/3462461654 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408294529 0 0) 0x7fdd9c069f90 con 0x7fdd9c069a80 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdd9b7fe700 1 -- 172.21.15.7:0/3462461654 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd8c003570 con 0x7fdd9c069a80 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdda0ce8700 1 -- 172.21.15.7:0/3462461654 >> v2:172.21.15.113:3300/0 conn(0x7fdd9c15d4d0 msgr2=0x7fdd9c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdda0ce8700 1 --2- 172.21.15.7:0/3462461654 >> v2:172.21.15.113:3300/0 conn(0x7fdd9c15d4d0 0x7fdd9c15d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdda0ce8700 1 -- 172.21.15.7:0/3462461654 >> v1:172.21.15.7:6789/0 conn(0x7fdd9c069a80 legacy=0x7fdd9c15a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdda0ce8700 1 -- 172.21.15.7:0/3462461654 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c06a830 con 0x7fdd9c15d100 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdda0ce8700 1 --2- 172.21.15.7:0/3462461654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d100 0x7fdd9c069540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd900098e0 tx=0x7fdd900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d55d228c55acf21 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.611+0000 7fdd9b7fe700 1 -- 172.21.15.7:0/3462461654 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90011630 con 0x7fdd9c15d100 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdd9b7fe700 1 -- 172.21.15.7:0/3462461654 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd90011790 con 0x7fdd9c15d100 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdd9b7fe700 1 -- 172.21.15.7:0/3462461654 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90011b30 con 0x7fdd9c15d100 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdda2f4c700 1 -- 172.21.15.7:0/3462461654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d100 msgr2=0x7fdd9c069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/3462461654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d100 0x7fdd9c069540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd900098e0 tx=0x7fdd900091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdda2f4c700 1 -- 172.21.15.7:0/3462461654 shutdown_connections 2024-01-11T07:12:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/3462461654 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d100 0x7fdd9c069540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/3462461654 >> v2:172.21.15.113:3300/0 conn(0x7fdd9c15d4d0 0x7fdd9c15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdda2f4c700 1 -- 172.21.15.7:0/3462461654 >> 172.21.15.7:0/3462461654 conn(0x7fdd9c0680b0 msgr2=0x7fdd9c0cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdda2f4c700 1 -- 172.21.15.7:0/3462461654 shutdown_connections 2024-01-11T07:12:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.612+0000 7fdda2f4c700 1 -- 172.21.15.7:0/3462461654 wait complete. 2024-01-11T07:12:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.613+0000 7fdda2f4c700 1 Processor -- start 2024-01-11T07:12:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.614+0000 7fdda2f4c700 1 -- start start 2024-01-11T07:12:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.614+0000 7fdda2f4c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c069a80 0x7fdd9c0c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.614+0000 7fdda2f4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd9c15d100 0x7fdd9c0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.614+0000 7fdda2f4c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d4d0 0x7fdd9c13c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.614+0000 7fdda2f4c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c16e630 con 0x7fdd9c15d100 2024-01-11T07:12:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.614+0000 7fdda2f4c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c16da20 con 0x7fdd9c069a80 2024-01-11T07:12:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.614+0000 7fdda2f4c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c16dd00 con 0x7fdd9c15d4d0 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd9c15d100 0x7fdd9c0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd9c15d100 0x7fdd9c0c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46758/0 (socket says 172.21.15.7:46758) 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 -- 172.21.15.7:0/1366221307 learned_addr learned my addr 172.21.15.7:0/1366221307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdda0ce8700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c069a80 0x7fdd9c0c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdda14e9700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d4d0 0x7fdd9c13c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 -- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d4d0 msgr2=0x7fdd9c13c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d4d0 0x7fdd9c13c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 -- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c069a80 msgr2=0x7fdd9c0c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c069a80 0x7fdd9c0c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 -- 172.21.15.7:0/1366221307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd90009420 con 0x7fdd9c15d100 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdda0ce8700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c069a80 0x7fdd9c0c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd9bfff700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd9c15d100 0x7fdd9c0c7f30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800e900 tx=0x7fdd880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd98ff9700 1 -- 172.21.15.7:0/1366221307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88013680 con 0x7fdd9c15d100 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd98ff9700 1 -- 172.21.15.7:0/1366221307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd880137e0 con 0x7fdd9c15d100 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9c16dfe0 con 0x7fdd9c15d100 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdd98ff9700 1 -- 172.21.15.7:0/1366221307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88013ba0 con 0x7fdd9c15d100 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.615+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c134e60 con 0x7fdd9c15d100 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.616+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9c04c3c0 con 0x7fdd9c15d100 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.617+0000 7fdd98ff9700 1 -- 172.21.15.7:0/1366221307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdd8800c460 con 0x7fdd9c15d100 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.617+0000 7fdd98ff9700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd80066f30 0x7fdd800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.617+0000 7fdd98ff9700 1 -- 172.21.15.7:0/1366221307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fdd88084920 con 0x7fdd9c15d100 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.617+0000 7fdda0ce8700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd80066f30 0x7fdd800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.618+0000 7fdda0ce8700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd80066f30 0x7fdd800693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdd900091e0 tx=0x7fdd90017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.622+0000 7fdd98ff9700 1 -- 172.21.15.7:0/1366221307 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd88051cf0 con 0x7fdd9c15d100 2024-01-11T07:12:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:36.826+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdd9c06ab10 con 0x7fdd9c15d100 2024-01-11T07:12:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.453+0000 7fdd98ff9700 1 -- 172.21.15.7:0/1366221307 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdd880170d0 con 0x7fdd9c15d100 2024-01-11T07:12:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd80066f30 msgr2=0x7fdd800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd80066f30 0x7fdd800693e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdd900091e0 tx=0x7fdd90017000 comp rx=0 tx=0).stop 2024-01-11T07:12:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd9c15d100 msgr2=0x7fdd9c0c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd9c15d100 0x7fdd9c0c7f30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800e900 tx=0x7fdd880091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 shutdown_connections 2024-01-11T07:12:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd80066f30 0x7fdd800693e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c15d4d0 0x7fdd9c13c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c069a80 0x7fdd9c0c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 --2- 172.21.15.7:0/1366221307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd9c15d100 0x7fdd9c0c7f30 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 >> 172.21.15.7:0/1366221307 conn(0x7fdd9c0680b0 msgr2=0x7fdd9c06e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 shutdown_connections 2024-01-11T07:12:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.460+0000 7fdda2f4c700 1 -- 172.21.15.7:0/1366221307 wait complete. 2024-01-11T07:12:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:12:37.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:12:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.615+0000 7f7f05724700 1 Processor -- start 2024-01-11T07:12:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.615+0000 7f7f05724700 1 -- start start 2024-01-11T07:12:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7f05724700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0015db90 0x7f7f0015df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7f05724700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f000c3690 0x7f7f0006e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7f05724700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0015e4a0 con 0x7f7f0006ec70 2024-01-11T07:12:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7f05724700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0006d3b0 con 0x7f7f000c3690 2024-01-11T07:12:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7f05724700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0006d690 con 0x7f7f0015db90 2024-01-11T07:12:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7efeffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0015db90 0x7f7f0015df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7efe7fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f000c3690 0x7f7f0006e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7efe7fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f000c3690 0x7f7f0006e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:34838/0 (socket says 172.21.15.7:34838) 2024-01-11T07:12:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7eff7fe700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7f0006ec70 0x7f7f000c9010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:42508/0 (socket says 172.21.15.7:42508) 2024-01-11T07:12:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7efeffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0015db90 0x7f7f0015df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:35864/0 (socket says 172.21.15.7:35864) 2024-01-11T07:12:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.616+0000 7f7efe7fc700 1 -- 172.21.15.7:0/132877155 learned_addr learned my addr 172.21.15.7:0/132877155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efeffd700 1 -- 172.21.15.7:0/132877155 >> v2:172.21.15.113:3300/0 conn(0x7f7f000c3690 msgr2=0x7f7f0006e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efeffd700 1 --2- 172.21.15.7:0/132877155 >> v2:172.21.15.113:3300/0 conn(0x7f7f000c3690 0x7f7f0006e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efeffd700 1 -- 172.21.15.7:0/132877155 >> v1:172.21.15.113:6789/0 conn(0x7f7f0006ec70 legacy=0x7f7f000c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efeffd700 1 -- 172.21.15.7:0/132877155 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0006d970 con 0x7f7f0015db90 2024-01-11T07:12:37.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efdffb700 1 -- 172.21.15.7:0/132877155 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556229509 0 0) 0x7f7f0015e4a0 con 0x7f7f0006ec70 2024-01-11T07:12:37.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efe7fc700 1 --2- 172.21.15.7:0/132877155 >> v2:172.21.15.113:3300/0 conn(0x7f7f000c3690 0x7f7f0006e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:37.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efeffd700 1 --2- 172.21.15.7:0/132877155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0015db90 0x7f7f0015df60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00098e0 tx=0x7f7ef00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11cd02890d92800 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:37.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efdffb700 1 -- 172.21.15.7:0/132877155 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000b640 con 0x7f7f0015db90 2024-01-11T07:12:37.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efdffb700 1 -- 172.21.15.7:0/132877155 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef000b7a0 con 0x7f7f0015db90 2024-01-11T07:12:37.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.617+0000 7f7efdffb700 1 -- 172.21.15.7:0/132877155 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000bb40 con 0x7f7f0015db90 2024-01-11T07:12:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.618+0000 7f7f05724700 1 -- 172.21.15.7:0/132877155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0015db90 msgr2=0x7f7f0015df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.618+0000 7f7f05724700 1 --2- 172.21.15.7:0/132877155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0015db90 0x7f7f0015df60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00098e0 tx=0x7f7ef00091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.618+0000 7f7f05724700 1 -- 172.21.15.7:0/132877155 shutdown_connections 2024-01-11T07:12:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.618+0000 7f7f05724700 1 --2- 172.21.15.7:0/132877155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0015db90 0x7f7f0015df60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.618+0000 7f7f05724700 1 --2- 172.21.15.7:0/132877155 >> v2:172.21.15.113:3300/0 conn(0x7f7f000c3690 0x7f7f0006e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.618+0000 7f7f05724700 1 -- 172.21.15.7:0/132877155 >> 172.21.15.7:0/132877155 conn(0x7f7f000cd070 msgr2=0x7f7f000cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.618+0000 7f7f05724700 1 -- 172.21.15.7:0/132877155 shutdown_connections 2024-01-11T07:12:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.618+0000 7f7f05724700 1 -- 172.21.15.7:0/132877155 wait complete. 2024-01-11T07:12:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.619+0000 7f7f05724700 1 Processor -- start 2024-01-11T07:12:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.619+0000 7f7f05724700 1 -- start start 2024-01-11T07:12:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7f05724700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0006ec70 0x7f7f000c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7f05724700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c3690 0x7f7f0015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7f05724700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f0015db90 0x7f7f0016a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7f05724700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000c4660 con 0x7f7f000c3690 2024-01-11T07:12:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7f05724700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000c5130 con 0x7f7f0015db90 2024-01-11T07:12:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7f05724700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f000c6090 con 0x7f7f0006ec70 2024-01-11T07:12:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7efe7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c3690 0x7f7f0015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7efe7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c3690 0x7f7f0015cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52210/0 (socket says 172.21.15.7:52210) 2024-01-11T07:12:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7efe7fc700 1 -- 172.21.15.7:0/2061758577 learned_addr learned my addr 172.21.15.7:0/2061758577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7efeffd700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0006ec70 0x7f7f000c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.620+0000 7f7eff7fe700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f0015db90 0x7f7f0016a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7efe7fc700 1 -- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0006ec70 msgr2=0x7f7f000c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7efe7fc700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0006ec70 0x7f7f000c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7efe7fc700 1 -- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f0015db90 msgr2=0x7f7f0016a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7efe7fc700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f0015db90 0x7f7f0016a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7efe7fc700 1 -- 172.21.15.7:0/2061758577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee80094f0 con 0x7f7f000c3690 2024-01-11T07:12:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7efe7fc700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c3690 0x7f7f0015cc90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800ec50 tx=0x7f7ee8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7ef6ffd700 1 -- 172.21.15.7:0/2061758577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800c850 con 0x7f7f000c3690 2024-01-11T07:12:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7ef6ffd700 1 -- 172.21.15.7:0/2061758577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee800c9b0 con 0x7f7f000c3690 2024-01-11T07:12:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef0009420 con 0x7f7f000c3690 2024-01-11T07:12:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7ef6ffd700 1 -- 172.21.15.7:0/2061758577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800cd30 con 0x7f7f000c3690 2024-01-11T07:12:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.621+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f000c4890 con 0x7f7f000c3690 2024-01-11T07:12:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.622+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f000c6200 con 0x7f7f000c3690 2024-01-11T07:12:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.623+0000 7f7ef6ffd700 1 -- 172.21.15.7:0/2061758577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7ee8013680 con 0x7f7f000c3690 2024-01-11T07:12:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.623+0000 7f7ef6ffd700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ee0066f30 0x7f7ee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.623+0000 7f7ef6ffd700 1 -- 172.21.15.7:0/2061758577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7ee80844e0 con 0x7f7f000c3690 2024-01-11T07:12:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.623+0000 7f7efeffd700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ee0066f30 0x7f7ee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.624+0000 7f7efeffd700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ee0066f30 0x7f7ee00693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00093f0 tx=0x7f7ef0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:37.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.628+0000 7f7ef6ffd700 1 -- 172.21.15.7:0/2061758577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ee8051a50 con 0x7f7f000c3690 2024-01-11T07:12:37.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.838+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7f0004c3c0 con 0x7f7f000c3690 2024-01-11T07:12:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.842+0000 7f7ef6ffd700 1 -- 172.21.15.7:0/2061758577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ee8055070 con 0x7f7f000c3690 2024-01-11T07:12:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ee0066f30 msgr2=0x7f7ee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ee0066f30 0x7f7ee00693e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00093f0 tx=0x7f7ef0017000 comp rx=0 tx=0).stop 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c3690 msgr2=0x7f7f0015cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c3690 0x7f7f0015cc90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800ec50 tx=0x7f7ee8009250 comp rx=0 tx=0).stop 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 shutdown_connections 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ee0066f30 0x7f7ee00693e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f0015db90 0x7f7f0016a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f0006ec70 0x7f7f000c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 --2- 172.21.15.7:0/2061758577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c3690 0x7f7f0015cc90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 >> 172.21.15.7:0/2061758577 conn(0x7f7f000cd070 msgr2=0x7f7f000698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 shutdown_connections 2024-01-11T07:12:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.844+0000 7f7f05724700 1 -- 172.21.15.7:0/2061758577 wait complete. 2024-01-11T07:12:37.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-01-11T07:12:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.991+0000 7fc989ee0700 1 Processor -- start 2024-01-11T07:12:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.991+0000 7fc989ee0700 1 -- start start 2024-01-11T07:12:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.991+0000 7fc989ee0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc984157c60 0x7fc98415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc989ee0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98415bdf0 con 0x7fc98406d2b0 2024-01-11T07:12:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc989ee0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98415c0d0 con 0x7fc984157890 2024-01-11T07:12:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc989ee0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9840c91c0 con 0x7fc984157c60 2024-01-11T07:12:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc982ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc984157c60 0x7fc98415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc983fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc98406d2b0 0x7fc98406ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47668/0 (socket says 172.21.15.7:47668) 2024-01-11T07:12:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc983fff700 1 -- 172.21.15.7:0/2172392221 learned_addr learned my addr 172.21.15.7:0/2172392221 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:37.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc982ffd700 1 -- 172.21.15.7:0/2172392221 >> v1:172.21.15.113:6789/0 conn(0x7fc984157890 legacy=0x7fc9840af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc982ffd700 1 -- 172.21.15.7:0/2172392221 >> v1:172.21.15.7:6789/0 conn(0x7fc98406d2b0 legacy=0x7fc98406ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc982ffd700 1 -- 172.21.15.7:0/2172392221 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98415c3b0 con 0x7fc984157c60 2024-01-11T07:12:37.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc9827fc700 1 -- 172.21.15.7:0/2172392221 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696241736 0 0) 0x7fc98415bdf0 con 0x7fc98406d2b0 2024-01-11T07:12:37.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc982ffd700 1 --2- 172.21.15.7:0/2172392221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157c60 0x7fc98415b6f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098e0 tx=0x7fc96c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21ecaec3ec071c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:37.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc9827fc700 1 -- 172.21.15.7:0/2172392221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c009e80 con 0x7fc984157c60 2024-01-11T07:12:37.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.992+0000 7fc9827fc700 1 -- 172.21.15.7:0/2172392221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00b860 con 0x7fc984157c60 2024-01-11T07:12:37.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.993+0000 7fc9827fc700 1 -- 172.21.15.7:0/2172392221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00bbe0 con 0x7fc984157c60 2024-01-11T07:12:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.993+0000 7fc989ee0700 1 -- 172.21.15.7:0/2172392221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157c60 msgr2=0x7fc98415b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.993+0000 7fc989ee0700 1 --2- 172.21.15.7:0/2172392221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157c60 0x7fc98415b6f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0098e0 tx=0x7fc96c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.993+0000 7fc989ee0700 1 -- 172.21.15.7:0/2172392221 shutdown_connections 2024-01-11T07:12:37.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.993+0000 7fc989ee0700 1 --2- 172.21.15.7:0/2172392221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157c60 0x7fc98415b6f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.993+0000 7fc989ee0700 1 -- 172.21.15.7:0/2172392221 >> 172.21.15.7:0/2172392221 conn(0x7fc9840ce9f0 msgr2=0x7fc9840cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.993+0000 7fc989ee0700 1 -- 172.21.15.7:0/2172392221 shutdown_connections 2024-01-11T07:12:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.993+0000 7fc989ee0700 1 -- 172.21.15.7:0/2172392221 wait complete. 2024-01-11T07:12:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.994+0000 7fc989ee0700 1 Processor -- start 2024-01-11T07:12:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc989ee0700 1 -- start start 2024-01-11T07:12:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc989ee0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc98406d2b0 0x7fc9840c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc989ee0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157890 0x7fc9840ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc989ee0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984157c60 0x7fc98416dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc989ee0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840c5270 con 0x7fc984157890 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc989ee0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840c4cb0 con 0x7fc984157c60 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc989ee0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840c4f90 con 0x7fc98406d2b0 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc982ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157890 0x7fc9840ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc982ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157890 0x7fc9840ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52238/0 (socket says 172.21.15.7:52238) 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.995+0000 7fc982ffd700 1 -- 172.21.15.7:0/3899557280 learned_addr learned my addr 172.21.15.7:0/3899557280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc983fff700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984157c60 0x7fc98416dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc9837fe700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc98406d2b0 0x7fc9840c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc982ffd700 1 -- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc98406d2b0 msgr2=0x7fc9840c2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc982ffd700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc98406d2b0 0x7fc9840c2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc982ffd700 1 -- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984157c60 msgr2=0x7fc98416dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc982ffd700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984157c60 0x7fc98416dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc982ffd700 1 -- 172.21.15.7:0/3899557280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c009420 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc983fff700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984157c60 0x7fc98416dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc982ffd700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157890 0x7fc9840ac350 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc9840ce710 tx=0x7fc96c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc97b7fe700 1 -- 172.21.15.7:0/3899557280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c01f070 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc97b7fe700 1 -- 172.21.15.7:0/3899557280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c011420 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc984067740 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc97b7fe700 1 -- 172.21.15.7:0/3899557280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00bbe0 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.996+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9840c49c0 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.997+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98404c3c0 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.998+0000 7fc97b7fe700 1 -- 172.21.15.7:0/3899557280 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc96c011a50 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.998+0000 7fc97b7fe700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc964066f30 0x7fc9640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.998+0000 7fc97b7fe700 1 -- 172.21.15.7:0/3899557280 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc96c0856f0 con 0x7fc984157890 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.998+0000 7fc9837fe700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc964066f30 0x7fc9640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:37.999+0000 7fc9837fe700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc964066f30 0x7fc9640693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc974007fd0 tx=0x7fc974009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.003+0000 7fc97b7fe700 1 -- 172.21.15.7:0/3899557280 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc96c052c60 con 0x7fc984157890 2024-01-11T07:12:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.290+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7fc98406d090 con 0x7fc984157890 2024-01-11T07:12:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.453+0000 7fc97b7fe700 1 -- 172.21.15.7:0/3899557280 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v287) v1 ==== 137+0+0 (secure 0 0 0) 0x7fc96c056280 con 0x7fc984157890 2024-01-11T07:12:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc964066f30 msgr2=0x7fc9640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc964066f30 0x7fc9640693e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc974007fd0 tx=0x7fc974009040 comp rx=0 tx=0).stop 2024-01-11T07:12:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157890 msgr2=0x7fc9840ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157890 0x7fc9840ac350 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc9840ce710 tx=0x7fc96c017000 comp rx=0 tx=0).stop 2024-01-11T07:12:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 shutdown_connections 2024-01-11T07:12:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc964066f30 0x7fc9640693e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc984157c60 0x7fc98416dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc98406d2b0 0x7fc9840c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 --2- 172.21.15.7:0/3899557280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc984157890 0x7fc9840ac350 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 >> 172.21.15.7:0/3899557280 conn(0x7fc9840ce9f0 msgr2=0x7fc9840cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 shutdown_connections 2024-01-11T07:12:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.455+0000 7fc989ee0700 1 -- 172.21.15.7:0/3899557280 wait complete. 2024-01-11T07:12:38.470 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-01-11T07:12:38.471 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-01-11T07:12:38.471 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2024-01-11T07:12:38.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-01-11T07:12:38.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.606+0000 7fa5d954a700 1 Processor -- start 2024-01-11T07:12:38.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.606+0000 7fa5d954a700 1 -- start start 2024-01-11T07:12:38.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.607+0000 7fa5d954a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5d415c330 0x7fa5d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.607+0000 7fa5d954a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40c87a0 con 0x7fa5d415b850 2024-01-11T07:12:38.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.607+0000 7fa5d954a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40c9cb0 con 0x7fa5d406e1b0 2024-01-11T07:12:38.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.607+0000 7fa5d954a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d4069280 con 0x7fa5d415c330 2024-01-11T07:12:38.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.607+0000 7fa5d2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa5d415b850 0x7fa5d415bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47678/0 (socket says 172.21.15.7:47678) 2024-01-11T07:12:38.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.607+0000 7fa5d2ffd700 1 -- 172.21.15.7:0/577616211 learned_addr learned my addr 172.21.15.7:0/577616211 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:38.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.607+0000 7fa5d27fc700 1 --2- 172.21.15.7:0/577616211 >> v2:172.21.15.113:3300/0 conn(0x7fa5d415c330 0x7fa5d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.607+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/577616211 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984009593 0 0) 0x7fa5d40c87a0 con 0x7fa5d415b850 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/577616211 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c0003570 con 0x7fa5d415b850 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/577616211 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394862477 0 0) 0x7fa5d40c9cb0 con 0x7fa5d406e1b0 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/577616211 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d40c87a0 con 0x7fa5d406e1b0 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/577616211 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3071152772 0 0) 0x7fa5c0003570 con 0x7fa5d415b850 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d27fc700 1 -- 172.21.15.7:0/577616211 >> v1:172.21.15.113:6789/0 conn(0x7fa5d406e1b0 legacy=0x7fa5d406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d27fc700 1 -- 172.21.15.7:0/577616211 >> v1:172.21.15.7:6789/0 conn(0x7fa5d415b850 legacy=0x7fa5d415bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d27fc700 1 -- 172.21.15.7:0/577616211 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4069560 con 0x7fa5d415c330 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d27fc700 1 --2- 172.21.15.7:0/577616211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d415c330 0x7fa5d4068bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0098e0 tx=0x7fa5bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b7b0fdecab2c5d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/577616211 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4001470 con 0x7fa5d415c330 2024-01-11T07:12:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/577616211 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc011420 con 0x7fa5d415c330 2024-01-11T07:12:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.608+0000 7fa5d1ffb700 1 -- 172.21.15.7:0/577616211 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0117a0 con 0x7fa5d415c330 2024-01-11T07:12:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.609+0000 7fa5d954a700 1 -- 172.21.15.7:0/577616211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d415c330 msgr2=0x7fa5d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.609+0000 7fa5d954a700 1 --2- 172.21.15.7:0/577616211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d415c330 0x7fa5d4068bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0098e0 tx=0x7fa5bc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.609+0000 7fa5d954a700 1 -- 172.21.15.7:0/577616211 shutdown_connections 2024-01-11T07:12:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.609+0000 7fa5d954a700 1 --2- 172.21.15.7:0/577616211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d415c330 0x7fa5d4068bf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.609+0000 7fa5d954a700 1 -- 172.21.15.7:0/577616211 >> 172.21.15.7:0/577616211 conn(0x7fa5d4063230 msgr2=0x7fa5d4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.609+0000 7fa5d954a700 1 -- 172.21.15.7:0/577616211 shutdown_connections 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.609+0000 7fa5d954a700 1 -- 172.21.15.7:0/577616211 wait complete. 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.610+0000 7fa5d954a700 1 Processor -- start 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.610+0000 7fa5d954a700 1 -- start start 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d954a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406e1b0 0x7fa5d4155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d954a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5d415b850 0x7fa5d415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d954a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5d415c330 0x7fa5d4156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d954a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d415a7e0 con 0x7fa5d415c330 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d954a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4159bd0 con 0x7fa5d406e1b0 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d954a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4159eb0 con 0x7fa5d415b850 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5d415c330 0x7fa5d4156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5d415c330 0x7fa5d4156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52252/0 (socket says 172.21.15.7:52252) 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d37fe700 1 -- 172.21.15.7:0/3664720913 learned_addr learned my addr 172.21.15.7:0/3664720913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d2ffd700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406e1b0 0x7fa5d4155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.611+0000 7fa5d27fc700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5d415b850 0x7fa5d415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d37fe700 1 -- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5d415b850 msgr2=0x7fa5d415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d37fe700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5d415b850 0x7fa5d415ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d37fe700 1 -- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406e1b0 msgr2=0x7fa5d4155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d37fe700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406e1b0 0x7fa5d4155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d37fe700 1 -- 172.21.15.7:0/3664720913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc009420 con 0x7fa5d415c330 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d27fc700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5d415b850 0x7fa5d415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d37fe700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5d415c330 0x7fa5d4156d80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c910 tx=0x7fa5cc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5caffd700 1 -- 172.21.15.7:0/3664720913 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc012630 con 0x7fa5d415c330 2024-01-11T07:12:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5caffd700 1 -- 172.21.15.7:0/3664720913 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0127b0 con 0x7fa5d415c330 2024-01-11T07:12:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d415a190 con 0x7fa5d415c330 2024-01-11T07:12:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5caffd700 1 -- 172.21.15.7:0/3664720913 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc012b70 con 0x7fa5d415c330 2024-01-11T07:12:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.612+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d41559c0 con 0x7fa5d415c330 2024-01-11T07:12:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.613+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d415a950 con 0x7fa5d415c330 2024-01-11T07:12:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.614+0000 7fa5caffd700 1 -- 172.21.15.7:0/3664720913 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa5cc01a3b0 con 0x7fa5d415c330 2024-01-11T07:12:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.614+0000 7fa5caffd700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa5b4066f30 0x7fa5b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.614+0000 7fa5caffd700 1 -- 172.21.15.7:0/3664720913 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa5cc083f20 con 0x7fa5d415c330 2024-01-11T07:12:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.614+0000 7fa5d2ffd700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa5b4066f30 0x7fa5b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.615+0000 7fa5d2ffd700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa5b4066f30 0x7fa5b40693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4001ed0 tx=0x7fa5c40019d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:38.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.619+0000 7fa5caffd700 1 -- 172.21.15.7:0/3664720913 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5cc051490 con 0x7fa5d415c330 2024-01-11T07:12:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:38.910+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7fa5d404c3c0 con 0x7fa5d415c330 2024-01-11T07:12:39.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.454+0000 7fa5caffd700 1 -- 172.21.15.7:0/3664720913 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v288) v1 ==== 131+0+0 (secure 0 0 0) 0x7fa5cc016020 con 0x7fa5d415c330 2024-01-11T07:12:39.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa5b4066f30 msgr2=0x7fa5b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:39.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa5b4066f30 0x7fa5b40693e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4001ed0 tx=0x7fa5c40019d0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5d415c330 msgr2=0x7fa5d4156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:39.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5d415c330 0x7fa5d4156d80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c910 tx=0x7fa5cc007f90 comp rx=0 tx=0).stop 2024-01-11T07:12:39.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 shutdown_connections 2024-01-11T07:12:39.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa5b4066f30 0x7fa5b40693e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5d415c330 0x7fa5d4156d80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406e1b0 0x7fa5d4155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 --2- 172.21.15.7:0/3664720913 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5d415b850 0x7fa5d415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 >> 172.21.15.7:0/3664720913 conn(0x7fa5d4063230 msgr2=0x7fa5d4156340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:39.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 shutdown_connections 2024-01-11T07:12:39.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.457+0000 7fa5d954a700 1 -- 172.21.15.7:0/3664720913 wait complete. 2024-01-11T07:12:39.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:12:39.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.612+0000 7f0184858700 1 Processor -- start 2024-01-11T07:12:39.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.612+0000 7f0184858700 1 -- start start 2024-01-11T07:12:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f0184858700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0180157aa0 0x7f018006dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f0184858700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f018006e140 0x7f01800b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f0184858700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01800b30b0 0x7f018006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:39.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f0184858700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01800b3530 con 0x7f018006e140 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f0184858700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0180063230 con 0x7f01800b30b0 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f0184858700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0180063ad0 con 0x7f0180157aa0 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f018006e140 0x7f01800b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f018006e140 0x7f01800b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52260/0 (socket says 172.21.15.7:52260) 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 -- 172.21.15.7:0/691335788 learned_addr learned my addr 172.21.15.7:0/691335788 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017e59c700 1 --2- 172.21.15.7:0/691335788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0180157aa0 0x7f018006dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017ed9d700 1 --2- 172.21.15.7:0/691335788 >> v2:172.21.15.113:3300/0 conn(0x7f01800b30b0 0x7f018006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 -- 172.21.15.7:0/691335788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0180157aa0 msgr2=0x7f018006dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 --2- 172.21.15.7:0/691335788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0180157aa0 0x7f018006dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 -- 172.21.15.7:0/691335788 >> v2:172.21.15.113:3300/0 conn(0x7f01800b30b0 msgr2=0x7f018006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 --2- 172.21.15.7:0/691335788 >> v2:172.21.15.113:3300/0 conn(0x7f01800b30b0 0x7f018006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 -- 172.21.15.7:0/691335788 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01800cb040 con 0x7f018006e140 2024-01-11T07:12:39.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.613+0000 7f017dd9b700 1 --2- 172.21.15.7:0/691335788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018006e140 0x7f01800b2b70 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f018006db90 tx=0x7f0174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6830c3fe1265548a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f017d59a700 1 -- 172.21.15.7:0/691335788 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400b640 con 0x7f018006e140 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f017d59a700 1 -- 172.21.15.7:0/691335788 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017400b7a0 con 0x7f018006e140 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f017d59a700 1 -- 172.21.15.7:0/691335788 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400bb60 con 0x7f018006e140 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 -- 172.21.15.7:0/691335788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018006e140 msgr2=0x7f01800b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 --2- 172.21.15.7:0/691335788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018006e140 0x7f01800b2b70 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f018006db90 tx=0x7f0174009250 comp rx=0 tx=0).stop 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 -- 172.21.15.7:0/691335788 shutdown_connections 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 --2- 172.21.15.7:0/691335788 >> v2:172.21.15.113:3300/0 conn(0x7f01800b30b0 0x7f018006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 --2- 172.21.15.7:0/691335788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0180157aa0 0x7f018006dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 --2- 172.21.15.7:0/691335788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018006e140 0x7f01800b2b70 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 -- 172.21.15.7:0/691335788 >> 172.21.15.7:0/691335788 conn(0x7f01800cd910 msgr2=0x7f01800cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 -- 172.21.15.7:0/691335788 shutdown_connections 2024-01-11T07:12:39.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.614+0000 7f0184858700 1 -- 172.21.15.7:0/691335788 wait complete. 2024-01-11T07:12:39.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.616+0000 7f0184858700 1 Processor -- start 2024-01-11T07:12:39.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.616+0000 7f0184858700 1 -- start start 2024-01-11T07:12:39.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.616+0000 7f0184858700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f018006e140 0x7f0180167b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:39.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.616+0000 7f0184858700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01800b30b0 0x7f0180152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:39.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.616+0000 7f0184858700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0180157aa0 0x7f01800cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.616+0000 7f0184858700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01800c4c60 con 0x7f01800b30b0 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.616+0000 7f0184858700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0180168070 con 0x7f0180157aa0 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.616+0000 7f0184858700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01800c4650 con 0x7f018006e140 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01800b30b0 0x7f0180152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01800b30b0 0x7f0180152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52262/0 (socket says 172.21.15.7:52262) 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 -- 172.21.15.7:0/1127511903 learned_addr learned my addr 172.21.15.7:0/1127511903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017e59c700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f018006e140 0x7f0180167b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017ed9d700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0180157aa0 0x7f01800cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 -- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f018006e140 msgr2=0x7f0180167b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f018006e140 0x7f0180167b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 -- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0180157aa0 msgr2=0x7f01800cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0180157aa0 0x7f01800cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 -- 172.21.15.7:0/1127511903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740094f0 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f017dd9b700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01800b30b0 0x7f0180152b80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f018006cfd0 tx=0x7f0174017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f016e7fc700 1 -- 172.21.15.7:0/1127511903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400b910 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f016e7fc700 1 -- 172.21.15.7:0/1127511903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017400ba70 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01800c76d0 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f016e7fc700 1 -- 172.21.15.7:0/1127511903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174011630 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.617+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01800c49a0 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.618+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01800c4e10 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.619+0000 7f016e7fc700 1 -- 172.21.15.7:0/1127511903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f01740119a0 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.619+0000 7f016e7fc700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0160066f30 0x7f01600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.619+0000 7f016e7fc700 1 -- 172.21.15.7:0/1127511903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0174085840 con 0x7f01800b30b0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.619+0000 7f017e59c700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0160066f30 0x7f01600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:39.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.620+0000 7f017e59c700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0160066f30 0x7f01600693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0170009d10 tx=0x7f01700091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:39.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.624+0000 7f016e7fc700 1 -- 172.21.15.7:0/1127511903 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01800c4e10 con 0x7f01800b30b0 2024-01-11T07:12:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.832+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f018004c430 con 0x7f01800b30b0 2024-01-11T07:12:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.836+0000 7f016e7fc700 1 -- 172.21.15.7:0/1127511903 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f018004c430 con 0x7f01800b30b0 2024-01-11T07:12:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0160066f30 msgr2=0x7f01600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0160066f30 0x7f01600693e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0170009d10 tx=0x7f01700091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01800b30b0 msgr2=0x7f0180152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01800b30b0 0x7f0180152b80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f018006cfd0 tx=0x7f0174017000 comp rx=0 tx=0).stop 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 shutdown_connections 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0160066f30 0x7f01600693e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0180157aa0 0x7f01800cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f018006e140 0x7f0180167b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 --2- 172.21.15.7:0/1127511903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01800b30b0 0x7f0180152b80 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 >> 172.21.15.7:0/1127511903 conn(0x7f01800cd910 msgr2=0x7f0180158080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 shutdown_connections 2024-01-11T07:12:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:39.838+0000 7f0184858700 1 -- 172.21.15.7:0/1127511903 wait complete. 2024-01-11T07:12:39.852 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:12:39.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:12:39.896 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:12:39.947 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:12:39.947 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:12:39.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.088+0000 7f71c1e36700 1 Processor -- start 2024-01-11T07:12:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.088+0000 7f71c1e36700 1 -- start start 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.089+0000 7f71c1e36700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71bc159e20 0x7f71bc15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.089+0000 7f71c1e36700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc15e010 con 0x7f71bc06bc50 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.089+0000 7f71c1e36700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc0acf80 con 0x7f71bc159a50 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.089+0000 7f71c1e36700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc15d9c0 con 0x7f71bc159e20 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.089+0000 7f71bbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f71bc06bc50 0x7f71bc06dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47694/0 (socket says 172.21.15.7:47694) 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.089+0000 7f71bbfff700 1 -- 172.21.15.7:0/2808762046 learned_addr learned my addr 172.21.15.7:0/2808762046 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71baffd700 1 --2- 172.21.15.7:0/2808762046 >> v2:172.21.15.113:3300/0 conn(0x7f71bc159e20 0x7f71bc15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313580060 0 0) 0x7f71bc15e010 con 0x7f71bc06bc50 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a8003570 con 0x7f71bc06bc50 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477411775 0 0) 0x7f71a8003570 con 0x7f71bc06bc50 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 >> v2:172.21.15.113:3300/0 conn(0x7f71bc159e20 msgr2=0x7f71bc15e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 --2- 172.21.15.7:0/2808762046 >> v2:172.21.15.113:3300/0 conn(0x7f71bc159e20 0x7f71bc15e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 >> v1:172.21.15.113:6789/0 conn(0x7f71bc159a50 legacy=0x7f71bc0ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc15dca0 con 0x7f71bc06bc50 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f71b40017f0 con 0x7f71bc06bc50 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 >> v1:172.21.15.7:6789/0 conn(0x7f71bc06bc50 legacy=0x7f71bc06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a8003a60 0x7f71a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a8004810 0x7f71a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a80051c0 0x7f71a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc15d9c0 con 0x7f71a8003a60 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a800c0d0 con 0x7f71a80051c0 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a800c290 con 0x7f71a8004810 2024-01-11T07:12:40.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71bb7fe700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a8003a60 0x7f71a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71baffd700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a8004810 0x7f71a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.090+0000 7f71bbfff700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a80051c0 0x7f71a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71bb7fe700 1 -- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a8004810 msgr2=0x7f71a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71bb7fe700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71a8004810 0x7f71a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71bb7fe700 1 -- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a80051c0 msgr2=0x7f71a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71bb7fe700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a80051c0 0x7f71a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71bb7fe700 1 -- 172.21.15.7:0/2808762046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a8003570 con 0x7f71a8003a60 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71bbfff700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a80051c0 0x7f71a800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71bbfff700 1 -- 172.21.15.7:0/2808762046 reap_dead start 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71bb7fe700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a8003a60 0x7f71a8003e70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f71a40018b0 tx=0x7f71a4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb3d0d3c5d799d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4012420 con 0x7f71a8003a60 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a40125a0 con 0x7f71a8003a60 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71ba7fc700 1 -- 172.21.15.7:0/2808762046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4012960 con 0x7f71a8003a60 2024-01-11T07:12:40.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71c1e36700 1 -- 172.21.15.7:0/2808762046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a8003a60 msgr2=0x7f71a8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.091+0000 7f71c1e36700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a8003a60 0x7f71a8003e70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f71a40018b0 tx=0x7f71a4007d30 comp rx=0 tx=0).stop 2024-01-11T07:12:40.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.092+0000 7f71c1e36700 1 -- 172.21.15.7:0/2808762046 shutdown_connections 2024-01-11T07:12:40.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.092+0000 7f71c1e36700 1 --2- 172.21.15.7:0/2808762046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a8003a60 0x7f71a8003e70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.092+0000 7f71c1e36700 1 -- 172.21.15.7:0/2808762046 >> 172.21.15.7:0/2808762046 conn(0x7f71bc0cd910 msgr2=0x7f71bc0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:40.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.092+0000 7f71c1e36700 1 -- 172.21.15.7:0/2808762046 shutdown_connections 2024-01-11T07:12:40.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.092+0000 7f71c1e36700 1 -- 172.21.15.7:0/2808762046 wait complete. 2024-01-11T07:12:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.093+0000 7f71c1e36700 1 Processor -- start 2024-01-11T07:12:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.093+0000 7f71c1e36700 1 -- start start 2024-01-11T07:12:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71c1e36700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8003a60 0x7f71bc144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71c1e36700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc156aa0 0x7f71bc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71c1e36700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71bc0adef0 0x7f71bc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71c1e36700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc15a9d0 con 0x7f71bc156aa0 2024-01-11T07:12:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71c1e36700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc15a3c0 con 0x7f71a8003a60 2024-01-11T07:12:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71c1e36700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0ae6d0 con 0x7f71bc0adef0 2024-01-11T07:12:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc156aa0 0x7f71bc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc156aa0 0x7f71bc0af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52278/0 (socket says 172.21.15.7:52278) 2024-01-11T07:12:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 -- 172.21.15.7:0/1009735593 learned_addr learned my addr 172.21.15.7:0/1009735593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71bbfff700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71bc0adef0 0x7f71bc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71bb7fe700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8003a60 0x7f71bc144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 -- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71bc0adef0 msgr2=0x7f71bc15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71bc0adef0 0x7f71bc15c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 -- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8003a60 msgr2=0x7f71bc144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8003a60 0x7f71bc144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 -- 172.21.15.7:0/1009735593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc15a660 con 0x7f71bc156aa0 2024-01-11T07:12:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.094+0000 7f71baffd700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc156aa0 0x7f71bc0af9a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f71ac011990 tx=0x7f71ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:40.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.095+0000 7f71b37fe700 1 -- 172.21.15.7:0/1009735593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac015670 con 0x7f71bc156aa0 2024-01-11T07:12:40.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.095+0000 7f71b37fe700 1 -- 172.21.15.7:0/1009735593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac0157d0 con 0x7f71bc156aa0 2024-01-11T07:12:40.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.095+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc0ae9b0 con 0x7f71bc156aa0 2024-01-11T07:12:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.095+0000 7f71b37fe700 1 -- 172.21.15.7:0/1009735593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac015b90 con 0x7f71bc156aa0 2024-01-11T07:12:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.095+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc15cd60 con 0x7f71bc156aa0 2024-01-11T07:12:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.096+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc15acb0 con 0x7f71bc156aa0 2024-01-11T07:12:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.096+0000 7f71b37fe700 1 -- 172.21.15.7:0/1009735593 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f71ac013410 con 0x7f71bc156aa0 2024-01-11T07:12:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.096+0000 7f71b37fe700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f719c066f30 0x7f719c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.097+0000 7f71b37fe700 1 -- 172.21.15.7:0/1009735593 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f71ac084880 con 0x7f71bc156aa0 2024-01-11T07:12:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.097+0000 7f71bb7fe700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f719c066f30 0x7f719c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.097+0000 7f71bb7fe700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f719c066f30 0x7f719c0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f71a4001010 tx=0x7f71a4015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.101+0000 7f71b37fe700 1 -- 172.21.15.7:0/1009735593 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71ac051df0 con 0x7f71bc156aa0 2024-01-11T07:12:40.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.352+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71bc04c3c0 con 0x7f71bc156aa0 2024-01-11T07:12:40.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.353+0000 7f71b37fe700 1 -- 172.21.15.7:0/1009735593 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f71ac022090 con 0x7f71bc156aa0 2024-01-11T07:12:40.353 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:40.353 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5639,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3071166276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3071166276},{"type":"v1","addr":"172.21.15.113:6837","nonce":3071166276}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2324048079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2324048079},{"type":"v1","addr":"172.21.15.7:6833","nonce":2324048079}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6169,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/3639617240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3639617240},{"type":"v1","addr":"172.21.15.113:6836","nonce":3639617240}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45},{"gid":6648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4166896547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":4166896547},{"type":"v1","addr":"172.21.15.7:6835","nonce":4166896547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[]} 2024-01-11T07:12:40.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f719c066f30 msgr2=0x7f719c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f719c066f30 0x7f719c0693e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f71a4001010 tx=0x7f71a4015000 comp rx=0 tx=0).stop 2024-01-11T07:12:40.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc156aa0 msgr2=0x7f71bc0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc156aa0 0x7f71bc0af9a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f71ac011990 tx=0x7f71ac008040 comp rx=0 tx=0).stop 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 shutdown_connections 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f719c066f30 0x7f719c0693e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f71bc0adef0 0x7f71bc15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71a8003a60 0x7f71bc144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.355+0000 7f71c1e36700 1 --2- 172.21.15.7:0/1009735593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc156aa0 0x7f71bc0af9a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.356+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 >> 172.21.15.7:0/1009735593 conn(0x7f71bc0cd910 msgr2=0x7f71bc0b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.356+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 shutdown_connections 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.356+0000 7f71c1e36700 1 -- 172.21.15.7:0/1009735593 wait complete. 2024-01-11T07:12:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-01-11T07:12:40.369 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:12:40.369 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:12:40.370 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:12:40.370 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:12:40.370 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:12:40.370 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:12:40.370 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:12:40.370 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:12:40.370 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:12:40.370 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:12:40.371 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:12:40.371 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:12:40.371 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:12:40.371 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:12:40.371 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:12:40.371 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:12:40.379 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:12:40.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:12:40.381 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:12:40.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:12:40.382 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:12:40.383 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:12:40.384 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:12:40.384 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:12:40.422 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:12:40.422 DEBUG:teuthology.parallel:result is None 2024-01-11T07:12:40.423 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:12:40.423 DEBUG:teuthology.parallel:result is None 2024-01-11T07:12:40.425 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:12:40.425 DEBUG:teuthology.parallel:result is None 2024-01-11T07:12:40.426 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:12:40.426 DEBUG:teuthology.parallel:result is None 2024-01-11T07:12:40.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:12:40.509 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:12:40.509 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:12:40.510 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:12:40.511 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:12:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.567+0000 7f48d4a4c700 1 Processor -- start 2024-01-11T07:12:40.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.567+0000 7f48d4a4c700 1 -- start start 2024-01-11T07:12:40.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.568+0000 7f48d4a4c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d006f170 0x7f48d006a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.568+0000 7f48d4a4c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48d0156bc0 0x7f48d006ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.568+0000 7f48d4a4c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d006e0d0 con 0x7f48d01567f0 2024-01-11T07:12:40.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.568+0000 7f48d4a4c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d00ccb00 con 0x7f48d0156bc0 2024-01-11T07:12:40.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.568+0000 7f48d4a4c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d006e3b0 con 0x7f48d006f170 2024-01-11T07:12:40.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48ced9d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f48d0156bc0 0x7f48d006ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48cdd9b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d006f170 0x7f48d006a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48ce59c700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f48d01567f0 0x7f48d00c86c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:42552/0 (socket says 172.21.15.7:42552) 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48ce59c700 1 -- 172.21.15.7:0/323554688 learned_addr learned my addr 172.21.15.7:0/323554688 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48ced9d700 1 -- 172.21.15.7:0/323554688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d006f170 msgr2=0x7f48d006a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48ced9d700 1 --2- 172.21.15.7:0/323554688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d006f170 0x7f48d006a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48ced9d700 1 -- 172.21.15.7:0/323554688 >> v1:172.21.15.113:6789/0 conn(0x7f48d01567f0 legacy=0x7f48d00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48ced9d700 1 -- 172.21.15.7:0/323554688 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d006e690 con 0x7f48d0156bc0 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48cd59a700 1 -- 172.21.15.7:0/323554688 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099540985 0 0) 0x7f48d006e0d0 con 0x7f48d01567f0 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48ced9d700 1 --2- 172.21.15.7:0/323554688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d0156bc0 0x7f48d006ac60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009e60 tx=0x7f48c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7aad90ac4979ef server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.569+0000 7f48cd59a700 1 -- 172.21.15.7:0/323554688 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800e070 con 0x7f48d0156bc0 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48cd59a700 1 -- 172.21.15.7:0/323554688 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c800bbf0 con 0x7f48d0156bc0 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48cd59a700 1 -- 172.21.15.7:0/323554688 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8012430 con 0x7f48d0156bc0 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48d4a4c700 1 -- 172.21.15.7:0/323554688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d0156bc0 msgr2=0x7f48d006ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/323554688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d0156bc0 0x7f48d006ac60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009e60 tx=0x7f48c8009780 comp rx=0 tx=0).stop 2024-01-11T07:12:40.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48d4a4c700 1 -- 172.21.15.7:0/323554688 shutdown_connections 2024-01-11T07:12:40.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/323554688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d0156bc0 0x7f48d006ac60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/323554688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d006f170 0x7f48d006a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48d4a4c700 1 -- 172.21.15.7:0/323554688 >> 172.21.15.7:0/323554688 conn(0x7f48d00cd090 msgr2=0x7f48d00cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:40.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48d4a4c700 1 -- 172.21.15.7:0/323554688 shutdown_connections 2024-01-11T07:12:40.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.570+0000 7f48d4a4c700 1 -- 172.21.15.7:0/323554688 wait complete. 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.571+0000 7f48d4a4c700 1 Processor -- start 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.571+0000 7f48d4a4c700 1 -- start start 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48d4a4c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d006f170 0x7f48d014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48d4a4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d01567f0 0x7f48d00c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48d4a4c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d0156bc0 0x7f48d006a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48d4a4c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00caf60 con 0x7f48d01567f0 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48d4a4c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00cb3f0 con 0x7f48d006f170 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48d4a4c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00ca350 con 0x7f48d0156bc0 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48cdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d01567f0 0x7f48d00c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48cdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d01567f0 0x7f48d00c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52322/0 (socket says 172.21.15.7:52322) 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48cdd9b700 1 -- 172.21.15.7:0/1113144144 learned_addr learned my addr 172.21.15.7:0/1113144144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:40.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48ce59c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d006f170 0x7f48d014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.572+0000 7f48ced9d700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d0156bc0 0x7f48d006a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48cdd9b700 1 -- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d0156bc0 msgr2=0x7f48d006a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48cdd9b700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d0156bc0 0x7f48d006a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48cdd9b700 1 -- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d006f170 msgr2=0x7f48d014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48cdd9b700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d006f170 0x7f48d014d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48cdd9b700 1 -- 172.21.15.7:0/1113144144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c8009a20 con 0x7f48d01567f0 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48ce59c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d006f170 0x7f48d014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48cdd9b700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d01567f0 0x7f48d00c9b00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f48c400ea70 tx=0x7f48c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48be7fc700 1 -- 172.21.15.7:0/1113144144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c400eeb0 con 0x7f48d01567f0 2024-01-11T07:12:40.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48be7fc700 1 -- 172.21.15.7:0/1113144144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c400c850 con 0x7f48d01567f0 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d00ca630 con 0x7f48d01567f0 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48be7fc700 1 -- 172.21.15.7:0/1113144144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c400cc30 con 0x7f48d01567f0 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.573+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d00cabf0 con 0x7f48d01567f0 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.574+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d004c3c0 con 0x7f48d01567f0 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.575+0000 7f48be7fc700 1 -- 172.21.15.7:0/1113144144 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f48c4013680 con 0x7f48d01567f0 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.575+0000 7f48be7fc700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f48b0066f30 0x7f48b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.575+0000 7f48be7fc700 1 -- 172.21.15.7:0/1113144144 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f48c4084460 con 0x7f48d01567f0 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.575+0000 7f48ce59c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f48b0066f30 0x7f48b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:40.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.576+0000 7f48ce59c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f48b0066f30 0x7f48b00693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f48c0007fd0 tx=0x7f48c0007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:40.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.580+0000 7f48be7fc700 1 -- 172.21.15.7:0/1113144144 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48c40519d0 con 0x7f48d01567f0 2024-01-11T07:12:40.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:40.797+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f48d006e970 con 0x7f48d01567f0 2024-01-11T07:12:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.847+0000 7f48be7fc700 1 -- 172.21.15.7:0/1113144144 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7f48c4054ff0 con 0x7f48d01567f0 2024-01-11T07:12:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f48b0066f30 msgr2=0x7f48b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f48b0066f30 0x7f48b00693e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f48c0007fd0 tx=0x7f48c0007d30 comp rx=0 tx=0).stop 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d01567f0 msgr2=0x7f48d00c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d01567f0 0x7f48d00c9b00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f48c400ea70 tx=0x7f48c40091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 shutdown_connections 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f48b0066f30 0x7f48b00693e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f48d0156bc0 0x7f48d006a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f48d006f170 0x7f48d014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 --2- 172.21.15.7:0/1113144144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d01567f0 0x7f48d00c9b00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.850+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 >> 172.21.15.7:0/1113144144 conn(0x7f48d00cd090 msgr2=0x7f48d00c3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.851+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 shutdown_connections 2024-01-11T07:12:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:41.851+0000 7f48d4a4c700 1 -- 172.21.15.7:0/1113144144 wait complete. 2024-01-11T07:12:41.852 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:12:41.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:12:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.001+0000 7f035ecbf700 1 Processor -- start 2024-01-11T07:12:42.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.001+0000 7f035ecbf700 1 -- start start 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.002+0000 7f035ecbf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03580b0b40 0x7f0358162770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.002+0000 7f035ecbf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0358159300 con 0x7f03580cafb0 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.002+0000 7f035ecbf700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03580cf000 con 0x7f0358158280 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.002+0000 7f035ecbf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03580cf260 con 0x7f03580b0b40 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.002+0000 7f035d25c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03580b0b40 0x7f0358162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.002+0000 7f035ca5b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f03580cafb0 0x7f03580ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47730/0 (socket says 172.21.15.7:47730) 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.003+0000 7f035ca5b700 1 -- 172.21.15.7:0/958022905 learned_addr learned my addr 172.21.15.7:0/958022905 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.003+0000 7f035d25c700 1 -- 172.21.15.7:0/958022905 >> v1:172.21.15.113:6789/0 conn(0x7f0358158280 legacy=0x7f03580b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.003+0000 7f035d25c700 1 -- 172.21.15.7:0/958022905 >> v1:172.21.15.7:6789/0 conn(0x7f03580cafb0 legacy=0x7f03580ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.003+0000 7f035d25c700 1 -- 172.21.15.7:0/958022905 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03580cf540 con 0x7f03580b0b40 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.003+0000 7f035d25c700 1 --2- 172.21.15.7:0/958022905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580b0b40 0x7f0358162770 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0350009e60 tx=0x7f0350009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a67b0f5f187c851 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.003+0000 7f03577fe700 1 -- 172.21.15.7:0/958022905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035000e070 con 0x7f03580b0b40 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.003+0000 7f03577fe700 1 -- 172.21.15.7:0/958022905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035000bc30 con 0x7f03580b0b40 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.003+0000 7f03577fe700 1 -- 172.21.15.7:0/958022905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350012430 con 0x7f03580b0b40 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.004+0000 7f035ecbf700 1 -- 172.21.15.7:0/958022905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580b0b40 msgr2=0x7f0358162770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.004+0000 7f035ecbf700 1 --2- 172.21.15.7:0/958022905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580b0b40 0x7f0358162770 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0350009e60 tx=0x7f0350009780 comp rx=0 tx=0).stop 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.004+0000 7f035ecbf700 1 -- 172.21.15.7:0/958022905 shutdown_connections 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.004+0000 7f035ecbf700 1 --2- 172.21.15.7:0/958022905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580b0b40 0x7f0358162770 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.004+0000 7f035ecbf700 1 -- 172.21.15.7:0/958022905 >> 172.21.15.7:0/958022905 conn(0x7f0358071100 msgr2=0x7f035806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.004+0000 7f035ecbf700 1 -- 172.21.15.7:0/958022905 shutdown_connections 2024-01-11T07:12:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.004+0000 7f035ecbf700 1 -- 172.21.15.7:0/958022905 wait complete. 2024-01-11T07:12:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.005+0000 7f035ecbf700 1 Processor -- start 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.005+0000 7f035ecbf700 1 -- start start 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f035ecbf700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03580b0b40 0x7f03580acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f035ecbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580cafb0 0x7f03580a7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f035ecbf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0358158280 0x7f03580b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f035ecbf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0358168870 con 0x7f03580cafb0 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f035ecbf700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0358167f40 con 0x7f03580b0b40 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f035ecbf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0358168220 con 0x7f0358158280 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f0357fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580cafb0 0x7f03580a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f0357fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580cafb0 0x7f03580a7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52328/0 (socket says 172.21.15.7:52328) 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f0357fff700 1 -- 172.21.15.7:0/626068936 learned_addr learned my addr 172.21.15.7:0/626068936 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f035d25c700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0358158280 0x7f03580b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.006+0000 7f035ca5b700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03580b0b40 0x7f03580acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0357fff700 1 -- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0358158280 msgr2=0x7f03580b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0357fff700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0358158280 0x7f03580b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0357fff700 1 -- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03580b0b40 msgr2=0x7f03580acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0357fff700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03580b0b40 0x7f03580acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0357fff700 1 -- 172.21.15.7:0/626068936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0350009a20 con 0x7f03580cafb0 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f035ca5b700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03580b0b40 0x7f03580acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0357fff700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580cafb0 0x7f03580a7680 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f034400c4e0 tx=0x7f0344007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0354ff9700 1 -- 172.21.15.7:0/626068936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344011590 con 0x7f03580cafb0 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0354ff9700 1 -- 172.21.15.7:0/626068936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03440116f0 con 0x7f03580cafb0 2024-01-11T07:12:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0358168500 con 0x7f03580cafb0 2024-01-11T07:12:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f0354ff9700 1 -- 172.21.15.7:0/626068936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344011ab0 con 0x7f03580cafb0 2024-01-11T07:12:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.007+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0358167bc0 con 0x7f03580cafb0 2024-01-11T07:12:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.008+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0358168a20 con 0x7f03580cafb0 2024-01-11T07:12:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.008+0000 7f0354ff9700 1 -- 172.21.15.7:0/626068936 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f034401b3b0 con 0x7f03580cafb0 2024-01-11T07:12:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.009+0000 7f0354ff9700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f033c066f30 0x7f033c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.009+0000 7f0354ff9700 1 -- 172.21.15.7:0/626068936 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0344083520 con 0x7f03580cafb0 2024-01-11T07:12:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.009+0000 7f035ca5b700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f033c066f30 0x7f033c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.010+0000 7f035ca5b700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f033c066f30 0x7f033c0693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f034c0018c0 tx=0x7f034c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.013+0000 7f0354ff9700 1 -- 172.21.15.7:0/626068936 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0358168a20 con 0x7f03580cafb0 2024-01-11T07:12:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.301+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f035804c3c0 con 0x7f03580cafb0 2024-01-11T07:12:42.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.301+0000 7f0354ff9700 1 -- 172.21.15.7:0/626068936 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0344017020 con 0x7f03580cafb0 2024-01-11T07:12:42.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:42.301 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:42.301 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:12:42.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.302 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:12:42.303 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.305 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.307 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:12:42.309 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:12:42.311 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:12:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:12:42.313 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:12:42.315 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f033c066f30 msgr2=0x7f033c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f033c066f30 0x7f033c0693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f034c0018c0 tx=0x7f034c009040 comp rx=0 tx=0).stop 2024-01-11T07:12:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580cafb0 msgr2=0x7f03580a7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580cafb0 0x7f03580a7680 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f034400c4e0 tx=0x7f0344007f00 comp rx=0 tx=0).stop 2024-01-11T07:12:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 shutdown_connections 2024-01-11T07:12:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f033c066f30 0x7f033c0693e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0358158280 0x7f03580b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f03580b0b40 0x7f03580acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 --2- 172.21.15.7:0/626068936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03580cafb0 0x7f03580a7680 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 >> 172.21.15.7:0/626068936 conn(0x7f0358071100 msgr2=0x7f03580cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 shutdown_connections 2024-01-11T07:12:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.304+0000 7f035ecbf700 1 -- 172.21.15.7:0/626068936 wait complete. 2024-01-11T07:12:42.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.452+0000 7fcde8e83700 1 Processor -- start 2024-01-11T07:12:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.452+0000 7fcde8e83700 1 -- start start 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.453+0000 7fcde8e83700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcde4063230 0x7fcde4063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.453+0000 7fcde8e83700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcde40ac5b0 0x7fcde40ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.453+0000 7fcde8e83700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde40ab100 con 0x7fcde415a280 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.453+0000 7fcde8e83700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde415d880 con 0x7fcde40ac5b0 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.453+0000 7fcde8e83700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde406c5b0 con 0x7fcde4063230 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.453+0000 7fcde2d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcde40ac5b0 0x7fcde40ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde1d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcde415a280 0x7fcde40aa9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47738/0 (socket says 172.21.15.7:47738) 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde2d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcde40ac5b0 0x7fcde40ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52344/0 (socket says 172.21.15.7:52344) 2024-01-11T07:12:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde1d9b700 1 -- 172.21.15.7:0/1334538390 learned_addr learned my addr 172.21.15.7:0/1334538390 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde259c700 1 --2- 172.21.15.7:0/1334538390 >> v2:172.21.15.113:3300/0 conn(0x7fcde4063230 0x7fcde4063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde2d9d700 1 -- 172.21.15.7:0/1334538390 >> v2:172.21.15.113:3300/0 conn(0x7fcde4063230 msgr2=0x7fcde4063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde2d9d700 1 --2- 172.21.15.7:0/1334538390 >> v2:172.21.15.113:3300/0 conn(0x7fcde4063230 0x7fcde4063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde2d9d700 1 -- 172.21.15.7:0/1334538390 >> v1:172.21.15.7:6789/0 conn(0x7fcde415a280 legacy=0x7fcde40aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde2d9d700 1 -- 172.21.15.7:0/1334538390 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde4063b40 con 0x7fcde40ac5b0 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde259c700 1 --2- 172.21.15.7:0/1334538390 >> v2:172.21.15.113:3300/0 conn(0x7fcde4063230 0x7fcde4063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde159a700 1 -- 172.21.15.7:0/1334538390 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871459164 0 0) 0x7fcde40ab100 con 0x7fcde415a280 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde2d9d700 1 --2- 172.21.15.7:0/1334538390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40ac9f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcde40ad1f0 tx=0x7fcddc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcd70b0fe0eef73 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde159a700 1 -- 172.21.15.7:0/1334538390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00e070 con 0x7fcde40ac5b0 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde159a700 1 -- 172.21.15.7:0/1334538390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc00bc30 con 0x7fcde40ac5b0 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.454+0000 7fcde159a700 1 -- 172.21.15.7:0/1334538390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc012430 con 0x7fcde40ac5b0 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.455+0000 7fcde8e83700 1 -- 172.21.15.7:0/1334538390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 msgr2=0x7fcde40ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.455+0000 7fcde8e83700 1 --2- 172.21.15.7:0/1334538390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40ac9f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcde40ad1f0 tx=0x7fcddc009780 comp rx=0 tx=0).stop 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.455+0000 7fcde8e83700 1 -- 172.21.15.7:0/1334538390 shutdown_connections 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.455+0000 7fcde8e83700 1 --2- 172.21.15.7:0/1334538390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40ac9f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.455+0000 7fcde8e83700 1 --2- 172.21.15.7:0/1334538390 >> v2:172.21.15.113:3300/0 conn(0x7fcde4063230 0x7fcde4063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.455+0000 7fcde8e83700 1 -- 172.21.15.7:0/1334538390 >> 172.21.15.7:0/1334538390 conn(0x7fcde40680e0 msgr2=0x7fcde40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.455+0000 7fcde8e83700 1 -- 172.21.15.7:0/1334538390 shutdown_connections 2024-01-11T07:12:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.455+0000 7fcde8e83700 1 -- 172.21.15.7:0/1334538390 wait complete. 2024-01-11T07:12:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.456+0000 7fcde8e83700 1 Processor -- start 2024-01-11T07:12:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.456+0000 7fcde8e83700 1 -- start start 2024-01-11T07:12:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde8e83700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcde4063230 0x7fcde4153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde8e83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde8e83700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde415a280 0x7fcde40c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde8e83700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415fe20 con 0x7fcde40ac5b0 2024-01-11T07:12:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde8e83700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4160070 con 0x7fcde415a280 2024-01-11T07:12:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde8e83700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4160350 con 0x7fcde4063230 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40c21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52350/0 (socket says 172.21.15.7:52350) 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde1d9b700 1 -- 172.21.15.7:0/3439276007 learned_addr learned my addr 172.21.15.7:0/3439276007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde259c700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcde4063230 0x7fcde4153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.457+0000 7fcde2d9d700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde415a280 0x7fcde40c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcde1d9b700 1 -- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcde4063230 msgr2=0x7fcde4153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcde1d9b700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcde4063230 0x7fcde4153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcde1d9b700 1 -- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde415a280 msgr2=0x7fcde40c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcde1d9b700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde415a280 0x7fcde40c2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcde1d9b700 1 -- 172.21.15.7:0/3439276007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc009a20 con 0x7fcde40ac5b0 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcde1d9b700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40c21c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c7e0 tx=0x7fcdd8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcdd27fc700 1 -- 172.21.15.7:0/3439276007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8005b30 con 0x7fcde40ac5b0 2024-01-11T07:12:42.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcdd27fc700 1 -- 172.21.15.7:0/3439276007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8005c90 con 0x7fcde40ac5b0 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde4160630 con 0x7fcde40ac5b0 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcdd27fc700 1 -- 172.21.15.7:0/3439276007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8007300 con 0x7fcde40ac5b0 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.458+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde41542a0 con 0x7fcde40ac5b0 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.459+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde415dda0 con 0x7fcde40ac5b0 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.460+0000 7fcdd27fc700 1 -- 172.21.15.7:0/3439276007 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcdd801b3b0 con 0x7fcde40ac5b0 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.460+0000 7fcdd27fc700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcdc4066f30 0x7fcdc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.460+0000 7fcdd27fc700 1 -- 172.21.15.7:0/3439276007 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcdd8083700 con 0x7fcde40ac5b0 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.461+0000 7fcde259c700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcdc4066f30 0x7fcdc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.461+0000 7fcde259c700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcdc4066f30 0x7fcdc40693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4006f00 tx=0x7fcdd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.466+0000 7fcdd27fc700 1 -- 172.21.15.7:0/3439276007 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcde415dda0 con 0x7fcde40ac5b0 2024-01-11T07:12:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.671+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcde415dda0 con 0x7fcde40ac5b0 2024-01-11T07:12:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.672+0000 7fcdd27fc700 1 -- 172.21.15.7:0/3439276007 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+12089 (secure 0 0 0) 0x7fcde415dda0 con 0x7fcde40ac5b0 2024-01-11T07:12:42.672 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:42.672 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:41.844726+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcdc4066f30 msgr2=0x7fcdc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcdc4066f30 0x7fcdc40693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4006f00 tx=0x7fcdd4008040 comp rx=0 tx=0).stop 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 msgr2=0x7fcde40c21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40c21c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c7e0 tx=0x7fcdd8007df0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 shutdown_connections 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcdc4066f30 0x7fcdc40693e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcde415a280 0x7fcde40c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcde4063230 0x7fcde4153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 --2- 172.21.15.7:0/3439276007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcde40ac5b0 0x7fcde40c21c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 >> 172.21.15.7:0/3439276007 conn(0x7fcde40680e0 msgr2=0x7fcde415b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 shutdown_connections 2024-01-11T07:12:42.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.674+0000 7fcde8e83700 1 -- 172.21.15.7:0/3439276007 wait complete. 2024-01-11T07:12:42.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:12:42.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:12:42.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.823+0000 7fd81206a700 1 Processor -- start 2024-01-11T07:12:42.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.823+0000 7fd81206a700 1 -- start start 2024-01-11T07:12:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd81206a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd80c06a850 0x7fd80c0b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd81206a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd80c156490 0x7fd80c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd81206a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c0ae4b0 con 0x7fd80c06ac20 2024-01-11T07:12:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd81206a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80c0af950 con 0x7fd80c156490 2024-01-11T07:12:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd81206a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80c0b1560 con 0x7fd80c06a850 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd80bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd80c156490 0x7fd80c1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd80bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd80c156490 0x7fd80c1568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52360/0 (socket says 172.21.15.7:52360) 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd80bfff700 1 -- 172.21.15.7:0/253532940 learned_addr learned my addr 172.21.15.7:0/253532940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.824+0000 7fd80b7fe700 1 --2- 172.21.15.7:0/253532940 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06a850 0x7fd80c0b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.825+0000 7fd80bfff700 1 -- 172.21.15.7:0/253532940 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06a850 msgr2=0x7fd80c0b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.825+0000 7fd80bfff700 1 --2- 172.21.15.7:0/253532940 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06a850 0x7fd80c0b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.825+0000 7fd80bfff700 1 -- 172.21.15.7:0/253532940 >> v1:172.21.15.113:6789/0 conn(0x7fd80c06ac20 legacy=0x7fd80c0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.825+0000 7fd80bfff700 1 -- 172.21.15.7:0/253532940 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c0adb80 con 0x7fd80c156490 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.825+0000 7fd80bfff700 1 --2- 172.21.15.7:0/253532940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c1568f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd80c071520 tx=0x7fd804009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c467ddd9ef0036 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.825+0000 7fd80a7fc700 1 -- 172.21.15.7:0/253532940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80400e070 con 0x7fd80c156490 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.825+0000 7fd80a7fc700 1 -- 172.21.15.7:0/253532940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80400ba20 con 0x7fd80c156490 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.825+0000 7fd80a7fc700 1 -- 172.21.15.7:0/253532940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8040092e0 con 0x7fd80c156490 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.826+0000 7fd81206a700 1 -- 172.21.15.7:0/253532940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 msgr2=0x7fd80c1568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.826+0000 7fd81206a700 1 --2- 172.21.15.7:0/253532940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c1568f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd80c071520 tx=0x7fd804009840 comp rx=0 tx=0).stop 2024-01-11T07:12:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.826+0000 7fd81206a700 1 -- 172.21.15.7:0/253532940 shutdown_connections 2024-01-11T07:12:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.826+0000 7fd81206a700 1 --2- 172.21.15.7:0/253532940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c1568f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.826+0000 7fd81206a700 1 --2- 172.21.15.7:0/253532940 >> v2:172.21.15.113:3300/0 conn(0x7fd80c06a850 0x7fd80c0b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.826+0000 7fd81206a700 1 -- 172.21.15.7:0/253532940 >> 172.21.15.7:0/253532940 conn(0x7fd80c0cd070 msgr2=0x7fd80c0cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.826+0000 7fd81206a700 1 -- 172.21.15.7:0/253532940 shutdown_connections 2024-01-11T07:12:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.826+0000 7fd81206a700 1 -- 172.21.15.7:0/253532940 wait complete. 2024-01-11T07:12:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.827+0000 7fd81206a700 1 Processor -- start 2024-01-11T07:12:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.827+0000 7fd81206a700 1 -- start start 2024-01-11T07:12:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd81206a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c06a850 0x7fd80c167cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd81206a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c06ac20 0x7fd80c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd81206a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd81206a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0aa540 con 0x7fd80c156490 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd81206a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0aa820 con 0x7fd80c06ac20 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd81206a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0aab00 con 0x7fd80c06a850 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52372/0 (socket says 172.21.15.7:52372) 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80bfff700 1 -- 172.21.15.7:0/2753336241 learned_addr learned my addr 172.21.15.7:0/2753336241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80b7fe700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c06a850 0x7fd80c167cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80affd700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c06ac20 0x7fd80c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80bfff700 1 -- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c06a850 msgr2=0x7fd80c167cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80bfff700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c06a850 0x7fd80c167cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80bfff700 1 -- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c06ac20 msgr2=0x7fd80c06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80bfff700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c06ac20 0x7fd80c06dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.828+0000 7fd80bfff700 1 -- 172.21.15.7:0/2753336241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd804009ae0 con 0x7fd80c156490 2024-01-11T07:12:42.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.829+0000 7fd80bfff700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c150860 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd804009880 tx=0x7fd804018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.829+0000 7fd8037fe700 1 -- 172.21.15.7:0/2753336241 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804008a80 con 0x7fd80c156490 2024-01-11T07:12:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.829+0000 7fd8037fe700 1 -- 172.21.15.7:0/2753336241 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd804008be0 con 0x7fd80c156490 2024-01-11T07:12:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.829+0000 7fd8037fe700 1 -- 172.21.15.7:0/2753336241 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804025460 con 0x7fd80c156490 2024-01-11T07:12:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.829+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c0aade0 con 0x7fd80c156490 2024-01-11T07:12:42.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.829+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c06e160 con 0x7fd80c156490 2024-01-11T07:12:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.830+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c04c3c0 con 0x7fd80c156490 2024-01-11T07:12:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.830+0000 7fd8037fe700 1 -- 172.21.15.7:0/2753336241 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd804003040 con 0x7fd80c156490 2024-01-11T07:12:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.831+0000 7fd8037fe700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7ec066f30 0x7fd7ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.831+0000 7fd8037fe700 1 -- 172.21.15.7:0/2753336241 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd8040856b0 con 0x7fd80c156490 2024-01-11T07:12:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.831+0000 7fd80b7fe700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7ec066f30 0x7fd7ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:42.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.832+0000 7fd80b7fe700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7ec066f30 0x7fd7ec0693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009db0 tx=0x7fd7fc0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:42.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:42.835+0000 7fd8037fe700 1 -- 172.21.15.7:0/2753336241 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd804052c70 con 0x7fd80c156490 2024-01-11T07:12:43.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.039+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd80c0ab300 con 0x7fd80c156490 2024-01-11T07:12:43.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.039+0000 7fd8037fe700 1 -- 172.21.15.7:0/2753336241 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd80c0ab300 con 0x7fd80c156490 2024-01-11T07:12:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:12:43.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.041+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7ec066f30 msgr2=0x7fd7ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.041+0000 7fd81206a700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7ec066f30 0x7fd7ec0693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009db0 tx=0x7fd7fc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.041+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 msgr2=0x7fd80c150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.041+0000 7fd81206a700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c150860 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd804009880 tx=0x7fd804018000 comp rx=0 tx=0).stop 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.042+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 shutdown_connections 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.042+0000 7fd81206a700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7ec066f30 0x7fd7ec0693e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.042+0000 7fd81206a700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd80c156490 0x7fd80c150860 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.042+0000 7fd81206a700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd80c06a850 0x7fd80c167cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.042+0000 7fd81206a700 1 --2- 172.21.15.7:0/2753336241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd80c06ac20 0x7fd80c06dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.042+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 >> 172.21.15.7:0/2753336241 conn(0x7fd80c0cd070 msgr2=0x7fd80c0ace90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.042+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 shutdown_connections 2024-01-11T07:12:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.042+0000 7fd81206a700 1 -- 172.21.15.7:0/2753336241 wait complete. 2024-01-11T07:12:43.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:12:43.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.188+0000 7f5f34ff9700 1 Processor -- start 2024-01-11T07:12:43.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.188+0000 7f5f34ff9700 1 -- start start 2024-01-11T07:12:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f34ff9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159f70 0x7f5f300c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f34ff9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f300c9570 0x7f5f300680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f34ff9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f30067c60 con 0x7f5f30159ba0 2024-01-11T07:12:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f34ff9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3006d730 con 0x7f5f300c9570 2024-01-11T07:12:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f34ff9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30069230 con 0x7f5f30159f70 2024-01-11T07:12:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f2ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f300c9570 0x7f5f300680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f2e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5f30159ba0 0x7f5f3006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47748/0 (socket says 172.21.15.7:47748) 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f2e59c700 1 -- 172.21.15.7:0/3730219337 learned_addr learned my addr 172.21.15.7:0/3730219337 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/3730219337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159f70 0x7f5f300c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f2ed9d700 1 -- 172.21.15.7:0/3730219337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159f70 msgr2=0x7f5f300c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f2ed9d700 1 --2- 172.21.15.7:0/3730219337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159f70 0x7f5f300c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f2ed9d700 1 -- 172.21.15.7:0/3730219337 >> v1:172.21.15.7:6789/0 conn(0x7f5f30159ba0 legacy=0x7f5f3006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f2ed9d700 1 -- 172.21.15.7:0/3730219337 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30069510 con 0x7f5f300c9570 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f2dd9b700 1 -- 172.21.15.7:0/3730219337 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473175602 0 0) 0x7f5f30067c60 con 0x7f5f30159ba0 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.189+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/3730219337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159f70 0x7f5f300c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f2ed9d700 1 --2- 172.21.15.7:0/3730219337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f300680f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009e60 tx=0x7f5f28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24d9759dee3e42a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f2dd9b700 1 -- 172.21.15.7:0/3730219337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800e070 con 0x7f5f300c9570 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f2dd9b700 1 -- 172.21.15.7:0/3730219337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2800bc30 con 0x7f5f300c9570 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f2dd9b700 1 -- 172.21.15.7:0/3730219337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28012430 con 0x7f5f300c9570 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3730219337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 msgr2=0x7f5f300680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3730219337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f300680f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009e60 tx=0x7f5f28009780 comp rx=0 tx=0).stop 2024-01-11T07:12:43.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3730219337 shutdown_connections 2024-01-11T07:12:43.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3730219337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f300680f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3730219337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159f70 0x7f5f300c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3730219337 >> 172.21.15.7:0/3730219337 conn(0x7f5f3006ae00 msgr2=0x7f5f3006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:43.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3730219337 shutdown_connections 2024-01-11T07:12:43.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.190+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3730219337 wait complete. 2024-01-11T07:12:43.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.192+0000 7f5f34ff9700 1 Processor -- start 2024-01-11T07:12:43.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.192+0000 7f5f34ff9700 1 -- start start 2024-01-11T07:12:43.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f34ff9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f30152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f34ff9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159ba0 0x7f5f300c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f34ff9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f30159f70 0x7f5f3015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f34ff9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300b2a80 con 0x7f5f300c9570 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f34ff9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300c6860 con 0x7f5f30159f70 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f34ff9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300b24c0 con 0x7f5f30159ba0 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f30152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f30152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52392/0 (socket says 172.21.15.7:52392) 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 -- 172.21.15.7:0/3608566436 learned_addr learned my addr 172.21.15.7:0/3608566436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159ba0 0x7f5f300c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2ed9d700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f30159f70 0x7f5f3015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 -- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159ba0 msgr2=0x7f5f300c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159ba0 0x7f5f300c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 -- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f30159f70 msgr2=0x7f5f3015b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f30159f70 0x7f5f3015b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 -- 172.21.15.7:0/3608566436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28009a20 con 0x7f5f300c9570 2024-01-11T07:12:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f2e59c700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f30152ee0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c7e0 tx=0x7f5f20007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f26ffd700 1 -- 172.21.15.7:0/3608566436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20005b30 con 0x7f5f300c9570 2024-01-11T07:12:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f26ffd700 1 -- 172.21.15.7:0/3608566436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20005c90 con 0x7f5f300c9570 2024-01-11T07:12:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f300b27a0 con 0x7f5f300c9570 2024-01-11T07:12:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.193+0000 7f5f26ffd700 1 -- 172.21.15.7:0/3608566436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20007300 con 0x7f5f300c9570 2024-01-11T07:12:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.194+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f300c5fe0 con 0x7f5f300c9570 2024-01-11T07:12:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.195+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f300c6350 con 0x7f5f300c9570 2024-01-11T07:12:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.195+0000 7f5f26ffd700 1 -- 172.21.15.7:0/3608566436 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5f2001b3b0 con 0x7f5f300c9570 2024-01-11T07:12:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.195+0000 7f5f26ffd700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f10066f30 0x7f5f100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.196+0000 7f5f26ffd700 1 -- 172.21.15.7:0/3608566436 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5f20083730 con 0x7f5f300c9570 2024-01-11T07:12:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.196+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f10066f30 0x7f5f100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.196+0000 7f5f25d9b700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f10066f30 0x7f5f100693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5f300b1b90 tx=0x7f5f18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.200+0000 7f5f26ffd700 1 -- 172.21.15.7:0/3608566436 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f300c6350 con 0x7f5f300c9570 2024-01-11T07:12:43.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.405+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5f300c6350 con 0x7f5f300c9570 2024-01-11T07:12:43.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.405+0000 7f5f26ffd700 1 -- 172.21.15.7:0/3608566436 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5f300c6350 con 0x7f5f300c9570 2024-01-11T07:12:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:43.405 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:12:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:12:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:12:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:12:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:12:43.406 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:12:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.407+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f10066f30 msgr2=0x7f5f100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.407+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f10066f30 0x7f5f100693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5f300b1b90 tx=0x7f5f18008040 comp rx=0 tx=0).stop 2024-01-11T07:12:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.407+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 msgr2=0x7f5f30152ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.407+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f30152ee0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c7e0 tx=0x7f5f20007df0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.408+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 shutdown_connections 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.408+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5f10066f30 0x7f5f100693e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.408+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f30159f70 0x7f5f3015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.408+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f300c9570 0x7f5f30152ee0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.408+0000 7f5f34ff9700 1 --2- 172.21.15.7:0/3608566436 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f30159ba0 0x7f5f300c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.408+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 >> 172.21.15.7:0/3608566436 conn(0x7f5f3006ae00 msgr2=0x7f5f300680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.408+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 shutdown_connections 2024-01-11T07:12:43.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.408+0000 7f5f34ff9700 1 -- 172.21.15.7:0/3608566436 wait complete. 2024-01-11T07:12:43.421 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:12:43.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:12:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.557+0000 7f6c91663700 1 Processor -- start 2024-01-11T07:12:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.557+0000 7f6c91663700 1 -- start start 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.558+0000 7f6c91663700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c143190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.558+0000 7f6c91663700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c0d03b0 con 0x7f6c8c06e250 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.558+0000 7f6c91663700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c0cd7d0 con 0x7f6c8c06de80 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.558+0000 7f6c91663700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0cf7a0 con 0x7f6c8c0cf230 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.558+0000 7f6c8b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6c8c06e250 0x7f6c8c0b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47758/0 (socket says 172.21.15.7:47758) 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.558+0000 7f6c8b7fe700 1 -- 172.21.15.7:0/1138079059 learned_addr learned my addr 172.21.15.7:0/1138079059 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.558+0000 7f6c8a7fc700 1 --2- 172.21.15.7:0/1138079059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c89ffb700 1 -- 172.21.15.7:0/1138079059 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992567885 0 0) 0x7f6c8c0d03b0 con 0x7f6c8c06e250 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c89ffb700 1 -- 172.21.15.7:0/1138079059 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c78003570 con 0x7f6c8c06e250 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c89ffb700 1 -- 172.21.15.7:0/1138079059 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575772878 0 0) 0x7f6c8c0cd7d0 con 0x7f6c8c06de80 2024-01-11T07:12:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c89ffb700 1 -- 172.21.15.7:0/1138079059 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c8c0d03b0 con 0x7f6c8c06de80 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c89ffb700 1 -- 172.21.15.7:0/1138079059 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2164060041 0 0) 0x7f6c78003570 con 0x7f6c8c06e250 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c8a7fc700 1 -- 172.21.15.7:0/1138079059 >> v1:172.21.15.113:6789/0 conn(0x7f6c8c06de80 legacy=0x7f6c8c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c8a7fc700 1 -- 172.21.15.7:0/1138079059 >> v1:172.21.15.7:6789/0 conn(0x7f6c8c06e250 legacy=0x7f6c8c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c8a7fc700 1 -- 172.21.15.7:0/1138079059 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c0cfa80 con 0x7f6c8c0cf230 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c8a7fc700 1 --2- 172.21.15.7:0/1138079059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c143190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009910 tx=0x7f6c740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16586330b04f17bd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c89ffb700 1 -- 172.21.15.7:0/1138079059 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c840017f0 con 0x7f6c8c0cf230 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c89ffb700 1 -- 172.21.15.7:0/1138079059 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c74011420 con 0x7f6c8c0cf230 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.559+0000 7f6c89ffb700 1 -- 172.21.15.7:0/1138079059 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740117c0 con 0x7f6c8c0cf230 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.560+0000 7f6c91663700 1 -- 172.21.15.7:0/1138079059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 msgr2=0x7f6c8c143190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.560+0000 7f6c91663700 1 --2- 172.21.15.7:0/1138079059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c143190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009910 tx=0x7f6c740091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.560+0000 7f6c91663700 1 -- 172.21.15.7:0/1138079059 shutdown_connections 2024-01-11T07:12:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.560+0000 7f6c91663700 1 --2- 172.21.15.7:0/1138079059 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c143190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.560+0000 7f6c91663700 1 -- 172.21.15.7:0/1138079059 >> 172.21.15.7:0/1138079059 conn(0x7f6c8c070690 msgr2=0x7f6c8c070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.560+0000 7f6c91663700 1 -- 172.21.15.7:0/1138079059 shutdown_connections 2024-01-11T07:12:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.560+0000 7f6c91663700 1 -- 172.21.15.7:0/1138079059 wait complete. 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.561+0000 7f6c91663700 1 Processor -- start 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.561+0000 7f6c91663700 1 -- start start 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c91663700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8c06de80 0x7f6c8c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c91663700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c8c06e250 0x7f6c8c0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c91663700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c91663700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0cc8c0 con 0x7f6c8c06e250 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c91663700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0c6dd0 con 0x7f6c8c06de80 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c91663700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0cc300 con 0x7f6c8c0cf230 2024-01-11T07:12:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c8a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c8c06e250 0x7f6c8c0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c8a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c8c06e250 0x7f6c8c0cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52404/0 (socket says 172.21.15.7:52404) 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c8a7fc700 1 -- 172.21.15.7:0/3201254906 learned_addr learned my addr 172.21.15.7:0/3201254906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.562+0000 7f6c8affd700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8c06de80 0x7f6c8c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c8b7fe700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c8a7fc700 1 -- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 msgr2=0x7f6c8c169f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c8a7fc700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c169f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c8a7fc700 1 -- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8c06de80 msgr2=0x7f6c8c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c8a7fc700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8c06de80 0x7f6c8c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c8a7fc700 1 -- 172.21.15.7:0/3201254906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c74009420 con 0x7f6c8c06e250 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c8b7fe700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c8a7fc700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c8c06e250 0x7f6c8c0cbaa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6c740091e0 tx=0x7f6c74016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c82ffd700 1 -- 172.21.15.7:0/3201254906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7401cbe0 con 0x7f6c8c06e250 2024-01-11T07:12:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c82ffd700 1 -- 172.21.15.7:0/3201254906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7401cd40 con 0x7f6c8c06e250 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c0cc5e0 con 0x7f6c8c06e250 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c82ffd700 1 -- 172.21.15.7:0/3201254906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7401a6a0 con 0x7f6c8c06e250 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.563+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c0cbfe0 con 0x7f6c8c06e250 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.564+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c0c6f80 con 0x7f6c8c06e250 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.565+0000 7f6c82ffd700 1 -- 172.21.15.7:0/3201254906 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6c7401aad0 con 0x7f6c8c06e250 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.565+0000 7f6c82ffd700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c6c066f30 0x7f6c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.565+0000 7f6c82ffd700 1 -- 172.21.15.7:0/3201254906 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6c74085e40 con 0x7f6c8c06e250 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.565+0000 7f6c8affd700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c6c066f30 0x7f6c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.566+0000 7f6c8affd700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c6c066f30 0x7f6c6c0693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c160ce0 tx=0x7f6c7c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:43.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.569+0000 7f6c82ffd700 1 -- 172.21.15.7:0/3201254906 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c8c0c6f80 con 0x7f6c8c06e250 2024-01-11T07:12:43.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:43.773+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6c8c0c6f40 con 0x7f6c8c06e250 2024-01-11T07:12:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.468+0000 7f6c82ffd700 1 -- 172.21.15.7:0/3201254906 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6c74053400 con 0x7f6c8c06e250 2024-01-11T07:12:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c6c066f30 msgr2=0x7f6c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c6c066f30 0x7f6c6c0693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c160ce0 tx=0x7f6c7c007d50 comp rx=0 tx=0).stop 2024-01-11T07:12:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c8c06e250 msgr2=0x7f6c8c0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c8c06e250 0x7f6c8c0cbaa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6c740091e0 tx=0x7f6c74016000 comp rx=0 tx=0).stop 2024-01-11T07:12:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 shutdown_connections 2024-01-11T07:12:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c6c066f30 0x7f6c6c0693e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8c0cf230 0x7f6c8c169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8c06de80 0x7f6c8c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 --2- 172.21.15.7:0/3201254906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c8c06e250 0x7f6c8c0cbaa0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 >> 172.21.15.7:0/3201254906 conn(0x7f6c8c070690 msgr2=0x7f6c8c070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 shutdown_connections 2024-01-11T07:12:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.473+0000 7f6c91663700 1 -- 172.21.15.7:0/3201254906 wait complete. 2024-01-11T07:12:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:12:44.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:12:44.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.632+0000 7f011ed57700 1 Processor -- start 2024-01-11T07:12:44.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.632+0000 7f011ed57700 1 -- start start 2024-01-11T07:12:44.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f011ed57700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0118063230 0x7f0118063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f011ed57700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f011ed57700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011815d880 con 0x7f01180ac5b0 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f011ed57700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011806c5b0 con 0x7f0118063230 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f011ed57700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0118063b40 con 0x7f011815a280 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f011caf3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0118063230 0x7f0118063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f011caf3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0118063230 0x7f0118063600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52420/0 (socket says 172.21.15.7:52420) 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f011caf3700 1 -- 172.21.15.7:0/2300650559 learned_addr learned my addr 172.21.15.7:0/2300650559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.633+0000 7f0117fff700 1 --2- 172.21.15.7:0/2300650559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.634+0000 7f011caf3700 1 -- 172.21.15.7:0/2300650559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 msgr2=0x7f01180aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.634+0000 7f011caf3700 1 --2- 172.21.15.7:0/2300650559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.634+0000 7f011caf3700 1 -- 172.21.15.7:0/2300650559 >> v1:172.21.15.113:6789/0 conn(0x7f01180ac5b0 legacy=0x7f01180aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.634+0000 7f011caf3700 1 -- 172.21.15.7:0/2300650559 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011815a7f0 con 0x7f0118063230 2024-01-11T07:12:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.634+0000 7f011caf3700 1 --2- 172.21.15.7:0/2300650559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118063230 0x7f0118063600 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f01080099e0 tx=0x7f0108009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6d6fa1edc38f35 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.634+0000 7f01177fe700 1 -- 172.21.15.7:0/2300650559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800b820 con 0x7f0118063230 2024-01-11T07:12:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.634+0000 7f01177fe700 1 -- 172.21.15.7:0/2300650559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010800b980 con 0x7f0118063230 2024-01-11T07:12:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.634+0000 7f01177fe700 1 -- 172.21.15.7:0/2300650559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010800bd20 con 0x7f0118063230 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.635+0000 7f011ed57700 1 -- 172.21.15.7:0/2300650559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118063230 msgr2=0x7f0118063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.635+0000 7f011ed57700 1 --2- 172.21.15.7:0/2300650559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118063230 0x7f0118063600 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f01080099e0 tx=0x7f0108009250 comp rx=0 tx=0).stop 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.635+0000 7f011ed57700 1 -- 172.21.15.7:0/2300650559 shutdown_connections 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.635+0000 7f011ed57700 1 --2- 172.21.15.7:0/2300650559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0118063230 0x7f0118063600 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.635+0000 7f011ed57700 1 --2- 172.21.15.7:0/2300650559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.635+0000 7f011ed57700 1 -- 172.21.15.7:0/2300650559 >> 172.21.15.7:0/2300650559 conn(0x7f01180680e0 msgr2=0x7f01180cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.635+0000 7f011ed57700 1 -- 172.21.15.7:0/2300650559 shutdown_connections 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.635+0000 7f011ed57700 1 -- 172.21.15.7:0/2300650559 wait complete. 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.636+0000 7f011ed57700 1 Processor -- start 2024-01-11T07:12:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.636+0000 7f011ed57700 1 -- start start 2024-01-11T07:12:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011ed57700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0118063230 0x7f0118153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011ed57700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01180ac5b0 0x7f01180afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011ed57700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011ed57700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0118154380 con 0x7f01180ac5b0 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011ed57700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011815f750 con 0x7f0118063230 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011ed57700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01180b0050 con 0x7f011815a280 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01180ac5b0 0x7f01180afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01180ac5b0 0x7f01180afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52428/0 (socket says 172.21.15.7:52428) 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 -- 172.21.15.7:0/3950988024 learned_addr learned my addr 172.21.15.7:0/3950988024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011caf3700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0118063230 0x7f0118153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011d2f4700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 -- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 msgr2=0x7f01180b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 -- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0118063230 msgr2=0x7f0118153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0118063230 0x7f0118153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 -- 172.21.15.7:0/3950988024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080094f0 con 0x7f01180ac5b0 2024-01-11T07:12:44.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f011d2f4700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.637+0000 7f0117fff700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01180ac5b0 0x7f01180afb10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f010c00d890 tx=0x7f010c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.638+0000 7f0114ff9700 1 -- 172.21.15.7:0/3950988024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00b600 con 0x7f01180ac5b0 2024-01-11T07:12:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.638+0000 7f0114ff9700 1 -- 172.21.15.7:0/3950988024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c00b760 con 0x7f01180ac5b0 2024-01-11T07:12:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.638+0000 7f0114ff9700 1 -- 172.21.15.7:0/3950988024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00bae0 con 0x7f01180ac5b0 2024-01-11T07:12:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.638+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01180af0f0 con 0x7f01180ac5b0 2024-01-11T07:12:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.638+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011815ee20 con 0x7f01180ac5b0 2024-01-11T07:12:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.639+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011815f900 con 0x7f01180ac5b0 2024-01-11T07:12:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.639+0000 7f0114ff9700 1 -- 172.21.15.7:0/3950988024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f010c01e460 con 0x7f01180ac5b0 2024-01-11T07:12:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.640+0000 7f0114ff9700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0104066f30 0x7f01040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.640+0000 7f0114ff9700 1 -- 172.21.15.7:0/3950988024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f010c084ad0 con 0x7f01180ac5b0 2024-01-11T07:12:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.640+0000 7f011caf3700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0104066f30 0x7f01040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.641+0000 7f011caf3700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0104066f30 0x7f01040693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f01080099e0 tx=0x7f0108017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:44.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.645+0000 7f0114ff9700 1 -- 172.21.15.7:0/3950988024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f011815f900 con 0x7f01180ac5b0 2024-01-11T07:12:44.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:44.847+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f01180aae60 con 0x7f01180ac5b0 2024-01-11T07:12:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.476+0000 7f0114ff9700 1 -- 172.21.15.7:0/3950988024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7f010c055910 con 0x7f01180ac5b0 2024-01-11T07:12:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.487+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0104066f30 msgr2=0x7f01040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.487+0000 7f011ed57700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0104066f30 0x7f01040693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f01080099e0 tx=0x7f0108017000 comp rx=0 tx=0).stop 2024-01-11T07:12:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.487+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01180ac5b0 msgr2=0x7f01180afb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:45.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.487+0000 7f011ed57700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01180ac5b0 0x7f01180afb10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f010c00d890 tx=0x7f010c008040 comp rx=0 tx=0).stop 2024-01-11T07:12:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.488+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 shutdown_connections 2024-01-11T07:12:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.488+0000 7f011ed57700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0104066f30 0x7f01040693e0 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f01080099e0 tx=0x7f0108017000 comp rx=0 tx=0).stop 2024-01-11T07:12:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.488+0000 7f011ed57700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f011815a280 0x7f01180b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.488+0000 7f011ed57700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0118063230 0x7f0118153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.488+0000 7f011ed57700 1 --2- 172.21.15.7:0/3950988024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01180ac5b0 0x7f01180afb10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.488+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 >> 172.21.15.7:0/3950988024 conn(0x7f01180680e0 msgr2=0x7f011815b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.488+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 shutdown_connections 2024-01-11T07:12:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.488+0000 7f011ed57700 1 -- 172.21.15.7:0/3950988024 wait complete. 2024-01-11T07:12:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:12:45.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:12:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.642+0000 7f0082cec700 1 Processor -- start 2024-01-11T07:12:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.642+0000 7f0082cec700 1 -- start start 2024-01-11T07:12:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f0082cec700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 0x7f007c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f0082cec700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f007c15e870 0x7f007c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f0082cec700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007c0ad520 con 0x7f007c15df60 2024-01-11T07:12:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f0082cec700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007c0cdbf0 con 0x7f007c15e870 2024-01-11T07:12:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f0082cec700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c06a6f0 con 0x7f007c0cd780 2024-01-11T07:12:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f0080a88700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f007c15df60 0x7f007c0afe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47770/0 (socket says 172.21.15.7:47770) 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f0080a88700 1 -- 172.21.15.7:0/747040638 learned_addr learned my addr 172.21.15.7:0/747040638 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007bfff700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 0x7f007c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f0081289700 1 --2- 172.21.15.7:0/747040638 >> v2:172.21.15.113:3300/0 conn(0x7f007c15e870 0x7f007c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036692646 0 0) 0x7f007c0ad520 con 0x7f007c15df60 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f006c003570 con 0x7f007c15df60 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2701762086 0 0) 0x7f006c003570 con 0x7f007c15df60 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 msgr2=0x7f007c15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 0x7f007c15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 >> v2:172.21.15.113:3300/0 conn(0x7f007c15e870 msgr2=0x7f007c0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 --2- 172.21.15.7:0/747040638 >> v2:172.21.15.113:3300/0 conn(0x7f007c15e870 0x7f007c0ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c0b0520 con 0x7f007c15df60 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f0070001470 con 0x7f007c15df60 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007bfff700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 0x7f007c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.643+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 >> v1:172.21.15.7:6789/0 conn(0x7f007c15df60 legacy=0x7f007c0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007b7fe700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 0x7f006c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007b7fe700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f006c003fa0 0x7f006c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007b7fe700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f006c004d50 0x7f006c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0cdbf0 con 0x7f006c003fa0 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c06a6f0 con 0x7f006c004d50 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c00ba60 con 0x7f007c0cd780 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007bfff700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f006c003fa0 0x7f006c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f0080a88700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 0x7f006c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f0081289700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f006c004d50 0x7f006c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007bfff700 1 -- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 msgr2=0x7f006c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007bfff700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f007c0cd780 0x7f006c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007bfff700 1 -- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f006c004d50 msgr2=0x7f006c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007bfff700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f006c004d50 0x7f006c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007bfff700 1 -- 172.21.15.7:0/747040638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068009420 con 0x7f006c003fa0 2024-01-11T07:12:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007bfff700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f006c003fa0 0x7f006c004810 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0068009df0 tx=0x7f00680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3053a736c6ec4b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.644+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068009740 con 0x7f006c003fa0 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006800e920 con 0x7f006c003fa0 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f007b7fe700 1 -- 172.21.15.7:0/747040638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006800ece0 con 0x7f006c003fa0 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f0082cec700 1 -- 172.21.15.7:0/747040638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f006c003fa0 msgr2=0x7f006c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f0082cec700 1 --2- 172.21.15.7:0/747040638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f006c003fa0 0x7f006c004810 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0068009df0 tx=0x7f00680091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f0081289700 1 -- 172.21.15.7:0/747040638 reap_dead start 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f0082cec700 1 -- 172.21.15.7:0/747040638 shutdown_connections 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f0082cec700 1 -- 172.21.15.7:0/747040638 >> 172.21.15.7:0/747040638 conn(0x7f007c0680a0 msgr2=0x7f007c0cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f0082cec700 1 -- 172.21.15.7:0/747040638 shutdown_connections 2024-01-11T07:12:45.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.645+0000 7f0082cec700 1 -- 172.21.15.7:0/747040638 wait complete. 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.646+0000 7f0082cec700 1 Processor -- start 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.647+0000 7f0082cec700 1 -- start start 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.647+0000 7f0082cec700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c003fa0 0x7f007c0b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.647+0000 7f0082cec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c0b1fc0 0x7f007c14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.647+0000 7f0082cec700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f007c152eb0 0x7f007c153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.647+0000 7f0082cec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c156ba0 con 0x7f007c0b1fc0 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.647+0000 7f0082cec700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c153860 con 0x7f007c152eb0 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.647+0000 7f0082cec700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0b1610 con 0x7f006c003fa0 2024-01-11T07:12:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c0b1fc0 0x7f007c14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c0b1fc0 0x7f007c14dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52438/0 (socket says 172.21.15.7:52438) 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 -- 172.21.15.7:0/982608069 learned_addr learned my addr 172.21.15.7:0/982608069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f0080a88700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c003fa0 0x7f007c0b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f0081289700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f007c152eb0 0x7f007c153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 -- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c003fa0 msgr2=0x7f007c0b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c003fa0 0x7f007c0b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 -- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f007c152eb0 msgr2=0x7f007c153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f007c152eb0 0x7f007c153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 -- 172.21.15.7:0/982608069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c14e1f0 con 0x7f007c0b1fc0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f007bfff700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c0b1fc0 0x7f007c14dcb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f006c0046f0 tx=0x7f006801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f0078ff9700 1 -- 172.21.15.7:0/982608069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068022070 con 0x7f007c0b1fc0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f0078ff9700 1 -- 172.21.15.7:0/982608069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0068013ab0 con 0x7f007c0b1fc0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007c0b2510 con 0x7f007c0b1fc0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f0078ff9700 1 -- 172.21.15.7:0/982608069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068013e70 con 0x7f007c0b1fc0 2024-01-11T07:12:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.648+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c0b2ad0 con 0x7f007c0b1fc0 2024-01-11T07:12:45.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.649+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007c14fa20 con 0x7f007c0b1fc0 2024-01-11T07:12:45.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.650+0000 7f0078ff9700 1 -- 172.21.15.7:0/982608069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0068026070 con 0x7f007c0b1fc0 2024-01-11T07:12:45.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.650+0000 7f0078ff9700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0060066f30 0x7f00600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:45.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.650+0000 7f0078ff9700 1 -- 172.21.15.7:0/982608069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0068085cd0 con 0x7f007c0b1fc0 2024-01-11T07:12:45.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.651+0000 7f0080a88700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0060066f30 0x7f00600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:45.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.651+0000 7f0080a88700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0060066f30 0x7f00600693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0070000e50 tx=0x7f007c0ae890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:45.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.655+0000 7f0078ff9700 1 -- 172.21.15.7:0/982608069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f007c14fa20 con 0x7f007c0b1fc0 2024-01-11T07:12:45.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:45.903+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f007c14fa20 con 0x7f007c0b1fc0 2024-01-11T07:12:46.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.487+0000 7f0078ff9700 1 -- 172.21.15.7:0/982608069 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f007c14fa20 con 0x7f007c0b1fc0 2024-01-11T07:12:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0060066f30 msgr2=0x7f00600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0060066f30 0x7f00600693e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0070000e50 tx=0x7f007c0ae890 comp rx=0 tx=0).stop 2024-01-11T07:12:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c0b1fc0 msgr2=0x7f007c14dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c0b1fc0 0x7f007c14dcb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f006c0046f0 tx=0x7f006801a000 comp rx=0 tx=0).stop 2024-01-11T07:12:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 shutdown_connections 2024-01-11T07:12:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0060066f30 0x7f00600693e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f007c152eb0 0x7f007c153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f006c003fa0 0x7f007c0b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 --2- 172.21.15.7:0/982608069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f007c0b1fc0 0x7f007c14dcb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 >> 172.21.15.7:0/982608069 conn(0x7f007c0680a0 msgr2=0x7f007c06d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 shutdown_connections 2024-01-11T07:12:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.506+0000 7f0082cec700 1 -- 172.21.15.7:0/982608069 wait complete. 2024-01-11T07:12:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 14 and data pool 15 2024-01-11T07:12:46.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.657+0000 7fe313f7e700 1 Processor -- start 2024-01-11T07:12:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.657+0000 7fe313f7e700 1 -- start start 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.658+0000 7fe313f7e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe30c158280 0x7fe30c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.658+0000 7fe313f7e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c159300 con 0x7fe30c0cafb0 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.658+0000 7fe313f7e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c0cf000 con 0x7fe30c15aa80 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.658+0000 7fe313f7e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c0cf260 con 0x7fe30c158280 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.658+0000 7fe311d1a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe30c0cafb0 0x7fe30c0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47778/0 (socket says 172.21.15.7:47778) 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.658+0000 7fe311d1a700 1 -- 172.21.15.7:0/3942159750 learned_addr learned my addr 172.21.15.7:0/3942159750 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe311519700 1 --2- 172.21.15.7:0/3942159750 >> v2:172.21.15.113:3300/0 conn(0x7fe30c158280 0x7fe30c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561534241 0 0) 0x7fe30c159300 con 0x7fe30c0cafb0 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f8003570 con 0x7fe30c0cafb0 2024-01-11T07:12:46.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744873766 0 0) 0x7fe2f8003570 con 0x7fe30c0cafb0 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 >> v2:172.21.15.113:3300/0 conn(0x7fe30c158280 msgr2=0x7fe30c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 --2- 172.21.15.7:0/3942159750 >> v2:172.21.15.113:3300/0 conn(0x7fe30c158280 0x7fe30c0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 >> v1:172.21.15.113:6789/0 conn(0x7fe30c15aa80 legacy=0x7fe30c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0cf540 con 0x7fe30c0cafb0 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fe2fc001470 con 0x7fe30c0cafb0 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 >> v1:172.21.15.7:6789/0 conn(0x7fe30c0cafb0 legacy=0x7fe30c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe2f8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2f8004810 0x7fe2f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f80051c0 0x7fe2f800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0cf260 con 0x7fe2f8003a60 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f800c0d0 con 0x7fe2f80051c0 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f800c290 con 0x7fe2f8004810 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe311519700 1 --2- 172.21.15.7:0/3942159750 >> v2:172.21.15.113:3300/0 conn(0x7fe30c158280 0x7fe30c0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe311d1a700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe2f8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe31251b700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f80051c0 0x7fe2f800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.659+0000 7fe311519700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2f8004810 0x7fe2f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe311d1a700 1 -- 172.21.15.7:0/3942159750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2f8004810 msgr2=0x7fe2f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe311d1a700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe2f8004810 0x7fe2f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe311d1a700 1 -- 172.21.15.7:0/3942159750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f80051c0 msgr2=0x7fe2f800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe311d1a700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2f80051c0 0x7fe2f800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe311d1a700 1 -- 172.21.15.7:0/3942159750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f8003570 con 0x7fe2f8003a60 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe31251b700 1 -- 172.21.15.7:0/3942159750 reap_dead start 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe311d1a700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe2f8003e70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c390 tx=0x7fe30c158130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350d5a402b0b9037 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00cc60 con 0x7fe2f8003a60 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc00cdc0 con 0x7fe2f8003a60 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe310d18700 1 -- 172.21.15.7:0/3942159750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc005b70 con 0x7fe2f8003a60 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe313f7e700 1 -- 172.21.15.7:0/3942159750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 msgr2=0x7fe2f8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe313f7e700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe2f8003e70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c390 tx=0x7fe30c158130 comp rx=0 tx=0).stop 2024-01-11T07:12:46.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe313f7e700 1 -- 172.21.15.7:0/3942159750 shutdown_connections 2024-01-11T07:12:46.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe313f7e700 1 --2- 172.21.15.7:0/3942159750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe2f8003e70 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.660+0000 7fe313f7e700 1 -- 172.21.15.7:0/3942159750 >> 172.21.15.7:0/3942159750 conn(0x7fe30c071100 msgr2=0x7fe30c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:46.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.661+0000 7fe313f7e700 1 -- 172.21.15.7:0/3942159750 shutdown_connections 2024-01-11T07:12:46.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.661+0000 7fe313f7e700 1 -- 172.21.15.7:0/3942159750 wait complete. 2024-01-11T07:12:46.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.662+0000 7fe313f7e700 1 Processor -- start 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.662+0000 7fe313f7e700 1 -- start start 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe313f7e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe30c1476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe313f7e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0b03e0 0x7fe30c0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe313f7e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c0b2d10 0x7fe30c067610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe313f7e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0b3190 con 0x7fe2f8003a60 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe313f7e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c069250 con 0x7fe30c0b03e0 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe313f7e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0ca150 con 0x7fe30c0b2d10 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe30c1476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe30c1476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52454/0 (socket says 172.21.15.7:52454) 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 -- 172.21.15.7:0/355692581 learned_addr learned my addr 172.21.15.7:0/355692581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311519700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0b03e0 0x7fe30c0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe31251b700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c0b2d10 0x7fe30c067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 -- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c0b2d10 msgr2=0x7fe30c067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c0b2d10 0x7fe30c067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 -- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0b03e0 msgr2=0x7fe30c0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0b03e0 0x7fe30c0b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 -- 172.21.15.7:0/355692581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0ca430 con 0x7fe2f8003a60 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe311d1a700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe30c1476b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c0a0 tx=0x7fe2fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe301ffb700 1 -- 172.21.15.7:0/355692581 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc000bc0 con 0x7fe2f8003a60 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe301ffb700 1 -- 172.21.15.7:0/355692581 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc0070d0 con 0x7fe2f8003a60 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c0ae390 con 0x7fe2f8003a60 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.663+0000 7fe301ffb700 1 -- 172.21.15.7:0/355692581 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc007490 con 0x7fe2f8003a60 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.664+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c0c9e40 con 0x7fe2f8003a60 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.665+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c04c3c0 con 0x7fe2f8003a60 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.665+0000 7fe301ffb700 1 -- 172.21.15.7:0/355692581 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe2fc028460 con 0x7fe2f8003a60 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.666+0000 7fe301ffb700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.666+0000 7fe301ffb700 1 -- 172.21.15.7:0/355692581 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2fc085500 con 0x7fe2f8003a60 2024-01-11T07:12:46.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.666+0000 7fe311519700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:46.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.666+0000 7fe311519700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2f0066f30 0x7fe2f00693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe30c158650 tx=0x7fe308008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:46.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.670+0000 7fe301ffb700 1 -- 172.21.15.7:0/355692581 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2fc052830 con 0x7fe2f8003a60 2024-01-11T07:12:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.875+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe30c0cf820 con 0x7fe2f8003a60 2024-01-11T07:12:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.876+0000 7fe301ffb700 1 -- 172.21.15.7:0/355692581 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+15330 (secure 0 0 0) 0x7fe2fc055e50 con 0x7fe2f8003a60 2024-01-11T07:12:46.876 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:46.876 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":119,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:45.905416+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:12:43.805689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-11T07:12:44.880093+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:46.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2f0066f30 msgr2=0x7fe2f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2f0066f30 0x7fe2f00693e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe30c158650 tx=0x7fe308008040 comp rx=0 tx=0).stop 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 msgr2=0x7fe30c1476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe30c1476b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c0a0 tx=0x7fe2fc015000 comp rx=0 tx=0).stop 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 shutdown_connections 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c0b2d10 0x7fe30c067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f8003a60 0x7fe30c1476b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 --2- 172.21.15.7:0/355692581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0b03e0 0x7fe30c0b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 >> 172.21.15.7:0/355692581 conn(0x7fe30c071100 msgr2=0x7fe30c0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 shutdown_connections 2024-01-11T07:12:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:46.878+0000 7fe313f7e700 1 -- 172.21.15.7:0/355692581 wait complete. 2024-01-11T07:12:46.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.024+0000 7fc3032fd700 1 Processor -- start 2024-01-11T07:12:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.024+0000 7fc3032fd700 1 -- start start 2024-01-11T07:12:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.025+0000 7fc3032fd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.025+0000 7fc3032fd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2fc0b1820 0x7fc2fc0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.025+0000 7fc3032fd700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc161b90 con 0x7fc2fc0b0430 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.025+0000 7fc3032fd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc0b36c0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.025+0000 7fc3032fd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0b4950 con 0x7fc2fc1617c0 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.025+0000 7fc300898700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2fc0b1820 0x7fc2fc0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.025+0000 7fc300898700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2fc0b1820 0x7fc2fc0b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52468/0 (socket says 172.21.15.7:52468) 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.025+0000 7fc300898700 1 -- 172.21.15.7:0/2434831759 learned_addr learned my addr 172.21.15.7:0/2434831759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc301099700 1 --2- 172.21.15.7:0/2434831759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc300898700 1 -- 172.21.15.7:0/2434831759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 msgr2=0x7fc2fc0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc300898700 1 --2- 172.21.15.7:0/2434831759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc300898700 1 -- 172.21.15.7:0/2434831759 >> v1:172.21.15.113:6789/0 conn(0x7fc2fc0b0430 legacy=0x7fc2fc067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc300898700 1 -- 172.21.15.7:0/2434831759 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0b3930 con 0x7fc2fc0b1820 2024-01-11T07:12:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc301099700 1 --2- 172.21.15.7:0/2434831759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc300898700 1 --2- 172.21.15.7:0/2434831759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc0b4410 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0099e0 tx=0x7fc2ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee58a6c8433500c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc2f3fff700 1 -- 172.21.15.7:0/2434831759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00b7e0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc2f3fff700 1 -- 172.21.15.7:0/2434831759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec00b940 con 0x7fc2fc0b1820 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.026+0000 7fc2f3fff700 1 -- 172.21.15.7:0/2434831759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00bce0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.027+0000 7fc3032fd700 1 -- 172.21.15.7:0/2434831759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 msgr2=0x7fc2fc0b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.027+0000 7fc3032fd700 1 --2- 172.21.15.7:0/2434831759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc0b4410 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0099e0 tx=0x7fc2ec009250 comp rx=0 tx=0).stop 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.027+0000 7fc3032fd700 1 -- 172.21.15.7:0/2434831759 shutdown_connections 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.027+0000 7fc3032fd700 1 --2- 172.21.15.7:0/2434831759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.027+0000 7fc3032fd700 1 --2- 172.21.15.7:0/2434831759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc0b4410 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.027+0000 7fc3032fd700 1 -- 172.21.15.7:0/2434831759 >> 172.21.15.7:0/2434831759 conn(0x7fc2fc071100 msgr2=0x7fc2fc06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.027+0000 7fc3032fd700 1 -- 172.21.15.7:0/2434831759 shutdown_connections 2024-01-11T07:12:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.027+0000 7fc3032fd700 1 -- 172.21.15.7:0/2434831759 wait complete. 2024-01-11T07:12:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.028+0000 7fc3032fd700 1 Processor -- start 2024-01-11T07:12:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.028+0000 7fc3032fd700 1 -- start start 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc3032fd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2fc0b0430 0x7fc2fc0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc3032fd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc172ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc3032fd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc3032fd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0a81b0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc3032fd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0a8410 con 0x7fc2fc0b0430 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc3032fd700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0a86f0 con 0x7fc2fc1617c0 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc300898700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc172ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc300898700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc172ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52478/0 (socket says 172.21.15.7:52478) 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc300898700 1 -- 172.21.15.7:0/416270971 learned_addr learned my addr 172.21.15.7:0/416270971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc301099700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2fc0b0430 0x7fc2fc0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc30189a700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc300898700 1 -- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 msgr2=0x7fc2fc0a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc300898700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0a7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc300898700 1 -- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2fc0b0430 msgr2=0x7fc2fc0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc300898700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2fc0b0430 0x7fc2fc0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc300898700 1 -- 172.21.15.7:0/416270971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0094f0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc301099700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2fc0b0430 0x7fc2fc0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.029+0000 7fc30189a700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.030+0000 7fc300898700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc172ba0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc06a1d0 tx=0x7fc2ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.030+0000 7fc2f17fa700 1 -- 172.21.15.7:0/416270971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec011dd0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.030+0000 7fc2f17fa700 1 -- 172.21.15.7:0/416270971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec020070 con 0x7fc2fc0b1820 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.030+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc1720c0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.030+0000 7fc2f17fa700 1 -- 172.21.15.7:0/416270971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01b680 con 0x7fc2fc0b1820 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.030+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc172680 con 0x7fc2fc0b1820 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.031+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc04c3c0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.031+0000 7fc2f17fa700 1 -- 172.21.15.7:0/416270971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc2ec022030 con 0x7fc2fc0b1820 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.032+0000 7fc2f17fa700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc2e0066f30 0x7fc2e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.032+0000 7fc2f17fa700 1 -- 172.21.15.7:0/416270971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2ec086510 con 0x7fc2fc0b1820 2024-01-11T07:12:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.032+0000 7fc301099700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc2e0066f30 0x7fc2e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.033+0000 7fc301099700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc2e0066f30 0x7fc2e00693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006fd0 tx=0x7fc2f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.036+0000 7fc2f17fa700 1 -- 172.21.15.7:0/416270971 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2ec053780 con 0x7fc2fc0b1820 2024-01-11T07:12:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.244+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2fc04c3c0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.245+0000 7fc2f17fa700 1 -- 172.21.15.7:0/416270971 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+15330 (secure 0 0 0) 0x7fc2ec056da0 con 0x7fc2fc0b1820 2024-01-11T07:12:47.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:47.246 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":119,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:45.905416+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:12:43.805689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-11T07:12:44.880093+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.247+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc2e0066f30 msgr2=0x7fc2e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.247+0000 7fc3032fd700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc2e0066f30 0x7fc2e00693e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006fd0 tx=0x7fc2f8008040 comp rx=0 tx=0).stop 2024-01-11T07:12:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.247+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 msgr2=0x7fc2fc172ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.247+0000 7fc3032fd700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc172ba0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc06a1d0 tx=0x7fc2ec016000 comp rx=0 tx=0).stop 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.247+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 shutdown_connections 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.247+0000 7fc3032fd700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc2e0066f30 0x7fc2e00693e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.247+0000 7fc3032fd700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc2fc1617c0 0x7fc2fc0a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.247+0000 7fc3032fd700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2fc0b0430 0x7fc2fc0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.248+0000 7fc3032fd700 1 --2- 172.21.15.7:0/416270971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2fc0b1820 0x7fc2fc172ba0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.248+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 >> 172.21.15.7:0/416270971 conn(0x7fc2fc071100 msgr2=0x7fc2fc067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.248+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 shutdown_connections 2024-01-11T07:12:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.248+0000 7fc3032fd700 1 -- 172.21.15.7:0/416270971 wait complete. 2024-01-11T07:12:47.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:12:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.395+0000 7fae5f5ce700 1 Processor -- start 2024-01-11T07:12:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.396+0000 7fae5f5ce700 1 -- start start 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.396+0000 7fae5f5ce700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae58157a90 0x7fae58157e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5f5ce700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae580cf1b0 0x7fae580c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5f5ce700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580ca3d0 con 0x7fae580cf5f0 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5f5ce700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae580b1790 con 0x7fae58157a90 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5f5ce700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580cd9a0 con 0x7fae580cf1b0 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5d36a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae58157a90 0x7fae58157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5d36a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fae58157a90 0x7fae58157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60914/0 (socket says 172.21.15.7:60914) 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5d36a700 1 -- 172.21.15.7:0/1026192549 learned_addr learned my addr 172.21.15.7:0/1026192549 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5cb69700 1 --2- 172.21.15.7:0/1026192549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae580cf1b0 0x7fae580c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5d36a700 1 -- 172.21.15.7:0/1026192549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae580cf1b0 msgr2=0x7fae580c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5d36a700 1 --2- 172.21.15.7:0/1026192549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae580cf1b0 0x7fae580c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5d36a700 1 -- 172.21.15.7:0/1026192549 >> v1:172.21.15.113:6789/0 conn(0x7fae580cf5f0 legacy=0x7fae580cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5d36a700 1 -- 172.21.15.7:0/1026192549 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580cdc80 con 0x7fae58157a90 2024-01-11T07:12:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae5d36a700 1 --2- 172.21.15.7:0/1026192549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae58157a90 0x7fae58157e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fae54009990 tx=0x7fae54009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c6663e9fae4c15 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.397+0000 7fae4ffff700 1 -- 172.21.15.7:0/1026192549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54005c10 con 0x7fae58157a90 2024-01-11T07:12:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae4ffff700 1 -- 172.21.15.7:0/1026192549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae54005d70 con 0x7fae58157a90 2024-01-11T07:12:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae4ffff700 1 -- 172.21.15.7:0/1026192549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400b880 con 0x7fae58157a90 2024-01-11T07:12:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1026192549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae58157a90 msgr2=0x7fae58157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1026192549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae58157a90 0x7fae58157e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fae54009990 tx=0x7fae54009220 comp rx=0 tx=0).stop 2024-01-11T07:12:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1026192549 shutdown_connections 2024-01-11T07:12:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1026192549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae58157a90 0x7fae58157e60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1026192549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae580cf1b0 0x7fae580c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1026192549 >> 172.21.15.7:0/1026192549 conn(0x7fae58097a50 msgr2=0x7fae580ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1026192549 shutdown_connections 2024-01-11T07:12:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.398+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1026192549 wait complete. 2024-01-11T07:12:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.399+0000 7fae5f5ce700 1 Processor -- start 2024-01-11T07:12:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.399+0000 7fae5f5ce700 1 -- start start 2024-01-11T07:12:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.400+0000 7fae5f5ce700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae580cf1b0 0x7fae58152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.400+0000 7fae5f5ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae580cf5f0 0x7fae5809ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.400+0000 7fae5f5ce700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae58157a90 0x7fae5809b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.400+0000 7fae5f5ce700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5809c140 con 0x7fae580cf5f0 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.400+0000 7fae5f5ce700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5809b8a0 con 0x7fae580cf1b0 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.400+0000 7fae5f5ce700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5809bb80 con 0x7fae58157a90 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae580cf5f0 0x7fae5809ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae580cf5f0 0x7fae5809ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60928/0 (socket says 172.21.15.7:60928) 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 -- 172.21.15.7:0/1405123116 learned_addr learned my addr 172.21.15.7:0/1405123116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5d36a700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae580cf1b0 0x7fae58152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5db6b700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae58157a90 0x7fae5809b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 -- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae58157a90 msgr2=0x7fae5809b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae58157a90 0x7fae5809b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 -- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae580cf1b0 msgr2=0x7fae58152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae580cf1b0 0x7fae58152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 -- 172.21.15.7:0/1405123116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54009480 con 0x7fae580cf5f0 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5cb69700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae580cf5f0 0x7fae5809ab20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fae4800d890 tx=0x7fae48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae4d7fa700 1 -- 172.21.15.7:0/1405123116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae480125f0 con 0x7fae580cf5f0 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae4d7fa700 1 -- 172.21.15.7:0/1405123116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae48012750 con 0x7fae580cf5f0 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5809be60 con 0x7fae580cf5f0 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae4d7fa700 1 -- 172.21.15.7:0/1405123116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48012af0 con 0x7fae580cf5f0 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.401+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5809b5a0 con 0x7fae580cf5f0 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.402+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5804c3c0 con 0x7fae580cf5f0 2024-01-11T07:12:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.403+0000 7fae4d7fa700 1 -- 172.21.15.7:0/1405123116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fae4801e460 con 0x7fae580cf5f0 2024-01-11T07:12:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.403+0000 7fae4d7fa700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fae3c066f30 0x7fae3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.403+0000 7fae4d7fa700 1 -- 172.21.15.7:0/1405123116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae48084a60 con 0x7fae580cf5f0 2024-01-11T07:12:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.404+0000 7fae5d36a700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fae3c066f30 0x7fae3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.404+0000 7fae5d36a700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fae3c066f30 0x7fae3c0693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae54009260 tx=0x7fae54016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.408+0000 7fae4d7fa700 1 -- 172.21.15.7:0/1405123116 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae48051cd0 con 0x7fae580cf5f0 2024-01-11T07:12:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:47.660+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fae580c3740 con 0x7fae580cf5f0 2024-01-11T07:12:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.492+0000 7fae4d7fa700 1 -- 172.21.15.7:0/1405123116 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7fae580c3740 con 0x7fae580cf5f0 2024-01-11T07:12:48.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.494+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fae3c066f30 msgr2=0x7fae3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:48.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.494+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fae3c066f30 0x7fae3c0693e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae54009260 tx=0x7fae54016000 comp rx=0 tx=0).stop 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.494+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae580cf5f0 msgr2=0x7fae5809ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.494+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae580cf5f0 0x7fae5809ab20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fae4800d890 tx=0x7fae48008040 comp rx=0 tx=0).stop 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.495+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 shutdown_connections 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.495+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fae3c066f30 0x7fae3c0693e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.495+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fae58157a90 0x7fae5809b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.495+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae580cf1b0 0x7fae58152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.495+0000 7fae5f5ce700 1 --2- 172.21.15.7:0/1405123116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae580cf5f0 0x7fae5809ab20 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.495+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 >> 172.21.15.7:0/1405123116 conn(0x7fae58097a50 msgr2=0x7fae580b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.495+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 shutdown_connections 2024-01-11T07:12:48.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.495+0000 7fae5f5ce700 1 -- 172.21.15.7:0/1405123116 wait complete. 2024-01-11T07:12:48.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:12:48.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.650+0000 7f83d6ea5700 1 Processor -- start 2024-01-11T07:12:48.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.650+0000 7f83d6ea5700 1 -- start start 2024-01-11T07:12:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.651+0000 7f83d6ea5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d015d900 0x7f83d0063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.651+0000 7f83d6ea5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00cd6c0 con 0x7f83d0069d10 2024-01-11T07:12:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.651+0000 7f83d6ea5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00cb4c0 con 0x7f83d0159dc0 2024-01-11T07:12:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.651+0000 7f83d6ea5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015a190 con 0x7f83d015d900 2024-01-11T07:12:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.651+0000 7f83d5442700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f83d0069d10 0x7f83d00cb950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44948/0 (socket says 172.21.15.7:44948) 2024-01-11T07:12:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.651+0000 7f83d5442700 1 -- 172.21.15.7:0/2501235609 learned_addr learned my addr 172.21.15.7:0/2501235609 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cffff700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d015d900 0x7f83d0063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243657986 0 0) 0x7f83c8001490 con 0x7f83d0069d10 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c0003570 con 0x7f83d0069d10 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171097111 0 0) 0x7f83d00cb4c0 con 0x7f83d0159dc0 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c8001490 con 0x7f83d0159dc0 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1334824037 0 0) 0x7f83c0003570 con 0x7f83d0069d10 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d015d900 msgr2=0x7f83d0063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d015d900 0x7f83d0063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 >> v1:172.21.15.113:6789/0 conn(0x7f83d0159dc0 legacy=0x7f83d00c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00c87d0 con 0x7f83d0069d10 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f83c8001980 con 0x7f83d0069d10 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 >> v1:172.21.15.7:6789/0 conn(0x7f83d0069d10 legacy=0x7f83d00cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d015d900 0x7f83c0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c0004180 0x7f83c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015a190 con 0x7f83c0004d50 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000bba0 con 0x7f83c0004180 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000bd60 con 0x7f83d015d900 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83d5442700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83d4c41700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d015d900 0x7f83c0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.652+0000 7f83cffff700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c0004180 0x7f83c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d5442700 1 -- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d015d900 msgr2=0x7f83c0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d5442700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d015d900 0x7f83c0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d5442700 1 -- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c0004180 msgr2=0x7f83c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d5442700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83c0004180 0x7f83c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d5442700 1 -- 172.21.15.7:0/2501235609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0003570 con 0x7f83c0004d50 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d5442700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83c00051c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f83c800aa60 tx=0x7f83c800a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb11afa96035f63 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800a3f0 con 0x7f83c0004d50 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c800c5d0 con 0x7f83c0004d50 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83cf7fe700 1 -- 172.21.15.7:0/2501235609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800c950 con 0x7f83c0004d50 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d6ea5700 1 -- 172.21.15.7:0/2501235609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 msgr2=0x7f83c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d6ea5700 1 --2- 172.21.15.7:0/2501235609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83c00051c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f83c800aa60 tx=0x7f83c800a120 comp rx=0 tx=0).stop 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.653+0000 7f83d5442700 1 -- 172.21.15.7:0/2501235609 reap_dead start 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.654+0000 7f83d6ea5700 1 -- 172.21.15.7:0/2501235609 shutdown_connections 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.654+0000 7f83d6ea5700 1 -- 172.21.15.7:0/2501235609 >> 172.21.15.7:0/2501235609 conn(0x7f83d00cd860 msgr2=0x7f83d00cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.654+0000 7f83d6ea5700 1 -- 172.21.15.7:0/2501235609 shutdown_connections 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.654+0000 7f83d6ea5700 1 -- 172.21.15.7:0/2501235609 wait complete. 2024-01-11T07:12:48.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.655+0000 7f83d6ea5700 1 Processor -- start 2024-01-11T07:12:48.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.655+0000 7f83d6ea5700 1 -- start start 2024-01-11T07:12:48.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d6ea5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83d00b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:48.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d6ea5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d0154bf0 0x7f83d00c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:48.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d6ea5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d0155060 0x7f83d015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:48.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d6ea5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015c000 con 0x7f83c0004d50 2024-01-11T07:12:48.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d6ea5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015b3f0 con 0x7f83d0154bf0 2024-01-11T07:12:48.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d6ea5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015b6d0 con 0x7f83d0155060 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83d00b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83d00b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60944/0 (socket says 172.21.15.7:60944) 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 -- 172.21.15.7:0/3678477370 learned_addr learned my addr 172.21.15.7:0/3678477370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d5442700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d0155060 0x7f83d015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83cffff700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d0154bf0 0x7f83d00c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 -- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d0155060 msgr2=0x7f83d015c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d0155060 0x7f83d015c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 -- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d0154bf0 msgr2=0x7f83d00c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d0154bf0 0x7f83d00c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 -- 172.21.15.7:0/3678477370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d015b9b0 con 0x7f83c0004d50 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83d4c41700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83d00b0600 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f83c4002b50 tx=0x7f83c4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.656+0000 7f83ccff9700 1 -- 172.21.15.7:0/3678477370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c400f8d0 con 0x7f83c0004d50 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.657+0000 7f83ccff9700 1 -- 172.21.15.7:0/3678477370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c400fa50 con 0x7f83c0004d50 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.657+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d015bc90 con 0x7f83c0004d50 2024-01-11T07:12:48.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.657+0000 7f83ccff9700 1 -- 172.21.15.7:0/3678477370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c400fe10 con 0x7f83c0004d50 2024-01-11T07:12:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.657+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d00c5820 con 0x7f83c0004d50 2024-01-11T07:12:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.658+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d004c3c0 con 0x7f83c0004d50 2024-01-11T07:12:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.658+0000 7f83ccff9700 1 -- 172.21.15.7:0/3678477370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f83c4013600 con 0x7f83c0004d50 2024-01-11T07:12:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.658+0000 7f83ccff9700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f83b4066f30 0x7f83b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.659+0000 7f83ccff9700 1 -- 172.21.15.7:0/3678477370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f83c4084bf0 con 0x7f83c0004d50 2024-01-11T07:12:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.659+0000 7f83cffff700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f83b4066f30 0x7f83b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.659+0000 7f83cffff700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f83b4066f30 0x7f83b40693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f83d015f980 tx=0x7f83bc0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.663+0000 7f83ccff9700 1 -- 172.21.15.7:0/3678477370 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83c4051dc0 con 0x7f83c0004d50 2024-01-11T07:12:48.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:48.919+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f83d015de70 con 0x7f83c0004d50 2024-01-11T07:12:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.497+0000 7f83ccff9700 1 -- 172.21.15.7:0/3678477370 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83c40170c0 con 0x7f83c0004d50 2024-01-11T07:12:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f83b4066f30 msgr2=0x7f83b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f83b4066f30 0x7f83b40693e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f83d015f980 tx=0x7f83bc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 msgr2=0x7f83d00b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83d00b0600 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f83c4002b50 tx=0x7f83c4007d30 comp rx=0 tx=0).stop 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 shutdown_connections 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f83b4066f30 0x7f83b40693e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83d0155060 0x7f83d015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83c0004d50 0x7f83d00b0600 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 --2- 172.21.15.7:0/3678477370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83d0154bf0 0x7f83d00c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 >> 172.21.15.7:0/3678477370 conn(0x7f83d00cd860 msgr2=0x7f83d006ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 shutdown_connections 2024-01-11T07:12:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.500+0000 7f83d6ea5700 1 -- 172.21.15.7:0/3678477370 wait complete. 2024-01-11T07:12:49.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.651+0000 7fb24b024700 1 Processor -- start 2024-01-11T07:12:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.651+0000 7fb24b024700 1 -- start start 2024-01-11T07:12:49.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb24b024700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440cafb0 0x7fb2440ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb24b024700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2440b0b40 0x7fb244161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb24b024700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb244159300 con 0x7fb244158280 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb24b024700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2440cf260 con 0x7fb2440b0b40 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb24b024700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440cf540 con 0x7fb2440cafb0 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb2495c1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2440b0b40 0x7fb244161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb23bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb244158280 0x7fb2440b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44964/0 (socket says 172.21.15.7:44964) 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb23bfff700 1 -- 172.21.15.7:0/3729571588 learned_addr learned my addr 172.21.15.7:0/3729571588 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.652+0000 7fb248dc0700 1 --2- 172.21.15.7:0/3729571588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440cafb0 0x7fb2440ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb2495c1700 1 -- 172.21.15.7:0/3729571588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440cafb0 msgr2=0x7fb2440ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb2495c1700 1 --2- 172.21.15.7:0/3729571588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440cafb0 0x7fb2440ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb2495c1700 1 -- 172.21.15.7:0/3729571588 >> v1:172.21.15.7:6789/0 conn(0x7fb244158280 legacy=0x7fb2440b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb2495c1700 1 -- 172.21.15.7:0/3729571588 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2440cf820 con 0x7fb2440b0b40 2024-01-11T07:12:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb23b7fe700 1 -- 172.21.15.7:0/3729571588 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618581259 0 0) 0x7fb244159300 con 0x7fb244158280 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb248dc0700 1 --2- 172.21.15.7:0/3729571588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440cafb0 0x7fb2440ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb2495c1700 1 --2- 172.21.15.7:0/3729571588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440b0b40 0x7fb244161760 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009e60 tx=0x7fb23c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d4952579c70b5f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb23b7fe700 1 -- 172.21.15.7:0/3729571588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00e070 con 0x7fb2440b0b40 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb23b7fe700 1 -- 172.21.15.7:0/3729571588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c00bc30 con 0x7fb2440b0b40 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb23b7fe700 1 -- 172.21.15.7:0/3729571588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c012430 con 0x7fb2440b0b40 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb24b024700 1 -- 172.21.15.7:0/3729571588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440b0b40 msgr2=0x7fb244161760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.653+0000 7fb24b024700 1 --2- 172.21.15.7:0/3729571588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440b0b40 0x7fb244161760 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009e60 tx=0x7fb23c009780 comp rx=0 tx=0).stop 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.654+0000 7fb24b024700 1 -- 172.21.15.7:0/3729571588 shutdown_connections 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.654+0000 7fb24b024700 1 --2- 172.21.15.7:0/3729571588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440b0b40 0x7fb244161760 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.654+0000 7fb24b024700 1 --2- 172.21.15.7:0/3729571588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440cafb0 0x7fb2440ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.654+0000 7fb24b024700 1 -- 172.21.15.7:0/3729571588 >> 172.21.15.7:0/3729571588 conn(0x7fb244071100 msgr2=0x7fb24406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.654+0000 7fb24b024700 1 -- 172.21.15.7:0/3729571588 shutdown_connections 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.654+0000 7fb24b024700 1 -- 172.21.15.7:0/3729571588 wait complete. 2024-01-11T07:12:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.655+0000 7fb24b024700 1 Processor -- start 2024-01-11T07:12:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.655+0000 7fb24b024700 1 -- start start 2024-01-11T07:12:49.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb24b024700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440b0b40 0x7fb2440c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb24b024700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440cafb0 0x7fb244155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb24b024700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb244158280 0x7fb24415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb24b024700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440c8150 con 0x7fb2440cafb0 2024-01-11T07:12:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb24b024700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440c9b00 con 0x7fb244158280 2024-01-11T07:12:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb24b024700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440c9de0 con 0x7fb2440b0b40 2024-01-11T07:12:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440cafb0 0x7fb244155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440cafb0 0x7fb244155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60964/0 (socket says 172.21.15.7:60964) 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 -- 172.21.15.7:0/535148022 learned_addr learned my addr 172.21.15.7:0/535148022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb248dc0700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440b0b40 0x7fb2440c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb2495c1700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb244158280 0x7fb24415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 -- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440b0b40 msgr2=0x7fb2440c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440b0b40 0x7fb2440c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 -- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb244158280 msgr2=0x7fb24415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb244158280 0x7fb24415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 -- 172.21.15.7:0/535148022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c009a20 con 0x7fb2440cafb0 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb2495c1700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb244158280 0x7fb24415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.656+0000 7fb23bfff700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440cafb0 0x7fb244155560 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb23400c5e0 tx=0x7fb234000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.657+0000 7fb238ff9700 1 -- 172.21.15.7:0/535148022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234005b30 con 0x7fb2440cafb0 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.657+0000 7fb238ff9700 1 -- 172.21.15.7:0/535148022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb234005c90 con 0x7fb2440cafb0 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.657+0000 7fb238ff9700 1 -- 172.21.15.7:0/535148022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2340072e0 con 0x7fb2440cafb0 2024-01-11T07:12:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.657+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2440ca0c0 con 0x7fb2440cafb0 2024-01-11T07:12:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.657+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb244155aa0 con 0x7fb2440cafb0 2024-01-11T07:12:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.658+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24404c3c0 con 0x7fb2440cafb0 2024-01-11T07:12:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.658+0000 7fb238ff9700 1 -- 172.21.15.7:0/535148022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb23401b3b0 con 0x7fb2440cafb0 2024-01-11T07:12:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.659+0000 7fb238ff9700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb230066f30 0x7fb2300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.659+0000 7fb238ff9700 1 -- 172.21.15.7:0/535148022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb234083880 con 0x7fb2440cafb0 2024-01-11T07:12:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.659+0000 7fb248dc0700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb230066f30 0x7fb2300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.659+0000 7fb248dc0700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb230066f30 0x7fb2300693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb240006f00 tx=0x7fb240008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.663+0000 7fb238ff9700 1 -- 172.21.15.7:0/535148022 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb23404b5a0 con 0x7fb2440cafb0 2024-01-11T07:12:49.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.914+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb244159550 con 0x7fb2440cafb0 2024-01-11T07:12:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.915+0000 7fb238ff9700 1 -- 172.21.15.7:0/535148022 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fb234017090 con 0x7fb2440cafb0 2024-01-11T07:12:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:49.916 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:12:45.905827+0000","modified":"2024-01-11T07:12:49.492046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2493499246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2493499246},{"type":"v1","addr":"172.21.15.7:6835","nonce":2493499246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-11T07:12:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb230066f30 msgr2=0x7fb2300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb230066f30 0x7fb2300693e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb240006f00 tx=0x7fb240008040 comp rx=0 tx=0).stop 2024-01-11T07:12:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440cafb0 msgr2=0x7fb244155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440cafb0 0x7fb244155560 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb23400c5e0 tx=0x7fb234000fe0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 shutdown_connections 2024-01-11T07:12:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb230066f30 0x7fb2300693e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb244158280 0x7fb24415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2440b0b40 0x7fb2440c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 --2- 172.21.15.7:0/535148022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2440cafb0 0x7fb244155560 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 >> 172.21.15.7:0/535148022 conn(0x7fb244071100 msgr2=0x7fb2440ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 shutdown_connections 2024-01-11T07:12:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:49.917+0000 7fb24b024700 1 -- 172.21.15.7:0/535148022 wait complete. 2024-01-11T07:12:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-01-11T07:12:49.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.062+0000 7f1500b3d700 1 Processor -- start 2024-01-11T07:12:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.062+0000 7f1500b3d700 1 -- start start 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.063+0000 7f1500b3d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14fc159ca0 0x7f14fc06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f1500b3d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14fc15a070 0x7f14fc06e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f1500b3d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14fc0d18c0 con 0x7f14fc0ce7f0 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f1500b3d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14fc0d0010 con 0x7f14fc159ca0 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f1500b3d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14fc0cf400 con 0x7f14fc15a070 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14fa59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14fc159ca0 0x7f14fc06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14fa59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14fc159ca0 0x7f14fc06dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60980/0 (socket says 172.21.15.7:60980) 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14fa59c700 1 -- 172.21.15.7:0/1321622114 learned_addr learned my addr 172.21.15.7:0/1321622114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14f9d9b700 1 --2- 172.21.15.7:0/1321622114 >> v2:172.21.15.113:3300/0 conn(0x7f14fc15a070 0x7f14fc06e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14fa59c700 1 -- 172.21.15.7:0/1321622114 >> v2:172.21.15.113:3300/0 conn(0x7f14fc15a070 msgr2=0x7f14fc06e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14fa59c700 1 --2- 172.21.15.7:0/1321622114 >> v2:172.21.15.113:3300/0 conn(0x7f14fc15a070 0x7f14fc06e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14fa59c700 1 -- 172.21.15.7:0/1321622114 >> v1:172.21.15.113:6789/0 conn(0x7f14fc0ce7f0 legacy=0x7f14fc0cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14fa59c700 1 -- 172.21.15.7:0/1321622114 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc0cf6e0 con 0x7f14fc159ca0 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14f9d9b700 1 --2- 172.21.15.7:0/1321622114 >> v2:172.21.15.113:3300/0 conn(0x7f14fc15a070 0x7f14fc06e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14fa59c700 1 --2- 172.21.15.7:0/1321622114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc06dc30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f14fc15a510 tx=0x7f14ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41df51d0516c604a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14f959a700 1 -- 172.21.15.7:0/1321622114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00b860 con 0x7f14fc159ca0 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14f959a700 1 -- 172.21.15.7:0/1321622114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec00b9c0 con 0x7f14fc159ca0 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.064+0000 7f14f959a700 1 -- 172.21.15.7:0/1321622114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00bd60 con 0x7f14fc159ca0 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.065+0000 7f1500b3d700 1 -- 172.21.15.7:0/1321622114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 msgr2=0x7f14fc06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.065+0000 7f1500b3d700 1 --2- 172.21.15.7:0/1321622114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc06dc30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f14fc15a510 tx=0x7f14ec009250 comp rx=0 tx=0).stop 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.065+0000 7f1500b3d700 1 -- 172.21.15.7:0/1321622114 shutdown_connections 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.065+0000 7f1500b3d700 1 --2- 172.21.15.7:0/1321622114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc06dc30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.065+0000 7f1500b3d700 1 --2- 172.21.15.7:0/1321622114 >> v2:172.21.15.113:3300/0 conn(0x7f14fc15a070 0x7f14fc06e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.065+0000 7f1500b3d700 1 -- 172.21.15.7:0/1321622114 >> 172.21.15.7:0/1321622114 conn(0x7f14fc063cc0 msgr2=0x7f14fc0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.065+0000 7f1500b3d700 1 -- 172.21.15.7:0/1321622114 shutdown_connections 2024-01-11T07:12:50.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.065+0000 7f1500b3d700 1 -- 172.21.15.7:0/1321622114 wait complete. 2024-01-11T07:12:50.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.066+0000 7f1500b3d700 1 Processor -- start 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.066+0000 7f1500b3d700 1 -- start start 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f1500b3d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14fc0ce7f0 0x7f14fc0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f1500b3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f1500b3d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14fc15a070 0x7f14fc0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f1500b3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0b3d10 con 0x7f14fc159ca0 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f1500b3d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc16ac30 con 0x7f14fc15a070 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f1500b3d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc16af10 con 0x7f14fc0ce7f0 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f14f9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f14f9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc0b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60990/0 (socket says 172.21.15.7:60990) 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f14f9d9b700 1 -- 172.21.15.7:0/3726783932 learned_addr learned my addr 172.21.15.7:0/3726783932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f14fa59c700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14fc0ce7f0 0x7f14fc0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.067+0000 7f14fad9d700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14fc15a070 0x7f14fc0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14f9d9b700 1 -- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14fc0ce7f0 msgr2=0x7f14fc0b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14f9d9b700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14fc0ce7f0 0x7f14fc0b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14f9d9b700 1 -- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14fc15a070 msgr2=0x7f14fc0cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14f9d9b700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14fc15a070 0x7f14fc0cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14f9d9b700 1 -- 172.21.15.7:0/3726783932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0094f0 con 0x7f14fc159ca0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14f9d9b700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc0b3f80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f14f000d860 tx=0x7f14f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14ea7fc700 1 -- 172.21.15.7:0/3726783932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f00125f0 con 0x7f14fc159ca0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14ea7fc700 1 -- 172.21.15.7:0/3726783932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f0012750 con 0x7f14fc159ca0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14fc16b1f0 con 0x7f14fc159ca0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f14ea7fc700 1 -- 172.21.15.7:0/3726783932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0012af0 con 0x7f14fc159ca0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.068+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc0b6860 con 0x7f14fc159ca0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.069+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14fc16ba60 con 0x7f14fc159ca0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.070+0000 7f14ea7fc700 1 -- 172.21.15.7:0/3726783932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f14f0013480 con 0x7f14fc159ca0 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.070+0000 7f14ea7fc700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f14dc066f30 0x7f14dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.070+0000 7f14ea7fc700 1 -- 172.21.15.7:0/3726783932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f14f0084670 con 0x7f14fc159ca0 2024-01-11T07:12:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.070+0000 7f14fa59c700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f14dc066f30 0x7f14dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.071+0000 7f14fa59c700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f14dc066f30 0x7f14dc0693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f14fc15a510 tx=0x7f14ec017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.074+0000 7f14ea7fc700 1 -- 172.21.15.7:0/3726783932 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14fc16ba60 con 0x7f14fc159ca0 2024-01-11T07:12:50.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.278+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14fc16ba60 con 0x7f14fc159ca0 2024-01-11T07:12:50.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.279+0000 7f14ea7fc700 1 -- 172.21.15.7:0/3726783932 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+15317 (secure 0 0 0) 0x7f14fc16ba60 con 0x7f14fc159ca0 2024-01-11T07:12:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:50.280 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":120,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:47.481414+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:12:43.805689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-11T07:12:44.880093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:50.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f14dc066f30 msgr2=0x7f14dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f14dc066f30 0x7f14dc0693e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f14fc15a510 tx=0x7f14ec017000 comp rx=0 tx=0).stop 2024-01-11T07:12:50.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 msgr2=0x7f14fc0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc0b3f80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f14f000d860 tx=0x7f14f0008040 comp rx=0 tx=0).stop 2024-01-11T07:12:50.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 shutdown_connections 2024-01-11T07:12:50.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f14dc066f30 0x7f14dc0693e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14fc15a070 0x7f14fc0cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14fc0ce7f0 0x7f14fc0b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 --2- 172.21.15.7:0/3726783932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14fc159ca0 0x7f14fc0b3f80 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 >> 172.21.15.7:0/3726783932 conn(0x7f14fc063cc0 msgr2=0x7f14fc0c66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:50.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 shutdown_connections 2024-01-11T07:12:50.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.281+0000 7f1500b3d700 1 -- 172.21.15.7:0/3726783932 wait complete. 2024-01-11T07:12:50.296 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:12:50.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:12:50.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.428+0000 7fa77ce1c700 1 Processor -- start 2024-01-11T07:12:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.428+0000 7fa77ce1c700 1 -- start start 2024-01-11T07:12:50.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa77ce1c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa778159c20 0x7fa7780b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa77ce1c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa77806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa77ce1c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa778067de0 con 0x7fa778068270 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa77ce1c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa778069570 con 0x7fa778159c20 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa77ce1c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780b20e0 con 0x7fa778159ff0 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa776d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa778068270 0x7fa7780b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44976/0 (socket says 172.21.15.7:44976) 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa776d9d700 1 -- 172.21.15.7:0/2653150668 learned_addr learned my addr 172.21.15.7:0/2653150668 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa775d9b700 1 --2- 172.21.15.7:0/2653150668 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa77806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa77659c700 1 --2- 172.21.15.7:0/2653150668 >> v2:172.21.15.113:3300/0 conn(0x7fa778159c20 0x7fa7780b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa77559a700 1 -- 172.21.15.7:0/2653150668 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610035537 0 0) 0x7fa778067de0 con 0x7fa778068270 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.429+0000 7fa77559a700 1 -- 172.21.15.7:0/2653150668 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa760003570 con 0x7fa778068270 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.430+0000 7fa775d9b700 1 -- 172.21.15.7:0/2653150668 >> v2:172.21.15.113:3300/0 conn(0x7fa778159c20 msgr2=0x7fa7780b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.430+0000 7fa775d9b700 1 --2- 172.21.15.7:0/2653150668 >> v2:172.21.15.113:3300/0 conn(0x7fa778159c20 0x7fa7780b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.430+0000 7fa775d9b700 1 -- 172.21.15.7:0/2653150668 >> v1:172.21.15.7:6789/0 conn(0x7fa778068270 legacy=0x7fa7780b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.430+0000 7fa775d9b700 1 -- 172.21.15.7:0/2653150668 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7780b1e00 con 0x7fa778159ff0 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.430+0000 7fa775d9b700 1 --2- 172.21.15.7:0/2653150668 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa77806f4d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098e0 tx=0x7fa76c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8fefcd655195265 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.430+0000 7fa77559a700 1 -- 172.21.15.7:0/2653150668 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c011630 con 0x7fa778159ff0 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.430+0000 7fa77559a700 1 -- 172.21.15.7:0/2653150668 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c011790 con 0x7fa778159ff0 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.430+0000 7fa77559a700 1 -- 172.21.15.7:0/2653150668 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c011b30 con 0x7fa778159ff0 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.431+0000 7fa77ce1c700 1 -- 172.21.15.7:0/2653150668 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 msgr2=0x7fa77806f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.431+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/2653150668 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa77806f4d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098e0 tx=0x7fa76c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.431+0000 7fa77ce1c700 1 -- 172.21.15.7:0/2653150668 shutdown_connections 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.431+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/2653150668 >> v2:172.21.15.113:3300/0 conn(0x7fa778159c20 0x7fa7780b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.431+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/2653150668 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa77806f4d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.431+0000 7fa77ce1c700 1 -- 172.21.15.7:0/2653150668 >> 172.21.15.7:0/2653150668 conn(0x7fa778071160 msgr2=0x7fa77806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.431+0000 7fa77ce1c700 1 -- 172.21.15.7:0/2653150668 shutdown_connections 2024-01-11T07:12:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.431+0000 7fa77ce1c700 1 -- 172.21.15.7:0/2653150668 wait complete. 2024-01-11T07:12:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.432+0000 7fa77ce1c700 1 Processor -- start 2024-01-11T07:12:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.432+0000 7fa77ce1c700 1 -- start start 2024-01-11T07:12:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa77ce1c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa778068270 0x7fa77815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa77ce1c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa778159c20 0x7fa7780c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa77ce1c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa7780ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa77ce1c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780cac80 con 0x7fa778159c20 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa77ce1c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780caf60 con 0x7fa778068270 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa77ce1c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780cb240 con 0x7fa778159ff0 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa778159c20 0x7fa7780c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa778159c20 0x7fa7780c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60998/0 (socket says 172.21.15.7:60998) 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 -- 172.21.15.7:0/3753013558 learned_addr learned my addr 172.21.15.7:0/3753013558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa77659c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa778068270 0x7fa77815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa776d9d700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa7780ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 -- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 msgr2=0x7fa7780ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa7780ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 -- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa778068270 msgr2=0x7fa77815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa778068270 0x7fa77815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 -- 172.21.15.7:0/3753013558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c009420 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa77659c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa778068270 0x7fa77815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.433+0000 7fa775d9b700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa778159c20 0x7fa7780c9f10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa7780b2220 tx=0x7fa76c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.434+0000 7fa7667fc700 1 -- 172.21.15.7:0/3753013558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c011b10 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.434+0000 7fa7667fc700 1 -- 172.21.15.7:0/3753013558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c0233f0 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.434+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7780cb520 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.434+0000 7fa7667fc700 1 -- 172.21.15.7:0/3753013558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c023770 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.434+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa778169ca0 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.435+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77804c3c0 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.435+0000 7fa7667fc700 1 -- 172.21.15.7:0/3753013558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa76c02c3b0 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.436+0000 7fa7667fc700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa758066f30 0x7fa7580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.436+0000 7fa7667fc700 1 -- 172.21.15.7:0/3753013558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa76c085a20 con 0x7fa778159c20 2024-01-11T07:12:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.436+0000 7fa77659c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa758066f30 0x7fa7580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.436+0000 7fa77659c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa758066f30 0x7fa7580693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa768006f00 tx=0x7fa768008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.440+0000 7fa7667fc700 1 -- 172.21.15.7:0/3753013558 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa76c052d10 con 0x7fa778159c20 2024-01-11T07:12:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.644+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7780cb0d0 con 0x7fa758066f30 2024-01-11T07:12:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.653+0000 7fa7667fc700 1 -- 172.21.15.7:0/3753013558 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285561 (secure 0 0 0) 0x7fa7780cb0d0 con 0x7fa758066f30 2024-01-11T07:12:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa758066f30 msgr2=0x7fa7580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa758066f30 0x7fa7580693e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa768006f00 tx=0x7fa768008040 comp rx=0 tx=0).stop 2024-01-11T07:12:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa778159c20 msgr2=0x7fa7780c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa778159c20 0x7fa7780c9f10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa7780b2220 tx=0x7fa76c017000 comp rx=0 tx=0).stop 2024-01-11T07:12:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 shutdown_connections 2024-01-11T07:12:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa758066f30 0x7fa7580693e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa778159ff0 0x7fa7780ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa778068270 0x7fa77815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 --2- 172.21.15.7:0/3753013558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa778159c20 0x7fa7780c9f10 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 >> 172.21.15.7:0/3753013558 conn(0x7fa778071160 msgr2=0x7fa7780ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:50.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 shutdown_connections 2024-01-11T07:12:50.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.656+0000 7fa77ce1c700 1 -- 172.21.15.7:0/3753013558 wait complete. 2024-01-11T07:12:50.662 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:12:50.670 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-01-11T07:12:49.922846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6280,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556219,"statfs":{"total":1600000000,"available":1593568516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454217"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.493536+0000","last_change":"2024-01-11T07:12:46.493452+0000","last_active":"2024-01-11T07:12:46.493536+0000","last_peered":"2024-01-11T07:12:46.493536+0000","last_clean":"2024-01-11T07:12:46.493536+0000","last_became_active":"2024-01-11T07:12:46.493177+0000","last_became_peered":"2024-01-11T07:12:46.493177+0000","last_unstale":"2024-01-11T07:12:46.493536+0000","last_undegraded":"2024-01-11T07:12:46.493536+0000","last_fullsized":"2024-01-11T07:12:46.493536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:01:32.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484826+0000","last_change":"2024-01-11T07:12:46.484666+0000","last_active":"2024-01-11T07:12:46.484826+0000","last_peered":"2024-01-11T07:12:46.484826+0000","last_clean":"2024-01-11T07:12:46.484826+0000","last_became_active":"2024-01-11T07:12:46.483727+0000","last_became_peered":"2024-01-11T07:12:46.483727+0000","last_unstale":"2024-01-11T07:12:46.484826+0000","last_undegraded":"2024-01-11T07:12:46.484826+0000","last_fullsized":"2024-01-11T07:12:46.484826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:58:53.624169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489928+0000","last_change":"2024-01-11T07:12:46.489759+0000","last_active":"2024-01-11T07:12:46.489928+0000","last_peered":"2024-01-11T07:12:46.489928+0000","last_clean":"2024-01-11T07:12:46.489928+0000","last_became_active":"2024-01-11T07:12:46.489495+0000","last_became_peered":"2024-01-11T07:12:46.489495+0000","last_unstale":"2024-01-11T07:12:46.489928+0000","last_undegraded":"2024-01-11T07:12:46.489928+0000","last_fullsized":"2024-01-11T07:12:46.489928+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:59:55.250999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500683+0000","last_change":"2024-01-11T07:12:46.500409+0000","last_active":"2024-01-11T07:12:46.500683+0000","last_peered":"2024-01-11T07:12:46.500683+0000","last_clean":"2024-01-11T07:12:46.500683+0000","last_became_active":"2024-01-11T07:12:46.499802+0000","last_became_peered":"2024-01-11T07:12:46.499802+0000","last_unstale":"2024-01-11T07:12:46.500683+0000","last_undegraded":"2024-01-11T07:12:46.500683+0000","last_fullsized":"2024-01-11T07:12:46.500683+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:57:29.700660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.495788+0000","last_change":"2024-01-11T07:12:46.495727+0000","last_active":"2024-01-11T07:12:46.495788+0000","last_peered":"2024-01-11T07:12:46.495788+0000","last_clean":"2024-01-11T07:12:46.495788+0000","last_became_active":"2024-01-11T07:12:46.495515+0000","last_became_peered":"2024-01-11T07:12:46.495515+0000","last_unstale":"2024-01-11T07:12:46.495788+0000","last_undegraded":"2024-01-11T07:12:46.495788+0000","last_fullsized":"2024-01-11T07:12:46.495788+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:42.973252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484967+0000","last_change":"2024-01-11T07:12:46.482720+0000","last_active":"2024-01-11T07:12:46.484967+0000","last_peered":"2024-01-11T07:12:46.484967+0000","last_clean":"2024-01-11T07:12:46.484967+0000","last_became_active":"2024-01-11T07:12:46.482434+0000","last_became_peered":"2024-01-11T07:12:46.482434+0000","last_unstale":"2024-01-11T07:12:46.484967+0000","last_undegraded":"2024-01-11T07:12:46.484967+0000","last_fullsized":"2024-01-11T07:12:46.484967+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:32.957971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484988+0000","last_change":"2024-01-11T07:12:46.482994+0000","last_active":"2024-01-11T07:12:46.484988+0000","last_peered":"2024-01-11T07:12:46.484988+0000","last_clean":"2024-01-11T07:12:46.484988+0000","last_became_active":"2024-01-11T07:12:46.482749+0000","last_became_peered":"2024-01-11T07:12:46.482749+0000","last_unstale":"2024-01-11T07:12:46.484988+0000","last_undegraded":"2024-01-11T07:12:46.484988+0000","last_fullsized":"2024-01-11T07:12:46.484988+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:16.225134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496585+0000","last_change":"2024-01-11T07:12:46.493197+0000","last_active":"2024-01-11T07:12:46.496585+0000","last_peered":"2024-01-11T07:12:46.496585+0000","last_clean":"2024-01-11T07:12:46.496585+0000","last_became_active":"2024-01-11T07:12:46.487933+0000","last_became_peered":"2024-01-11T07:12:46.487933+0000","last_unstale":"2024-01-11T07:12:46.496585+0000","last_undegraded":"2024-01-11T07:12:46.496585+0000","last_fullsized":"2024-01-11T07:12:46.496585+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:48:28.000032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500524+0000","last_change":"2024-01-11T07:12:46.500478+0000","last_active":"2024-01-11T07:12:46.500524+0000","last_peered":"2024-01-11T07:12:46.500524+0000","last_clean":"2024-01-11T07:12:46.500524+0000","last_became_active":"2024-01-11T07:12:46.500344+0000","last_became_peered":"2024-01-11T07:12:46.500344+0000","last_unstale":"2024-01-11T07:12:46.500524+0000","last_undegraded":"2024-01-11T07:12:46.500524+0000","last_fullsized":"2024-01-11T07:12:46.500524+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:12.892522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.487386+0000","last_change":"2024-01-11T07:12:46.487295+0000","last_active":"2024-01-11T07:12:46.487386+0000","last_peered":"2024-01-11T07:12:46.487386+0000","last_clean":"2024-01-11T07:12:46.487386+0000","last_became_active":"2024-01-11T07:12:46.487020+0000","last_became_peered":"2024-01-11T07:12:46.487020+0000","last_unstale":"2024-01-11T07:12:46.487386+0000","last_undegraded":"2024-01-11T07:12:46.487386+0000","last_fullsized":"2024-01-11T07:12:46.487386+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:44:04.558093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485047+0000","last_change":"2024-01-11T07:12:46.484280+0000","last_active":"2024-01-11T07:12:46.485047+0000","last_peered":"2024-01-11T07:12:46.485047+0000","last_clean":"2024-01-11T07:12:46.485047+0000","last_became_active":"2024-01-11T07:12:46.483491+0000","last_became_peered":"2024-01-11T07:12:46.483491+0000","last_unstale":"2024-01-11T07:12:46.485047+0000","last_undegraded":"2024-01-11T07:12:46.485047+0000","last_fullsized":"2024-01-11T07:12:46.485047+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:33:04.278431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.494801+0000","last_change":"2024-01-11T07:12:46.494583+0000","last_active":"2024-01-11T07:12:46.494801+0000","last_peered":"2024-01-11T07:12:46.494801+0000","last_clean":"2024-01-11T07:12:46.494801+0000","last_became_active":"2024-01-11T07:12:46.494353+0000","last_became_peered":"2024-01-11T07:12:46.494353+0000","last_unstale":"2024-01-11T07:12:46.494801+0000","last_undegraded":"2024-01-11T07:12:46.494801+0000","last_fullsized":"2024-01-11T07:12:46.494801+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:16.225134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496153+0000","last_change":"2024-01-11T07:12:46.495599+0000","last_active":"2024-01-11T07:12:46.496153+0000","last_peered":"2024-01-11T07:12:46.496153+0000","last_clean":"2024-01-11T07:12:46.496153+0000","last_became_active":"2024-01-11T07:12:46.494941+0000","last_became_peered":"2024-01-11T07:12:46.494941+0000","last_unstale":"2024-01-11T07:12:46.496153+0000","last_undegraded":"2024-01-11T07:12:46.496153+0000","last_fullsized":"2024-01-11T07:12:46.496153+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:04:59.382310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.491503+0000","last_change":"2024-01-11T07:12:46.491393+0000","last_active":"2024-01-11T07:12:46.491503+0000","last_peered":"2024-01-11T07:12:46.491503+0000","last_clean":"2024-01-11T07:12:46.491503+0000","last_became_active":"2024-01-11T07:12:46.491148+0000","last_became_peered":"2024-01-11T07:12:46.491148+0000","last_unstale":"2024-01-11T07:12:46.491503+0000","last_undegraded":"2024-01-11T07:12:46.491503+0000","last_fullsized":"2024-01-11T07:12:46.491503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:11:10.842352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.503245+0000","last_change":"2024-01-11T07:12:46.503028+0000","last_active":"2024-01-11T07:12:46.503245+0000","last_peered":"2024-01-11T07:12:46.503245+0000","last_clean":"2024-01-11T07:12:46.503245+0000","last_became_active":"2024-01-11T07:12:46.502513+0000","last_became_peered":"2024-01-11T07:12:46.502513+0000","last_unstale":"2024-01-11T07:12:46.503245+0000","last_undegraded":"2024-01-11T07:12:46.503245+0000","last_fullsized":"2024-01-11T07:12:46.503245+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:50.736749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.480554+0000","last_change":"2024-01-11T07:12:46.479991+0000","last_active":"2024-01-11T07:12:46.480554+0000","last_peered":"2024-01-11T07:12:46.480554+0000","last_clean":"2024-01-11T07:12:46.480554+0000","last_became_active":"2024-01-11T07:12:46.479757+0000","last_became_peered":"2024-01-11T07:12:46.479757+0000","last_unstale":"2024-01-11T07:12:46.480554+0000","last_undegraded":"2024-01-11T07:12:46.480554+0000","last_fullsized":"2024-01-11T07:12:46.480554+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:49:24.784110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500518+0000","last_change":"2024-01-11T07:12:46.500286+0000","last_active":"2024-01-11T07:12:46.500518+0000","last_peered":"2024-01-11T07:12:46.500518+0000","last_clean":"2024-01-11T07:12:46.500518+0000","last_became_active":"2024-01-11T07:12:46.499778+0000","last_became_peered":"2024-01-11T07:12:46.499778+0000","last_unstale":"2024-01-11T07:12:46.500518+0000","last_undegraded":"2024-01-11T07:12:46.500518+0000","last_fullsized":"2024-01-11T07:12:46.500518+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:48:31.698041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.499099+0000","last_change":"2024-01-11T07:12:46.496242+0000","last_active":"2024-01-11T07:12:46.499099+0000","last_peered":"2024-01-11T07:12:46.499099+0000","last_clean":"2024-01-11T07:12:46.499099+0000","last_became_active":"2024-01-11T07:12:46.495871+0000","last_became_peered":"2024-01-11T07:12:46.495871+0000","last_unstale":"2024-01-11T07:12:46.499099+0000","last_undegraded":"2024-01-11T07:12:46.499099+0000","last_fullsized":"2024-01-11T07:12:46.499099+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:16.225134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490609+0000","last_change":"2024-01-11T07:12:46.490188+0000","last_active":"2024-01-11T07:12:46.490609+0000","last_peered":"2024-01-11T07:12:46.490609+0000","last_clean":"2024-01-11T07:12:46.490609+0000","last_became_active":"2024-01-11T07:12:46.488730+0000","last_became_peered":"2024-01-11T07:12:46.488730+0000","last_unstale":"2024-01-11T07:12:46.490609+0000","last_undegraded":"2024-01-11T07:12:46.490609+0000","last_fullsized":"2024-01-11T07:12:46.490609+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:55:18.207452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497199+0000","last_change":"2024-01-11T07:12:46.497130+0000","last_active":"2024-01-11T07:12:46.497199+0000","last_peered":"2024-01-11T07:12:46.497199+0000","last_clean":"2024-01-11T07:12:46.497199+0000","last_became_active":"2024-01-11T07:12:46.496900+0000","last_became_peered":"2024-01-11T07:12:46.496900+0000","last_unstale":"2024-01-11T07:12:46.497199+0000","last_undegraded":"2024-01-11T07:12:46.497199+0000","last_fullsized":"2024-01-11T07:12:46.497199+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:26.573595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.486715+0000","last_change":"2024-01-11T07:12:46.486575+0000","last_active":"2024-01-11T07:12:46.486715+0000","last_peered":"2024-01-11T07:12:46.486715+0000","last_clean":"2024-01-11T07:12:46.486715+0000","last_became_active":"2024-01-11T07:12:46.486317+0000","last_became_peered":"2024-01-11T07:12:46.486317+0000","last_unstale":"2024-01-11T07:12:46.486715+0000","last_undegraded":"2024-01-11T07:12:46.486715+0000","last_fullsized":"2024-01-11T07:12:46.486715+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:37:29.109240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484810+0000","last_change":"2024-01-11T07:12:46.484495+0000","last_active":"2024-01-11T07:12:46.484810+0000","last_peered":"2024-01-11T07:12:46.484810+0000","last_clean":"2024-01-11T07:12:46.484810+0000","last_became_active":"2024-01-11T07:12:46.483377+0000","last_became_peered":"2024-01-11T07:12:46.483377+0000","last_unstale":"2024-01-11T07:12:46.484810+0000","last_undegraded":"2024-01-11T07:12:46.484810+0000","last_fullsized":"2024-01-11T07:12:46.484810+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:25:21.010285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496549+0000","last_change":"2024-01-11T07:12:46.488178+0000","last_active":"2024-01-11T07:12:46.496549+0000","last_peered":"2024-01-11T07:12:46.496549+0000","last_clean":"2024-01-11T07:12:46.496549+0000","last_became_active":"2024-01-11T07:12:46.487106+0000","last_became_peered":"2024-01-11T07:12:46.487106+0000","last_unstale":"2024-01-11T07:12:46.496549+0000","last_undegraded":"2024-01-11T07:12:46.496549+0000","last_fullsized":"2024-01-11T07:12:46.496549+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:58:08.855673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490013+0000","last_change":"2024-01-11T07:12:46.489944+0000","last_active":"2024-01-11T07:12:46.490013+0000","last_peered":"2024-01-11T07:12:46.490013+0000","last_clean":"2024-01-11T07:12:46.490013+0000","last_became_active":"2024-01-11T07:12:46.489698+0000","last_became_peered":"2024-01-11T07:12:46.489698+0000","last_unstale":"2024-01-11T07:12:46.490013+0000","last_undegraded":"2024-01-11T07:12:46.490013+0000","last_fullsized":"2024-01-11T07:12:46.490013+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:04:59.382310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484027+0000","last_change":"2024-01-11T07:12:46.483843+0000","last_active":"2024-01-11T07:12:46.484027+0000","last_peered":"2024-01-11T07:12:46.484027+0000","last_clean":"2024-01-11T07:12:46.484027+0000","last_became_active":"2024-01-11T07:12:46.483613+0000","last_became_peered":"2024-01-11T07:12:46.483613+0000","last_unstale":"2024-01-11T07:12:46.484027+0000","last_undegraded":"2024-01-11T07:12:46.484027+0000","last_fullsized":"2024-01-11T07:12:46.484027+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:15:20.309612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484791+0000","last_change":"2024-01-11T07:12:46.484374+0000","last_active":"2024-01-11T07:12:46.484791+0000","last_peered":"2024-01-11T07:12:46.484791+0000","last_clean":"2024-01-11T07:12:46.484791+0000","last_became_active":"2024-01-11T07:12:46.484119+0000","last_became_peered":"2024-01-11T07:12:46.484119+0000","last_unstale":"2024-01-11T07:12:46.484791+0000","last_undegraded":"2024-01-11T07:12:46.484791+0000","last_fullsized":"2024-01-11T07:12:46.484791+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:28.177221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485474+0000","last_change":"2024-01-11T07:12:46.485362+0000","last_active":"2024-01-11T07:12:46.485474+0000","last_peered":"2024-01-11T07:12:46.485474+0000","last_clean":"2024-01-11T07:12:46.485474+0000","last_became_active":"2024-01-11T07:12:46.485129+0000","last_became_peered":"2024-01-11T07:12:46.485129+0000","last_unstale":"2024-01-11T07:12:46.485474+0000","last_undegraded":"2024-01-11T07:12:46.485474+0000","last_fullsized":"2024-01-11T07:12:46.485474+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:22:33.981202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496373+0000","last_change":"2024-01-11T07:12:46.496302+0000","last_active":"2024-01-11T07:12:46.496373+0000","last_peered":"2024-01-11T07:12:46.496373+0000","last_clean":"2024-01-11T07:12:46.496373+0000","last_became_active":"2024-01-11T07:12:46.495541+0000","last_became_peered":"2024-01-11T07:12:46.495541+0000","last_unstale":"2024-01-11T07:12:46.496373+0000","last_undegraded":"2024-01-11T07:12:46.496373+0000","last_fullsized":"2024-01-11T07:12:46.496373+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:35:56.154787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496526+0000","last_change":"2024-01-11T07:12:46.486216+0000","last_active":"2024-01-11T07:12:46.496526+0000","last_peered":"2024-01-11T07:12:46.496526+0000","last_clean":"2024-01-11T07:12:46.496526+0000","last_became_active":"2024-01-11T07:12:46.485965+0000","last_became_peered":"2024-01-11T07:12:46.485965+0000","last_unstale":"2024-01-11T07:12:46.496526+0000","last_undegraded":"2024-01-11T07:12:46.496526+0000","last_fullsized":"2024-01-11T07:12:46.496526+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:05.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.492768+0000","last_change":"2024-01-11T07:12:46.492676+0000","last_active":"2024-01-11T07:12:46.492768+0000","last_peered":"2024-01-11T07:12:46.492768+0000","last_clean":"2024-01-11T07:12:46.492768+0000","last_became_active":"2024-01-11T07:12:46.492434+0000","last_became_peered":"2024-01-11T07:12:46.492434+0000","last_unstale":"2024-01-11T07:12:46.492768+0000","last_undegraded":"2024-01-11T07:12:46.492768+0000","last_fullsized":"2024-01-11T07:12:46.492768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:24:42.079612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.499158+0000","last_change":"2024-01-11T07:12:46.498816+0000","last_active":"2024-01-11T07:12:46.499158+0000","last_peered":"2024-01-11T07:12:46.499158+0000","last_clean":"2024-01-11T07:12:46.499158+0000","last_became_active":"2024-01-11T07:12:46.498005+0000","last_became_peered":"2024-01-11T07:12:46.498005+0000","last_unstale":"2024-01-11T07:12:46.499158+0000","last_undegraded":"2024-01-11T07:12:46.499158+0000","last_fullsized":"2024-01-11T07:12:46.499158+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.486708+0000","last_change":"2024-01-11T07:12:46.486097+0000","last_active":"2024-01-11T07:12:46.486708+0000","last_peered":"2024-01-11T07:12:46.486708+0000","last_clean":"2024-01-11T07:12:46.486708+0000","last_became_active":"2024-01-11T07:12:46.485824+0000","last_became_peered":"2024-01-11T07:12:46.485824+0000","last_unstale":"2024-01-11T07:12:46.486708+0000","last_undegraded":"2024-01-11T07:12:46.486708+0000","last_fullsized":"2024-01-11T07:12:46.486708+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:37.888735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489802+0000","last_change":"2024-01-11T07:12:46.489132+0000","last_active":"2024-01-11T07:12:46.489802+0000","last_peered":"2024-01-11T07:12:46.489802+0000","last_clean":"2024-01-11T07:12:46.489802+0000","last_became_active":"2024-01-11T07:12:46.488365+0000","last_became_peered":"2024-01-11T07:12:46.488365+0000","last_unstale":"2024-01-11T07:12:46.489802+0000","last_undegraded":"2024-01-11T07:12:46.489802+0000","last_fullsized":"2024-01-11T07:12:46.489802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:21:16.163722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.505167+0000","last_change":"2024-01-11T07:12:45.486817+0000","last_active":"2024-01-11T07:12:46.505167+0000","last_peered":"2024-01-11T07:12:46.505167+0000","last_clean":"2024-01-11T07:12:46.505167+0000","last_became_active":"2024-01-11T07:12:45.486275+0000","last_became_peered":"2024-01-11T07:12:45.486275+0000","last_unstale":"2024-01-11T07:12:46.505167+0000","last_undegraded":"2024-01-11T07:12:46.505167+0000","last_fullsized":"2024-01-11T07:12:46.505167+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:15.220175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496229+0000","last_change":"2024-01-11T07:12:46.496100+0000","last_active":"2024-01-11T07:12:46.496229+0000","last_peered":"2024-01-11T07:12:46.496229+0000","last_clean":"2024-01-11T07:12:46.496229+0000","last_became_active":"2024-01-11T07:12:46.495337+0000","last_became_peered":"2024-01-11T07:12:46.495337+0000","last_unstale":"2024-01-11T07:12:46.496229+0000","last_undegraded":"2024-01-11T07:12:46.496229+0000","last_fullsized":"2024-01-11T07:12:46.496229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:31:51.803993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500236+0000","last_change":"2024-01-11T07:12:45.488589+0000","last_active":"2024-01-11T07:12:46.500236+0000","last_peered":"2024-01-11T07:12:46.500236+0000","last_clean":"2024-01-11T07:12:46.500236+0000","last_became_active":"2024-01-11T07:12:45.488356+0000","last_became_peered":"2024-01-11T07:12:45.488356+0000","last_unstale":"2024-01-11T07:12:46.500236+0000","last_undegraded":"2024-01-11T07:12:46.500236+0000","last_fullsized":"2024-01-11T07:12:46.500236+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:12:29.270653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490608+0000","last_change":"2024-01-11T07:12:46.490527+0000","last_active":"2024-01-11T07:12:46.490608+0000","last_peered":"2024-01-11T07:12:46.490608+0000","last_clean":"2024-01-11T07:12:46.490608+0000","last_became_active":"2024-01-11T07:12:46.490281+0000","last_became_peered":"2024-01-11T07:12:46.490281+0000","last_unstale":"2024-01-11T07:12:46.490608+0000","last_undegraded":"2024-01-11T07:12:46.490608+0000","last_fullsized":"2024-01-11T07:12:46.490608+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:59.846219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.501319+0000","last_change":"2024-01-11T07:12:45.485614+0000","last_active":"2024-01-11T07:12:46.501319+0000","last_peered":"2024-01-11T07:12:46.501319+0000","last_clean":"2024-01-11T07:12:46.501319+0000","last_became_active":"2024-01-11T07:12:45.485293+0000","last_became_peered":"2024-01-11T07:12:45.485293+0000","last_unstale":"2024-01-11T07:12:46.501319+0000","last_undegraded":"2024-01-11T07:12:46.501319+0000","last_fullsized":"2024-01-11T07:12:46.501319+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:31.953012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489259+0000","last_change":"2024-01-11T07:12:46.488522+0000","last_active":"2024-01-11T07:12:46.489259+0000","last_peered":"2024-01-11T07:12:46.489259+0000","last_clean":"2024-01-11T07:12:46.489259+0000","last_became_active":"2024-01-11T07:12:46.488125+0000","last_became_peered":"2024-01-11T07:12:46.488125+0000","last_unstale":"2024-01-11T07:12:46.489259+0000","last_undegraded":"2024-01-11T07:12:46.489259+0000","last_fullsized":"2024-01-11T07:12:46.489259+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:32.957971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.502462+0000","last_change":"2024-01-11T07:12:45.489292+0000","last_active":"2024-01-11T07:12:46.502462+0000","last_peered":"2024-01-11T07:12:46.502462+0000","last_clean":"2024-01-11T07:12:46.502462+0000","last_became_active":"2024-01-11T07:12:45.488073+0000","last_became_peered":"2024-01-11T07:12:45.488073+0000","last_unstale":"2024-01-11T07:12:46.502462+0000","last_undegraded":"2024-01-11T07:12:46.502462+0000","last_fullsized":"2024-01-11T07:12:46.502462+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:25.568635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490358+0000","last_change":"2024-01-11T07:12:46.490282+0000","last_active":"2024-01-11T07:12:46.490358+0000","last_peered":"2024-01-11T07:12:46.490358+0000","last_clean":"2024-01-11T07:12:46.490358+0000","last_became_active":"2024-01-11T07:12:46.490027+0000","last_became_peered":"2024-01-11T07:12:46.490027+0000","last_unstale":"2024-01-11T07:12:46.490358+0000","last_undegraded":"2024-01-11T07:12:46.490358+0000","last_fullsized":"2024-01-11T07:12:46.490358+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:37.888735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.499964+0000","last_change":"2024-01-11T07:12:45.485661+0000","last_active":"2024-01-11T07:12:46.499964+0000","last_peered":"2024-01-11T07:12:46.499964+0000","last_clean":"2024-01-11T07:12:46.499964+0000","last_became_active":"2024-01-11T07:12:45.485148+0000","last_became_peered":"2024-01-11T07:12:45.485148+0000","last_unstale":"2024-01-11T07:12:46.499964+0000","last_undegraded":"2024-01-11T07:12:46.499964+0000","last_fullsized":"2024-01-11T07:12:46.499964+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:13:34.267082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.479269+0000","last_change":"2024-01-11T07:12:46.479196+0000","last_active":"2024-01-11T07:12:46.479269+0000","last_peered":"2024-01-11T07:12:46.479269+0000","last_clean":"2024-01-11T07:12:46.479269+0000","last_became_active":"2024-01-11T07:12:46.478866+0000","last_became_peered":"2024-01-11T07:12:46.478866+0000","last_unstale":"2024-01-11T07:12:46.479269+0000","last_undegraded":"2024-01-11T07:12:46.479269+0000","last_fullsized":"2024-01-11T07:12:46.479269+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:57:29.700660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.505228+0000","last_change":"2024-01-11T07:12:45.484742+0000","last_active":"2024-01-11T07:12:46.505228+0000","last_peered":"2024-01-11T07:12:46.505228+0000","last_clean":"2024-01-11T07:12:46.505228+0000","last_became_active":"2024-01-11T07:12:45.477948+0000","last_became_peered":"2024-01-11T07:12:45.477948+0000","last_unstale":"2024-01-11T07:12:46.505228+0000","last_undegraded":"2024-01-11T07:12:46.505228+0000","last_fullsized":"2024-01-11T07:12:46.505228+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:41.968293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.503255+0000","last_change":"2024-01-11T07:12:46.503184+0000","last_active":"2024-01-11T07:12:46.503255+0000","last_peered":"2024-01-11T07:12:46.503255+0000","last_clean":"2024-01-11T07:12:46.503255+0000","last_became_active":"2024-01-11T07:12:46.502859+0000","last_became_peered":"2024-01-11T07:12:46.502859+0000","last_unstale":"2024-01-11T07:12:46.503255+0000","last_undegraded":"2024-01-11T07:12:46.503255+0000","last_fullsized":"2024-01-11T07:12:46.503255+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:38:11.890732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.478476+0000","last_change":"2024-01-11T07:12:45.489677+0000","last_active":"2024-01-11T07:12:46.478476+0000","last_peered":"2024-01-11T07:12:46.478476+0000","last_clean":"2024-01-11T07:12:46.478476+0000","last_became_active":"2024-01-11T07:12:45.489435+0000","last_became_peered":"2024-01-11T07:12:45.489435+0000","last_unstale":"2024-01-11T07:12:46.478476+0000","last_undegraded":"2024-01-11T07:12:46.478476+0000","last_fullsized":"2024-01-11T07:12:46.478476+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:59:54.246040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489275+0000","last_change":"2024-01-11T07:12:46.488546+0000","last_active":"2024-01-11T07:12:46.489275+0000","last_peered":"2024-01-11T07:12:46.489275+0000","last_clean":"2024-01-11T07:12:46.489275+0000","last_became_active":"2024-01-11T07:12:46.488160+0000","last_became_peered":"2024-01-11T07:12:46.488160+0000","last_unstale":"2024-01-11T07:12:46.489275+0000","last_undegraded":"2024-01-11T07:12:46.489275+0000","last_fullsized":"2024-01-11T07:12:46.489275+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:42.973252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.482659+0000","last_change":"2024-01-11T07:12:45.489125+0000","last_active":"2024-01-11T07:12:46.482659+0000","last_peered":"2024-01-11T07:12:46.482659+0000","last_clean":"2024-01-11T07:12:46.482659+0000","last_became_active":"2024-01-11T07:12:45.488883+0000","last_became_peered":"2024-01-11T07:12:45.488883+0000","last_unstale":"2024-01-11T07:12:46.482659+0000","last_undegraded":"2024-01-11T07:12:46.482659+0000","last_fullsized":"2024-01-11T07:12:46.482659+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:27:20.031769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.482060+0000","last_change":"2024-01-11T07:12:46.481815+0000","last_active":"2024-01-11T07:12:46.482060+0000","last_peered":"2024-01-11T07:12:46.482060+0000","last_clean":"2024-01-11T07:12:46.482060+0000","last_became_active":"2024-01-11T07:12:46.481547+0000","last_became_peered":"2024-01-11T07:12:46.481547+0000","last_unstale":"2024-01-11T07:12:46.482060+0000","last_undegraded":"2024-01-11T07:12:46.482060+0000","last_fullsized":"2024-01-11T07:12:46.482060+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:45.892227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.481349+0000","last_change":"2024-01-11T07:12:45.489426+0000","last_active":"2024-01-11T07:12:46.481349+0000","last_peered":"2024-01-11T07:12:46.481349+0000","last_clean":"2024-01-11T07:12:46.481349+0000","last_became_active":"2024-01-11T07:12:45.488430+0000","last_became_peered":"2024-01-11T07:12:45.488430+0000","last_unstale":"2024-01-11T07:12:46.481349+0000","last_undegraded":"2024-01-11T07:12:46.481349+0000","last_fullsized":"2024-01-11T07:12:46.481349+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:00:03.913579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489239+0000","last_change":"2024-01-11T07:12:46.488516+0000","last_active":"2024-01-11T07:12:46.489239+0000","last_peered":"2024-01-11T07:12:46.489239+0000","last_clean":"2024-01-11T07:12:46.489239+0000","last_became_active":"2024-01-11T07:12:46.488110+0000","last_became_peered":"2024-01-11T07:12:46.488110+0000","last_unstale":"2024-01-11T07:12:46.489239+0000","last_undegraded":"2024-01-11T07:12:46.489239+0000","last_fullsized":"2024-01-11T07:12:46.489239+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:15:20.309612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.482313+0000","last_change":"2024-01-11T07:12:45.487160+0000","last_active":"2024-01-11T07:12:46.482313+0000","last_peered":"2024-01-11T07:12:46.482313+0000","last_clean":"2024-01-11T07:12:46.482313+0000","last_became_active":"2024-01-11T07:12:45.486914+0000","last_became_peered":"2024-01-11T07:12:45.486914+0000","last_unstale":"2024-01-11T07:12:46.482313+0000","last_undegraded":"2024-01-11T07:12:46.482313+0000","last_fullsized":"2024-01-11T07:12:46.482313+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:20:56.690624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.480807+0000","last_change":"2024-01-11T07:12:46.480100+0000","last_active":"2024-01-11T07:12:46.480807+0000","last_peered":"2024-01-11T07:12:46.480807+0000","last_clean":"2024-01-11T07:12:46.480807+0000","last_became_active":"2024-01-11T07:12:46.479857+0000","last_became_peered":"2024-01-11T07:12:46.479857+0000","last_unstale":"2024-01-11T07:12:46.480807+0000","last_undegraded":"2024-01-11T07:12:46.480807+0000","last_fullsized":"2024-01-11T07:12:46.480807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:12.892522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"119'3","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.495868+0000","last_change":"2024-01-11T07:12:45.482145+0000","last_active":"2024-01-11T07:12:46.495868+0000","last_peered":"2024-01-11T07:12:46.495868+0000","last_clean":"2024-01-11T07:12:46.495868+0000","last_became_active":"2024-01-11T07:12:45.481896+0000","last_became_peered":"2024-01-11T07:12:45.481896+0000","last_unstale":"2024-01-11T07:12:46.495868+0000","last_undegraded":"2024-01-11T07:12:46.495868+0000","last_fullsized":"2024-01-11T07:12:46.495868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:29:56.530253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.487692+0000","last_change":"2024-01-11T07:12:46.487616+0000","last_active":"2024-01-11T07:12:46.487692+0000","last_peered":"2024-01-11T07:12:46.487692+0000","last_clean":"2024-01-11T07:12:46.487692+0000","last_became_active":"2024-01-11T07:12:46.487368+0000","last_became_peered":"2024-01-11T07:12:46.487368+0000","last_unstale":"2024-01-11T07:12:46.487692+0000","last_undegraded":"2024-01-11T07:12:46.487692+0000","last_fullsized":"2024-01-11T07:12:46.487692+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:26.573595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.504164+0000","last_change":"2024-01-11T07:12:45.483782+0000","last_active":"2024-01-11T07:12:46.504164+0000","last_peered":"2024-01-11T07:12:46.504164+0000","last_clean":"2024-01-11T07:12:46.504164+0000","last_became_active":"2024-01-11T07:12:45.483521+0000","last_became_peered":"2024-01-11T07:12:45.483521+0000","last_unstale":"2024-01-11T07:12:46.504164+0000","last_undegraded":"2024-01-11T07:12:46.504164+0000","last_fullsized":"2024-01-11T07:12:46.504164+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:26:41.167127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.499123+0000","last_change":"2024-01-11T07:12:46.497719+0000","last_active":"2024-01-11T07:12:46.499123+0000","last_peered":"2024-01-11T07:12:46.499123+0000","last_clean":"2024-01-11T07:12:46.499123+0000","last_became_active":"2024-01-11T07:12:46.494288+0000","last_became_peered":"2024-01-11T07:12:46.494288+0000","last_unstale":"2024-01-11T07:12:46.499123+0000","last_undegraded":"2024-01-11T07:12:46.499123+0000","last_fullsized":"2024-01-11T07:12:46.499123+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:49:24.784110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.498297+0000","last_change":"2024-01-11T07:12:45.487212+0000","last_active":"2024-01-11T07:12:46.498297+0000","last_peered":"2024-01-11T07:12:46.498297+0000","last_clean":"2024-01-11T07:12:46.498297+0000","last_became_active":"2024-01-11T07:12:45.486879+0000","last_became_peered":"2024-01-11T07:12:45.486879+0000","last_unstale":"2024-01-11T07:12:46.498297+0000","last_undegraded":"2024-01-11T07:12:46.498297+0000","last_fullsized":"2024-01-11T07:12:46.498297+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:30:54.683822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489740+0000","last_change":"2024-01-11T07:12:46.489472+0000","last_active":"2024-01-11T07:12:46.489740+0000","last_peered":"2024-01-11T07:12:46.489740+0000","last_clean":"2024-01-11T07:12:46.489740+0000","last_became_active":"2024-01-11T07:12:46.488711+0000","last_became_peered":"2024-01-11T07:12:46.488711+0000","last_unstale":"2024-01-11T07:12:46.489740+0000","last_undegraded":"2024-01-11T07:12:46.489740+0000","last_fullsized":"2024-01-11T07:12:46.489740+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:38.059444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.483616+0000","last_change":"2024-01-11T07:12:45.488890+0000","last_active":"2024-01-11T07:12:46.483616+0000","last_peered":"2024-01-11T07:12:46.483616+0000","last_clean":"2024-01-11T07:12:46.483616+0000","last_became_active":"2024-01-11T07:12:45.488621+0000","last_became_peered":"2024-01-11T07:12:45.488621+0000","last_unstale":"2024-01-11T07:12:46.483616+0000","last_undegraded":"2024-01-11T07:12:46.483616+0000","last_fullsized":"2024-01-11T07:12:46.483616+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:48:31.873076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489502+0000","last_change":"2024-01-11T07:12:46.489379+0000","last_active":"2024-01-11T07:12:46.489502+0000","last_peered":"2024-01-11T07:12:46.489502+0000","last_clean":"2024-01-11T07:12:46.489502+0000","last_became_active":"2024-01-11T07:12:46.489150+0000","last_became_peered":"2024-01-11T07:12:46.489150+0000","last_unstale":"2024-01-11T07:12:46.489502+0000","last_undegraded":"2024-01-11T07:12:46.489502+0000","last_fullsized":"2024-01-11T07:12:46.489502+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:44:04.558093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.481021+0000","last_change":"2024-01-11T07:12:45.482088+0000","last_active":"2024-01-11T07:12:46.481021+0000","last_peered":"2024-01-11T07:12:46.481021+0000","last_clean":"2024-01-11T07:12:46.481021+0000","last_became_active":"2024-01-11T07:12:45.481778+0000","last_became_peered":"2024-01-11T07:12:45.481778+0000","last_unstale":"2024-01-11T07:12:46.481021+0000","last_undegraded":"2024-01-11T07:12:46.481021+0000","last_fullsized":"2024-01-11T07:12:46.481021+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:29:56.530253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489884+0000","last_change":"2024-01-11T07:12:46.489766+0000","last_active":"2024-01-11T07:12:46.489884+0000","last_peered":"2024-01-11T07:12:46.489884+0000","last_clean":"2024-01-11T07:12:46.489884+0000","last_became_active":"2024-01-11T07:12:46.489277+0000","last_became_peered":"2024-01-11T07:12:46.489277+0000","last_unstale":"2024-01-11T07:12:46.489884+0000","last_undegraded":"2024-01-11T07:12:46.489884+0000","last_fullsized":"2024-01-11T07:12:46.489884+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:16:26.786403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500646+0000","last_change":"2024-01-11T07:12:45.487798+0000","last_active":"2024-01-11T07:12:46.500646+0000","last_peered":"2024-01-11T07:12:46.500646+0000","last_clean":"2024-01-11T07:12:46.500646+0000","last_became_active":"2024-01-11T07:12:45.487545+0000","last_became_peered":"2024-01-11T07:12:45.487545+0000","last_unstale":"2024-01-11T07:12:46.500646+0000","last_undegraded":"2024-01-11T07:12:46.500646+0000","last_fullsized":"2024-01-11T07:12:46.500646+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:45:30.253518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485413+0000","last_change":"2024-01-11T07:12:46.485226+0000","last_active":"2024-01-11T07:12:46.485413+0000","last_peered":"2024-01-11T07:12:46.485413+0000","last_clean":"2024-01-11T07:12:46.485413+0000","last_became_active":"2024-01-11T07:12:46.484759+0000","last_became_peered":"2024-01-11T07:12:46.484759+0000","last_unstale":"2024-01-11T07:12:46.485413+0000","last_undegraded":"2024-01-11T07:12:46.485413+0000","last_fullsized":"2024-01-11T07:12:46.485413+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:20:57.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497980+0000","last_change":"2024-01-11T07:12:45.486498+0000","last_active":"2024-01-11T07:12:46.497980+0000","last_peered":"2024-01-11T07:12:46.497980+0000","last_clean":"2024-01-11T07:12:46.497980+0000","last_became_active":"2024-01-11T07:12:45.486259+0000","last_became_peered":"2024-01-11T07:12:45.486259+0000","last_unstale":"2024-01-11T07:12:46.497980+0000","last_undegraded":"2024-01-11T07:12:46.497980+0000","last_fullsized":"2024-01-11T07:12:46.497980+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:22.967591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484936+0000","last_change":"2024-01-11T07:12:46.484846+0000","last_active":"2024-01-11T07:12:46.484936+0000","last_peered":"2024-01-11T07:12:46.484936+0000","last_clean":"2024-01-11T07:12:46.484936+0000","last_became_active":"2024-01-11T07:12:46.484626+0000","last_became_peered":"2024-01-11T07:12:46.484626+0000","last_unstale":"2024-01-11T07:12:46.484936+0000","last_undegraded":"2024-01-11T07:12:46.484936+0000","last_fullsized":"2024-01-11T07:12:46.484936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:59:55.250999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497118+0000","last_change":"2024-01-11T07:12:45.485211+0000","last_active":"2024-01-11T07:12:46.497118+0000","last_peered":"2024-01-11T07:12:46.497118+0000","last_clean":"2024-01-11T07:12:46.497118+0000","last_became_active":"2024-01-11T07:12:45.484963+0000","last_became_peered":"2024-01-11T07:12:45.484963+0000","last_unstale":"2024-01-11T07:12:46.497118+0000","last_undegraded":"2024-01-11T07:12:46.497118+0000","last_fullsized":"2024-01-11T07:12:46.497118+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:55:36.288232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485029+0000","last_change":"2024-01-11T07:12:46.484140+0000","last_active":"2024-01-11T07:12:46.485029+0000","last_peered":"2024-01-11T07:12:46.485029+0000","last_clean":"2024-01-11T07:12:46.485029+0000","last_became_active":"2024-01-11T07:12:46.483200+0000","last_became_peered":"2024-01-11T07:12:46.483200+0000","last_unstale":"2024-01-11T07:12:46.485029+0000","last_undegraded":"2024-01-11T07:12:46.485029+0000","last_fullsized":"2024-01-11T07:12:46.485029+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:12.892522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.477704+0000","last_change":"2024-01-11T07:12:45.486893+0000","last_active":"2024-01-11T07:12:46.477704+0000","last_peered":"2024-01-11T07:12:46.477704+0000","last_clean":"2024-01-11T07:12:46.477704+0000","last_became_active":"2024-01-11T07:12:45.486644+0000","last_became_peered":"2024-01-11T07:12:45.486644+0000","last_unstale":"2024-01-11T07:12:46.477704+0000","last_undegraded":"2024-01-11T07:12:46.477704+0000","last_fullsized":"2024-01-11T07:12:46.477704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:33:03.273472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500559+0000","last_change":"2024-01-11T07:12:46.499504+0000","last_active":"2024-01-11T07:12:46.500559+0000","last_peered":"2024-01-11T07:12:46.500559+0000","last_clean":"2024-01-11T07:12:46.500559+0000","last_became_active":"2024-01-11T07:12:46.499265+0000","last_became_peered":"2024-01-11T07:12:46.499265+0000","last_unstale":"2024-01-11T07:12:46.500559+0000","last_undegraded":"2024-01-11T07:12:46.500559+0000","last_fullsized":"2024-01-11T07:12:46.500559+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:15:37.981680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.505106+0000","last_change":"2024-01-11T07:12:45.486525+0000","last_active":"2024-01-11T07:12:46.505106+0000","last_peered":"2024-01-11T07:12:46.505106+0000","last_clean":"2024-01-11T07:12:46.505106+0000","last_became_active":"2024-01-11T07:12:45.485349+0000","last_became_peered":"2024-01-11T07:12:45.485349+0000","last_unstale":"2024-01-11T07:12:46.505106+0000","last_undegraded":"2024-01-11T07:12:46.505106+0000","last_fullsized":"2024-01-11T07:12:46.505106+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:04:58.377351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484815+0000","last_change":"2024-01-11T07:12:46.484732+0000","last_active":"2024-01-11T07:12:46.484815+0000","last_peered":"2024-01-11T07:12:46.484815+0000","last_clean":"2024-01-11T07:12:46.484815+0000","last_became_active":"2024-01-11T07:12:46.484484+0000","last_became_peered":"2024-01-11T07:12:46.484484+0000","last_unstale":"2024-01-11T07:12:46.484815+0000","last_undegraded":"2024-01-11T07:12:46.484815+0000","last_fullsized":"2024-01-11T07:12:46.484815+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:15:07.322551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.482845+0000","last_change":"2024-01-11T07:12:45.487290+0000","last_active":"2024-01-11T07:12:46.482845+0000","last_peered":"2024-01-11T07:12:46.482845+0000","last_clean":"2024-01-11T07:12:46.482845+0000","last_became_active":"2024-01-11T07:12:45.487059+0000","last_became_peered":"2024-01-11T07:12:45.487059+0000","last_unstale":"2024-01-11T07:12:46.482845+0000","last_undegraded":"2024-01-11T07:12:46.482845+0000","last_fullsized":"2024-01-11T07:12:46.482845+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:55:05.550733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.502356+0000","last_change":"2024-01-11T07:12:46.502274+0000","last_active":"2024-01-11T07:12:46.502356+0000","last_peered":"2024-01-11T07:12:46.502356+0000","last_clean":"2024-01-11T07:12:46.502356+0000","last_became_active":"2024-01-11T07:12:46.502072+0000","last_became_peered":"2024-01-11T07:12:46.502072+0000","last_unstale":"2024-01-11T07:12:46.502356+0000","last_undegraded":"2024-01-11T07:12:46.502356+0000","last_fullsized":"2024-01-11T07:12:46.502356+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:26:47.737864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.480954+0000","last_change":"2024-01-11T07:12:45.488799+0000","last_active":"2024-01-11T07:12:46.480954+0000","last_peered":"2024-01-11T07:12:46.480954+0000","last_clean":"2024-01-11T07:12:46.480954+0000","last_became_active":"2024-01-11T07:12:45.488547+0000","last_became_peered":"2024-01-11T07:12:45.488547+0000","last_unstale":"2024-01-11T07:12:46.480954+0000","last_undegraded":"2024-01-11T07:12:46.480954+0000","last_fullsized":"2024-01-11T07:12:46.480954+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:23:30.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490782+0000","last_change":"2024-01-11T07:12:46.490715+0000","last_active":"2024-01-11T07:12:46.490782+0000","last_peered":"2024-01-11T07:12:46.490782+0000","last_clean":"2024-01-11T07:12:46.490782+0000","last_became_active":"2024-01-11T07:12:46.490282+0000","last_became_peered":"2024-01-11T07:12:46.490282+0000","last_unstale":"2024-01-11T07:12:46.490782+0000","last_undegraded":"2024-01-11T07:12:46.490782+0000","last_fullsized":"2024-01-11T07:12:46.490782+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:08:41.946213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.476777+0000","last_change":"2024-01-11T07:12:45.485891+0000","last_active":"2024-01-11T07:12:46.476777+0000","last_peered":"2024-01-11T07:12:46.476777+0000","last_clean":"2024-01-11T07:12:46.476777+0000","last_became_active":"2024-01-11T07:12:45.479745+0000","last_became_peered":"2024-01-11T07:12:45.479745+0000","last_unstale":"2024-01-11T07:12:46.476777+0000","last_undegraded":"2024-01-11T07:12:46.476777+0000","last_fullsized":"2024-01-11T07:12:46.476777+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:22:32.976243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496567+0000","last_change":"2024-01-11T07:12:46.493008+0000","last_active":"2024-01-11T07:12:46.496567+0000","last_peered":"2024-01-11T07:12:46.496567+0000","last_clean":"2024-01-11T07:12:46.496567+0000","last_became_active":"2024-01-11T07:12:46.487432+0000","last_became_peered":"2024-01-11T07:12:46.487432+0000","last_unstale":"2024-01-11T07:12:46.496567+0000","last_undegraded":"2024-01-11T07:12:46.496567+0000","last_fullsized":"2024-01-11T07:12:46.496567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:08:39.516302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.495036+0000","last_change":"2024-01-11T07:12:45.482766+0000","last_active":"2024-01-11T07:12:46.495036+0000","last_peered":"2024-01-11T07:12:46.495036+0000","last_clean":"2024-01-11T07:12:46.495036+0000","last_became_active":"2024-01-11T07:12:45.482510+0000","last_became_peered":"2024-01-11T07:12:45.482510+0000","last_unstale":"2024-01-11T07:12:46.495036+0000","last_undegraded":"2024-01-11T07:12:46.495036+0000","last_fullsized":"2024-01-11T07:12:46.495036+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:46:13.141613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489700+0000","last_change":"2024-01-11T07:12:46.487694+0000","last_active":"2024-01-11T07:12:46.489700+0000","last_peered":"2024-01-11T07:12:46.489700+0000","last_clean":"2024-01-11T07:12:46.489700+0000","last_became_active":"2024-01-11T07:12:46.487420+0000","last_became_peered":"2024-01-11T07:12:46.487420+0000","last_unstale":"2024-01-11T07:12:46.489700+0000","last_undegraded":"2024-01-11T07:12:46.489700+0000","last_fullsized":"2024-01-11T07:12:46.489700+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:09:28.624228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.478146+0000","last_change":"2024-01-11T07:12:45.486679+0000","last_active":"2024-01-11T07:12:46.478146+0000","last_peered":"2024-01-11T07:12:46.478146+0000","last_clean":"2024-01-11T07:12:46.478146+0000","last_became_active":"2024-01-11T07:12:45.485687+0000","last_became_peered":"2024-01-11T07:12:45.485687+0000","last_unstale":"2024-01-11T07:12:46.478146+0000","last_undegraded":"2024-01-11T07:12:46.478146+0000","last_fullsized":"2024-01-11T07:12:46.478146+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:44:03.553134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489763+0000","last_change":"2024-01-11T07:12:46.489578+0000","last_active":"2024-01-11T07:12:46.489763+0000","last_peered":"2024-01-11T07:12:46.489763+0000","last_clean":"2024-01-11T07:12:46.489763+0000","last_became_active":"2024-01-11T07:12:46.489118+0000","last_became_peered":"2024-01-11T07:12:46.489118+0000","last_unstale":"2024-01-11T07:12:46.489763+0000","last_undegraded":"2024-01-11T07:12:46.489763+0000","last_fullsized":"2024-01-11T07:12:46.489763+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:46:14.146572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.476559+0000","last_change":"2024-01-11T07:12:45.489422+0000","last_active":"2024-01-11T07:12:46.476559+0000","last_peered":"2024-01-11T07:12:46.476559+0000","last_clean":"2024-01-11T07:12:46.476559+0000","last_became_active":"2024-01-11T07:12:45.488739+0000","last_became_peered":"2024-01-11T07:12:45.488739+0000","last_unstale":"2024-01-11T07:12:46.476559+0000","last_undegraded":"2024-01-11T07:12:46.476559+0000","last_fullsized":"2024-01-11T07:12:46.476559+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:57:28.695701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485425+0000","last_change":"2024-01-11T07:12:46.485340+0000","last_active":"2024-01-11T07:12:46.485425+0000","last_peered":"2024-01-11T07:12:46.485425+0000","last_clean":"2024-01-11T07:12:46.485425+0000","last_became_active":"2024-01-11T07:12:46.485018+0000","last_became_peered":"2024-01-11T07:12:46.485018+0000","last_unstale":"2024-01-11T07:12:46.485425+0000","last_undegraded":"2024-01-11T07:12:46.485425+0000","last_fullsized":"2024-01-11T07:12:46.485425+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:04.611411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.479543+0000","last_change":"2024-01-11T07:12:45.485762+0000","last_active":"2024-01-11T07:12:46.479543+0000","last_peered":"2024-01-11T07:12:46.479543+0000","last_clean":"2024-01-11T07:12:46.479543+0000","last_became_active":"2024-01-11T07:12:45.485503+0000","last_became_peered":"2024-01-11T07:12:45.485503+0000","last_unstale":"2024-01-11T07:12:46.479543+0000","last_undegraded":"2024-01-11T07:12:46.479543+0000","last_fullsized":"2024-01-11T07:12:46.479543+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:36.883776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489851+0000","last_change":"2024-01-11T07:12:46.489568+0000","last_active":"2024-01-11T07:12:46.489851+0000","last_peered":"2024-01-11T07:12:46.489851+0000","last_clean":"2024-01-11T07:12:46.489851+0000","last_became_active":"2024-01-11T07:12:46.488911+0000","last_became_peered":"2024-01-11T07:12:46.488911+0000","last_unstale":"2024-01-11T07:12:46.489851+0000","last_undegraded":"2024-01-11T07:12:46.489851+0000","last_fullsized":"2024-01-11T07:12:46.489851+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:05:50.062232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.476921+0000","last_change":"2024-01-11T07:12:45.489119+0000","last_active":"2024-01-11T07:12:46.476921+0000","last_peered":"2024-01-11T07:12:46.476921+0000","last_clean":"2024-01-11T07:12:46.476921+0000","last_became_active":"2024-01-11T07:12:45.487696+0000","last_became_peered":"2024-01-11T07:12:45.487696+0000","last_unstale":"2024-01-11T07:12:46.476921+0000","last_undegraded":"2024-01-11T07:12:46.476921+0000","last_fullsized":"2024-01-11T07:12:46.476921+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:11.887562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.483768+0000","last_change":"2024-01-11T07:12:46.483650+0000","last_active":"2024-01-11T07:12:46.483768+0000","last_peered":"2024-01-11T07:12:46.483768+0000","last_clean":"2024-01-11T07:12:46.483768+0000","last_became_active":"2024-01-11T07:12:46.483407+0000","last_became_peered":"2024-01-11T07:12:46.483407+0000","last_unstale":"2024-01-11T07:12:46.483768+0000","last_undegraded":"2024-01-11T07:12:46.483768+0000","last_fullsized":"2024-01-11T07:12:46.483768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:57:29.700660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.476887+0000","last_change":"2024-01-11T07:12:45.488319+0000","last_active":"2024-01-11T07:12:46.476887+0000","last_peered":"2024-01-11T07:12:46.476887+0000","last_clean":"2024-01-11T07:12:46.476887+0000","last_became_active":"2024-01-11T07:12:45.488027+0000","last_became_peered":"2024-01-11T07:12:45.488027+0000","last_unstale":"2024-01-11T07:12:46.476887+0000","last_undegraded":"2024-01-11T07:12:46.476887+0000","last_fullsized":"2024-01-11T07:12:46.476887+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:01:31.568694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490628+0000","last_change":"2024-01-11T07:12:46.490430+0000","last_active":"2024-01-11T07:12:46.490628+0000","last_peered":"2024-01-11T07:12:46.490628+0000","last_clean":"2024-01-11T07:12:46.490628+0000","last_became_active":"2024-01-11T07:12:46.489456+0000","last_became_peered":"2024-01-11T07:12:46.489456+0000","last_unstale":"2024-01-11T07:12:46.490628+0000","last_undegraded":"2024-01-11T07:12:46.490628+0000","last_fullsized":"2024-01-11T07:12:46.490628+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:57.235755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484355+0000","last_change":"2024-01-11T07:12:45.484489+0000","last_active":"2024-01-11T07:12:46.484355+0000","last_peered":"2024-01-11T07:12:46.484355+0000","last_clean":"2024-01-11T07:12:46.484355+0000","last_became_active":"2024-01-11T07:12:45.484197+0000","last_became_peered":"2024-01-11T07:12:45.484197+0000","last_unstale":"2024-01-11T07:12:46.484355+0000","last_undegraded":"2024-01-11T07:12:46.484355+0000","last_fullsized":"2024-01-11T07:12:46.484355+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:03.606452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496567+0000","last_change":"2024-01-11T07:12:46.496500+0000","last_active":"2024-01-11T07:12:46.496567+0000","last_peered":"2024-01-11T07:12:46.496567+0000","last_clean":"2024-01-11T07:12:46.496567+0000","last_became_active":"2024-01-11T07:12:46.495890+0000","last_became_peered":"2024-01-11T07:12:46.495890+0000","last_unstale":"2024-01-11T07:12:46.496567+0000","last_undegraded":"2024-01-11T07:12:46.496567+0000","last_fullsized":"2024-01-11T07:12:46.496567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:02:14.806403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.487498+0000","last_change":"2024-01-11T07:12:45.487476+0000","last_active":"2024-01-11T07:12:46.487498+0000","last_peered":"2024-01-11T07:12:46.487498+0000","last_clean":"2024-01-11T07:12:46.487498+0000","last_became_active":"2024-01-11T07:12:45.486460+0000","last_became_peered":"2024-01-11T07:12:45.486460+0000","last_unstale":"2024-01-11T07:12:46.487498+0000","last_undegraded":"2024-01-11T07:12:46.487498+0000","last_fullsized":"2024-01-11T07:12:46.487498+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:23:49.736311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497770+0000","last_change":"2024-01-11T07:12:46.497700+0000","last_active":"2024-01-11T07:12:46.497770+0000","last_peered":"2024-01-11T07:12:46.497770+0000","last_clean":"2024-01-11T07:12:46.497770+0000","last_became_active":"2024-01-11T07:12:46.497467+0000","last_became_peered":"2024-01-11T07:12:46.497467+0000","last_unstale":"2024-01-11T07:12:46.497770+0000","last_undegraded":"2024-01-11T07:12:46.497770+0000","last_fullsized":"2024-01-11T07:12:46.497770+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:42:54.731518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497028+0000","last_change":"2024-01-11T07:12:45.487098+0000","last_active":"2024-01-11T07:12:46.497028+0000","last_peered":"2024-01-11T07:12:46.497028+0000","last_clean":"2024-01-11T07:12:46.497028+0000","last_became_active":"2024-01-11T07:12:45.486865+0000","last_became_peered":"2024-01-11T07:12:45.486865+0000","last_unstale":"2024-01-11T07:12:46.497028+0000","last_undegraded":"2024-01-11T07:12:46.497028+0000","last_fullsized":"2024-01-11T07:12:46.497028+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:35:11.863797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":251,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.477315+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:12:46.477315+0000","last_peered":"2024-01-11T07:12:46.477315+0000","last_clean":"2024-01-11T07:12:46.477315+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:12:46.477315+0000","last_undegraded":"2024-01-11T07:12:46.477315+0000","last_fullsized":"2024-01-11T07:12:46.477315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194094,"statfs":{"total":200000000,"available":198752967,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64400000000000002,"5min":0.754,"15min":0.754},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.496}]},{"osd":1,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70099999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.75900000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.53300000000000003},"min":{"1min":0.46500000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76100000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.747,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48299999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77900000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.378},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79900000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82899999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342823,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.42399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68000000000000005,"5min":1.157,"15min":1.157},"last":0.66200000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70799999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69399999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51800000000000002,"15min":0.54600000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66600000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53400000000000003,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.439}]},{"osd":5,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.379},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59699999999999998,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.53300000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67800000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71299999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":635,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194677,"statfs":{"total":200000000,"available":199349735,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.60699999999999998,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56699999999999995,"5min":20.41,"15min":20.41},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.57199999999999995,"15min":0.46600000000000003},"min":{"1min":0.313,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69199999999999995,"5min":15.611000000000001,"15min":15.611000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.69099999999999995,"15min":0.54900000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65300000000000002,"5min":20.199999999999999,"15min":20.199999999999999},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.66800000000000004,"15min":0.53900000000000003},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":20.369,"15min":20.369},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42699999999999999,"15min":0.437},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61399999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.70499999999999996,"15min":0.64300000000000002},"min":{"1min":0.41399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.83099999999999996,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.748,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85799999999999998,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":0.72899999999999998,"5min":1.413,"15min":1.413},"last":0.748},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72699999999999998,"15min":0.67400000000000004},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95299999999999996,"5min":9.891,"15min":9.891},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.746,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.88700000000000001,"15min":0.77100000000000002},"min":{"1min":0.48199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.91000000000000003,"5min":20.324000000000002,"15min":20.324000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69499999999999995,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92600000000000005,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194094,"statfs":{"total":200000000,"available":198752525,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42199999999999999,"15min":0.46999999999999997},"min":{"1min":0.27400000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.58899999999999997,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.627,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.379}]},{"osd":2,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.504,"15min":0.71199999999999997},"min":{"1min":0.32700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64200000000000002,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.47199999999999998,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.72099999999999997,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.504,"15min":0.72499999999999998},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56000000000000005,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51100000000000001,"15min":0.57399999999999995},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.65900000000000003,"5min":2.855,"15min":2.855},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61799999999999999,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71199999999999997,"15min":0.89300000000000002},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94999999999999996,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.48399999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67000000000000004,"15min":0.71199999999999997},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92200000000000004,"5min":2.819,"15min":2.819},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.498,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.94999999999999996},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.046,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.503},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.48099999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.014,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.54200000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93500000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76300000000000001,"15min":0.94399999999999995},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194671,"statfs":{"total":200000000,"available":199343413,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66800000000000004,"5min":1.472,"15min":1.472},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.439,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58199999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.59499999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.498}]},{"osd":3,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.74399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73499999999999999,"5min":1.139,"15min":1.139},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":1.099},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68999999999999995,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0229999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.82599999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.52900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98199999999999998,"5min":1.052,"15min":1.052},"last":1.006}]},{"osd":7,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67000000000000004,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":1.0409999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194669,"statfs":{"total":200000000,"available":199341889,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.50600000000000001,"15min":0.78700000000000003},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":11.481,"15min":11.481},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.48299999999999998},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.55900000000000005,"15min":0.97699999999999998},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60799999999999998,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.30199999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.56799999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.63,"15min":1.034},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.376}]},{"osd":4,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.40899999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91300000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.496},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.84899999999999998,"15min":1.2669999999999999},"min":{"1min":0.38500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84299999999999997,"5min":17.474,"15min":17.474},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.78900000000000003,"15min":1.2010000000000001},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.754,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.877,"5min":11.394,"15min":11.394},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.76500000000000001,"15min":1.0309999999999999},"min":{"1min":0.498,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.63},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.55200000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.84399999999999997,"15min":1.238},"min":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1839999999999999,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.52200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194669,"statfs":{"total":200000000,"available":199341787,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.77700000000000002,"15min":1.056},"min":{"1min":0.39900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.81399999999999995,"5min":8.484,"15min":8.484},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.79100000000000004,"15min":1.1120000000000001},"min":{"1min":0.45800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82799999999999996,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.68899999999999995}]},{"osd":1,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68200000000000005,"15min":0.86899999999999999},"min":{"1min":0.42299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.95299999999999996,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.73399999999999999,"15min":1.01},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.504}]},{"osd":2,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.76500000000000001,"15min":0.97899999999999998},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79300000000000004,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.66500000000000004,"15min":0.879},"min":{"1min":0.439,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85699999999999998,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.71599999999999997,"15min":1.05},"min":{"1min":0.375,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90800000000000003,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.79500000000000004,"15min":1.1220000000000001},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89100000000000001,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.59299999999999997,"15min":0.94199999999999995},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.60099999999999998,"15min":0.85999999999999999},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64200000000000002,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.38300000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.63300000000000001,"15min":0.91100000000000003},"min":{"1min":0.371,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70599999999999996,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69599999999999995,"5min":1.119,"15min":1.119},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63500000000000001,"15min":0.89100000000000001},"min":{"1min":0.39900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.629},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.72099999999999997,"15min":0.98899999999999999},"min":{"1min":0.44800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.751,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194671,"statfs":{"total":200000000,"available":199343377,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72799999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.48999999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76700000000000002,"5min":0.87,"15min":0.87},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.44400000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40899999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76600000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.46600000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78100000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.49399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.625,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.45000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81699999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.754,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.61699999999999999},"min":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.626}]}]}],"pool_statfs":[]}} 2024-01-11T07:12:50.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:12:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.844+0000 7f9a0133b700 1 Processor -- start 2024-01-11T07:12:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.844+0000 7f9a0133b700 1 -- start start 2024-01-11T07:12:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.845+0000 7f9a0133b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f99fc159ca0 0x7f99fc06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.845+0000 7f9a0133b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc0d18c0 con 0x7f99fc15a070 2024-01-11T07:12:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.845+0000 7f9a0133b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc0d0010 con 0x7f99fc0cef00 2024-01-11T07:12:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.845+0000 7f9a0133b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc0cf400 con 0x7f99fc159ca0 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.845+0000 7f99faffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f99fc159ca0 0x7f99fc06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.845+0000 7f99fa7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f99fc15a070 0x7f99fc0ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44980/0 (socket says 172.21.15.7:44980) 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.845+0000 7f99fa7fc700 1 -- 172.21.15.7:0/240959728 learned_addr learned my addr 172.21.15.7:0/240959728 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.846+0000 7f99faffd700 1 -- 172.21.15.7:0/240959728 >> v1:172.21.15.113:6789/0 conn(0x7f99fc0cef00 legacy=0x7f99fc0b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.846+0000 7f99faffd700 1 -- 172.21.15.7:0/240959728 >> v1:172.21.15.7:6789/0 conn(0x7f99fc15a070 legacy=0x7f99fc0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.846+0000 7f99faffd700 1 -- 172.21.15.7:0/240959728 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0cf6e0 con 0x7f99fc159ca0 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.846+0000 7f99f9ffb700 1 -- 172.21.15.7:0/240959728 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147461145 0 0) 0x7f99fc0d18c0 con 0x7f99fc15a070 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.846+0000 7f99faffd700 1 --2- 172.21.15.7:0/240959728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc06dc30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0098e0 tx=0x7f99ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9426e7a43bca8690 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.846+0000 7f99f9ffb700 1 -- 172.21.15.7:0/240959728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec009e80 con 0x7f99fc159ca0 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.846+0000 7f99f9ffb700 1 -- 172.21.15.7:0/240959728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec00b820 con 0x7f99fc159ca0 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.846+0000 7f99f9ffb700 1 -- 172.21.15.7:0/240959728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00bba0 con 0x7f99fc159ca0 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.847+0000 7f9a0133b700 1 -- 172.21.15.7:0/240959728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 msgr2=0x7f99fc06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.847+0000 7f9a0133b700 1 --2- 172.21.15.7:0/240959728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc06dc30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0098e0 tx=0x7f99ec0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.847+0000 7f9a0133b700 1 -- 172.21.15.7:0/240959728 shutdown_connections 2024-01-11T07:12:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.847+0000 7f9a0133b700 1 --2- 172.21.15.7:0/240959728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc06dc30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.847+0000 7f9a0133b700 1 -- 172.21.15.7:0/240959728 >> 172.21.15.7:0/240959728 conn(0x7f99fc063cc0 msgr2=0x7f99fc0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.847+0000 7f9a0133b700 1 -- 172.21.15.7:0/240959728 shutdown_connections 2024-01-11T07:12:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.847+0000 7f9a0133b700 1 -- 172.21.15.7:0/240959728 wait complete. 2024-01-11T07:12:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.849+0000 7f9a0133b700 1 Processor -- start 2024-01-11T07:12:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.849+0000 7f9a0133b700 1 -- start start 2024-01-11T07:12:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f9a0133b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99fc0cef00 0x7f99fc0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f9a0133b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f9a0133b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc15a070 0x7f99fc148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f9a0133b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0b6220 con 0x7f99fc159ca0 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f9a0133b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0c7f90 con 0x7f99fc15a070 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f9a0133b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0b5610 con 0x7f99fc0cef00 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc0b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32786/0 (socket says 172.21.15.7:32786) 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 -- 172.21.15.7:0/333732708 learned_addr learned my addr 172.21.15.7:0/333732708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99faffd700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99fc0cef00 0x7f99fc0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fb7fe700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc15a070 0x7f99fc148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 -- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99fc0cef00 msgr2=0x7f99fc0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99fc0cef00 0x7f99fc0ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 -- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc15a070 msgr2=0x7f99fc148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc15a070 0x7f99fc148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 -- 172.21.15.7:0/333732708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec009420 con 0x7f99fc159ca0 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99faffd700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99fc0cef00 0x7f99fc0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99fa7fc700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc0b4f10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f99e400c5b0 tx=0x7f99e4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99f2ffd700 1 -- 172.21.15.7:0/333732708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4005b30 con 0x7f99fc159ca0 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99f2ffd700 1 -- 172.21.15.7:0/333732708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e4005c90 con 0x7f99fc159ca0 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.850+0000 7f99f2ffd700 1 -- 172.21.15.7:0/333732708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4007300 con 0x7f99fc159ca0 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.851+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc0b58f0 con 0x7f99fc159ca0 2024-01-11T07:12:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.851+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc0b5eb0 con 0x7f99fc159ca0 2024-01-11T07:12:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.852+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc0b63d0 con 0x7f99fc159ca0 2024-01-11T07:12:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.852+0000 7f99f2ffd700 1 -- 172.21.15.7:0/333732708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f99e401b3b0 con 0x7f99fc159ca0 2024-01-11T07:12:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.853+0000 7f99f2ffd700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f99dc066f30 0x7f99dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.853+0000 7f99f2ffd700 1 -- 172.21.15.7:0/333732708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f99e4083a70 con 0x7f99fc159ca0 2024-01-11T07:12:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.853+0000 7f99faffd700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f99dc066f30 0x7f99dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.853+0000 7f99faffd700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f99dc066f30 0x7f99dc0693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0030c0 tx=0x7f99ec016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:50.857+0000 7f99f2ffd700 1 -- 172.21.15.7:0/333732708 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99fc0b63d0 con 0x7f99fc159ca0 2024-01-11T07:12:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.055+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99fc04c3c0 con 0x7f99dc066f30 2024-01-11T07:12:51.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.064+0000 7f99f2ffd700 1 -- 172.21.15.7:0/333732708 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285561 (secure 0 0 0) 0x7f99fc04c3c0 con 0x7f99dc066f30 2024-01-11T07:12:51.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f99dc066f30 msgr2=0x7f99dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f99dc066f30 0x7f99dc0693e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0030c0 tx=0x7f99ec016000 comp rx=0 tx=0).stop 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 msgr2=0x7f99fc0b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc0b4f10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f99e400c5b0 tx=0x7f99e4001070 comp rx=0 tx=0).stop 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 shutdown_connections 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f99dc066f30 0x7f99dc0693e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99fc15a070 0x7f99fc148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99fc0cef00 0x7f99fc0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 --2- 172.21.15.7:0/333732708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99fc159ca0 0x7f99fc0b4f10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 >> 172.21.15.7:0/333732708 conn(0x7f99fc063cc0 msgr2=0x7f99fc06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 shutdown_connections 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.067+0000 7f9a0133b700 1 -- 172.21.15.7:0/333732708 wait complete. 2024-01-11T07:12:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:12:51.084 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-01-11T07:12:49.922846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6280,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556219,"statfs":{"total":1600000000,"available":1593568516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454217"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.493536+0000","last_change":"2024-01-11T07:12:46.493452+0000","last_active":"2024-01-11T07:12:46.493536+0000","last_peered":"2024-01-11T07:12:46.493536+0000","last_clean":"2024-01-11T07:12:46.493536+0000","last_became_active":"2024-01-11T07:12:46.493177+0000","last_became_peered":"2024-01-11T07:12:46.493177+0000","last_unstale":"2024-01-11T07:12:46.493536+0000","last_undegraded":"2024-01-11T07:12:46.493536+0000","last_fullsized":"2024-01-11T07:12:46.493536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:01:32.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484826+0000","last_change":"2024-01-11T07:12:46.484666+0000","last_active":"2024-01-11T07:12:46.484826+0000","last_peered":"2024-01-11T07:12:46.484826+0000","last_clean":"2024-01-11T07:12:46.484826+0000","last_became_active":"2024-01-11T07:12:46.483727+0000","last_became_peered":"2024-01-11T07:12:46.483727+0000","last_unstale":"2024-01-11T07:12:46.484826+0000","last_undegraded":"2024-01-11T07:12:46.484826+0000","last_fullsized":"2024-01-11T07:12:46.484826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:58:53.624169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489928+0000","last_change":"2024-01-11T07:12:46.489759+0000","last_active":"2024-01-11T07:12:46.489928+0000","last_peered":"2024-01-11T07:12:46.489928+0000","last_clean":"2024-01-11T07:12:46.489928+0000","last_became_active":"2024-01-11T07:12:46.489495+0000","last_became_peered":"2024-01-11T07:12:46.489495+0000","last_unstale":"2024-01-11T07:12:46.489928+0000","last_undegraded":"2024-01-11T07:12:46.489928+0000","last_fullsized":"2024-01-11T07:12:46.489928+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:59:55.250999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500683+0000","last_change":"2024-01-11T07:12:46.500409+0000","last_active":"2024-01-11T07:12:46.500683+0000","last_peered":"2024-01-11T07:12:46.500683+0000","last_clean":"2024-01-11T07:12:46.500683+0000","last_became_active":"2024-01-11T07:12:46.499802+0000","last_became_peered":"2024-01-11T07:12:46.499802+0000","last_unstale":"2024-01-11T07:12:46.500683+0000","last_undegraded":"2024-01-11T07:12:46.500683+0000","last_fullsized":"2024-01-11T07:12:46.500683+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:57:29.700660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.495788+0000","last_change":"2024-01-11T07:12:46.495727+0000","last_active":"2024-01-11T07:12:46.495788+0000","last_peered":"2024-01-11T07:12:46.495788+0000","last_clean":"2024-01-11T07:12:46.495788+0000","last_became_active":"2024-01-11T07:12:46.495515+0000","last_became_peered":"2024-01-11T07:12:46.495515+0000","last_unstale":"2024-01-11T07:12:46.495788+0000","last_undegraded":"2024-01-11T07:12:46.495788+0000","last_fullsized":"2024-01-11T07:12:46.495788+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:42.973252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484967+0000","last_change":"2024-01-11T07:12:46.482720+0000","last_active":"2024-01-11T07:12:46.484967+0000","last_peered":"2024-01-11T07:12:46.484967+0000","last_clean":"2024-01-11T07:12:46.484967+0000","last_became_active":"2024-01-11T07:12:46.482434+0000","last_became_peered":"2024-01-11T07:12:46.482434+0000","last_unstale":"2024-01-11T07:12:46.484967+0000","last_undegraded":"2024-01-11T07:12:46.484967+0000","last_fullsized":"2024-01-11T07:12:46.484967+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:32.957971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484988+0000","last_change":"2024-01-11T07:12:46.482994+0000","last_active":"2024-01-11T07:12:46.484988+0000","last_peered":"2024-01-11T07:12:46.484988+0000","last_clean":"2024-01-11T07:12:46.484988+0000","last_became_active":"2024-01-11T07:12:46.482749+0000","last_became_peered":"2024-01-11T07:12:46.482749+0000","last_unstale":"2024-01-11T07:12:46.484988+0000","last_undegraded":"2024-01-11T07:12:46.484988+0000","last_fullsized":"2024-01-11T07:12:46.484988+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:16.225134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496585+0000","last_change":"2024-01-11T07:12:46.493197+0000","last_active":"2024-01-11T07:12:46.496585+0000","last_peered":"2024-01-11T07:12:46.496585+0000","last_clean":"2024-01-11T07:12:46.496585+0000","last_became_active":"2024-01-11T07:12:46.487933+0000","last_became_peered":"2024-01-11T07:12:46.487933+0000","last_unstale":"2024-01-11T07:12:46.496585+0000","last_undegraded":"2024-01-11T07:12:46.496585+0000","last_fullsized":"2024-01-11T07:12:46.496585+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:48:28.000032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500524+0000","last_change":"2024-01-11T07:12:46.500478+0000","last_active":"2024-01-11T07:12:46.500524+0000","last_peered":"2024-01-11T07:12:46.500524+0000","last_clean":"2024-01-11T07:12:46.500524+0000","last_became_active":"2024-01-11T07:12:46.500344+0000","last_became_peered":"2024-01-11T07:12:46.500344+0000","last_unstale":"2024-01-11T07:12:46.500524+0000","last_undegraded":"2024-01-11T07:12:46.500524+0000","last_fullsized":"2024-01-11T07:12:46.500524+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:12.892522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.487386+0000","last_change":"2024-01-11T07:12:46.487295+0000","last_active":"2024-01-11T07:12:46.487386+0000","last_peered":"2024-01-11T07:12:46.487386+0000","last_clean":"2024-01-11T07:12:46.487386+0000","last_became_active":"2024-01-11T07:12:46.487020+0000","last_became_peered":"2024-01-11T07:12:46.487020+0000","last_unstale":"2024-01-11T07:12:46.487386+0000","last_undegraded":"2024-01-11T07:12:46.487386+0000","last_fullsized":"2024-01-11T07:12:46.487386+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:44:04.558093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485047+0000","last_change":"2024-01-11T07:12:46.484280+0000","last_active":"2024-01-11T07:12:46.485047+0000","last_peered":"2024-01-11T07:12:46.485047+0000","last_clean":"2024-01-11T07:12:46.485047+0000","last_became_active":"2024-01-11T07:12:46.483491+0000","last_became_peered":"2024-01-11T07:12:46.483491+0000","last_unstale":"2024-01-11T07:12:46.485047+0000","last_undegraded":"2024-01-11T07:12:46.485047+0000","last_fullsized":"2024-01-11T07:12:46.485047+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:33:04.278431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.494801+0000","last_change":"2024-01-11T07:12:46.494583+0000","last_active":"2024-01-11T07:12:46.494801+0000","last_peered":"2024-01-11T07:12:46.494801+0000","last_clean":"2024-01-11T07:12:46.494801+0000","last_became_active":"2024-01-11T07:12:46.494353+0000","last_became_peered":"2024-01-11T07:12:46.494353+0000","last_unstale":"2024-01-11T07:12:46.494801+0000","last_undegraded":"2024-01-11T07:12:46.494801+0000","last_fullsized":"2024-01-11T07:12:46.494801+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:16.225134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496153+0000","last_change":"2024-01-11T07:12:46.495599+0000","last_active":"2024-01-11T07:12:46.496153+0000","last_peered":"2024-01-11T07:12:46.496153+0000","last_clean":"2024-01-11T07:12:46.496153+0000","last_became_active":"2024-01-11T07:12:46.494941+0000","last_became_peered":"2024-01-11T07:12:46.494941+0000","last_unstale":"2024-01-11T07:12:46.496153+0000","last_undegraded":"2024-01-11T07:12:46.496153+0000","last_fullsized":"2024-01-11T07:12:46.496153+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:04:59.382310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.491503+0000","last_change":"2024-01-11T07:12:46.491393+0000","last_active":"2024-01-11T07:12:46.491503+0000","last_peered":"2024-01-11T07:12:46.491503+0000","last_clean":"2024-01-11T07:12:46.491503+0000","last_became_active":"2024-01-11T07:12:46.491148+0000","last_became_peered":"2024-01-11T07:12:46.491148+0000","last_unstale":"2024-01-11T07:12:46.491503+0000","last_undegraded":"2024-01-11T07:12:46.491503+0000","last_fullsized":"2024-01-11T07:12:46.491503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:11:10.842352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.503245+0000","last_change":"2024-01-11T07:12:46.503028+0000","last_active":"2024-01-11T07:12:46.503245+0000","last_peered":"2024-01-11T07:12:46.503245+0000","last_clean":"2024-01-11T07:12:46.503245+0000","last_became_active":"2024-01-11T07:12:46.502513+0000","last_became_peered":"2024-01-11T07:12:46.502513+0000","last_unstale":"2024-01-11T07:12:46.503245+0000","last_undegraded":"2024-01-11T07:12:46.503245+0000","last_fullsized":"2024-01-11T07:12:46.503245+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:25:50.736749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.480554+0000","last_change":"2024-01-11T07:12:46.479991+0000","last_active":"2024-01-11T07:12:46.480554+0000","last_peered":"2024-01-11T07:12:46.480554+0000","last_clean":"2024-01-11T07:12:46.480554+0000","last_became_active":"2024-01-11T07:12:46.479757+0000","last_became_peered":"2024-01-11T07:12:46.479757+0000","last_unstale":"2024-01-11T07:12:46.480554+0000","last_undegraded":"2024-01-11T07:12:46.480554+0000","last_fullsized":"2024-01-11T07:12:46.480554+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:49:24.784110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500518+0000","last_change":"2024-01-11T07:12:46.500286+0000","last_active":"2024-01-11T07:12:46.500518+0000","last_peered":"2024-01-11T07:12:46.500518+0000","last_clean":"2024-01-11T07:12:46.500518+0000","last_became_active":"2024-01-11T07:12:46.499778+0000","last_became_peered":"2024-01-11T07:12:46.499778+0000","last_unstale":"2024-01-11T07:12:46.500518+0000","last_undegraded":"2024-01-11T07:12:46.500518+0000","last_fullsized":"2024-01-11T07:12:46.500518+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:48:31.698041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.499099+0000","last_change":"2024-01-11T07:12:46.496242+0000","last_active":"2024-01-11T07:12:46.499099+0000","last_peered":"2024-01-11T07:12:46.499099+0000","last_clean":"2024-01-11T07:12:46.499099+0000","last_became_active":"2024-01-11T07:12:46.495871+0000","last_became_peered":"2024-01-11T07:12:46.495871+0000","last_unstale":"2024-01-11T07:12:46.499099+0000","last_undegraded":"2024-01-11T07:12:46.499099+0000","last_fullsized":"2024-01-11T07:12:46.499099+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:16.225134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490609+0000","last_change":"2024-01-11T07:12:46.490188+0000","last_active":"2024-01-11T07:12:46.490609+0000","last_peered":"2024-01-11T07:12:46.490609+0000","last_clean":"2024-01-11T07:12:46.490609+0000","last_became_active":"2024-01-11T07:12:46.488730+0000","last_became_peered":"2024-01-11T07:12:46.488730+0000","last_unstale":"2024-01-11T07:12:46.490609+0000","last_undegraded":"2024-01-11T07:12:46.490609+0000","last_fullsized":"2024-01-11T07:12:46.490609+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:55:18.207452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497199+0000","last_change":"2024-01-11T07:12:46.497130+0000","last_active":"2024-01-11T07:12:46.497199+0000","last_peered":"2024-01-11T07:12:46.497199+0000","last_clean":"2024-01-11T07:12:46.497199+0000","last_became_active":"2024-01-11T07:12:46.496900+0000","last_became_peered":"2024-01-11T07:12:46.496900+0000","last_unstale":"2024-01-11T07:12:46.497199+0000","last_undegraded":"2024-01-11T07:12:46.497199+0000","last_fullsized":"2024-01-11T07:12:46.497199+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:26.573595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.486715+0000","last_change":"2024-01-11T07:12:46.486575+0000","last_active":"2024-01-11T07:12:46.486715+0000","last_peered":"2024-01-11T07:12:46.486715+0000","last_clean":"2024-01-11T07:12:46.486715+0000","last_became_active":"2024-01-11T07:12:46.486317+0000","last_became_peered":"2024-01-11T07:12:46.486317+0000","last_unstale":"2024-01-11T07:12:46.486715+0000","last_undegraded":"2024-01-11T07:12:46.486715+0000","last_fullsized":"2024-01-11T07:12:46.486715+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:37:29.109240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484810+0000","last_change":"2024-01-11T07:12:46.484495+0000","last_active":"2024-01-11T07:12:46.484810+0000","last_peered":"2024-01-11T07:12:46.484810+0000","last_clean":"2024-01-11T07:12:46.484810+0000","last_became_active":"2024-01-11T07:12:46.483377+0000","last_became_peered":"2024-01-11T07:12:46.483377+0000","last_unstale":"2024-01-11T07:12:46.484810+0000","last_undegraded":"2024-01-11T07:12:46.484810+0000","last_fullsized":"2024-01-11T07:12:46.484810+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:25:21.010285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496549+0000","last_change":"2024-01-11T07:12:46.488178+0000","last_active":"2024-01-11T07:12:46.496549+0000","last_peered":"2024-01-11T07:12:46.496549+0000","last_clean":"2024-01-11T07:12:46.496549+0000","last_became_active":"2024-01-11T07:12:46.487106+0000","last_became_peered":"2024-01-11T07:12:46.487106+0000","last_unstale":"2024-01-11T07:12:46.496549+0000","last_undegraded":"2024-01-11T07:12:46.496549+0000","last_fullsized":"2024-01-11T07:12:46.496549+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:58:08.855673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490013+0000","last_change":"2024-01-11T07:12:46.489944+0000","last_active":"2024-01-11T07:12:46.490013+0000","last_peered":"2024-01-11T07:12:46.490013+0000","last_clean":"2024-01-11T07:12:46.490013+0000","last_became_active":"2024-01-11T07:12:46.489698+0000","last_became_peered":"2024-01-11T07:12:46.489698+0000","last_unstale":"2024-01-11T07:12:46.490013+0000","last_undegraded":"2024-01-11T07:12:46.490013+0000","last_fullsized":"2024-01-11T07:12:46.490013+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:04:59.382310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484027+0000","last_change":"2024-01-11T07:12:46.483843+0000","last_active":"2024-01-11T07:12:46.484027+0000","last_peered":"2024-01-11T07:12:46.484027+0000","last_clean":"2024-01-11T07:12:46.484027+0000","last_became_active":"2024-01-11T07:12:46.483613+0000","last_became_peered":"2024-01-11T07:12:46.483613+0000","last_unstale":"2024-01-11T07:12:46.484027+0000","last_undegraded":"2024-01-11T07:12:46.484027+0000","last_fullsized":"2024-01-11T07:12:46.484027+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:15:20.309612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484791+0000","last_change":"2024-01-11T07:12:46.484374+0000","last_active":"2024-01-11T07:12:46.484791+0000","last_peered":"2024-01-11T07:12:46.484791+0000","last_clean":"2024-01-11T07:12:46.484791+0000","last_became_active":"2024-01-11T07:12:46.484119+0000","last_became_peered":"2024-01-11T07:12:46.484119+0000","last_unstale":"2024-01-11T07:12:46.484791+0000","last_undegraded":"2024-01-11T07:12:46.484791+0000","last_fullsized":"2024-01-11T07:12:46.484791+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:28.177221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485474+0000","last_change":"2024-01-11T07:12:46.485362+0000","last_active":"2024-01-11T07:12:46.485474+0000","last_peered":"2024-01-11T07:12:46.485474+0000","last_clean":"2024-01-11T07:12:46.485474+0000","last_became_active":"2024-01-11T07:12:46.485129+0000","last_became_peered":"2024-01-11T07:12:46.485129+0000","last_unstale":"2024-01-11T07:12:46.485474+0000","last_undegraded":"2024-01-11T07:12:46.485474+0000","last_fullsized":"2024-01-11T07:12:46.485474+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:22:33.981202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496373+0000","last_change":"2024-01-11T07:12:46.496302+0000","last_active":"2024-01-11T07:12:46.496373+0000","last_peered":"2024-01-11T07:12:46.496373+0000","last_clean":"2024-01-11T07:12:46.496373+0000","last_became_active":"2024-01-11T07:12:46.495541+0000","last_became_peered":"2024-01-11T07:12:46.495541+0000","last_unstale":"2024-01-11T07:12:46.496373+0000","last_undegraded":"2024-01-11T07:12:46.496373+0000","last_fullsized":"2024-01-11T07:12:46.496373+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:35:56.154787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496526+0000","last_change":"2024-01-11T07:12:46.486216+0000","last_active":"2024-01-11T07:12:46.496526+0000","last_peered":"2024-01-11T07:12:46.496526+0000","last_clean":"2024-01-11T07:12:46.496526+0000","last_became_active":"2024-01-11T07:12:46.485965+0000","last_became_peered":"2024-01-11T07:12:46.485965+0000","last_unstale":"2024-01-11T07:12:46.496526+0000","last_undegraded":"2024-01-11T07:12:46.496526+0000","last_fullsized":"2024-01-11T07:12:46.496526+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:05.852720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.492768+0000","last_change":"2024-01-11T07:12:46.492676+0000","last_active":"2024-01-11T07:12:46.492768+0000","last_peered":"2024-01-11T07:12:46.492768+0000","last_clean":"2024-01-11T07:12:46.492768+0000","last_became_active":"2024-01-11T07:12:46.492434+0000","last_became_peered":"2024-01-11T07:12:46.492434+0000","last_unstale":"2024-01-11T07:12:46.492768+0000","last_undegraded":"2024-01-11T07:12:46.492768+0000","last_fullsized":"2024-01-11T07:12:46.492768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:24:42.079612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.499158+0000","last_change":"2024-01-11T07:12:46.498816+0000","last_active":"2024-01-11T07:12:46.499158+0000","last_peered":"2024-01-11T07:12:46.499158+0000","last_clean":"2024-01-11T07:12:46.499158+0000","last_became_active":"2024-01-11T07:12:46.498005+0000","last_became_peered":"2024-01-11T07:12:46.498005+0000","last_unstale":"2024-01-11T07:12:46.499158+0000","last_undegraded":"2024-01-11T07:12:46.499158+0000","last_fullsized":"2024-01-11T07:12:46.499158+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:45:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.486708+0000","last_change":"2024-01-11T07:12:46.486097+0000","last_active":"2024-01-11T07:12:46.486708+0000","last_peered":"2024-01-11T07:12:46.486708+0000","last_clean":"2024-01-11T07:12:46.486708+0000","last_became_active":"2024-01-11T07:12:46.485824+0000","last_became_peered":"2024-01-11T07:12:46.485824+0000","last_unstale":"2024-01-11T07:12:46.486708+0000","last_undegraded":"2024-01-11T07:12:46.486708+0000","last_fullsized":"2024-01-11T07:12:46.486708+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:37.888735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489802+0000","last_change":"2024-01-11T07:12:46.489132+0000","last_active":"2024-01-11T07:12:46.489802+0000","last_peered":"2024-01-11T07:12:46.489802+0000","last_clean":"2024-01-11T07:12:46.489802+0000","last_became_active":"2024-01-11T07:12:46.488365+0000","last_became_peered":"2024-01-11T07:12:46.488365+0000","last_unstale":"2024-01-11T07:12:46.489802+0000","last_undegraded":"2024-01-11T07:12:46.489802+0000","last_fullsized":"2024-01-11T07:12:46.489802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:21:16.163722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.505167+0000","last_change":"2024-01-11T07:12:45.486817+0000","last_active":"2024-01-11T07:12:46.505167+0000","last_peered":"2024-01-11T07:12:46.505167+0000","last_clean":"2024-01-11T07:12:46.505167+0000","last_became_active":"2024-01-11T07:12:45.486275+0000","last_became_peered":"2024-01-11T07:12:45.486275+0000","last_unstale":"2024-01-11T07:12:46.505167+0000","last_undegraded":"2024-01-11T07:12:46.505167+0000","last_fullsized":"2024-01-11T07:12:46.505167+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:08:15.220175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496229+0000","last_change":"2024-01-11T07:12:46.496100+0000","last_active":"2024-01-11T07:12:46.496229+0000","last_peered":"2024-01-11T07:12:46.496229+0000","last_clean":"2024-01-11T07:12:46.496229+0000","last_became_active":"2024-01-11T07:12:46.495337+0000","last_became_peered":"2024-01-11T07:12:46.495337+0000","last_unstale":"2024-01-11T07:12:46.496229+0000","last_undegraded":"2024-01-11T07:12:46.496229+0000","last_fullsized":"2024-01-11T07:12:46.496229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:31:51.803993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500236+0000","last_change":"2024-01-11T07:12:45.488589+0000","last_active":"2024-01-11T07:12:46.500236+0000","last_peered":"2024-01-11T07:12:46.500236+0000","last_clean":"2024-01-11T07:12:46.500236+0000","last_became_active":"2024-01-11T07:12:45.488356+0000","last_became_peered":"2024-01-11T07:12:45.488356+0000","last_unstale":"2024-01-11T07:12:46.500236+0000","last_undegraded":"2024-01-11T07:12:46.500236+0000","last_fullsized":"2024-01-11T07:12:46.500236+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:12:29.270653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490608+0000","last_change":"2024-01-11T07:12:46.490527+0000","last_active":"2024-01-11T07:12:46.490608+0000","last_peered":"2024-01-11T07:12:46.490608+0000","last_clean":"2024-01-11T07:12:46.490608+0000","last_became_active":"2024-01-11T07:12:46.490281+0000","last_became_peered":"2024-01-11T07:12:46.490281+0000","last_unstale":"2024-01-11T07:12:46.490608+0000","last_undegraded":"2024-01-11T07:12:46.490608+0000","last_fullsized":"2024-01-11T07:12:46.490608+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:52:59.846219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.501319+0000","last_change":"2024-01-11T07:12:45.485614+0000","last_active":"2024-01-11T07:12:46.501319+0000","last_peered":"2024-01-11T07:12:46.501319+0000","last_clean":"2024-01-11T07:12:46.501319+0000","last_became_active":"2024-01-11T07:12:45.485293+0000","last_became_peered":"2024-01-11T07:12:45.485293+0000","last_unstale":"2024-01-11T07:12:46.501319+0000","last_undegraded":"2024-01-11T07:12:46.501319+0000","last_fullsized":"2024-01-11T07:12:46.501319+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:31.953012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489259+0000","last_change":"2024-01-11T07:12:46.488522+0000","last_active":"2024-01-11T07:12:46.489259+0000","last_peered":"2024-01-11T07:12:46.489259+0000","last_clean":"2024-01-11T07:12:46.489259+0000","last_became_active":"2024-01-11T07:12:46.488125+0000","last_became_peered":"2024-01-11T07:12:46.488125+0000","last_unstale":"2024-01-11T07:12:46.489259+0000","last_undegraded":"2024-01-11T07:12:46.489259+0000","last_fullsized":"2024-01-11T07:12:46.489259+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:32.957971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.502462+0000","last_change":"2024-01-11T07:12:45.489292+0000","last_active":"2024-01-11T07:12:46.502462+0000","last_peered":"2024-01-11T07:12:46.502462+0000","last_clean":"2024-01-11T07:12:46.502462+0000","last_became_active":"2024-01-11T07:12:45.488073+0000","last_became_peered":"2024-01-11T07:12:45.488073+0000","last_unstale":"2024-01-11T07:12:46.502462+0000","last_undegraded":"2024-01-11T07:12:46.502462+0000","last_fullsized":"2024-01-11T07:12:46.502462+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:25.568635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490358+0000","last_change":"2024-01-11T07:12:46.490282+0000","last_active":"2024-01-11T07:12:46.490358+0000","last_peered":"2024-01-11T07:12:46.490358+0000","last_clean":"2024-01-11T07:12:46.490358+0000","last_became_active":"2024-01-11T07:12:46.490027+0000","last_became_peered":"2024-01-11T07:12:46.490027+0000","last_unstale":"2024-01-11T07:12:46.490358+0000","last_undegraded":"2024-01-11T07:12:46.490358+0000","last_fullsized":"2024-01-11T07:12:46.490358+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:37.888735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.499964+0000","last_change":"2024-01-11T07:12:45.485661+0000","last_active":"2024-01-11T07:12:46.499964+0000","last_peered":"2024-01-11T07:12:46.499964+0000","last_clean":"2024-01-11T07:12:46.499964+0000","last_became_active":"2024-01-11T07:12:45.485148+0000","last_became_peered":"2024-01-11T07:12:45.485148+0000","last_unstale":"2024-01-11T07:12:46.499964+0000","last_undegraded":"2024-01-11T07:12:46.499964+0000","last_fullsized":"2024-01-11T07:12:46.499964+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:13:34.267082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.479269+0000","last_change":"2024-01-11T07:12:46.479196+0000","last_active":"2024-01-11T07:12:46.479269+0000","last_peered":"2024-01-11T07:12:46.479269+0000","last_clean":"2024-01-11T07:12:46.479269+0000","last_became_active":"2024-01-11T07:12:46.478866+0000","last_became_peered":"2024-01-11T07:12:46.478866+0000","last_unstale":"2024-01-11T07:12:46.479269+0000","last_undegraded":"2024-01-11T07:12:46.479269+0000","last_fullsized":"2024-01-11T07:12:46.479269+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:57:29.700660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.505228+0000","last_change":"2024-01-11T07:12:45.484742+0000","last_active":"2024-01-11T07:12:46.505228+0000","last_peered":"2024-01-11T07:12:46.505228+0000","last_clean":"2024-01-11T07:12:46.505228+0000","last_became_active":"2024-01-11T07:12:45.477948+0000","last_became_peered":"2024-01-11T07:12:45.477948+0000","last_unstale":"2024-01-11T07:12:46.505228+0000","last_undegraded":"2024-01-11T07:12:46.505228+0000","last_fullsized":"2024-01-11T07:12:46.505228+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:41.968293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.503255+0000","last_change":"2024-01-11T07:12:46.503184+0000","last_active":"2024-01-11T07:12:46.503255+0000","last_peered":"2024-01-11T07:12:46.503255+0000","last_clean":"2024-01-11T07:12:46.503255+0000","last_became_active":"2024-01-11T07:12:46.502859+0000","last_became_peered":"2024-01-11T07:12:46.502859+0000","last_unstale":"2024-01-11T07:12:46.503255+0000","last_undegraded":"2024-01-11T07:12:46.503255+0000","last_fullsized":"2024-01-11T07:12:46.503255+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:38:11.890732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.478476+0000","last_change":"2024-01-11T07:12:45.489677+0000","last_active":"2024-01-11T07:12:46.478476+0000","last_peered":"2024-01-11T07:12:46.478476+0000","last_clean":"2024-01-11T07:12:46.478476+0000","last_became_active":"2024-01-11T07:12:45.489435+0000","last_became_peered":"2024-01-11T07:12:45.489435+0000","last_unstale":"2024-01-11T07:12:46.478476+0000","last_undegraded":"2024-01-11T07:12:46.478476+0000","last_fullsized":"2024-01-11T07:12:46.478476+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:59:54.246040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489275+0000","last_change":"2024-01-11T07:12:46.488546+0000","last_active":"2024-01-11T07:12:46.489275+0000","last_peered":"2024-01-11T07:12:46.489275+0000","last_clean":"2024-01-11T07:12:46.489275+0000","last_became_active":"2024-01-11T07:12:46.488160+0000","last_became_peered":"2024-01-11T07:12:46.488160+0000","last_unstale":"2024-01-11T07:12:46.489275+0000","last_undegraded":"2024-01-11T07:12:46.489275+0000","last_fullsized":"2024-01-11T07:12:46.489275+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:19:42.973252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.482659+0000","last_change":"2024-01-11T07:12:45.489125+0000","last_active":"2024-01-11T07:12:46.482659+0000","last_peered":"2024-01-11T07:12:46.482659+0000","last_clean":"2024-01-11T07:12:46.482659+0000","last_became_active":"2024-01-11T07:12:45.488883+0000","last_became_peered":"2024-01-11T07:12:45.488883+0000","last_unstale":"2024-01-11T07:12:46.482659+0000","last_undegraded":"2024-01-11T07:12:46.482659+0000","last_fullsized":"2024-01-11T07:12:46.482659+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:27:20.031769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.482060+0000","last_change":"2024-01-11T07:12:46.481815+0000","last_active":"2024-01-11T07:12:46.482060+0000","last_peered":"2024-01-11T07:12:46.482060+0000","last_clean":"2024-01-11T07:12:46.482060+0000","last_became_active":"2024-01-11T07:12:46.481547+0000","last_became_peered":"2024-01-11T07:12:46.481547+0000","last_unstale":"2024-01-11T07:12:46.482060+0000","last_undegraded":"2024-01-11T07:12:46.482060+0000","last_fullsized":"2024-01-11T07:12:46.482060+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:45.892227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.481349+0000","last_change":"2024-01-11T07:12:45.489426+0000","last_active":"2024-01-11T07:12:46.481349+0000","last_peered":"2024-01-11T07:12:46.481349+0000","last_clean":"2024-01-11T07:12:46.481349+0000","last_became_active":"2024-01-11T07:12:45.488430+0000","last_became_peered":"2024-01-11T07:12:45.488430+0000","last_unstale":"2024-01-11T07:12:46.481349+0000","last_undegraded":"2024-01-11T07:12:46.481349+0000","last_fullsized":"2024-01-11T07:12:46.481349+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:00:03.913579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489239+0000","last_change":"2024-01-11T07:12:46.488516+0000","last_active":"2024-01-11T07:12:46.489239+0000","last_peered":"2024-01-11T07:12:46.489239+0000","last_clean":"2024-01-11T07:12:46.489239+0000","last_became_active":"2024-01-11T07:12:46.488110+0000","last_became_peered":"2024-01-11T07:12:46.488110+0000","last_unstale":"2024-01-11T07:12:46.489239+0000","last_undegraded":"2024-01-11T07:12:46.489239+0000","last_fullsized":"2024-01-11T07:12:46.489239+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:15:20.309612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.482313+0000","last_change":"2024-01-11T07:12:45.487160+0000","last_active":"2024-01-11T07:12:46.482313+0000","last_peered":"2024-01-11T07:12:46.482313+0000","last_clean":"2024-01-11T07:12:46.482313+0000","last_became_active":"2024-01-11T07:12:45.486914+0000","last_became_peered":"2024-01-11T07:12:45.486914+0000","last_unstale":"2024-01-11T07:12:46.482313+0000","last_undegraded":"2024-01-11T07:12:46.482313+0000","last_fullsized":"2024-01-11T07:12:46.482313+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:20:56.690624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.480807+0000","last_change":"2024-01-11T07:12:46.480100+0000","last_active":"2024-01-11T07:12:46.480807+0000","last_peered":"2024-01-11T07:12:46.480807+0000","last_clean":"2024-01-11T07:12:46.480807+0000","last_became_active":"2024-01-11T07:12:46.479857+0000","last_became_peered":"2024-01-11T07:12:46.479857+0000","last_unstale":"2024-01-11T07:12:46.480807+0000","last_undegraded":"2024-01-11T07:12:46.480807+0000","last_fullsized":"2024-01-11T07:12:46.480807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:12.892522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"119'3","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.495868+0000","last_change":"2024-01-11T07:12:45.482145+0000","last_active":"2024-01-11T07:12:46.495868+0000","last_peered":"2024-01-11T07:12:46.495868+0000","last_clean":"2024-01-11T07:12:46.495868+0000","last_became_active":"2024-01-11T07:12:45.481896+0000","last_became_peered":"2024-01-11T07:12:45.481896+0000","last_unstale":"2024-01-11T07:12:46.495868+0000","last_undegraded":"2024-01-11T07:12:46.495868+0000","last_fullsized":"2024-01-11T07:12:46.495868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:29:56.530253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.487692+0000","last_change":"2024-01-11T07:12:46.487616+0000","last_active":"2024-01-11T07:12:46.487692+0000","last_peered":"2024-01-11T07:12:46.487692+0000","last_clean":"2024-01-11T07:12:46.487692+0000","last_became_active":"2024-01-11T07:12:46.487368+0000","last_became_peered":"2024-01-11T07:12:46.487368+0000","last_unstale":"2024-01-11T07:12:46.487692+0000","last_undegraded":"2024-01-11T07:12:46.487692+0000","last_fullsized":"2024-01-11T07:12:46.487692+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:58:26.573595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.504164+0000","last_change":"2024-01-11T07:12:45.483782+0000","last_active":"2024-01-11T07:12:46.504164+0000","last_peered":"2024-01-11T07:12:46.504164+0000","last_clean":"2024-01-11T07:12:46.504164+0000","last_became_active":"2024-01-11T07:12:45.483521+0000","last_became_peered":"2024-01-11T07:12:45.483521+0000","last_unstale":"2024-01-11T07:12:46.504164+0000","last_undegraded":"2024-01-11T07:12:46.504164+0000","last_fullsized":"2024-01-11T07:12:46.504164+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:26:41.167127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.499123+0000","last_change":"2024-01-11T07:12:46.497719+0000","last_active":"2024-01-11T07:12:46.499123+0000","last_peered":"2024-01-11T07:12:46.499123+0000","last_clean":"2024-01-11T07:12:46.499123+0000","last_became_active":"2024-01-11T07:12:46.494288+0000","last_became_peered":"2024-01-11T07:12:46.494288+0000","last_unstale":"2024-01-11T07:12:46.499123+0000","last_undegraded":"2024-01-11T07:12:46.499123+0000","last_fullsized":"2024-01-11T07:12:46.499123+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:49:24.784110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.498297+0000","last_change":"2024-01-11T07:12:45.487212+0000","last_active":"2024-01-11T07:12:46.498297+0000","last_peered":"2024-01-11T07:12:46.498297+0000","last_clean":"2024-01-11T07:12:46.498297+0000","last_became_active":"2024-01-11T07:12:45.486879+0000","last_became_peered":"2024-01-11T07:12:45.486879+0000","last_unstale":"2024-01-11T07:12:46.498297+0000","last_undegraded":"2024-01-11T07:12:46.498297+0000","last_fullsized":"2024-01-11T07:12:46.498297+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:30:54.683822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489740+0000","last_change":"2024-01-11T07:12:46.489472+0000","last_active":"2024-01-11T07:12:46.489740+0000","last_peered":"2024-01-11T07:12:46.489740+0000","last_clean":"2024-01-11T07:12:46.489740+0000","last_became_active":"2024-01-11T07:12:46.488711+0000","last_became_peered":"2024-01-11T07:12:46.488711+0000","last_unstale":"2024-01-11T07:12:46.489740+0000","last_undegraded":"2024-01-11T07:12:46.489740+0000","last_fullsized":"2024-01-11T07:12:46.489740+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:48:38.059444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.483616+0000","last_change":"2024-01-11T07:12:45.488890+0000","last_active":"2024-01-11T07:12:46.483616+0000","last_peered":"2024-01-11T07:12:46.483616+0000","last_clean":"2024-01-11T07:12:46.483616+0000","last_became_active":"2024-01-11T07:12:45.488621+0000","last_became_peered":"2024-01-11T07:12:45.488621+0000","last_unstale":"2024-01-11T07:12:46.483616+0000","last_undegraded":"2024-01-11T07:12:46.483616+0000","last_fullsized":"2024-01-11T07:12:46.483616+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:48:31.873076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489502+0000","last_change":"2024-01-11T07:12:46.489379+0000","last_active":"2024-01-11T07:12:46.489502+0000","last_peered":"2024-01-11T07:12:46.489502+0000","last_clean":"2024-01-11T07:12:46.489502+0000","last_became_active":"2024-01-11T07:12:46.489150+0000","last_became_peered":"2024-01-11T07:12:46.489150+0000","last_unstale":"2024-01-11T07:12:46.489502+0000","last_undegraded":"2024-01-11T07:12:46.489502+0000","last_fullsized":"2024-01-11T07:12:46.489502+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:44:04.558093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.481021+0000","last_change":"2024-01-11T07:12:45.482088+0000","last_active":"2024-01-11T07:12:46.481021+0000","last_peered":"2024-01-11T07:12:46.481021+0000","last_clean":"2024-01-11T07:12:46.481021+0000","last_became_active":"2024-01-11T07:12:45.481778+0000","last_became_peered":"2024-01-11T07:12:45.481778+0000","last_unstale":"2024-01-11T07:12:46.481021+0000","last_undegraded":"2024-01-11T07:12:46.481021+0000","last_fullsized":"2024-01-11T07:12:46.481021+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:29:56.530253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489884+0000","last_change":"2024-01-11T07:12:46.489766+0000","last_active":"2024-01-11T07:12:46.489884+0000","last_peered":"2024-01-11T07:12:46.489884+0000","last_clean":"2024-01-11T07:12:46.489884+0000","last_became_active":"2024-01-11T07:12:46.489277+0000","last_became_peered":"2024-01-11T07:12:46.489277+0000","last_unstale":"2024-01-11T07:12:46.489884+0000","last_undegraded":"2024-01-11T07:12:46.489884+0000","last_fullsized":"2024-01-11T07:12:46.489884+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:16:26.786403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500646+0000","last_change":"2024-01-11T07:12:45.487798+0000","last_active":"2024-01-11T07:12:46.500646+0000","last_peered":"2024-01-11T07:12:46.500646+0000","last_clean":"2024-01-11T07:12:46.500646+0000","last_became_active":"2024-01-11T07:12:45.487545+0000","last_became_peered":"2024-01-11T07:12:45.487545+0000","last_unstale":"2024-01-11T07:12:46.500646+0000","last_undegraded":"2024-01-11T07:12:46.500646+0000","last_fullsized":"2024-01-11T07:12:46.500646+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:45:30.253518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485413+0000","last_change":"2024-01-11T07:12:46.485226+0000","last_active":"2024-01-11T07:12:46.485413+0000","last_peered":"2024-01-11T07:12:46.485413+0000","last_clean":"2024-01-11T07:12:46.485413+0000","last_became_active":"2024-01-11T07:12:46.484759+0000","last_became_peered":"2024-01-11T07:12:46.484759+0000","last_unstale":"2024-01-11T07:12:46.485413+0000","last_undegraded":"2024-01-11T07:12:46.485413+0000","last_fullsized":"2024-01-11T07:12:46.485413+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:20:57.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497980+0000","last_change":"2024-01-11T07:12:45.486498+0000","last_active":"2024-01-11T07:12:46.497980+0000","last_peered":"2024-01-11T07:12:46.497980+0000","last_clean":"2024-01-11T07:12:46.497980+0000","last_became_active":"2024-01-11T07:12:45.486259+0000","last_became_peered":"2024-01-11T07:12:45.486259+0000","last_unstale":"2024-01-11T07:12:46.497980+0000","last_undegraded":"2024-01-11T07:12:46.497980+0000","last_fullsized":"2024-01-11T07:12:46.497980+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:09:22.967591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484936+0000","last_change":"2024-01-11T07:12:46.484846+0000","last_active":"2024-01-11T07:12:46.484936+0000","last_peered":"2024-01-11T07:12:46.484936+0000","last_clean":"2024-01-11T07:12:46.484936+0000","last_became_active":"2024-01-11T07:12:46.484626+0000","last_became_peered":"2024-01-11T07:12:46.484626+0000","last_unstale":"2024-01-11T07:12:46.484936+0000","last_undegraded":"2024-01-11T07:12:46.484936+0000","last_fullsized":"2024-01-11T07:12:46.484936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:59:55.250999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497118+0000","last_change":"2024-01-11T07:12:45.485211+0000","last_active":"2024-01-11T07:12:46.497118+0000","last_peered":"2024-01-11T07:12:46.497118+0000","last_clean":"2024-01-11T07:12:46.497118+0000","last_became_active":"2024-01-11T07:12:45.484963+0000","last_became_peered":"2024-01-11T07:12:45.484963+0000","last_unstale":"2024-01-11T07:12:46.497118+0000","last_undegraded":"2024-01-11T07:12:46.497118+0000","last_fullsized":"2024-01-11T07:12:46.497118+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:55:36.288232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485029+0000","last_change":"2024-01-11T07:12:46.484140+0000","last_active":"2024-01-11T07:12:46.485029+0000","last_peered":"2024-01-11T07:12:46.485029+0000","last_clean":"2024-01-11T07:12:46.485029+0000","last_became_active":"2024-01-11T07:12:46.483200+0000","last_became_peered":"2024-01-11T07:12:46.483200+0000","last_unstale":"2024-01-11T07:12:46.485029+0000","last_undegraded":"2024-01-11T07:12:46.485029+0000","last_fullsized":"2024-01-11T07:12:46.485029+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:12.892522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.477704+0000","last_change":"2024-01-11T07:12:45.486893+0000","last_active":"2024-01-11T07:12:46.477704+0000","last_peered":"2024-01-11T07:12:46.477704+0000","last_clean":"2024-01-11T07:12:46.477704+0000","last_became_active":"2024-01-11T07:12:45.486644+0000","last_became_peered":"2024-01-11T07:12:45.486644+0000","last_unstale":"2024-01-11T07:12:46.477704+0000","last_undegraded":"2024-01-11T07:12:46.477704+0000","last_fullsized":"2024-01-11T07:12:46.477704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:33:03.273472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.500559+0000","last_change":"2024-01-11T07:12:46.499504+0000","last_active":"2024-01-11T07:12:46.500559+0000","last_peered":"2024-01-11T07:12:46.500559+0000","last_clean":"2024-01-11T07:12:46.500559+0000","last_became_active":"2024-01-11T07:12:46.499265+0000","last_became_peered":"2024-01-11T07:12:46.499265+0000","last_unstale":"2024-01-11T07:12:46.500559+0000","last_undegraded":"2024-01-11T07:12:46.500559+0000","last_fullsized":"2024-01-11T07:12:46.500559+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:15:37.981680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.505106+0000","last_change":"2024-01-11T07:12:45.486525+0000","last_active":"2024-01-11T07:12:46.505106+0000","last_peered":"2024-01-11T07:12:46.505106+0000","last_clean":"2024-01-11T07:12:46.505106+0000","last_became_active":"2024-01-11T07:12:45.485349+0000","last_became_peered":"2024-01-11T07:12:45.485349+0000","last_unstale":"2024-01-11T07:12:46.505106+0000","last_undegraded":"2024-01-11T07:12:46.505106+0000","last_fullsized":"2024-01-11T07:12:46.505106+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:04:58.377351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484815+0000","last_change":"2024-01-11T07:12:46.484732+0000","last_active":"2024-01-11T07:12:46.484815+0000","last_peered":"2024-01-11T07:12:46.484815+0000","last_clean":"2024-01-11T07:12:46.484815+0000","last_became_active":"2024-01-11T07:12:46.484484+0000","last_became_peered":"2024-01-11T07:12:46.484484+0000","last_unstale":"2024-01-11T07:12:46.484815+0000","last_undegraded":"2024-01-11T07:12:46.484815+0000","last_fullsized":"2024-01-11T07:12:46.484815+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:15:07.322551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.482845+0000","last_change":"2024-01-11T07:12:45.487290+0000","last_active":"2024-01-11T07:12:46.482845+0000","last_peered":"2024-01-11T07:12:46.482845+0000","last_clean":"2024-01-11T07:12:46.482845+0000","last_became_active":"2024-01-11T07:12:45.487059+0000","last_became_peered":"2024-01-11T07:12:45.487059+0000","last_unstale":"2024-01-11T07:12:46.482845+0000","last_undegraded":"2024-01-11T07:12:46.482845+0000","last_fullsized":"2024-01-11T07:12:46.482845+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:55:05.550733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.502356+0000","last_change":"2024-01-11T07:12:46.502274+0000","last_active":"2024-01-11T07:12:46.502356+0000","last_peered":"2024-01-11T07:12:46.502356+0000","last_clean":"2024-01-11T07:12:46.502356+0000","last_became_active":"2024-01-11T07:12:46.502072+0000","last_became_peered":"2024-01-11T07:12:46.502072+0000","last_unstale":"2024-01-11T07:12:46.502356+0000","last_undegraded":"2024-01-11T07:12:46.502356+0000","last_fullsized":"2024-01-11T07:12:46.502356+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:26:47.737864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.480954+0000","last_change":"2024-01-11T07:12:45.488799+0000","last_active":"2024-01-11T07:12:46.480954+0000","last_peered":"2024-01-11T07:12:46.480954+0000","last_clean":"2024-01-11T07:12:46.480954+0000","last_became_active":"2024-01-11T07:12:45.488547+0000","last_became_peered":"2024-01-11T07:12:45.488547+0000","last_unstale":"2024-01-11T07:12:46.480954+0000","last_undegraded":"2024-01-11T07:12:46.480954+0000","last_fullsized":"2024-01-11T07:12:46.480954+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:23:30.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490782+0000","last_change":"2024-01-11T07:12:46.490715+0000","last_active":"2024-01-11T07:12:46.490782+0000","last_peered":"2024-01-11T07:12:46.490782+0000","last_clean":"2024-01-11T07:12:46.490782+0000","last_became_active":"2024-01-11T07:12:46.490282+0000","last_became_peered":"2024-01-11T07:12:46.490282+0000","last_unstale":"2024-01-11T07:12:46.490782+0000","last_undegraded":"2024-01-11T07:12:46.490782+0000","last_fullsized":"2024-01-11T07:12:46.490782+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:08:41.946213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.476777+0000","last_change":"2024-01-11T07:12:45.485891+0000","last_active":"2024-01-11T07:12:46.476777+0000","last_peered":"2024-01-11T07:12:46.476777+0000","last_clean":"2024-01-11T07:12:46.476777+0000","last_became_active":"2024-01-11T07:12:45.479745+0000","last_became_peered":"2024-01-11T07:12:45.479745+0000","last_unstale":"2024-01-11T07:12:46.476777+0000","last_undegraded":"2024-01-11T07:12:46.476777+0000","last_fullsized":"2024-01-11T07:12:46.476777+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:22:32.976243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496567+0000","last_change":"2024-01-11T07:12:46.493008+0000","last_active":"2024-01-11T07:12:46.496567+0000","last_peered":"2024-01-11T07:12:46.496567+0000","last_clean":"2024-01-11T07:12:46.496567+0000","last_became_active":"2024-01-11T07:12:46.487432+0000","last_became_peered":"2024-01-11T07:12:46.487432+0000","last_unstale":"2024-01-11T07:12:46.496567+0000","last_undegraded":"2024-01-11T07:12:46.496567+0000","last_fullsized":"2024-01-11T07:12:46.496567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:08:39.516302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.495036+0000","last_change":"2024-01-11T07:12:45.482766+0000","last_active":"2024-01-11T07:12:46.495036+0000","last_peered":"2024-01-11T07:12:46.495036+0000","last_clean":"2024-01-11T07:12:46.495036+0000","last_became_active":"2024-01-11T07:12:45.482510+0000","last_became_peered":"2024-01-11T07:12:45.482510+0000","last_unstale":"2024-01-11T07:12:46.495036+0000","last_undegraded":"2024-01-11T07:12:46.495036+0000","last_fullsized":"2024-01-11T07:12:46.495036+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:46:13.141613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489700+0000","last_change":"2024-01-11T07:12:46.487694+0000","last_active":"2024-01-11T07:12:46.489700+0000","last_peered":"2024-01-11T07:12:46.489700+0000","last_clean":"2024-01-11T07:12:46.489700+0000","last_became_active":"2024-01-11T07:12:46.487420+0000","last_became_peered":"2024-01-11T07:12:46.487420+0000","last_unstale":"2024-01-11T07:12:46.489700+0000","last_undegraded":"2024-01-11T07:12:46.489700+0000","last_fullsized":"2024-01-11T07:12:46.489700+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:09:28.624228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.478146+0000","last_change":"2024-01-11T07:12:45.486679+0000","last_active":"2024-01-11T07:12:46.478146+0000","last_peered":"2024-01-11T07:12:46.478146+0000","last_clean":"2024-01-11T07:12:46.478146+0000","last_became_active":"2024-01-11T07:12:45.485687+0000","last_became_peered":"2024-01-11T07:12:45.485687+0000","last_unstale":"2024-01-11T07:12:46.478146+0000","last_undegraded":"2024-01-11T07:12:46.478146+0000","last_fullsized":"2024-01-11T07:12:46.478146+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:44:03.553134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489763+0000","last_change":"2024-01-11T07:12:46.489578+0000","last_active":"2024-01-11T07:12:46.489763+0000","last_peered":"2024-01-11T07:12:46.489763+0000","last_clean":"2024-01-11T07:12:46.489763+0000","last_became_active":"2024-01-11T07:12:46.489118+0000","last_became_peered":"2024-01-11T07:12:46.489118+0000","last_unstale":"2024-01-11T07:12:46.489763+0000","last_undegraded":"2024-01-11T07:12:46.489763+0000","last_fullsized":"2024-01-11T07:12:46.489763+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:46:14.146572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.476559+0000","last_change":"2024-01-11T07:12:45.489422+0000","last_active":"2024-01-11T07:12:46.476559+0000","last_peered":"2024-01-11T07:12:46.476559+0000","last_clean":"2024-01-11T07:12:46.476559+0000","last_became_active":"2024-01-11T07:12:45.488739+0000","last_became_peered":"2024-01-11T07:12:45.488739+0000","last_unstale":"2024-01-11T07:12:46.476559+0000","last_undegraded":"2024-01-11T07:12:46.476559+0000","last_fullsized":"2024-01-11T07:12:46.476559+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:57:28.695701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.485425+0000","last_change":"2024-01-11T07:12:46.485340+0000","last_active":"2024-01-11T07:12:46.485425+0000","last_peered":"2024-01-11T07:12:46.485425+0000","last_clean":"2024-01-11T07:12:46.485425+0000","last_became_active":"2024-01-11T07:12:46.485018+0000","last_became_peered":"2024-01-11T07:12:46.485018+0000","last_unstale":"2024-01-11T07:12:46.485425+0000","last_undegraded":"2024-01-11T07:12:46.485425+0000","last_fullsized":"2024-01-11T07:12:46.485425+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:04.611411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.479543+0000","last_change":"2024-01-11T07:12:45.485762+0000","last_active":"2024-01-11T07:12:46.479543+0000","last_peered":"2024-01-11T07:12:46.479543+0000","last_clean":"2024-01-11T07:12:46.479543+0000","last_became_active":"2024-01-11T07:12:45.485503+0000","last_became_peered":"2024-01-11T07:12:45.485503+0000","last_unstale":"2024-01-11T07:12:46.479543+0000","last_undegraded":"2024-01-11T07:12:46.479543+0000","last_fullsized":"2024-01-11T07:12:46.479543+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:39:36.883776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.489851+0000","last_change":"2024-01-11T07:12:46.489568+0000","last_active":"2024-01-11T07:12:46.489851+0000","last_peered":"2024-01-11T07:12:46.489851+0000","last_clean":"2024-01-11T07:12:46.489851+0000","last_became_active":"2024-01-11T07:12:46.488911+0000","last_became_peered":"2024-01-11T07:12:46.488911+0000","last_unstale":"2024-01-11T07:12:46.489851+0000","last_undegraded":"2024-01-11T07:12:46.489851+0000","last_fullsized":"2024-01-11T07:12:46.489851+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:05:50.062232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.476921+0000","last_change":"2024-01-11T07:12:45.489119+0000","last_active":"2024-01-11T07:12:46.476921+0000","last_peered":"2024-01-11T07:12:46.476921+0000","last_clean":"2024-01-11T07:12:46.476921+0000","last_became_active":"2024-01-11T07:12:45.487696+0000","last_became_peered":"2024-01-11T07:12:45.487696+0000","last_unstale":"2024-01-11T07:12:46.476921+0000","last_undegraded":"2024-01-11T07:12:46.476921+0000","last_fullsized":"2024-01-11T07:12:46.476921+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:11.887562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.483768+0000","last_change":"2024-01-11T07:12:46.483650+0000","last_active":"2024-01-11T07:12:46.483768+0000","last_peered":"2024-01-11T07:12:46.483768+0000","last_clean":"2024-01-11T07:12:46.483768+0000","last_became_active":"2024-01-11T07:12:46.483407+0000","last_became_peered":"2024-01-11T07:12:46.483407+0000","last_unstale":"2024-01-11T07:12:46.483768+0000","last_undegraded":"2024-01-11T07:12:46.483768+0000","last_fullsized":"2024-01-11T07:12:46.483768+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:57:29.700660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.476887+0000","last_change":"2024-01-11T07:12:45.488319+0000","last_active":"2024-01-11T07:12:46.476887+0000","last_peered":"2024-01-11T07:12:46.476887+0000","last_clean":"2024-01-11T07:12:46.476887+0000","last_became_active":"2024-01-11T07:12:45.488027+0000","last_became_peered":"2024-01-11T07:12:45.488027+0000","last_unstale":"2024-01-11T07:12:46.476887+0000","last_undegraded":"2024-01-11T07:12:46.476887+0000","last_fullsized":"2024-01-11T07:12:46.476887+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:01:31.568694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.490628+0000","last_change":"2024-01-11T07:12:46.490430+0000","last_active":"2024-01-11T07:12:46.490628+0000","last_peered":"2024-01-11T07:12:46.490628+0000","last_clean":"2024-01-11T07:12:46.490628+0000","last_became_active":"2024-01-11T07:12:46.489456+0000","last_became_peered":"2024-01-11T07:12:46.489456+0000","last_unstale":"2024-01-11T07:12:46.490628+0000","last_undegraded":"2024-01-11T07:12:46.490628+0000","last_fullsized":"2024-01-11T07:12:46.490628+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:57.235755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.484355+0000","last_change":"2024-01-11T07:12:45.484489+0000","last_active":"2024-01-11T07:12:46.484355+0000","last_peered":"2024-01-11T07:12:46.484355+0000","last_clean":"2024-01-11T07:12:46.484355+0000","last_became_active":"2024-01-11T07:12:45.484197+0000","last_became_peered":"2024-01-11T07:12:45.484197+0000","last_unstale":"2024-01-11T07:12:46.484355+0000","last_undegraded":"2024-01-11T07:12:46.484355+0000","last_fullsized":"2024-01-11T07:12:46.484355+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:03.606452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.496567+0000","last_change":"2024-01-11T07:12:46.496500+0000","last_active":"2024-01-11T07:12:46.496567+0000","last_peered":"2024-01-11T07:12:46.496567+0000","last_clean":"2024-01-11T07:12:46.496567+0000","last_became_active":"2024-01-11T07:12:46.495890+0000","last_became_peered":"2024-01-11T07:12:46.495890+0000","last_unstale":"2024-01-11T07:12:46.496567+0000","last_undegraded":"2024-01-11T07:12:46.496567+0000","last_fullsized":"2024-01-11T07:12:46.496567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:02:14.806403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.487498+0000","last_change":"2024-01-11T07:12:45.487476+0000","last_active":"2024-01-11T07:12:46.487498+0000","last_peered":"2024-01-11T07:12:46.487498+0000","last_clean":"2024-01-11T07:12:46.487498+0000","last_became_active":"2024-01-11T07:12:45.486460+0000","last_became_peered":"2024-01-11T07:12:45.486460+0000","last_unstale":"2024-01-11T07:12:46.487498+0000","last_undegraded":"2024-01-11T07:12:46.487498+0000","last_fullsized":"2024-01-11T07:12:46.487498+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:23:49.736311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497770+0000","last_change":"2024-01-11T07:12:46.497700+0000","last_active":"2024-01-11T07:12:46.497770+0000","last_peered":"2024-01-11T07:12:46.497770+0000","last_clean":"2024-01-11T07:12:46.497770+0000","last_became_active":"2024-01-11T07:12:46.497467+0000","last_became_peered":"2024-01-11T07:12:46.497467+0000","last_unstale":"2024-01-11T07:12:46.497770+0000","last_undegraded":"2024-01-11T07:12:46.497770+0000","last_fullsized":"2024-01-11T07:12:46.497770+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:45.467032+0000","last_clean_scrub_stamp":"2024-01-11T07:12:45.467032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:42:54.731518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.497028+0000","last_change":"2024-01-11T07:12:45.487098+0000","last_active":"2024-01-11T07:12:46.497028+0000","last_peered":"2024-01-11T07:12:46.497028+0000","last_clean":"2024-01-11T07:12:46.497028+0000","last_became_active":"2024-01-11T07:12:45.486865+0000","last_became_peered":"2024-01-11T07:12:45.486865+0000","last_unstale":"2024-01-11T07:12:46.497028+0000","last_undegraded":"2024-01-11T07:12:46.497028+0000","last_fullsized":"2024-01-11T07:12:46.497028+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:12:44.462073+0000","last_clean_scrub_stamp":"2024-01-11T07:12:44.462073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:35:11.863797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":251,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-01-11T07:12:46.477315+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:12:46.477315+0000","last_peered":"2024-01-11T07:12:46.477315+0000","last_clean":"2024-01-11T07:12:46.477315+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:12:46.477315+0000","last_undegraded":"2024-01-11T07:12:46.477315+0000","last_fullsized":"2024-01-11T07:12:46.477315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194094,"statfs":{"total":200000000,"available":198752967,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64400000000000002,"5min":0.754,"15min":0.754},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.75900000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.496}]},{"osd":1,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.71899999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.47599999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55300000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70099999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58299999999999996,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.75900000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.53300000000000003},"min":{"1min":0.46500000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76100000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.747,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48299999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77900000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.378},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.498,"15min":0.47999999999999998},"min":{"1min":0.32800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79900000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85499999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:11:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.58099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82899999999999996,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342823,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.42399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68000000000000005,"5min":1.157,"15min":1.157},"last":0.66200000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.39300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70499999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70799999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.38700000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69399999999999995,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73899999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.51800000000000002,"15min":0.54600000000000004},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66600000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53400000000000003,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72399999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.439}]},{"osd":5,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.68999999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.379},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59699999999999998,"15min":0.622},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:11:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51700000000000002,"15min":0.53300000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67800000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55400000000000005,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71299999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":635,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194677,"statfs":{"total":200000000,"available":199349735,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.60699999999999998,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56699999999999995,"5min":20.41,"15min":20.41},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.57199999999999995,"15min":0.46600000000000003},"min":{"1min":0.313,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69199999999999995,"5min":15.611000000000001,"15min":15.611000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.69099999999999995,"15min":0.54900000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65300000000000002,"5min":20.199999999999999,"15min":20.199999999999999},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.66800000000000004,"15min":0.53900000000000003},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":20.369,"15min":20.369},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42699999999999999,"15min":0.437},"min":{"1min":0.318,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61399999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.70499999999999996,"15min":0.64300000000000002},"min":{"1min":0.41399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.83099999999999996,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.748,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85799999999999998,"5min":12.449999999999999,"15min":12.449999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":0.72899999999999998,"5min":1.413,"15min":1.413},"last":0.748},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72699999999999998,"15min":0.67400000000000004},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95299999999999996,"5min":9.891,"15min":9.891},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.746,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94599999999999995,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:11:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.88700000000000001,"15min":0.77100000000000002},"min":{"1min":0.48199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.91000000000000003,"5min":20.324000000000002,"15min":20.324000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69499999999999995,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92600000000000005,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194094,"statfs":{"total":200000000,"available":198752525,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42199999999999999,"15min":0.46999999999999997},"min":{"1min":0.27400000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.58899999999999997,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.627,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.379}]},{"osd":2,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.504,"15min":0.71199999999999997},"min":{"1min":0.32700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64200000000000002,"5min":7.7960000000000003,"15min":7.7960000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.47199999999999998,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.72099999999999997,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.504,"15min":0.72499999999999998},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56000000000000005,"5min":8.0730000000000004,"15min":8.0730000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51100000000000001,"15min":0.57399999999999995},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.65900000000000003,"5min":2.855,"15min":2.855},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61799999999999999,"15min":0.60899999999999999},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71199999999999997,"15min":0.89300000000000002},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94999999999999996,"5min":8.0399999999999991,"15min":8.0399999999999991},"last":0.48399999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67000000000000004,"15min":0.71199999999999997},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92200000000000004,"5min":2.819,"15min":2.819},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.498,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.748,"15min":0.94999999999999996},"min":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.046,"5min":7.9539999999999997,"15min":7.9539999999999997},"last":0.503},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77700000000000002,"15min":0.77500000000000002},"min":{"1min":0.48099999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.014,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:12:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.54200000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93500000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76300000000000001,"15min":0.94399999999999995},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":8.0030000000000001,"15min":8.0030000000000001},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194671,"statfs":{"total":200000000,"available":199343413,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.41399999999999998,"15min":0.42199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66800000000000004,"5min":1.472,"15min":1.472},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39600000000000002},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.439,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58199999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.59499999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.498}]},{"osd":3,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.74399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.36099999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73499999999999999,"5min":1.139,"15min":1.139},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.66800000000000004},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94399999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63900000000000001,"15min":0.622},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":1.099},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68999999999999995,"15min":0.70799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0229999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.82599999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66000000000000003},"min":{"1min":0.52900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.65600000000000003,"15min":0.66700000000000004},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98199999999999998,"5min":1.052,"15min":1.052},"last":1.006}]},{"osd":7,"last update":"Thu Jan 11 07:11:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67000000000000004,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":1.0409999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194669,"statfs":{"total":200000000,"available":199341889,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.50600000000000001,"15min":0.78700000000000003},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53600000000000003,"5min":11.481,"15min":11.481},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.48299999999999998},"min":{"1min":0.34799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.55900000000000005,"15min":0.97699999999999998},"min":{"1min":0.252,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60799999999999998,"5min":17.516999999999999,"15min":17.516999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.437,"15min":0.41899999999999998},"min":{"1min":0.30199999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42899999999999999,"15min":0.42799999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.56799999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.63,"15min":1.034},"min":{"1min":0.38300000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69499999999999995,"5min":17.553000000000001,"15min":17.553000000000001},"last":0.376}]},{"osd":4,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.40899999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91300000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.496},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.84899999999999998,"15min":1.2669999999999999},"min":{"1min":0.38500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84299999999999997,"5min":17.474,"15min":17.474},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.78900000000000003,"15min":1.2010000000000001},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85599999999999998,"5min":17.393000000000001,"15min":17.393000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.754,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.877,"5min":11.394,"15min":11.394},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.76500000000000001,"15min":1.0309999999999999},"min":{"1min":0.498,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":11.436999999999999,"15min":11.436999999999999},"last":0.63},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.67200000000000004},"min":{"1min":0.55200000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.84399999999999997,"15min":1.238},"min":{"1min":0.47699999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1839999999999999,"5min":17.434000000000001,"15min":17.434000000000001},"last":0.52200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194669,"statfs":{"total":200000000,"available":199341787,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.77700000000000002,"15min":1.056},"min":{"1min":0.39900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.81399999999999995,"5min":8.484,"15min":8.484},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.79100000000000004,"15min":1.1120000000000001},"min":{"1min":0.45800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82799999999999996,"5min":9.2599999999999998,"15min":9.2599999999999998},"last":0.68899999999999995}]},{"osd":1,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68200000000000005,"15min":0.86899999999999999},"min":{"1min":0.42299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.95299999999999996,"5min":8.5340000000000007,"15min":8.5340000000000007},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.73399999999999999,"15min":1.01},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":9.2279999999999998,"15min":9.2279999999999998},"last":0.504}]},{"osd":2,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.76500000000000001,"15min":0.97899999999999998},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79300000000000004,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.66500000000000004,"15min":0.879},"min":{"1min":0.439,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85699999999999998,"5min":6.8639999999999999,"15min":6.8639999999999999},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.71599999999999997,"15min":1.05},"min":{"1min":0.375,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90800000000000003,"5min":8.9779999999999998,"15min":8.9779999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.79500000000000004,"15min":1.1220000000000001},"min":{"1min":0.48099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89100000000000001,"5min":8.7309999999999999,"15min":8.7309999999999999},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.59299999999999997,"15min":0.94199999999999995},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":11.388999999999999,"15min":11.388999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.60099999999999998,"15min":0.85999999999999999},"min":{"1min":0.35799999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64200000000000002,"5min":8.6509999999999998,"15min":8.6509999999999998},"last":0.38300000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.63300000000000001,"15min":0.91100000000000003},"min":{"1min":0.371,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70599999999999996,"5min":9.1449999999999996,"15min":9.1449999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.39900000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.69599999999999995,"5min":1.119,"15min":1.119},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:12:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63500000000000001,"15min":0.89100000000000001},"min":{"1min":0.39900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":8.6050000000000004,"15min":8.6050000000000004},"last":0.629},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.72099999999999997,"15min":0.98899999999999999},"min":{"1min":0.44800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.751,"5min":9.4640000000000004,"15min":9.4640000000000004},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194671,"statfs":{"total":200000000,"available":199343377,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.45800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72799999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.48999999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76700000000000002,"5min":0.87,"15min":0.87},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.44400000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.40899999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76600000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59899999999999998,"15min":0.58299999999999996},"min":{"1min":0.46600000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78100000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.49399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78900000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.625,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.45000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81699999999999995,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.501,"15min":0.50800000000000001},"min":{"1min":0.33200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68899999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69399999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.754,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72799999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.61699999999999999},"min":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.626}]}]}],"pool_statfs":[]}} 2024-01-11T07:12:51.088 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:12:51.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:12:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.265+0000 7fb29d4d2700 1 Processor -- start 2024-01-11T07:12:51.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.265+0000 7fb29d4d2700 1 -- start start 2024-01-11T07:12:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.266+0000 7fb29d4d2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29806dd00 0x7fb2980c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.266+0000 7fb29d4d2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb29815a1c0 0x7fb298069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.266+0000 7fb29d4d2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29815a640 con 0x7fb29806e0d0 2024-01-11T07:12:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.266+0000 7fb29d4d2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29815b770 con 0x7fb29815a1c0 2024-01-11T07:12:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.266+0000 7fb29d4d2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29815abf0 con 0x7fb29806dd00 2024-01-11T07:12:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb2967fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb29806e0d0 0x7fb298159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44994/0 (socket says 172.21.15.7:44994) 2024-01-11T07:12:51.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb2967fc700 1 -- 172.21.15.7:0/3838341345 learned_addr learned my addr 172.21.15.7:0/3838341345 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb2977fe700 1 --2- 172.21.15.7:0/3838341345 >> v2:172.21.15.113:3300/0 conn(0x7fb29815a1c0 0x7fb298069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb296ffd700 1 --2- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29806dd00 0x7fb2980c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb295ffb700 1 -- 172.21.15.7:0/3838341345 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568148043 0 0) 0x7fb28c001110 con 0x7fb29806e0d0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb295ffb700 1 -- 172.21.15.7:0/3838341345 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb280003570 con 0x7fb29806e0d0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb2977fe700 1 -- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29806dd00 msgr2=0x7fb2980c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb2977fe700 1 --2- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29806dd00 0x7fb2980c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb2977fe700 1 -- 172.21.15.7:0/3838341345 >> v1:172.21.15.7:6789/0 conn(0x7fb29806e0d0 legacy=0x7fb298159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb295ffb700 1 -- 172.21.15.7:0/3838341345 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 381764823 0 0) 0x7fb280003570 con 0x7fb29806e0d0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb2977fe700 1 -- 172.21.15.7:0/3838341345 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29815aed0 con 0x7fb29815a1c0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb296ffd700 1 --2- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29806dd00 0x7fb2980c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.267+0000 7fb2977fe700 1 --2- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29815a1c0 0x7fb298069fb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb290009e60 tx=0x7fb290009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26d0fb062909ebb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb295ffb700 1 -- 172.21.15.7:0/3838341345 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29000e070 con 0x7fb29815a1c0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb295ffb700 1 -- 172.21.15.7:0/3838341345 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29000b7a0 con 0x7fb29815a1c0 2024-01-11T07:12:51.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb295ffb700 1 -- 172.21.15.7:0/3838341345 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c001600 con 0x7fb29815a1c0 2024-01-11T07:12:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29815a1c0 msgr2=0x7fb298069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29815a1c0 0x7fb298069fb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb290009e60 tx=0x7fb290009780 comp rx=0 tx=0).stop 2024-01-11T07:12:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3838341345 shutdown_connections 2024-01-11T07:12:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29815a1c0 0x7fb298069fb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3838341345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29806dd00 0x7fb2980c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3838341345 >> 172.21.15.7:0/3838341345 conn(0x7fb2980680e0 msgr2=0x7fb2980cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3838341345 shutdown_connections 2024-01-11T07:12:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.268+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3838341345 wait complete. 2024-01-11T07:12:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.270+0000 7fb29d4d2700 1 Processor -- start 2024-01-11T07:12:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.270+0000 7fb29d4d2700 1 -- start start 2024-01-11T07:12:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.270+0000 7fb29d4d2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29806dd00 0x7fb2980c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.270+0000 7fb29d4d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29806e0d0 0x7fb29815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.270+0000 7fb29d4d2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29815a1c0 0x7fb298158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb29d4d2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980aaf20 con 0x7fb29806e0d0 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb29d4d2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29815c650 con 0x7fb29806dd00 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb29d4d2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29815c930 con 0x7fb29815a1c0 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29806e0d0 0x7fb29815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29806e0d0 0x7fb29815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32800/0 (socket says 172.21.15.7:32800) 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 -- 172.21.15.7:0/3422137688 learned_addr learned my addr 172.21.15.7:0/3422137688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb296ffd700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29806dd00 0x7fb2980c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2977fe700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29815a1c0 0x7fb298158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 -- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29815a1c0 msgr2=0x7fb298158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29815a1c0 0x7fb298158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 -- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29806dd00 msgr2=0x7fb2980c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29806dd00 0x7fb2980c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 -- 172.21.15.7:0/3422137688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288009420 con 0x7fb29806e0d0 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2977fe700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29815a1c0 0x7fb298158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb2967fc700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29806e0d0 0x7fb29815ced0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00cb90 tx=0x7fb28c001450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb27effd700 1 -- 172.21.15.7:0/3422137688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c007e90 con 0x7fb29806e0d0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb27effd700 1 -- 172.21.15.7:0/3422137688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c007360 con 0x7fb29806e0d0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb290009a20 con 0x7fb29806e0d0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb27effd700 1 -- 172.21.15.7:0/3422137688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c004830 con 0x7fb29806e0d0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.271+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2980aabb0 con 0x7fb29806e0d0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.273+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29804c3c0 con 0x7fb29806e0d0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.273+0000 7fb27effd700 1 -- 172.21.15.7:0/3422137688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb28c004c20 con 0x7fb29806e0d0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.274+0000 7fb27effd700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb284066f30 0x7fb2840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.274+0000 7fb27effd700 1 -- 172.21.15.7:0/3422137688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb28c083ca0 con 0x7fb29806e0d0 2024-01-11T07:12:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.274+0000 7fb296ffd700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb284066f30 0x7fb2840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.274+0000 7fb296ffd700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb284066f30 0x7fb2840693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2880093f0 tx=0x7fb288008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.278+0000 7fb27effd700 1 -- 172.21.15.7:0/3422137688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb28c050f90 con 0x7fb29806e0d0 2024-01-11T07:12:51.464 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-11T07:12:51.463+0000 7f2ea6a80700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:12:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.570+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb29815a640 con 0x7fb29806e0d0 2024-01-11T07:12:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.577+0000 7fb27effd700 1 -- 172.21.15.7:0/3422137688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb29815a640 con 0x7fb29806e0d0 2024-01-11T07:12:51.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb284066f30 msgr2=0x7fb2840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb284066f30 0x7fb2840693e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2880093f0 tx=0x7fb288008040 comp rx=0 tx=0).stop 2024-01-11T07:12:51.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29806e0d0 msgr2=0x7fb29815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29806e0d0 0x7fb29815ced0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00cb90 tx=0x7fb28c001450 comp rx=0 tx=0).stop 2024-01-11T07:12:51.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 shutdown_connections 2024-01-11T07:12:51.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb284066f30 0x7fb2840693e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb29815a1c0 0x7fb298158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29806dd00 0x7fb2980c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 --2- 172.21.15.7:0/3422137688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb29806e0d0 0x7fb29815ced0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 >> 172.21.15.7:0/3422137688 conn(0x7fb2980680e0 msgr2=0x7fb298063230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:51.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 shutdown_connections 2024-01-11T07:12:51.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.579+0000 7fb29d4d2700 1 -- 172.21.15.7:0/3422137688 wait complete. 2024-01-11T07:12:51.581 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:12:51.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.725+0000 7fba59ba4700 1 Processor -- start 2024-01-11T07:12:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.725+0000 7fba59ba4700 1 -- start start 2024-01-11T07:12:51.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba59ba4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba54063230 0x7fba5415e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba59ba4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba5415aa40 0x7fba5406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba59ba4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5415df80 con 0x7fba54063600 2024-01-11T07:12:51.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba59ba4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5415e950 con 0x7fba54063230 2024-01-11T07:12:51.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba59ba4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540aab70 con 0x7fba5415aa40 2024-01-11T07:12:51.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba52ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fba54063600 0x7fba5415a330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44998/0 (socket says 172.21.15.7:44998) 2024-01-11T07:12:51.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba52ffd700 1 -- 172.21.15.7:0/1587765293 learned_addr learned my addr 172.21.15.7:0/1587765293 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:51.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba537fe700 1 --2- 172.21.15.7:0/1587765293 >> v2:172.21.15.113:3300/0 conn(0x7fba54063230 0x7fba5415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.726+0000 7fba53fff700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba5415aa40 0x7fba5406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579889433 0 0) 0x7fba5415df80 con 0x7fba54063600 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3c003570 con 0x7fba54063600 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1436482239 0 0) 0x7fba3c003570 con 0x7fba54063600 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba5415aa40 msgr2=0x7fba5406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba5415aa40 0x7fba5406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 >> v2:172.21.15.113:3300/0 conn(0x7fba54063230 msgr2=0x7fba5415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 --2- 172.21.15.7:0/1587765293 >> v2:172.21.15.113:3300/0 conn(0x7fba54063230 0x7fba5415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5415af10 con 0x7fba54063600 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fba48001470 con 0x7fba54063600 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 >> v1:172.21.15.7:6789/0 conn(0x7fba54063600 legacy=0x7fba5415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba3c003a60 0x7fba3c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c004810 0x7fba3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba5415aa40 0x7fba3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5415e950 con 0x7fba3c003a60 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540aab70 con 0x7fba3c004810 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c00ba60 con 0x7fba5415aa40 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba537fe700 1 --2- 172.21.15.7:0/1587765293 >> v2:172.21.15.113:3300/0 conn(0x7fba54063230 0x7fba5415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba52ffd700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c004810 0x7fba3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba537fe700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba3c003a60 0x7fba3c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba53fff700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba5415aa40 0x7fba3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba537fe700 1 -- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba5415aa40 msgr2=0x7fba3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.727+0000 7fba537fe700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba5415aa40 0x7fba3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba537fe700 1 -- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c004810 msgr2=0x7fba3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba537fe700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba3c004810 0x7fba3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba537fe700 1 -- 172.21.15.7:0/1587765293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c003570 con 0x7fba3c003a60 2024-01-11T07:12:51.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba537fe700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba3c003a60 0x7fba3c003e70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fba44009de0 tx=0x7fba440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586af416c9d3b4f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400e920 con 0x7fba3c003a60 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4400ea80 con 0x7fba3c003a60 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba527fc700 1 -- 172.21.15.7:0/1587765293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400ee20 con 0x7fba3c003a60 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba59ba4700 1 -- 172.21.15.7:0/1587765293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba3c003a60 msgr2=0x7fba3c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba59ba4700 1 --2- 172.21.15.7:0/1587765293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba3c003a60 0x7fba3c003e70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fba44009de0 tx=0x7fba440091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba53fff700 1 -- 172.21.15.7:0/1587765293 reap_dead start 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba59ba4700 1 -- 172.21.15.7:0/1587765293 shutdown_connections 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba59ba4700 1 -- 172.21.15.7:0/1587765293 >> 172.21.15.7:0/1587765293 conn(0x7fba540cd910 msgr2=0x7fba540cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.728+0000 7fba59ba4700 1 -- 172.21.15.7:0/1587765293 shutdown_connections 2024-01-11T07:12:51.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.729+0000 7fba59ba4700 1 -- 172.21.15.7:0/1587765293 wait complete. 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.730+0000 7fba59ba4700 1 Processor -- start 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.730+0000 7fba59ba4700 1 -- start start 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.731+0000 7fba59ba4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c003a60 0x7fba5414c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.731+0000 7fba59ba4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba540cb0a0 0x7fba540cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.731+0000 7fba59ba4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba54154ea0 0x7fba54151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.731+0000 7fba59ba4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b1470 con 0x7fba540cb0a0 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.731+0000 7fba59ba4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b0860 con 0x7fba54154ea0 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.731+0000 7fba59ba4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b0b40 con 0x7fba3c003a60 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba540cb0a0 0x7fba540cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba540cb0a0 0x7fba540cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32822/0 (socket says 172.21.15.7:32822) 2024-01-11T07:12:51.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 -- 172.21.15.7:0/20035440 learned_addr learned my addr 172.21.15.7:0/20035440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba53fff700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba54154ea0 0x7fba54151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba537fe700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c003a60 0x7fba5414c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 -- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c003a60 msgr2=0x7fba5414c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c003a60 0x7fba5414c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 -- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba54154ea0 msgr2=0x7fba54151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba54154ea0 0x7fba54151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 -- 172.21.15.7:0/20035440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540b0e20 con 0x7fba540cb0a0 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba52ffd700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba540cb0a0 0x7fba540cbef0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fba480073c0 tx=0x7fba48001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba3b7fe700 1 -- 172.21.15.7:0/20035440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48018030 con 0x7fba540cb0a0 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba3b7fe700 1 -- 172.21.15.7:0/20035440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4800f850 con 0x7fba540cb0a0 2024-01-11T07:12:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba540b1100 con 0x7fba540cb0a0 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba3b7fe700 1 -- 172.21.15.7:0/20035440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4800fbf0 con 0x7fba540cb0a0 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.732+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba540cc430 con 0x7fba540cb0a0 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.733+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba540b1620 con 0x7fba540cb0a0 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.734+0000 7fba3b7fe700 1 -- 172.21.15.7:0/20035440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fba4801b070 con 0x7fba540cb0a0 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.735+0000 7fba3b7fe700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fba40066f30 0x7fba400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.735+0000 7fba3b7fe700 1 -- 172.21.15.7:0/20035440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba48019070 con 0x7fba540cb0a0 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.735+0000 7fba537fe700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fba40066f30 0x7fba400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.736+0000 7fba537fe700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fba40066f30 0x7fba400693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fba44006ed0 tx=0x7fba44019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:51.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.738+0000 7fba3b7fe700 1 -- 172.21.15.7:0/20035440 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba48051f30 con 0x7fba540cb0a0 2024-01-11T07:12:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.994+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba5404c3c0 con 0x7fba540cb0a0 2024-01-11T07:12:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.994+0000 7fba3b7fe700 1 -- 172.21.15.7:0/20035440 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fba48055550 con 0x7fba540cb0a0 2024-01-11T07:12:51.995 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:51.995 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:12:45.905827+0000","modified":"2024-01-11T07:12:49.492046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2493499246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2493499246},{"type":"v1","addr":"172.21.15.7:6835","nonce":2493499246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-11T07:12:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fba40066f30 msgr2=0x7fba400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fba40066f30 0x7fba400693e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fba44006ed0 tx=0x7fba44019000 comp rx=0 tx=0).stop 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba540cb0a0 msgr2=0x7fba540cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba540cb0a0 0x7fba540cbef0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fba480073c0 tx=0x7fba48001e90 comp rx=0 tx=0).stop 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 shutdown_connections 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fba40066f30 0x7fba400693e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba54154ea0 0x7fba54151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba3c003a60 0x7fba5414c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 --2- 172.21.15.7:0/20035440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba540cb0a0 0x7fba540cbef0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 >> 172.21.15.7:0/20035440 conn(0x7fba540cd910 msgr2=0x7fba5406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 shutdown_connections 2024-01-11T07:12:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:51.997+0000 7fba59ba4700 1 -- 172.21.15.7:0/20035440 wait complete. 2024-01-11T07:12:51.999 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-01-11T07:12:52.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:12:45.905827+0000', 'modified': '2024-01-11T07:12:49.492046+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6720}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6720': {'gid': 6720, 'name': 'c', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/2493499246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6832', 'nonce': 2493499246}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 2493499246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:12:52.415 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:12:52.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-11T07:12:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.559+0000 7f43854bb700 1 Processor -- start 2024-01-11T07:12:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.559+0000 7f43854bb700 1 -- start start 2024-01-11T07:12:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.559+0000 7f43854bb700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f438006e340 0x7f43800ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.559+0000 7f43854bb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f438015a070 0x7f43800ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.559+0000 7f43854bb700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43800d18c0 con 0x7f4380159ca0 2024-01-11T07:12:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.559+0000 7f43854bb700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43800d0010 con 0x7f438006e340 2024-01-11T07:12:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.559+0000 7f43854bb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43800cf400 con 0x7f438015a070 2024-01-11T07:12:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437e7fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f438006e340 0x7f43800ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f438015a070 0x7f43800ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437e7fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f438006e340 0x7f43800ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:42576/0 (socket says 172.21.15.7:42576) 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f438015a070 0x7f43800ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:35286/0 (socket says 172.21.15.7:35286) 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437e7fc700 1 -- 172.21.15.7:0/4044218396 learned_addr learned my addr 172.21.15.7:0/4044218396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437e7fc700 1 -- 172.21.15.7:0/4044218396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f438015a070 msgr2=0x7f43800ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437e7fc700 1 --2- 172.21.15.7:0/4044218396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f438015a070 0x7f43800ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437e7fc700 1 -- 172.21.15.7:0/4044218396 >> v1:172.21.15.113:6789/0 conn(0x7f4380159ca0 legacy=0x7f438006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437e7fc700 1 -- 172.21.15.7:0/4044218396 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43800cf6e0 con 0x7f438006e340 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.560+0000 7f437e7fc700 1 --2- 172.21.15.7:0/4044218396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438006e340 0x7f43800ce7f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43680099b0 tx=0x7f4368009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61009bcdd9e8dda5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f437dffb700 1 -- 172.21.15.7:0/4044218396 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368009ea0 con 0x7f438006e340 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f437dffb700 1 -- 172.21.15.7:0/4044218396 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436800b860 con 0x7f438006e340 2024-01-11T07:12:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f437dffb700 1 -- 172.21.15.7:0/4044218396 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436800bc00 con 0x7f438006e340 2024-01-11T07:12:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f43854bb700 1 -- 172.21.15.7:0/4044218396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438006e340 msgr2=0x7f43800ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f43854bb700 1 --2- 172.21.15.7:0/4044218396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438006e340 0x7f43800ce7f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43680099b0 tx=0x7f4368009250 comp rx=0 tx=0).stop 2024-01-11T07:12:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f43854bb700 1 -- 172.21.15.7:0/4044218396 shutdown_connections 2024-01-11T07:12:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f43854bb700 1 --2- 172.21.15.7:0/4044218396 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f438015a070 0x7f43800ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f43854bb700 1 --2- 172.21.15.7:0/4044218396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438006e340 0x7f43800ce7f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f43854bb700 1 -- 172.21.15.7:0/4044218396 >> 172.21.15.7:0/4044218396 conn(0x7f4380063cc0 msgr2=0x7f43800d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f43854bb700 1 -- 172.21.15.7:0/4044218396 shutdown_connections 2024-01-11T07:12:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.561+0000 7f43854bb700 1 -- 172.21.15.7:0/4044218396 wait complete. 2024-01-11T07:12:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.563+0000 7f43854bb700 1 Processor -- start 2024-01-11T07:12:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.563+0000 7f43854bb700 1 -- start start 2024-01-11T07:12:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f43854bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f438006e340 0x7f43800c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f43854bb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4380159ca0 0x7f43800b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f43854bb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438015a070 0x7f43800c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f43854bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43800ca420 con 0x7f438006e340 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f43854bb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43800ca700 con 0x7f438015a070 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f43854bb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43800ca9e0 con 0x7f4380159ca0 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f437effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f438006e340 0x7f43800c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f437effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f438006e340 0x7f43800c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32836/0 (socket says 172.21.15.7:32836) 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.564+0000 7f437effd700 1 -- 172.21.15.7:0/2344661091 learned_addr learned my addr 172.21.15.7:0/2344661091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437e7fc700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4380159ca0 0x7f43800b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437f7fe700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438015a070 0x7f43800c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437effd700 1 -- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4380159ca0 msgr2=0x7f43800b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437effd700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4380159ca0 0x7f43800b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437effd700 1 -- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438015a070 msgr2=0x7f43800c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437effd700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438015a070 0x7f43800c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437effd700 1 -- 172.21.15.7:0/2344661091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680094f0 con 0x7f438006e340 2024-01-11T07:12:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437e7fc700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4380159ca0 0x7f43800b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f437effd700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f438006e340 0x7f43800c8230 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f437000c590 tx=0x7f437000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f4376ffd700 1 -- 172.21.15.7:0/2344661091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011590 con 0x7f438006e340 2024-01-11T07:12:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f4376ffd700 1 -- 172.21.15.7:0/2344661091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43700116f0 con 0x7f438006e340 2024-01-11T07:12:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f43854bb700 1 -- 172.21.15.7:0/2344661091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43800cacc0 con 0x7f438006e340 2024-01-11T07:12:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f4376ffd700 1 -- 172.21.15.7:0/2344661091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011ab0 con 0x7f438006e340 2024-01-11T07:12:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.565+0000 7f43854bb700 1 -- 172.21.15.7:0/2344661091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4380143670 con 0x7f438006e340 2024-01-11T07:12:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.567+0000 7f4376ffd700 1 -- 172.21.15.7:0/2344661091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f437001b3b0 con 0x7f438006e340 2024-01-11T07:12:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.567+0000 7f4376ffd700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4360066f30 0x7f43600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.567+0000 7f4376ffd700 1 -- 172.21.15.7:0/2344661091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4370083ae0 con 0x7f438006e340 2024-01-11T07:12:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.567+0000 7f437e7fc700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4360066f30 0x7f43600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.568+0000 7f437e7fc700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4360066f30 0x7f43600693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f43680099b0 tx=0x7f4368017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.569+0000 7f438e7bbb80 1 Processor -- start 2024-01-11T07:12:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.569+0000 7f438e7bbb80 1 -- start start 2024-01-11T07:12:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065127c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e065128170 0x55e065148360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06505a3e0 con 0x55e06512bfa0 2024-01-11T07:12:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e065057be0 con 0x55e06512c8b0 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0650539d0 con 0x55e065128170 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f437effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f437effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32844/0 (socket says 172.21.15.7:32844) 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f437effd700 1 -- 172.21.15.7:0/1259519723 learned_addr learned my addr 172.21.15.7:0/1259519723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f437f7fe700 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e065128170 0x55e065148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.570+0000 7f437e7fc700 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065127c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f437effd700 1 -- 172.21.15.7:0/1259519723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e065128170 msgr2=0x55e065148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f437effd700 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e065128170 0x55e065148360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f437effd700 1 -- 172.21.15.7:0/1259519723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 msgr2=0x55e065127c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f437effd700 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065127c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f437effd700 1 -- 172.21.15.7:0/1259519723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e064ee7360 con 0x55e06512bfa0 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f437effd700 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512c370 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4370050f10 tx=0x7f4370081f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bcae3b83b47b2c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f435ffff700 1 -- 172.21.15.7:0/1259519723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370055170 con 0x55e06512bfa0 2024-01-11T07:12:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f435ffff700 1 -- 172.21.15.7:0/1259519723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370054a10 con 0x55e06512bfa0 2024-01-11T07:12:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.571+0000 7f435ffff700 1 -- 172.21.15.7:0/1259519723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370057c60 con 0x55e06512bfa0 2024-01-11T07:12:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1259519723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 msgr2=0x55e06512c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512c370 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4370050f10 tx=0x7f4370081f60 comp rx=0 tx=0).stop 2024-01-11T07:12:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1259519723 shutdown_connections 2024-01-11T07:12:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e065128170 0x55e065148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512c370 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/1259519723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065127c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1259519723 >> 172.21.15.7:0/1259519723 conn(0x55e065102440 msgr2=0x55e06514ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1259519723 shutdown_connections 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1259519723 wait complete. 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.572+0000 7f438e7bbb80 1 -- start start 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.573+0000 7f438e7bbb80 1 Processor -- start 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e065128170 0x55e06514e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512c8b0 0x55e065126b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0650001b0 con 0x55e06512bfa0 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e064fffe50 con 0x55e065128170 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e064f4ce40 con 0x55e06512c8b0 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f438e7bbb80 10 client.0 _ll_register_callbacks cb 0x55e064f4f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32858/0 (socket says 172.21.15.7:32858) 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 -- 172.21.15.7:0/3930129776 learned_addr learned my addr 172.21.15.7:0/3930129776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437f7fe700 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512c8b0 0x55e065126b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437effd700 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e065128170 0x55e06514e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 -- 172.21.15.7:0/3930129776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512c8b0 msgr2=0x55e065126b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512c8b0 0x55e065126b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 -- 172.21.15.7:0/3930129776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e065128170 msgr2=0x55e06514e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e065128170 0x55e06514e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 -- 172.21.15.7:0/3930129776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4380143670 con 0x55e06512bfa0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f437e7fc700 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512a660 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4368006fd0 tx=0x7f4368020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df35ee411e869de server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f435d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f435d7fa700 1 -- 172.21.15.7:0/3930129776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436802f070 con 0x55e06512bfa0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f438e7bbb80 1 -- 172.21.15.7:0/3930129776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0650d4eb0 con 0x55e06512bfa0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.574+0000 7f435d7fa700 1 -- 172.21.15.7:0/3930129776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4368026430 con 0x55e06512bfa0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f435d7fa700 1 -- 172.21.15.7:0/3930129776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4368020070 con 0x55e06512bfa0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f438e7bbb80 10 client.6849 fetch_fsmap learned FSMap version 57 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f438e7bbb80 1 -- 172.21.15.7:0/3930129776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55e064e93b10 con 0x55e06512bfa0 2024-01-11T07:12:52.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f435d7fa700 1 -- 172.21.15.7:0/3930129776 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0650d4eb0 con 0x55e06512bfa0 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f435d7fa700 1 -- 172.21.15.7:0/3930129776 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4368026800 con 0x55e06512bfa0 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f438e7bbb80 10 client.6849 fetch_fsmap finished waiting for FSMap version 57 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f438e7bbb80 10 client.6849 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f438e7bbb80 20 client.6849 populate_metadata read hostname 'smithi007' 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] conn(0x55e065143e50 0x55e065144220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f438e7bbb80 4 client.6849 mds_command: new command op to 6720 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f438e7bbb80 1 -- 172.21.15.7:0/3930129776 --> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e06502a880 con 0x55e065143e50 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.575+0000 7f437effd700 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] conn(0x55e065143e50 0x55e065144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.576+0000 7f437effd700 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] conn(0x55e065143e50 0x55e065144220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=326b60bd058b64a6 server_cookie=bb0562d2d1235705 in_seq=0 out_seq=0 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.576+0000 7f435d7fa700 10 client.6849 ms_handle_connect on v2:172.21.15.7:6832/2493499246 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.576+0000 7f435d7fa700 1 -- 172.21.15.7:0/3930129776 <== mds.0 v2:172.21.15.7:6832/2493499246 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e06502a880 con 0x55e065143e50 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.576+0000 7f435d7fa700 10 client.6849 handle_command_reply: tid=0 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.576+0000 7f438e7bbb80 1 client.6849 shutdown 2024-01-11T07:12:52.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 -- 172.21.15.7:0/3930129776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 msgr2=0x55e06512a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512a660 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4368006fd0 tx=0x7f4368020000 comp rx=0 tx=0).stop 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 -- 172.21.15.7:0/3930129776 shutdown_connections 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512c8b0 0x55e065126b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] conn(0x55e065143e50 0x55e065144220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e065128170 0x55e06514e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/3930129776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e06512bfa0 0x55e06512a660 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f435d7fa700 0 client.6849 ms_handle_reset on v2:172.21.15.7:6832/2493499246 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 -- 172.21.15.7:0/3930129776 >> 172.21.15.7:0/3930129776 conn(0x55e065102440 msgr2=0x55e065102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 -- 172.21.15.7:0/3930129776 shutdown_connections 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 1 -- 172.21.15.7:0/3930129776 wait complete. 2024-01-11T07:12:52.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.577+0000 7f438e7bbb80 20 client.6849 trim_cache size 0 max 16384 2024-01-11T07:12:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.595+0000 7f438e7bbb80 1 Processor -- start 2024-01-11T07:12:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.595+0000 7f438e7bbb80 1 -- start start 2024-01-11T07:12:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e06514e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065101980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065143380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e065106010 con 0x55e065128170 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e065127700 con 0x55e06512c8b0 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0650f9af0 con 0x55e06512bfa0 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e06514e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e06514e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32870/0 (socket says 172.21.15.7:32870) 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 -- 172.21.15.7:0/1375146479 learned_addr learned my addr 172.21.15.7:0/1375146479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437f7fe700 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437e7fc700 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 -- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 msgr2=0x55e065101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 -- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 msgr2=0x55e065143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 -- 172.21.15.7:0/1375146479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e065022290 con 0x55e065128170 2024-01-11T07:12:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437f7fe700 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437e7fc700 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.596+0000 7f437effd700 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e06514e450 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f437000a040 tx=0x7f4370061250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8bf4bacab3dfb8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f4374ff9700 1 -- 172.21.15.7:0/1375146479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370081040 con 0x55e065128170 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f4374ff9700 1 -- 172.21.15.7:0/1375146479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437005b9b0 con 0x55e065128170 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f4374ff9700 1 -- 172.21.15.7:0/1375146479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370054a10 con 0x55e065128170 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1375146479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 msgr2=0x55e06514e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e06514e450 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f437000a040 tx=0x7f4370061250 comp rx=0 tx=0).stop 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1375146479 shutdown_connections 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e065143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e06514e450 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/1375146479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1375146479 >> 172.21.15.7:0/1375146479 conn(0x55e065102440 msgr2=0x55e065102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1375146479 shutdown_connections 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.597+0000 7f438e7bbb80 1 -- 172.21.15.7:0/1375146479 wait complete. 2024-01-11T07:12:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.598+0000 7f438e7bbb80 1 -- start start 2024-01-11T07:12:52.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.598+0000 7f438e7bbb80 1 Processor -- start 2024-01-11T07:12:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e0650fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065104d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f438e7bbb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e0651052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e065101ad0 con 0x55e065128170 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0651af9c0 con 0x55e06512c8b0 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f438e7bbb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e065125570 con 0x55e06512bfa0 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f438e7bbb80 10 client.0 _ll_register_callbacks cb 0x7f436801a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f437effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e0650fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f437effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e0650fa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32884/0 (socket says 172.21.15.7:32884) 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f437effd700 1 -- 172.21.15.7:0/2235365639 learned_addr learned my addr 172.21.15.7:0/2235365639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f437f7fe700 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e0651052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.599+0000 7f437e7fc700 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065104d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f437effd700 1 -- 172.21.15.7:0/2235365639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 msgr2=0x55e065104d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f437effd700 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065104d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f437effd700 1 -- 172.21.15.7:0/2235365639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 msgr2=0x55e0651052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f437effd700 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e0651052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f437effd700 1 -- 172.21.15.7:0/2235365639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e064ee7360 con 0x55e065128170 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f437effd700 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e0650fa6e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4370067170 tx=0x7f4370065000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581d07a2d34dff19 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f435d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f435d7fa700 1 -- 172.21.15.7:0/2235365639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370081040 con 0x55e065128170 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f435d7fa700 1 -- 172.21.15.7:0/2235365639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4370054a10 con 0x55e065128170 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f438e7bbb80 1 -- 172.21.15.7:0/2235365639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0650f8490 con 0x55e065128170 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f435d7fa700 1 -- 172.21.15.7:0/2235365639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4370054110 con 0x55e065128170 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f438e7bbb80 10 client.6855 fetch_fsmap learned FSMap version 57 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f438e7bbb80 1 -- 172.21.15.7:0/2235365639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55e064eebc80 con 0x55e065128170 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f435d7fa700 1 -- 172.21.15.7:0/2235365639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0650f8490 con 0x55e065128170 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f435d7fa700 1 -- 172.21.15.7:0/2235365639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 57) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f437005b520 con 0x55e065128170 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f438e7bbb80 10 client.6855 fetch_fsmap finished waiting for FSMap version 57 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f438e7bbb80 10 client.6855 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.600+0000 7f438e7bbb80 20 client.6855 populate_metadata read hostname 'smithi007' 2024-01-11T07:12:52.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] conn(0x55e065148240 0x55e065148610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f438e7bbb80 4 client.6855 mds_command: new command op to 6720 tid=0[{"prefix": "status"}] 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f438e7bbb80 1 -- 172.21.15.7:0/2235365639 --> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e064ff6910 con 0x55e065148240 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f437e7fc700 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] conn(0x55e065148240 0x55e065148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f435d7fa700 10 client.6855 ms_handle_connect on v2:172.21.15.7:6832/2493499246 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f437e7fc700 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] conn(0x55e065148240 0x55e065148610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df59b37ff6cbd897 server_cookie=fafaa4994eb1f473 in_seq=0 out_seq=0 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f435d7fa700 1 -- 172.21.15.7:0/2235365639 <== mds.0 v2:172.21.15.7:6832/2493499246 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e064ff6910 con 0x55e065148240 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f435d7fa700 10 client.6855 handle_command_reply: tid=0 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f438e7bbb80 1 client.6855 shutdown 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f438e7bbb80 1 -- 172.21.15.7:0/2235365639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 msgr2=0x55e0650fa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.601+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e0650fa6e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4370067170 tx=0x7f4370065000 comp rx=0 tx=0).stop 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 1 -- 172.21.15.7:0/2235365639 shutdown_connections 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.7:6832/2493499246,v1:172.21.15.7:6835/2493499246] conn(0x55e065148240 0x55e065148610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e06512c8b0 0x55e0651052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e065128170 0x55e0650fa6e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f435d7fa700 0 client.6855 ms_handle_reset on v2:172.21.15.7:6832/2493499246 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 1 --2- 172.21.15.7:0/2235365639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e06512bfa0 0x55e065104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 1 -- 172.21.15.7:0/2235365639 >> 172.21.15.7:0/2235365639 conn(0x55e065102440 msgr2=0x55e06514e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 1 -- 172.21.15.7:0/2235365639 shutdown_connections 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 1 -- 172.21.15.7:0/2235365639 wait complete. 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f438e7bbb80 20 client.6855 trim_cache size 0 max 16384 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6720, 2024-01-11T07:12:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:12:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:12:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:12:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1213649539999997, 2024-01-11T07:12:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 57, 2024-01-11T07:12:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 120, 2024-01-11T07:12:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 120, 2024-01-11T07:12:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 12.089154166 2024-01-11T07:12:52.609 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:12:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 -- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4360066f30 msgr2=0x7f43600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4360066f30 0x7f43600693e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f43680099b0 tx=0x7f4368017000 comp rx=0 tx=0).stop 2024-01-11T07:12:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 -- 172.21.15.7:0/2344661091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f438006e340 msgr2=0x7f43800c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f438006e340 0x7f43800c8230 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f437000c590 tx=0x7f437000c950 comp rx=0 tx=0).stop 2024-01-11T07:12:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 -- 172.21.15.7:0/2344661091 shutdown_connections 2024-01-11T07:12:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4360066f30 0x7f43600693e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f438015a070 0x7f43800c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f438006e340 0x7f43800c8230 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 --2- 172.21.15.7:0/2344661091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4380159ca0 0x7f43800b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 -- 172.21.15.7:0/2344661091 >> 172.21.15.7:0/2344661091 conn(0x7f4380063cc0 msgr2=0x7f43800c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 -- 172.21.15.7:0/2344661091 shutdown_connections 2024-01-11T07:12:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.602+0000 7f43854bb700 1 -- 172.21.15.7:0/2344661091 wait complete. 2024-01-11T07:12:52.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:12:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.747+0000 7fd9db72e700 1 Processor -- start 2024-01-11T07:12:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.747+0000 7fd9db72e700 1 -- start start 2024-01-11T07:12:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9db72e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9d4159e20 0x7fd9d415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9db72e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd9d406bc50 0x7fd9d406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9db72e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d415e010 con 0x7fd9d4159a50 2024-01-11T07:12:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9db72e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d406c5e0 con 0x7fd9d4159e20 2024-01-11T07:12:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9db72e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40acf80 con 0x7fd9d406bc50 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d8cc9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9d4159e20 0x7fd9d415e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d94ca700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd9d4159a50 0x7fd9d40ac450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45012/0 (socket says 172.21.15.7:45012) 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d8cc9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9d4159e20 0x7fd9d415e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32888/0 (socket says 172.21.15.7:32888) 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d94ca700 1 -- 172.21.15.7:0/663786757 learned_addr learned my addr 172.21.15.7:0/663786757 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d9ccb700 1 --2- 172.21.15.7:0/663786757 >> v2:172.21.15.113:3300/0 conn(0x7fd9d406bc50 0x7fd9d406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d8cc9700 1 -- 172.21.15.7:0/663786757 >> v2:172.21.15.113:3300/0 conn(0x7fd9d406bc50 msgr2=0x7fd9d406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d8cc9700 1 --2- 172.21.15.7:0/663786757 >> v2:172.21.15.113:3300/0 conn(0x7fd9d406bc50 0x7fd9d406c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d8cc9700 1 -- 172.21.15.7:0/663786757 >> v1:172.21.15.7:6789/0 conn(0x7fd9d4159a50 legacy=0x7fd9d40ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9cbfff700 1 -- 172.21.15.7:0/663786757 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854985583 0 0) 0x7fd9d415e010 con 0x7fd9d4159a50 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.748+0000 7fd9d8cc9700 1 -- 172.21.15.7:0/663786757 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d415d9c0 con 0x7fd9d4159e20 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9d8cc9700 1 --2- 172.21.15.7:0/663786757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d4159e20 0x7fd9d415e4a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd9d406aec0 tx=0x7fd9c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1328882d479e5f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9cbfff700 1 -- 172.21.15.7:0/663786757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4011630 con 0x7fd9d4159e20 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9cbfff700 1 -- 172.21.15.7:0/663786757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4011790 con 0x7fd9d4159e20 2024-01-11T07:12:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9cbfff700 1 -- 172.21.15.7:0/663786757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4011b30 con 0x7fd9d4159e20 2024-01-11T07:12:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9db72e700 1 -- 172.21.15.7:0/663786757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d4159e20 msgr2=0x7fd9d415e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9db72e700 1 --2- 172.21.15.7:0/663786757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d4159e20 0x7fd9d415e4a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd9d406aec0 tx=0x7fd9c40091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9db72e700 1 -- 172.21.15.7:0/663786757 shutdown_connections 2024-01-11T07:12:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9db72e700 1 --2- 172.21.15.7:0/663786757 >> v2:172.21.15.113:3300/0 conn(0x7fd9d406bc50 0x7fd9d406c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9db72e700 1 --2- 172.21.15.7:0/663786757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d4159e20 0x7fd9d415e4a0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9db72e700 1 -- 172.21.15.7:0/663786757 >> 172.21.15.7:0/663786757 conn(0x7fd9d40cd910 msgr2=0x7fd9d40cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9db72e700 1 -- 172.21.15.7:0/663786757 shutdown_connections 2024-01-11T07:12:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.749+0000 7fd9db72e700 1 -- 172.21.15.7:0/663786757 wait complete. 2024-01-11T07:12:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.751+0000 7fd9db72e700 1 Processor -- start 2024-01-11T07:12:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.751+0000 7fd9db72e700 1 -- start start 2024-01-11T07:12:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9db72e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d406bc50 0x7fd9d414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9db72e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d4159a50 0x7fd9d41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9db72e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d4159e20 0x7fd9d414a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9db72e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40ad4a0 con 0x7fd9d406bc50 2024-01-11T07:12:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9db72e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4152c10 con 0x7fd9d4159e20 2024-01-11T07:12:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9db72e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d414ace0 con 0x7fd9d4159a50 2024-01-11T07:12:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d406bc50 0x7fd9d414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d406bc50 0x7fd9d414c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32896/0 (socket says 172.21.15.7:32896) 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 -- 172.21.15.7:0/4038494585 learned_addr learned my addr 172.21.15.7:0/4038494585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d9ccb700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d4159e20 0x7fd9d414a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d8cc9700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d4159a50 0x7fd9d41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 -- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d4159a50 msgr2=0x7fd9d41526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d4159a50 0x7fd9d41526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 -- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d4159e20 msgr2=0x7fd9d414a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d4159e20 0x7fd9d414a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 -- 172.21.15.7:0/4038494585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4009420 con 0x7fd9d406bc50 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d9ccb700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d4159e20 0x7fd9d414a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d8cc9700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d4159a50 0x7fd9d41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9d94ca700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d406bc50 0x7fd9d414c1f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c7e0 tx=0x7fd9d0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9c97fa700 1 -- 172.21.15.7:0/4038494585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0005b30 con 0x7fd9d406bc50 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9c97fa700 1 -- 172.21.15.7:0/4038494585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0005c90 con 0x7fd9d406bc50 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9c97fa700 1 -- 172.21.15.7:0/4038494585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0007300 con 0x7fd9d406bc50 2024-01-11T07:12:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.752+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d414c8b0 con 0x7fd9d406bc50 2024-01-11T07:12:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.753+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d414ce70 con 0x7fd9d406bc50 2024-01-11T07:12:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.754+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d404c3c0 con 0x7fd9d406bc50 2024-01-11T07:12:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.754+0000 7fd9c97fa700 1 -- 172.21.15.7:0/4038494585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd9d001b400 con 0x7fd9d406bc50 2024-01-11T07:12:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.754+0000 7fd9c97fa700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9b8066f30 0x7fd9b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.755+0000 7fd9c97fa700 1 -- 172.21.15.7:0/4038494585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd9d0084030 con 0x7fd9d406bc50 2024-01-11T07:12:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.755+0000 7fd9d8cc9700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9b8066f30 0x7fd9b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.755+0000 7fd9d8cc9700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9b8066f30 0x7fd9b80693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40093f0 tx=0x7fd9c4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:52.759+0000 7fd9c97fa700 1 -- 172.21.15.7:0/4038494585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9d0051320 con 0x7fd9d406bc50 2024-01-11T07:12:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.015+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9d415dca0 con 0x7fd9d406bc50 2024-01-11T07:12:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.016+0000 7fd9c97fa700 1 -- 172.21.15.7:0/4038494585 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd9d0054940 con 0x7fd9d406bc50 2024-01-11T07:12:53.016 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9b8066f30 msgr2=0x7fd9b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9b8066f30 0x7fd9b80693e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40093f0 tx=0x7fd9c4017000 comp rx=0 tx=0).stop 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d406bc50 msgr2=0x7fd9d414c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d406bc50 0x7fd9d414c1f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c7e0 tx=0x7fd9d0007df0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 shutdown_connections 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd9b8066f30 0x7fd9b80693e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d4159e20 0x7fd9d414a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d406bc50 0x7fd9d414c1f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 --2- 172.21.15.7:0/4038494585 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d4159a50 0x7fd9d41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 >> 172.21.15.7:0/4038494585 conn(0x7fd9d40cd910 msgr2=0x7fd9d414d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 shutdown_connections 2024-01-11T07:12:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.018+0000 7fd9db72e700 1 -- 172.21.15.7:0/4038494585 wait complete. 2024-01-11T07:12:53.032 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:12:53.032 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:12:53.033 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:12:53.033 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:12:53.033 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:53.033 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:12:53.033 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:12:53.033 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:12:53.033 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:12:53.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:53.033 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:12:53.053 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:12:53.053 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:12:53.053 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:12:53.053 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:53.053 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:12:53.053 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:53.053 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:53.053 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42045sec preferred_lft 42045sec 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:12:53.054 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:12:53.055 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:53.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:53.056 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:12:53.124 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:12:53.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:53.126 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:12:53.189 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:12:53.189 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:12:53.189 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:12:53.189 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:12:53.189 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:12:53.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:53.190 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:12:53.190 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:12:53.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:12:53.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:12:53.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:12:53.190 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:12:53.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:53.313 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:12:53.333 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:12:53.333 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:12:53.333 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:12:53.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:53.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:12:53.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:12:53.409 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:53.470 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:12:53.471 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:12:53.471 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:53.533 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:12:53.533 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:12:53.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:12:53.626 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:12:53.626 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:12:53.626 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:12:53.636 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:12:53.636 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:12:53.636 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:12:53.636 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:12:53.636 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:12:53.654 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:12:53.654 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:12:53.654 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:12:53.654 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:12:53.654 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:12:53.654 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:12:53.654 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:12:53.654 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:12:53.656 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:12:53.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:53.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:12:53.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:12:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.868+0000 7f849ba69700 1 Processor -- start 2024-01-11T07:12:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.869+0000 7f849ba69700 1 -- start start 2024-01-11T07:12:53.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.869+0000 7f849ba69700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 0x7f84940ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:53.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.869+0000 7f849ba69700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8494158280 0x7f84940b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:53.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.869+0000 7f849ba69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84940b0970 0x7f8494159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:53.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.869+0000 7f849ba69700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84940b0df0 con 0x7f84940b0970 2024-01-11T07:12:53.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.869+0000 7f849ba69700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8494159300 con 0x7f8494158280 2024-01-11T07:12:53.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.869+0000 7f849ba69700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84940cf260 con 0x7f84940cafb0 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84940b0970 0x7f8494159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84940b0970 0x7f8494159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32910/0 (socket says 172.21.15.7:32910) 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 -- 172.21.15.7:0/289262096 learned_addr learned my addr 172.21.15.7:0/289262096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f8499805700 1 --2- 172.21.15.7:0/289262096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 0x7f84940ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f8499004700 1 --2- 172.21.15.7:0/289262096 >> v2:172.21.15.113:3300/0 conn(0x7f8494158280 0x7f84940b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 -- 172.21.15.7:0/289262096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 msgr2=0x7f84940ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 --2- 172.21.15.7:0/289262096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 0x7f84940ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 -- 172.21.15.7:0/289262096 >> v2:172.21.15.113:3300/0 conn(0x7f8494158280 msgr2=0x7f84940b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 --2- 172.21.15.7:0/289262096 >> v2:172.21.15.113:3300/0 conn(0x7f8494158280 0x7f84940b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 -- 172.21.15.7:0/289262096 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940cf540 con 0x7f84940b0970 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f849a006700 1 --2- 172.21.15.7:0/289262096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84940b0970 0x7f8494159a70 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8494064280 tx=0x7f848c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c60f6e6526bbe6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f8498803700 1 -- 172.21.15.7:0/289262096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c00e070 con 0x7f84940b0970 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f8498803700 1 -- 172.21.15.7:0/289262096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c00ba20 con 0x7f84940b0970 2024-01-11T07:12:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.870+0000 7f8498803700 1 -- 172.21.15.7:0/289262096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c0092e0 con 0x7f84940b0970 2024-01-11T07:12:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 -- 172.21.15.7:0/289262096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84940b0970 msgr2=0x7f8494159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 --2- 172.21.15.7:0/289262096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84940b0970 0x7f8494159a70 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8494064280 tx=0x7f848c009840 comp rx=0 tx=0).stop 2024-01-11T07:12:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 -- 172.21.15.7:0/289262096 shutdown_connections 2024-01-11T07:12:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 --2- 172.21.15.7:0/289262096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84940b0970 0x7f8494159a70 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 --2- 172.21.15.7:0/289262096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 0x7f84940ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 --2- 172.21.15.7:0/289262096 >> v2:172.21.15.113:3300/0 conn(0x7f8494158280 0x7f84940b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 -- 172.21.15.7:0/289262096 >> 172.21.15.7:0/289262096 conn(0x7f8494071100 msgr2=0x7f849406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:53.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 -- 172.21.15.7:0/289262096 shutdown_connections 2024-01-11T07:12:53.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.871+0000 7f849ba69700 1 -- 172.21.15.7:0/289262096 wait complete. 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.872+0000 7f849ba69700 1 Processor -- start 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.873+0000 7f849ba69700 1 -- start start 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.873+0000 7f849ba69700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84940b0970 0x7f8494068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.873+0000 7f849ba69700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 0x7f84941470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.873+0000 7f849ba69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8494158280 0x7f84940c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.873+0000 7f849ba69700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941475f0 con 0x7f8494158280 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.873+0000 7f849ba69700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84940c8b10 con 0x7f84940b0970 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.873+0000 7f849ba69700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84940c8d20 con 0x7f84940cafb0 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.873+0000 7f849a006700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8494158280 0x7f84940c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849a006700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8494158280 0x7f84940c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32916/0 (socket says 172.21.15.7:32916) 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849a006700 1 -- 172.21.15.7:0/1239893205 learned_addr learned my addr 172.21.15.7:0/1239893205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f8499805700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84940b0970 0x7f8494068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f8499004700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 0x7f84941470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:53.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849a006700 1 -- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 msgr2=0x7f84941470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849a006700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 0x7f84941470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849a006700 1 -- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84940b0970 msgr2=0x7f8494068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849a006700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84940b0970 0x7f8494068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849a006700 1 -- 172.21.15.7:0/1239893205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009ae0 con 0x7f8494158280 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849a006700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8494158280 0x7f84940c85d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f849406a1d0 tx=0x7f848c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f8489ffb700 1 -- 172.21.15.7:0/1239893205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c0032f0 con 0x7f8494158280 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f8489ffb700 1 -- 172.21.15.7:0/1239893205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c008e50 con 0x7f8494158280 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84940c9000 con 0x7f8494158280 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.874+0000 7f8489ffb700 1 -- 172.21.15.7:0/1239893205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c0266b0 con 0x7f8494158280 2024-01-11T07:12:53.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.875+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84940c95c0 con 0x7f8494158280 2024-01-11T07:12:53.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.875+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84940c9c10 con 0x7f8494158280 2024-01-11T07:12:53.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.876+0000 7f8489ffb700 1 -- 172.21.15.7:0/1239893205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f848c003040 con 0x7f8494158280 2024-01-11T07:12:53.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.876+0000 7f8489ffb700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8478066f30 0x7f84780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:53.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.876+0000 7f8489ffb700 1 -- 172.21.15.7:0/1239893205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f848c0865d0 con 0x7f8494158280 2024-01-11T07:12:53.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.877+0000 7f8499805700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8478066f30 0x7f84780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:53.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.877+0000 7f8499805700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8478066f30 0x7f84780693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8490009db0 tx=0x7f84900091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:53.881+0000 7f8489ffb700 1 -- 172.21.15.7:0/1239893205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f848c0537a0 con 0x7f8494158280 2024-01-11T07:12:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.135+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f849404c3c0 con 0x7f8494158280 2024-01-11T07:12:54.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.135+0000 7f8489ffb700 1 -- 172.21.15.7:0/1239893205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f848c056dc0 con 0x7f8494158280 2024-01-11T07:12:54.136 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8478066f30 msgr2=0x7f84780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8478066f30 0x7f84780693e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8490009db0 tx=0x7f84900091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8494158280 msgr2=0x7f84940c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8494158280 0x7f84940c85d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f849406a1d0 tx=0x7f848c018000 comp rx=0 tx=0).stop 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 shutdown_connections 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8478066f30 0x7f84780693e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8494158280 0x7f84940c85d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84940b0970 0x7f8494068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 --2- 172.21.15.7:0/1239893205 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84940cafb0 0x7f84941470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 >> 172.21.15.7:0/1239893205 conn(0x7f8494071100 msgr2=0x7f8494155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 shutdown_connections 2024-01-11T07:12:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.138+0000 7f849ba69700 1 -- 172.21.15.7:0/1239893205 wait complete. 2024-01-11T07:12:54.152 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:12:54.152 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:12:54.152 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:12:54.152 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:12:54.152 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:54.152 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:12:54.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:12:54.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:12:54.153 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:12:54.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:54.153 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:12:54.175 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42086sec preferred_lft 42086sec 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:12:54.176 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:12:54.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:54.178 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:12:54.245 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:12:54.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:54.246 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:12:54.307 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:12:54.307 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:12:54.307 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:12:54.307 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:12:54.307 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:12:54.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:54.307 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:12:54.307 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:12:54.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:12:54.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:12:54.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:12:54.308 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:12:54.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:54.418 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:12:54.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:12:54.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:12:54.436 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:12:54.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:54.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:12:54.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:12:54.505 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:54.562 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:12:54.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:12:54.563 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:54.619 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:12:54.619 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:12:54.619 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:12:54.702 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:12:54.702 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:12:54.702 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:12:54.710 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:12:54.710 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:12:54.711 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:12:54.711 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:12:54.711 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:12:54.728 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:12:54.728 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:12:54.728 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:12:54.728 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:12:54.729 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:12:54.729 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:12:54.729 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:12:54.729 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:12:54.731 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:12:54.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:54.732 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:12:54.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:12:54.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.939+0000 7f7bf311d700 1 Processor -- start 2024-01-11T07:12:54.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.939+0000 7f7bf311d700 1 -- start start 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf311d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bec15a280 0x7f7bec0aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf311d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bec0ac5b0 0x7f7bec0ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf311d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bec15d880 con 0x7f7bec063230 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf311d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bec06c5b0 con 0x7f7bec0ac5b0 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf311d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bec15a7f0 con 0x7f7bec15a280 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf16ba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bec0ac5b0 0x7f7bec0ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf16ba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bec0ac5b0 0x7f7bec0ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32918/0 (socket says 172.21.15.7:32918) 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf16ba700 1 -- 172.21.15.7:0/4102062414 learned_addr learned my addr 172.21.15.7:0/4102062414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bebfff700 1 --2- 172.21.15.7:0/4102062414 >> v2:172.21.15.113:3300/0 conn(0x7f7bec15a280 0x7f7bec0aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:54.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf16ba700 1 -- 172.21.15.7:0/4102062414 >> v2:172.21.15.113:3300/0 conn(0x7f7bec15a280 msgr2=0x7f7bec0aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf16ba700 1 --2- 172.21.15.7:0/4102062414 >> v2:172.21.15.113:3300/0 conn(0x7f7bec15a280 0x7f7bec0aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf16ba700 1 -- 172.21.15.7:0/4102062414 >> v1:172.21.15.113:6789/0 conn(0x7f7bec063230 legacy=0x7f7bec063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf16ba700 1 -- 172.21.15.7:0/4102062414 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec15aad0 con 0x7f7bec0ac5b0 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bebfff700 1 --2- 172.21.15.7:0/4102062414 >> v2:172.21.15.113:3300/0 conn(0x7f7bec15a280 0x7f7bec0aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.940+0000 7f7bf16ba700 1 --2- 172.21.15.7:0/4102062414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec0ac5b0 0x7f7bec0ac9f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0b39b0 tx=0x7f7be4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4773e788f950cd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7beb7fe700 1 -- 172.21.15.7:0/4102062414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be400e070 con 0x7f7bec0ac5b0 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7beb7fe700 1 -- 172.21.15.7:0/4102062414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be400ba50 con 0x7f7bec0ac5b0 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7beb7fe700 1 -- 172.21.15.7:0/4102062414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be40092e0 con 0x7f7bec0ac5b0 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7bf311d700 1 -- 172.21.15.7:0/4102062414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec0ac5b0 msgr2=0x7f7bec0ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4102062414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec0ac5b0 0x7f7bec0ac9f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0b39b0 tx=0x7f7be4009840 comp rx=0 tx=0).stop 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7bf311d700 1 -- 172.21.15.7:0/4102062414 shutdown_connections 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4102062414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec0ac5b0 0x7f7bec0ac9f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4102062414 >> v2:172.21.15.113:3300/0 conn(0x7f7bec15a280 0x7f7bec0aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7bf311d700 1 -- 172.21.15.7:0/4102062414 >> 172.21.15.7:0/4102062414 conn(0x7f7bec0680e0 msgr2=0x7f7bec0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:54.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7bf311d700 1 -- 172.21.15.7:0/4102062414 shutdown_connections 2024-01-11T07:12:54.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.941+0000 7f7bf311d700 1 -- 172.21.15.7:0/4102062414 wait complete. 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.942+0000 7f7bf311d700 1 Processor -- start 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.943+0000 7f7bf311d700 1 -- start start 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.943+0000 7f7bf311d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec063230 0x7f7bec15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.943+0000 7f7bf311d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec0ac5b0 0x7f7bec0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.943+0000 7f7bf311d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bec15a280 0x7f7bec0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.943+0000 7f7bf311d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec153740 con 0x7f7bec063230 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.943+0000 7f7bf311d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0c8cf0 con 0x7f7bec0ac5b0 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.943+0000 7f7bf311d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0b3000 con 0x7f7bec15a280 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec063230 0x7f7bec15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec063230 0x7f7bec15e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32932/0 (socket says 172.21.15.7:32932) 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 -- 172.21.15.7:0/4187084606 learned_addr learned my addr 172.21.15.7:0/4187084606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:54.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bebfff700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec0ac5b0 0x7f7bec0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf16ba700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bec15a280 0x7f7bec0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 -- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bec15a280 msgr2=0x7f7bec0b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bec15a280 0x7f7bec0b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 -- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec0ac5b0 msgr2=0x7f7bec0c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec0ac5b0 0x7f7bec0c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 -- 172.21.15.7:0/4187084606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be4009ae0 con 0x7f7bec063230 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf0eb9700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec063230 0x7f7bec15e4b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c4e0 tx=0x7f7bdc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7be8ff9700 1 -- 172.21.15.7:0/4187084606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc011590 con 0x7f7bec063230 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7be8ff9700 1 -- 172.21.15.7:0/4187084606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bdc011710 con 0x7f7bec063230 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec0b2740 con 0x7f7bec063230 2024-01-11T07:12:54.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7be8ff9700 1 -- 172.21.15.7:0/4187084606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc011ad0 con 0x7f7bec063230 2024-01-11T07:12:54.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.944+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec0b2c90 con 0x7f7bec063230 2024-01-11T07:12:54.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.945+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec04c3c0 con 0x7f7bec063230 2024-01-11T07:12:54.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.946+0000 7f7be8ff9700 1 -- 172.21.15.7:0/4187084606 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7bdc01b3b0 con 0x7f7bec063230 2024-01-11T07:12:54.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.946+0000 7f7be8ff9700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7bd8066f30 0x7f7bd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:54.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.946+0000 7f7be8ff9700 1 -- 172.21.15.7:0/4187084606 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bdc083a70 con 0x7f7bec063230 2024-01-11T07:12:54.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.946+0000 7f7bebfff700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7bd8066f30 0x7f7bd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:54.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.947+0000 7f7bebfff700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7bd8066f30 0x7f7bd80693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7be0006f00 tx=0x7f7be0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:54.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:54.951+0000 7f7be8ff9700 1 -- 172.21.15.7:0/4187084606 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bdc050ce0 con 0x7f7bec063230 2024-01-11T07:12:55.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.202+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bec15adb0 con 0x7f7bec063230 2024-01-11T07:12:55.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.203+0000 7f7be8ff9700 1 -- 172.21.15.7:0/4187084606 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f7bdc017070 con 0x7f7bec063230 2024-01-11T07:12:55.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:55.204 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:12:45.905827+0000","modified":"2024-01-11T07:12:49.492046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2493499246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2493499246},{"type":"v1","addr":"172.21.15.7:6835","nonce":2493499246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-11T07:12:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7bd8066f30 msgr2=0x7f7bd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7bd8066f30 0x7f7bd80693e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7be0006f00 tx=0x7f7be0008040 comp rx=0 tx=0).stop 2024-01-11T07:12:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec063230 msgr2=0x7f7bec15e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec063230 0x7f7bec15e4b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c4e0 tx=0x7f7bdc00c8a0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 shutdown_connections 2024-01-11T07:12:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7bd8066f30 0x7f7bd80693e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7bec15a280 0x7f7bec0b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bec063230 0x7f7bec15e4b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 --2- 172.21.15.7:0/4187084606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bec0ac5b0 0x7f7bec0c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 >> 172.21.15.7:0/4187084606 conn(0x7f7bec0680e0 msgr2=0x7f7bec0c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 shutdown_connections 2024-01-11T07:12:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.206+0000 7f7bf311d700 1 -- 172.21.15.7:0/4187084606 wait complete. 2024-01-11T07:12:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-01-11T07:12:55.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:12:55.373 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":6720,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8812209790000001,"mdsmap_epoch":57,"osdmap_epoch":120,"osdmap_epoch_barrier":120,"uptime":14.849010442000001} 2024-01-11T07:12:55.374 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 6720, "mdsmap_epoch": 57, "osdmap_epoch": 120, "osdmap_epoch_barrier": 120, "rank_uptime": 8.881220979, "state": "up:active", "uptime": 14.849010442, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:12:55.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-11T07:12:55.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.510+0000 7fe3e9eb7700 1 Processor -- start 2024-01-11T07:12:55.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.510+0000 7fe3e9eb7700 1 -- start start 2024-01-11T07:12:55.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e9eb7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3e40b3cc0 0x7fe3e415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e9eb7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 0x7fe3e40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e9eb7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e415b680 con 0x7fe3e40adf00 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e9eb7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e415b960 con 0x7fe3e40b3cc0 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e9eb7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4158860 con 0x7fe3e415c330 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3e40b3cc0 0x7fe3e415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe3e40adf00 0x7fe3e4158080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45020/0 (socket says 172.21.15.7:45020) 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e37fe700 1 -- 172.21.15.7:0/1774434962 learned_addr learned my addr 172.21.15.7:0/1774434962 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e3fff700 1 --2- 172.21.15.7:0/1774434962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 0x7fe3e40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e2ffd700 1 -- 172.21.15.7:0/1774434962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 msgr2=0x7fe3e40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e2ffd700 1 --2- 172.21.15.7:0/1774434962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 0x7fe3e40ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e2ffd700 1 -- 172.21.15.7:0/1774434962 >> v1:172.21.15.7:6789/0 conn(0x7fe3e40adf00 legacy=0x7fe3e4158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e27fc700 1 -- 172.21.15.7:0/1774434962 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966082196 0 0) 0x7fe3e415b680 con 0x7fe3e40adf00 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e2ffd700 1 -- 172.21.15.7:0/1774434962 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e4158b40 con 0x7fe3e40b3cc0 2024-01-11T07:12:55.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e2ffd700 1 --2- 172.21.15.7:0/1774434962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40b3cc0 0x7fe3e415bdf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0098e0 tx=0x7fe3cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcfa04efd0f8350 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e27fc700 1 -- 172.21.15.7:0/1774434962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc009e80 con 0x7fe3e40b3cc0 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e27fc700 1 -- 172.21.15.7:0/1774434962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc00b860 con 0x7fe3e40b3cc0 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.512+0000 7fe3e27fc700 1 -- 172.21.15.7:0/1774434962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00bbe0 con 0x7fe3e40b3cc0 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.513+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1774434962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40b3cc0 msgr2=0x7fe3e415bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.513+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1774434962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40b3cc0 0x7fe3e415bdf0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0098e0 tx=0x7fe3cc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.513+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1774434962 shutdown_connections 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.513+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1774434962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 0x7fe3e40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.513+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1774434962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40b3cc0 0x7fe3e415bdf0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.513+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1774434962 >> 172.21.15.7:0/1774434962 conn(0x7fe3e406b910 msgr2=0x7fe3e406ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.513+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1774434962 shutdown_connections 2024-01-11T07:12:55.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.513+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1774434962 wait complete. 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.514+0000 7fe3e9eb7700 1 Processor -- start 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.514+0000 7fe3e9eb7700 1 -- start start 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e9eb7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40adf00 0x7fe3e4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e9eb7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40b3cc0 0x7fe3e40ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e9eb7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 0x7fe3e40cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e9eb7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e415b610 con 0x7fe3e40adf00 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e9eb7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40b0a70 con 0x7fe3e40b3cc0 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e9eb7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40af9d0 con 0x7fe3e415c330 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40adf00 0x7fe3e4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40adf00 0x7fe3e4159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32960/0 (socket says 172.21.15.7:32960) 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e37fe700 1 -- 172.21.15.7:0/1379778883 learned_addr learned my addr 172.21.15.7:0/1379778883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:55.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e3fff700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 0x7fe3e40cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e2ffd700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40b3cc0 0x7fe3e40ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.515+0000 7fe3e37fe700 1 -- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 msgr2=0x7fe3e40cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3e37fe700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 0x7fe3e40cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3e37fe700 1 -- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40b3cc0 msgr2=0x7fe3e40ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3e37fe700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40b3cc0 0x7fe3e40ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3e37fe700 1 -- 172.21.15.7:0/1379778883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc009420 con 0x7fe3e40adf00 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3e2ffd700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40b3cc0 0x7fe3e40ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3e37fe700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40adf00 0x7fe3e4159c50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00c5b0 tx=0x7fe3dc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3db7fe700 1 -- 172.21.15.7:0/1379778883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc005b30 con 0x7fe3e40adf00 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3db7fe700 1 -- 172.21.15.7:0/1379778883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc005c90 con 0x7fe3e40adf00 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e40afcb0 con 0x7fe3e40adf00 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3db7fe700 1 -- 172.21.15.7:0/1379778883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc007300 con 0x7fe3e40adf00 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.516+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e40b0270 con 0x7fe3e40adf00 2024-01-11T07:12:55.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.517+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e404c3c0 con 0x7fe3e40adf00 2024-01-11T07:12:55.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.517+0000 7fe3db7fe700 1 -- 172.21.15.7:0/1379778883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe3dc01b3b0 con 0x7fe3e40adf00 2024-01-11T07:12:55.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.518+0000 7fe3db7fe700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3c4066f30 0x7fe3c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:55.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.518+0000 7fe3db7fe700 1 -- 172.21.15.7:0/1379778883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe3dc083b10 con 0x7fe3e40adf00 2024-01-11T07:12:55.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.518+0000 7fe3e2ffd700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3c4066f30 0x7fe3c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:55.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.519+0000 7fe3e2ffd700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3c4066f30 0x7fe3c40693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0091e0 tx=0x7fe3cc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:55.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.523+0000 7fe3db7fe700 1 -- 172.21.15.7:0/1379778883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3dc050d80 con 0x7fe3e40adf00 2024-01-11T07:12:55.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:55.730+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fe3e406d530 con 0x7fe3e40adf00 2024-01-11T07:12:56.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.591+0000 7fe3db7fe700 1 -- 172.21.15.7:0/1379778883 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe3dc0170c0 con 0x7fe3e40adf00 2024-01-11T07:12:56.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3c4066f30 msgr2=0x7fe3c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:56.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3c4066f30 0x7fe3c40693e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0091e0 tx=0x7fe3cc017000 comp rx=0 tx=0).stop 2024-01-11T07:12:56.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40adf00 msgr2=0x7fe3e4159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:56.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40adf00 0x7fe3e4159c50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00c5b0 tx=0x7fe3dc001070 comp rx=0 tx=0).stop 2024-01-11T07:12:56.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 shutdown_connections 2024-01-11T07:12:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe3c4066f30 0x7fe3c40693e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3e415c330 0x7fe3e40cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e40adf00 0x7fe3e4159c50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 --2- 172.21.15.7:0/1379778883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40b3cc0 0x7fe3e40ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.594+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 >> 172.21.15.7:0/1379778883 conn(0x7fe3e406b910 msgr2=0x7fe3e406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.595+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 shutdown_connections 2024-01-11T07:12:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:56.595+0000 7fe3e9eb7700 1 -- 172.21.15.7:0/1379778883 wait complete. 2024-01-11T07:12:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-11T07:12:56.610 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:12:56.610 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:12:56.610 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-11T07:12:56.610 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-11T07:12:56.610 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-11T07:12:56.610 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-11T07:12:56.610 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:12:56.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:12:56.877 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T07:12:56.960 INFO:teuthology.orchestra.run.smithi007.stdout:client6637 2024-01-11T07:12:56.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:12:56.961 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c0c5e15-bc38-4016-89e4-891c3586efbb.client6637/osdmap') 2024-01-11T07:12:57.053 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 120 barrier 120 flags 0x588000 2024-01-11T07:12:57.053 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:12:57.053 INFO:teuthology.orchestra.run.smithi007.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:12:57.053 INFO:teuthology.orchestra.run.smithi007.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:12:57.053 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-11T07:12:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-11T07:12:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-11T07:12:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-11T07:12:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.113:6801 100% (exists, up) 100% -1 2024-01-11T07:12:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.113:6817 100% (exists, up) 100% -1 2024-01-11T07:12:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.113:6809 100% (exists, up) 100% -1 2024-01-11T07:12:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.113:6825 100% (exists, up) 100% -1 2024-01-11T07:12:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-11T07:12:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-11T07:12:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:795 bytes copied, 4.3572e-05 s, 18.2 MB/s 2024-01-11T07:12:57.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-01-11T07:12:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.193+0000 7f4aedfe1700 1 Processor -- start 2024-01-11T07:12:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.193+0000 7f4aedfe1700 1 -- start start 2024-01-11T07:12:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4aedfe1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ae8157da0 0x7f4ae80cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4aedfe1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae80ce220 0x7f4ae80ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4aedfe1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae806d270 con 0x7f4ae81579d0 2024-01-11T07:12:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4aedfe1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae80cda40 con 0x7f4ae8157da0 2024-01-11T07:12:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4aedfe1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae80cd5b0 con 0x7f4ae80ce220 2024-01-11T07:12:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4ae6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ae8157da0 0x7f4ae80cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4ae6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ae8157da0 0x7f4ae80cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32976/0 (socket says 172.21.15.7:32976) 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4ae6ffd700 1 -- 172.21.15.7:0/3536987389 learned_addr learned my addr 172.21.15.7:0/3536987389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4ae7fff700 1 --2- 172.21.15.7:0/3536987389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae80ce220 0x7f4ae80ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4ae6ffd700 1 -- 172.21.15.7:0/3536987389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae80ce220 msgr2=0x7f4ae80ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.194+0000 7f4ae6ffd700 1 --2- 172.21.15.7:0/3536987389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae80ce220 0x7f4ae80ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4ae6ffd700 1 -- 172.21.15.7:0/3536987389 >> v1:172.21.15.113:6789/0 conn(0x7f4ae81579d0 legacy=0x7f4ae815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4ae6ffd700 1 -- 172.21.15.7:0/3536987389 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae80ce660 con 0x7f4ae8157da0 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4ae6ffd700 1 --2- 172.21.15.7:0/3536987389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae8157da0 0x7f4ae80cdce0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80ce7f0 tx=0x7f4ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dda08b6137e03a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4ae67fc700 1 -- 172.21.15.7:0/3536987389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000b860 con 0x7f4ae8157da0 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4ae67fc700 1 -- 172.21.15.7:0/3536987389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad000b9c0 con 0x7f4ae8157da0 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4ae67fc700 1 -- 172.21.15.7:0/3536987389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000bd80 con 0x7f4ae8157da0 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4aedfe1700 1 -- 172.21.15.7:0/3536987389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae8157da0 msgr2=0x7f4ae80cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/3536987389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae8157da0 0x7f4ae80cdce0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80ce7f0 tx=0x7f4ad0009250 comp rx=0 tx=0).stop 2024-01-11T07:12:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4aedfe1700 1 -- 172.21.15.7:0/3536987389 shutdown_connections 2024-01-11T07:12:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/3536987389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae80ce220 0x7f4ae80ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/3536987389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae8157da0 0x7f4ae80cdce0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.195+0000 7f4aedfe1700 1 -- 172.21.15.7:0/3536987389 >> 172.21.15.7:0/3536987389 conn(0x7f4ae806ae00 msgr2=0x7f4ae806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.196+0000 7f4aedfe1700 1 -- 172.21.15.7:0/3536987389 shutdown_connections 2024-01-11T07:12:57.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.196+0000 7f4aedfe1700 1 -- 172.21.15.7:0/3536987389 wait complete. 2024-01-11T07:12:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.197+0000 7f4aedfe1700 1 Processor -- start 2024-01-11T07:12:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.197+0000 7f4aedfe1700 1 -- start start 2024-01-11T07:12:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4aedfe1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ae80ce220 0x7f4ae8140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4aedfe1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81579d0 0x7f4ae815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4aedfe1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae8157da0 0x7f4ae80cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4aedfe1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae80c6bf0 con 0x7f4ae81579d0 2024-01-11T07:12:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4aedfe1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae80c5d90 con 0x7f4ae80ce220 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4aedfe1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae80c5fe0 con 0x7f4ae8157da0 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81579d0 0x7f4ae815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81579d0 0x7f4ae815a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32992/0 (socket says 172.21.15.7:32992) 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 -- 172.21.15.7:0/282356291 learned_addr learned my addr 172.21.15.7:0/282356291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae7fff700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae8157da0 0x7f4ae80cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae77fe700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ae80ce220 0x7f4ae8140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 -- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae8157da0 msgr2=0x7f4ae80cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae8157da0 0x7f4ae80cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 -- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ae80ce220 msgr2=0x7f4ae8140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ae80ce220 0x7f4ae8140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 -- 172.21.15.7:0/282356291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00094f0 con 0x7f4ae81579d0 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae77fe700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ae80ce220 0x7f4ae8140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.198+0000 7f4ae6ffd700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81579d0 0x7f4ae815a8f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009290 tx=0x7f4ad0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.199+0000 7f4adf7fe700 1 -- 172.21.15.7:0/282356291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0011e20 con 0x7f4ae81579d0 2024-01-11T07:12:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.199+0000 7f4adf7fe700 1 -- 172.21.15.7:0/282356291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad001c690 con 0x7f4ae81579d0 2024-01-11T07:12:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.199+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae80c62c0 con 0x7f4ae81579d0 2024-01-11T07:12:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.199+0000 7f4adf7fe700 1 -- 172.21.15.7:0/282356291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad001ca10 con 0x7f4ae81579d0 2024-01-11T07:12:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.199+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae80c6880 con 0x7f4ae81579d0 2024-01-11T07:12:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.200+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae804c3c0 con 0x7f4ae81579d0 2024-01-11T07:12:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.200+0000 7f4adf7fe700 1 -- 172.21.15.7:0/282356291 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4ad001a450 con 0x7f4ae81579d0 2024-01-11T07:12:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.200+0000 7f4adf7fe700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ac8066f30 0x7f4ac80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.200+0000 7f4adf7fe700 1 -- 172.21.15.7:0/282356291 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ad0086790 con 0x7f4ae81579d0 2024-01-11T07:12:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.201+0000 7f4ae77fe700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ac8066f30 0x7f4ac80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.201+0000 7f4ae77fe700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ac8066f30 0x7f4ac80693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8007fd0 tx=0x7f4ad8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:57.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.205+0000 7f4adf7fe700 1 -- 172.21.15.7:0/282356291 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ad0053a00 con 0x7f4ae81579d0 2024-01-11T07:12:57.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:57.420+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f4ae80cc9a0 con 0x7f4ae81579d0 2024-01-11T07:12:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.231+0000 7f4adf7fe700 1 -- 172.21.15.7:0/282356291 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v122) v1 ==== 93+0+0 (secure 0 0 0) 0x7f4ad0057020 con 0x7f4ae81579d0 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ac8066f30 msgr2=0x7f4ac80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ac8066f30 0x7f4ac80693e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8007fd0 tx=0x7f4ad8009040 comp rx=0 tx=0).stop 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81579d0 msgr2=0x7f4ae815a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81579d0 0x7f4ae815a8f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009290 tx=0x7f4ad0016000 comp rx=0 tx=0).stop 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 shutdown_connections 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ac8066f30 0x7f4ac80693e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4ae8157da0 0x7f4ae80cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ae80ce220 0x7f4ae8140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 --2- 172.21.15.7:0/282356291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ae81579d0 0x7f4ae815a8f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 >> 172.21.15.7:0/282356291 conn(0x7f4ae806ae00 msgr2=0x7f4ae81535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 shutdown_connections 2024-01-11T07:12:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.234+0000 7f4aedfe1700 1 -- 172.21.15.7:0/282356291 wait complete. 2024-01-11T07:12:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:pauserd,pausewr is set 2024-01-11T07:12:58.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-01-11T07:12:58.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.389+0000 7f1d0a6b6700 1 Processor -- start 2024-01-11T07:12:58.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.389+0000 7f1d0a6b6700 1 -- start start 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.390+0000 7f1d0a6b6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d040cafb0 0x7f1d040ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.390+0000 7f1d0a6b6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d040b0b40 0x7f1d04161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.390+0000 7f1d0a6b6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04159300 con 0x7f1d04158280 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.390+0000 7f1d0a6b6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d040cf260 con 0x7f1d040b0b40 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.390+0000 7f1d0a6b6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040cf540 con 0x7f1d040cafb0 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.390+0000 7f1d08c53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d040b0b40 0x7f1d04161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d08c53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d040b0b40 0x7f1d04161760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41766/0 (socket says 172.21.15.7:41766) 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d08c53700 1 -- 172.21.15.7:0/502227824 learned_addr learned my addr 172.21.15.7:0/502227824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d03fff700 1 --2- 172.21.15.7:0/502227824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d040cafb0 0x7f1d040ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d08c53700 1 -- 172.21.15.7:0/502227824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d040cafb0 msgr2=0x7f1d040ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d08c53700 1 --2- 172.21.15.7:0/502227824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d040cafb0 0x7f1d040ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d08c53700 1 -- 172.21.15.7:0/502227824 >> v1:172.21.15.113:6789/0 conn(0x7f1d04158280 legacy=0x7f1d040b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d08c53700 1 -- 172.21.15.7:0/502227824 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d040cf820 con 0x7f1d040b0b40 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d03fff700 1 --2- 172.21.15.7:0/502227824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d040cafb0 0x7f1d040ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d08c53700 1 --2- 172.21.15.7:0/502227824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040b0b40 0x7f1d04161760 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009f20 tx=0x7f1cfc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcd02e8c5c2e23b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d02ffd700 1 -- 172.21.15.7:0/502227824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00e070 con 0x7f1d040b0b40 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d02ffd700 1 -- 172.21.15.7:0/502227824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc00ba80 con 0x7f1d040b0b40 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.391+0000 7f1d02ffd700 1 -- 172.21.15.7:0/502227824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0092e0 con 0x7f1d040b0b40 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.392+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/502227824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040b0b40 msgr2=0x7f1d04161760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.392+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/502227824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040b0b40 0x7f1d04161760 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009f20 tx=0x7f1cfc009840 comp rx=0 tx=0).stop 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.392+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/502227824 shutdown_connections 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.392+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/502227824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040b0b40 0x7f1d04161760 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.392+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/502227824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d040cafb0 0x7f1d040ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.392+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/502227824 >> 172.21.15.7:0/502227824 conn(0x7f1d04071100 msgr2=0x7f1d0406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.392+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/502227824 shutdown_connections 2024-01-11T07:12:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.392+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/502227824 wait complete. 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.393+0000 7f1d0a6b6700 1 Processor -- start 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.393+0000 7f1d0a6b6700 1 -- start start 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d0a6b6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d040b0b40 0x7f1d0416a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d0a6b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040cafb0 0x7f1d040c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d0a6b6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d04158280 0x7f1d04154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d0a6b6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04067610 con 0x7f1d040cafb0 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d0a6b6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0416a770 con 0x7f1d040b0b40 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d0a6b6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040b3580 con 0x7f1d04158280 2024-01-11T07:12:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040cafb0 0x7f1d040c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040cafb0 0x7f1d040c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41774/0 (socket says 172.21.15.7:41774) 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d037fe700 1 -- 172.21.15.7:0/456935609 learned_addr learned my addr 172.21.15.7:0/456935609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d08c53700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d04158280 0x7f1d04154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d03fff700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d040b0b40 0x7f1d0416a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d037fe700 1 -- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d04158280 msgr2=0x7f1d04154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d037fe700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d04158280 0x7f1d04154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d037fe700 1 -- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d040b0b40 msgr2=0x7f1d0416a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.394+0000 7f1d037fe700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d040b0b40 0x7f1d0416a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.395+0000 7f1d037fe700 1 -- 172.21.15.7:0/456935609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc009ae0 con 0x7f1d040cafb0 2024-01-11T07:12:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.395+0000 7f1d037fe700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040cafb0 0x7f1d040c7c10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800c5c0 tx=0x7f1cf8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.395+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8011590 con 0x7f1d040cafb0 2024-01-11T07:12:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.395+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf8011710 con 0x7f1d040cafb0 2024-01-11T07:12:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.395+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d04068c20 con 0x7f1d040cafb0 2024-01-11T07:12:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.395+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8011ad0 con 0x7f1d040cafb0 2024-01-11T07:12:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.395+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d040691e0 con 0x7f1d040cafb0 2024-01-11T07:12:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.396+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d040677c0 con 0x7f1d040cafb0 2024-01-11T07:12:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.396+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1cf8012420 con 0x7f1d040cafb0 2024-01-11T07:12:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.397+0000 7f1cebfff700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1cf0066f30 0x7f1cf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.397+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1cf8083800 con 0x7f1d040cafb0 2024-01-11T07:12:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.397+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=123+}) v3 -- 0x7f1cf006ea50 con 0x7f1d040cafb0 2024-01-11T07:12:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.397+0000 7f1d03fff700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1cf0066f30 0x7f1cf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.398+0000 7f1d03fff700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1cf0066f30 0x7f1cf00693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006f00 tx=0x7f1cf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.401+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cf804b5a0 con 0x7f1d040cafb0 2024-01-11T07:12:58.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:58.620+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f1d0404c3c0 con 0x7f1d040cafb0 2024-01-11T07:12:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.599+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(123..123 src has 1..123) v4 ==== 280+0+0 (secure 0 0 0) 0x7f1cf80547e0 con 0x7f1d040cafb0 2024-01-11T07:12:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.600+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=124}) v3 -- 0x7f1cf006f3a0 con 0x7f1d040cafb0 2024-01-11T07:12:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.601+0000 7f1cebfff700 1 -- 172.21.15.7:0/456935609 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1cf8017070 con 0x7f1d040cafb0 2024-01-11T07:12:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1cf0066f30 msgr2=0x7f1cf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1cf0066f30 0x7f1cf00693e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006f00 tx=0x7f1cf4008040 comp rx=0 tx=0).stop 2024-01-11T07:12:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040cafb0 msgr2=0x7f1d040c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040cafb0 0x7f1d040c7c10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800c5c0 tx=0x7f1cf8007f50 comp rx=0 tx=0).stop 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 shutdown_connections 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1cf0066f30 0x7f1cf00693e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d04158280 0x7f1d04154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d040b0b40 0x7f1d0416a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 --2- 172.21.15.7:0/456935609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d040cafb0 0x7f1d040c7c10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.604+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 >> 172.21.15.7:0/456935609 conn(0x7f1d04071100 msgr2=0x7f1d040cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.605+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 shutdown_connections 2024-01-11T07:12:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.605+0000 7f1d0a6b6700 1 -- 172.21.15.7:0/456935609 wait complete. 2024-01-11T07:12:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:pauserd,pausewr is unset 2024-01-11T07:12:59.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:12:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.757+0000 7fb8ca5d9700 1 Processor -- start 2024-01-11T07:12:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.757+0000 7fb8ca5d9700 1 -- start start 2024-01-11T07:12:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8ca5d9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 0x7fb8c40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8ca5d9700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8c40b0b40 0x7fb8c4161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8ca5d9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c4159300 con 0x7fb8c4158280 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8ca5d9700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c40cf260 con 0x7fb8c40b0b40 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8ca5d9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40cf540 con 0x7fb8c40cafb0 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8c37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8c4158280 0x7fb8c40b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39718/0 (socket says 172.21.15.7:39718) 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8c37fe700 1 -- 172.21.15.7:0/4079694355 learned_addr learned my addr 172.21.15.7:0/4079694355 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8c8b76700 1 --2- 172.21.15.7:0/4079694355 >> v2:172.21.15.113:3300/0 conn(0x7fb8c40b0b40 0x7fb8c4161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8c3fff700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 0x7fb8c40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299195191 0 0) 0x7fb8c4159300 con 0x7fb8c4158280 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.758+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b0003570 con 0x7fb8c4158280 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1128761917 0 0) 0x7fb8b0003570 con 0x7fb8c4158280 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 msgr2=0x7fb8c40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 0x7fb8c40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 >> v2:172.21.15.113:3300/0 conn(0x7fb8c40b0b40 msgr2=0x7fb8c4161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 --2- 172.21.15.7:0/4079694355 >> v2:172.21.15.113:3300/0 conn(0x7fb8c40b0b40 0x7fb8c4161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40cf820 con 0x7fb8c4158280 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fb8ac001470 con 0x7fb8c4158280 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 >> v1:172.21.15.7:6789/0 conn(0x7fb8c4158280 legacy=0x7fb8c40b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 0x7fb8b0003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b00040b0 0x7fb8b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b0004d50 0x7fb8b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40cf260 con 0x7fb8b00040b0 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40cf540 con 0x7fb8b0004d50 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b000bb60 con 0x7fb8c40cafb0 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c37fe700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b00040b0 0x7fb8b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c3fff700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 0x7fb8b0003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c8b76700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b0004d50 0x7fb8b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c37fe700 1 -- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 msgr2=0x7fb8b0003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c37fe700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 0x7fb8b0003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c37fe700 1 -- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b0004d50 msgr2=0x7fb8b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c37fe700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b0004d50 0x7fb8b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c37fe700 1 -- 172.21.15.7:0/4079694355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0003570 con 0x7fb8b00040b0 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.759+0000 7fb8c3fff700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c40cafb0 0x7fb8b0003ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8c37fe700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b00040b0 0x7fb8b0004810 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4158650 tx=0x7fb8ac001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95292df5306e4e03 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00cb40 con 0x7fb8b00040b0 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac00ccc0 con 0x7fb8b00040b0 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8c2ffd700 1 -- 172.21.15.7:0/4079694355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac005b70 con 0x7fb8b00040b0 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/4079694355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b00040b0 msgr2=0x7fb8b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8ca5d9700 1 --2- 172.21.15.7:0/4079694355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b00040b0 0x7fb8b0004810 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4158650 tx=0x7fb8ac001dc0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8c8b76700 1 -- 172.21.15.7:0/4079694355 reap_dead start 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/4079694355 shutdown_connections 2024-01-11T07:12:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/4079694355 >> 172.21.15.7:0/4079694355 conn(0x7fb8c4071100 msgr2=0x7fb8c406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/4079694355 shutdown_connections 2024-01-11T07:12:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.760+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/4079694355 wait complete. 2024-01-11T07:12:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.761+0000 7fb8ca5d9700 1 Processor -- start 2024-01-11T07:12:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.762+0000 7fb8ca5d9700 1 -- start start 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.762+0000 7fb8ca5d9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8b00040b0 0x7fb8c41534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8ca5d9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c416dc40 0x7fb8c416d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8ca5d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c416d560 0x7fb8c4068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8ca5d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c416da00 con 0x7fb8c416d560 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8ca5d9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b2340 con 0x7fb8c416dc40 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8ca5d9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c40b2620 con 0x7fb8b00040b0 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c416d560 0x7fb8c4068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c416d560 0x7fb8c4068020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41792/0 (socket says 172.21.15.7:41792) 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 -- 172.21.15.7:0/2744046048 learned_addr learned my addr 172.21.15.7:0/2744046048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c37fe700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c416dc40 0x7fb8c416d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c3fff700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8b00040b0 0x7fb8c41534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 -- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8b00040b0 msgr2=0x7fb8c41534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8b00040b0 0x7fb8c41534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 -- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c416dc40 msgr2=0x7fb8c416d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c416dc40 0x7fb8c416d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 -- 172.21.15.7:0/2744046048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40afc70 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8c8b76700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c416d560 0x7fb8c4068020 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0094d0 tx=0x7fb8bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8bbfff700 1 -- 172.21.15.7:0/2744046048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00e040 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8bbfff700 1 -- 172.21.15.7:0/2744046048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc00f040 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c40aff50 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.763+0000 7fb8bbfff700 1 -- 172.21.15.7:0/2744046048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00fd70 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.764+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c41539f0 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.764+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c40b00e0 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.765+0000 7fb8bbfff700 1 -- 172.21.15.7:0/2744046048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb8bc015640 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.766+0000 7fb8bbfff700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8a4066f30 0x7fb8a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.766+0000 7fb8bbfff700 1 -- 172.21.15.7:0/2744046048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb8bc085220 con 0x7fb8c416d560 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.766+0000 7fb8c3fff700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8a4066f30 0x7fb8a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:12:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.767+0000 7fb8c3fff700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8a4066f30 0x7fb8a40693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4006f00 tx=0x7fb8b40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:12:59.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.770+0000 7fb8bbfff700 1 -- 172.21.15.7:0/2744046048 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8bc052510 con 0x7fb8c416d560 2024-01-11T07:12:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.975+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8c404c3c0 con 0x7fb8c416d560 2024-01-11T07:12:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.976+0000 7fb8bbfff700 1 -- 172.21.15.7:0/2744046048 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+15324 (secure 0 0 0) 0x7fb8bc055b30 con 0x7fb8c416d560 2024-01-11T07:12:59.976 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:12:59.976 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":123,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:59.596060+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:12:43.805689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-11T07:12:44.880093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:12:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8a4066f30 msgr2=0x7fb8a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8a4066f30 0x7fb8a40693e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4006f00 tx=0x7fb8b40091a0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c416d560 msgr2=0x7fb8c4068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c416d560 0x7fb8c4068020 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc0094d0 tx=0x7fb8bc00b040 comp rx=0 tx=0).stop 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 shutdown_connections 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8a4066f30 0x7fb8a40693e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c416d560 0x7fb8c4068020 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8b00040b0 0x7fb8c41534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 --2- 172.21.15.7:0/2744046048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c416dc40 0x7fb8c416d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 >> 172.21.15.7:0/2744046048 conn(0x7fb8c4071100 msgr2=0x7fb8c41553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 shutdown_connections 2024-01-11T07:12:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:12:59.978+0000 7fb8ca5d9700 1 -- 172.21.15.7:0/2744046048 wait complete. 2024-01-11T07:12:59.992 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:12:59.992 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:12:59.992 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-11T07:12:59.992 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-11T07:12:59.992 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-11T07:12:59.992 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-11T07:12:59.992 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:13:00.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:00.255 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T07:13:00.337 INFO:teuthology.orchestra.run.smithi007.stdout:client6637 2024-01-11T07:13:00.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:00.338 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c0c5e15-bc38-4016-89e4-891c3586efbb.client6637/osdmap') 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 120 barrier 120 flags 0x588000 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.113:6801 100% (exists, up) 100% -1 2024-01-11T07:13:00.430 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.113:6817 100% (exists, up) 100% -1 2024-01-11T07:13:00.431 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.113:6809 100% (exists, up) 100% -1 2024-01-11T07:13:00.431 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.113:6825 100% (exists, up) 100% -1 2024-01-11T07:13:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-11T07:13:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-11T07:13:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:795 bytes copied, 4.3399e-05 s, 18.3 MB/s 2024-01-11T07:13:00.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.570+0000 7f9a3b37d700 1 Processor -- start 2024-01-11T07:13:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.570+0000 7f9a3b37d700 1 -- start start 2024-01-11T07:13:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a3b37d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a3b37d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a34069320 0x7f9a340b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a3b37d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a340ce9b0 con 0x7f9a34159c90 2024-01-11T07:13:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a3b37d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a340b1f30 con 0x7f9a34069320 2024-01-11T07:13:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a3b37d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340cf530 con 0x7f9a3415a060 2024-01-11T07:13:00.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a3991a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a34069320 0x7f9a340b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a3991a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a34069320 0x7f9a340b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41798/0 (socket says 172.21.15.7:41798) 2024-01-11T07:13:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a3991a700 1 -- 172.21.15.7:0/2782832302 learned_addr learned my addr 172.21.15.7:0/2782832302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.571+0000 7f9a38918700 1 --2- 172.21.15.7:0/2782832302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3991a700 1 -- 172.21.15.7:0/2782832302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 msgr2=0x7f9a340b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3991a700 1 --2- 172.21.15.7:0/2782832302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3991a700 1 -- 172.21.15.7:0/2782832302 >> v1:172.21.15.113:6789/0 conn(0x7f9a34159c90 legacy=0x7f9a340b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:00.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3991a700 1 -- 172.21.15.7:0/2782832302 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340cf9c0 con 0x7f9a34069320 2024-01-11T07:13:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a38918700 1 --2- 172.21.15.7:0/2782832302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3991a700 1 --2- 172.21.15.7:0/2782832302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340b0970 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009f20 tx=0x7f9a2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462d571dfcb0d741 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a2bfff700 1 -- 172.21.15.7:0/2782832302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00e070 con 0x7f9a34069320 2024-01-11T07:13:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a2bfff700 1 -- 172.21.15.7:0/2782832302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c00ba80 con 0x7f9a34069320 2024-01-11T07:13:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a2bfff700 1 -- 172.21.15.7:0/2782832302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0092e0 con 0x7f9a34069320 2024-01-11T07:13:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3b37d700 1 -- 172.21.15.7:0/2782832302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 msgr2=0x7f9a340b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/2782832302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340b0970 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009f20 tx=0x7f9a2c009840 comp rx=0 tx=0).stop 2024-01-11T07:13:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3b37d700 1 -- 172.21.15.7:0/2782832302 shutdown_connections 2024-01-11T07:13:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/2782832302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340b0970 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/2782832302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.572+0000 7f9a3b37d700 1 -- 172.21.15.7:0/2782832302 >> 172.21.15.7:0/2782832302 conn(0x7f9a34071100 msgr2=0x7f9a3406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.573+0000 7f9a3b37d700 1 -- 172.21.15.7:0/2782832302 shutdown_connections 2024-01-11T07:13:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.573+0000 7f9a3b37d700 1 -- 172.21.15.7:0/2782832302 wait complete. 2024-01-11T07:13:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.574+0000 7f9a3b37d700 1 Processor -- start 2024-01-11T07:13:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.574+0000 7f9a3b37d700 1 -- start start 2024-01-11T07:13:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a3b37d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a3b37d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a34159c90 0x7f9a34155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a3b37d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:00.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a3b37d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340c9670 con 0x7f9a34069320 2024-01-11T07:13:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a3b37d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340c9090 con 0x7f9a34159c90 2024-01-11T07:13:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a3b37d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340ca040 con 0x7f9a3415a060 2024-01-11T07:13:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41812/0 (socket says 172.21.15.7:41812) 2024-01-11T07:13:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 -- 172.21.15.7:0/1559886677 learned_addr learned my addr 172.21.15.7:0/1559886677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a38918700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a34159c90 0x7f9a34155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a3991a700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 -- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 msgr2=0x7f9a340c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 -- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a34159c90 msgr2=0x7f9a34155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a34159c90 0x7f9a34155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 -- 172.21.15.7:0/1559886677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c009ae0 con 0x7f9a34069320 2024-01-11T07:13:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.575+0000 7f9a39119700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340c9b00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c5c0 tx=0x7f9a3000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.576+0000 7f9a297fa700 1 -- 172.21.15.7:0/1559886677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30011590 con 0x7f9a34069320 2024-01-11T07:13:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.576+0000 7f9a297fa700 1 -- 172.21.15.7:0/1559886677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a30011710 con 0x7f9a34069320 2024-01-11T07:13:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.576+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a34156850 con 0x7f9a34069320 2024-01-11T07:13:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.576+0000 7f9a297fa700 1 -- 172.21.15.7:0/1559886677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30011ad0 con 0x7f9a34069320 2024-01-11T07:13:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.576+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a340c8a90 con 0x7f9a34069320 2024-01-11T07:13:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.577+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3404c3c0 con 0x7f9a34069320 2024-01-11T07:13:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.577+0000 7f9a297fa700 1 -- 172.21.15.7:0/1559886677 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9a3001b3b0 con 0x7f9a34069320 2024-01-11T07:13:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.578+0000 7f9a297fa700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a18066f30 0x7f9a180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.578+0000 7f9a297fa700 1 -- 172.21.15.7:0/1559886677 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a300839b0 con 0x7f9a34069320 2024-01-11T07:13:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.578+0000 7f9a38918700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a18066f30 0x7f9a180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.579+0000 7f9a38918700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a18066f30 0x7f9a180693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a34160870 tx=0x7f9a24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:00.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.582+0000 7f9a297fa700 1 -- 172.21.15.7:0/1559886677 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a30050c20 con 0x7f9a34069320 2024-01-11T07:13:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.830+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a341592d0 con 0x7f9a34069320 2024-01-11T07:13:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.831+0000 7f9a297fa700 1 -- 172.21.15.7:0/1559886677 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f9a30017070 con 0x7f9a34069320 2024-01-11T07:13:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:00.831 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:12:45.905827+0000","modified":"2024-01-11T07:12:49.492046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2493499246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2493499246},{"type":"v1","addr":"172.21.15.7:6835","nonce":2493499246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a18066f30 msgr2=0x7f9a180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a18066f30 0x7f9a180693e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a34160870 tx=0x7f9a24008040 comp rx=0 tx=0).stop 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 msgr2=0x7f9a340c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340c9b00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000c5c0 tx=0x7f9a3000c8d0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 shutdown_connections 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a18066f30 0x7f9a180693e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a3415a060 0x7f9a340c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a34069320 0x7f9a340c9b00 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 --2- 172.21.15.7:0/1559886677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a34159c90 0x7f9a34155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 >> 172.21.15.7:0/1559886677 conn(0x7f9a34071100 msgr2=0x7f9a340b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 shutdown_connections 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:00.833+0000 7f9a3b37d700 1 -- 172.21.15.7:0/1559886677 wait complete. 2024-01-11T07:13:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-01-11T07:13:00.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 123 2024-01-11T07:13:00.997 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-01-11T07:13:00.998 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:13:00.998 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:13:00.998 DEBUG:teuthology.orchestra.run.smithi007:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-01-11T07:13:00.998 DEBUG:teuthology.orchestra.run.smithi007:> os.fchmod(fd, 0O666) 2024-01-11T07:13:00.998 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(fd) 2024-01-11T07:13:00.998 DEBUG:teuthology.orchestra.run.smithi007:> os.close(fd) 2024-01-11T07:13:00.998 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:13:01.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:13:01.144 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:13:01.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:01.274 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T07:13:01.317 INFO:teuthology.orchestra.run.smithi007.stdout:client6637 2024-01-11T07:13:01.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:01.317 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c0c5e15-bc38-4016-89e4-891c3586efbb.client6637/osdmap') 2024-01-11T07:13:01.410 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 123 barrier 123 flags 0x588000 2024-01-11T07:13:01.410 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:01.410 INFO:teuthology.orchestra.run.smithi007.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:01.410 INFO:teuthology.orchestra.run.smithi007.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:01.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-11T07:13:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-11T07:13:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-11T07:13:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-11T07:13:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.113:6801 100% (exists, up) 100% -1 2024-01-11T07:13:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.113:6817 100% (exists, up) 100% -1 2024-01-11T07:13:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.113:6809 100% (exists, up) 100% -1 2024-01-11T07:13:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.113:6825 100% (exists, up) 100% -1 2024-01-11T07:13:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-11T07:13:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-11T07:13:01.412 INFO:teuthology.orchestra.run.smithi007.stderr:795 bytes copied, 4.2802e-05 s, 18.6 MB/s 2024-01-11T07:13:01.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:01.412 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-11T07:13:01.452 INFO:teuthology.orchestra.run.smithi007.stdout:client6637 2024-01-11T07:13:01.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:01.452 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9c0c5e15-bc38-4016-89e4-891c3586efbb.client6637/osdmap') 2024-01-11T07:13:01.545 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 123 barrier 123 flags 0x588000 2024-01-11T07:13:01.545 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:01.545 INFO:teuthology.orchestra.run.smithi007.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:01.545 INFO:teuthology.orchestra.run.smithi007.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2024-01-11T07:13:01.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd0 (1)172.21.15.7:6817 100% (exists, up) 100% -1 2024-01-11T07:13:01.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd1 (1)172.21.15.7:6809 100% (exists, up) 100% -1 2024-01-11T07:13:01.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd2 (1)172.21.15.7:6825 100% (exists, up) 100% -1 2024-01-11T07:13:01.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd3 (1)172.21.15.7:6801 100% (exists, up) 100% -1 2024-01-11T07:13:01.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd4 (1)172.21.15.113:6801 100% (exists, up) 100% -1 2024-01-11T07:13:01.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd5 (1)172.21.15.113:6817 100% (exists, up) 100% -1 2024-01-11T07:13:01.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd6 (1)172.21.15.113:6809 100% (exists, up) 100% -1 2024-01-11T07:13:01.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd7 (1)172.21.15.113:6825 100% (exists, up) 100% -1 2024-01-11T07:13:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records in 2024-01-11T07:13:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:1+1 records out 2024-01-11T07:13:01.547 INFO:teuthology.orchestra.run.smithi007.stderr:795 bytes copied, 4.307e-05 s, 18.5 MB/s 2024-01-11T07:13:01.547 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:13:01.547 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:13:01.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:13:01.601 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:13:01.653 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:13:01.653 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:13:01.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:01.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:13:01.726 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:13:01.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:01.727 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:13:01.756 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:13:01.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:01.757 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:13:01.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:13:01.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:01.806 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:13:01.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.964+0000 7f24a32a2700 1 Processor -- start 2024-01-11T07:13:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.964+0000 7f24a32a2700 1 -- start start 2024-01-11T07:13:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a32a2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f249c15baa0 0x7f249c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a32a2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c0cd8c0 con 0x7f249c0b2b50 2024-01-11T07:13:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a32a2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249c15f210 con 0x7f249c15be70 2024-01-11T07:13:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a32a2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249c0ce290 con 0x7f249c15baa0 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a103e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f249c15baa0 0x7f249c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a183f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f249c0b2b50 0x7f249c0ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39732/0 (socket says 172.21.15.7:39732) 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a183f700 1 -- 172.21.15.7:0/2247521806 learned_addr learned my addr 172.21.15.7:0/2247521806 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a103e700 1 -- 172.21.15.7:0/2247521806 >> v1:172.21.15.113:6789/0 conn(0x7f249c15be70 legacy=0x7f249c15e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a103e700 1 -- 172.21.15.7:0/2247521806 >> v1:172.21.15.7:6789/0 conn(0x7f249c0b2b50 legacy=0x7f249c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f24a103e700 1 -- 172.21.15.7:0/2247521806 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c15f4f0 con 0x7f249c15baa0 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.965+0000 7f2493fff700 1 -- 172.21.15.7:0/2247521806 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460390980 0 0) 0x7f249c0cd8c0 con 0x7f249c0b2b50 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f24a103e700 1 --2- 172.21.15.7:0/2247521806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15baa0 0x7f249c0cdd50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f24980098e0 tx=0x7f24980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f298f3ffe51652fb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f2493fff700 1 -- 172.21.15.7:0/2247521806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498009e80 con 0x7f249c15baa0 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f2493fff700 1 -- 172.21.15.7:0/2247521806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249800b820 con 0x7f249c15baa0 2024-01-11T07:13:01.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f2493fff700 1 -- 172.21.15.7:0/2247521806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249800bba0 con 0x7f249c15baa0 2024-01-11T07:13:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f24a32a2700 1 -- 172.21.15.7:0/2247521806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15baa0 msgr2=0x7f249c0cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f24a32a2700 1 --2- 172.21.15.7:0/2247521806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15baa0 0x7f249c0cdd50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f24980098e0 tx=0x7f24980091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f24a32a2700 1 -- 172.21.15.7:0/2247521806 shutdown_connections 2024-01-11T07:13:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f24a32a2700 1 --2- 172.21.15.7:0/2247521806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15baa0 0x7f249c0cdd50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f24a32a2700 1 -- 172.21.15.7:0/2247521806 >> 172.21.15.7:0/2247521806 conn(0x7f249c071120 msgr2=0x7f249c06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.966+0000 7f24a32a2700 1 -- 172.21.15.7:0/2247521806 shutdown_connections 2024-01-11T07:13:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.967+0000 7f24a32a2700 1 -- 172.21.15.7:0/2247521806 wait complete. 2024-01-11T07:13:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.968+0000 7f24a32a2700 1 Processor -- start 2024-01-11T07:13:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.968+0000 7f24a32a2700 1 -- start start 2024-01-11T07:13:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a32a2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f249c0b2b50 0x7f249c171850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a32a2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f249c15baa0 0x7f249c14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a32a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15be70 0x7f249c16ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a32a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c16e270 con 0x7f249c15be70 2024-01-11T07:13:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a32a2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c171d90 con 0x7f249c0b2b50 2024-01-11T07:13:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a32a2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c172030 con 0x7f249c15baa0 2024-01-11T07:13:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15be70 0x7f249c16ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15be70 0x7f249c16ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41830/0 (socket says 172.21.15.7:41830) 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 -- 172.21.15.7:0/3928522903 learned_addr learned my addr 172.21.15.7:0/3928522903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a103e700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f249c0b2b50 0x7f249c171850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a083d700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f249c15baa0 0x7f249c14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 -- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f249c15baa0 msgr2=0x7f249c14fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f249c15baa0 0x7f249c14fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 -- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f249c0b2b50 msgr2=0x7f249c171850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f249c0b2b50 0x7f249c171850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 -- 172.21.15.7:0/3928522903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498009420 con 0x7f249c15be70 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a183f700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15be70 0x7f249c16ea30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f249400cc90 tx=0x7f2494001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24917fa700 1 -- 172.21.15.7:0/3928522903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249400cea0 con 0x7f249c15be70 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24917fa700 1 -- 172.21.15.7:0/3928522903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249400ab70 con 0x7f249c15be70 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249c172310 con 0x7f249c15be70 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.969+0000 7f24917fa700 1 -- 172.21.15.7:0/3928522903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24940070f0 con 0x7f249c15be70 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.970+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c16e510 con 0x7f249c15be70 2024-01-11T07:13:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.970+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c04c3c0 con 0x7f249c15be70 2024-01-11T07:13:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.971+0000 7f24917fa700 1 -- 172.21.15.7:0/3928522903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f24940074e0 con 0x7f249c15be70 2024-01-11T07:13:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.971+0000 7f24917fa700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.971+0000 7f24917fa700 1 -- 172.21.15.7:0/3928522903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2494084920 con 0x7f249c15be70 2024-01-11T07:13:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.972+0000 7f24a103e700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.972+0000 7f24a103e700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2480066f30 0x7f24800693e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f24980030c0 tx=0x7f2498016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:01.976+0000 7f24917fa700 1 -- 172.21.15.7:0/3928522903 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2494051c10 con 0x7f249c15be70 2024-01-11T07:13:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.229+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f249c16e700 con 0x7f249c15be70 2024-01-11T07:13:02.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.231+0000 7f24917fa700 1 -- 172.21.15.7:0/3928522903 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f2494055230 con 0x7f249c15be70 2024-01-11T07:13:02.230 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:02.230 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:12:45.905827+0000","modified":"2024-01-11T07:12:49.492046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2493499246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2493499246},{"type":"v1","addr":"172.21.15.7:6835","nonce":2493499246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-11T07:13:02.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2480066f30 msgr2=0x7f24800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2480066f30 0x7f24800693e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f24980030c0 tx=0x7f2498016000 comp rx=0 tx=0).stop 2024-01-11T07:13:02.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15be70 msgr2=0x7f249c16ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15be70 0x7f249c16ea30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f249400cc90 tx=0x7f2494001a80 comp rx=0 tx=0).stop 2024-01-11T07:13:02.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 shutdown_connections 2024-01-11T07:13:02.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2480066f30 0x7f24800693e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f249c15be70 0x7f249c16ea30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f249c0b2b50 0x7f249c171850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 --2- 172.21.15.7:0/3928522903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f249c15baa0 0x7f249c14fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 >> 172.21.15.7:0/3928522903 conn(0x7f249c071120 msgr2=0x7f249c0c6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:02.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 shutdown_connections 2024-01-11T07:13:02.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.233+0000 7f24a32a2700 1 -- 172.21.15.7:0/3928522903 wait complete. 2024-01-11T07:13:02.234 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-01-11T07:13:02.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.378+0000 7fa4e9e3f700 1 Processor -- start 2024-01-11T07:13:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.378+0000 7fa4e9e3f700 1 -- start start 2024-01-11T07:13:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e9e3f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4e40b0b40 0x7fa4e4162790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e9e3f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e40cc2a0 con 0x7fa4e40adb40 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e9e3f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e40b3b50 con 0x7fa4e40adf10 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e9e3f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40b3e30 con 0x7fa4e40b0b40 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa4e40adb40 0x7fa4e40b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39746/0 (socket says 172.21.15.7:39746) 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4e40b0b40 0x7fa4e4162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e37fe700 1 -- 172.21.15.7:0/3774886060 learned_addr learned my addr 172.21.15.7:0/3774886060 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e3fff700 1 -- 172.21.15.7:0/3774886060 >> v1:172.21.15.113:6789/0 conn(0x7fa4e40adf10 legacy=0x7fa4e40b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e3fff700 1 -- 172.21.15.7:0/3774886060 >> v1:172.21.15.7:6789/0 conn(0x7fa4e40adb40 legacy=0x7fa4e40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.379+0000 7fa4e3fff700 1 -- 172.21.15.7:0/3774886060 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40cc540 con 0x7fa4e40b0b40 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.380+0000 7fa4e3fff700 1 --2- 172.21.15.7:0/3774886060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40b0b40 0x7fa4e4162790 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009e60 tx=0x7fa4dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc40defedd044bf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.380+0000 7fa4e27fc700 1 -- 172.21.15.7:0/3774886060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00e070 con 0x7fa4e40b0b40 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.380+0000 7fa4e27fc700 1 -- 172.21.15.7:0/3774886060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc00bbf0 con 0x7fa4e40b0b40 2024-01-11T07:13:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.380+0000 7fa4e27fc700 1 -- 172.21.15.7:0/3774886060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc012430 con 0x7fa4e40b0b40 2024-01-11T07:13:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.380+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/3774886060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40b0b40 msgr2=0x7fa4e4162790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.380+0000 7fa4e9e3f700 1 --2- 172.21.15.7:0/3774886060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40b0b40 0x7fa4e4162790 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009e60 tx=0x7fa4dc009780 comp rx=0 tx=0).stop 2024-01-11T07:13:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.380+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/3774886060 shutdown_connections 2024-01-11T07:13:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.381+0000 7fa4e9e3f700 1 --2- 172.21.15.7:0/3774886060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40b0b40 0x7fa4e4162790 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.381+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/3774886060 >> 172.21.15.7:0/3774886060 conn(0x7fa4e4071160 msgr2=0x7fa4e406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.381+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/3774886060 shutdown_connections 2024-01-11T07:13:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.381+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/3774886060 wait complete. 2024-01-11T07:13:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.382+0000 7fa4e9e3f700 1 Processor -- start 2024-01-11T07:13:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.382+0000 7fa4e9e3f700 1 -- start start 2024-01-11T07:13:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.383+0000 7fa4e9e3f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40adb40 0x7fa4e415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.383+0000 7fa4e9e3f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e40adf10 0x7fa4e40cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.383+0000 7fa4e9e3f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e40b0b40 0x7fa4e415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.383+0000 7fa4e9e3f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e415c210 con 0x7fa4e40adb40 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.383+0000 7fa4e9e3f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e415c4f0 con 0x7fa4e40adf10 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.383+0000 7fa4e9e3f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e415c7d0 con 0x7fa4e40b0b40 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40adb40 0x7fa4e415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40adb40 0x7fa4e415d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41856/0 (socket says 172.21.15.7:41856) 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 -- 172.21.15.7:0/701373056 learned_addr learned my addr 172.21.15.7:0/701373056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e3fff700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e40b0b40 0x7fa4e415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e2ffd700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e40adf10 0x7fa4e40cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 -- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e40b0b40 msgr2=0x7fa4e415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e40b0b40 0x7fa4e415b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 -- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e40adf10 msgr2=0x7fa4e40cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e40adf10 0x7fa4e40cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 -- 172.21.15.7:0/701373056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc009a20 con 0x7fa4e40adb40 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e2ffd700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e40adf10 0x7fa4e40cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e37fe700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40adb40 0x7fa4e415d290 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400c7c0 tx=0x7fa4d4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4cb7fe700 1 -- 172.21.15.7:0/701373056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4005b30 con 0x7fa4e40adb40 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4cb7fe700 1 -- 172.21.15.7:0/701373056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4005c90 con 0x7fa4e40adb40 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e415cab0 con 0x7fa4e40adb40 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4cb7fe700 1 -- 172.21.15.7:0/701373056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4004a80 con 0x7fa4e40adb40 2024-01-11T07:13:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.384+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e40a6670 con 0x7fa4e40adb40 2024-01-11T07:13:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.386+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e404c3c0 con 0x7fa4e40adb40 2024-01-11T07:13:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.386+0000 7fa4cb7fe700 1 -- 172.21.15.7:0/701373056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa4d40070d0 con 0x7fa4e40adb40 2024-01-11T07:13:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.386+0000 7fa4cb7fe700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4d0066f30 0x7fa4d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.386+0000 7fa4cb7fe700 1 -- 172.21.15.7:0/701373056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa4d4083900 con 0x7fa4e40adb40 2024-01-11T07:13:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.387+0000 7fa4e2ffd700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4d0066f30 0x7fa4d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.387+0000 7fa4e2ffd700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4d0066f30 0x7fa4d00693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8007fd0 tx=0x7fa4d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:02.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.391+0000 7fa4cb7fe700 1 -- 172.21.15.7:0/701373056 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4d404b5a0 con 0x7fa4e40adb40 2024-01-11T07:13:02.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.599+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4e40b4110 con 0x7fa4e40adb40 2024-01-11T07:13:02.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.600+0000 7fa4cb7fe700 1 -- 172.21.15.7:0/701373056 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+15324 (secure 0 0 0) 0x7fa4d4050b70 con 0x7fa4e40adb40 2024-01-11T07:13:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:02.600 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":123,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:59.596060+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:12:43.805689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-11T07:12:44.880093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4d0066f30 msgr2=0x7fa4d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4d0066f30 0x7fa4d00693e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8007fd0 tx=0x7fa4d8009040 comp rx=0 tx=0).stop 2024-01-11T07:13:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40adb40 msgr2=0x7fa4e415d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40adb40 0x7fa4e415d290 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400c7c0 tx=0x7fa4d4007f70 comp rx=0 tx=0).stop 2024-01-11T07:13:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 shutdown_connections 2024-01-11T07:13:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4d0066f30 0x7fa4d00693e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4e40b0b40 0x7fa4e415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4e40adb40 0x7fa4e415d290 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 --2- 172.21.15.7:0/701373056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4e40adf10 0x7fa4e40cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 >> 172.21.15.7:0/701373056 conn(0x7fa4e4071160 msgr2=0x7fa4e40ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 shutdown_connections 2024-01-11T07:13:02.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.602+0000 7fa4e9e3f700 1 -- 172.21.15.7:0/701373056 wait complete. 2024-01-11T07:13:02.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:12:43.805689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:12:44.880093+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:13:02.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:13:02.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.752+0000 7f2b756bf700 1 Processor -- start 2024-01-11T07:13:02.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.752+0000 7f2b756bf700 1 -- start start 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b756bf700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b70159e30 0x7f2b700af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b756bf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b700d16e0 0x7f2b7006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b756bf700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7006e5d0 con 0x7f2b7006e170 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b756bf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7015c080 con 0x7f2b700d16e0 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b756bf700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b700b00f0 con 0x7f2b70159e30 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b6e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b700d16e0 0x7f2b7006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b6e7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b700d16e0 0x7f2b7006dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41866/0 (socket says 172.21.15.7:41866) 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b6e7fc700 1 -- 172.21.15.7:0/2653327531 learned_addr learned my addr 172.21.15.7:0/2653327531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b6effd700 1 --2- 172.21.15.7:0/2653327531 >> v2:172.21.15.113:3300/0 conn(0x7f2b70159e30 0x7f2b700af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b6e7fc700 1 -- 172.21.15.7:0/2653327531 >> v2:172.21.15.113:3300/0 conn(0x7f2b70159e30 msgr2=0x7f2b700af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b6e7fc700 1 --2- 172.21.15.7:0/2653327531 >> v2:172.21.15.113:3300/0 conn(0x7f2b70159e30 0x7f2b700af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b6e7fc700 1 -- 172.21.15.7:0/2653327531 >> v1:172.21.15.113:6789/0 conn(0x7f2b7006e170 legacy=0x7f2b7015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.753+0000 7f2b6e7fc700 1 -- 172.21.15.7:0/2653327531 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7015ba30 con 0x7f2b700d16e0 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b6e7fc700 1 --2- 172.21.15.7:0/2653327531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b7006dc30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2b70068c10 tx=0x7f2b64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996f406bc8224c26 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b6dffb700 1 -- 172.21.15.7:0/2653327531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64009e70 con 0x7f2b700d16e0 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b6dffb700 1 -- 172.21.15.7:0/2653327531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6400b860 con 0x7f2b700d16e0 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b6dffb700 1 -- 172.21.15.7:0/2653327531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400bc00 con 0x7f2b700d16e0 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b756bf700 1 -- 172.21.15.7:0/2653327531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 msgr2=0x7f2b7006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b756bf700 1 --2- 172.21.15.7:0/2653327531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b7006dc30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2b70068c10 tx=0x7f2b64009250 comp rx=0 tx=0).stop 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b756bf700 1 -- 172.21.15.7:0/2653327531 shutdown_connections 2024-01-11T07:13:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b756bf700 1 --2- 172.21.15.7:0/2653327531 >> v2:172.21.15.113:3300/0 conn(0x7f2b70159e30 0x7f2b700af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b756bf700 1 --2- 172.21.15.7:0/2653327531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b7006dc30 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b756bf700 1 -- 172.21.15.7:0/2653327531 >> 172.21.15.7:0/2653327531 conn(0x7f2b70063d00 msgr2=0x7f2b700d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b756bf700 1 -- 172.21.15.7:0/2653327531 shutdown_connections 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.754+0000 7f2b756bf700 1 -- 172.21.15.7:0/2653327531 wait complete. 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.756+0000 7f2b756bf700 1 Processor -- start 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.756+0000 7f2b756bf700 1 -- start start 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b756bf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7006e170 0x7f2b70155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b756bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b700ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b756bf700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b70159e30 0x7f2b700b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b756bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b700c9ed0 con 0x7f2b700d16e0 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b756bf700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b700c7cd0 con 0x7f2b70159e30 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b756bf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b701561c0 con 0x7f2b7006e170 2024-01-11T07:13:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b700ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b700ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41878/0 (socket says 172.21.15.7:41878) 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 -- 172.21.15.7:0/3995440271 learned_addr learned my addr 172.21.15.7:0/3995440271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6f7fe700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b70159e30 0x7f2b700b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6effd700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7006e170 0x7f2b70155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 -- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7006e170 msgr2=0x7f2b70155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7006e170 0x7f2b70155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 -- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b70159e30 msgr2=0x7f2b700b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b70159e30 0x7f2b700b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 -- 172.21.15.7:0/3995440271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b640094f0 con 0x7f2b700d16e0 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b6e7fc700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b700ca3e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2b64006fd0 tx=0x7f2b64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b56ffd700 1 -- 172.21.15.7:0/3995440271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400bbe0 con 0x7f2b700d16e0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b56ffd700 1 -- 172.21.15.7:0/3995440271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b64011420 con 0x7f2b700d16e0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b700b46d0 con 0x7f2b700d16e0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.757+0000 7f2b56ffd700 1 -- 172.21.15.7:0/3995440271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b640117c0 con 0x7f2b700d16e0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.758+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b700c8200 con 0x7f2b700d16e0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.758+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b700b43e0 con 0x7f2b700d16e0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.759+0000 7f2b56ffd700 1 -- 172.21.15.7:0/3995440271 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2b64011bf0 con 0x7f2b700d16e0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.759+0000 7f2b56ffd700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b5c066f30 0x7f2b5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.760+0000 7f2b56ffd700 1 -- 172.21.15.7:0/3995440271 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b64085980 con 0x7f2b700d16e0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.760+0000 7f2b6effd700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b5c066f30 0x7f2b5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.760+0000 7f2b6effd700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b5c066f30 0x7f2b5c0693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009db0 tx=0x7f2b600091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:02.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.764+0000 7f2b56ffd700 1 -- 172.21.15.7:0/3995440271 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b64052c70 con 0x7f2b700d16e0 2024-01-11T07:13:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.968+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b700c8420 con 0x7f2b700d16e0 2024-01-11T07:13:02.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.968+0000 7f2b56ffd700 1 -- 172.21.15.7:0/3995440271 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b64056290 con 0x7f2b700d16e0 2024-01-11T07:13:02.968 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:13:02.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b5c066f30 msgr2=0x7f2b5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b5c066f30 0x7f2b5c0693e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009db0 tx=0x7f2b600091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 msgr2=0x7f2b700ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b700ca3e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2b64006fd0 tx=0x7f2b64017000 comp rx=0 tx=0).stop 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 shutdown_connections 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2b5c066f30 0x7f2b5c0693e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b70159e30 0x7f2b700b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b7006e170 0x7f2b70155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 --2- 172.21.15.7:0/3995440271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b700d16e0 0x7f2b700ca3e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 >> 172.21.15.7:0/3995440271 conn(0x7f2b70063d00 msgr2=0x7f2b700c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 shutdown_connections 2024-01-11T07:13:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:02.971+0000 7f2b756bf700 1 -- 172.21.15.7:0/3995440271 wait complete. 2024-01-11T07:13:02.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:13:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.119+0000 7f5ed5e41700 1 Processor -- start 2024-01-11T07:13:03.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.119+0000 7f5ed5e41700 1 -- start start 2024-01-11T07:13:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ed5e41700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ed0159e30 0x7f5ed00af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ed5e41700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed00d16e0 0x7f5ed006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ed5e41700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed015c080 con 0x7f5ed006e170 2024-01-11T07:13:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ed5e41700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed006e5a0 con 0x7f5ed0159e30 2024-01-11T07:13:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ed5e41700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00b00f0 con 0x7f5ed00d16e0 2024-01-11T07:13:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ecf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ed0159e30 0x7f5ed00af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ecffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5ed006e170 0x7f5ed015a200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39762/0 (socket says 172.21.15.7:39762) 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ecffff700 1 -- 172.21.15.7:0/2957184681 learned_addr learned my addr 172.21.15.7:0/2957184681 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5eceffd700 1 --2- 172.21.15.7:0/2957184681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed00d16e0 0x7f5ed006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ecf7fe700 1 -- 172.21.15.7:0/2957184681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed00d16e0 msgr2=0x7f5ed006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ecf7fe700 1 --2- 172.21.15.7:0/2957184681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed00d16e0 0x7f5ed006dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ecf7fe700 1 -- 172.21.15.7:0/2957184681 >> v1:172.21.15.7:6789/0 conn(0x7f5ed006e170 legacy=0x7f5ed015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ecf7fe700 1 -- 172.21.15.7:0/2957184681 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed015ba30 con 0x7f5ed0159e30 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ece7fc700 1 -- 172.21.15.7:0/2957184681 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090885906 0 0) 0x7f5ed015c080 con 0x7f5ed006e170 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ecf7fe700 1 --2- 172.21.15.7:0/2957184681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00af7a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098e0 tx=0x7f5ec00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd707ee59377d41e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.120+0000 7f5ece7fc700 1 -- 172.21.15.7:0/2957184681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0009e80 con 0x7f5ed0159e30 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ece7fc700 1 -- 172.21.15.7:0/2957184681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000b860 con 0x7f5ed0159e30 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ece7fc700 1 -- 172.21.15.7:0/2957184681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000bbe0 con 0x7f5ed0159e30 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ed5e41700 1 -- 172.21.15.7:0/2957184681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 msgr2=0x7f5ed00af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/2957184681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00af7a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098e0 tx=0x7f5ec00091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ed5e41700 1 -- 172.21.15.7:0/2957184681 shutdown_connections 2024-01-11T07:13:03.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/2957184681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00af7a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/2957184681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed00d16e0 0x7f5ed006dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ed5e41700 1 -- 172.21.15.7:0/2957184681 >> 172.21.15.7:0/2957184681 conn(0x7f5ed0063d00 msgr2=0x7f5ed00d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:03.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ed5e41700 1 -- 172.21.15.7:0/2957184681 shutdown_connections 2024-01-11T07:13:03.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.121+0000 7f5ed5e41700 1 -- 172.21.15.7:0/2957184681 wait complete. 2024-01-11T07:13:03.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.123+0000 7f5ed5e41700 1 Processor -- start 2024-01-11T07:13:03.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.123+0000 7f5ed5e41700 1 -- start start 2024-01-11T07:13:03.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.123+0000 7f5ed5e41700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed006e170 0x7f5ed014b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.123+0000 7f5ed5e41700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ed00d16e0 0x7f5ed00c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.123+0000 7f5ed5e41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.123+0000 7f5ed5e41700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00c9450 con 0x7f5ed0159e30 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.123+0000 7f5ed5e41700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00c8840 con 0x7f5ed00d16e0 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.123+0000 7f5ed5e41700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00c8b20 con 0x7f5ed006e170 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41890/0 (socket says 172.21.15.7:41890) 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 -- 172.21.15.7:0/736891247 learned_addr learned my addr 172.21.15.7:0/736891247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5eceffd700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ed00d16e0 0x7f5ed00c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecf7fe700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed006e170 0x7f5ed014b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 -- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed006e170 msgr2=0x7f5ed014b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed006e170 0x7f5ed014b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 -- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ed00d16e0 msgr2=0x7f5ed00c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ed00d16e0 0x7f5ed00c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 -- 172.21.15.7:0/736891247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009420 con 0x7f5ed0159e30 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecf7fe700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed006e170 0x7f5ed014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ecffff700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00c98e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800cbf0 tx=0x7f5ec80019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5eb77fe700 1 -- 172.21.15.7:0/736891247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800a910 con 0x7f5ed0159e30 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5eb77fe700 1 -- 172.21.15.7:0/736891247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec800aa70 con 0x7f5ed0159e30 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5eb77fe700 1 -- 172.21.15.7:0/736891247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800ae30 con 0x7f5ed0159e30 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed00c8e00 con 0x7f5ed0159e30 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.124+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed00ca990 con 0x7f5ed0159e30 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.125+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed004c3c0 con 0x7f5ed0159e30 2024-01-11T07:13:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.126+0000 7f5eb77fe700 1 -- 172.21.15.7:0/736891247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5ec8007300 con 0x7f5ed0159e30 2024-01-11T07:13:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.126+0000 7f5eb77fe700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ebc066f30 0x7f5ebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.126+0000 7f5eb77fe700 1 -- 172.21.15.7:0/736891247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ec8084860 con 0x7f5ed0159e30 2024-01-11T07:13:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.126+0000 7f5ecf7fe700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ebc066f30 0x7f5ebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.127+0000 7f5ecf7fe700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ebc066f30 0x7f5ebc0693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00091e0 tx=0x7f5ec0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.131+0000 7f5eb77fe700 1 -- 172.21.15.7:0/736891247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ec8051b50 con 0x7f5ed0159e30 2024-01-11T07:13:03.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.332+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ed0157070 con 0x7f5ed0159e30 2024-01-11T07:13:03.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.333+0000 7f5eb77fe700 1 -- 172.21.15.7:0/736891247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ec8055170 con 0x7f5ed0159e30 2024-01-11T07:13:03.333 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:13:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ebc066f30 msgr2=0x7f5ebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ebc066f30 0x7f5ebc0693e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00091e0 tx=0x7f5ec0017000 comp rx=0 tx=0).stop 2024-01-11T07:13:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 msgr2=0x7f5ed00c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00c98e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800cbf0 tx=0x7f5ec80019b0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 shutdown_connections 2024-01-11T07:13:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ebc066f30 0x7f5ebc0693e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ed0159e30 0x7f5ed00c98e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ed006e170 0x7f5ed014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 --2- 172.21.15.7:0/736891247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ed00d16e0 0x7f5ed00c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 >> 172.21.15.7:0/736891247 conn(0x7f5ed0063d00 msgr2=0x7f5ed00b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 shutdown_connections 2024-01-11T07:13:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.335+0000 7f5ed5e41700 1 -- 172.21.15.7:0/736891247 wait complete. 2024-01-11T07:13:03.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:13:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.483+0000 7f42bbc5e700 1 Processor -- start 2024-01-11T07:13:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.483+0000 7f42bbc5e700 1 -- start start 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.484+0000 7f42bbc5e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b40b0390 0x7f42b406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.484+0000 7f42bbc5e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42b415a2a0 0x7f42b406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.484+0000 7f42bbc5e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b40d0130 con 0x7f42b4159ed0 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.484+0000 7f42bbc5e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b40cc5d0 con 0x7f42b415a2a0 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.484+0000 7f42bbc5e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40cfae0 con 0x7f42b40b0390 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.484+0000 7f42ba1fb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42b415a2a0 0x7f42b406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.484+0000 7f42ba1fb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42b415a2a0 0x7f42b406e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41892/0 (socket says 172.21.15.7:41892) 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42ba1fb700 1 -- 172.21.15.7:0/4260748748 learned_addr learned my addr 172.21.15.7:0/4260748748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42b91f9700 1 --2- 172.21.15.7:0/4260748748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b40b0390 0x7f42b406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42ba1fb700 1 -- 172.21.15.7:0/4260748748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b40b0390 msgr2=0x7f42b406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42ba1fb700 1 --2- 172.21.15.7:0/4260748748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b40b0390 0x7f42b406dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42ba1fb700 1 -- 172.21.15.7:0/4260748748 >> v1:172.21.15.113:6789/0 conn(0x7f42b4159ed0 legacy=0x7f42b40cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42b89f8700 1 -- 172.21.15.7:0/4260748748 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249732097 0 0) 0x7f42b40d0130 con 0x7f42b4159ed0 2024-01-11T07:13:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42ba1fb700 1 -- 172.21.15.7:0/4260748748 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b40cfdc0 con 0x7f42b415a2a0 2024-01-11T07:13:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42ba1fb700 1 --2- 172.21.15.7:0/4260748748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b415a2a0 0x7f42b406e170 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009f20 tx=0x7f42ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6638609465b957e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42b89f8700 1 -- 172.21.15.7:0/4260748748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00e070 con 0x7f42b415a2a0 2024-01-11T07:13:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42b89f8700 1 -- 172.21.15.7:0/4260748748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac00b9d0 con 0x7f42b415a2a0 2024-01-11T07:13:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.485+0000 7f42b89f8700 1 -- 172.21.15.7:0/4260748748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0092e0 con 0x7f42b415a2a0 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.486+0000 7f42bbc5e700 1 -- 172.21.15.7:0/4260748748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b415a2a0 msgr2=0x7f42b406e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.486+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/4260748748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b415a2a0 0x7f42b406e170 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009f20 tx=0x7f42ac009840 comp rx=0 tx=0).stop 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.486+0000 7f42bbc5e700 1 -- 172.21.15.7:0/4260748748 shutdown_connections 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.486+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/4260748748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b415a2a0 0x7f42b406e170 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.486+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/4260748748 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b40b0390 0x7f42b406dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.486+0000 7f42bbc5e700 1 -- 172.21.15.7:0/4260748748 >> 172.21.15.7:0/4260748748 conn(0x7f42b4063d00 msgr2=0x7f42b40d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.486+0000 7f42bbc5e700 1 -- 172.21.15.7:0/4260748748 shutdown_connections 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.486+0000 7f42bbc5e700 1 -- 172.21.15.7:0/4260748748 wait complete. 2024-01-11T07:13:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.487+0000 7f42bbc5e700 1 Processor -- start 2024-01-11T07:13:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.487+0000 7f42bbc5e700 1 -- start start 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42bbc5e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b40b0390 0x7f42b415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42bbc5e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b4159ed0 0x7f42b4169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42bbc5e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42b415a2a0 0x7f42b416ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42bbc5e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b415d210 con 0x7f42b40b0390 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42bbc5e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b416a460 con 0x7f42b415a2a0 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42bbc5e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b416a740 con 0x7f42b4159ed0 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b99fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b40b0390 0x7f42b415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b99fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b40b0390 0x7f42b415ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41896/0 (socket says 172.21.15.7:41896) 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b99fa700 1 -- 172.21.15.7:0/982174356 learned_addr learned my addr 172.21.15.7:0/982174356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b91f9700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b4159ed0 0x7f42b4169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42ba1fb700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42b415a2a0 0x7f42b416ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b99fa700 1 -- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b4159ed0 msgr2=0x7f42b4169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b99fa700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b4159ed0 0x7f42b4169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b99fa700 1 -- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42b415a2a0 msgr2=0x7f42b416ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b99fa700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42b415a2a0 0x7f42b416ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.488+0000 7f42b99fa700 1 -- 172.21.15.7:0/982174356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac009ae0 con 0x7f42b40b0390 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.489+0000 7f42b99fa700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b40b0390 0x7f42b415ccd0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f42b000c7e0 tx=0x7f42b0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.489+0000 7f42a9ffb700 1 -- 172.21.15.7:0/982174356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0012700 con 0x7f42b40b0390 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.489+0000 7f42a9ffb700 1 -- 172.21.15.7:0/982174356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b0012880 con 0x7f42b40b0390 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.489+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b416aa20 con 0x7f42b40b0390 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.489+0000 7f42a9ffb700 1 -- 172.21.15.7:0/982174356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0012c40 con 0x7f42b40b0390 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.489+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b415a980 con 0x7f42b40b0390 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.490+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b415acf0 con 0x7f42b40b0390 2024-01-11T07:13:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.490+0000 7f42a9ffb700 1 -- 172.21.15.7:0/982174356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f42b001b400 con 0x7f42b40b0390 2024-01-11T07:13:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.491+0000 7f42a9ffb700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4298066f30 0x7f42980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.491+0000 7f42a9ffb700 1 -- 172.21.15.7:0/982174356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42b0084100 con 0x7f42b40b0390 2024-01-11T07:13:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.491+0000 7f42b91f9700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4298066f30 0x7f42980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.492+0000 7f42b91f9700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4298066f30 0x7f42980693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42b415bce0 tx=0x7f42a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.495+0000 7f42a9ffb700 1 -- 172.21.15.7:0/982174356 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42b0051370 con 0x7f42b40b0390 2024-01-11T07:13:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.697+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42b406f660 con 0x7f42b40b0390 2024-01-11T07:13:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.697+0000 7f42a9ffb700 1 -- 172.21.15.7:0/982174356 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42b0054990 con 0x7f42b40b0390 2024-01-11T07:13:03.697 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:13:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4298066f30 msgr2=0x7f42980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4298066f30 0x7f42980693e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42b415bce0 tx=0x7f42a4008040 comp rx=0 tx=0).stop 2024-01-11T07:13:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b40b0390 msgr2=0x7f42b415ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b40b0390 0x7f42b415ccd0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f42b000c7e0 tx=0x7f42b0007df0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 shutdown_connections 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4298066f30 0x7f42980693e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42b415a2a0 0x7f42b416ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b40b0390 0x7f42b415ccd0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.699+0000 7f42bbc5e700 1 --2- 172.21.15.7:0/982174356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f42b4159ed0 0x7f42b4169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.700+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 >> 172.21.15.7:0/982174356 conn(0x7f42b4063d00 msgr2=0x7f42b40b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.700+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 shutdown_connections 2024-01-11T07:13:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.700+0000 7f42bbc5e700 1 -- 172.21.15.7:0/982174356 wait complete. 2024-01-11T07:13:03.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:03.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.845+0000 7f1f19af6700 1 Processor -- start 2024-01-11T07:13:03.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.845+0000 7f1f19af6700 1 -- start start 2024-01-11T07:13:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f19af6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1415b850 0x7f1f14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f19af6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f1415bc20 0x7f1f1415c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f19af6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f1406e1b0 0x7f1f1406e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f19af6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f140c87a0 con 0x7f1f1415bc20 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f19af6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f14069280 con 0x7f1f1406e1b0 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f19af6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f14069560 con 0x7f1f1415b850 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f12ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f1415bc20 0x7f1f1415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f12ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f1415bc20 0x7f1f1415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41910/0 (socket says 172.21.15.7:41910) 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f12ffd700 1 -- 172.21.15.7:0/687702614 learned_addr learned my addr 172.21.15.7:0/687702614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f137fe700 1 --2- 172.21.15.7:0/687702614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1415b850 0x7f1f14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f13fff700 1 --2- 172.21.15.7:0/687702614 >> v2:172.21.15.113:3300/0 conn(0x7f1f1406e1b0 0x7f1f1406e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f12ffd700 1 -- 172.21.15.7:0/687702614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1415b850 msgr2=0x7f1f14068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f12ffd700 1 --2- 172.21.15.7:0/687702614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1415b850 0x7f1f14068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f12ffd700 1 -- 172.21.15.7:0/687702614 >> v2:172.21.15.113:3300/0 conn(0x7f1f1406e1b0 msgr2=0x7f1f1406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f12ffd700 1 --2- 172.21.15.7:0/687702614 >> v2:172.21.15.113:3300/0 conn(0x7f1f1406e1b0 0x7f1f1406e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.846+0000 7f1f12ffd700 1 -- 172.21.15.7:0/687702614 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14069840 con 0x7f1f1415bc20 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f12ffd700 1 --2- 172.21.15.7:0/687702614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f1415c090 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1f140641d0 tx=0x7f1f08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c139c6b66beeda server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f127fc700 1 -- 172.21.15.7:0/687702614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800b860 con 0x7f1f1415bc20 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f127fc700 1 -- 172.21.15.7:0/687702614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0800b9c0 con 0x7f1f1415bc20 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f127fc700 1 -- 172.21.15.7:0/687702614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800bd60 con 0x7f1f1415bc20 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 -- 172.21.15.7:0/687702614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 msgr2=0x7f1f1415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 --2- 172.21.15.7:0/687702614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f1415c090 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1f140641d0 tx=0x7f1f08009250 comp rx=0 tx=0).stop 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 -- 172.21.15.7:0/687702614 shutdown_connections 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 --2- 172.21.15.7:0/687702614 >> v2:172.21.15.113:3300/0 conn(0x7f1f1406e1b0 0x7f1f1406e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 --2- 172.21.15.7:0/687702614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1415b850 0x7f1f14068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 --2- 172.21.15.7:0/687702614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f1415c090 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 -- 172.21.15.7:0/687702614 >> 172.21.15.7:0/687702614 conn(0x7f1f14063230 msgr2=0x7f1f14063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 -- 172.21.15.7:0/687702614 shutdown_connections 2024-01-11T07:13:03.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.847+0000 7f1f19af6700 1 -- 172.21.15.7:0/687702614 wait complete. 2024-01-11T07:13:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.849+0000 7f1f19af6700 1 Processor -- start 2024-01-11T07:13:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.849+0000 7f1f19af6700 1 -- start start 2024-01-11T07:13:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f19af6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1406e1b0 0x7f1f14147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f19af6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415b850 0x7f1f140ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f19af6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f140b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f19af6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f14147860 con 0x7f1f1415bc20 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f19af6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140cb340 con 0x7f1f1415b850 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f19af6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140c5040 con 0x7f1f1406e1b0 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f140b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f140b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41916/0 (socket says 172.21.15.7:41916) 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1406e1b0 0x7f1f14147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 -- 172.21.15.7:0/3012849507 learned_addr learned my addr 172.21.15.7:0/3012849507 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f12ffd700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415b850 0x7f1f140ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 -- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1406e1b0 msgr2=0x7f1f14147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1406e1b0 0x7f1f14147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 -- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415b850 msgr2=0x7f1f140ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415b850 0x7f1f140ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 -- 172.21.15.7:0/3012849507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080094f0 con 0x7f1f1415bc20 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f12ffd700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415b850 0x7f1f140ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.850+0000 7f1f13fff700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f140b0cf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009fa0 tx=0x7f1f0c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.851+0000 7f1efb7fe700 1 -- 172.21.15.7:0/3012849507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00ebc0 con 0x7f1f1415bc20 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.851+0000 7f1efb7fe700 1 -- 172.21.15.7:0/3012849507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c00ed20 con 0x7f1f1415bc20 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.851+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f140cad80 con 0x7f1f1415bc20 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.851+0000 7f1efb7fe700 1 -- 172.21.15.7:0/3012849507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c014670 con 0x7f1f1415bc20 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.851+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f140c4cd0 con 0x7f1f1415bc20 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.852+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f140c4b30 con 0x7f1f1415bc20 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.852+0000 7f1efb7fe700 1 -- 172.21.15.7:0/3012849507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f0c0149e0 con 0x7f1f1415bc20 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.853+0000 7f1efb7fe700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.853+0000 7f1efb7fe700 1 -- 172.21.15.7:0/3012849507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f0c084c70 con 0x7f1f1415bc20 2024-01-11T07:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.853+0000 7f1f137fe700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.853+0000 7f1f137fe700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f04006fd0 tx=0x7f1f04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:03.857+0000 7f1efb7fe700 1 -- 172.21.15.7:0/3012849507 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f140c4b30 con 0x7f1f1415bc20 2024-01-11T07:13:04.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.106+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f140c4b30 con 0x7f1f1415bc20 2024-01-11T07:13:04.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.106+0000 7f1efb7fe700 1 -- 172.21.15.7:0/3012849507 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f1f140c4b30 con 0x7f1f1415bc20 2024-01-11T07:13:04.106 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:04.106 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:12:45.905827+0000","modified":"2024-01-11T07:12:49.492046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2493499246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2493499246},{"type":"v1","addr":"172.21.15.7:6835","nonce":2493499246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-11T07:13:04.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 msgr2=0x7f1f000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1f04006fd0 tx=0x7f1f04008040 comp rx=0 tx=0).stop 2024-01-11T07:13:04.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 msgr2=0x7f1f140b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f140b0cf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009fa0 tx=0x7f1f0c009900 comp rx=0 tx=0).stop 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 shutdown_connections 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1415bc20 0x7f1f140b0cf0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1406e1b0 0x7f1f14147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 --2- 172.21.15.7:0/3012849507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1415b850 0x7f1f140ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 >> 172.21.15.7:0/3012849507 conn(0x7f1f14063230 msgr2=0x7f1f1406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 shutdown_connections 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.108+0000 7f1f19af6700 1 -- 172.21.15.7:0/3012849507 wait complete. 2024-01-11T07:13:04.109 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-01-11T07:13:04.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.253+0000 7fd01ed71700 1 Processor -- start 2024-01-11T07:13:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.253+0000 7fd01ed71700 1 -- start start 2024-01-11T07:13:04.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.254+0000 7fd01ed71700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd018157de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.254+0000 7fd01ed71700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0180b2b70 0x7fd0180b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.254+0000 7fd01ed71700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0180b34f0 con 0x7fd0180acd10 2024-01-11T07:13:04.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.254+0000 7fd01ed71700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0180ac9e0 con 0x7fd0180b2b70 2024-01-11T07:13:04.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.254+0000 7fd01ed71700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01815c000 con 0x7fd018157a10 2024-01-11T07:13:04.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.254+0000 7fd01d30e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0180b2b70 0x7fd0180b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.254+0000 7fd017fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0180acd10 0x7fd01815b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39770/0 (socket says 172.21.15.7:39770) 2024-01-11T07:13:04.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.254+0000 7fd017fff700 1 -- 172.21.15.7:0/3913958889 learned_addr learned my addr 172.21.15.7:0/3913958889 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd01cb0d700 1 --2- 172.21.15.7:0/3913958889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd018157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd01d30e700 1 -- 172.21.15.7:0/3913958889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 msgr2=0x7fd018157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd01d30e700 1 --2- 172.21.15.7:0/3913958889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd018157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd01d30e700 1 -- 172.21.15.7:0/3913958889 >> v1:172.21.15.7:6789/0 conn(0x7fd0180acd10 legacy=0x7fd01815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd01d30e700 1 -- 172.21.15.7:0/3913958889 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01806df50 con 0x7fd0180b2b70 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd0177fe700 1 -- 172.21.15.7:0/3913958889 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028532530 0 0) 0x7fd0180b34f0 con 0x7fd0180acd10 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd01cb0d700 1 --2- 172.21.15.7:0/3913958889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd018157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd01d30e700 1 --2- 172.21.15.7:0/3913958889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180b2fb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd010009e60 tx=0x7fd010009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba850cc680021164 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd0177fe700 1 -- 172.21.15.7:0/3913958889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000e070 con 0x7fd0180b2b70 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd0177fe700 1 -- 172.21.15.7:0/3913958889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01000bc30 con 0x7fd0180b2b70 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.255+0000 7fd0177fe700 1 -- 172.21.15.7:0/3913958889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010012430 con 0x7fd0180b2b70 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.256+0000 7fd01ed71700 1 -- 172.21.15.7:0/3913958889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 msgr2=0x7fd0180b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.256+0000 7fd01ed71700 1 --2- 172.21.15.7:0/3913958889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180b2fb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd010009e60 tx=0x7fd010009780 comp rx=0 tx=0).stop 2024-01-11T07:13:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.256+0000 7fd01ed71700 1 -- 172.21.15.7:0/3913958889 shutdown_connections 2024-01-11T07:13:04.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.256+0000 7fd01ed71700 1 --2- 172.21.15.7:0/3913958889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180b2fb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.256+0000 7fd01ed71700 1 --2- 172.21.15.7:0/3913958889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd018157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.256+0000 7fd01ed71700 1 -- 172.21.15.7:0/3913958889 >> 172.21.15.7:0/3913958889 conn(0x7fd0180cd910 msgr2=0x7fd0180cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:04.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.256+0000 7fd01ed71700 1 -- 172.21.15.7:0/3913958889 shutdown_connections 2024-01-11T07:13:04.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.256+0000 7fd01ed71700 1 -- 172.21.15.7:0/3913958889 wait complete. 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.257+0000 7fd01ed71700 1 Processor -- start 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.257+0000 7fd01ed71700 1 -- start start 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd01ed71700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0180acd10 0x7fd01815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd01ed71700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd01ed71700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd0180adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd01ed71700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01814fb30 con 0x7fd0180b2b70 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd01ed71700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01815acd0 con 0x7fd0180acd10 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd01ed71700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01815a0c0 con 0x7fd018157a10 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd017fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd017fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41930/0 (socket says 172.21.15.7:41930) 2024-01-11T07:13:04.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd017fff700 1 -- 172.21.15.7:0/673655797 learned_addr learned my addr 172.21.15.7:0/673655797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd01d30e700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd0180adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.258+0000 7fd01cb0d700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0180acd10 0x7fd01815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd017fff700 1 -- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 msgr2=0x7fd0180adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd017fff700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd0180adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd017fff700 1 -- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0180acd10 msgr2=0x7fd01815b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd017fff700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0180acd10 0x7fd01815b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd017fff700 1 -- 172.21.15.7:0/673655797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd010009a20 con 0x7fd0180b2b70 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd017fff700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180cc4b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c7e0 tx=0x7fd004007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd014ff9700 1 -- 172.21.15.7:0/673655797 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004005b30 con 0x7fd0180b2b70 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd014ff9700 1 -- 172.21.15.7:0/673655797 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd004005c90 con 0x7fd0180b2b70 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd014ff9700 1 -- 172.21.15.7:0/673655797 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004007300 con 0x7fd0180b2b70 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01815a3a0 con 0x7fd0180b2b70 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.259+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01815a960 con 0x7fd0180b2b70 2024-01-11T07:13:04.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.260+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd018064a40 con 0x7fd0180b2b70 2024-01-11T07:13:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.261+0000 7fd014ff9700 1 -- 172.21.15.7:0/673655797 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd00401b3b0 con 0x7fd0180b2b70 2024-01-11T07:13:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.261+0000 7fd014ff9700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcffc066f30 0x7fcffc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.261+0000 7fd014ff9700 1 -- 172.21.15.7:0/673655797 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd004083af0 con 0x7fd0180b2b70 2024-01-11T07:13:04.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.262+0000 7fd01cb0d700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcffc066f30 0x7fcffc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.262+0000 7fd01cb0d700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcffc066f30 0x7fcffc0693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006f00 tx=0x7fd00c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:04.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.266+0000 7fd014ff9700 1 -- 172.21.15.7:0/673655797 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd004050cc0 con 0x7fd0180b2b70 2024-01-11T07:13:04.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.468+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd01815ae40 con 0x7fd0180b2b70 2024-01-11T07:13:04.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.469+0000 7fd014ff9700 1 -- 172.21.15.7:0/673655797 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+15324 (secure 0 0 0) 0x7fd004017020 con 0x7fd0180b2b70 2024-01-11T07:13:04.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:04.469 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":123,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:59.596060+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:12:43.805689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-11T07:12:44.880093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcffc066f30 msgr2=0x7fcffc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcffc066f30 0x7fcffc0693e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006f00 tx=0x7fd00c008040 comp rx=0 tx=0).stop 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 msgr2=0x7fd0180cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180cc4b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c7e0 tx=0x7fd004007df0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 shutdown_connections 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcffc066f30 0x7fcffc0693e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd018157a10 0x7fd0180adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0180acd10 0x7fd01815b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 --2- 172.21.15.7:0/673655797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0180b2b70 0x7fd0180cc4b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.471+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 >> 172.21.15.7:0/673655797 conn(0x7fd0180cd910 msgr2=0x7fd0180cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.472+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 shutdown_connections 2024-01-11T07:13:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.472+0000 7fd01ed71700 1 -- 172.21.15.7:0/673655797 wait complete. 2024-01-11T07:13:04.485 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:13:04.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:04.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.617+0000 7f55c8220700 1 Processor -- start 2024-01-11T07:13:04.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.617+0000 7f55c8220700 1 -- start start 2024-01-11T07:13:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c8220700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 0x7f55c0162770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c8220700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c0159300 con 0x7f55c00cafb0 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c8220700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c00cf000 con 0x7f55c0158280 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c8220700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00cf260 con 0x7f55c00b0b40 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c5fbc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f55c00cafb0 0x7f55c00ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39784/0 (socket says 172.21.15.7:39784) 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c5fbc700 1 -- 172.21.15.7:0/3047283122 learned_addr learned my addr 172.21.15.7:0/3047283122 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c67bd700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 0x7f55c0162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183857529 0 0) 0x7f55c0159300 con 0x7f55c00cafb0 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.618+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55ac003570 con 0x7f55c00cafb0 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147932709 0 0) 0x7f55c00cf000 con 0x7f55c0158280 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c0159300 con 0x7f55c0158280 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 370994878 0 0) 0x7f55ac003570 con 0x7f55c00cafb0 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 msgr2=0x7f55c0162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 0x7f55c0162770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 >> v1:172.21.15.113:6789/0 conn(0x7f55c0158280 legacy=0x7f55c00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00cf540 con 0x7f55c00cafb0 2024-01-11T07:13:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f55b40013f0 con 0x7f55c00cafb0 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c67bd700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 0x7f55c0162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 >> v1:172.21.15.7:6789/0 conn(0x7f55c00cafb0 legacy=0x7f55c00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac003c20 0x7f55ac004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 0x7f55ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55ac004d50 0x7f55ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00cf260 con 0x7f55ac004d50 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac00bb70 con 0x7f55ac003c20 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac00bd30 con 0x7f55c00b0b40 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c67bd700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55ac004d50 0x7f55ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c5fbc700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac003c20 0x7f55ac004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c57bb700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 0x7f55ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c67bd700 1 -- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 msgr2=0x7f55ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c67bd700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c00b0b40 0x7f55ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c67bd700 1 -- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac003c20 msgr2=0x7f55ac004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c67bd700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac003c20 0x7f55ac004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.619+0000 7f55c67bd700 1 -- 172.21.15.7:0/3047283122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009a20 con 0x7f55ac004d50 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c67bd700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55ac004d50 0x7f55ac0051c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f55b800e3c0 tx=0x7f55b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c893527d5f26cf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8009c50 con 0x7f55ac004d50 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b800f040 con 0x7f55ac004d50 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c4fba700 1 -- 172.21.15.7:0/3047283122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800eea0 con 0x7f55ac004d50 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c8220700 1 -- 172.21.15.7:0/3047283122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55ac004d50 msgr2=0x7f55ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c8220700 1 --2- 172.21.15.7:0/3047283122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55ac004d50 0x7f55ac0051c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f55b800e3c0 tx=0x7f55b800b040 comp rx=0 tx=0).stop 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c67bd700 1 -- 172.21.15.7:0/3047283122 reap_dead start 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c8220700 1 -- 172.21.15.7:0/3047283122 shutdown_connections 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.620+0000 7f55c8220700 1 -- 172.21.15.7:0/3047283122 >> 172.21.15.7:0/3047283122 conn(0x7f55c0071100 msgr2=0x7f55c006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.621+0000 7f55c8220700 1 -- 172.21.15.7:0/3047283122 shutdown_connections 2024-01-11T07:13:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.621+0000 7f55c8220700 1 -- 172.21.15.7:0/3047283122 wait complete. 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.622+0000 7f55c8220700 1 Processor -- start 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.622+0000 7f55c8220700 1 -- start start 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c8220700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac004d50 0x7f55c0151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c8220700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55c015e170 0x7f55c015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c8220700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c015e5e0 0x7f55c015c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c8220700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00b3760 con 0x7f55c015e170 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c8220700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01529a0 con 0x7f55ac004d50 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c8220700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0151f50 con 0x7f55c015e5e0 2024-01-11T07:13:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55c015e170 0x7f55c015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55c015e170 0x7f55c015c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41950/0 (socket says 172.21.15.7:41950) 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 -- 172.21.15.7:0/1417834456 learned_addr learned my addr 172.21.15.7:0/1417834456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c5fbc700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac004d50 0x7f55c0151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c67bd700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c015e5e0 0x7f55c015c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 -- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c015e5e0 msgr2=0x7f55c015c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c015e5e0 0x7f55c015c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 -- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac004d50 msgr2=0x7f55c0151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac004d50 0x7f55c0151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 -- 172.21.15.7:0/1417834456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c015b980 con 0x7f55c015e170 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55c57bb700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55c015e170 0x7f55c015c080 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f55bc002b20 tx=0x7f55bc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.623+0000 7f55b1ffb700 1 -- 172.21.15.7:0/1417834456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00f8d0 con 0x7f55c015e170 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.624+0000 7f55b1ffb700 1 -- 172.21.15.7:0/1417834456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc00fa30 con 0x7f55c015e170 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.624+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c00b31a0 con 0x7f55c015e170 2024-01-11T07:13:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.624+0000 7f55b1ffb700 1 -- 172.21.15.7:0/1417834456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00fdf0 con 0x7f55c015e170 2024-01-11T07:13:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.624+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c0152630 con 0x7f55c015e170 2024-01-11T07:13:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.625+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c0152490 con 0x7f55c015e170 2024-01-11T07:13:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.625+0000 7f55b1ffb700 1 -- 172.21.15.7:0/1417834456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f55bc013600 con 0x7f55c015e170 2024-01-11T07:13:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.626+0000 7f55b1ffb700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55a4066f30 0x7f55a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.626+0000 7f55b1ffb700 1 -- 172.21.15.7:0/1417834456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55bc084bf0 con 0x7f55c015e170 2024-01-11T07:13:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.626+0000 7f55c5fbc700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55a4066f30 0x7f55a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.627+0000 7f55c5fbc700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55a4066f30 0x7f55a40693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f55b4001b50 tx=0x7f55b4001cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:04.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.630+0000 7f55b1ffb700 1 -- 172.21.15.7:0/1417834456 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55c0152490 con 0x7f55c015e170 2024-01-11T07:13:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.882+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55c0152490 con 0x7f55c015e170 2024-01-11T07:13:04.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.883+0000 7f55b1ffb700 1 -- 172.21.15.7:0/1417834456 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f55c0152490 con 0x7f55c015e170 2024-01-11T07:13:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:04.883 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:12:45.905827+0000","modified":"2024-01-11T07:12:49.492046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2493499246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2493499246},{"type":"v1","addr":"172.21.15.7:6835","nonce":2493499246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55a4066f30 msgr2=0x7f55a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55a4066f30 0x7f55a40693e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f55b4001b50 tx=0x7f55b4001cc0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55c015e170 msgr2=0x7f55c015c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55c015e170 0x7f55c015c080 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f55bc002b20 tx=0x7f55bc007d30 comp rx=0 tx=0).stop 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 shutdown_connections 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55a4066f30 0x7f55a40693e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c015e5e0 0x7f55c015c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55ac004d50 0x7f55c0151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 --2- 172.21.15.7:0/1417834456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55c015e170 0x7f55c015c080 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 >> 172.21.15.7:0/1417834456 conn(0x7f55c0071100 msgr2=0x7f55c0157860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 shutdown_connections 2024-01-11T07:13:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:04.886+0000 7f55c8220700 1 -- 172.21.15.7:0/1417834456 wait complete. 2024-01-11T07:13:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-01-11T07:13:04.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.030+0000 7f068a009700 1 Processor -- start 2024-01-11T07:13:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.031+0000 7f068a009700 1 -- start start 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.031+0000 7f068a009700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 0x7f06840cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.031+0000 7f068a009700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06840ce220 0x7f06840ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.031+0000 7f068a009700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068406d270 con 0x7f06841579d0 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.031+0000 7f068a009700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06840ce6a0 con 0x7f06840ce220 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.031+0000 7f068a009700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06840cda40 con 0x7f0684157da0 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0683fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06840ce220 0x7f06840ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0683fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06840ce220 0x7f06840ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41960/0 (socket says 172.21.15.7:41960) 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0683fff700 1 -- 172.21.15.7:0/2706991182 learned_addr learned my addr 172.21.15.7:0/2706991182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0682ffd700 1 --2- 172.21.15.7:0/2706991182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 0x7f06840cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0683fff700 1 -- 172.21.15.7:0/2706991182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 msgr2=0x7f06840cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:05.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0683fff700 1 --2- 172.21.15.7:0/2706991182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 0x7f06840cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0683fff700 1 -- 172.21.15.7:0/2706991182 >> v1:172.21.15.113:6789/0 conn(0x7f06841579d0 legacy=0x7f068415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0683fff700 1 -- 172.21.15.7:0/2706991182 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840cd5b0 con 0x7f06840ce220 2024-01-11T07:13:05.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f0683fff700 1 --2- 172.21.15.7:0/2706991182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840ca500 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f067c009f20 tx=0x7f067c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cb3717e9efdf01 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f06827fc700 1 -- 172.21.15.7:0/2706991182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c00e070 con 0x7f06840ce220 2024-01-11T07:13:05.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f06827fc700 1 -- 172.21.15.7:0/2706991182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c00ba80 con 0x7f06840ce220 2024-01-11T07:13:05.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.032+0000 7f06827fc700 1 -- 172.21.15.7:0/2706991182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0092e0 con 0x7f06840ce220 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.033+0000 7f068a009700 1 -- 172.21.15.7:0/2706991182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 msgr2=0x7f06840ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.033+0000 7f068a009700 1 --2- 172.21.15.7:0/2706991182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840ca500 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f067c009f20 tx=0x7f067c009840 comp rx=0 tx=0).stop 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.033+0000 7f068a009700 1 -- 172.21.15.7:0/2706991182 shutdown_connections 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.033+0000 7f068a009700 1 --2- 172.21.15.7:0/2706991182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840ca500 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.033+0000 7f068a009700 1 --2- 172.21.15.7:0/2706991182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 0x7f06840cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.033+0000 7f068a009700 1 -- 172.21.15.7:0/2706991182 >> 172.21.15.7:0/2706991182 conn(0x7f068406ae00 msgr2=0x7f068406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.033+0000 7f068a009700 1 -- 172.21.15.7:0/2706991182 shutdown_connections 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.033+0000 7f068a009700 1 -- 172.21.15.7:0/2706991182 wait complete. 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.034+0000 7f068a009700 1 Processor -- start 2024-01-11T07:13:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.034+0000 7f068a009700 1 -- start start 2024-01-11T07:13:05.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f068a009700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f068a009700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06841579d0 0x7f06840b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f068a009700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 0x7f06841562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f068a009700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068416a420 con 0x7f06840ce220 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f068a009700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068416a700 con 0x7f06841579d0 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f068a009700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068416a9e0 con 0x7f0684157da0 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f06837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f06837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41972/0 (socket says 172.21.15.7:41972) 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.035+0000 7f06837fe700 1 -- 172.21.15.7:0/4123477846 learned_addr learned my addr 172.21.15.7:0/4123477846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f0683fff700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 0x7f06841562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f0682ffd700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06841579d0 0x7f06840b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f06837fe700 1 -- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 msgr2=0x7f06841562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f06837fe700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 0x7f06841562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f06837fe700 1 -- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06841579d0 msgr2=0x7f06840b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f06837fe700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06841579d0 0x7f06840b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f06837fe700 1 -- 172.21.15.7:0/4123477846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c009ae0 con 0x7f06840ce220 2024-01-11T07:13:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f0682ffd700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06841579d0 0x7f06840b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f06837fe700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840afcc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f067400c590 tx=0x7f067400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f066b7fe700 1 -- 172.21.15.7:0/4123477846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674011590 con 0x7f06840ce220 2024-01-11T07:13:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f066b7fe700 1 -- 172.21.15.7:0/4123477846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0674011710 con 0x7f06840ce220 2024-01-11T07:13:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f066b7fe700 1 -- 172.21.15.7:0/4123477846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674011ad0 con 0x7f06840ce220 2024-01-11T07:13:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068416acc0 con 0x7f06840ce220 2024-01-11T07:13:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.036+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0684156810 con 0x7f06840ce220 2024-01-11T07:13:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.037+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068404c3c0 con 0x7f06840ce220 2024-01-11T07:13:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.038+0000 7f066b7fe700 1 -- 172.21.15.7:0/4123477846 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f067401b3b0 con 0x7f06840ce220 2024-01-11T07:13:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.038+0000 7f066b7fe700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0670066f00 0x7f06700693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.038+0000 7f066b7fe700 1 -- 172.21.15.7:0/4123477846 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f06740839f0 con 0x7f06840ce220 2024-01-11T07:13:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.038+0000 7f0682ffd700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0670066f00 0x7f06700693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.039+0000 7f0682ffd700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0670066f00 0x7f06700693b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f06840b0740 tx=0x7f0678008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.042+0000 7f066b7fe700 1 -- 172.21.15.7:0/4123477846 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0674050c60 con 0x7f06840ce220 2024-01-11T07:13:05.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.292+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f068416a560 con 0x7f06840ce220 2024-01-11T07:13:05.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.293+0000 7f066b7fe700 1 -- 172.21.15.7:0/4123477846 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f068416a560 con 0x7f06840ce220 2024-01-11T07:13:05.293 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:05.293 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:12:45.905827+0000","modified":"2024-01-11T07:12:49.492046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2493499246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2493499246},{"type":"v1","addr":"172.21.15.7:6835","nonce":2493499246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2024-01-11T07:13:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0670066f00 msgr2=0x7f06700693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0670066f00 0x7f06700693b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f06840b0740 tx=0x7f0678008040 comp rx=0 tx=0).stop 2024-01-11T07:13:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 msgr2=0x7f06840afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840afcc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f067400c590 tx=0x7f067400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 shutdown_connections 2024-01-11T07:13:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0670066f00 0x7f06700693b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0684157da0 0x7f06841562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06840ce220 0x7f06840afcc0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 --2- 172.21.15.7:0/4123477846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06841579d0 0x7f06840b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 >> 172.21.15.7:0/4123477846 conn(0x7f068406ae00 msgr2=0x7f06841510d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:05.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 shutdown_connections 2024-01-11T07:13:05.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.295+0000 7f068a009700 1 -- 172.21.15.7:0/4123477846 wait complete. 2024-01-11T07:13:05.296 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2024-01-11T07:13:05.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:05.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.440+0000 7f8d3438c700 1 Processor -- start 2024-01-11T07:13:05.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.441+0000 7f8d3438c700 1 -- start start 2024-01-11T07:13:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d3438c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c154740 0x7f8d2c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d3438c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0b1c10 con 0x7f8d2c157ad0 2024-01-11T07:13:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d3438c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0b1ef0 con 0x7f8d2c154370 2024-01-11T07:13:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d3438c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15a7d0 con 0x7f8d2c154740 2024-01-11T07:13:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d32929700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8d2c157ad0 0x7f8d2c069c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39786/0 (socket says 172.21.15.7:39786) 2024-01-11T07:13:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d32929700 1 -- 172.21.15.7:0/1384939649 learned_addr learned my addr 172.21.15.7:0/1384939649 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d31927700 1 --2- 172.21.15.7:0/1384939649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c154740 0x7f8d2c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d31126700 1 -- 172.21.15.7:0/1384939649 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427150460 0 0) 0x7f8d2c0b1c10 con 0x7f8d2c157ad0 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d31126700 1 -- 172.21.15.7:0/1384939649 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d18003570 con 0x7f8d2c157ad0 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d31126700 1 -- 172.21.15.7:0/1384939649 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868873489 0 0) 0x7f8d2c0b1ef0 con 0x7f8d2c154370 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d31126700 1 -- 172.21.15.7:0/1384939649 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d2c0b1c10 con 0x7f8d2c154370 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d31927700 1 -- 172.21.15.7:0/1384939649 >> v1:172.21.15.113:6789/0 conn(0x7f8d2c154370 legacy=0x7f8d2c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d31927700 1 -- 172.21.15.7:0/1384939649 >> v1:172.21.15.7:6789/0 conn(0x7f8d2c157ad0 legacy=0x7f8d2c069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.442+0000 7f8d31927700 1 -- 172.21.15.7:0/1384939649 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c15aab0 con 0x7f8d2c154740 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d31927700 1 --2- 172.21.15.7:0/1384939649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c154740 0x7f8d2c0b1530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009910 tx=0x7f8d1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e6e16e4e197215 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d31126700 1 -- 172.21.15.7:0/1384939649 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00f6b0 con 0x7f8d2c154740 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d31126700 1 -- 172.21.15.7:0/1384939649 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c00f810 con 0x7f8d2c154740 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d31126700 1 -- 172.21.15.7:0/1384939649 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00fbb0 con 0x7f8d2c154740 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d3438c700 1 -- 172.21.15.7:0/1384939649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c154740 msgr2=0x7f8d2c0b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d3438c700 1 --2- 172.21.15.7:0/1384939649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c154740 0x7f8d2c0b1530 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009910 tx=0x7f8d1c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d3438c700 1 -- 172.21.15.7:0/1384939649 shutdown_connections 2024-01-11T07:13:05.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d3438c700 1 --2- 172.21.15.7:0/1384939649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c154740 0x7f8d2c0b1530 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.443+0000 7f8d3438c700 1 -- 172.21.15.7:0/1384939649 >> 172.21.15.7:0/1384939649 conn(0x7f8d2c0cd910 msgr2=0x7f8d2c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:05.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.444+0000 7f8d3438c700 1 -- 172.21.15.7:0/1384939649 shutdown_connections 2024-01-11T07:13:05.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.444+0000 7f8d3438c700 1 -- 172.21.15.7:0/1384939649 wait complete. 2024-01-11T07:13:05.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.445+0000 7f8d3438c700 1 Processor -- start 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.445+0000 7f8d3438c700 1 -- start start 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.445+0000 7f8d3438c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d2c154370 0x7f8d2c14cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.445+0000 7f8d3438c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c154740 0x7f8d2c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.445+0000 7f8d3438c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c157ad0 0x7f8d2c0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.445+0000 7f8d3438c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c8070 con 0x7f8d2c154740 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.445+0000 7f8d3438c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c8350 con 0x7f8d2c154370 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.445+0000 7f8d3438c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c8630 con 0x7f8d2c157ad0 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c154740 0x7f8d2c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c154740 0x7f8d2c0c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41988/0 (socket says 172.21.15.7:41988) 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 -- 172.21.15.7:0/3036722208 learned_addr learned my addr 172.21.15.7:0/3036722208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d32128700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d2c154370 0x7f8d2c14cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d32929700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c157ad0 0x7f8d2c0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 -- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c157ad0 msgr2=0x7f8d2c0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c157ad0 0x7f8d2c0cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 -- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d2c154370 msgr2=0x7f8d2c14cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d2c154370 0x7f8d2c14cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 -- 172.21.15.7:0/3036722208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c009420 con 0x7f8d2c154740 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d32929700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c157ad0 0x7f8d2c0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d31927700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c154740 0x7f8d2c0c9110 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0093f0 tx=0x7f8d1c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d227fc700 1 -- 172.21.15.7:0/3036722208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c000bc0 con 0x7f8d2c154740 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d227fc700 1 -- 172.21.15.7:0/3036722208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c00f980 con 0x7f8d2c154740 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c0c8910 con 0x7f8d2c154740 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d227fc700 1 -- 172.21.15.7:0/3036722208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c010470 con 0x7f8d2c154740 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.446+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c0caf30 con 0x7f8d2c154740 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.447+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c0c8db0 con 0x7f8d2c154740 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.448+0000 7f8d227fc700 1 -- 172.21.15.7:0/3036722208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8d1c022070 con 0x7f8d2c154740 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.448+0000 7f8d227fc700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d10066f30 0x7f8d100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.448+0000 7f8d227fc700 1 -- 172.21.15.7:0/3036722208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d1c014070 con 0x7f8d2c154740 2024-01-11T07:13:05.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.449+0000 7f8d32128700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d10066f30 0x7f8d100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.449+0000 7f8d32128700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d10066f30 0x7f8d100693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8d28007fd0 tx=0x7f8d28007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.453+0000 7f8d227fc700 1 -- 172.21.15.7:0/3036722208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d1c052820 con 0x7f8d2c154740 2024-01-11T07:13:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.657+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d2c04c3c0 con 0x7f8d2c154740 2024-01-11T07:13:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.658+0000 7f8d227fc700 1 -- 172.21.15.7:0/3036722208 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f8d1c029090 con 0x7f8d2c154740 2024-01-11T07:13:05.658 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:05.658 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":123,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:12:59.596060+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:12:43.805689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-11T07:12:44.880093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d10066f30 msgr2=0x7f8d100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d10066f30 0x7f8d100693e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8d28007fd0 tx=0x7f8d28007d50 comp rx=0 tx=0).stop 2024-01-11T07:13:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c154740 msgr2=0x7f8d2c0c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c154740 0x7f8d2c0c9110 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0093f0 tx=0x7f8d1c019000 comp rx=0 tx=0).stop 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 shutdown_connections 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8d10066f30 0x7f8d100693e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d2c157ad0 0x7f8d2c0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d2c154370 0x7f8d2c14cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 --2- 172.21.15.7:0/3036722208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d2c154740 0x7f8d2c0c9110 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 >> 172.21.15.7:0/3036722208 conn(0x7f8d2c0cd910 msgr2=0x7f8d2c06e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 shutdown_connections 2024-01-11T07:13:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.660+0000 7f8d3438c700 1 -- 172.21.15.7:0/3036722208 wait complete. 2024-01-11T07:13:05.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:13:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.807+0000 7f15e752e700 1 Processor -- start 2024-01-11T07:13:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.808+0000 7f15e752e700 1 -- start start 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.808+0000 7f15e752e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15e00aff20 0x7f15e00aab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.808+0000 7f15e752e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15e0063230 0x7f15e0063680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.808+0000 7f15e752e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e0063bc0 con 0x7f15e006dae0 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.808+0000 7f15e752e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e00b02f0 con 0x7f15e0063230 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.808+0000 7f15e752e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e0154860 con 0x7f15e00aff20 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e4ac9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15e0063230 0x7f15e0063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e5acb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f15e006dae0 0x7f15e006dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39794/0 (socket says 172.21.15.7:39794) 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e5acb700 1 -- 172.21.15.7:0/1305809374 learned_addr learned my addr 172.21.15.7:0/1305809374 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e52ca700 1 --2- 172.21.15.7:0/1305809374 >> v2:172.21.15.113:3300/0 conn(0x7f15e00aff20 0x7f15e00aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e4ac9700 1 -- 172.21.15.7:0/1305809374 >> v2:172.21.15.113:3300/0 conn(0x7f15e00aff20 msgr2=0x7f15e00aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e4ac9700 1 --2- 172.21.15.7:0/1305809374 >> v2:172.21.15.113:3300/0 conn(0x7f15e00aff20 0x7f15e00aab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e4ac9700 1 -- 172.21.15.7:0/1305809374 >> v1:172.21.15.7:6789/0 conn(0x7f15e006dae0 legacy=0x7f15e006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e4ac9700 1 -- 172.21.15.7:0/1305809374 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e00ab040 con 0x7f15e0063230 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15d7fff700 1 -- 172.21.15.7:0/1305809374 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563746803 0 0) 0x7f15e0063bc0 con 0x7f15e006dae0 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15e4ac9700 1 --2- 172.21.15.7:0/1305809374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e0063230 0x7f15e0063680 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f15e006d9e0 tx=0x7f15d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe3f731ff67d2a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15d7fff700 1 -- 172.21.15.7:0/1305809374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000b6b0 con 0x7f15e0063230 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15d7fff700 1 -- 172.21.15.7:0/1305809374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d000b810 con 0x7f15e0063230 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.809+0000 7f15d7fff700 1 -- 172.21.15.7:0/1305809374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000bbb0 con 0x7f15e0063230 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.810+0000 7f15e752e700 1 -- 172.21.15.7:0/1305809374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e0063230 msgr2=0x7f15e0063680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.810+0000 7f15e752e700 1 --2- 172.21.15.7:0/1305809374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e0063230 0x7f15e0063680 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f15e006d9e0 tx=0x7f15d00091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.810+0000 7f15e752e700 1 -- 172.21.15.7:0/1305809374 shutdown_connections 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.810+0000 7f15e752e700 1 --2- 172.21.15.7:0/1305809374 >> v2:172.21.15.113:3300/0 conn(0x7f15e00aff20 0x7f15e00aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.810+0000 7f15e752e700 1 --2- 172.21.15.7:0/1305809374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e0063230 0x7f15e0063680 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.810+0000 7f15e752e700 1 -- 172.21.15.7:0/1305809374 >> 172.21.15.7:0/1305809374 conn(0x7f15e0068090 msgr2=0x7f15e00cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:05.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.810+0000 7f15e752e700 1 -- 172.21.15.7:0/1305809374 shutdown_connections 2024-01-11T07:13:05.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.810+0000 7f15e752e700 1 -- 172.21.15.7:0/1305809374 wait complete. 2024-01-11T07:13:05.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.811+0000 7f15e752e700 1 Processor -- start 2024-01-11T07:13:05.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.811+0000 7f15e752e700 1 -- start start 2024-01-11T07:13:05.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e752e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e0063230 0x7f15e0152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e752e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e006dae0 0x7f15e00b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e752e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15e00aff20 0x7f15e006bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e752e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e00b19d0 con 0x7f15e006dae0 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e752e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e00b1cb0 con 0x7f15e0063230 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e752e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e00b1f90 con 0x7f15e00aff20 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e4ac9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e006dae0 0x7f15e00b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e4ac9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e006dae0 0x7f15e00b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42004/0 (socket says 172.21.15.7:42004) 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e4ac9700 1 -- 172.21.15.7:0/3076237722 learned_addr learned my addr 172.21.15.7:0/3076237722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e52ca700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e0063230 0x7f15e0152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.812+0000 7f15e5acb700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15e00aff20 0x7f15e006bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15e4ac9700 1 -- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15e00aff20 msgr2=0x7f15e006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15e4ac9700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15e00aff20 0x7f15e006bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15e4ac9700 1 -- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e0063230 msgr2=0x7f15e0152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15e4ac9700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e0063230 0x7f15e0152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15e4ac9700 1 -- 172.21.15.7:0/3076237722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0009420 con 0x7f15e006dae0 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15e4ac9700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e006dae0 0x7f15e00b1010 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f15e00cd400 tx=0x7f15d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15d57fa700 1 -- 172.21.15.7:0/3076237722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0000d10 con 0x7f15e006dae0 2024-01-11T07:13:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15d57fa700 1 -- 172.21.15.7:0/3076237722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d000b980 con 0x7f15e006dae0 2024-01-11T07:13:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e00b2270 con 0x7f15e006dae0 2024-01-11T07:13:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15d57fa700 1 -- 172.21.15.7:0/3076237722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0025420 con 0x7f15e006dae0 2024-01-11T07:13:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.813+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e0153170 con 0x7f15e006dae0 2024-01-11T07:13:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.814+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e0153360 con 0x7f15e006dae0 2024-01-11T07:13:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.815+0000 7f15d57fa700 1 -- 172.21.15.7:0/3076237722 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f15d0023070 con 0x7f15e006dae0 2024-01-11T07:13:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.815+0000 7f15d57fa700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15c4066f30 0x7f15c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.815+0000 7f15d57fa700 1 -- 172.21.15.7:0/3076237722 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15d0085b30 con 0x7f15e006dae0 2024-01-11T07:13:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.815+0000 7f15e52ca700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15c4066f30 0x7f15c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.816+0000 7f15e52ca700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15c4066f30 0x7f15c40693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f15dc006fd0 tx=0x7f15dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:05.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:05.820+0000 7f15d57fa700 1 -- 172.21.15.7:0/3076237722 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15d0052da0 con 0x7f15e006dae0 2024-01-11T07:13:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.071+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f15e004c3c0 con 0x7f15e006dae0 2024-01-11T07:13:06.071 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:13:06.071+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:13:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.076+0000 7f15d57fa700 1 -- 172.21.15.7:0/3076237722 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f15d00563c0 con 0x7f15e006dae0 2024-01-11T07:13:06.077 INFO:tasks.ceph.mds.c.smithi007.stderr: -3446> 2024-01-11T07:12:51.463+0000 7f2ea6a80700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:13:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.078+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15c4066f30 msgr2=0x7f15c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.078+0000 7f15e752e700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15c4066f30 0x7f15c40693e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f15dc006fd0 tx=0x7f15dc008040 comp rx=0 tx=0).stop 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.078+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e006dae0 msgr2=0x7f15e00b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.078+0000 7f15e752e700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e006dae0 0x7f15e00b1010 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f15e00cd400 tx=0x7f15d0017000 comp rx=0 tx=0).stop 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.079+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 shutdown_connections 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.079+0000 7f15e752e700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f15c4066f30 0x7f15c40693e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.079+0000 7f15e752e700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f15e00aff20 0x7f15e006bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.079+0000 7f15e752e700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15e0063230 0x7f15e0152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.079+0000 7f15e752e700 1 --2- 172.21.15.7:0/3076237722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15e006dae0 0x7f15e00b1010 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.079+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 >> 172.21.15.7:0/3076237722 conn(0x7f15e0068090 msgr2=0x7f15e0153ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.079+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 shutdown_connections 2024-01-11T07:13:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.079+0000 7f15e752e700 1 -- 172.21.15.7:0/3076237722 wait complete. 2024-01-11T07:13:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:13:06.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:13:06.105 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:13:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.228+0000 7fa4da54b700 1 Processor -- start 2024-01-11T07:13:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.228+0000 7fa4da54b700 1 -- start start 2024-01-11T07:13:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.229+0000 7fa4da54b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4d40af7a0 0x7fa4d406a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.229+0000 7fa4da54b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d415b860 con 0x7fa4d40ade40 2024-01-11T07:13:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.229+0000 7fa4da54b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d415a9d0 con 0x7fa4d40ada70 2024-01-11T07:13:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.229+0000 7fa4da54b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d415b530 con 0x7fa4d40af7a0 2024-01-11T07:13:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.229+0000 7fa4d8ae8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa4d40ade40 0x7fa4d415bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39798/0 (socket says 172.21.15.7:39798) 2024-01-11T07:13:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.229+0000 7fa4d8ae8700 1 -- 172.21.15.7:0/3586980868 learned_addr learned my addr 172.21.15.7:0/3586980868 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.229+0000 7fa4d37fe700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4d40af7a0 0x7fa4d406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163252362 0 0) 0x7fa4d415b860 con 0x7fa4d40ade40 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4bc003570 con 0x7fa4d40ade40 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4227060396 0 0) 0x7fa4bc003570 con 0x7fa4d40ade40 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4d40af7a0 msgr2=0x7fa4d406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4d40af7a0 0x7fa4d406a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 >> v1:172.21.15.113:6789/0 conn(0x7fa4d40ada70 legacy=0x7fa4d415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d415ac30 con 0x7fa4d40ade40 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa4cc0017f0 con 0x7fa4d40ade40 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 >> v1:172.21.15.7:6789/0 conn(0x7fa4d40ade40 legacy=0x7fa4d415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4d40af7a0 0x7fa4bc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4bc003fd0 0x7fa4bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4bc004d50 0x7fa4bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d415b530 con 0x7fa4bc004d50 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc00bb40 con 0x7fa4bc003fd0 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc00bd00 con 0x7fa4d40af7a0 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d8ae8700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4bc004d50 0x7fa4bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d37fe700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4bc003fd0 0x7fa4bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d3fff700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4d40af7a0 0x7fa4bc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d8ae8700 1 -- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4d40af7a0 msgr2=0x7fa4bc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d8ae8700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4d40af7a0 0x7fa4bc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d8ae8700 1 -- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4bc003fd0 msgr2=0x7fa4bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d8ae8700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4bc003fd0 0x7fa4bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.230+0000 7fa4d8ae8700 1 -- 172.21.15.7:0/3586980868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc003570 con 0x7fa4bc004d50 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4d8ae8700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4bc004d50 0x7fa4bc0051c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00a950 tx=0x7fa4cc0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb4f72d70cf9207 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0070d0 con 0x7fa4bc004d50 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc007230 con 0x7fa4bc004d50 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4d2ffd700 1 -- 172.21.15.7:0/3586980868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0075f0 con 0x7fa4bc004d50 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4da54b700 1 -- 172.21.15.7:0/3586980868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4bc004d50 msgr2=0x7fa4bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4da54b700 1 --2- 172.21.15.7:0/3586980868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4bc004d50 0x7fa4bc0051c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00a950 tx=0x7fa4cc0021f0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4d8ae8700 1 -- 172.21.15.7:0/3586980868 reap_dead start 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4da54b700 1 -- 172.21.15.7:0/3586980868 shutdown_connections 2024-01-11T07:13:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4da54b700 1 -- 172.21.15.7:0/3586980868 >> 172.21.15.7:0/3586980868 conn(0x7fa4d40cd070 msgr2=0x7fa4d40cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:06.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.231+0000 7fa4da54b700 1 -- 172.21.15.7:0/3586980868 shutdown_connections 2024-01-11T07:13:06.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.232+0000 7fa4da54b700 1 -- 172.21.15.7:0/3586980868 wait complete. 2024-01-11T07:13:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.233+0000 7fa4da54b700 1 Processor -- start 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.233+0000 7fa4da54b700 1 -- start start 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.234+0000 7fa4da54b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4bc004d50 0x7fa4d4159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.234+0000 7fa4da54b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4d41599d0 0x7fa4d40ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.234+0000 7fa4da54b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4d40caa30 0x7fa4d4142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.234+0000 7fa4da54b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d41540d0 con 0x7fa4d41599d0 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.234+0000 7fa4da54b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d40cb190 con 0x7fa4d40caa30 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.234+0000 7fa4da54b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d40cb470 con 0x7fa4bc004d50 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4d41599d0 0x7fa4d40ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4d41599d0 0x7fa4d40ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42026/0 (socket says 172.21.15.7:42026) 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 -- 172.21.15.7:0/2541166140 learned_addr learned my addr 172.21.15.7:0/2541166140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d8ae8700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4d40caa30 0x7fa4d4142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d3fff700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4bc004d50 0x7fa4d4159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 -- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4bc004d50 msgr2=0x7fa4d4159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4bc004d50 0x7fa4d4159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 -- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4d40caa30 msgr2=0x7fa4d4142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4d40caa30 0x7fa4d4142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 -- 172.21.15.7:0/2541166140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d40cb750 con 0x7fa4d41599d0 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d3fff700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4bc004d50 0x7fa4d4159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d8ae8700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4d40caa30 0x7fa4d4142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4d37fe700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4d41599d0 0x7fa4d40ca4f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8011b40 tx=0x7fa4c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4bbfff700 1 -- 172.21.15.7:0/2541166140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c800fa10 con 0x7fa4d41599d0 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4bbfff700 1 -- 172.21.15.7:0/2541166140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c800fb70 con 0x7fa4d41599d0 2024-01-11T07:13:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d40cba30 con 0x7fa4d41599d0 2024-01-11T07:13:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4bbfff700 1 -- 172.21.15.7:0/2541166140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8014440 con 0x7fa4d41599d0 2024-01-11T07:13:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.235+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d40caeb0 con 0x7fa4d41599d0 2024-01-11T07:13:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.237+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d404c3c0 con 0x7fa4d41599d0 2024-01-11T07:13:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.237+0000 7fa4bbfff700 1 -- 172.21.15.7:0/2541166140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa4c8014870 con 0x7fa4d41599d0 2024-01-11T07:13:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.238+0000 7fa4bbfff700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4c0066f30 0x7fa4c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.238+0000 7fa4bbfff700 1 -- 172.21.15.7:0/2541166140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fa4c8018030 con 0x7fa4d41599d0 2024-01-11T07:13:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.238+0000 7fa4d3fff700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4c0066f30 0x7fa4c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.238+0000 7fa4d3fff700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4c0066f30 0x7fa4c00693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4007a60 tx=0x7fa4c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.243+0000 7fa4bbfff700 1 -- 172.21.15.7:0/2541166140 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4c8052610 con 0x7fa4d41599d0 2024-01-11T07:13:06.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:06.494+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa4d415af10 con 0x7fa4d41599d0 2024-01-11T07:13:07.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.077+0000 7fa4bbfff700 1 -- 172.21.15.7:0/2541166140 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa4c8055c30 con 0x7fa4d41599d0 2024-01-11T07:13:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.080+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4c0066f30 msgr2=0x7fa4c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.080+0000 7fa4da54b700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4c0066f30 0x7fa4c00693e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4007a60 tx=0x7fa4c4009040 comp rx=0 tx=0).stop 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.080+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4d41599d0 msgr2=0x7fa4d40ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.080+0000 7fa4da54b700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4d41599d0 0x7fa4d40ca4f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8011b40 tx=0x7fa4c80091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.081+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 shutdown_connections 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.081+0000 7fa4da54b700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4c0066f30 0x7fa4c00693e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.081+0000 7fa4da54b700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4d40caa30 0x7fa4d4142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.081+0000 7fa4da54b700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4bc004d50 0x7fa4d4159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.081+0000 7fa4da54b700 1 --2- 172.21.15.7:0/2541166140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4d41599d0 0x7fa4d40ca4f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.081+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 >> 172.21.15.7:0/2541166140 conn(0x7fa4d40cd070 msgr2=0x7fa4d415cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.081+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 shutdown_connections 2024-01-11T07:13:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.081+0000 7fa4da54b700 1 -- 172.21.15.7:0/2541166140 wait complete. 2024-01-11T07:13:07.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:13:07.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.283+0000 7f4d5d53c700 1 Processor -- start 2024-01-11T07:13:07.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.283+0000 7f4d5d53c700 1 -- start start 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.284+0000 7f4d5d53c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d58159cb0 0x7f4d5815a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.284+0000 7f4d5d53c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4d5815db10 0x7f4d580cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.284+0000 7f4d5d53c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d580b39f0 con 0x7f4d5806dc30 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.284+0000 7f4d5d53c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d580d0600 con 0x7f4d58159cb0 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.284+0000 7f4d5d53c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d580cf9f0 con 0x7f4d5815db10 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.284+0000 7f4d56ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d58159cb0 0x7f4d5815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.284+0000 7f4d577fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4d5806dc30 0x7f4d580b5d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39806/0 (socket says 172.21.15.7:39806) 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.284+0000 7f4d577fe700 1 -- 172.21.15.7:0/1181630828 learned_addr learned my addr 172.21.15.7:0/1181630828 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d567fc700 1 --2- 172.21.15.7:0/1181630828 >> v2:172.21.15.113:3300/0 conn(0x7f4d5815db10 0x7f4d580cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d56ffd700 1 -- 172.21.15.7:0/1181630828 >> v2:172.21.15.113:3300/0 conn(0x7f4d5815db10 msgr2=0x7f4d580cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d56ffd700 1 --2- 172.21.15.7:0/1181630828 >> v2:172.21.15.113:3300/0 conn(0x7f4d5815db10 0x7f4d580cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d56ffd700 1 -- 172.21.15.7:0/1181630828 >> v1:172.21.15.7:6789/0 conn(0x7f4d5806dc30 legacy=0x7f4d580b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d56ffd700 1 -- 172.21.15.7:0/1181630828 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d580cfcd0 con 0x7f4d58159cb0 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d55ffb700 1 -- 172.21.15.7:0/1181630828 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821791801 0 0) 0x7f4d580b39f0 con 0x7f4d5806dc30 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d56ffd700 1 --2- 172.21.15.7:0/1181630828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d5815a080 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4d580b2f40 tx=0x7f4d480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e436fbc79abf9a65 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d55ffb700 1 -- 172.21.15.7:0/1181630828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800b6b0 con 0x7f4d58159cb0 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d55ffb700 1 -- 172.21.15.7:0/1181630828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4800b810 con 0x7f4d58159cb0 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.285+0000 7f4d55ffb700 1 -- 172.21.15.7:0/1181630828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800bbb0 con 0x7f4d58159cb0 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.286+0000 7f4d5d53c700 1 -- 172.21.15.7:0/1181630828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 msgr2=0x7f4d5815a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.286+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/1181630828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d5815a080 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4d580b2f40 tx=0x7f4d480091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.286+0000 7f4d5d53c700 1 -- 172.21.15.7:0/1181630828 shutdown_connections 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.286+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/1181630828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d5815a080 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.286+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/1181630828 >> v2:172.21.15.113:3300/0 conn(0x7f4d5815db10 0x7f4d580cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.286+0000 7f4d5d53c700 1 -- 172.21.15.7:0/1181630828 >> 172.21.15.7:0/1181630828 conn(0x7f4d58063cc0 msgr2=0x7f4d580b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.286+0000 7f4d5d53c700 1 -- 172.21.15.7:0/1181630828 shutdown_connections 2024-01-11T07:13:07.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.286+0000 7f4d5d53c700 1 -- 172.21.15.7:0/1181630828 wait complete. 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.287+0000 7f4d5d53c700 1 Processor -- start 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.287+0000 7f4d5d53c700 1 -- start start 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d5d53c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d5806dc30 0x7f4d58153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d5d53c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d580c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d5d53c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d5815db10 0x7f4d580c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d5d53c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58159520 con 0x7f4d58159cb0 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d5d53c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58155e60 con 0x7f4d5815db10 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d5d53c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58154120 con 0x7f4d5806dc30 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d567fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d580c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d567fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d580c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42042/0 (socket says 172.21.15.7:42042) 2024-01-11T07:13:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d567fc700 1 -- 172.21.15.7:0/225048190 learned_addr learned my addr 172.21.15.7:0/225048190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d56ffd700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d5806dc30 0x7f4d58153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d577fe700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d5815db10 0x7f4d580c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d567fc700 1 -- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d5806dc30 msgr2=0x7f4d58153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d567fc700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d5806dc30 0x7f4d58153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d567fc700 1 -- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d5815db10 msgr2=0x7f4d580c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d567fc700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d5815db10 0x7f4d580c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.288+0000 7f4d567fc700 1 -- 172.21.15.7:0/225048190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48009420 con 0x7f4d58159cb0 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.289+0000 7f4d567fc700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d580c65e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00c880 tx=0x7f4d4c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.289+0000 7f4d3effd700 1 -- 172.21.15.7:0/225048190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c011600 con 0x7f4d58159cb0 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.289+0000 7f4d3effd700 1 -- 172.21.15.7:0/225048190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c011760 con 0x7f4d58159cb0 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.289+0000 7f4d3effd700 1 -- 172.21.15.7:0/225048190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c011b00 con 0x7f4d58159cb0 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.289+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d58158c80 con 0x7f4d58159cb0 2024-01-11T07:13:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.289+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d58159240 con 0x7f4d58159cb0 2024-01-11T07:13:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.290+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d58155c30 con 0x7f4d58159cb0 2024-01-11T07:13:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.290+0000 7f4d3effd700 1 -- 172.21.15.7:0/225048190 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4d4c018360 con 0x7f4d58159cb0 2024-01-11T07:13:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.291+0000 7f4d3effd700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d44066f30 0x7f4d440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.291+0000 7f4d3effd700 1 -- 172.21.15.7:0/225048190 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f4d4c084400 con 0x7f4d58159cb0 2024-01-11T07:13:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.291+0000 7f4d56ffd700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d44066f30 0x7f4d440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.292+0000 7f4d56ffd700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d44066f30 0x7f4d440693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4d480093f0 tx=0x7f4d48017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.296+0000 7f4d3effd700 1 -- 172.21.15.7:0/225048190 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d58155c30 con 0x7f4d58159cb0 2024-01-11T07:13:07.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:07.502+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4d58155c30 con 0x7f4d58159cb0 2024-01-11T07:13:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.256+0000 7f4d3effd700 1 -- 172.21.15.7:0/225048190 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4d58155c30 con 0x7f4d58159cb0 2024-01-11T07:13:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d44066f30 msgr2=0x7f4d440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d44066f30 0x7f4d440693e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4d480093f0 tx=0x7f4d48017000 comp rx=0 tx=0).stop 2024-01-11T07:13:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 msgr2=0x7f4d580c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d580c65e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00c880 tx=0x7f4d4c008040 comp rx=0 tx=0).stop 2024-01-11T07:13:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 shutdown_connections 2024-01-11T07:13:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d44066f30 0x7f4d440693e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d5815db10 0x7f4d580c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d5806dc30 0x7f4d58153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 --2- 172.21.15.7:0/225048190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d58159cb0 0x7f4d580c65e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 >> 172.21.15.7:0/225048190 conn(0x7f4d58063cc0 msgr2=0x7f4d58154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 shutdown_connections 2024-01-11T07:13:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.262+0000 7f4d5d53c700 1 -- 172.21.15.7:0/225048190 wait complete. 2024-01-11T07:13:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:13:08.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:13:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.419+0000 7f34e18f8700 1 Processor -- start 2024-01-11T07:13:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.419+0000 7f34e18f8700 1 -- start start 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34e18f8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f34dc0cf230 0x7f34dc143190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34e18f8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc06e250 0x7f34dc1436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34e18f8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc0d03b0 con 0x7f34dc06de80 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34e18f8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc0cd7d0 con 0x7f34dc0cf230 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34e18f8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc0cf7a0 con 0x7f34dc06e250 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34daffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f34dc06de80 0x7f34dc0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47476/0 (socket says 172.21.15.7:47476) 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34daffd700 1 -- 172.21.15.7:0/3646970023 learned_addr learned my addr 172.21.15.7:0/3646970023 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34db7fe700 1 --2- 172.21.15.7:0/3646970023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc06e250 0x7f34dc1436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34d3fff700 1 --2- 172.21.15.7:0/3646970023 >> v2:172.21.15.113:3300/0 conn(0x7f34dc0cf230 0x7f34dc143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.420+0000 7f34da7fc700 1 -- 172.21.15.7:0/3646970023 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152124500 0 0) 0x7f34dc0d03b0 con 0x7f34dc06de80 2024-01-11T07:13:08.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34da7fc700 1 -- 172.21.15.7:0/3646970023 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c8003570 con 0x7f34dc06de80 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34da7fc700 1 -- 172.21.15.7:0/3646970023 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3469461550 0 0) 0x7f34c8003570 con 0x7f34dc06de80 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34db7fe700 1 -- 172.21.15.7:0/3646970023 >> v2:172.21.15.113:3300/0 conn(0x7f34dc0cf230 msgr2=0x7f34dc143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34db7fe700 1 --2- 172.21.15.7:0/3646970023 >> v2:172.21.15.113:3300/0 conn(0x7f34dc0cf230 0x7f34dc143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34db7fe700 1 -- 172.21.15.7:0/3646970023 >> v1:172.21.15.7:6789/0 conn(0x7f34dc06de80 legacy=0x7f34dc0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34db7fe700 1 -- 172.21.15.7:0/3646970023 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0cfa80 con 0x7f34dc06e250 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34db7fe700 1 --2- 172.21.15.7:0/3646970023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc06e250 0x7f34dc1436d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009e60 tx=0x7f34d4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97be001116f71f1b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34da7fc700 1 -- 172.21.15.7:0/3646970023 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d400e070 con 0x7f34dc06e250 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34da7fc700 1 -- 172.21.15.7:0/3646970023 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d400b7a0 con 0x7f34dc06e250 2024-01-11T07:13:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.421+0000 7f34da7fc700 1 -- 172.21.15.7:0/3646970023 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc001470 con 0x7f34dc06e250 2024-01-11T07:13:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.422+0000 7f34e18f8700 1 -- 172.21.15.7:0/3646970023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc06e250 msgr2=0x7f34dc1436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.422+0000 7f34e18f8700 1 --2- 172.21.15.7:0/3646970023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc06e250 0x7f34dc1436d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009e60 tx=0x7f34d4009780 comp rx=0 tx=0).stop 2024-01-11T07:13:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.422+0000 7f34e18f8700 1 -- 172.21.15.7:0/3646970023 shutdown_connections 2024-01-11T07:13:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.422+0000 7f34e18f8700 1 --2- 172.21.15.7:0/3646970023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc06e250 0x7f34dc1436d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.422+0000 7f34e18f8700 1 --2- 172.21.15.7:0/3646970023 >> v2:172.21.15.113:3300/0 conn(0x7f34dc0cf230 0x7f34dc143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.422+0000 7f34e18f8700 1 -- 172.21.15.7:0/3646970023 >> 172.21.15.7:0/3646970023 conn(0x7f34dc070690 msgr2=0x7f34dc070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.422+0000 7f34e18f8700 1 -- 172.21.15.7:0/3646970023 shutdown_connections 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.422+0000 7f34e18f8700 1 -- 172.21.15.7:0/3646970023 wait complete. 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.423+0000 7f34e18f8700 1 Processor -- start 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.423+0000 7f34e18f8700 1 -- start start 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34e18f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34dc06de80 0x7f34dc15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34e18f8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34dc06e250 0x7f34dc15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34e18f8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc0cf230 0x7f34dc159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34e18f8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc15b9f0 con 0x7f34dc06de80 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34e18f8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc15f360 con 0x7f34dc06e250 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34e18f8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc15a230 con 0x7f34dc0cf230 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34daffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34dc06de80 0x7f34dc15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34daffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34dc06de80 0x7f34dc15b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33298/0 (socket says 172.21.15.7:33298) 2024-01-11T07:13:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34daffd700 1 -- 172.21.15.7:0/2946565314 learned_addr learned my addr 172.21.15.7:0/2946565314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34db7fe700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc0cf230 0x7f34dc159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34d3fff700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34dc06e250 0x7f34dc15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.424+0000 7f34daffd700 1 -- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc0cf230 msgr2=0x7f34dc159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34daffd700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc0cf230 0x7f34dc159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34daffd700 1 -- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34dc06e250 msgr2=0x7f34dc15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34daffd700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34dc06e250 0x7f34dc15ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34daffd700 1 -- 172.21.15.7:0/2946565314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d4009a20 con 0x7f34dc06de80 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34db7fe700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc0cf230 0x7f34dc159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34d3fff700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34dc06e250 0x7f34dc15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34daffd700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34dc06de80 0x7f34dc15b4b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00cb90 tx=0x7f34cc001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34d2ffd700 1 -- 172.21.15.7:0/2946565314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc007360 con 0x7f34dc06de80 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34d2ffd700 1 -- 172.21.15.7:0/2946565314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc0074e0 con 0x7f34dc06de80 2024-01-11T07:13:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc169a10 con 0x7f34dc06de80 2024-01-11T07:13:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34d2ffd700 1 -- 172.21.15.7:0/2946565314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc004830 con 0x7f34dc06de80 2024-01-11T07:13:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.425+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc15bf30 con 0x7f34dc06de80 2024-01-11T07:13:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.426+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc04c3c0 con 0x7f34dc06de80 2024-01-11T07:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.426+0000 7f34d2ffd700 1 -- 172.21.15.7:0/2946565314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f34cc004ba0 con 0x7f34dc06de80 2024-01-11T07:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.427+0000 7f34d2ffd700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34bc066f30 0x7f34bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.427+0000 7f34d2ffd700 1 -- 172.21.15.7:0/2946565314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f34cc083d70 con 0x7f34dc06de80 2024-01-11T07:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.427+0000 7f34d3fff700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34bc066f30 0x7f34bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.428+0000 7f34d3fff700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34bc066f30 0x7f34bc0693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f34dc142380 tx=0x7f34c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:08.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.431+0000 7f34d2ffd700 1 -- 172.21.15.7:0/2946565314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34cc0511c0 con 0x7f34dc06de80 2024-01-11T07:13:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:08.633+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34dc0cfd60 con 0x7f34dc06de80 2024-01-11T07:13:09.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.256+0000 7f34d2ffd700 1 -- 172.21.15.7:0/2946565314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7f34cc018020 con 0x7f34dc06de80 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34bc066f30 msgr2=0x7f34bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34bc066f30 0x7f34bc0693e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f34dc142380 tx=0x7f34c4008040 comp rx=0 tx=0).stop 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34dc06de80 msgr2=0x7f34dc15b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34dc06de80 0x7f34dc15b4b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00cb90 tx=0x7f34cc001b50 comp rx=0 tx=0).stop 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 shutdown_connections 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34bc066f30 0x7f34bc0693e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34dc0cf230 0x7f34dc159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34dc06de80 0x7f34dc15b4b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 --2- 172.21.15.7:0/2946565314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34dc06e250 0x7f34dc15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 >> 172.21.15.7:0/2946565314 conn(0x7f34dc070690 msgr2=0x7f34dc142550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 shutdown_connections 2024-01-11T07:13:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.259+0000 7f34e18f8700 1 -- 172.21.15.7:0/2946565314 wait complete. 2024-01-11T07:13:09.260 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:13:09.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:13:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.414+0000 7f5207238700 1 Processor -- start 2024-01-11T07:13:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.414+0000 7f5207238700 1 -- start start 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.415+0000 7f5207238700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5200157f90 0x7f52000c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.415+0000 7f5207238700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f52000c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.415+0000 7f5207238700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52000c9e40 con 0x7f5200157bc0 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.415+0000 7f5207238700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52000ca640 con 0x7f5200157f90 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.415+0000 7f5207238700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52000b1ca0 con 0x7f52000c94a0 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.415+0000 7f5204fd4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5200157bc0 0x7f52000cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47488/0 (socket says 172.21.15.7:47488) 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.415+0000 7f5204fd4700 1 -- 172.21.15.7:0/3731645681 learned_addr learned my addr 172.21.15.7:0/3731645681 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f51fffff700 1 --2- 172.21.15.7:0/3731645681 >> v2:172.21.15.113:3300/0 conn(0x7f5200157f90 0x7f52000c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.415+0000 7f52057d5700 1 --2- 172.21.15.7:0/3731645681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f52000c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f51ff7fe700 1 -- 172.21.15.7:0/3731645681 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568025085 0 0) 0x7f51f4001290 con 0x7f5200157bc0 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f51ff7fe700 1 -- 172.21.15.7:0/3731645681 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f0003570 con 0x7f5200157bc0 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f52057d5700 1 -- 172.21.15.7:0/3731645681 >> v2:172.21.15.113:3300/0 conn(0x7f5200157f90 msgr2=0x7f52000c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f52057d5700 1 --2- 172.21.15.7:0/3731645681 >> v2:172.21.15.113:3300/0 conn(0x7f5200157f90 0x7f52000c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f52057d5700 1 -- 172.21.15.7:0/3731645681 >> v1:172.21.15.7:6789/0 conn(0x7f5200157bc0 legacy=0x7f52000cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f52057d5700 1 -- 172.21.15.7:0/3731645681 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52000ca080 con 0x7f52000c94a0 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f52057d5700 1 --2- 172.21.15.7:0/3731645681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f52000c9900 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009e60 tx=0x7f51f8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f9530e55927f1aa server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f51ff7fe700 1 -- 172.21.15.7:0/3731645681 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f800e070 con 0x7f52000c94a0 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f51ff7fe700 1 -- 172.21.15.7:0/3731645681 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f800bab0 con 0x7f52000c94a0 2024-01-11T07:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.416+0000 7f51ff7fe700 1 -- 172.21.15.7:0/3731645681 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f800be70 con 0x7f52000c94a0 2024-01-11T07:13:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.417+0000 7f5207238700 1 -- 172.21.15.7:0/3731645681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 msgr2=0x7f52000c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.417+0000 7f5207238700 1 --2- 172.21.15.7:0/3731645681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f52000c9900 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009e60 tx=0x7f51f8009780 comp rx=0 tx=0).stop 2024-01-11T07:13:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.417+0000 7f5207238700 1 -- 172.21.15.7:0/3731645681 shutdown_connections 2024-01-11T07:13:09.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.417+0000 7f5207238700 1 --2- 172.21.15.7:0/3731645681 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f52000c9900 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.417+0000 7f5207238700 1 --2- 172.21.15.7:0/3731645681 >> v2:172.21.15.113:3300/0 conn(0x7f5200157f90 0x7f52000c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.417+0000 7f5207238700 1 -- 172.21.15.7:0/3731645681 >> 172.21.15.7:0/3731645681 conn(0x7f5200097a50 msgr2=0x7f52000ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:09.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.417+0000 7f5207238700 1 -- 172.21.15.7:0/3731645681 shutdown_connections 2024-01-11T07:13:09.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.417+0000 7f5207238700 1 -- 172.21.15.7:0/3731645681 wait complete. 2024-01-11T07:13:09.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.419+0000 7f5207238700 1 Processor -- start 2024-01-11T07:13:09.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.419+0000 7f5207238700 1 -- start start 2024-01-11T07:13:09.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f5207238700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f5200154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f5207238700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5200157bc0 0x7f5200151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f5207238700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5200157f90 0x7f52000689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f5207238700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5200154db0 con 0x7f5200157bc0 2024-01-11T07:13:09.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f5207238700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52000c69c0 con 0x7f5200157f90 2024-01-11T07:13:09.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f5207238700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52000c5db0 con 0x7f52000c94a0 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f51fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5200157bc0 0x7f5200151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f51fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5200157bc0 0x7f5200151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33314/0 (socket says 172.21.15.7:33314) 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f51fffff700 1 -- 172.21.15.7:0/3909433704 learned_addr learned my addr 172.21.15.7:0/3909433704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f5204fd4700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f5200154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f52057d5700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5200157f90 0x7f52000689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f51fffff700 1 -- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 msgr2=0x7f5200154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f51fffff700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f5200154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f51fffff700 1 -- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5200157f90 msgr2=0x7f52000689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f51fffff700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5200157f90 0x7f52000689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.420+0000 7f51fffff700 1 -- 172.21.15.7:0/3909433704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f8009a20 con 0x7f5200157bc0 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.421+0000 7f51fffff700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5200157bc0 0x7f5200151fb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00e960 tx=0x7f51ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.421+0000 7f51fcff9700 1 -- 172.21.15.7:0/3909433704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00c850 con 0x7f5200157bc0 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.421+0000 7f51fcff9700 1 -- 172.21.15.7:0/3909433704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00c9b0 con 0x7f5200157bc0 2024-01-11T07:13:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.421+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52000c6090 con 0x7f5200157bc0 2024-01-11T07:13:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.421+0000 7f51fcff9700 1 -- 172.21.15.7:0/3909433704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00cd50 con 0x7f5200157bc0 2024-01-11T07:13:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.421+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52000c6650 con 0x7f5200157bc0 2024-01-11T07:13:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.422+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520004c3c0 con 0x7f5200157bc0 2024-01-11T07:13:09.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.422+0000 7f51fcff9700 1 -- 172.21.15.7:0/3909433704 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f51ec013680 con 0x7f5200157bc0 2024-01-11T07:13:09.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.423+0000 7f51fcff9700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51e4066f30 0x7f51e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.423+0000 7f51fcff9700 1 -- 172.21.15.7:0/3909433704 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f51ec084460 con 0x7f5200157bc0 2024-01-11T07:13:09.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.423+0000 7f5204fd4700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51e4066f30 0x7f51e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.424+0000 7f5204fd4700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51e4066f30 0x7f51e40693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f51f40013f0 tx=0x7f51f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:09.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.427+0000 7f51fcff9700 1 -- 172.21.15.7:0/3909433704 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51ec0519d0 con 0x7f5200157bc0 2024-01-11T07:13:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.630+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f52000ca360 con 0x7f5200157bc0 2024-01-11T07:13:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.634+0000 7f51fcff9700 1 -- 172.21.15.7:0/3909433704 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f51ec054ff0 con 0x7f5200157bc0 2024-01-11T07:13:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51e4066f30 msgr2=0x7f51e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51e4066f30 0x7f51e40693e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f51f40013f0 tx=0x7f51f4009040 comp rx=0 tx=0).stop 2024-01-11T07:13:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5200157bc0 msgr2=0x7f5200151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5200157bc0 0x7f5200151fb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00e960 tx=0x7f51ec0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 shutdown_connections 2024-01-11T07:13:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51e4066f30 0x7f51e40693e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5200157f90 0x7f52000689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52000c94a0 0x7f5200154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 --2- 172.21.15.7:0/3909433704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5200157bc0 0x7f5200151fb0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 >> 172.21.15.7:0/3909433704 conn(0x7f5200097a50 msgr2=0x7f520006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 shutdown_connections 2024-01-11T07:13:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.636+0000 7f5207238700 1 -- 172.21.15.7:0/3909433704 wait complete. 2024-01-11T07:13:09.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-01-11T07:13:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.781+0000 7f67844d5700 1 Processor -- start 2024-01-11T07:13:09.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.781+0000 7f67844d5700 1 -- start start 2024-01-11T07:13:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f67844d5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f677c069320 0x7f677c1649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f67844d5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677c0ce9b0 con 0x7f677c15a060 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f67844d5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677c0cf530 con 0x7f677c159c90 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f67844d5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677c0b1f30 con 0x7f677c069320 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f6781a70700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f677c15a060 0x7f677c0b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47502/0 (socket says 172.21.15.7:47502) 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f6781a70700 1 -- 172.21.15.7:0/2094885059 learned_addr learned my addr 172.21.15.7:0/2094885059 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f6782a72700 1 --2- 172.21.15.7:0/2094885059 >> v2:172.21.15.113:3300/0 conn(0x7f677c069320 0x7f677c1649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f678126f700 1 -- 172.21.15.7:0/2094885059 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091869229 0 0) 0x7f677c0ce9b0 con 0x7f677c15a060 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f678126f700 1 -- 172.21.15.7:0/2094885059 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6768003570 con 0x7f677c15a060 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f678126f700 1 -- 172.21.15.7:0/2094885059 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916235706 0 0) 0x7f677c0cf530 con 0x7f677c159c90 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f678126f700 1 -- 172.21.15.7:0/2094885059 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f677c0ce9b0 con 0x7f677c159c90 2024-01-11T07:13:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f678126f700 1 -- 172.21.15.7:0/2094885059 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1538014703 0 0) 0x7f6768003570 con 0x7f677c15a060 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f6782a72700 1 -- 172.21.15.7:0/2094885059 >> v1:172.21.15.113:6789/0 conn(0x7f677c159c90 legacy=0x7f677c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f6782a72700 1 -- 172.21.15.7:0/2094885059 >> v1:172.21.15.7:6789/0 conn(0x7f677c15a060 legacy=0x7f677c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.782+0000 7f6782a72700 1 -- 172.21.15.7:0/2094885059 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c0cf9c0 con 0x7f677c069320 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.783+0000 7f6782a72700 1 --2- 172.21.15.7:0/2094885059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c069320 0x7f677c1649a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6774009e60 tx=0x7f6774009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869a23ab48d1260a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.783+0000 7f678126f700 1 -- 172.21.15.7:0/2094885059 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400e070 con 0x7f677c069320 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.783+0000 7f678126f700 1 -- 172.21.15.7:0/2094885059 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677400b7a0 con 0x7f677c069320 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.783+0000 7f678126f700 1 -- 172.21.15.7:0/2094885059 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770001470 con 0x7f677c069320 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.783+0000 7f67844d5700 1 -- 172.21.15.7:0/2094885059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c069320 msgr2=0x7f677c1649a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.783+0000 7f67844d5700 1 --2- 172.21.15.7:0/2094885059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c069320 0x7f677c1649a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6774009e60 tx=0x7f6774009780 comp rx=0 tx=0).stop 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.784+0000 7f67844d5700 1 -- 172.21.15.7:0/2094885059 shutdown_connections 2024-01-11T07:13:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.784+0000 7f67844d5700 1 --2- 172.21.15.7:0/2094885059 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c069320 0x7f677c1649a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.784+0000 7f67844d5700 1 -- 172.21.15.7:0/2094885059 >> 172.21.15.7:0/2094885059 conn(0x7f677c071100 msgr2=0x7f677c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.784+0000 7f67844d5700 1 -- 172.21.15.7:0/2094885059 shutdown_connections 2024-01-11T07:13:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.784+0000 7f67844d5700 1 -- 172.21.15.7:0/2094885059 wait complete. 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.785+0000 7f67844d5700 1 Processor -- start 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.785+0000 7f67844d5700 1 -- start start 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f67844d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c069320 0x7f677c144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f67844d5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f677c159c90 0x7f677c16d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f67844d5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c15a060 0x7f677c0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f67844d5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c1454c0 con 0x7f677c069320 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f67844d5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c1456d0 con 0x7f677c15a060 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f67844d5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c1459b0 con 0x7f677c159c90 2024-01-11T07:13:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c069320 0x7f677c144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c069320 0x7f677c144c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33318/0 (socket says 172.21.15.7:33318) 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 -- 172.21.15.7:0/822176362 learned_addr learned my addr 172.21.15.7:0/822176362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782a72700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c15a060 0x7f677c0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6781a70700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f677c159c90 0x7f677c16d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 -- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f677c159c90 msgr2=0x7f677c16d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f677c159c90 0x7f677c16d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 -- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c15a060 msgr2=0x7f677c0afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c15a060 0x7f677c0afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 -- 172.21.15.7:0/822176362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774009a20 con 0x7f677c069320 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f6782271700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c069320 0x7f677c144c10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f677800c790 tx=0x7f6778007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f676e7fc700 1 -- 172.21.15.7:0/822176362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778010510 con 0x7f677c069320 2024-01-11T07:13:09.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f676e7fc700 1 -- 172.21.15.7:0/822176362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6778010670 con 0x7f677c069320 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c145c90 con 0x7f677c069320 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.786+0000 7f676e7fc700 1 -- 172.21.15.7:0/822176362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778010a10 con 0x7f677c069320 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.787+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c146250 con 0x7f677c069320 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.788+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677c04c3c0 con 0x7f677c069320 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.788+0000 7f676e7fc700 1 -- 172.21.15.7:0/822176362 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f677801b3b0 con 0x7f677c069320 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.788+0000 7f676e7fc700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6760066f30 0x7f67600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.788+0000 7f676e7fc700 1 -- 172.21.15.7:0/822176362 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6778083ad0 con 0x7f677c069320 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.789+0000 7f6781a70700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6760066f30 0x7f67600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.789+0000 7f6781a70700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6760066f30 0x7f67600693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6770001ed0 tx=0x7f67700019d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:09.793+0000 7f676e7fc700 1 -- 172.21.15.7:0/822176362 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6778051040 con 0x7f677c069320 2024-01-11T07:13:10.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.080+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f677c069100 con 0x7f677c069320 2024-01-11T07:13:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.262+0000 7f676e7fc700 1 -- 172.21.15.7:0/822176362 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v318) v1 ==== 128+0+0 (secure 0 0 0) 0x7f6778017020 con 0x7f677c069320 2024-01-11T07:13:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6760066f30 msgr2=0x7f67600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:10.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6760066f30 0x7f67600693e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6770001ed0 tx=0x7f67700019d0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c069320 msgr2=0x7f677c144c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c069320 0x7f677c144c10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f677800c790 tx=0x7f6778007e10 comp rx=0 tx=0).stop 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 shutdown_connections 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6760066f30 0x7f67600693e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f677c15a060 0x7f677c0afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c069320 0x7f677c144c10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 --2- 172.21.15.7:0/822176362 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f677c159c90 0x7f677c16d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 >> 172.21.15.7:0/822176362 conn(0x7f677c071100 msgr2=0x7f677c067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 shutdown_connections 2024-01-11T07:13:10.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.265+0000 7f67844d5700 1 -- 172.21.15.7:0/822176362 wait complete. 2024-01-11T07:13:10.282 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-01-11T07:13:10.282 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2024-01-11T07:13:10.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-01-11T07:13:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.419+0000 7f16f19d7700 1 Processor -- start 2024-01-11T07:13:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.419+0000 7f16f19d7700 1 -- start start 2024-01-11T07:13:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.420+0000 7f16f19d7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16ec0cd6d0 0x7f16ec0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.420+0000 7f16f19d7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16ec15e450 0x7f16ec159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.420+0000 7f16f19d7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec15acf0 con 0x7f16ec15d8a0 2024-01-11T07:13:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.420+0000 7f16f19d7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec15afd0 con 0x7f16ec15e450 2024-01-11T07:13:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.420+0000 7f16f19d7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec0ce980 con 0x7f16ec0cd6d0 2024-01-11T07:13:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16ea7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16ec15d8a0 0x7f16ec15dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47516/0 (socket says 172.21.15.7:47516) 2024-01-11T07:13:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16ea7fc700 1 -- 172.21.15.7:0/1321168744 learned_addr learned my addr 172.21.15.7:0/1321168744 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16eb7fe700 1 --2- 172.21.15.7:0/1321168744 >> v2:172.21.15.7:3300/0 conn(0x7f16ec15e450 0x7f16ec159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16eaffd700 1 --2- 172.21.15.7:0/1321168744 >> v2:172.21.15.113:3300/0 conn(0x7f16ec0cd6d0 0x7f16ec0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16e9ffb700 1 -- 172.21.15.7:0/1321168744 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392009938 0 0) 0x7f16ec15acf0 con 0x7f16ec15d8a0 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16e9ffb700 1 -- 172.21.15.7:0/1321168744 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d8003570 con 0x7f16ec15d8a0 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16eb7fe700 1 -- 172.21.15.7:0/1321168744 >> v2:172.21.15.113:3300/0 conn(0x7f16ec0cd6d0 msgr2=0x7f16ec0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16eb7fe700 1 --2- 172.21.15.7:0/1321168744 >> v2:172.21.15.113:3300/0 conn(0x7f16ec0cd6d0 0x7f16ec0cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16eb7fe700 1 -- 172.21.15.7:0/1321168744 >> v1:172.21.15.7:6789/0 conn(0x7f16ec15d8a0 legacy=0x7f16ec15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16eb7fe700 1 -- 172.21.15.7:0/1321168744 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec0cec60 con 0x7f16ec15e450 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16eb7fe700 1 --2- 172.21.15.7:0/1321168744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15e450 0x7f16ec159600 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0677c0 tx=0x7f16e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2216d4298b6f8a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16e9ffb700 1 -- 172.21.15.7:0/1321168744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e400e070 con 0x7f16ec15e450 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16e9ffb700 1 -- 172.21.15.7:0/1321168744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e400bab0 con 0x7f16ec15e450 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.421+0000 7f16e9ffb700 1 -- 172.21.15.7:0/1321168744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e400be70 con 0x7f16ec15e450 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.422+0000 7f16f19d7700 1 -- 172.21.15.7:0/1321168744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15e450 msgr2=0x7f16ec159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.422+0000 7f16f19d7700 1 --2- 172.21.15.7:0/1321168744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15e450 0x7f16ec159600 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0677c0 tx=0x7f16e4009780 comp rx=0 tx=0).stop 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.422+0000 7f16f19d7700 1 -- 172.21.15.7:0/1321168744 shutdown_connections 2024-01-11T07:13:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.422+0000 7f16f19d7700 1 --2- 172.21.15.7:0/1321168744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15e450 0x7f16ec159600 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.422+0000 7f16f19d7700 1 --2- 172.21.15.7:0/1321168744 >> v2:172.21.15.113:3300/0 conn(0x7f16ec0cd6d0 0x7f16ec0cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.422+0000 7f16f19d7700 1 -- 172.21.15.7:0/1321168744 >> 172.21.15.7:0/1321168744 conn(0x7f16ec071120 msgr2=0x7f16ec06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:10.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.422+0000 7f16f19d7700 1 -- 172.21.15.7:0/1321168744 shutdown_connections 2024-01-11T07:13:10.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.422+0000 7f16f19d7700 1 -- 172.21.15.7:0/1321168744 wait complete. 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.424+0000 7f16f19d7700 1 Processor -- start 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.424+0000 7f16f19d7700 1 -- start start 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16f19d7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ec0cd6d0 0x7f16ec151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16f19d7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15d8a0 0x7f16ec1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16f19d7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ec15e450 0x7f16ec1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16f19d7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1578a0 con 0x7f16ec15d8a0 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16f19d7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec156c90 con 0x7f16ec0cd6d0 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16f19d7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec156f70 con 0x7f16ec15e450 2024-01-11T07:13:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16ea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15d8a0 0x7f16ec1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16ea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15d8a0 0x7f16ec1560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33336/0 (socket says 172.21.15.7:33336) 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16ea7fc700 1 -- 172.21.15.7:0/4236806424 learned_addr learned my addr 172.21.15.7:0/4236806424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16eaffd700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ec0cd6d0 0x7f16ec151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16eb7fe700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ec15e450 0x7f16ec1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16ea7fc700 1 -- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ec15e450 msgr2=0x7f16ec1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16ea7fc700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ec15e450 0x7f16ec1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16ea7fc700 1 -- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ec0cd6d0 msgr2=0x7f16ec151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16ea7fc700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ec0cd6d0 0x7f16ec151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.425+0000 7f16ea7fc700 1 -- 172.21.15.7:0/4236806424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e4009a20 con 0x7f16ec15d8a0 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.426+0000 7f16ea7fc700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15d8a0 0x7f16ec1560b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c7c0 tx=0x7f16dc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:10.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.426+0000 7f16e2ffd700 1 -- 172.21.15.7:0/4236806424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc005cd0 con 0x7f16ec15d8a0 2024-01-11T07:13:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.426+0000 7f16e2ffd700 1 -- 172.21.15.7:0/4236806424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16dc005e30 con 0x7f16ec15d8a0 2024-01-11T07:13:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.426+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec157250 con 0x7f16ec15d8a0 2024-01-11T07:13:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.426+0000 7f16e2ffd700 1 -- 172.21.15.7:0/4236806424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc007320 con 0x7f16ec15d8a0 2024-01-11T07:13:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.426+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1520e0 con 0x7f16ec15d8a0 2024-01-11T07:13:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.427+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ec04c3c0 con 0x7f16ec15d8a0 2024-01-11T07:13:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.427+0000 7f16e2ffd700 1 -- 172.21.15.7:0/4236806424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f16dc01b3b0 con 0x7f16ec15d8a0 2024-01-11T07:13:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.428+0000 7f16e2ffd700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f16cc066f30 0x7f16cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.428+0000 7f16e2ffd700 1 -- 172.21.15.7:0/4236806424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f16dc0838d0 con 0x7f16ec15d8a0 2024-01-11T07:13:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.428+0000 7f16eaffd700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f16cc066f30 0x7f16cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.429+0000 7f16eaffd700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f16cc066f30 0x7f16cc0693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009db0 tx=0x7f16d40091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:10.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.432+0000 7f16e2ffd700 1 -- 172.21.15.7:0/4236806424 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16dc050e40 con 0x7f16ec15d8a0 2024-01-11T07:13:10.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:10.729+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f16ec1576f0 con 0x7f16ec15d8a0 2024-01-11T07:13:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.264+0000 7f16e2ffd700 1 -- 172.21.15.7:0/4236806424 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v319) v1 ==== 143+0+0 (secure 0 0 0) 0x7f16dc017020 con 0x7f16ec15d8a0 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f16cc066f30 msgr2=0x7f16cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f16cc066f30 0x7f16cc0693e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009db0 tx=0x7f16d40091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15d8a0 msgr2=0x7f16ec1560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15d8a0 0x7f16ec1560b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c7c0 tx=0x7f16dc007e00 comp rx=0 tx=0).stop 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 shutdown_connections 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f16cc066f30 0x7f16cc0693e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16ec15e450 0x7f16ec1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16ec0cd6d0 0x7f16ec151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 --2- 172.21.15.7:0/4236806424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16ec15d8a0 0x7f16ec1560b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 >> 172.21.15.7:0/4236806424 conn(0x7f16ec071120 msgr2=0x7f16ec15dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 shutdown_connections 2024-01-11T07:13:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.267+0000 7f16f19d7700 1 -- 172.21.15.7:0/4236806424 wait complete. 2024-01-11T07:13:11.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:13:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.421+0000 7fc32a2a4700 1 Processor -- start 2024-01-11T07:13:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.421+0000 7fc32a2a4700 1 -- start start 2024-01-11T07:13:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc32a2a4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc32a2a4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc32406ad10 0x7fc32415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc32a2a4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32415b860 con 0x7fc3240af7a0 2024-01-11T07:13:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc32a2a4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32415b530 con 0x7fc32406ad10 2024-01-11T07:13:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc32a2a4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32415ac30 con 0x7fc3240ada70 2024-01-11T07:13:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc3237fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc3240af7a0 0x7fc32406a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47518/0 (socket says 172.21.15.7:47518) 2024-01-11T07:13:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc3237fe700 1 -- 172.21.15.7:0/3990844469 learned_addr learned my addr 172.21.15.7:0/3990844469 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc328841700 1 --2- 172.21.15.7:0/3990844469 >> v2:172.21.15.113:3300/0 conn(0x7fc32406ad10 0x7fc32415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc323fff700 1 --2- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc322ffd700 1 -- 172.21.15.7:0/3990844469 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177934143 0 0) 0x7fc32415b860 con 0x7fc3240af7a0 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.422+0000 7fc322ffd700 1 -- 172.21.15.7:0/3990844469 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc30c003570 con 0x7fc3240af7a0 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc328841700 1 -- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 msgr2=0x7fc32415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc322ffd700 1 -- 172.21.15.7:0/3990844469 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 347451727 0 0) 0x7fc30c003570 con 0x7fc3240af7a0 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc328841700 1 --2- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc328841700 1 -- 172.21.15.7:0/3990844469 >> v1:172.21.15.7:6789/0 conn(0x7fc3240af7a0 legacy=0x7fc32406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc328841700 1 -- 172.21.15.7:0/3990844469 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32415af10 con 0x7fc32406ad10 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc323fff700 1 --2- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc328841700 1 --2- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 0x7fc32415bcf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009e60 tx=0x7fc31c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8f1d84d37b4e05 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc322ffd700 1 -- 172.21.15.7:0/3990844469 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00e070 con 0x7fc32406ad10 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc322ffd700 1 -- 172.21.15.7:0/3990844469 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c00b7a0 con 0x7fc32406ad10 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.423+0000 7fc322ffd700 1 -- 172.21.15.7:0/3990844469 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc318001470 con 0x7fc32406ad10 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.424+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 msgr2=0x7fc32415bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.424+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 0x7fc32415bcf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009e60 tx=0x7fc31c009780 comp rx=0 tx=0).stop 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.424+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3990844469 shutdown_connections 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.424+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 0x7fc32415bcf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.424+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3990844469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.424+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3990844469 >> 172.21.15.7:0/3990844469 conn(0x7fc3240cd070 msgr2=0x7fc3240cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.424+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3990844469 shutdown_connections 2024-01-11T07:13:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.424+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3990844469 wait complete. 2024-01-11T07:13:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.425+0000 7fc32a2a4700 1 Processor -- start 2024-01-11T07:13:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.425+0000 7fc32a2a4700 1 -- start start 2024-01-11T07:13:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc32a2a4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 0x7fc32416cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc32a2a4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32406d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc32a2a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3240af7a0 0x7fc3240cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc32a2a4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32416de80 con 0x7fc3240af7a0 2024-01-11T07:13:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc32a2a4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3240ca6a0 con 0x7fc32406ad10 2024-01-11T07:13:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc32a2a4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32416d8c0 con 0x7fc3240ada70 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc328841700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3240af7a0 0x7fc3240cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc328841700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3240af7a0 0x7fc3240cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33350/0 (socket says 172.21.15.7:33350) 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc328841700 1 -- 172.21.15.7:0/3735174450 learned_addr learned my addr 172.21.15.7:0/3735174450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc3237fe700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc323fff700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 0x7fc32416cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc328841700 1 -- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 msgr2=0x7fc32406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc328841700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32406d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc328841700 1 -- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 msgr2=0x7fc32416cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc328841700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 0x7fc32416cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.426+0000 7fc328841700 1 -- 172.21.15.7:0/3735174450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc314009420 con 0x7fc3240af7a0 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.427+0000 7fc328841700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3240af7a0 0x7fc3240cab30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc324063d40 tx=0x7fc31c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.427+0000 7fc30bfff700 1 -- 172.21.15.7:0/3735174450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c012d20 con 0x7fc3240af7a0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.427+0000 7fc30bfff700 1 -- 172.21.15.7:0/3735174450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c012e80 con 0x7fc3240af7a0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.427+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31c009a20 con 0x7fc3240af7a0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.427+0000 7fc30bfff700 1 -- 172.21.15.7:0/3735174450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0276c0 con 0x7fc3240af7a0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.427+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32416d500 con 0x7fc3240af7a0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.428+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3240ca810 con 0x7fc3240af7a0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.428+0000 7fc30bfff700 1 -- 172.21.15.7:0/3735174450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc31c024070 con 0x7fc3240af7a0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.429+0000 7fc30bfff700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc310066f30 0x7fc3100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.429+0000 7fc30bfff700 1 -- 172.21.15.7:0/3735174450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc31c0867c0 con 0x7fc3240af7a0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.429+0000 7fc323fff700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc310066f30 0x7fc3100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.429+0000 7fc323fff700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc310066f30 0x7fc3100693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc3140093f0 tx=0x7fc314008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.433+0000 7fc30bfff700 1 -- 172.21.15.7:0/3735174450 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc31c053d30 con 0x7fc3240af7a0 2024-01-11T07:13:11.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.636+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc32404c3c0 con 0x7fc3240af7a0 2024-01-11T07:13:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.640+0000 7fc30bfff700 1 -- 172.21.15.7:0/3735174450 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc31c057350 con 0x7fc3240af7a0 2024-01-11T07:13:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc310066f30 msgr2=0x7fc3100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc310066f30 0x7fc3100693e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc3140093f0 tx=0x7fc314008040 comp rx=0 tx=0).stop 2024-01-11T07:13:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3240af7a0 msgr2=0x7fc3240cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3240af7a0 0x7fc3240cab30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc324063d40 tx=0x7fc31c018000 comp rx=0 tx=0).stop 2024-01-11T07:13:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 shutdown_connections 2024-01-11T07:13:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc310066f30 0x7fc3100693e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3240af7a0 0x7fc3240cab30 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc32406ad10 0x7fc32416cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 --2- 172.21.15.7:0/3735174450 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc3240ada70 0x7fc32406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 >> 172.21.15.7:0/3735174450 conn(0x7fc3240cd070 msgr2=0x7fc324153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.642+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 shutdown_connections 2024-01-11T07:13:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.643+0000 7fc32a2a4700 1 -- 172.21.15.7:0/3735174450 wait complete. 2024-01-11T07:13:11.656 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:13:11.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:13:11.700 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:13:11.744 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:13:11.744 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:13:11.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.881+0000 7fb47208b700 1 Processor -- start 2024-01-11T07:13:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.881+0000 7fb47208b700 1 -- start start 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb47208b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb46c069d60 0x7fb46c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb47208b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c063db0 0x7fb46c0b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb47208b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c0ad300 con 0x7fb46c155650 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb47208b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c0ace30 con 0x7fb46c069d60 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb47208b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c063920 con 0x7fb46c063db0 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb46affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb46c069d60 0x7fb46c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb46b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb46c155650 0x7fb46c0ac720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47528/0 (socket says 172.21.15.7:47528) 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb46b7fe700 1 -- 172.21.15.7:0/341608782 learned_addr learned my addr 172.21.15.7:0/341608782 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb46bfff700 1 --2- 172.21.15.7:0/341608782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c063db0 0x7fb46c0b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb46a7fc700 1 -- 172.21.15.7:0/341608782 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540723148 0 0) 0x7fb46c0ad300 con 0x7fb46c155650 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.882+0000 7fb46a7fc700 1 -- 172.21.15.7:0/341608782 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb454003570 con 0x7fb46c155650 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46affd700 1 -- 172.21.15.7:0/341608782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c063db0 msgr2=0x7fb46c0b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46affd700 1 --2- 172.21.15.7:0/341608782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c063db0 0x7fb46c0b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46affd700 1 -- 172.21.15.7:0/341608782 >> v1:172.21.15.7:6789/0 conn(0x7fb46c155650 legacy=0x7fb46c0ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46affd700 1 -- 172.21.15.7:0/341608782 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0ad020 con 0x7fb46c069d60 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46bfff700 1 --2- 172.21.15.7:0/341608782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c063db0 0x7fb46c0b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46affd700 1 --2- 172.21.15.7:0/341608782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c0b2b70 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb4600098e0 tx=0x7fb4600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6863a10728602af server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46a7fc700 1 -- 172.21.15.7:0/341608782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46000b640 con 0x7fb46c069d60 2024-01-11T07:13:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46a7fc700 1 -- 172.21.15.7:0/341608782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46000b7a0 con 0x7fb46c069d60 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb46a7fc700 1 -- 172.21.15.7:0/341608782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46000bb40 con 0x7fb46c069d60 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb47208b700 1 -- 172.21.15.7:0/341608782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 msgr2=0x7fb46c0b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.883+0000 7fb47208b700 1 --2- 172.21.15.7:0/341608782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c0b2b70 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb4600098e0 tx=0x7fb4600091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.884+0000 7fb47208b700 1 -- 172.21.15.7:0/341608782 shutdown_connections 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.884+0000 7fb47208b700 1 --2- 172.21.15.7:0/341608782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c063db0 0x7fb46c0b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.884+0000 7fb47208b700 1 --2- 172.21.15.7:0/341608782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c0b2b70 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.884+0000 7fb47208b700 1 -- 172.21.15.7:0/341608782 >> 172.21.15.7:0/341608782 conn(0x7fb46c0cd910 msgr2=0x7fb46c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.884+0000 7fb47208b700 1 -- 172.21.15.7:0/341608782 shutdown_connections 2024-01-11T07:13:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.884+0000 7fb47208b700 1 -- 172.21.15.7:0/341608782 wait complete. 2024-01-11T07:13:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.885+0000 7fb47208b700 1 Processor -- start 2024-01-11T07:13:11.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.885+0000 7fb47208b700 1 -- start start 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb47208b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c063db0 0x7fb46c168db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb47208b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb47208b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c155650 0x7fb46c153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb47208b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c157a10 con 0x7fb46c069d60 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb47208b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c157120 con 0x7fb46c063db0 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb47208b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c168a70 con 0x7fb46c155650 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33380/0 (socket says 172.21.15.7:33380) 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 -- 172.21.15.7:0/1295221782 learned_addr learned my addr 172.21.15.7:0/1295221782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46b7fe700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c063db0 0x7fb46c168db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46bfff700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c155650 0x7fb46c153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 -- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c155650 msgr2=0x7fb46c153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c155650 0x7fb46c153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 -- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c063db0 msgr2=0x7fb46c168db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c063db0 0x7fb46c168db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 -- 172.21.15.7:0/1295221782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460009420 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.886+0000 7fb46affd700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c153aa0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb46c06cfd0 tx=0x7fb460017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.887+0000 7fb4537fe700 1 -- 172.21.15.7:0/1295221782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46001bda0 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.887+0000 7fb4537fe700 1 -- 172.21.15.7:0/1295221782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460011420 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.887+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c1573c0 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.887+0000 7fb4537fe700 1 -- 172.21.15.7:0/1295221782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4600117e0 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.887+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c1521e0 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.888+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c04c3c0 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.888+0000 7fb4537fe700 1 -- 172.21.15.7:0/1295221782 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb460011b50 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.889+0000 7fb4537fe700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb458066f30 0x7fb4580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.889+0000 7fb4537fe700 1 -- 172.21.15.7:0/1295221782 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb4600859a0 con 0x7fb46c069d60 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.889+0000 7fb46b7fe700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb458066f30 0x7fb4580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.889+0000 7fb46b7fe700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb458066f30 0x7fb4580693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb46c154670 tx=0x7fb45c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:11.893+0000 7fb4537fe700 1 -- 172.21.15.7:0/1295221782 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb460052e70 con 0x7fb46c069d60 2024-01-11T07:13:12.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.143+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb46c157b50 con 0x7fb46c069d60 2024-01-11T07:13:12.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.143+0000 7fb4537fe700 1 -- 172.21.15.7:0/1295221782 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fb460015de0 con 0x7fb46c069d60 2024-01-11T07:13:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:12.143 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5882,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4073188648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4073188648},{"type":"v1","addr":"172.21.15.113:6836","nonce":4073188648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1245691399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1245691399},{"type":"v1","addr":"172.21.15.113:6837","nonce":1245691399}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2698040945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":2698040945},{"type":"v1","addr":"172.21.15.7:6834","nonce":2698040945}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53},{"gid":6996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/18962714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":18962714},{"type":"v1","addr":"172.21.15.7:6835","nonce":18962714}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[]} 2024-01-11T07:13:12.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.145+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb458066f30 msgr2=0x7fb4580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.145+0000 7fb47208b700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb458066f30 0x7fb4580693e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb46c154670 tx=0x7fb45c009040 comp rx=0 tx=0).stop 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.145+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 msgr2=0x7fb46c153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.145+0000 7fb47208b700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c153aa0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb46c06cfd0 tx=0x7fb460017000 comp rx=0 tx=0).stop 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.146+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 shutdown_connections 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.146+0000 7fb47208b700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb458066f30 0x7fb4580693e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.146+0000 7fb47208b700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c155650 0x7fb46c153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.146+0000 7fb47208b700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c063db0 0x7fb46c168db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.146+0000 7fb47208b700 1 --2- 172.21.15.7:0/1295221782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46c069d60 0x7fb46c153aa0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.146+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 >> 172.21.15.7:0/1295221782 conn(0x7fb46c0cd910 msgr2=0x7fb46c06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.146+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 shutdown_connections 2024-01-11T07:13:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.146+0000 7fb47208b700 1 -- 172.21.15.7:0/1295221782 wait complete. 2024-01-11T07:13:12.147 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 59 2024-01-11T07:13:12.159 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:13:12.159 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:13:12.159 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:13:12.159 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:13:12.160 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:13:12.160 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:13:12.160 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:13:12.160 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:13:12.160 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:13:12.160 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:13:12.161 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:13:12.161 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:13:12.161 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:13:12.161 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:13:12.161 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:13:12.161 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:13:12.168 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:13:12.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:13:12.169 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:13:12.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:13:12.172 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:13:12.172 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:13:12.173 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:13:12.173 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:13:12.211 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:13:12.211 DEBUG:teuthology.parallel:result is None 2024-01-11T07:13:12.212 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:13:12.212 DEBUG:teuthology.parallel:result is None 2024-01-11T07:13:12.214 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:13:12.214 DEBUG:teuthology.parallel:result is None 2024-01-11T07:13:12.215 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:13:12.215 DEBUG:teuthology.parallel:result is None 2024-01-11T07:13:12.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:13:12.300 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:13:12.300 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:13:12.301 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:13:12.301 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:13:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.357+0000 7f8987735700 1 Processor -- start 2024-01-11T07:13:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.357+0000 7f8987735700 1 -- start start 2024-01-11T07:13:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.358+0000 7f8987735700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f898006afc0 0x7f898015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.358+0000 7f8987735700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898015b770 con 0x7f898006dd00 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.358+0000 7f8987735700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898015abf0 con 0x7f898006e0d0 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.358+0000 7f8987735700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898015aed0 con 0x7f898006afc0 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.358+0000 7f89854d1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f898006dd00 0x7f89800c98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47542/0 (socket says 172.21.15.7:47542) 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.358+0000 7f89854d1700 1 -- 172.21.15.7:0/356803453 learned_addr learned my addr 172.21.15.7:0/356803453 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.358+0000 7f8985cd2700 1 --2- 172.21.15.7:0/356803453 >> v2:172.21.15.113:3300/0 conn(0x7f898006afc0 0x7f898015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8977fff700 1 -- 172.21.15.7:0/356803453 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213239418 0 0) 0x7f898015b770 con 0x7f898006dd00 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8977fff700 1 -- 172.21.15.7:0/356803453 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f896c003570 con 0x7f898006dd00 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8985cd2700 1 -- 172.21.15.7:0/356803453 >> v1:172.21.15.113:6789/0 conn(0x7f898006e0d0 legacy=0x7f8980159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8985cd2700 1 -- 172.21.15.7:0/356803453 >> v1:172.21.15.7:6789/0 conn(0x7f898006dd00 legacy=0x7f89800c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8977fff700 1 -- 172.21.15.7:0/356803453 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453444131 0 0) 0x7f898015abf0 con 0x7f898006e0d0 2024-01-11T07:13:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8985cd2700 1 -- 172.21.15.7:0/356803453 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898015b1b0 con 0x7f898006afc0 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8985cd2700 1 --2- 172.21.15.7:0/356803453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006afc0 0x7f898015a1c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8978009e60 tx=0x7f8978009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee173857b48ed626 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8977fff700 1 -- 172.21.15.7:0/356803453 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897800e070 con 0x7f898006afc0 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8977fff700 1 -- 172.21.15.7:0/356803453 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897800bb60 con 0x7f898006afc0 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.359+0000 7f8977fff700 1 -- 172.21.15.7:0/356803453 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978012430 con 0x7f898006afc0 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.360+0000 7f8987735700 1 -- 172.21.15.7:0/356803453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006afc0 msgr2=0x7f898015a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.360+0000 7f8987735700 1 --2- 172.21.15.7:0/356803453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006afc0 0x7f898015a1c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8978009e60 tx=0x7f8978009780 comp rx=0 tx=0).stop 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.360+0000 7f8987735700 1 -- 172.21.15.7:0/356803453 shutdown_connections 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.360+0000 7f8987735700 1 --2- 172.21.15.7:0/356803453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006afc0 0x7f898015a1c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.360+0000 7f8987735700 1 -- 172.21.15.7:0/356803453 >> 172.21.15.7:0/356803453 conn(0x7f89800680e0 msgr2=0x7f89800cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.360+0000 7f8987735700 1 -- 172.21.15.7:0/356803453 shutdown_connections 2024-01-11T07:13:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.360+0000 7f8987735700 1 -- 172.21.15.7:0/356803453 wait complete. 2024-01-11T07:13:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.361+0000 7f8987735700 1 Processor -- start 2024-01-11T07:13:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.362+0000 7f8987735700 1 -- start start 2024-01-11T07:13:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f8987735700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f898006afc0 0x7f89800a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f8987735700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f898006dd00 0x7f89800af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f8987735700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006e0d0 0x7f89800af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f8987735700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800a8330 con 0x7f898006afc0 2024-01-11T07:13:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f8987735700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800a67f0 con 0x7f898006e0d0 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f8987735700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898016b690 con 0x7f898006dd00 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f89854d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f898006afc0 0x7f89800a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f89854d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f898006afc0 0x7f89800a6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33420/0 (socket says 172.21.15.7:33420) 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f89854d1700 1 -- 172.21.15.7:0/1187890791 learned_addr learned my addr 172.21.15.7:0/1187890791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f8984cd0700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f898006dd00 0x7f89800af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f8985cd2700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006e0d0 0x7f89800af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f89854d1700 1 -- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f898006dd00 msgr2=0x7f89800af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f89854d1700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f898006dd00 0x7f89800af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f89854d1700 1 -- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006e0d0 msgr2=0x7f89800af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f89854d1700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006e0d0 0x7f89800af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.363+0000 7f89854d1700 1 -- 172.21.15.7:0/1187890791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8978009a20 con 0x7f898006afc0 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.364+0000 7f89854d1700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f898006afc0 0x7f89800a6e50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f897c00c790 tx=0x7f897c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.364+0000 7f89757fa700 1 -- 172.21.15.7:0/1187890791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c005b30 con 0x7f898006afc0 2024-01-11T07:13:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.364+0000 7f89757fa700 1 -- 172.21.15.7:0/1187890791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c005c90 con 0x7f898006afc0 2024-01-11T07:13:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.364+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898016b080 con 0x7f898006afc0 2024-01-11T07:13:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.364+0000 7f89757fa700 1 -- 172.21.15.7:0/1187890791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c007320 con 0x7f898006afc0 2024-01-11T07:13:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.364+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898016aa60 con 0x7f898006afc0 2024-01-11T07:13:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.365+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898004c3c0 con 0x7f898006afc0 2024-01-11T07:13:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.365+0000 7f89757fa700 1 -- 172.21.15.7:0/1187890791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f897c007690 con 0x7f898006afc0 2024-01-11T07:13:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.366+0000 7f89757fa700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8964066f30 0x7f89640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.366+0000 7f89757fa700 1 -- 172.21.15.7:0/1187890791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f897c0836e0 con 0x7f898006afc0 2024-01-11T07:13:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.366+0000 7f8984cd0700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8964066f30 0x7f89640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.366+0000 7f8984cd0700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8964066f30 0x7f89640693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f89800aca30 tx=0x7f8970007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:12.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.370+0000 7f89757fa700 1 -- 172.21.15.7:0/1187890791 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f897c050bb0 con 0x7f898006afc0 2024-01-11T07:13:12.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-01-11T07:13:12.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-01-11T07:13:12.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-01-11T07:13:12.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-01-11T07:13:12.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:12.592+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f898015b490 con 0x7f898006afc0 2024-01-11T07:13:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.655+0000 7f89757fa700 1 -- 172.21.15.7:0/1187890791 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v131) v1 ==== 97+0+0 (secure 0 0 0) 0x7f897c017080 con 0x7f898006afc0 2024-01-11T07:13:13.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8964066f30 msgr2=0x7f89640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:13.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8964066f30 0x7f89640693e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f89800aca30 tx=0x7f8970007ec0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f898006afc0 msgr2=0x7f89800a6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f898006afc0 0x7f89800a6e50 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f897c00c790 tx=0x7f897c007df0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 shutdown_connections 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8964066f30 0x7f89640693e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f898006e0d0 0x7f89800af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f898006afc0 0x7f89800a6e50 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.658+0000 7f8987735700 1 --2- 172.21.15.7:0/1187890791 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f898006dd00 0x7f89800af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.659+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 >> 172.21.15.7:0/1187890791 conn(0x7f89800680e0 msgr2=0x7f89800adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.659+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 shutdown_connections 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.659+0000 7f8987735700 1 -- 172.21.15.7:0/1187890791 wait complete. 2024-01-11T07:13:13.659 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:13:13.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.811+0000 7fa379dec700 1 Processor -- start 2024-01-11T07:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.811+0000 7fa379dec700 1 -- start start 2024-01-11T07:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa379dec700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa374158280 0x7fa3740b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa379dec700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa37415aa80 0x7fa3740b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa379dec700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa374159300 con 0x7fa3740cafb0 2024-01-11T07:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa379dec700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3740cf000 con 0x7fa374158280 2024-01-11T07:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa379dec700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740cf260 con 0x7fa37415aa80 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa372ffd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa374158280 0x7fa3740b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa373fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa37415aa80 0x7fa3740b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa372ffd700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa374158280 0x7fa3740b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:34802/0 (socket says 172.21.15.7:34802) 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa373fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa37415aa80 0x7fa3740b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:54194/0 (socket says 172.21.15.7:54194) 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa3737fe700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa3740cafb0 0x7fa3740ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:45288/0 (socket says 172.21.15.7:45288) 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.812+0000 7fa372ffd700 1 -- 172.21.15.7:0/2449948301 learned_addr learned my addr 172.21.15.7:0/2449948301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa373fff700 1 -- 172.21.15.7:0/2449948301 >> v2:172.21.15.113:3300/0 conn(0x7fa374158280 msgr2=0x7fa3740b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa373fff700 1 --2- 172.21.15.7:0/2449948301 >> v2:172.21.15.113:3300/0 conn(0x7fa374158280 0x7fa3740b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa373fff700 1 -- 172.21.15.7:0/2449948301 >> v1:172.21.15.113:6789/0 conn(0x7fa3740cafb0 legacy=0x7fa3740ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa373fff700 1 -- 172.21.15.7:0/2449948301 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740cf540 con 0x7fa37415aa80 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa372ffd700 1 --2- 172.21.15.7:0/2449948301 >> v2:172.21.15.113:3300/0 conn(0x7fa374158280 0x7fa3740b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa3727fc700 1 -- 172.21.15.7:0/2449948301 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234639074 0 0) 0x7fa374159300 con 0x7fa3740cafb0 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa373fff700 1 --2- 172.21.15.7:0/2449948301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa37415aa80 0x7fa3740b0970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009e60 tx=0x7fa36c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f5d85916c6378fa server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa3727fc700 1 -- 172.21.15.7:0/2449948301 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c00e070 con 0x7fa37415aa80 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa3727fc700 1 -- 172.21.15.7:0/2449948301 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c00bb60 con 0x7fa37415aa80 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.813+0000 7fa3727fc700 1 -- 172.21.15.7:0/2449948301 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c012430 con 0x7fa37415aa80 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.814+0000 7fa379dec700 1 -- 172.21.15.7:0/2449948301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa37415aa80 msgr2=0x7fa3740b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.814+0000 7fa379dec700 1 --2- 172.21.15.7:0/2449948301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa37415aa80 0x7fa3740b0970 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009e60 tx=0x7fa36c009780 comp rx=0 tx=0).stop 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.814+0000 7fa379dec700 1 -- 172.21.15.7:0/2449948301 shutdown_connections 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.814+0000 7fa379dec700 1 --2- 172.21.15.7:0/2449948301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa37415aa80 0x7fa3740b0970 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.814+0000 7fa379dec700 1 --2- 172.21.15.7:0/2449948301 >> v2:172.21.15.113:3300/0 conn(0x7fa374158280 0x7fa3740b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.814+0000 7fa379dec700 1 -- 172.21.15.7:0/2449948301 >> 172.21.15.7:0/2449948301 conn(0x7fa374071100 msgr2=0x7fa37406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.814+0000 7fa379dec700 1 -- 172.21.15.7:0/2449948301 shutdown_connections 2024-01-11T07:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.814+0000 7fa379dec700 1 -- 172.21.15.7:0/2449948301 wait complete. 2024-01-11T07:13:13.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.815+0000 7fa379dec700 1 Processor -- start 2024-01-11T07:13:13.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.815+0000 7fa379dec700 1 -- start start 2024-01-11T07:13:13.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa379dec700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3740cafb0 0x7fa374068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:13.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa379dec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa374158280 0x7fa3740b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa379dec700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37415aa80 0x7fa3740ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa379dec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740ae7b0 con 0x7fa374158280 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa379dec700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740b2820 con 0x7fa37415aa80 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa379dec700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740ca820 con 0x7fa3740cafb0 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa372ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa374158280 0x7fa3740b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa372ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa374158280 0x7fa3740b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33434/0 (socket says 172.21.15.7:33434) 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa372ffd700 1 -- 172.21.15.7:0/3827301249 learned_addr learned my addr 172.21.15.7:0/3827301249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa3737fe700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3740cafb0 0x7fa374068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa373fff700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37415aa80 0x7fa3740ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.816+0000 7fa372ffd700 1 -- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3740cafb0 msgr2=0x7fa374068b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa372ffd700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3740cafb0 0x7fa374068b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa372ffd700 1 -- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37415aa80 msgr2=0x7fa3740ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa372ffd700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37415aa80 0x7fa3740ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa372ffd700 1 -- 172.21.15.7:0/3827301249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0094f0 con 0x7fa374158280 2024-01-11T07:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa372ffd700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa374158280 0x7fa3740b22e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00ec80 tx=0x7fa35c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:13.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa36b7fe700 1 -- 172.21.15.7:0/3827301249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00c850 con 0x7fa374158280 2024-01-11T07:13:13.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa36b7fe700 1 -- 172.21.15.7:0/3827301249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c00c9b0 con 0x7fa374158280 2024-01-11T07:13:13.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c009a20 con 0x7fa374158280 2024-01-11T07:13:13.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa36b7fe700 1 -- 172.21.15.7:0/3827301249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00cd30 con 0x7fa374158280 2024-01-11T07:13:13.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.817+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3740b2d60 con 0x7fa374158280 2024-01-11T07:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.818+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3740ae8f0 con 0x7fa374158280 2024-01-11T07:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.819+0000 7fa36b7fe700 1 -- 172.21.15.7:0/3827301249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa35c013680 con 0x7fa374158280 2024-01-11T07:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.819+0000 7fa36b7fe700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa354066f30 0x7fa3540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.819+0000 7fa36b7fe700 1 -- 172.21.15.7:0/3827301249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa35c084860 con 0x7fa374158280 2024-01-11T07:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.820+0000 7fa3737fe700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa354066f30 0x7fa3540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.820+0000 7fa3737fe700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa354066f30 0x7fa3540693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa364007fd0 tx=0x7fa364007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:13.824+0000 7fa36b7fe700 1 -- 172.21.15.7:0/3827301249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa35c051e20 con 0x7fa374158280 2024-01-11T07:13:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.115+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa37404c3c0 con 0x7fa374158280 2024-01-11T07:13:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.116+0000 7fa36b7fe700 1 -- 172.21.15.7:0/3827301249 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa35c017090 con 0x7fa374158280 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:13:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:13:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:13:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:13:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:13:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:13:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:13:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:13:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:13:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:13:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:13:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:13:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:13:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:13:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:14.132 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa354066f30 msgr2=0x7fa3540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa354066f30 0x7fa3540693e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa364007fd0 tx=0x7fa364007d50 comp rx=0 tx=0).stop 2024-01-11T07:13:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa374158280 msgr2=0x7fa3740b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa374158280 0x7fa3740b22e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00ec80 tx=0x7fa35c009250 comp rx=0 tx=0).stop 2024-01-11T07:13:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 shutdown_connections 2024-01-11T07:13:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa354066f30 0x7fa3540693e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa37415aa80 0x7fa3740ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3740cafb0 0x7fa374068b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 --2- 172.21.15.7:0/3827301249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa374158280 0x7fa3740b22e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 >> 172.21.15.7:0/3827301249 conn(0x7fa374071100 msgr2=0x7fa3740ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 shutdown_connections 2024-01-11T07:13:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.118+0000 7fa379dec700 1 -- 172.21.15.7:0/3827301249 wait complete. 2024-01-11T07:13:14.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.268+0000 7fabbe8fe700 1 Processor -- start 2024-01-11T07:13:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.268+0000 7fabbe8fe700 1 -- start start 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.269+0000 7fabbe8fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabb8158280 0x7fabb80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.269+0000 7fabbe8fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb815aa80 0x7fabb80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.269+0000 7fabbe8fe700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb8159300 con 0x7fabb80cafb0 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.269+0000 7fabbe8fe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb80cf000 con 0x7fabb8158280 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.269+0000 7fabbe8fe700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80cf260 con 0x7fabb815aa80 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.269+0000 7fabb77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabb8158280 0x7fabb80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabb77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabb8158280 0x7fabb80b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33442/0 (socket says 172.21.15.7:33442) 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabbce9b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb815aa80 0x7fabb80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabb7fff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fabb80cafb0 0x7fabb80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:45298/0 (socket says 172.21.15.7:45298) 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabb77fe700 1 -- 172.21.15.7:0/896673664 learned_addr learned my addr 172.21.15.7:0/896673664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabb77fe700 1 -- 172.21.15.7:0/896673664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb815aa80 msgr2=0x7fabb80b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabb77fe700 1 --2- 172.21.15.7:0/896673664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb815aa80 0x7fabb80b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabb77fe700 1 -- 172.21.15.7:0/896673664 >> v1:172.21.15.113:6789/0 conn(0x7fabb80cafb0 legacy=0x7fabb80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabb77fe700 1 -- 172.21.15.7:0/896673664 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80cf540 con 0x7fabb8158280 2024-01-11T07:13:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.270+0000 7fabb77fe700 1 --2- 172.21.15.7:0/896673664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8158280 0x7fabb80b0430 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fabac0099b0 tx=0x7fabac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a920cdbaace1191 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabb6ffd700 1 -- 172.21.15.7:0/896673664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac009ea0 con 0x7fabb8158280 2024-01-11T07:13:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabb6ffd700 1 -- 172.21.15.7:0/896673664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac00b860 con 0x7fabb8158280 2024-01-11T07:13:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabb6ffd700 1 -- 172.21.15.7:0/896673664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00bc00 con 0x7fabb8158280 2024-01-11T07:13:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabbe8fe700 1 -- 172.21.15.7:0/896673664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8158280 msgr2=0x7fabb80b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/896673664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8158280 0x7fabb80b0430 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fabac0099b0 tx=0x7fabac009250 comp rx=0 tx=0).stop 2024-01-11T07:13:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabbe8fe700 1 -- 172.21.15.7:0/896673664 shutdown_connections 2024-01-11T07:13:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/896673664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb815aa80 0x7fabb80b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/896673664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8158280 0x7fabb80b0430 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.272+0000 7fabbe8fe700 1 -- 172.21.15.7:0/896673664 >> 172.21.15.7:0/896673664 conn(0x7fabb8071100 msgr2=0x7fabb806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.273+0000 7fabbe8fe700 1 -- 172.21.15.7:0/896673664 shutdown_connections 2024-01-11T07:13:14.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.273+0000 7fabbe8fe700 1 -- 172.21.15.7:0/896673664 wait complete. 2024-01-11T07:13:14.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.274+0000 7fabbe8fe700 1 Processor -- start 2024-01-11T07:13:14.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.274+0000 7fabbe8fe700 1 -- start start 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabbe8fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb80cafb0 0x7fabb81555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabbe8fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb8158280 0x7fabb80aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabbe8fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb815aa80 0x7fabb80af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabbe8fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb8155b10 con 0x7fabb80cafb0 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabbe8fe700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80c4f40 con 0x7fabb815aa80 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabbe8fe700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80c5220 con 0x7fabb8158280 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb80cafb0 0x7fabb81555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb80cafb0 0x7fabb81555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33458/0 (socket says 172.21.15.7:33458) 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 -- 172.21.15.7:0/1980132233 learned_addr learned my addr 172.21.15.7:0/1980132233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb77fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb8158280 0x7fabb80aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabbce9b700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb815aa80 0x7fabb80af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 -- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb8158280 msgr2=0x7fabb80aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb8158280 0x7fabb80aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 -- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb815aa80 msgr2=0x7fabb80af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb815aa80 0x7fabb80af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 -- 172.21.15.7:0/1980132233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0094f0 con 0x7fabb80cafb0 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.275+0000 7fabb7fff700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb80cafb0 0x7fabb81555d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7faba800c7c0 tx=0x7faba8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.276+0000 7fab9ffff700 1 -- 172.21.15.7:0/1980132233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005a70 con 0x7fabb80cafb0 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.276+0000 7fab9ffff700 1 -- 172.21.15.7:0/1980132233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba8005bd0 con 0x7fabb80cafb0 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.276+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb80c5500 con 0x7fabb80cafb0 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.276+0000 7fab9ffff700 1 -- 172.21.15.7:0/1980132233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80070d0 con 0x7fabb80cafb0 2024-01-11T07:13:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.276+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb8168030 con 0x7fabb80cafb0 2024-01-11T07:13:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.277+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb804c3c0 con 0x7fabb80cafb0 2024-01-11T07:13:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.277+0000 7fab9ffff700 1 -- 172.21.15.7:0/1980132233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faba8007440 con 0x7fabb80cafb0 2024-01-11T07:13:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.278+0000 7fab9ffff700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faba4066f30 0x7faba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.278+0000 7fab9ffff700 1 -- 172.21.15.7:0/1980132233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4853+0+0 (secure 0 0 0) 0x7faba80835a0 con 0x7fabb80cafb0 2024-01-11T07:13:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.278+0000 7fabb77fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faba4066f30 0x7faba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.278+0000 7fabb77fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faba4066f30 0x7faba40693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fabac0099b0 tx=0x7fabac017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.282+0000 7fab9ffff700 1 -- 172.21.15.7:0/1980132233 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faba8050b60 con 0x7fabb80cafb0 2024-01-11T07:13:14.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.489+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabb80cf820 con 0x7fabb80cafb0 2024-01-11T07:13:14.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.490+0000 7fab9ffff700 1 -- 172.21.15.7:0/1980132233 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+12097 (secure 0 0 0) 0x7faba8017090 con 0x7fabb80cafb0 2024-01-11T07:13:14.490 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:14.490 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:13:13.652718+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:14.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faba4066f30 msgr2=0x7faba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faba4066f30 0x7faba40693e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fabac0099b0 tx=0x7fabac017000 comp rx=0 tx=0).stop 2024-01-11T07:13:14.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb80cafb0 msgr2=0x7fabb81555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb80cafb0 0x7fabb81555d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7faba800c7c0 tx=0x7faba8007f30 comp rx=0 tx=0).stop 2024-01-11T07:13:14.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 shutdown_connections 2024-01-11T07:13:14.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faba4066f30 0x7faba40693e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabb815aa80 0x7fabb80af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb80cafb0 0x7fabb81555d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 --2- 172.21.15.7:0/1980132233 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabb8158280 0x7fabb80aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 >> 172.21.15.7:0/1980132233 conn(0x7fabb8071100 msgr2=0x7fabb8155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:14.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 shutdown_connections 2024-01-11T07:13:14.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.492+0000 7fabbe8fe700 1 -- 172.21.15.7:0/1980132233 wait complete. 2024-01-11T07:13:14.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:13:14.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:13:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.640+0000 7f7f87d9f700 1 Processor -- start 2024-01-11T07:13:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.640+0000 7f7f87d9f700 1 -- start start 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f87d9f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f8006dd00 0x7f7f800c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f87d9f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f87d9f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8015b770 con 0x7f7f80159ff0 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f87d9f700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8015abf0 con 0x7f7f8006dd00 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f87d9f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8015aed0 con 0x7f7f8006e0d0 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f8633c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f80159ff0 0x7f7f800c7e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47548/0 (socket says 172.21.15.7:47548) 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f8633c700 1 -- 172.21.15.7:0/148435098 learned_addr learned my addr 172.21.15.7:0/148435098 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f85b3b700 1 --2- 172.21.15.7:0/148435098 >> v2:172.21.15.113:3300/0 conn(0x7f7f8006dd00 0x7f7f800c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.641+0000 7f7f8533a700 1 --2- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f84b39700 1 -- 172.21.15.7:0/148435098 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525649412 0 0) 0x7f7f8015b770 con 0x7f7f80159ff0 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f84b39700 1 -- 172.21.15.7:0/148435098 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f6c003570 con 0x7f7f80159ff0 2024-01-11T07:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f85b3b700 1 -- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 msgr2=0x7f7f80159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f85b3b700 1 --2- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f85b3b700 1 -- 172.21.15.7:0/148435098 >> v1:172.21.15.7:6789/0 conn(0x7f7f80159ff0 legacy=0x7f7f800c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f85b3b700 1 -- 172.21.15.7:0/148435098 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8015b1b0 con 0x7f7f8006dd00 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f8533a700 1 --2- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f85b3b700 1 --2- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f8006dd00 0x7f7f800c98d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0098e0 tx=0x7f7f7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b7171a57bf3eebc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f84b39700 1 -- 172.21.15.7:0/148435098 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c011630 con 0x7f7f8006dd00 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f84b39700 1 -- 172.21.15.7:0/148435098 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c011790 con 0x7f7f8006dd00 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.642+0000 7f7f84b39700 1 -- 172.21.15.7:0/148435098 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c011b30 con 0x7f7f8006dd00 2024-01-11T07:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.643+0000 7f7f87d9f700 1 -- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f8006dd00 msgr2=0x7f7f800c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.643+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f8006dd00 0x7f7f800c98d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0098e0 tx=0x7f7f7c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.643+0000 7f7f87d9f700 1 -- 172.21.15.7:0/148435098 shutdown_connections 2024-01-11T07:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.643+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f8006dd00 0x7f7f800c98d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.643+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/148435098 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.643+0000 7f7f87d9f700 1 -- 172.21.15.7:0/148435098 >> 172.21.15.7:0/148435098 conn(0x7f7f800680e0 msgr2=0x7f7f800cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.643+0000 7f7f87d9f700 1 -- 172.21.15.7:0/148435098 shutdown_connections 2024-01-11T07:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.643+0000 7f7f87d9f700 1 -- 172.21.15.7:0/148435098 wait complete. 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.644+0000 7f7f87d9f700 1 Processor -- start 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.644+0000 7f7f87d9f700 1 -- start start 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f87d9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8006dd00 0x7f7f8016b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f87d9f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f87d9f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f80159ff0 0x7f7f80147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f87d9f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8014a150 con 0x7f7f8006dd00 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f87d9f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f80149540 con 0x7f7f80159ff0 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f87d9f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f80149820 con 0x7f7f8006e0d0 2024-01-11T07:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f85b3b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8006dd00 0x7f7f8016b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f85b3b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8006dd00 0x7f7f8016b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33474/0 (socket says 172.21.15.7:33474) 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.645+0000 7f7f85b3b700 1 -- 172.21.15.7:0/3731645386 learned_addr learned my addr 172.21.15.7:0/3731645386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f8633c700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f80159ff0 0x7f7f80147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f8533a700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f85b3b700 1 -- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 msgr2=0x7f7f80147920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f85b3b700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80147920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f85b3b700 1 -- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f80159ff0 msgr2=0x7f7f80147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f85b3b700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f80159ff0 0x7f7f80147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f85b3b700 1 -- 172.21.15.7:0/3731645386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009420 con 0x7f7f8006dd00 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f8633c700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f80159ff0 0x7f7f80147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f85b3b700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8006dd00 0x7f7f8016b300 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7f8006d820 tx=0x7f7f7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f71ffb700 1 -- 172.21.15.7:0/3731645386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c01f070 con 0x7f7f8006dd00 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f71ffb700 1 -- 172.21.15.7:0/3731645386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c000bc0 con 0x7f7f8006dd00 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c009420 con 0x7f7f8006dd00 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f71ffb700 1 -- 172.21.15.7:0/3731645386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c011b30 con 0x7f7f8006dd00 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.646+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f80149de0 con 0x7f7f8006dd00 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.647+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8004c3c0 con 0x7f7f8006dd00 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.648+0000 7f7f71ffb700 1 -- 172.21.15.7:0/3731645386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7f7c023070 con 0x7f7f8006dd00 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.648+0000 7f7f71ffb700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.648+0000 7f7f71ffb700 1 -- 172.21.15.7:0/3731645386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7f7c0854e0 con 0x7f7f8006dd00 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.649+0000 7f7f8533a700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.649+0000 7f7f8533a700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f64066f30 0x7f7f640693e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006f00 tx=0x7f7f74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:14.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.653+0000 7f7f71ffb700 1 -- 172.21.15.7:0/3731645386 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f7c052b50 con 0x7f7f8006dd00 2024-01-11T07:13:14.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.855+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f8015b490 con 0x7f7f8006dd00 2024-01-11T07:13:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.855+0000 7f7f71ffb700 1 -- 172.21.15.7:0/3731645386 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f7c02a020 con 0x7f7f8006dd00 2024-01-11T07:13:14.855 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:13:14.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.857+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f64066f30 msgr2=0x7f7f640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.857+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f64066f30 0x7f7f640693e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006f00 tx=0x7f7f74008040 comp rx=0 tx=0).stop 2024-01-11T07:13:14.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.857+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8006dd00 msgr2=0x7f7f8016b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:14.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.857+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8006dd00 0x7f7f8016b300 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7f8006d820 tx=0x7f7f7c017000 comp rx=0 tx=0).stop 2024-01-11T07:13:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.858+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 shutdown_connections 2024-01-11T07:13:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.858+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f64066f30 0x7f7f640693e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.858+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f80159ff0 0x7f7f80147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.858+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f8006dd00 0x7f7f8016b300 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.858+0000 7f7f87d9f700 1 --2- 172.21.15.7:0/3731645386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f8006e0d0 0x7f7f80147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.858+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 >> 172.21.15.7:0/3731645386 conn(0x7f7f800680e0 msgr2=0x7f7f800cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.858+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 shutdown_connections 2024-01-11T07:13:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:14.858+0000 7f7f87d9f700 1 -- 172.21.15.7:0/3731645386 wait complete. 2024-01-11T07:13:14.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:13:15.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.003+0000 7fd1a1a9e700 1 Processor -- start 2024-01-11T07:13:15.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.003+0000 7fd1a1a9e700 1 -- start start 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd1a1a9e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd19c063230 0x7fd19c15e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd1a1a9e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063600 0x7fd19c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd1a1a9e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c15acd0 con 0x7fd19c15a870 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd1a1a9e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c15e950 con 0x7fd19c063230 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd1a1a9e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c15df80 con 0x7fd19c063600 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19affd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063600 0x7fd19c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19b7fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd19c063230 0x7fd19c15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19affd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063600 0x7fd19c15a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:54246/0 (socket says 172.21.15.7:54246) 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19b7fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd19c063230 0x7fd19c15e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:34842/0 (socket says 172.21.15.7:34842) 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19affd700 1 -- 172.21.15.7:0/2345725566 learned_addr learned my addr 172.21.15.7:0/2345725566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19bfff700 1 --1- 172.21.15.7:0/2345725566 >> v1:172.21.15.113:6789/0 conn(0x7fd19c15a870 0x7fd19c06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:45304/0 (socket says 172.21.15.7:45304) 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19affd700 1 -- 172.21.15.7:0/2345725566 >> v2:172.21.15.113:3300/0 conn(0x7fd19c063230 msgr2=0x7fd19c15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19affd700 1 --2- 172.21.15.7:0/2345725566 >> v2:172.21.15.113:3300/0 conn(0x7fd19c063230 0x7fd19c15e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19affd700 1 -- 172.21.15.7:0/2345725566 >> v1:172.21.15.113:6789/0 conn(0x7fd19c15a870 legacy=0x7fd19c06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19affd700 1 -- 172.21.15.7:0/2345725566 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0aab70 con 0x7fd19c063600 2024-01-11T07:13:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.004+0000 7fd19a7fc700 1 -- 172.21.15.7:0/2345725566 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168751339 0 0) 0x7fd194001490 con 0x7fd19c15a870 2024-01-11T07:13:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.005+0000 7fd19b7fe700 1 --2- 172.21.15.7:0/2345725566 >> v2:172.21.15.113:3300/0 conn(0x7fd19c063230 0x7fd19c15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.005+0000 7fd19affd700 1 --2- 172.21.15.7:0/2345725566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063600 0x7fd19c15a330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd1900099b0 tx=0x7fd190009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f05031de00b6cccf server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.005+0000 7fd19a7fc700 1 -- 172.21.15.7:0/2345725566 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190009ea0 con 0x7fd19c063600 2024-01-11T07:13:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.005+0000 7fd19a7fc700 1 -- 172.21.15.7:0/2345725566 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19000b860 con 0x7fd19c063600 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.005+0000 7fd19a7fc700 1 -- 172.21.15.7:0/2345725566 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19000bc00 con 0x7fd19c063600 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.005+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/2345725566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063600 msgr2=0x7fd19c15a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.005+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/2345725566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063600 0x7fd19c15a330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd1900099b0 tx=0x7fd190009250 comp rx=0 tx=0).stop 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.006+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/2345725566 shutdown_connections 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.006+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/2345725566 >> v2:172.21.15.113:3300/0 conn(0x7fd19c063230 0x7fd19c15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.006+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/2345725566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063600 0x7fd19c15a330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.006+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/2345725566 >> 172.21.15.7:0/2345725566 conn(0x7fd19c0cd910 msgr2=0x7fd19c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.006+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/2345725566 shutdown_connections 2024-01-11T07:13:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.006+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/2345725566 wait complete. 2024-01-11T07:13:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.007+0000 7fd1a1a9e700 1 Processor -- start 2024-01-11T07:13:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.007+0000 7fd1a1a9e700 1 -- start start 2024-01-11T07:13:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd1a1a9e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063230 0x7fd19c0adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd1a1a9e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c063600 0x7fd19c0ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd1a1a9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd19c15a870 0x7fd19c0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd1a1a9e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0cb890 con 0x7fd19c15a870 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd1a1a9e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c1498c0 con 0x7fd19c063600 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd1a1a9e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0cac80 con 0x7fd19c063230 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd19c15a870 0x7fd19c0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063230 0x7fd19c0adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19affd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c063600 0x7fd19c0ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19b7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063230 0x7fd19c0adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:54258/0 (socket says 172.21.15.7:54258) 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd19c15a870 0x7fd19c0cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33480/0 (socket says 172.21.15.7:33480) 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19b7fe700 1 -- 172.21.15.7:0/3842534213 learned_addr learned my addr 172.21.15.7:0/3842534213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19affd700 1 -- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063230 msgr2=0x7fd19c0adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19affd700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063230 0x7fd19c0adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19affd700 1 -- 172.21.15.7:0/3842534213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd19c15a870 msgr2=0x7fd19c0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19affd700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd19c15a870 0x7fd19c0cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19affd700 1 -- 172.21.15.7:0/3842534213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c009420 con 0x7fd19c063600 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.008+0000 7fd19b7fe700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063230 0x7fd19c0adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.009+0000 7fd19affd700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c063600 0x7fd19c0ca310 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd19c15d870 tx=0x7fd190017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.009+0000 7fd1837fe700 1 -- 172.21.15.7:0/3842534213 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190000d10 con 0x7fd19c063600 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.009+0000 7fd1837fe700 1 -- 172.21.15.7:0/3842534213 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19000bbe0 con 0x7fd19c063600 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.009+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900094f0 con 0x7fd19c063600 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.009+0000 7fd1837fe700 1 -- 172.21.15.7:0/3842534213 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190011650 con 0x7fd19c063600 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.009+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c0cb240 con 0x7fd19c063600 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.010+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c0cbb70 con 0x7fd19c063600 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.010+0000 7fd1837fe700 1 -- 172.21.15.7:0/3842534213 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd190011a80 con 0x7fd19c063600 2024-01-11T07:13:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.011+0000 7fd1837fe700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd188066f30 0x7fd1880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.011+0000 7fd1837fe700 1 -- 172.21.15.7:0/3842534213 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd190085b60 con 0x7fd19c063600 2024-01-11T07:13:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.011+0000 7fd19b7fe700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd188066f30 0x7fd1880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.012+0000 7fd19b7fe700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd188066f30 0x7fd1880693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd18c011640 tx=0x7fd18c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.016+0000 7fd1837fe700 1 -- 172.21.15.7:0/3842534213 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd190053080 con 0x7fd19c063600 2024-01-11T07:13:15.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.224+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd19c04c3c0 con 0x7fd19c063600 2024-01-11T07:13:15.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.225+0000 7fd1837fe700 1 -- 172.21.15.7:0/3842534213 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v131) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd190015090 con 0x7fd19c063600 2024-01-11T07:13:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:15.225 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:13:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:13:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:13:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:15.225 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:13:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd188066f30 msgr2=0x7fd1880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd188066f30 0x7fd1880693e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd18c011640 tx=0x7fd18c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c063600 msgr2=0x7fd19c0ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c063600 0x7fd19c0ca310 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd19c15d870 tx=0x7fd190017000 comp rx=0 tx=0).stop 2024-01-11T07:13:15.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 shutdown_connections 2024-01-11T07:13:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd188066f30 0x7fd1880693e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd19c15a870 0x7fd19c0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd19c063230 0x7fd19c0adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 --2- 172.21.15.7:0/3842534213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd19c063600 0x7fd19c0ca310 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 >> 172.21.15.7:0/3842534213 conn(0x7fd19c0cd910 msgr2=0x7fd19c157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 shutdown_connections 2024-01-11T07:13:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.227+0000 7fd1a1a9e700 1 -- 172.21.15.7:0/3842534213 wait complete. 2024-01-11T07:13:15.240 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:13:15.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:13:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.377+0000 7f324cf47700 1 Processor -- start 2024-01-11T07:13:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.377+0000 7f324cf47700 1 -- start start 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f324cf47700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248159bf0 0x7f3248069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f324cf47700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3248159fc0 0x7f3248063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f324cf47700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32480c8570 con 0x7f3248063770 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f324cf47700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3248063b80 con 0x7f3248159fc0 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f324cf47700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324806e920 con 0x7f3248159bf0 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f3245d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3248159fc0 0x7f3248063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f3245d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3248159fc0 0x7f3248063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33492/0 (socket says 172.21.15.7:33492) 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f3245d9b700 1 -- 172.21.15.7:0/3593434979 learned_addr learned my addr 172.21.15.7:0/3593434979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f324659c700 1 --2- 172.21.15.7:0/3593434979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248159bf0 0x7f3248069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f3245d9b700 1 -- 172.21.15.7:0/3593434979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248159bf0 msgr2=0x7f3248069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f3245d9b700 1 --2- 172.21.15.7:0/3593434979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248159bf0 0x7f3248069680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f3245d9b700 1 -- 172.21.15.7:0/3593434979 >> v1:172.21.15.113:6789/0 conn(0x7f3248063770 legacy=0x7f324806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.378+0000 7f3245d9b700 1 -- 172.21.15.7:0/3593434979 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324806ec00 con 0x7f3248159fc0 2024-01-11T07:13:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.379+0000 7f3245d9b700 1 --2- 172.21.15.7:0/3593434979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f3248063230 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f32300099b0 tx=0x7f3230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc89ae7b943474e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.379+0000 7f324559a700 1 -- 172.21.15.7:0/3593434979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000b820 con 0x7f3248159fc0 2024-01-11T07:13:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.379+0000 7f324559a700 1 -- 172.21.15.7:0/3593434979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000b980 con 0x7f3248159fc0 2024-01-11T07:13:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.379+0000 7f324559a700 1 -- 172.21.15.7:0/3593434979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000bd20 con 0x7f3248159fc0 2024-01-11T07:13:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.379+0000 7f324cf47700 1 -- 172.21.15.7:0/3593434979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 msgr2=0x7f3248063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.379+0000 7f324cf47700 1 --2- 172.21.15.7:0/3593434979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f3248063230 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f32300099b0 tx=0x7f3230009250 comp rx=0 tx=0).stop 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.380+0000 7f324cf47700 1 -- 172.21.15.7:0/3593434979 shutdown_connections 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.380+0000 7f324cf47700 1 --2- 172.21.15.7:0/3593434979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248159bf0 0x7f3248069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.380+0000 7f324cf47700 1 --2- 172.21.15.7:0/3593434979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f3248063230 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.380+0000 7f324cf47700 1 -- 172.21.15.7:0/3593434979 >> 172.21.15.7:0/3593434979 conn(0x7f32480680a0 msgr2=0x7f32480cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.380+0000 7f324cf47700 1 -- 172.21.15.7:0/3593434979 shutdown_connections 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.380+0000 7f324cf47700 1 -- 172.21.15.7:0/3593434979 wait complete. 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.381+0000 7f324cf47700 1 Processor -- start 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.381+0000 7f324cf47700 1 -- start start 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f324cf47700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248063770 0x7f32480b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f324cf47700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248159bf0 0x7f324815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f324cf47700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f324815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f324cf47700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32480c6fe0 con 0x7f3248159fc0 2024-01-11T07:13:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f324cf47700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248157990 con 0x7f3248159bf0 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f324cf47700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32480c6410 con 0x7f3248063770 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f324815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f324659c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248063770 0x7f32480b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3245d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248159bf0 0x7f324815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f324815c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33498/0 (socket says 172.21.15.7:33498) 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 -- 172.21.15.7:0/105988801 learned_addr learned my addr 172.21.15.7:0/105988801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 -- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248063770 msgr2=0x7f32480b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248063770 0x7f32480b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 -- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248159bf0 msgr2=0x7f324815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248159bf0 0x7f324815be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 -- 172.21.15.7:0/105988801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300094f0 con 0x7f3248159fc0 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.382+0000 7f3246d9d700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f324815c350 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f324000c930 tx=0x7f32400026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.383+0000 7f323e7fc700 1 -- 172.21.15.7:0/105988801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240012630 con 0x7f3248159fc0 2024-01-11T07:13:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.383+0000 7f323e7fc700 1 -- 172.21.15.7:0/105988801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32400127d0 con 0x7f3248159fc0 2024-01-11T07:13:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.383+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32480c66b0 con 0x7f3248159fc0 2024-01-11T07:13:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.383+0000 7f323e7fc700 1 -- 172.21.15.7:0/105988801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240012b90 con 0x7f3248159fc0 2024-01-11T07:13:15.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.383+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32480c6c70 con 0x7f3248159fc0 2024-01-11T07:13:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.384+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32480c7150 con 0x7f3248159fc0 2024-01-11T07:13:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.384+0000 7f323e7fc700 1 -- 172.21.15.7:0/105988801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f324001a3b0 con 0x7f3248159fc0 2024-01-11T07:13:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.384+0000 7f323e7fc700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3228066f30 0x7f32280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.384+0000 7f323e7fc700 1 -- 172.21.15.7:0/105988801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3240083e00 con 0x7f3248159fc0 2024-01-11T07:13:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.385+0000 7f324659c700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3228066f30 0x7f32280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.385+0000 7f324659c700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3228066f30 0x7f32280693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f323800acb0 tx=0x7f3238008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:15.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.389+0000 7f323e7fc700 1 -- 172.21.15.7:0/105988801 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3240054e50 con 0x7f3248159fc0 2024-01-11T07:13:15.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:15.593+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f324804c3c0 con 0x7f3248159fc0 2024-01-11T07:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.279+0000 7f323e7fc700 1 -- 172.21.15.7:0/105988801 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v132) v1 ==== 138+0+0 (secure 0 0 0) 0x7f32400549e0 con 0x7f3248159fc0 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3228066f30 msgr2=0x7f32280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3228066f30 0x7f32280693e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f323800acb0 tx=0x7f3238008040 comp rx=0 tx=0).stop 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 msgr2=0x7f324815c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f324815c350 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f324000c930 tx=0x7f32400026e0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 shutdown_connections 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3228066f30 0x7f32280693e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3248159fc0 0x7f324815c350 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3248063770 0x7f32480b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 --2- 172.21.15.7:0/105988801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248159bf0 0x7f324815be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 >> 172.21.15.7:0/105988801 conn(0x7f32480680a0 msgr2=0x7f32480cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 shutdown_connections 2024-01-11T07:13:16.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.285+0000 7f324cf47700 1 -- 172.21.15.7:0/105988801 wait complete. 2024-01-11T07:13:16.286 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:13:16.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:13:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.444+0000 7fab31a18700 1 Processor -- start 2024-01-11T07:13:16.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.444+0000 7fab31a18700 1 -- start start 2024-01-11T07:13:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab31a18700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab2c0adb40 0x7fab2c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab31a18700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab2c0adf10 0x7fab2c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab31a18700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab2c0cc2a0 con 0x7fab2c0b3590 2024-01-11T07:13:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab31a18700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c0b2930 con 0x7fab2c0adf10 2024-01-11T07:13:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab31a18700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab2c0b3b50 con 0x7fab2c0adb40 2024-01-11T07:13:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab2c0adf10 0x7fab2c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fab2c0b3590 0x7fab2c0cea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47560/0 (socket says 172.21.15.7:47560) 2024-01-11T07:13:16.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab2c0adf10 0x7fab2c0b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33506/0 (socket says 172.21.15.7:33506) 2024-01-11T07:13:16.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2b7fe700 1 -- 172.21.15.7:0/3138121551 learned_addr learned my addr 172.21.15.7:0/3138121551 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2affd700 1 --2- 172.21.15.7:0/3138121551 >> v2:172.21.15.113:3300/0 conn(0x7fab2c0adb40 0x7fab2c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2a7fc700 1 -- 172.21.15.7:0/3138121551 >> v2:172.21.15.113:3300/0 conn(0x7fab2c0adb40 msgr2=0x7fab2c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2a7fc700 1 --2- 172.21.15.7:0/3138121551 >> v2:172.21.15.113:3300/0 conn(0x7fab2c0adb40 0x7fab2c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2a7fc700 1 -- 172.21.15.7:0/3138121551 >> v1:172.21.15.7:6789/0 conn(0x7fab2c0b3590 legacy=0x7fab2c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab29ffb700 1 -- 172.21.15.7:0/3138121551 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893071673 0 0) 0x7fab2c0cc2a0 con 0x7fab2c0b3590 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.445+0000 7fab2a7fc700 1 -- 172.21.15.7:0/3138121551 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c0b3e30 con 0x7fab2c0adf10 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab2a7fc700 1 --2- 172.21.15.7:0/3138121551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c0b0430 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0cfb70 tx=0x7fab1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fb6c02e4f68d3e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab29ffb700 1 -- 172.21.15.7:0/3138121551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00b6b0 con 0x7fab2c0adf10 2024-01-11T07:13:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab29ffb700 1 -- 172.21.15.7:0/3138121551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c00b810 con 0x7fab2c0adf10 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab29ffb700 1 -- 172.21.15.7:0/3138121551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00bbb0 con 0x7fab2c0adf10 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab31a18700 1 -- 172.21.15.7:0/3138121551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 msgr2=0x7fab2c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab31a18700 1 --2- 172.21.15.7:0/3138121551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c0b0430 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0cfb70 tx=0x7fab1c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab31a18700 1 -- 172.21.15.7:0/3138121551 shutdown_connections 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab31a18700 1 --2- 172.21.15.7:0/3138121551 >> v2:172.21.15.113:3300/0 conn(0x7fab2c0adb40 0x7fab2c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab31a18700 1 --2- 172.21.15.7:0/3138121551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c0b0430 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab31a18700 1 -- 172.21.15.7:0/3138121551 >> 172.21.15.7:0/3138121551 conn(0x7fab2c071160 msgr2=0x7fab2c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab31a18700 1 -- 172.21.15.7:0/3138121551 shutdown_connections 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.446+0000 7fab31a18700 1 -- 172.21.15.7:0/3138121551 wait complete. 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.448+0000 7fab31a18700 1 Processor -- start 2024-01-11T07:13:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.448+0000 7fab31a18700 1 -- start start 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab31a18700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0adb40 0x7fab2c067610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab31a18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab31a18700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0b3590 0x7fab2c1480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab31a18700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c067d90 con 0x7fab2c0adf10 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab31a18700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c15be50 con 0x7fab2c0b3590 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab31a18700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c15c0c0 con 0x7fab2c0adb40 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c15dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33508/0 (socket says 172.21.15.7:33508) 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 -- 172.21.15.7:0/2259277117 learned_addr learned my addr 172.21.15.7:0/2259277117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2b7fe700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0b3590 0x7fab2c1480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2affd700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0adb40 0x7fab2c067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 -- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0adb40 msgr2=0x7fab2c067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0adb40 0x7fab2c067610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 -- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0b3590 msgr2=0x7fab2c1480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0b3590 0x7fab2c1480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 -- 172.21.15.7:0/2259277117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c009420 con 0x7fab2c0adf10 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2b7fe700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0b3590 0x7fab2c1480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2affd700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0adb40 0x7fab2c067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab2a7fc700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c15dc20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00b0e0 tx=0x7fab1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab22ffd700 1 -- 172.21.15.7:0/2259277117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c011420 con 0x7fab2c0adf10 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.449+0000 7fab22ffd700 1 -- 172.21.15.7:0/2259277117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c011580 con 0x7fab2c0adf10 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.450+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c15c3a0 con 0x7fab2c0adf10 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.450+0000 7fab22ffd700 1 -- 172.21.15.7:0/2259277117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c011940 con 0x7fab2c0adf10 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.450+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c156bd0 con 0x7fab2c0adf10 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.451+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2c04c3c0 con 0x7fab2c0adf10 2024-01-11T07:13:16.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.451+0000 7fab22ffd700 1 -- 172.21.15.7:0/2259277117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fab1c011d30 con 0x7fab2c0adf10 2024-01-11T07:13:16.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.452+0000 7fab22ffd700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab0c066f30 0x7fab0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:16.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.452+0000 7fab22ffd700 1 -- 172.21.15.7:0/2259277117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fab1c085d70 con 0x7fab2c0adf10 2024-01-11T07:13:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.452+0000 7fab2affd700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab0c066f30 0x7fab0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.453+0000 7fab2affd700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab0c066f30 0x7fab0c0693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fab14006fd0 tx=0x7fab14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.457+0000 7fab22ffd700 1 -- 172.21.15.7:0/2259277117 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab1c053120 con 0x7fab2c0adf10 2024-01-11T07:13:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:16.658+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fab2c0cc540 con 0x7fab2c0adf10 2024-01-11T07:13:17.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.284+0000 7fab22ffd700 1 -- 172.21.15.7:0/2259277117 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v133) v1 ==== 170+0+0 (secure 0 0 0) 0x7fab1c0150d0 con 0x7fab2c0adf10 2024-01-11T07:13:17.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab0c066f30 msgr2=0x7fab0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:17.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab0c066f30 0x7fab0c0693e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fab14006fd0 tx=0x7fab14008040 comp rx=0 tx=0).stop 2024-01-11T07:13:17.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 msgr2=0x7fab2c15dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:17.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c15dc20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00b0e0 tx=0x7fab1c017000 comp rx=0 tx=0).stop 2024-01-11T07:13:17.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 shutdown_connections 2024-01-11T07:13:17.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fab0c066f30 0x7fab0c0693e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:17.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0b3590 0x7fab2c1480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:17.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0adb40 0x7fab2c067610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:17.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 --2- 172.21.15.7:0/2259277117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab2c0adf10 0x7fab2c15dc20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:17.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.290+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 >> 172.21.15.7:0/2259277117 conn(0x7fab2c071160 msgr2=0x7fab2c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:17.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.291+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 shutdown_connections 2024-01-11T07:13:17.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.292+0000 7fab31a18700 1 -- 172.21.15.7:0/2259277117 wait complete. 2024-01-11T07:13:17.297 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:13:17.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:13:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.452+0000 7fb192b13700 1 Processor -- start 2024-01-11T07:13:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.452+0000 7fb192b13700 1 -- start start 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb192b13700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb18c1511b0 0x7fb18c0ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb192b13700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0acc30 con 0x7fb18c150de0 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb192b13700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0acf10 con 0x7fb18c06bf20 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb192b13700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c0ab260 con 0x7fb18c1511b0 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb1908af700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb18c150de0 0x7fb18c06a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59662/0 (socket says 172.21.15.7:59662) 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb1908af700 1 -- 172.21.15.7:0/3477941949 learned_addr learned my addr 172.21.15.7:0/3477941949 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb18bfff700 1 --2- 172.21.15.7:0/3477941949 >> v2:172.21.15.113:3300/0 conn(0x7fb18c1511b0 0x7fb18c0ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb18b7fe700 1 -- 172.21.15.7:0/3477941949 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739008361 0 0) 0x7fb18c0acc30 con 0x7fb18c150de0 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.453+0000 7fb18b7fe700 1 -- 172.21.15.7:0/3477941949 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb17c003570 con 0x7fb18c150de0 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18b7fe700 1 -- 172.21.15.7:0/3477941949 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629620098 0 0) 0x7fb18c0acf10 con 0x7fb18c06bf20 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18b7fe700 1 -- 172.21.15.7:0/3477941949 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb18c0acc30 con 0x7fb18c06bf20 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18bfff700 1 -- 172.21.15.7:0/3477941949 >> v1:172.21.15.113:6789/0 conn(0x7fb18c06bf20 legacy=0x7fb18c063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18bfff700 1 -- 172.21.15.7:0/3477941949 >> v1:172.21.15.7:6789/0 conn(0x7fb18c150de0 legacy=0x7fb18c06a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18b7fe700 1 -- 172.21.15.7:0/3477941949 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1653277451 0 0) 0x7fb17c003570 con 0x7fb18c150de0 2024-01-11T07:13:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18bfff700 1 -- 172.21.15.7:0/3477941949 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0ad1f0 con 0x7fb18c1511b0 2024-01-11T07:13:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18bfff700 1 --2- 172.21.15.7:0/3477941949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c1511b0 0x7fb18c0ac650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb1780098e0 tx=0x7fb1780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f9e40133e88f2a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18b7fe700 1 -- 172.21.15.7:0/3477941949 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180001470 con 0x7fb18c1511b0 2024-01-11T07:13:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18b7fe700 1 -- 172.21.15.7:0/3477941949 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb178011420 con 0x7fb18c1511b0 2024-01-11T07:13:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.454+0000 7fb18b7fe700 1 -- 172.21.15.7:0/3477941949 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780117c0 con 0x7fb18c1511b0 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.455+0000 7fb192b13700 1 -- 172.21.15.7:0/3477941949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c1511b0 msgr2=0x7fb18c0ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.455+0000 7fb192b13700 1 --2- 172.21.15.7:0/3477941949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c1511b0 0x7fb18c0ac650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb1780098e0 tx=0x7fb1780091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.455+0000 7fb192b13700 1 -- 172.21.15.7:0/3477941949 shutdown_connections 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.455+0000 7fb192b13700 1 --2- 172.21.15.7:0/3477941949 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c1511b0 0x7fb18c0ac650 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.455+0000 7fb192b13700 1 -- 172.21.15.7:0/3477941949 >> 172.21.15.7:0/3477941949 conn(0x7fb18c0cd910 msgr2=0x7fb18c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.455+0000 7fb192b13700 1 -- 172.21.15.7:0/3477941949 shutdown_connections 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.455+0000 7fb192b13700 1 -- 172.21.15.7:0/3477941949 wait complete. 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.456+0000 7fb192b13700 1 Processor -- start 2024-01-11T07:13:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.456+0000 7fb192b13700 1 -- start start 2024-01-11T07:13:17.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb192b13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb18c06bf20 0x7fb18c152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb192b13700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c150de0 0x7fb18c16ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb192b13700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb18c1511b0 0x7fb18c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb192b13700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c16b3a0 con 0x7fb18c06bf20 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb192b13700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c1525e0 con 0x7fb18c150de0 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb192b13700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c15a920 con 0x7fb18c1511b0 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1908af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb18c06bf20 0x7fb18c152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1908af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb18c06bf20 0x7fb18c152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45904/0 (socket says 172.21.15.7:45904) 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1908af700 1 -- 172.21.15.7:0/3565960097 learned_addr learned my addr 172.21.15.7:0/3565960097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb18bfff700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c150de0 0x7fb18c16ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1910b0700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb18c1511b0 0x7fb18c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1908af700 1 -- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb18c1511b0 msgr2=0x7fb18c159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1908af700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb18c1511b0 0x7fb18c159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1908af700 1 -- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c150de0 msgr2=0x7fb18c16ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1908af700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c150de0 0x7fb18c16ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb1908af700 1 -- 172.21.15.7:0/3565960097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb178009420 con 0x7fb18c06bf20 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.457+0000 7fb18bfff700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c150de0 0x7fb18c16ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.458+0000 7fb1908af700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb18c06bf20 0x7fb18c152a70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb18000cb70 tx=0x7fb180001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.458+0000 7fb188ff9700 1 -- 172.21.15.7:0/3565960097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180007e90 con 0x7fb18c06bf20 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.458+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c1519d0 con 0x7fb18c06bf20 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.458+0000 7fb188ff9700 1 -- 172.21.15.7:0/3565960097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb180007360 con 0x7fb18c06bf20 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.458+0000 7fb188ff9700 1 -- 172.21.15.7:0/3565960097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180004830 con 0x7fb18c06bf20 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.458+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c151f90 con 0x7fb18c06bf20 2024-01-11T07:13:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.459+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c064a40 con 0x7fb18c06bf20 2024-01-11T07:13:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.459+0000 7fb188ff9700 1 -- 172.21.15.7:0/3565960097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb180004ba0 con 0x7fb18c06bf20 2024-01-11T07:13:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.460+0000 7fb188ff9700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb170066f30 0x7fb1700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.460+0000 7fb188ff9700 1 -- 172.21.15.7:0/3565960097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fb1800838c0 con 0x7fb18c06bf20 2024-01-11T07:13:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.460+0000 7fb18bfff700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb170066f30 0x7fb1700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:17.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.461+0000 7fb18bfff700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb170066f30 0x7fb1700693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb1780093f0 tx=0x7fb178016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.465+0000 7fb188ff9700 1 -- 172.21.15.7:0/3565960097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb18004b5a0 con 0x7fb18c06bf20 2024-01-11T07:13:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:17.723+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb18c04c3c0 con 0x7fb18c06bf20 2024-01-11T07:13:18.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.300+0000 7fb188ff9700 1 -- 172.21.15.7:0/3565960097 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v61) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb180018020 con 0x7fb18c06bf20 2024-01-11T07:13:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb170066f30 msgr2=0x7fb1700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb170066f30 0x7fb1700693e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb1780093f0 tx=0x7fb178016000 comp rx=0 tx=0).stop 2024-01-11T07:13:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb18c06bf20 msgr2=0x7fb18c152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb18c06bf20 0x7fb18c152a70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb18000cb70 tx=0x7fb180001b50 comp rx=0 tx=0).stop 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 shutdown_connections 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb170066f30 0x7fb1700693e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb18c1511b0 0x7fb18c159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb18c06bf20 0x7fb18c152a70 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 --2- 172.21.15.7:0/3565960097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb18c150de0 0x7fb18c16ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.316+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 >> 172.21.15.7:0/3565960097 conn(0x7fb18c0cd910 msgr2=0x7fb18c0ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.317+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 shutdown_connections 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.317+0000 7fb192b13700 1 -- 172.21.15.7:0/3565960097 wait complete. 2024-01-11T07:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 16 and data pool 17 2024-01-11T07:13:18.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.464+0000 7f7483ad0700 1 Processor -- start 2024-01-11T07:13:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.464+0000 7f7483ad0700 1 -- start start 2024-01-11T07:13:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.465+0000 7f7483ad0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.465+0000 7f7483ad0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f747c06bc50 0x7f747c06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.465+0000 7f7483ad0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747c15e010 con 0x7f747c15e4a0 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.465+0000 7f7483ad0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c15e910 con 0x7f747c06bc50 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.465+0000 7f7483ad0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c06c5e0 con 0x7f747c159a50 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.465+0000 7f748206d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f747c06bc50 0x7f747c06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.465+0000 7f748206d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f747c06bc50 0x7f747c06c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45918/0 (socket says 172.21.15.7:45918) 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.465+0000 7f748206d700 1 -- 172.21.15.7:0/3886498413 learned_addr learned my addr 172.21.15.7:0/3886498413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748186c700 1 --2- 172.21.15.7:0/3886498413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748206d700 1 -- 172.21.15.7:0/3886498413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 msgr2=0x7f747c159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748206d700 1 --2- 172.21.15.7:0/3886498413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748206d700 1 -- 172.21.15.7:0/3886498413 >> v1:172.21.15.113:6789/0 conn(0x7f747c15e4a0 legacy=0x7f747c0ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748206d700 1 -- 172.21.15.7:0/3886498413 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0acf80 con 0x7f747c06bc50 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748206d700 1 --2- 172.21.15.7:0/3886498413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c06bc50 0x7f747c06c0a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7474009f20 tx=0x7f7474009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4007792256cd4f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748086a700 1 -- 172.21.15.7:0/3886498413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747400e070 con 0x7f747c06bc50 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748086a700 1 -- 172.21.15.7:0/3886498413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747400ba80 con 0x7f747c06bc50 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.466+0000 7f748086a700 1 -- 172.21.15.7:0/3886498413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740092e0 con 0x7f747c06bc50 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.467+0000 7f7483ad0700 1 -- 172.21.15.7:0/3886498413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c06bc50 msgr2=0x7f747c06c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.467+0000 7f7483ad0700 1 --2- 172.21.15.7:0/3886498413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c06bc50 0x7f747c06c0a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f7474009f20 tx=0x7f7474009840 comp rx=0 tx=0).stop 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.467+0000 7f7483ad0700 1 -- 172.21.15.7:0/3886498413 shutdown_connections 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.467+0000 7f7483ad0700 1 --2- 172.21.15.7:0/3886498413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c06bc50 0x7f747c06c0a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.467+0000 7f7483ad0700 1 --2- 172.21.15.7:0/3886498413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.467+0000 7f7483ad0700 1 -- 172.21.15.7:0/3886498413 >> 172.21.15.7:0/3886498413 conn(0x7f747c0cd910 msgr2=0x7f747c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.467+0000 7f7483ad0700 1 -- 172.21.15.7:0/3886498413 shutdown_connections 2024-01-11T07:13:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.467+0000 7f7483ad0700 1 -- 172.21.15.7:0/3886498413 wait complete. 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.468+0000 7f7483ad0700 1 Processor -- start 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.468+0000 7f7483ad0700 1 -- start start 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f7483ad0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f747c06bc50 0x7f747c14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f7483ad0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f7483ad0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c15e4a0 0x7f747c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f7483ad0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c155cf0 con 0x7f747c15e4a0 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f7483ad0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c16aab0 con 0x7f747c06bc50 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f7483ad0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c14da70 con 0x7f747c159a50 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748206d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c15e4a0 0x7f747c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748206d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c15e4a0 0x7f747c15c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45924/0 (socket says 172.21.15.7:45924) 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748206d700 1 -- 172.21.15.7:0/592948287 learned_addr learned my addr 172.21.15.7:0/592948287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748186c700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f747c06bc50 0x7f747c14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748106b700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748206d700 1 -- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 msgr2=0x7f747c156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748206d700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748206d700 1 -- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f747c06bc50 msgr2=0x7f747c14d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.469+0000 7f748206d700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f747c06bc50 0x7f747c14d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.470+0000 7f748206d700 1 -- 172.21.15.7:0/592948287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7474009ae0 con 0x7f747c15e4a0 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.470+0000 7f748106b700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.470+0000 7f748206d700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c15e4a0 0x7f747c15c590 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f747c06d170 tx=0x7f7474018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.470+0000 7f746dffb700 1 -- 172.21.15.7:0/592948287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740032f0 con 0x7f747c15e4a0 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.470+0000 7f746dffb700 1 -- 172.21.15.7:0/592948287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7474008e80 con 0x7f747c15e4a0 2024-01-11T07:13:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.470+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747c155f40 con 0x7f747c15e4a0 2024-01-11T07:13:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.470+0000 7f746dffb700 1 -- 172.21.15.7:0/592948287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474026690 con 0x7f747c15e4a0 2024-01-11T07:13:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.470+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747c1562d0 con 0x7f747c15e4a0 2024-01-11T07:13:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.471+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747c04c3c0 con 0x7f747c15e4a0 2024-01-11T07:13:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.471+0000 7f746dffb700 1 -- 172.21.15.7:0/592948287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7474003040 con 0x7f747c15e4a0 2024-01-11T07:13:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.472+0000 7f746dffb700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7460066f30 0x7f74600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.472+0000 7f746dffb700 1 -- 172.21.15.7:0/592948287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7474086680 con 0x7f747c15e4a0 2024-01-11T07:13:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.472+0000 7f748186c700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7460066f30 0x7f74600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.473+0000 7f748186c700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7460066f30 0x7f74600693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7470006fd0 tx=0x7f7470008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.476+0000 7f746dffb700 1 -- 172.21.15.7:0/592948287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74740539a0 con 0x7f747c15e4a0 2024-01-11T07:13:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.681+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f747c15d9c0 con 0x7f747c15e4a0 2024-01-11T07:13:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.682+0000 7f746dffb700 1 -- 172.21.15.7:0/592948287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f7474056fc0 con 0x7f747c15e4a0 2024-01-11T07:13:18.682 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:18.682 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":134,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:13:17.725013+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:13:15.625657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-11T07:13:16.691544+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7460066f30 msgr2=0x7f74600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7460066f30 0x7f74600693e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7470006fd0 tx=0x7f7470008040 comp rx=0 tx=0).stop 2024-01-11T07:13:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c15e4a0 msgr2=0x7f747c15c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c15e4a0 0x7f747c15c590 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f747c06d170 tx=0x7f7474018000 comp rx=0 tx=0).stop 2024-01-11T07:13:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 shutdown_connections 2024-01-11T07:13:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7460066f30 0x7f74600693e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f747c15e4a0 0x7f747c15c590 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f747c06bc50 0x7f747c14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 --2- 172.21.15.7:0/592948287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f747c159a50 0x7f747c156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 >> 172.21.15.7:0/592948287 conn(0x7f747c0cd910 msgr2=0x7f747c0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 shutdown_connections 2024-01-11T07:13:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.684+0000 7f7483ad0700 1 -- 172.21.15.7:0/592948287 wait complete. 2024-01-11T07:13:18.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.830+0000 7fbd3f97c700 1 Processor -- start 2024-01-11T07:13:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.831+0000 7fbd3f97c700 1 -- start start 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3f97c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd38158760 0x7fbd3815f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3f97c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd38158be0 con 0x7fbd38063230 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3f97c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd38161e30 con 0x7fbd38158390 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3f97c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3815f770 con 0x7fbd38158760 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3cf17700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd38158760 0x7fbd3815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3df19700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbd38063230 0x7fbd3806dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59668/0 (socket says 172.21.15.7:59668) 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3df19700 1 -- 172.21.15.7:0/670324582 learned_addr learned my addr 172.21.15.7:0/670324582 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3cf17700 1 -- 172.21.15.7:0/670324582 >> v1:172.21.15.113:6789/0 conn(0x7fbd38158390 legacy=0x7fbd380b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3cf17700 1 -- 172.21.15.7:0/670324582 >> v1:172.21.15.7:6789/0 conn(0x7fbd38063230 legacy=0x7fbd3806dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3cf17700 1 -- 172.21.15.7:0/670324582 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd38162110 con 0x7fbd38158760 2024-01-11T07:13:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.832+0000 7fbd3cf17700 1 --2- 172.21.15.7:0/670324582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd3815f230 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009910 tx=0x7fbd340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44690175021a9797 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd2ffff700 1 -- 172.21.15.7:0/670324582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400b820 con 0x7fbd38158760 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd2ffff700 1 -- 172.21.15.7:0/670324582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3400b980 con 0x7fbd38158760 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd2ffff700 1 -- 172.21.15.7:0/670324582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400bd20 con 0x7fbd38158760 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd3f97c700 1 -- 172.21.15.7:0/670324582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 msgr2=0x7fbd3815f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd3f97c700 1 --2- 172.21.15.7:0/670324582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd3815f230 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009910 tx=0x7fbd340091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd3f97c700 1 -- 172.21.15.7:0/670324582 shutdown_connections 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd3f97c700 1 --2- 172.21.15.7:0/670324582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd3815f230 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd3f97c700 1 -- 172.21.15.7:0/670324582 >> 172.21.15.7:0/670324582 conn(0x7fbd380cd910 msgr2=0x7fbd380cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd3f97c700 1 -- 172.21.15.7:0/670324582 shutdown_connections 2024-01-11T07:13:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.833+0000 7fbd3f97c700 1 -- 172.21.15.7:0/670324582 wait complete. 2024-01-11T07:13:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.834+0000 7fbd3f97c700 1 Processor -- start 2024-01-11T07:13:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.835+0000 7fbd3f97c700 1 -- start start 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.835+0000 7fbd3f97c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd38063230 0x7fbd38149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.835+0000 7fbd3f97c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd38158390 0x7fbd38148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.835+0000 7fbd3f97c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd381486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3f97c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3814a1c0 con 0x7fbd38158760 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3f97c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3814a4a0 con 0x7fbd38063230 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3f97c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3814a780 con 0x7fbd38158390 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd381486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd381486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45932/0 (socket says 172.21.15.7:45932) 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 -- 172.21.15.7:0/2627938108 learned_addr learned my addr 172.21.15.7:0/2627938108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3d718700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd38063230 0x7fbd38149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3cf17700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd38158390 0x7fbd38148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 -- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd38158390 msgr2=0x7fbd38148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd38158390 0x7fbd38148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 -- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd38063230 msgr2=0x7fbd38149970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd38063230 0x7fbd38149970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 -- 172.21.15.7:0/2627938108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34009420 con 0x7fbd38158760 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3d718700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd38063230 0x7fbd38149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3cf17700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd38158390 0x7fbd38148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3df19700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd381486d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000cbc0 tx=0x7fbd300019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd2d7fa700 1 -- 172.21.15.7:0/2627938108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3000a940 con 0x7fbd38158760 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd2d7fa700 1 -- 172.21.15.7:0/2627938108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3000aaa0 con 0x7fbd38158760 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3814aa60 con 0x7fbd38158760 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd2d7fa700 1 -- 172.21.15.7:0/2627938108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3000ae20 con 0x7fbd38158760 2024-01-11T07:13:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.836+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd38149eb0 con 0x7fbd38158760 2024-01-11T07:13:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.837+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd38145b00 con 0x7fbd38158760 2024-01-11T07:13:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.838+0000 7fbd2d7fa700 1 -- 172.21.15.7:0/2627938108 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbd30007300 con 0x7fbd38158760 2024-01-11T07:13:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.838+0000 7fbd2d7fa700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd1c066f30 0x7fbd1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.838+0000 7fbd2d7fa700 1 -- 172.21.15.7:0/2627938108 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbd30084930 con 0x7fbd38158760 2024-01-11T07:13:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.838+0000 7fbd3d718700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd1c066f30 0x7fbd1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.839+0000 7fbd3d718700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd1c066f30 0x7fbd1c0693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbd28007fd0 tx=0x7fbd28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:18.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:18.843+0000 7fbd2d7fa700 1 -- 172.21.15.7:0/2627938108 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd30051b90 con 0x7fbd38158760 2024-01-11T07:13:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.049+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd3804c3c0 con 0x7fbd38158760 2024-01-11T07:13:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.050+0000 7fbd2d7fa700 1 -- 172.21.15.7:0/2627938108 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+15330 (secure 0 0 0) 0x7fbd300551b0 con 0x7fbd38158760 2024-01-11T07:13:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:19.050 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":134,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:13:17.725013+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:13:15.625657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-11T07:13:16.691544+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd1c066f30 msgr2=0x7fbd1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd1c066f30 0x7fbd1c0693e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbd28007fd0 tx=0x7fbd28009040 comp rx=0 tx=0).stop 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 msgr2=0x7fbd381486d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd381486d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000cbc0 tx=0x7fbd300019c0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 shutdown_connections 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd1c066f30 0x7fbd1c0693e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd38158760 0x7fbd381486d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd38063230 0x7fbd38149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 --2- 172.21.15.7:0/2627938108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd38158390 0x7fbd38148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 >> 172.21.15.7:0/2627938108 conn(0x7fbd380cd910 msgr2=0x7fbd38157980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.052+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 shutdown_connections 2024-01-11T07:13:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.053+0000 7fbd3f97c700 1 -- 172.21.15.7:0/2627938108 wait complete. 2024-01-11T07:13:19.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:13:19.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-01-11T07:13:19.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-01-11T07:13:19.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-01-11T07:13:19.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-01-11T07:13:19.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.202+0000 7f82ff1f6700 1 Processor -- start 2024-01-11T07:13:19.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.202+0000 7f82ff1f6700 1 -- start start 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.203+0000 7f82ff1f6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f815d4a0 0x7f82f80b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.203+0000 7f82ff1f6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82f815d870 0x7f82f80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.203+0000 7f82ff1f6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82f80cca30 0x7f82f8068570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.203+0000 7f82ff1f6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f80cced0 con 0x7f82f815d870 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.203+0000 7f82ff1f6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f80680e0 con 0x7f82f80cca30 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.203+0000 7f82ff1f6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80677b0 con 0x7f82f815d4a0 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82f815d870 0x7f82f80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82f815d870 0x7f82f80cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45944/0 (socket says 172.21.15.7:45944) 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 -- 172.21.15.7:0/2112640918 learned_addr learned my addr 172.21.15.7:0/2112640918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82fcf92700 1 --2- 172.21.15.7:0/2112640918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f815d4a0 0x7f82f80b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82fd793700 1 --2- 172.21.15.7:0/2112640918 >> v2:172.21.15.113:3300/0 conn(0x7f82f80cca30 0x7f82f8068570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 -- 172.21.15.7:0/2112640918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f815d4a0 msgr2=0x7f82f80b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 --2- 172.21.15.7:0/2112640918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f815d4a0 0x7f82f80b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 -- 172.21.15.7:0/2112640918 >> v2:172.21.15.113:3300/0 conn(0x7f82f80cca30 msgr2=0x7f82f8068570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 --2- 172.21.15.7:0/2112640918 >> v2:172.21.15.113:3300/0 conn(0x7f82f80cca30 0x7f82f8068570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 -- 172.21.15.7:0/2112640918 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8067a90 con 0x7f82f815d870 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f7fff700 1 --2- 172.21.15.7:0/2112640918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f815d870 0x7f82f80cc4f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f82f806e100 tx=0x7f82e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4cca0283926e66 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f77fe700 1 -- 172.21.15.7:0/2112640918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000b860 con 0x7f82f815d870 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f77fe700 1 -- 172.21.15.7:0/2112640918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e000b9c0 con 0x7f82f815d870 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.204+0000 7f82f77fe700 1 -- 172.21.15.7:0/2112640918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e000bd60 con 0x7f82f815d870 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 -- 172.21.15.7:0/2112640918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f815d870 msgr2=0x7f82f80cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/2112640918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f815d870 0x7f82f80cc4f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f82f806e100 tx=0x7f82e0009250 comp rx=0 tx=0).stop 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 -- 172.21.15.7:0/2112640918 shutdown_connections 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/2112640918 >> v2:172.21.15.113:3300/0 conn(0x7f82f80cca30 0x7f82f8068570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/2112640918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f815d4a0 0x7f82f80b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/2112640918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f815d870 0x7f82f80cc4f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 -- 172.21.15.7:0/2112640918 >> 172.21.15.7:0/2112640918 conn(0x7f82f8097a50 msgr2=0x7f82f80ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:19.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 -- 172.21.15.7:0/2112640918 shutdown_connections 2024-01-11T07:13:19.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.205+0000 7f82ff1f6700 1 -- 172.21.15.7:0/2112640918 wait complete. 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.206+0000 7f82ff1f6700 1 Processor -- start 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82ff1f6700 1 -- start start 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82ff1f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f80cca30 0x7f82f80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82ff1f6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f815d4a0 0x7f82f816a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82ff1f6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f816dc60 0x7f82f80c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82ff1f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f816ad50 con 0x7f82f80cca30 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82ff1f6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80b2e30 con 0x7f82f815d4a0 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82ff1f6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80b2870 con 0x7f82f816dc60 2024-01-11T07:13:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82fcf92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f80cca30 0x7f82f80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82fcf92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f80cca30 0x7f82f80b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45956/0 (socket says 172.21.15.7:45956) 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.207+0000 7f82fcf92700 1 -- 172.21.15.7:0/560460604 learned_addr learned my addr 172.21.15.7:0/560460604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82fcf92700 1 -- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f816dc60 msgr2=0x7f82f80c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82fcf92700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f816dc60 0x7f82f80c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82f7fff700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f815d4a0 0x7f82f816a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82fcf92700 1 -- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f815d4a0 msgr2=0x7f82f816a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82fcf92700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f815d4a0 0x7f82f816a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82fcf92700 1 -- 172.21.15.7:0/560460604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00094f0 con 0x7f82f80cca30 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82fcf92700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f80cca30 0x7f82f80b21b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f82e800d840 tx=0x7f82e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82f4ff9700 1 -- 172.21.15.7:0/560460604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8014620 con 0x7f82f80cca30 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82f4ff9700 1 -- 172.21.15.7:0/560460604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8014780 con 0x7f82f80cca30 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f80b2b50 con 0x7f82f80cca30 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82f4ff9700 1 -- 172.21.15.7:0/560460604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8014b40 con 0x7f82f80cca30 2024-01-11T07:13:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.208+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f816d740 con 0x7f82f80cca30 2024-01-11T07:13:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.209+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f804c3c0 con 0x7f82f80cca30 2024-01-11T07:13:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.210+0000 7f82f4ff9700 1 -- 172.21.15.7:0/560460604 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f82e801d400 con 0x7f82f80cca30 2024-01-11T07:13:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.210+0000 7f82f4ff9700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82d8066f30 0x7f82d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.210+0000 7f82f4ff9700 1 -- 172.21.15.7:0/560460604 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82e8084dd0 con 0x7f82f80cca30 2024-01-11T07:13:19.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.211+0000 7f82f7fff700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82d8066f30 0x7f82d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:19.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.211+0000 7f82f7fff700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82d8066f30 0x7f82d80693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f82f806e100 tx=0x7f82e0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.215+0000 7f82f4ff9700 1 -- 172.21.15.7:0/560460604 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82e8052040 con 0x7f82f80cca30 2024-01-11T07:13:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:19.470+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f82f8067d70 con 0x7f82f80cca30 2024-01-11T07:13:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.306+0000 7f82f4ff9700 1 -- 172.21.15.7:0/560460604 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7f82e80860b0 con 0x7f82f80cca30 2024-01-11T07:13:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82d8066f30 msgr2=0x7f82d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82d8066f30 0x7f82d80693e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f82f806e100 tx=0x7f82e0017000 comp rx=0 tx=0).stop 2024-01-11T07:13:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f80cca30 msgr2=0x7f82f80b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f80cca30 0x7f82f80b21b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f82e800d840 tx=0x7f82e8008040 comp rx=0 tx=0).stop 2024-01-11T07:13:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 shutdown_connections 2024-01-11T07:13:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82d8066f30 0x7f82d80693e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82f816dc60 0x7f82f80c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82f80cca30 0x7f82f80b21b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 --2- 172.21.15.7:0/560460604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f815d4a0 0x7f82f816a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 >> 172.21.15.7:0/560460604 conn(0x7f82f8097a50 msgr2=0x7f82f8068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 shutdown_connections 2024-01-11T07:13:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.309+0000 7f82ff1f6700 1 -- 172.21.15.7:0/560460604 wait complete. 2024-01-11T07:13:20.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:13:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.465+0000 7fb48ea2c700 1 Processor -- start 2024-01-11T07:13:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.465+0000 7fb48ea2c700 1 -- start start 2024-01-11T07:13:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb48ea2c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb48815d430 0x7fb488068120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb48ea2c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4880c97e0 0x7fb4880c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb48ea2c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4880cdac0 con 0x7fb488068660 2024-01-11T07:13:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb48ea2c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4880cde10 con 0x7fb48815d430 2024-01-11T07:13:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb48ea2c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4880cd500 con 0x7fb4880c97e0 2024-01-11T07:13:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb487fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb48815d430 0x7fb488068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb487fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb48815d430 0x7fb488068120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45972/0 (socket says 172.21.15.7:45972) 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb487fff700 1 -- 172.21.15.7:0/957850738 learned_addr learned my addr 172.21.15.7:0/957850738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb48cfc9700 1 --2- 172.21.15.7:0/957850738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4880c97e0 0x7fb4880c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb487fff700 1 -- 172.21.15.7:0/957850738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4880c97e0 msgr2=0x7fb4880c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb487fff700 1 --2- 172.21.15.7:0/957850738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4880c97e0 0x7fb4880c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb487fff700 1 -- 172.21.15.7:0/957850738 >> v1:172.21.15.113:6789/0 conn(0x7fb488068660 legacy=0x7fb488068ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb487fff700 1 -- 172.21.15.7:0/957850738 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4880cf0d0 con 0x7fb48815d430 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.466+0000 7fb48cfc9700 1 --2- 172.21.15.7:0/957850738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4880c97e0 0x7fb4880c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb487fff700 1 --2- 172.21.15.7:0/957850738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb48815d430 0x7fb488068120 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb478009990 tx=0x7fb478009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66f253a9c307abb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb486ffd700 1 -- 172.21.15.7:0/957850738 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478005c10 con 0x7fb48815d430 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb486ffd700 1 -- 172.21.15.7:0/957850738 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb478005d70 con 0x7fb48815d430 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb486ffd700 1 -- 172.21.15.7:0/957850738 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47800b840 con 0x7fb48815d430 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb48ea2c700 1 -- 172.21.15.7:0/957850738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb48815d430 msgr2=0x7fb488068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/957850738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb48815d430 0x7fb488068120 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb478009990 tx=0x7fb478009220 comp rx=0 tx=0).stop 2024-01-11T07:13:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb48ea2c700 1 -- 172.21.15.7:0/957850738 shutdown_connections 2024-01-11T07:13:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/957850738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb4880c97e0 0x7fb4880c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/957850738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb48815d430 0x7fb488068120 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb48ea2c700 1 -- 172.21.15.7:0/957850738 >> 172.21.15.7:0/957850738 conn(0x7fb488097a50 msgr2=0x7fb4880ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb48ea2c700 1 -- 172.21.15.7:0/957850738 shutdown_connections 2024-01-11T07:13:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.467+0000 7fb48ea2c700 1 -- 172.21.15.7:0/957850738 wait complete. 2024-01-11T07:13:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.469+0000 7fb48ea2c700 1 Processor -- start 2024-01-11T07:13:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.469+0000 7fb48ea2c700 1 -- start start 2024-01-11T07:13:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb48ea2c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb488068660 0x7fb488149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb48ea2c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4880c97e0 0x7fb488159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb48ea2c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb48815d430 0x7fb488159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb48ea2c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb488159260 con 0x7fb4880c97e0 2024-01-11T07:13:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb48ea2c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb488149570 con 0x7fb488068660 2024-01-11T07:13:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb48ea2c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb488157a40 con 0x7fb48815d430 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4880c97e0 0x7fb488159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb487fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb488068660 0x7fb488149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4880c97e0 0x7fb488159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45984/0 (socket says 172.21.15.7:45984) 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 -- 172.21.15.7:0/450531600 learned_addr learned my addr 172.21.15.7:0/450531600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb48cfc9700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb48815d430 0x7fb488159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 -- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb48815d430 msgr2=0x7fb488159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb48815d430 0x7fb488159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 -- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb488068660 msgr2=0x7fb488149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb488068660 0x7fb488149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 -- 172.21.15.7:0/450531600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478009480 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb4877fe700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4880c97e0 0x7fb488159440 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb47000b5e0 tx=0x7fb470009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.470+0000 7fb47ffff700 1 -- 172.21.15.7:0/450531600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000f510 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.471+0000 7fb47ffff700 1 -- 172.21.15.7:0/450531600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47000f670 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.471+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb488157110 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.471+0000 7fb47ffff700 1 -- 172.21.15.7:0/450531600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000fa30 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.471+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4881576d0 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.472+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb488157bb0 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.472+0000 7fb47ffff700 1 -- 172.21.15.7:0/450531600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb47001a400 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.472+0000 7fb47ffff700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb468066f30 0x7fb4680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.472+0000 7fb47ffff700 1 -- 172.21.15.7:0/450531600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb470083f70 con 0x7fb4880c97e0 2024-01-11T07:13:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.473+0000 7fb487fff700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb468066f30 0x7fb4680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.473+0000 7fb487fff700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb468066f30 0x7fb4680693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb478003080 tx=0x7fb478015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.477+0000 7fb47ffff700 1 -- 172.21.15.7:0/450531600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4700511c0 con 0x7fb4880c97e0 2024-01-11T07:13:20.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:20.732+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb48804c3c0 con 0x7fb4880c97e0 2024-01-11T07:13:21.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.308+0000 7fb47ffff700 1 -- 172.21.15.7:0/450531600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb4700547e0 con 0x7fb4880c97e0 2024-01-11T07:13:21.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb468066f30 msgr2=0x7fb4680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb468066f30 0x7fb4680693e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb478003080 tx=0x7fb478015000 comp rx=0 tx=0).stop 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4880c97e0 msgr2=0x7fb488159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4880c97e0 0x7fb488159440 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb47000b5e0 tx=0x7fb470009120 comp rx=0 tx=0).stop 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 shutdown_connections 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb468066f30 0x7fb4680693e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb48815d430 0x7fb488159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb488068660 0x7fb488149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 --2- 172.21.15.7:0/450531600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4880c97e0 0x7fb488159440 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 >> 172.21.15.7:0/450531600 conn(0x7fb488097a50 msgr2=0x7fb48815bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 shutdown_connections 2024-01-11T07:13:21.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.311+0000 7fb48ea2c700 1 -- 172.21.15.7:0/450531600 wait complete. 2024-01-11T07:13:21.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.462+0000 7f7fb3e90700 1 Processor -- start 2024-01-11T07:13:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.462+0000 7f7fb3e90700 1 -- start start 2024-01-11T07:13:21.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb3e90700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063230 0x7f7fac15e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb3e90700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7fac15aa40 0x7f7fac06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb3e90700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fac15df80 con 0x7f7fac063600 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb3e90700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fac15e950 con 0x7f7fac15aa40 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb3e90700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac0aab70 con 0x7f7fac063230 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb142b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7fac063600 0x7f7fac15a330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59674/0 (socket says 172.21.15.7:59674) 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb142b700 1 -- 172.21.15.7:0/1135767596 learned_addr learned my addr 172.21.15.7:0/1135767596 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb242d700 1 --2- 172.21.15.7:0/1135767596 >> v2:172.21.15.113:3300/0 conn(0x7f7fac15aa40 0x7f7fac06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb1c2c700 1 --2- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063230 0x7f7fac15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb0c2a700 1 -- 172.21.15.7:0/1135767596 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594048961 0 0) 0x7f7fac15df80 con 0x7f7fac063600 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.463+0000 7f7fb0c2a700 1 -- 172.21.15.7:0/1135767596 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f98003570 con 0x7f7fac063600 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb242d700 1 -- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063230 msgr2=0x7f7fac15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb242d700 1 --2- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063230 0x7f7fac15e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb242d700 1 -- 172.21.15.7:0/1135767596 >> v1:172.21.15.7:6789/0 conn(0x7f7fac063600 legacy=0x7f7fac15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb242d700 1 -- 172.21.15.7:0/1135767596 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac15af10 con 0x7f7fac15aa40 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb1c2c700 1 --2- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063230 0x7f7fac15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb242d700 1 --2- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac15aa40 0x7f7fac06b530 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009e60 tx=0x7f7fa4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4731fc8ca8b035bd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb0c2a700 1 -- 172.21.15.7:0/1135767596 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400e070 con 0x7f7fac15aa40 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb0c2a700 1 -- 172.21.15.7:0/1135767596 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa400bab0 con 0x7f7fac15aa40 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.464+0000 7f7fb0c2a700 1 -- 172.21.15.7:0/1135767596 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400be70 con 0x7f7fac15aa40 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.465+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac15aa40 msgr2=0x7f7fac06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.465+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac15aa40 0x7f7fac06b530 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009e60 tx=0x7f7fa4009780 comp rx=0 tx=0).stop 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.465+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1135767596 shutdown_connections 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.465+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac15aa40 0x7f7fac06b530 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.465+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1135767596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063230 0x7f7fac15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.465+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1135767596 >> 172.21.15.7:0/1135767596 conn(0x7f7fac0cd910 msgr2=0x7f7fac0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.465+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1135767596 shutdown_connections 2024-01-11T07:13:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.465+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1135767596 wait complete. 2024-01-11T07:13:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.466+0000 7f7fb3e90700 1 Processor -- start 2024-01-11T07:13:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.466+0000 7f7fb3e90700 1 -- start start 2024-01-11T07:13:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb3e90700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac063230 0x7f7fac14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb3e90700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063600 0x7f7fac14b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb3e90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac15aa40 0x7f7fac0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb3e90700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac14ce60 con 0x7f7fac15aa40 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb3e90700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1501b0 con 0x7f7fac063230 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb3e90700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac0add60 con 0x7f7fac063600 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac15aa40 0x7f7fac0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac15aa40 0x7f7fac0b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45988/0 (socket says 172.21.15.7:45988) 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 -- 172.21.15.7:0/1580083726 learned_addr learned my addr 172.21.15.7:0/1580083726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb1c2c700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac063230 0x7f7fac14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb142b700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063600 0x7f7fac14b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 -- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063600 msgr2=0x7f7fac14b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063600 0x7f7fac14b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 -- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac063230 msgr2=0x7f7fac14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac063230 0x7f7fac14fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 -- 172.21.15.7:0/1580083726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8009420 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fb242d700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac15aa40 0x7f7fac0b21b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4013000 tx=0x7f7fa4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.467+0000 7f7fa1ffb700 1 -- 172.21.15.7:0/1580083726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4012e50 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.468+0000 7f7fa1ffb700 1 -- 172.21.15.7:0/1580083726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa4027470 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.468+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa4009a20 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.468+0000 7f7fa1ffb700 1 -- 172.21.15.7:0/1580083726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4027830 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.468+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac14d040 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.469+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fac04c3c0 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.469+0000 7f7fa1ffb700 1 -- 172.21.15.7:0/1580083726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7fa4024070 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.470+0000 7f7fa1ffb700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f90066f30 0x7f7f900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.470+0000 7f7fa1ffb700 1 -- 172.21.15.7:0/1580083726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7fa4086b70 con 0x7f7fac15aa40 2024-01-11T07:13:21.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.470+0000 7f7fb1c2c700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f90066f30 0x7f7f900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.471+0000 7f7fb1c2c700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f90066f30 0x7f7f900693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009f40 tx=0x7f7fa80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:21.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.474+0000 7f7fa1ffb700 1 -- 172.21.15.7:0/1580083726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fa4053d40 con 0x7f7fac15aa40 2024-01-11T07:13:21.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.727+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fac15b1f0 con 0x7f7fac15aa40 2024-01-11T07:13:21.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.728+0000 7f7fa1ffb700 1 -- 172.21.15.7:0/1580083726 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7fa4057360 con 0x7f7fac15aa40 2024-01-11T07:13:21.728 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:21.728 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f90066f30 msgr2=0x7f7f900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f90066f30 0x7f7f900693e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009f40 tx=0x7f7fa80091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac15aa40 msgr2=0x7f7fac0b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac15aa40 0x7f7fac0b21b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4013000 tx=0x7f7fa4019000 comp rx=0 tx=0).stop 2024-01-11T07:13:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 shutdown_connections 2024-01-11T07:13:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7f90066f30 0x7f7f900693e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac15aa40 0x7f7fac0b21b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fac063230 0x7f7fac14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 --2- 172.21.15.7:0/1580083726 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7fac063600 0x7f7fac14b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 >> 172.21.15.7:0/1580083726 conn(0x7f7fac0cd910 msgr2=0x7f7fac15d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:21.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 shutdown_connections 2024-01-11T07:13:21.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.730+0000 7f7fb3e90700 1 -- 172.21.15.7:0/1580083726 wait complete. 2024-01-11T07:13:21.731 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:21.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.875+0000 7f5591722700 1 Processor -- start 2024-01-11T07:13:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.875+0000 7f5591722700 1 -- start start 2024-01-11T07:13:21.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.876+0000 7f5591722700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c0cd6d0 0x7f558c0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.876+0000 7f5591722700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558c15acf0 con 0x7f558c15e420 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.876+0000 7f5591722700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558c15afd0 con 0x7f558c15d8a0 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.876+0000 7f5591722700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0ce980 con 0x7f558c0cd6d0 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.876+0000 7f558b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f558c15e420 0x7f558c159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59678/0 (socket says 172.21.15.7:59678) 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.876+0000 7f558b7fe700 1 -- 172.21.15.7:0/2026214302 learned_addr learned my addr 172.21.15.7:0/2026214302 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.876+0000 7f558affd700 1 --2- 172.21.15.7:0/2026214302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c0cd6d0 0x7f558c0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f5589ffb700 1 -- 172.21.15.7:0/2026214302 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435853480 0 0) 0x7f558c15acf0 con 0x7f558c15e420 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f5589ffb700 1 -- 172.21.15.7:0/2026214302 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5578003570 con 0x7f558c15e420 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f5589ffb700 1 -- 172.21.15.7:0/2026214302 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61259652 0 0) 0x7f558c15afd0 con 0x7f558c15d8a0 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f5589ffb700 1 -- 172.21.15.7:0/2026214302 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f558c15acf0 con 0x7f558c15d8a0 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f558affd700 1 -- 172.21.15.7:0/2026214302 >> v1:172.21.15.113:6789/0 conn(0x7f558c15d8a0 legacy=0x7f558c15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f558affd700 1 -- 172.21.15.7:0/2026214302 >> v1:172.21.15.7:6789/0 conn(0x7f558c15e420 legacy=0x7f558c159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f558affd700 1 -- 172.21.15.7:0/2026214302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c0cec60 con 0x7f558c0cd6d0 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f558affd700 1 --2- 172.21.15.7:0/2026214302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c0cd6d0 0x7f558c0cdaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f557c009910 tx=0x7f557c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5629b283be688523 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f5589ffb700 1 -- 172.21.15.7:0/2026214302 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c011630 con 0x7f558c0cd6d0 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f5589ffb700 1 -- 172.21.15.7:0/2026214302 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c011790 con 0x7f558c0cd6d0 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.877+0000 7f5589ffb700 1 -- 172.21.15.7:0/2026214302 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c011b30 con 0x7f558c0cd6d0 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.878+0000 7f5591722700 1 -- 172.21.15.7:0/2026214302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c0cd6d0 msgr2=0x7f558c0cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.878+0000 7f5591722700 1 --2- 172.21.15.7:0/2026214302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c0cd6d0 0x7f558c0cdaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f557c009910 tx=0x7f557c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.878+0000 7f5591722700 1 -- 172.21.15.7:0/2026214302 shutdown_connections 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.878+0000 7f5591722700 1 --2- 172.21.15.7:0/2026214302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c0cd6d0 0x7f558c0cdaa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.878+0000 7f5591722700 1 -- 172.21.15.7:0/2026214302 >> 172.21.15.7:0/2026214302 conn(0x7f558c071120 msgr2=0x7f558c06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.878+0000 7f5591722700 1 -- 172.21.15.7:0/2026214302 shutdown_connections 2024-01-11T07:13:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.878+0000 7f5591722700 1 -- 172.21.15.7:0/2026214302 wait complete. 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.879+0000 7f5591722700 1 Processor -- start 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.879+0000 7f5591722700 1 -- start start 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f5591722700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c0cd6d0 0x7f558c0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f5591722700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c15d8a0 0x7f558c149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f5591722700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c15e420 0x7f558c16a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f5591722700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0b6840 con 0x7f558c15d8a0 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f5591722700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0b2320 con 0x7f558c0cd6d0 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f5591722700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0b1560 con 0x7f558c15e420 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f558a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c15d8a0 0x7f558c149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f558a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c15d8a0 0x7f558c149730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45992/0 (socket says 172.21.15.7:45992) 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f558a7fc700 1 -- 172.21.15.7:0/2121034634 learned_addr learned my addr 172.21.15.7:0/2121034634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f558affd700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c0cd6d0 0x7f558c0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.880+0000 7f558b7fe700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c15e420 0x7f558c16a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f558a7fc700 1 -- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c15e420 msgr2=0x7f558c16a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f558a7fc700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c15e420 0x7f558c16a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f558a7fc700 1 -- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c0cd6d0 msgr2=0x7f558c0c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f558a7fc700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c0cd6d0 0x7f558c0c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f558a7fc700 1 -- 172.21.15.7:0/2121034634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c009420 con 0x7f558c15d8a0 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f558a7fc700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c15d8a0 0x7f558c149730 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f557400c790 tx=0x7f5574007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f5582ffd700 1 -- 172.21.15.7:0/2121034634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574010510 con 0x7f558c15d8a0 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f5582ffd700 1 -- 172.21.15.7:0/2121034634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5574010670 con 0x7f558c15d8a0 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558c0b1840 con 0x7f558c15d8a0 2024-01-11T07:13:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f5582ffd700 1 -- 172.21.15.7:0/2121034634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574010a10 con 0x7f558c15d8a0 2024-01-11T07:13:21.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.881+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c0b1e00 con 0x7f558c15d8a0 2024-01-11T07:13:21.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.882+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558c04c3c0 con 0x7f558c15d8a0 2024-01-11T07:13:21.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.883+0000 7f5582ffd700 1 -- 172.21.15.7:0/2121034634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f557401b3b0 con 0x7f558c15d8a0 2024-01-11T07:13:21.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.883+0000 7f5582ffd700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f556c066f30 0x7f556c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:21.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.883+0000 7f5582ffd700 1 -- 172.21.15.7:0/2121034634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5574083d90 con 0x7f558c15d8a0 2024-01-11T07:13:21.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.883+0000 7f558affd700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f556c066f30 0x7f556c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:21.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.884+0000 7f558affd700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f556c066f30 0x7f556c0693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f557c009860 tx=0x7f557c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:21.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:21.888+0000 7f5582ffd700 1 -- 172.21.15.7:0/2121034634 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5574051000 con 0x7f558c15d8a0 2024-01-11T07:13:22.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.093+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f558c0b2020 con 0x7f558c15d8a0 2024-01-11T07:13:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.094+0000 7f5582ffd700 1 -- 172.21.15.7:0/2121034634 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+15317 (secure 0 0 0) 0x7f5574017020 con 0x7f558c15d8a0 2024-01-11T07:13:22.093 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:22.093 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":135,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:13:19.284786+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:13:15.625657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-11T07:13:16.691544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f556c066f30 msgr2=0x7f556c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f556c066f30 0x7f556c0693e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f557c009860 tx=0x7f557c017000 comp rx=0 tx=0).stop 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c15d8a0 msgr2=0x7f558c149730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c15d8a0 0x7f558c149730 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f557400c790 tx=0x7f5574007e10 comp rx=0 tx=0).stop 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 shutdown_connections 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f556c066f30 0x7f556c0693e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f558c15e420 0x7f558c16a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f558c0cd6d0 0x7f558c0c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 --2- 172.21.15.7:0/2121034634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c15d8a0 0x7f558c149730 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 >> 172.21.15.7:0/2121034634 conn(0x7f558c071120 msgr2=0x7f558c1577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 shutdown_connections 2024-01-11T07:13:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.096+0000 7f5591722700 1 -- 172.21.15.7:0/2121034634 wait complete. 2024-01-11T07:13:22.110 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:13:22.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:13:22.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.241+0000 7febdb43f700 1 Processor -- start 2024-01-11T07:13:22.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.241+0000 7febdb43f700 1 -- start start 2024-01-11T07:13:22.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.242+0000 7febdb43f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40cafb0 0x7febd40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.242+0000 7febdb43f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7febd4158280 0x7febd40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.242+0000 7febdb43f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd4159300 con 0x7febd40b0970 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.242+0000 7febdb43f700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd40b0df0 con 0x7febd4158280 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.242+0000 7febdb43f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40cf260 con 0x7febd40cafb0 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febd99dc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7febd40b0970 0x7febd4159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59684/0 (socket says 172.21.15.7:59684) 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febd99dc700 1 -- 172.21.15.7:0/3264163710 learned_addr learned my addr 172.21.15.7:0/3264163710 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febd89da700 1 --2- 172.21.15.7:0/3264163710 >> v2:172.21.15.113:3300/0 conn(0x7febd4158280 0x7febd40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febd91db700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40cafb0 0x7febd40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333413745 0 0) 0x7febd4159300 con 0x7febd40b0970 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc0003570 con 0x7febd40b0970 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2627549765 0 0) 0x7febc0003570 con 0x7febd40b0970 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40cafb0 msgr2=0x7febd40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40cafb0 0x7febd40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 >> v2:172.21.15.113:3300/0 conn(0x7febd4158280 msgr2=0x7febd40b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 --2- 172.21.15.7:0/3264163710 >> v2:172.21.15.113:3300/0 conn(0x7febd4158280 0x7febd40b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40cf540 con 0x7febd40b0970 2024-01-11T07:13:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7febcc0017f0 con 0x7febd40b0970 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 >> v1:172.21.15.7:6789/0 conn(0x7febd40b0970 legacy=0x7febd4159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0003a60 0x7febc0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc0004810 0x7febc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40cafb0 0x7febc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40b0df0 con 0x7febc0004810 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40cf260 con 0x7febc0003a60 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc000fe70 con 0x7febd40cafb0 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febd89da700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc0004810 0x7febc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febd91db700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0003a60 0x7febc0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.243+0000 7febd99dc700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40cafb0 0x7febc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febd89da700 1 -- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40cafb0 msgr2=0x7febc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febd89da700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40cafb0 0x7febc00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febd89da700 1 -- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0003a60 msgr2=0x7febc0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febd89da700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0003a60 0x7febc0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febd89da700 1 -- 172.21.15.7:0/3264163710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0003570 con 0x7febc0004810 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febd89da700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc0004810 0x7febc0004c80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7febd4158650 tx=0x7febd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6149ff64fd22c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000e710 con 0x7febc0004810 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd000e870 con 0x7febc0004810 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.244+0000 7febc7fff700 1 -- 172.21.15.7:0/3264163710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000ec30 con 0x7febc0004810 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.245+0000 7febdb43f700 1 -- 172.21.15.7:0/3264163710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc0004810 msgr2=0x7febc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.245+0000 7febdb43f700 1 --2- 172.21.15.7:0/3264163710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc0004810 0x7febc0004c80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7febd4158650 tx=0x7febd00091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.245+0000 7febd99dc700 1 -- 172.21.15.7:0/3264163710 reap_dead start 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.245+0000 7febdb43f700 1 -- 172.21.15.7:0/3264163710 shutdown_connections 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.245+0000 7febdb43f700 1 -- 172.21.15.7:0/3264163710 >> 172.21.15.7:0/3264163710 conn(0x7febd4071100 msgr2=0x7febd406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.245+0000 7febdb43f700 1 -- 172.21.15.7:0/3264163710 shutdown_connections 2024-01-11T07:13:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.245+0000 7febdb43f700 1 -- 172.21.15.7:0/3264163710 wait complete. 2024-01-11T07:13:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.246+0000 7febdb43f700 1 Processor -- start 2024-01-11T07:13:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.246+0000 7febdb43f700 1 -- start start 2024-01-11T07:13:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febdb43f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0004810 0x7febd4156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febdb43f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40ca2a0 0x7febd40c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febdb43f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4156c50 0x7febd40c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febdb43f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40c4280 con 0x7febd4156c50 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febdb43f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41571b0 con 0x7febc0004810 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febdb43f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd4157490 con 0x7febd40ca2a0 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febd99dc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4156c50 0x7febd40c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febd99dc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4156c50 0x7febd40c3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46006/0 (socket says 172.21.15.7:46006) 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febd99dc700 1 -- 172.21.15.7:0/3178762115 learned_addr learned my addr 172.21.15.7:0/3178762115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febd91db700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0004810 0x7febd4156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.247+0000 7febd89da700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40ca2a0 0x7febd40c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febd99dc700 1 -- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40ca2a0 msgr2=0x7febd40c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febd99dc700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40ca2a0 0x7febd40c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febd99dc700 1 -- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0004810 msgr2=0x7febd4156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febd99dc700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0004810 0x7febd4156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febd99dc700 1 -- 172.21.15.7:0/3178762115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4157770 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febd99dc700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4156c50 0x7febd40c3d40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7febd4155d20 tx=0x7febcc002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febc57fa700 1 -- 172.21.15.7:0/3178762115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00ac70 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febc57fa700 1 -- 172.21.15.7:0/3178762115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00adf0 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40ae5f0 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febc57fa700 1 -- 172.21.15.7:0/3178762115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc013640 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.248+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd41531c0 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.249+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd41533b0 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.249+0000 7febc57fa700 1 -- 172.21.15.7:0/3178762115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7febcc0139f0 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.250+0000 7febc57fa700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7febb8066f30 0x7febb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.250+0000 7febc57fa700 1 -- 172.21.15.7:0/3178762115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7febcc084d30 con 0x7febd4156c50 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.250+0000 7febd91db700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7febb8066f30 0x7febb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.251+0000 7febd91db700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7febb8066f30 0x7febb80693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7febc8006f00 tx=0x7febc80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:22.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.254+0000 7febc57fa700 1 -- 172.21.15.7:0/3178762115 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febcc051fa0 con 0x7febd4156c50 2024-01-11T07:13:22.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.456+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7febd4069820 con 0x7febb8066f30 2024-01-11T07:13:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.465+0000 7febc57fa700 1 -- 172.21.15.7:0/3178762115 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285435 (secure 0 0 0) 0x7febc8014810 con 0x7febb8066f30 2024-01-11T07:13:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7febb8066f30 msgr2=0x7febb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7febb8066f30 0x7febb80693e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7febc8006f00 tx=0x7febc80091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4156c50 msgr2=0x7febd40c3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4156c50 0x7febd40c3d40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7febd4155d20 tx=0x7febcc002210 comp rx=0 tx=0).stop 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 shutdown_connections 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7febb8066f30 0x7febb80693e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4156c50 0x7febd40c3d40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febc0004810 0x7febd4156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.468+0000 7febdb43f700 1 --2- 172.21.15.7:0/3178762115 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7febd40ca2a0 0x7febd40c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.469+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 >> 172.21.15.7:0/3178762115 conn(0x7febd4071100 msgr2=0x7febd4067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.469+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 shutdown_connections 2024-01-11T07:13:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.469+0000 7febdb43f700 1 -- 172.21.15.7:0/3178762115 wait complete. 2024-01-11T07:13:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:13:22.482 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-01-11T07:13:21.929772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555608,"statfs":{"total":1600000000,"available":1592942969,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.650259"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292913+0000","last_change":"2024-01-11T07:13:18.292913+0000","last_active":"2024-01-11T07:13:18.292913+0000","last_peered":"2024-01-11T07:13:18.292913+0000","last_clean":"2024-01-11T07:13:18.292913+0000","last_became_active":"2024-01-11T07:13:18.292576+0000","last_became_peered":"2024-01-11T07:13:18.292576+0000","last_unstale":"2024-01-11T07:13:18.292913+0000","last_undegraded":"2024-01-11T07:13:18.292913+0000","last_fullsized":"2024-01-11T07:13:18.292913+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:19:56.151854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.308392+0000","last_change":"2024-01-11T07:13:18.308392+0000","last_active":"2024-01-11T07:13:18.308392+0000","last_peered":"2024-01-11T07:13:18.308392+0000","last_clean":"2024-01-11T07:13:18.308392+0000","last_became_active":"2024-01-11T07:13:18.308140+0000","last_became_peered":"2024-01-11T07:13:18.308140+0000","last_unstale":"2024-01-11T07:13:18.308392+0000","last_undegraded":"2024-01-11T07:13:18.308392+0000","last_fullsized":"2024-01-11T07:13:18.308392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:30:53.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.290108+0000","last_change":"2024-01-11T07:13:18.290108+0000","last_active":"2024-01-11T07:13:18.290108+0000","last_peered":"2024-01-11T07:13:18.290108+0000","last_clean":"2024-01-11T07:13:18.290108+0000","last_became_active":"2024-01-11T07:13:18.289283+0000","last_became_peered":"2024-01-11T07:13:18.289283+0000","last_unstale":"2024-01-11T07:13:18.290108+0000","last_undegraded":"2024-01-11T07:13:18.290108+0000","last_fullsized":"2024-01-11T07:13:18.290108+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:56:56.035612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.288576+0000","last_change":"2024-01-11T07:13:18.288576+0000","last_active":"2024-01-11T07:13:18.288576+0000","last_peered":"2024-01-11T07:13:18.288576+0000","last_clean":"2024-01-11T07:13:18.288576+0000","last_became_active":"2024-01-11T07:13:18.288271+0000","last_became_peered":"2024-01-11T07:13:18.288271+0000","last_unstale":"2024-01-11T07:13:18.288576+0000","last_undegraded":"2024-01-11T07:13:18.288576+0000","last_fullsized":"2024-01-11T07:13:18.288576+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:21:46.490695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306523+0000","last_change":"2024-01-11T07:13:18.306523+0000","last_active":"2024-01-11T07:13:18.306523+0000","last_peered":"2024-01-11T07:13:18.306523+0000","last_clean":"2024-01-11T07:13:18.306523+0000","last_became_active":"2024-01-11T07:13:18.306247+0000","last_became_peered":"2024-01-11T07:13:18.306247+0000","last_unstale":"2024-01-11T07:13:18.306523+0000","last_undegraded":"2024-01-11T07:13:18.306523+0000","last_fullsized":"2024-01-11T07:13:18.306523+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:25:52.822147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306933+0000","last_change":"2024-01-11T07:13:18.306933+0000","last_active":"2024-01-11T07:13:18.306933+0000","last_peered":"2024-01-11T07:13:18.306933+0000","last_clean":"2024-01-11T07:13:18.306933+0000","last_became_active":"2024-01-11T07:13:18.306139+0000","last_became_peered":"2024-01-11T07:13:18.306139+0000","last_unstale":"2024-01-11T07:13:18.306933+0000","last_undegraded":"2024-01-11T07:13:18.306933+0000","last_fullsized":"2024-01-11T07:13:18.306933+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:08:09.538380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.299814+0000","last_change":"2024-01-11T07:13:18.299814+0000","last_active":"2024-01-11T07:13:18.299814+0000","last_peered":"2024-01-11T07:13:18.299814+0000","last_clean":"2024-01-11T07:13:18.299814+0000","last_became_active":"2024-01-11T07:13:18.299431+0000","last_became_peered":"2024-01-11T07:13:18.299431+0000","last_unstale":"2024-01-11T07:13:18.299814+0000","last_undegraded":"2024-01-11T07:13:18.299814+0000","last_fullsized":"2024-01-11T07:13:18.299814+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:31:49.646204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.303527+0000","last_change":"2024-01-11T07:13:18.303527+0000","last_active":"2024-01-11T07:13:18.303527+0000","last_peered":"2024-01-11T07:13:18.303527+0000","last_clean":"2024-01-11T07:13:18.303527+0000","last_became_active":"2024-01-11T07:13:18.303354+0000","last_became_peered":"2024-01-11T07:13:18.303354+0000","last_unstale":"2024-01-11T07:13:18.303527+0000","last_undegraded":"2024-01-11T07:13:18.303527+0000","last_fullsized":"2024-01-11T07:13:18.303527+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:14:00.649612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295797+0000","last_change":"2024-01-11T07:13:18.295797+0000","last_active":"2024-01-11T07:13:18.295797+0000","last_peered":"2024-01-11T07:13:18.295797+0000","last_clean":"2024-01-11T07:13:18.295797+0000","last_became_active":"2024-01-11T07:13:18.295552+0000","last_became_peered":"2024-01-11T07:13:18.295552+0000","last_unstale":"2024-01-11T07:13:18.295797+0000","last_undegraded":"2024-01-11T07:13:18.295797+0000","last_fullsized":"2024-01-11T07:13:18.295797+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:32:31.066220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.294028+0000","last_change":"2024-01-11T07:13:18.294028+0000","last_active":"2024-01-11T07:13:18.294028+0000","last_peered":"2024-01-11T07:13:18.294028+0000","last_clean":"2024-01-11T07:13:18.294028+0000","last_became_active":"2024-01-11T07:13:18.293703+0000","last_became_peered":"2024-01-11T07:13:18.293703+0000","last_unstale":"2024-01-11T07:13:18.294028+0000","last_undegraded":"2024-01-11T07:13:18.294028+0000","last_fullsized":"2024-01-11T07:13:18.294028+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:25.049652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297727+0000","last_change":"2024-01-11T07:13:18.297727+0000","last_active":"2024-01-11T07:13:18.297727+0000","last_peered":"2024-01-11T07:13:18.297727+0000","last_clean":"2024-01-11T07:13:18.297727+0000","last_became_active":"2024-01-11T07:13:18.297531+0000","last_became_peered":"2024-01-11T07:13:18.297531+0000","last_unstale":"2024-01-11T07:13:18.297727+0000","last_undegraded":"2024-01-11T07:13:18.297727+0000","last_fullsized":"2024-01-11T07:13:18.297727+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:19:08.390105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292997+0000","last_change":"2024-01-11T07:13:18.292997+0000","last_active":"2024-01-11T07:13:18.292997+0000","last_peered":"2024-01-11T07:13:18.292997+0000","last_clean":"2024-01-11T07:13:18.292997+0000","last_became_active":"2024-01-11T07:13:18.292711+0000","last_became_peered":"2024-01-11T07:13:18.292711+0000","last_unstale":"2024-01-11T07:13:18.292997+0000","last_undegraded":"2024-01-11T07:13:18.292997+0000","last_fullsized":"2024-01-11T07:13:18.292997+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:57.927263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292720+0000","last_change":"2024-01-11T07:13:18.292720+0000","last_active":"2024-01-11T07:13:18.292720+0000","last_peered":"2024-01-11T07:13:18.292720+0000","last_clean":"2024-01-11T07:13:18.292720+0000","last_became_active":"2024-01-11T07:13:18.292436+0000","last_became_peered":"2024-01-11T07:13:18.292436+0000","last_unstale":"2024-01-11T07:13:18.292720+0000","last_undegraded":"2024-01-11T07:13:18.292720+0000","last_fullsized":"2024-01-11T07:13:18.292720+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:49:42.554340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.293542+0000","last_change":"2024-01-11T07:13:18.293542+0000","last_active":"2024-01-11T07:13:18.293542+0000","last_peered":"2024-01-11T07:13:18.293542+0000","last_clean":"2024-01-11T07:13:18.293542+0000","last_became_active":"2024-01-11T07:13:18.293278+0000","last_became_peered":"2024-01-11T07:13:18.293278+0000","last_unstale":"2024-01-11T07:13:18.293542+0000","last_undegraded":"2024-01-11T07:13:18.293542+0000","last_fullsized":"2024-01-11T07:13:18.293542+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:12:18.804791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.311803+0000","last_change":"2024-01-11T07:13:18.311803+0000","last_active":"2024-01-11T07:13:18.311803+0000","last_peered":"2024-01-11T07:13:18.311803+0000","last_clean":"2024-01-11T07:13:18.311803+0000","last_became_active":"2024-01-11T07:13:18.311304+0000","last_became_peered":"2024-01-11T07:13:18.311304+0000","last_unstale":"2024-01-11T07:13:18.311803+0000","last_undegraded":"2024-01-11T07:13:18.311803+0000","last_fullsized":"2024-01-11T07:13:18.311803+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:44.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.288672+0000","last_change":"2024-01-11T07:13:18.288672+0000","last_active":"2024-01-11T07:13:18.288672+0000","last_peered":"2024-01-11T07:13:18.288672+0000","last_clean":"2024-01-11T07:13:18.288672+0000","last_became_active":"2024-01-11T07:13:18.288430+0000","last_became_peered":"2024-01-11T07:13:18.288430+0000","last_unstale":"2024-01-11T07:13:18.288672+0000","last_undegraded":"2024-01-11T07:13:18.288672+0000","last_fullsized":"2024-01-11T07:13:18.288672+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:33:29.540323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295490+0000","last_change":"2024-01-11T07:13:18.295490+0000","last_active":"2024-01-11T07:13:18.295490+0000","last_peered":"2024-01-11T07:13:18.295490+0000","last_clean":"2024-01-11T07:13:18.295490+0000","last_became_active":"2024-01-11T07:13:18.295203+0000","last_became_peered":"2024-01-11T07:13:18.295203+0000","last_unstale":"2024-01-11T07:13:18.295490+0000","last_undegraded":"2024-01-11T07:13:18.295490+0000","last_fullsized":"2024-01-11T07:13:18.295490+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:02:14.868338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.298547+0000","last_change":"2024-01-11T07:13:18.298547+0000","last_active":"2024-01-11T07:13:18.298547+0000","last_peered":"2024-01-11T07:13:18.298547+0000","last_clean":"2024-01-11T07:13:18.298547+0000","last_became_active":"2024-01-11T07:13:18.298311+0000","last_became_peered":"2024-01-11T07:13:18.298311+0000","last_unstale":"2024-01-11T07:13:18.298547+0000","last_undegraded":"2024-01-11T07:13:18.298547+0000","last_fullsized":"2024-01-11T07:13:18.298547+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:39:10.000850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292616+0000","last_change":"2024-01-11T07:13:18.292616+0000","last_active":"2024-01-11T07:13:18.292616+0000","last_peered":"2024-01-11T07:13:18.292616+0000","last_clean":"2024-01-11T07:13:18.292616+0000","last_became_active":"2024-01-11T07:13:18.292376+0000","last_became_peered":"2024-01-11T07:13:18.292376+0000","last_unstale":"2024-01-11T07:13:18.292616+0000","last_undegraded":"2024-01-11T07:13:18.292616+0000","last_fullsized":"2024-01-11T07:13:18.292616+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:00:46.176711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.296824+0000","last_change":"2024-01-11T07:13:18.296824+0000","last_active":"2024-01-11T07:13:18.296824+0000","last_peered":"2024-01-11T07:13:18.296824+0000","last_clean":"2024-01-11T07:13:18.296824+0000","last_became_active":"2024-01-11T07:13:18.296263+0000","last_became_peered":"2024-01-11T07:13:18.296263+0000","last_unstale":"2024-01-11T07:13:18.296824+0000","last_undegraded":"2024-01-11T07:13:18.296824+0000","last_fullsized":"2024-01-11T07:13:18.296824+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:07.274571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.305797+0000","last_change":"2024-01-11T07:13:18.305797+0000","last_active":"2024-01-11T07:13:18.305797+0000","last_peered":"2024-01-11T07:13:18.305797+0000","last_clean":"2024-01-11T07:13:18.305797+0000","last_became_active":"2024-01-11T07:13:18.305571+0000","last_became_peered":"2024-01-11T07:13:18.305571+0000","last_unstale":"2024-01-11T07:13:18.305797+0000","last_undegraded":"2024-01-11T07:13:18.305797+0000","last_fullsized":"2024-01-11T07:13:18.305797+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:12:18.804791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.302118+0000","last_change":"2024-01-11T07:13:18.302118+0000","last_active":"2024-01-11T07:13:18.302118+0000","last_peered":"2024-01-11T07:13:18.302118+0000","last_clean":"2024-01-11T07:13:18.302118+0000","last_became_active":"2024-01-11T07:13:18.301874+0000","last_became_peered":"2024-01-11T07:13:18.301874+0000","last_unstale":"2024-01-11T07:13:18.302118+0000","last_undegraded":"2024-01-11T07:13:18.302118+0000","last_fullsized":"2024-01-11T07:13:18.302118+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:19:30.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.296671+0000","last_change":"2024-01-11T07:13:18.296671+0000","last_active":"2024-01-11T07:13:18.296671+0000","last_peered":"2024-01-11T07:13:18.296671+0000","last_clean":"2024-01-11T07:13:18.296671+0000","last_became_active":"2024-01-11T07:13:18.295591+0000","last_became_peered":"2024-01-11T07:13:18.295591+0000","last_unstale":"2024-01-11T07:13:18.296671+0000","last_undegraded":"2024-01-11T07:13:18.296671+0000","last_fullsized":"2024-01-11T07:13:18.296671+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:59:19.451948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297225+0000","last_change":"2024-01-11T07:13:18.297225+0000","last_active":"2024-01-11T07:13:18.297225+0000","last_peered":"2024-01-11T07:13:18.297225+0000","last_clean":"2024-01-11T07:13:18.297225+0000","last_became_active":"2024-01-11T07:13:18.296970+0000","last_became_peered":"2024-01-11T07:13:18.296970+0000","last_unstale":"2024-01-11T07:13:18.297225+0000","last_undegraded":"2024-01-11T07:13:18.297225+0000","last_fullsized":"2024-01-11T07:13:18.297225+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:17:48.754294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297190+0000","last_change":"2024-01-11T07:13:18.297190+0000","last_active":"2024-01-11T07:13:18.297190+0000","last_peered":"2024-01-11T07:13:18.297190+0000","last_clean":"2024-01-11T07:13:18.297190+0000","last_became_active":"2024-01-11T07:13:18.296586+0000","last_became_peered":"2024-01-11T07:13:18.296586+0000","last_unstale":"2024-01-11T07:13:18.297190+0000","last_undegraded":"2024-01-11T07:13:18.297190+0000","last_fullsized":"2024-01-11T07:13:18.297190+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:56:56.035612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.313148+0000","last_change":"2024-01-11T07:13:18.313148+0000","last_active":"2024-01-11T07:13:18.313148+0000","last_peered":"2024-01-11T07:13:18.313148+0000","last_clean":"2024-01-11T07:13:18.313148+0000","last_became_active":"2024-01-11T07:13:18.312739+0000","last_became_peered":"2024-01-11T07:13:18.312739+0000","last_unstale":"2024-01-11T07:13:18.313148+0000","last_undegraded":"2024-01-11T07:13:18.313148+0000","last_fullsized":"2024-01-11T07:13:18.313148+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:13:37.840332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306714+0000","last_change":"2024-01-11T07:13:18.306714+0000","last_active":"2024-01-11T07:13:18.306714+0000","last_peered":"2024-01-11T07:13:18.306714+0000","last_clean":"2024-01-11T07:13:18.306714+0000","last_became_active":"2024-01-11T07:13:18.306119+0000","last_became_peered":"2024-01-11T07:13:18.306119+0000","last_unstale":"2024-01-11T07:13:18.306714+0000","last_undegraded":"2024-01-11T07:13:18.306714+0000","last_fullsized":"2024-01-11T07:13:18.306714+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:24:03.561483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.302050+0000","last_change":"2024-01-11T07:13:18.302050+0000","last_active":"2024-01-11T07:13:18.302050+0000","last_peered":"2024-01-11T07:13:18.302050+0000","last_clean":"2024-01-11T07:13:18.302050+0000","last_became_active":"2024-01-11T07:13:18.301782+0000","last_became_peered":"2024-01-11T07:13:18.301782+0000","last_unstale":"2024-01-11T07:13:18.302050+0000","last_undegraded":"2024-01-11T07:13:18.302050+0000","last_fullsized":"2024-01-11T07:13:18.302050+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:41.118932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295273+0000","last_change":"2024-01-11T07:13:18.295273+0000","last_active":"2024-01-11T07:13:18.295273+0000","last_peered":"2024-01-11T07:13:18.295273+0000","last_clean":"2024-01-11T07:13:18.295273+0000","last_became_active":"2024-01-11T07:13:18.294956+0000","last_became_peered":"2024-01-11T07:13:18.294956+0000","last_unstale":"2024-01-11T07:13:18.295273+0000","last_undegraded":"2024-01-11T07:13:18.295273+0000","last_fullsized":"2024-01-11T07:13:18.295273+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:51.391309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.310815+0000","last_change":"2024-01-11T07:13:18.310815+0000","last_active":"2024-01-11T07:13:18.310815+0000","last_peered":"2024-01-11T07:13:18.310815+0000","last_clean":"2024-01-11T07:13:18.310815+0000","last_became_active":"2024-01-11T07:13:18.310627+0000","last_became_peered":"2024-01-11T07:13:18.310627+0000","last_unstale":"2024-01-11T07:13:18.310815+0000","last_undegraded":"2024-01-11T07:13:18.310815+0000","last_fullsized":"2024-01-11T07:13:18.310815+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:24:03.561483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.312965+0000","last_change":"2024-01-11T07:13:18.312965+0000","last_active":"2024-01-11T07:13:18.312965+0000","last_peered":"2024-01-11T07:13:18.312965+0000","last_clean":"2024-01-11T07:13:18.312965+0000","last_became_active":"2024-01-11T07:13:18.312501+0000","last_became_peered":"2024-01-11T07:13:18.312501+0000","last_unstale":"2024-01-11T07:13:18.312965+0000","last_undegraded":"2024-01-11T07:13:18.312965+0000","last_fullsized":"2024-01-11T07:13:18.312965+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:05.275296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.307485+0000","last_change":"2024-01-11T07:13:18.307485+0000","last_active":"2024-01-11T07:13:18.307485+0000","last_peered":"2024-01-11T07:13:18.307485+0000","last_clean":"2024-01-11T07:13:18.307485+0000","last_became_active":"2024-01-11T07:13:18.307229+0000","last_became_peered":"2024-01-11T07:13:18.307229+0000","last_unstale":"2024-01-11T07:13:18.307485+0000","last_undegraded":"2024-01-11T07:13:18.307485+0000","last_fullsized":"2024-01-11T07:13:18.307485+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:09.898464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.300149+0000","last_change":"2024-01-11T07:13:18.300149+0000","last_active":"2024-01-11T07:13:18.300149+0000","last_peered":"2024-01-11T07:13:18.300149+0000","last_clean":"2024-01-11T07:13:18.300149+0000","last_became_active":"2024-01-11T07:13:18.299918+0000","last_became_peered":"2024-01-11T07:13:18.299918+0000","last_unstale":"2024-01-11T07:13:18.300149+0000","last_undegraded":"2024-01-11T07:13:18.300149+0000","last_fullsized":"2024-01-11T07:13:18.300149+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:01:53.801845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"134'4","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.308549+0000","last_change":"2024-01-11T07:13:17.296028+0000","last_active":"2024-01-11T07:13:18.308549+0000","last_peered":"2024-01-11T07:13:18.308549+0000","last_clean":"2024-01-11T07:13:18.308549+0000","last_became_active":"2024-01-11T07:13:17.295770+0000","last_became_peered":"2024-01-11T07:13:17.295770+0000","last_unstale":"2024-01-11T07:13:18.308549+0000","last_undegraded":"2024-01-11T07:13:18.308549+0000","last_fullsized":"2024-01-11T07:13:18.308549+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:36.657856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292265+0000","last_change":"2024-01-11T07:13:18.292265+0000","last_active":"2024-01-11T07:13:18.292265+0000","last_peered":"2024-01-11T07:13:18.292265+0000","last_clean":"2024-01-11T07:13:18.292265+0000","last_became_active":"2024-01-11T07:13:18.291800+0000","last_became_peered":"2024-01-11T07:13:18.291800+0000","last_unstale":"2024-01-11T07:13:18.292265+0000","last_undegraded":"2024-01-11T07:13:18.292265+0000","last_fullsized":"2024-01-11T07:13:18.292265+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:00:04.530196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.288845+0000","last_change":"2024-01-11T07:13:17.294772+0000","last_active":"2024-01-11T07:13:18.288845+0000","last_peered":"2024-01-11T07:13:18.288845+0000","last_clean":"2024-01-11T07:13:18.288845+0000","last_became_active":"2024-01-11T07:13:17.292791+0000","last_became_peered":"2024-01-11T07:13:17.292791+0000","last_unstale":"2024-01-11T07:13:18.288845+0000","last_undegraded":"2024-01-11T07:13:18.288845+0000","last_fullsized":"2024-01-11T07:13:18.288845+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:00:03.523470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306769+0000","last_change":"2024-01-11T07:13:18.306769+0000","last_active":"2024-01-11T07:13:18.306769+0000","last_peered":"2024-01-11T07:13:18.306769+0000","last_clean":"2024-01-11T07:13:18.306769+0000","last_became_active":"2024-01-11T07:13:18.305779+0000","last_became_peered":"2024-01-11T07:13:18.305779+0000","last_unstale":"2024-01-11T07:13:18.306769+0000","last_undegraded":"2024-01-11T07:13:18.306769+0000","last_fullsized":"2024-01-11T07:13:18.306769+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:42:43.114680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.317061+0000","last_change":"2024-01-11T07:13:17.296375+0000","last_active":"2024-01-11T07:13:18.317061+0000","last_peered":"2024-01-11T07:13:18.317061+0000","last_clean":"2024-01-11T07:13:18.317061+0000","last_became_active":"2024-01-11T07:13:17.295991+0000","last_became_peered":"2024-01-11T07:13:17.295991+0000","last_unstale":"2024-01-11T07:13:18.317061+0000","last_undegraded":"2024-01-11T07:13:18.317061+0000","last_fullsized":"2024-01-11T07:13:18.317061+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:38:42.695868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.293555+0000","last_change":"2024-01-11T07:13:18.293555+0000","last_active":"2024-01-11T07:13:18.293555+0000","last_peered":"2024-01-11T07:13:18.293555+0000","last_clean":"2024-01-11T07:13:18.293555+0000","last_became_active":"2024-01-11T07:13:18.293372+0000","last_became_peered":"2024-01-11T07:13:18.293372+0000","last_unstale":"2024-01-11T07:13:18.293555+0000","last_undegraded":"2024-01-11T07:13:18.293555+0000","last_fullsized":"2024-01-11T07:13:18.293555+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:59:19.451948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.314142+0000","last_change":"2024-01-11T07:13:17.300690+0000","last_active":"2024-01-11T07:13:18.314142+0000","last_peered":"2024-01-11T07:13:18.314142+0000","last_clean":"2024-01-11T07:13:18.314142+0000","last_became_active":"2024-01-11T07:13:17.300338+0000","last_became_peered":"2024-01-11T07:13:17.300338+0000","last_unstale":"2024-01-11T07:13:18.314142+0000","last_undegraded":"2024-01-11T07:13:18.314142+0000","last_fullsized":"2024-01-11T07:13:18.314142+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:09:25.512858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.305801+0000","last_change":"2024-01-11T07:13:18.305801+0000","last_active":"2024-01-11T07:13:18.305801+0000","last_peered":"2024-01-11T07:13:18.305801+0000","last_clean":"2024-01-11T07:13:18.305801+0000","last_became_active":"2024-01-11T07:13:18.305331+0000","last_became_peered":"2024-01-11T07:13:18.305331+0000","last_unstale":"2024-01-11T07:13:18.305801+0000","last_undegraded":"2024-01-11T07:13:18.305801+0000","last_fullsized":"2024-01-11T07:13:18.305801+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:04:46.248050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297009+0000","last_change":"2024-01-11T07:13:17.297043+0000","last_active":"2024-01-11T07:13:18.297009+0000","last_peered":"2024-01-11T07:13:18.297009+0000","last_clean":"2024-01-11T07:13:18.297009+0000","last_became_active":"2024-01-11T07:13:17.296564+0000","last_became_peered":"2024-01-11T07:13:17.296564+0000","last_unstale":"2024-01-11T07:13:18.297009+0000","last_undegraded":"2024-01-11T07:13:18.297009+0000","last_fullsized":"2024-01-11T07:13:18.297009+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:51:33.647768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306446+0000","last_change":"2024-01-11T07:13:18.306446+0000","last_active":"2024-01-11T07:13:18.306446+0000","last_peered":"2024-01-11T07:13:18.306446+0000","last_clean":"2024-01-11T07:13:18.306446+0000","last_became_active":"2024-01-11T07:13:18.306239+0000","last_became_peered":"2024-01-11T07:13:18.306239+0000","last_unstale":"2024-01-11T07:13:18.306446+0000","last_undegraded":"2024-01-11T07:13:18.306446+0000","last_fullsized":"2024-01-11T07:13:18.306446+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:49:42.554340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295911+0000","last_change":"2024-01-11T07:13:17.297214+0000","last_active":"2024-01-11T07:13:18.295911+0000","last_peered":"2024-01-11T07:13:18.295911+0000","last_clean":"2024-01-11T07:13:18.295911+0000","last_became_active":"2024-01-11T07:13:17.296959+0000","last_became_peered":"2024-01-11T07:13:17.296959+0000","last_unstale":"2024-01-11T07:13:18.295911+0000","last_undegraded":"2024-01-11T07:13:18.295911+0000","last_fullsized":"2024-01-11T07:13:18.295911+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:38:32.291346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.304162+0000","last_change":"2024-01-11T07:13:18.304162+0000","last_active":"2024-01-11T07:13:18.304162+0000","last_peered":"2024-01-11T07:13:18.304162+0000","last_clean":"2024-01-11T07:13:18.304162+0000","last_became_active":"2024-01-11T07:13:18.303760+0000","last_became_peered":"2024-01-11T07:13:18.303760+0000","last_unstale":"2024-01-11T07:13:18.304162+0000","last_undegraded":"2024-01-11T07:13:18.304162+0000","last_fullsized":"2024-01-11T07:13:18.304162+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:21:34.087245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.295416+0000","last_change":"2024-01-11T07:13:17.295294+0000","last_active":"2024-01-11T07:13:17.295416+0000","last_peered":"2024-01-11T07:13:17.295416+0000","last_clean":"2024-01-11T07:13:17.295416+0000","last_became_active":"2024-01-11T07:13:17.295056+0000","last_became_peered":"2024-01-11T07:13:17.295056+0000","last_unstale":"2024-01-11T07:13:17.295416+0000","last_undegraded":"2024-01-11T07:13:17.295416+0000","last_fullsized":"2024-01-11T07:13:17.295416+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:14.944854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295509+0000","last_change":"2024-01-11T07:13:18.295509+0000","last_active":"2024-01-11T07:13:18.295509+0000","last_peered":"2024-01-11T07:13:18.295509+0000","last_clean":"2024-01-11T07:13:18.295509+0000","last_became_active":"2024-01-11T07:13:18.295278+0000","last_became_peered":"2024-01-11T07:13:18.295278+0000","last_unstale":"2024-01-11T07:13:18.295509+0000","last_undegraded":"2024-01-11T07:13:18.295509+0000","last_fullsized":"2024-01-11T07:13:18.295509+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:01:53.801845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.300446+0000","last_change":"2024-01-11T07:13:17.299335+0000","last_active":"2024-01-11T07:13:17.300446+0000","last_peered":"2024-01-11T07:13:17.300446+0000","last_clean":"2024-01-11T07:13:17.300446+0000","last_became_active":"2024-01-11T07:13:17.299089+0000","last_became_peered":"2024-01-11T07:13:17.299089+0000","last_unstale":"2024-01-11T07:13:17.300446+0000","last_undegraded":"2024-01-11T07:13:17.300446+0000","last_fullsized":"2024-01-11T07:13:17.300446+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:19:29.062828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.300962+0000","last_change":"2024-01-11T07:13:18.300962+0000","last_active":"2024-01-11T07:13:18.300962+0000","last_peered":"2024-01-11T07:13:18.300962+0000","last_clean":"2024-01-11T07:13:18.300962+0000","last_became_active":"2024-01-11T07:13:18.300795+0000","last_became_peered":"2024-01-11T07:13:18.300795+0000","last_unstale":"2024-01-11T07:13:18.300962+0000","last_undegraded":"2024-01-11T07:13:18.300962+0000","last_fullsized":"2024-01-11T07:13:18.300962+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:44.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.300614+0000","last_change":"2024-01-11T07:13:17.300549+0000","last_active":"2024-01-11T07:13:17.300614+0000","last_peered":"2024-01-11T07:13:17.300614+0000","last_clean":"2024-01-11T07:13:17.300614+0000","last_became_active":"2024-01-11T07:13:17.300007+0000","last_became_peered":"2024-01-11T07:13:17.300007+0000","last_unstale":"2024-01-11T07:13:17.300614+0000","last_undegraded":"2024-01-11T07:13:17.300614+0000","last_fullsized":"2024-01-11T07:13:17.300614+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:16:04.250535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.296696+0000","last_change":"2024-01-11T07:13:18.296696+0000","last_active":"2024-01-11T07:13:18.296696+0000","last_peered":"2024-01-11T07:13:18.296696+0000","last_clean":"2024-01-11T07:13:18.296696+0000","last_became_active":"2024-01-11T07:13:18.295962+0000","last_became_peered":"2024-01-11T07:13:18.295962+0000","last_unstale":"2024-01-11T07:13:18.296696+0000","last_undegraded":"2024-01-11T07:13:18.296696+0000","last_fullsized":"2024-01-11T07:13:18.296696+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:24:33.389471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.296625+0000","last_change":"2024-01-11T07:13:17.296548+0000","last_active":"2024-01-11T07:13:17.296625+0000","last_peered":"2024-01-11T07:13:17.296625+0000","last_clean":"2024-01-11T07:13:17.296625+0000","last_became_active":"2024-01-11T07:13:17.296286+0000","last_became_peered":"2024-01-11T07:13:17.296286+0000","last_unstale":"2024-01-11T07:13:17.296625+0000","last_undegraded":"2024-01-11T07:13:17.296625+0000","last_fullsized":"2024-01-11T07:13:17.296625+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:19:15.784688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.303867+0000","last_change":"2024-01-11T07:13:18.303867+0000","last_active":"2024-01-11T07:13:18.303867+0000","last_peered":"2024-01-11T07:13:18.303867+0000","last_clean":"2024-01-11T07:13:18.303867+0000","last_became_active":"2024-01-11T07:13:18.303605+0000","last_became_peered":"2024-01-11T07:13:18.303605+0000","last_unstale":"2024-01-11T07:13:18.303867+0000","last_undegraded":"2024-01-11T07:13:18.303867+0000","last_fullsized":"2024-01-11T07:13:18.303867+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:47:50.081608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"134'3","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.314161+0000","last_change":"2024-01-11T07:13:17.295227+0000","last_active":"2024-01-11T07:13:18.314161+0000","last_peered":"2024-01-11T07:13:18.314161+0000","last_clean":"2024-01-11T07:13:18.314161+0000","last_became_active":"2024-01-11T07:13:17.294983+0000","last_became_peered":"2024-01-11T07:13:17.294983+0000","last_unstale":"2024-01-11T07:13:18.314161+0000","last_undegraded":"2024-01-11T07:13:18.314161+0000","last_fullsized":"2024-01-11T07:13:18.314161+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:13:59.642885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.301514+0000","last_change":"2024-01-11T07:13:18.301514+0000","last_active":"2024-01-11T07:13:18.301514+0000","last_peered":"2024-01-11T07:13:18.301514+0000","last_clean":"2024-01-11T07:13:18.301514+0000","last_became_active":"2024-01-11T07:13:18.301322+0000","last_became_peered":"2024-01-11T07:13:18.301322+0000","last_unstale":"2024-01-11T07:13:18.301514+0000","last_undegraded":"2024-01-11T07:13:18.301514+0000","last_fullsized":"2024-01-11T07:13:18.301514+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:42:43.114680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"134'6","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.314521+0000","last_change":"2024-01-11T07:13:17.299287+0000","last_active":"2024-01-11T07:13:18.314521+0000","last_peered":"2024-01-11T07:13:18.314521+0000","last_clean":"2024-01-11T07:13:18.314521+0000","last_became_active":"2024-01-11T07:13:17.298550+0000","last_became_peered":"2024-01-11T07:13:17.298550+0000","last_unstale":"2024-01-11T07:13:18.314521+0000","last_undegraded":"2024-01-11T07:13:18.314521+0000","last_fullsized":"2024-01-11T07:13:18.314521+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:40.112206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.302865+0000","last_change":"2024-01-11T07:13:18.302865+0000","last_active":"2024-01-11T07:13:18.302865+0000","last_peered":"2024-01-11T07:13:18.302865+0000","last_clean":"2024-01-11T07:13:18.302865+0000","last_became_active":"2024-01-11T07:13:18.302633+0000","last_became_peered":"2024-01-11T07:13:18.302633+0000","last_unstale":"2024-01-11T07:13:18.302865+0000","last_undegraded":"2024-01-11T07:13:18.302865+0000","last_fullsized":"2024-01-11T07:13:18.302865+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:46:51.487989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.309309+0000","last_change":"2024-01-11T07:13:17.290909+0000","last_active":"2024-01-11T07:13:18.309309+0000","last_peered":"2024-01-11T07:13:18.309309+0000","last_clean":"2024-01-11T07:13:18.309309+0000","last_became_active":"2024-01-11T07:13:17.290606+0000","last_became_peered":"2024-01-11T07:13:17.290606+0000","last_unstale":"2024-01-11T07:13:18.309309+0000","last_undegraded":"2024-01-11T07:13:18.309309+0000","last_fullsized":"2024-01-11T07:13:18.309309+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:36.657856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.313675+0000","last_change":"2024-01-11T07:13:18.313675+0000","last_active":"2024-01-11T07:13:18.313675+0000","last_peered":"2024-01-11T07:13:18.313675+0000","last_clean":"2024-01-11T07:13:18.313675+0000","last_became_active":"2024-01-11T07:13:18.313203+0000","last_became_peered":"2024-01-11T07:13:18.313203+0000","last_unstale":"2024-01-11T07:13:18.313675+0000","last_undegraded":"2024-01-11T07:13:18.313675+0000","last_fullsized":"2024-01-11T07:13:18.313675+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:14:00.649612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.292232+0000","last_change":"2024-01-11T07:13:17.292098+0000","last_active":"2024-01-11T07:13:17.292232+0000","last_peered":"2024-01-11T07:13:17.292232+0000","last_clean":"2024-01-11T07:13:17.292232+0000","last_became_active":"2024-01-11T07:13:17.291852+0000","last_became_peered":"2024-01-11T07:13:17.291852+0000","last_unstale":"2024-01-11T07:13:17.292232+0000","last_undegraded":"2024-01-11T07:13:17.292232+0000","last_fullsized":"2024-01-11T07:13:17.292232+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:08.891738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.302745+0000","last_change":"2024-01-11T07:13:18.302745+0000","last_active":"2024-01-11T07:13:18.302745+0000","last_peered":"2024-01-11T07:13:18.302745+0000","last_clean":"2024-01-11T07:13:18.302745+0000","last_became_active":"2024-01-11T07:13:18.302468+0000","last_became_peered":"2024-01-11T07:13:18.302468+0000","last_unstale":"2024-01-11T07:13:18.302745+0000","last_undegraded":"2024-01-11T07:13:18.302745+0000","last_fullsized":"2024-01-11T07:13:18.302745+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:28:46.090843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"134'8","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.317649+0000","last_change":"2024-01-11T07:13:17.301276+0000","last_active":"2024-01-11T07:13:18.317649+0000","last_peered":"2024-01-11T07:13:18.317649+0000","last_clean":"2024-01-11T07:13:18.317649+0000","last_became_active":"2024-01-11T07:13:17.301027+0000","last_became_peered":"2024-01-11T07:13:17.301027+0000","last_unstale":"2024-01-11T07:13:18.317649+0000","last_undegraded":"2024-01-11T07:13:18.317649+0000","last_fullsized":"2024-01-11T07:13:18.317649+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:02:07.545480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295022+0000","last_change":"2024-01-11T07:13:18.295022+0000","last_active":"2024-01-11T07:13:18.295022+0000","last_peered":"2024-01-11T07:13:18.295022+0000","last_clean":"2024-01-11T07:13:18.295022+0000","last_became_active":"2024-01-11T07:13:18.294598+0000","last_became_peered":"2024-01-11T07:13:18.294598+0000","last_unstale":"2024-01-11T07:13:18.295022+0000","last_undegraded":"2024-01-11T07:13:18.295022+0000","last_fullsized":"2024-01-11T07:13:18.295022+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:59:25.436030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.286364+0000","last_change":"2024-01-11T07:13:17.292532+0000","last_active":"2024-01-11T07:13:18.286364+0000","last_peered":"2024-01-11T07:13:18.286364+0000","last_clean":"2024-01-11T07:13:18.286364+0000","last_became_active":"2024-01-11T07:13:17.292231+0000","last_became_peered":"2024-01-11T07:13:17.292231+0000","last_unstale":"2024-01-11T07:13:18.286364+0000","last_undegraded":"2024-01-11T07:13:18.286364+0000","last_fullsized":"2024-01-11T07:13:18.286364+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:17:47.747568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.300616+0000","last_change":"2024-01-11T07:13:18.300616+0000","last_active":"2024-01-11T07:13:18.300616+0000","last_peered":"2024-01-11T07:13:18.300616+0000","last_clean":"2024-01-11T07:13:18.300616+0000","last_became_active":"2024-01-11T07:13:18.300388+0000","last_became_peered":"2024-01-11T07:13:18.300388+0000","last_unstale":"2024-01-11T07:13:18.300616+0000","last_undegraded":"2024-01-11T07:13:18.300616+0000","last_fullsized":"2024-01-11T07:13:18.300616+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:49.962092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.317190+0000","last_change":"2024-01-11T07:13:17.296239+0000","last_active":"2024-01-11T07:13:18.317190+0000","last_peered":"2024-01-11T07:13:18.317190+0000","last_clean":"2024-01-11T07:13:18.317190+0000","last_became_active":"2024-01-11T07:13:17.295632+0000","last_became_peered":"2024-01-11T07:13:17.295632+0000","last_unstale":"2024-01-11T07:13:18.317190+0000","last_undegraded":"2024-01-11T07:13:18.317190+0000","last_fullsized":"2024-01-11T07:13:18.317190+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:49:55.589246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.291734+0000","last_change":"2024-01-11T07:13:18.291734+0000","last_active":"2024-01-11T07:13:18.291734+0000","last_peered":"2024-01-11T07:13:18.291734+0000","last_clean":"2024-01-11T07:13:18.291734+0000","last_became_active":"2024-01-11T07:13:18.291447+0000","last_became_peered":"2024-01-11T07:13:18.291447+0000","last_unstale":"2024-01-11T07:13:18.291734+0000","last_undegraded":"2024-01-11T07:13:18.291734+0000","last_fullsized":"2024-01-11T07:13:18.291734+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:07.274571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.291641+0000","last_change":"2024-01-11T07:13:17.295147+0000","last_active":"2024-01-11T07:13:18.291641+0000","last_peered":"2024-01-11T07:13:18.291641+0000","last_clean":"2024-01-11T07:13:18.291641+0000","last_became_active":"2024-01-11T07:13:17.294847+0000","last_became_peered":"2024-01-11T07:13:17.294847+0000","last_unstale":"2024-01-11T07:13:18.291641+0000","last_undegraded":"2024-01-11T07:13:18.291641+0000","last_fullsized":"2024-01-11T07:13:18.291641+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:43.685649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.301780+0000","last_change":"2024-01-11T07:13:18.301780+0000","last_active":"2024-01-11T07:13:18.301780+0000","last_peered":"2024-01-11T07:13:18.301780+0000","last_clean":"2024-01-11T07:13:18.301780+0000","last_became_active":"2024-01-11T07:13:18.301558+0000","last_became_peered":"2024-01-11T07:13:18.301558+0000","last_unstale":"2024-01-11T07:13:18.301780+0000","last_undegraded":"2024-01-11T07:13:18.301780+0000","last_fullsized":"2024-01-11T07:13:18.301780+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:30:53.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.302973+0000","last_change":"2024-01-11T07:13:17.302902+0000","last_active":"2024-01-11T07:13:17.302973+0000","last_peered":"2024-01-11T07:13:17.302973+0000","last_clean":"2024-01-11T07:13:17.302973+0000","last_became_active":"2024-01-11T07:13:17.302669+0000","last_became_peered":"2024-01-11T07:13:17.302669+0000","last_unstale":"2024-01-11T07:13:17.302973+0000","last_undegraded":"2024-01-11T07:13:17.302973+0000","last_fullsized":"2024-01-11T07:13:17.302973+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:48:58.805168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.290298+0000","last_change":"2024-01-11T07:13:18.290298+0000","last_active":"2024-01-11T07:13:18.290298+0000","last_peered":"2024-01-11T07:13:18.290298+0000","last_clean":"2024-01-11T07:13:18.290298+0000","last_became_active":"2024-01-11T07:13:18.290002+0000","last_became_peered":"2024-01-11T07:13:18.290002+0000","last_unstale":"2024-01-11T07:13:18.290298+0000","last_undegraded":"2024-01-11T07:13:18.290298+0000","last_fullsized":"2024-01-11T07:13:18.290298+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:51.391309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.317821+0000","last_change":"2024-01-11T07:13:17.296046+0000","last_active":"2024-01-11T07:13:18.317821+0000","last_peered":"2024-01-11T07:13:18.317821+0000","last_clean":"2024-01-11T07:13:18.317821+0000","last_became_active":"2024-01-11T07:13:17.295737+0000","last_became_peered":"2024-01-11T07:13:17.295737+0000","last_unstale":"2024-01-11T07:13:18.317821+0000","last_undegraded":"2024-01-11T07:13:18.317821+0000","last_fullsized":"2024-01-11T07:13:18.317821+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:58.982357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.290137+0000","last_change":"2024-01-11T07:13:18.290137+0000","last_active":"2024-01-11T07:13:18.290137+0000","last_peered":"2024-01-11T07:13:18.290137+0000","last_clean":"2024-01-11T07:13:18.290137+0000","last_became_active":"2024-01-11T07:13:18.289875+0000","last_became_peered":"2024-01-11T07:13:18.289875+0000","last_unstale":"2024-01-11T07:13:18.290137+0000","last_undegraded":"2024-01-11T07:13:18.290137+0000","last_fullsized":"2024-01-11T07:13:18.290137+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:02.877332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.296433+0000","last_change":"2024-01-11T07:13:17.295777+0000","last_active":"2024-01-11T07:13:17.296433+0000","last_peered":"2024-01-11T07:13:17.296433+0000","last_clean":"2024-01-11T07:13:17.296433+0000","last_became_active":"2024-01-11T07:13:17.295453+0000","last_became_peered":"2024-01-11T07:13:17.295453+0000","last_unstale":"2024-01-11T07:13:17.296433+0000","last_undegraded":"2024-01-11T07:13:17.296433+0000","last_fullsized":"2024-01-11T07:13:17.296433+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:26:21.541884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292705+0000","last_change":"2024-01-11T07:13:18.292705+0000","last_active":"2024-01-11T07:13:18.292705+0000","last_peered":"2024-01-11T07:13:18.292705+0000","last_clean":"2024-01-11T07:13:18.292705+0000","last_became_active":"2024-01-11T07:13:18.292443+0000","last_became_peered":"2024-01-11T07:13:18.292443+0000","last_unstale":"2024-01-11T07:13:18.292705+0000","last_undegraded":"2024-01-11T07:13:18.292705+0000","last_fullsized":"2024-01-11T07:13:18.292705+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:57:57.007824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.299273+0000","last_change":"2024-01-11T07:13:17.299169+0000","last_active":"2024-01-11T07:13:17.299273+0000","last_peered":"2024-01-11T07:13:17.299273+0000","last_clean":"2024-01-11T07:13:17.299273+0000","last_became_active":"2024-01-11T07:13:17.298704+0000","last_became_peered":"2024-01-11T07:13:17.298704+0000","last_unstale":"2024-01-11T07:13:17.299273+0000","last_undegraded":"2024-01-11T07:13:17.299273+0000","last_fullsized":"2024-01-11T07:13:17.299273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:48:58.805168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.313500+0000","last_change":"2024-01-11T07:13:18.313500+0000","last_active":"2024-01-11T07:13:18.313500+0000","last_peered":"2024-01-11T07:13:18.313500+0000","last_clean":"2024-01-11T07:13:18.313500+0000","last_became_active":"2024-01-11T07:13:18.312722+0000","last_became_peered":"2024-01-11T07:13:18.312722+0000","last_unstale":"2024-01-11T07:13:18.313500+0000","last_undegraded":"2024-01-11T07:13:18.313500+0000","last_fullsized":"2024-01-11T07:13:18.313500+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:27:49.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.298010+0000","last_change":"2024-01-11T07:13:17.297670+0000","last_active":"2024-01-11T07:13:17.298010+0000","last_peered":"2024-01-11T07:13:17.298010+0000","last_clean":"2024-01-11T07:13:17.298010+0000","last_became_active":"2024-01-11T07:13:17.297143+0000","last_became_peered":"2024-01-11T07:13:17.297143+0000","last_unstale":"2024-01-11T07:13:17.298010+0000","last_undegraded":"2024-01-11T07:13:17.298010+0000","last_fullsized":"2024-01-11T07:13:17.298010+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:41:00.934039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292408+0000","last_change":"2024-01-11T07:13:18.292408+0000","last_active":"2024-01-11T07:13:18.292408+0000","last_peered":"2024-01-11T07:13:18.292408+0000","last_clean":"2024-01-11T07:13:18.292408+0000","last_became_active":"2024-01-11T07:13:18.292062+0000","last_became_peered":"2024-01-11T07:13:18.292062+0000","last_unstale":"2024-01-11T07:13:18.292408+0000","last_undegraded":"2024-01-11T07:13:18.292408+0000","last_fullsized":"2024-01-11T07:13:18.292408+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:11:47.850216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.315337+0000","last_change":"2024-01-11T07:13:17.303712+0000","last_active":"2024-01-11T07:13:18.315337+0000","last_peered":"2024-01-11T07:13:18.315337+0000","last_clean":"2024-01-11T07:13:18.315337+0000","last_became_active":"2024-01-11T07:13:17.303437+0000","last_became_peered":"2024-01-11T07:13:17.303437+0000","last_unstale":"2024-01-11T07:13:18.315337+0000","last_undegraded":"2024-01-11T07:13:18.315337+0000","last_fullsized":"2024-01-11T07:13:18.315337+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:06:11.543004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.298557+0000","last_change":"2024-01-11T07:13:18.298557+0000","last_active":"2024-01-11T07:13:18.298557+0000","last_peered":"2024-01-11T07:13:18.298557+0000","last_clean":"2024-01-11T07:13:18.298557+0000","last_became_active":"2024-01-11T07:13:18.298285+0000","last_became_peered":"2024-01-11T07:13:18.298285+0000","last_unstale":"2024-01-11T07:13:18.298557+0000","last_undegraded":"2024-01-11T07:13:18.298557+0000","last_fullsized":"2024-01-11T07:13:18.298557+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:41:22.502204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.294987+0000","last_change":"2024-01-11T07:13:17.294509+0000","last_active":"2024-01-11T07:13:17.294987+0000","last_peered":"2024-01-11T07:13:17.294987+0000","last_clean":"2024-01-11T07:13:17.294987+0000","last_became_active":"2024-01-11T07:13:17.293881+0000","last_became_peered":"2024-01-11T07:13:17.293881+0000","last_unstale":"2024-01-11T07:13:17.294987+0000","last_undegraded":"2024-01-11T07:13:17.294987+0000","last_fullsized":"2024-01-11T07:13:17.294987+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:08:08.531654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.304008+0000","last_change":"2024-01-11T07:13:18.304008+0000","last_active":"2024-01-11T07:13:18.304008+0000","last_peered":"2024-01-11T07:13:18.304008+0000","last_clean":"2024-01-11T07:13:18.304008+0000","last_became_active":"2024-01-11T07:13:18.303442+0000","last_became_peered":"2024-01-11T07:13:18.303442+0000","last_unstale":"2024-01-11T07:13:18.304008+0000","last_undegraded":"2024-01-11T07:13:18.304008+0000","last_fullsized":"2024-01-11T07:13:18.304008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:39:19.050764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.300412+0000","last_change":"2024-01-11T07:13:17.299139+0000","last_active":"2024-01-11T07:13:17.300412+0000","last_peered":"2024-01-11T07:13:17.300412+0000","last_clean":"2024-01-11T07:13:17.300412+0000","last_became_active":"2024-01-11T07:13:17.298176+0000","last_became_peered":"2024-01-11T07:13:17.298176+0000","last_unstale":"2024-01-11T07:13:17.300412+0000","last_undegraded":"2024-01-11T07:13:17.300412+0000","last_fullsized":"2024-01-11T07:13:17.300412+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:27:48.185341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.293069+0000","last_change":"2024-01-11T07:13:18.293069+0000","last_active":"2024-01-11T07:13:18.293069+0000","last_peered":"2024-01-11T07:13:18.293069+0000","last_clean":"2024-01-11T07:13:18.293069+0000","last_became_active":"2024-01-11T07:13:18.292641+0000","last_became_peered":"2024-01-11T07:13:18.292641+0000","last_unstale":"2024-01-11T07:13:18.293069+0000","last_undegraded":"2024-01-11T07:13:18.293069+0000","last_fullsized":"2024-01-11T07:13:18.293069+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:46:12.687312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.296680+0000","last_change":"2024-01-11T07:13:17.296602+0000","last_active":"2024-01-11T07:13:17.296680+0000","last_peered":"2024-01-11T07:13:17.296680+0000","last_clean":"2024-01-11T07:13:17.296680+0000","last_became_active":"2024-01-11T07:13:17.296329+0000","last_became_peered":"2024-01-11T07:13:17.296329+0000","last_unstale":"2024-01-11T07:13:17.296680+0000","last_undegraded":"2024-01-11T07:13:17.296680+0000","last_fullsized":"2024-01-11T07:13:17.296680+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:57:56.001098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.299494+0000","last_change":"2024-01-11T07:13:18.299494+0000","last_active":"2024-01-11T07:13:18.299494+0000","last_peered":"2024-01-11T07:13:18.299494+0000","last_clean":"2024-01-11T07:13:18.299494+0000","last_became_active":"2024-01-11T07:13:18.299086+0000","last_became_peered":"2024-01-11T07:13:18.299086+0000","last_unstale":"2024-01-11T07:13:18.299494+0000","last_undegraded":"2024-01-11T07:13:18.299494+0000","last_fullsized":"2024-01-11T07:13:18.299494+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:01:34.907281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.294313+0000","last_change":"2024-01-11T07:13:17.294195+0000","last_active":"2024-01-11T07:13:17.294313+0000","last_peered":"2024-01-11T07:13:17.294313+0000","last_clean":"2024-01-11T07:13:17.294313+0000","last_became_active":"2024-01-11T07:13:17.293589+0000","last_became_peered":"2024-01-11T07:13:17.293589+0000","last_unstale":"2024-01-11T07:13:17.294313+0000","last_undegraded":"2024-01-11T07:13:17.294313+0000","last_fullsized":"2024-01-11T07:13:17.294313+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:38:55.748896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297297+0000","last_change":"2024-01-11T07:13:18.297297+0000","last_active":"2024-01-11T07:13:18.297297+0000","last_peered":"2024-01-11T07:13:18.297297+0000","last_clean":"2024-01-11T07:13:18.297297+0000","last_became_active":"2024-01-11T07:13:18.297083+0000","last_became_peered":"2024-01-11T07:13:18.297083+0000","last_unstale":"2024-01-11T07:13:18.297297+0000","last_undegraded":"2024-01-11T07:13:18.297297+0000","last_fullsized":"2024-01-11T07:13:18.297297+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:00:46.176711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.290204+0000","last_change":"2024-01-11T07:13:17.290129+0000","last_active":"2024-01-11T07:13:17.290204+0000","last_peered":"2024-01-11T07:13:17.290204+0000","last_clean":"2024-01-11T07:13:17.290204+0000","last_became_active":"2024-01-11T07:13:17.289878+0000","last_became_peered":"2024-01-11T07:13:17.289878+0000","last_unstale":"2024-01-11T07:13:17.290204+0000","last_undegraded":"2024-01-11T07:13:17.290204+0000","last_fullsized":"2024-01-11T07:13:17.290204+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:59:11.300237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292085+0000","last_change":"2024-01-11T07:13:18.292085+0000","last_active":"2024-01-11T07:13:18.292085+0000","last_peered":"2024-01-11T07:13:18.292085+0000","last_clean":"2024-01-11T07:13:18.292085+0000","last_became_active":"2024-01-11T07:13:18.291824+0000","last_became_peered":"2024-01-11T07:13:18.291824+0000","last_unstale":"2024-01-11T07:13:18.292085+0000","last_undegraded":"2024-01-11T07:13:18.292085+0000","last_fullsized":"2024-01-11T07:13:18.292085+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:19:08.390105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.296449+0000","last_change":"2024-01-11T07:13:17.296023+0000","last_active":"2024-01-11T07:13:17.296449+0000","last_peered":"2024-01-11T07:13:17.296449+0000","last_clean":"2024-01-11T07:13:17.296449+0000","last_became_active":"2024-01-11T07:13:17.295201+0000","last_became_peered":"2024-01-11T07:13:17.295201+0000","last_unstale":"2024-01-11T07:13:17.296449+0000","last_undegraded":"2024-01-11T07:13:17.296449+0000","last_fullsized":"2024-01-11T07:13:17.296449+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:32:22.609129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.301984+0000","last_change":"2024-01-11T07:13:18.301984+0000","last_active":"2024-01-11T07:13:18.301984+0000","last_peered":"2024-01-11T07:13:18.301984+0000","last_clean":"2024-01-11T07:13:18.301984+0000","last_became_active":"2024-01-11T07:13:18.301711+0000","last_became_peered":"2024-01-11T07:13:18.301711+0000","last_unstale":"2024-01-11T07:13:18.301984+0000","last_undegraded":"2024-01-11T07:13:18.301984+0000","last_fullsized":"2024-01-11T07:13:18.301984+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:43:50.526144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.300390+0000","last_change":"2024-01-11T07:13:17.298989+0000","last_active":"2024-01-11T07:13:17.300390+0000","last_peered":"2024-01-11T07:13:17.300390+0000","last_clean":"2024-01-11T07:13:17.300390+0000","last_became_active":"2024-01-11T07:13:17.297773+0000","last_became_peered":"2024-01-11T07:13:17.297773+0000","last_unstale":"2024-01-11T07:13:17.300390+0000","last_undegraded":"2024-01-11T07:13:17.300390+0000","last_fullsized":"2024-01-11T07:13:17.300390+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:28:45.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.299562+0000","last_change":"2024-01-11T07:13:18.299562+0000","last_active":"2024-01-11T07:13:18.299562+0000","last_peered":"2024-01-11T07:13:18.299562+0000","last_clean":"2024-01-11T07:13:18.299562+0000","last_became_active":"2024-01-11T07:13:18.299317+0000","last_became_peered":"2024-01-11T07:13:18.299317+0000","last_unstale":"2024-01-11T07:13:18.299562+0000","last_undegraded":"2024-01-11T07:13:18.299562+0000","last_fullsized":"2024-01-11T07:13:18.299562+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:41:01.940765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.293969+0000","last_change":"2024-01-11T07:13:17.293893+0000","last_active":"2024-01-11T07:13:17.293969+0000","last_peered":"2024-01-11T07:13:17.293969+0000","last_clean":"2024-01-11T07:13:17.293969+0000","last_became_active":"2024-01-11T07:13:17.293625+0000","last_became_peered":"2024-01-11T07:13:17.293625+0000","last_unstale":"2024-01-11T07:13:17.293969+0000","last_undegraded":"2024-01-11T07:13:17.293969+0000","last_fullsized":"2024-01-11T07:13:17.293969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:58.982357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":263,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.287495+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:13:17.287495+0000","last_peered":"2024-01-11T07:13:17.287495+0000","last_clean":"2024-01-11T07:13:17.287495+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:13:17.287495+0000","last_undegraded":"2024-01-11T07:13:17.287495+0000","last_fullsized":"2024-01-11T07:13:17.287495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1296,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194015,"statfs":{"total":200000000,"available":198672326,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.33200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":0.754},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72399999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57399999999999995,"15min":0.52400000000000002},"min":{"1min":0.44500000000000001,"5min":0.34000000000000002,"15min":0.311},"max":{"1min":0.753,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.69999999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58999999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.52400000000000002},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.53600000000000003},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76600000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.499,"15min":0.48699999999999999},"min":{"1min":0.29899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.747,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51500000000000001,"15min":0.48499999999999999},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.79900000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81499999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46700000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199263212,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.81100000000000005,"15min":0.81399999999999995},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.46300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76000000000000001,"5min":0.81200000000000006,"15min":1.157},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.5720000000000001,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.55400000000000005},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.85099999999999998},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72699999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.36599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70399999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.621,"5min":2.621,"15min":2.621},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45300000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.69899999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.745},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.61699999999999999},"min":{"1min":0.33500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.53100000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.749,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.65000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199270045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.67600000000000005,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.216,"15min":0.216},"max":{"1min":8.5150000000000006,"5min":20.41,"15min":20.41},"last":0.753},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.33000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.7029999999999998,"5min":3.7029999999999998,"15min":3.7029999999999998},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.48799999999999999},"min":{"1min":0.29399999999999998,"5min":0.27000000000000002,"15min":0.26700000000000002},"max":{"1min":7.8099999999999996,"5min":15.611000000000001,"15min":15.611000000000001},"last":3.597},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":6.1390000000000002,"5min":20.199999999999999,"15min":20.199999999999999},"last":3.0710000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70999999999999996,"15min":0.55300000000000005},"min":{"1min":0.33100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.6449999999999996,"5min":20.369,"15min":20.369},"last":3.8460000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.496,"15min":0.46000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":8.5790000000000006,"5min":8.5790000000000006,"15min":8.5790000000000006},"last":4.7569999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.76800000000000002,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.317,"15min":0.317},"max":{"1min":7.6749999999999998,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80100000000000005,"15min":0.67900000000000005},"min":{"1min":0.46300000000000002,"5min":0.44,"15min":0.41899999999999998},"max":{"1min":5.7830000000000004,"5min":12.449999999999999,"15min":12.449999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64600000000000002,"15min":0.61899999999999999},"min":{"1min":0.45400000000000001,"5min":0.379,"15min":0.376},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.78800000000000003,"15min":0.69399999999999995},"min":{"1min":0.46300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":7.7480000000000002,"5min":9.891,"15min":9.891},"last":1.177}]},{"osd":6,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.995,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.389},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.79300000000000004,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.7770000000000001,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":2.9049999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.92100000000000004,"15min":0.78300000000000003},"min":{"1min":0.499,"5min":0.442,"15min":0.442},"max":{"1min":5.8449999999999998,"5min":20.324000000000002,"15min":20.324000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.69499999999999995,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46800000000000003},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.268}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1289,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194023,"statfs":{"total":200000000,"available":198679677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39900000000000002,"15min":0.46300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.224},"max":{"1min":0.56699999999999995,"5min":0.628,"15min":2.7429999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51500000000000001,"5min":0.627,"15min":0.72199999999999998},"last":0.26400000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.68600000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.73199999999999998,"15min":7.7960000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44600000000000001,"15min":0.52400000000000002},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63400000000000001,"5min":0.72099999999999997,"15min":2.5880000000000001},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.69699999999999995},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":8.0730000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.56599999999999995},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.755,"5min":0.77700000000000002,"15min":2.855},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.443},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.87},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40999999999999998},"max":{"1min":0.86599999999999999,"5min":0.94999999999999996,"15min":8.0399999999999991},"last":0.52500000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65700000000000003,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.872,"5min":0.97099999999999997,"15min":2.819},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.92300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":1.046,"15min":7.9539999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.48099999999999998,"15min":0.41799999999999998},"max":{"1min":0.91700000000000004,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.501}]},{"osd":7,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68100000000000005,"15min":0.66500000000000004},"min":{"1min":0.45200000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68700000000000006,"15min":0.91900000000000004},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.44900000000000001},"max":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":8.0030000000000001},"last":0.495}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194593,"statfs":{"total":200000000,"available":199263282,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.25800000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.505,"5min":1.472,"15min":1.472},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.312}]},{"osd":1,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.432,"15min":0.44},"min":{"1min":0.313,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54000000000000004,"5min":1.0309999999999999,"15min":1.2430000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39100000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.246},"max":{"1min":0.56999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.29299999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.65500000000000003,"5min":0.74399999999999999,"15min":0.76800000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":1.081,"15min":1.139},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"max":{"1min":0.92400000000000004,"5min":1.0169999999999999,"15min":1.0660000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.624},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97799999999999998,"5min":1.089,"15min":1.089},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68000000000000005,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89200000000000002,"5min":1.145,"15min":1.3100000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.499,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.052},"last":0.495}]},{"osd":7,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0409999999999999,"5min":1.042,"15min":1.042},"last":0.622}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194591,"statfs":{"total":200000000,"available":199261676,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39400000000000002,"15min":0.75},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.58699999999999997,"15min":11.481},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.57299999999999995,"5min":0.65400000000000003,"15min":1.6200000000000001},"last":0.38}]},{"osd":1,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39500000000000002,"15min":0.92300000000000004},"min":{"1min":0.27200000000000002,"5min":0.252,"15min":0.23899999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":17.516999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.29199999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":0.67400000000000004},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42599999999999999,"15min":0.42699999999999999},"min":{"1min":0.30499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.76500000000000001},"last":0.433},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46899999999999997,"15min":0.97999999999999998},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.30599999999999999},"max":{"1min":0.54300000000000004,"5min":0.79300000000000004,"15min":17.553000000000001},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.93899999999999995,"15min":0.94599999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68400000000000005,"15min":1.212},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84299999999999997,"5min":0.95499999999999996,"15min":17.474},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":1.149},"min":{"1min":0.42599999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":0.98799999999999999,"15min":17.393000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65700000000000003,"15min":0.98199999999999998},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.002,"5min":1.0149999999999999,"15min":11.394},"last":0.46400000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66300000000000003,"15min":0.997},"min":{"1min":0.46700000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":11.436999999999999},"last":0.503},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94799999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68899999999999995,"15min":1.1870000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":1.1839999999999999,"15min":17.434000000000001},"last":0.53000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199270045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.64800000000000002,"15min":1.0129999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.248},"max":{"1min":0.753,"5min":3.5579999999999998,"15min":8.484},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65200000000000002,"15min":1.0660000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.76700000000000002,"5min":3.5329999999999999,"15min":9.2599999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.59599999999999997,"15min":0.83999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.30199999999999999},"max":{"1min":0.92700000000000005,"5min":1.2250000000000001,"15min":8.5340000000000007},"last":0.505},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61299999999999999,"15min":0.96999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74099999999999999,"5min":4.2560000000000002,"15min":9.2279999999999998},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65900000000000003,"15min":0.94399999999999995},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":5.9770000000000003,"15min":9.5640000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.84999999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90600000000000003,"5min":0.93000000000000005,"15min":6.8639999999999999},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57899999999999996,"15min":1.004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81499999999999995,"5min":1.208,"15min":8.9779999999999998},"last":0.625},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65300000000000002,"15min":1.075},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73799999999999999,"5min":4.8680000000000003,"15min":8.7309999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46200000000000002,"15min":0.89800000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":11.388999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.82799999999999996},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.76800000000000002,"5min":1.0880000000000001,"15min":8.6509999999999998},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53100000000000003,"15min":0.877},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91800000000000004,"15min":9.1449999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.378,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.81200000000000006,"5min":0.86099999999999999,"15min":1.119},"last":0.69199999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.85999999999999999},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.83299999999999996,"5min":0.95999999999999996,"15min":8.6050000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.95499999999999996},"min":{"1min":0.40200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.80000000000000004,"5min":0.874,"15min":9.4640000000000004},"last":0.73599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":720,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199262706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81200000000000006,"5min":0.82699999999999996,"15min":0.90000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.504}]},{"osd":1,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.39900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.87},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.84099999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.81599999999999995,"15min":0.82799999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79500000000000004,"5min":0.874,"15min":0.93999999999999995},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76700000000000002,"5min":0.84199999999999997,"15min":1.0389999999999999},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.41799999999999998,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71799999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.61699999999999999},"min":{"1min":0.40699999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":0.85199999999999998,"15min":0.88200000000000001},"last":0.45700000000000002}]}]}],"pool_statfs":[]}} 2024-01-11T07:13:22.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.657+0000 7f35528de700 1 Processor -- start 2024-01-11T07:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.658+0000 7f35528de700 1 -- start start 2024-01-11T07:13:22.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.658+0000 7f35528de700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f354c15a060 0x7f354c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.658+0000 7f35528de700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c0ce9b0 con 0x7f354c069320 2024-01-11T07:13:22.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.658+0000 7f35528de700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c0cf530 con 0x7f354c159c90 2024-01-11T07:13:22.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.658+0000 7f35528de700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c0b1f30 con 0x7f354c15a060 2024-01-11T07:13:22.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f354c15a060 0x7f354c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f354c15a060 0x7f354c0b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46018/0 (socket says 172.21.15.7:46018) 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354b7fe700 1 -- 172.21.15.7:0/862704289 learned_addr learned my addr 172.21.15.7:0/862704289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f3550e7b700 1 --1- 172.21.15.7:0/862704289 >> v1:172.21.15.7:6789/0 conn(0x7f354c069320 0x7f354c0b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59694/0 (socket says 172.21.15.7:59694) 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354b7fe700 1 -- 172.21.15.7:0/862704289 >> v1:172.21.15.113:6789/0 conn(0x7f354c159c90 legacy=0x7f354c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354b7fe700 1 -- 172.21.15.7:0/862704289 >> v1:172.21.15.7:6789/0 conn(0x7f354c069320 legacy=0x7f354c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354b7fe700 1 -- 172.21.15.7:0/862704289 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0cf9c0 con 0x7f354c15a060 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354affd700 1 -- 172.21.15.7:0/862704289 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767703690 0 0) 0x7f354c0ce9b0 con 0x7f354c069320 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354b7fe700 1 --2- 172.21.15.7:0/862704289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0b0430 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f35400099b0 tx=0x7f3540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62058318a7af1b5f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354affd700 1 -- 172.21.15.7:0/862704289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354000b860 con 0x7f354c15a060 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354affd700 1 -- 172.21.15.7:0/862704289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354000b9c0 con 0x7f354c15a060 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.659+0000 7f354affd700 1 -- 172.21.15.7:0/862704289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354000bd60 con 0x7f354c15a060 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.660+0000 7f35528de700 1 -- 172.21.15.7:0/862704289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 msgr2=0x7f354c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.660+0000 7f35528de700 1 --2- 172.21.15.7:0/862704289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0b0430 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f35400099b0 tx=0x7f3540009250 comp rx=0 tx=0).stop 2024-01-11T07:13:22.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.660+0000 7f35528de700 1 -- 172.21.15.7:0/862704289 shutdown_connections 2024-01-11T07:13:22.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.660+0000 7f35528de700 1 --2- 172.21.15.7:0/862704289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0b0430 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.660+0000 7f35528de700 1 -- 172.21.15.7:0/862704289 >> 172.21.15.7:0/862704289 conn(0x7f354c071100 msgr2=0x7f354c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:22.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.660+0000 7f35528de700 1 -- 172.21.15.7:0/862704289 shutdown_connections 2024-01-11T07:13:22.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.660+0000 7f35528de700 1 -- 172.21.15.7:0/862704289 wait complete. 2024-01-11T07:13:22.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.661+0000 7f35528de700 1 Processor -- start 2024-01-11T07:13:22.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.662+0000 7f35528de700 1 -- start start 2024-01-11T07:13:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.662+0000 7f35528de700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f354c069320 0x7f354c155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.662+0000 7f35528de700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c159c90 0x7f354c0af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.662+0000 7f35528de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.662+0000 7f35528de700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0aee90 con 0x7f354c15a060 2024-01-11T07:13:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.662+0000 7f35528de700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c156430 con 0x7f354c159c90 2024-01-11T07:13:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.662+0000 7f35528de700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0ae280 con 0x7f354c069320 2024-01-11T07:13:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.662+0000 7f3550e7b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3550e7b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46022/0 (socket says 172.21.15.7:46022) 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3550e7b700 1 -- 172.21.15.7:0/134121349 learned_addr learned my addr 172.21.15.7:0/134121349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f354b7fe700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c159c90 0x7f354c0af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f354bfff700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f354c069320 0x7f354c155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3550e7b700 1 -- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f354c069320 msgr2=0x7f354c155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3550e7b700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f354c069320 0x7f354c155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3550e7b700 1 -- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c159c90 msgr2=0x7f354c0af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3550e7b700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c159c90 0x7f354c0af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3550e7b700 1 -- 172.21.15.7:0/134121349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35400094f0 con 0x7f354c15a060 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3550e7b700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0af860 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f354400ccc0 tx=0x7f3544001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3533fff700 1 -- 172.21.15.7:0/134121349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544005df0 con 0x7f354c15a060 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3533fff700 1 -- 172.21.15.7:0/134121349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354400a600 con 0x7f354c15a060 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c0ae560 con 0x7f354c15a060 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f3533fff700 1 -- 172.21.15.7:0/134121349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400a9c0 con 0x7f354c15a060 2024-01-11T07:13:22.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.663+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c0aeb20 con 0x7f354c15a060 2024-01-11T07:13:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.664+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c04c3c0 con 0x7f354c15a060 2024-01-11T07:13:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.665+0000 7f3533fff700 1 -- 172.21.15.7:0/134121349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f35440070f0 con 0x7f354c15a060 2024-01-11T07:13:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.665+0000 7f3533fff700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.665+0000 7f3533fff700 1 -- 172.21.15.7:0/134121349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3544084660 con 0x7f354c15a060 2024-01-11T07:13:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.666+0000 7f354bfff700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:22.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.666+0000 7f354bfff700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3538066f30 0x7f35380693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f353c003a80 tx=0x7f353c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:22.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.670+0000 7f3533fff700 1 -- 172.21.15.7:0/134121349 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3544051960 con 0x7f354c15a060 2024-01-11T07:13:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.869+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f354c0aefd0 con 0x7f3538066f30 2024-01-11T07:13:22.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.878+0000 7f3533fff700 1 -- 172.21.15.7:0/134121349 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285435 (secure 0 0 0) 0x7f354c0aefd0 con 0x7f3538066f30 2024-01-11T07:13:22.881 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3538066f30 msgr2=0x7f35380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3538066f30 0x7f35380693e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f353c003a80 tx=0x7f353c007ec0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 msgr2=0x7f354c0af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0af860 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f354400ccc0 tx=0x7f3544001a70 comp rx=0 tx=0).stop 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 shutdown_connections 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3538066f30 0x7f35380693e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c15a060 0x7f354c0af860 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f354c069320 0x7f354c155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 --2- 172.21.15.7:0/134121349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f354c159c90 0x7f354c0af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 >> 172.21.15.7:0/134121349 conn(0x7f354c071100 msgr2=0x7f354c067610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:22.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 shutdown_connections 2024-01-11T07:13:22.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:22.881+0000 7f35528de700 1 -- 172.21.15.7:0/134121349 wait complete. 2024-01-11T07:13:22.885 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:13:22.894 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-01-11T07:13:21.929772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555608,"statfs":{"total":1600000000,"available":1592942969,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.650259"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292913+0000","last_change":"2024-01-11T07:13:18.292913+0000","last_active":"2024-01-11T07:13:18.292913+0000","last_peered":"2024-01-11T07:13:18.292913+0000","last_clean":"2024-01-11T07:13:18.292913+0000","last_became_active":"2024-01-11T07:13:18.292576+0000","last_became_peered":"2024-01-11T07:13:18.292576+0000","last_unstale":"2024-01-11T07:13:18.292913+0000","last_undegraded":"2024-01-11T07:13:18.292913+0000","last_fullsized":"2024-01-11T07:13:18.292913+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:19:56.151854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.308392+0000","last_change":"2024-01-11T07:13:18.308392+0000","last_active":"2024-01-11T07:13:18.308392+0000","last_peered":"2024-01-11T07:13:18.308392+0000","last_clean":"2024-01-11T07:13:18.308392+0000","last_became_active":"2024-01-11T07:13:18.308140+0000","last_became_peered":"2024-01-11T07:13:18.308140+0000","last_unstale":"2024-01-11T07:13:18.308392+0000","last_undegraded":"2024-01-11T07:13:18.308392+0000","last_fullsized":"2024-01-11T07:13:18.308392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:30:53.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.290108+0000","last_change":"2024-01-11T07:13:18.290108+0000","last_active":"2024-01-11T07:13:18.290108+0000","last_peered":"2024-01-11T07:13:18.290108+0000","last_clean":"2024-01-11T07:13:18.290108+0000","last_became_active":"2024-01-11T07:13:18.289283+0000","last_became_peered":"2024-01-11T07:13:18.289283+0000","last_unstale":"2024-01-11T07:13:18.290108+0000","last_undegraded":"2024-01-11T07:13:18.290108+0000","last_fullsized":"2024-01-11T07:13:18.290108+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:56:56.035612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.288576+0000","last_change":"2024-01-11T07:13:18.288576+0000","last_active":"2024-01-11T07:13:18.288576+0000","last_peered":"2024-01-11T07:13:18.288576+0000","last_clean":"2024-01-11T07:13:18.288576+0000","last_became_active":"2024-01-11T07:13:18.288271+0000","last_became_peered":"2024-01-11T07:13:18.288271+0000","last_unstale":"2024-01-11T07:13:18.288576+0000","last_undegraded":"2024-01-11T07:13:18.288576+0000","last_fullsized":"2024-01-11T07:13:18.288576+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:21:46.490695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306523+0000","last_change":"2024-01-11T07:13:18.306523+0000","last_active":"2024-01-11T07:13:18.306523+0000","last_peered":"2024-01-11T07:13:18.306523+0000","last_clean":"2024-01-11T07:13:18.306523+0000","last_became_active":"2024-01-11T07:13:18.306247+0000","last_became_peered":"2024-01-11T07:13:18.306247+0000","last_unstale":"2024-01-11T07:13:18.306523+0000","last_undegraded":"2024-01-11T07:13:18.306523+0000","last_fullsized":"2024-01-11T07:13:18.306523+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:25:52.822147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306933+0000","last_change":"2024-01-11T07:13:18.306933+0000","last_active":"2024-01-11T07:13:18.306933+0000","last_peered":"2024-01-11T07:13:18.306933+0000","last_clean":"2024-01-11T07:13:18.306933+0000","last_became_active":"2024-01-11T07:13:18.306139+0000","last_became_peered":"2024-01-11T07:13:18.306139+0000","last_unstale":"2024-01-11T07:13:18.306933+0000","last_undegraded":"2024-01-11T07:13:18.306933+0000","last_fullsized":"2024-01-11T07:13:18.306933+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:08:09.538380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.299814+0000","last_change":"2024-01-11T07:13:18.299814+0000","last_active":"2024-01-11T07:13:18.299814+0000","last_peered":"2024-01-11T07:13:18.299814+0000","last_clean":"2024-01-11T07:13:18.299814+0000","last_became_active":"2024-01-11T07:13:18.299431+0000","last_became_peered":"2024-01-11T07:13:18.299431+0000","last_unstale":"2024-01-11T07:13:18.299814+0000","last_undegraded":"2024-01-11T07:13:18.299814+0000","last_fullsized":"2024-01-11T07:13:18.299814+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:31:49.646204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.303527+0000","last_change":"2024-01-11T07:13:18.303527+0000","last_active":"2024-01-11T07:13:18.303527+0000","last_peered":"2024-01-11T07:13:18.303527+0000","last_clean":"2024-01-11T07:13:18.303527+0000","last_became_active":"2024-01-11T07:13:18.303354+0000","last_became_peered":"2024-01-11T07:13:18.303354+0000","last_unstale":"2024-01-11T07:13:18.303527+0000","last_undegraded":"2024-01-11T07:13:18.303527+0000","last_fullsized":"2024-01-11T07:13:18.303527+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:14:00.649612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295797+0000","last_change":"2024-01-11T07:13:18.295797+0000","last_active":"2024-01-11T07:13:18.295797+0000","last_peered":"2024-01-11T07:13:18.295797+0000","last_clean":"2024-01-11T07:13:18.295797+0000","last_became_active":"2024-01-11T07:13:18.295552+0000","last_became_peered":"2024-01-11T07:13:18.295552+0000","last_unstale":"2024-01-11T07:13:18.295797+0000","last_undegraded":"2024-01-11T07:13:18.295797+0000","last_fullsized":"2024-01-11T07:13:18.295797+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:32:31.066220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.294028+0000","last_change":"2024-01-11T07:13:18.294028+0000","last_active":"2024-01-11T07:13:18.294028+0000","last_peered":"2024-01-11T07:13:18.294028+0000","last_clean":"2024-01-11T07:13:18.294028+0000","last_became_active":"2024-01-11T07:13:18.293703+0000","last_became_peered":"2024-01-11T07:13:18.293703+0000","last_unstale":"2024-01-11T07:13:18.294028+0000","last_undegraded":"2024-01-11T07:13:18.294028+0000","last_fullsized":"2024-01-11T07:13:18.294028+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:07:25.049652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297727+0000","last_change":"2024-01-11T07:13:18.297727+0000","last_active":"2024-01-11T07:13:18.297727+0000","last_peered":"2024-01-11T07:13:18.297727+0000","last_clean":"2024-01-11T07:13:18.297727+0000","last_became_active":"2024-01-11T07:13:18.297531+0000","last_became_peered":"2024-01-11T07:13:18.297531+0000","last_unstale":"2024-01-11T07:13:18.297727+0000","last_undegraded":"2024-01-11T07:13:18.297727+0000","last_fullsized":"2024-01-11T07:13:18.297727+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:19:08.390105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292997+0000","last_change":"2024-01-11T07:13:18.292997+0000","last_active":"2024-01-11T07:13:18.292997+0000","last_peered":"2024-01-11T07:13:18.292997+0000","last_clean":"2024-01-11T07:13:18.292997+0000","last_became_active":"2024-01-11T07:13:18.292711+0000","last_became_peered":"2024-01-11T07:13:18.292711+0000","last_unstale":"2024-01-11T07:13:18.292997+0000","last_undegraded":"2024-01-11T07:13:18.292997+0000","last_fullsized":"2024-01-11T07:13:18.292997+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:22:57.927263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292720+0000","last_change":"2024-01-11T07:13:18.292720+0000","last_active":"2024-01-11T07:13:18.292720+0000","last_peered":"2024-01-11T07:13:18.292720+0000","last_clean":"2024-01-11T07:13:18.292720+0000","last_became_active":"2024-01-11T07:13:18.292436+0000","last_became_peered":"2024-01-11T07:13:18.292436+0000","last_unstale":"2024-01-11T07:13:18.292720+0000","last_undegraded":"2024-01-11T07:13:18.292720+0000","last_fullsized":"2024-01-11T07:13:18.292720+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:49:42.554340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.293542+0000","last_change":"2024-01-11T07:13:18.293542+0000","last_active":"2024-01-11T07:13:18.293542+0000","last_peered":"2024-01-11T07:13:18.293542+0000","last_clean":"2024-01-11T07:13:18.293542+0000","last_became_active":"2024-01-11T07:13:18.293278+0000","last_became_peered":"2024-01-11T07:13:18.293278+0000","last_unstale":"2024-01-11T07:13:18.293542+0000","last_undegraded":"2024-01-11T07:13:18.293542+0000","last_fullsized":"2024-01-11T07:13:18.293542+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:12:18.804791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.311803+0000","last_change":"2024-01-11T07:13:18.311803+0000","last_active":"2024-01-11T07:13:18.311803+0000","last_peered":"2024-01-11T07:13:18.311803+0000","last_clean":"2024-01-11T07:13:18.311803+0000","last_became_active":"2024-01-11T07:13:18.311304+0000","last_became_peered":"2024-01-11T07:13:18.311304+0000","last_unstale":"2024-01-11T07:13:18.311803+0000","last_undegraded":"2024-01-11T07:13:18.311803+0000","last_fullsized":"2024-01-11T07:13:18.311803+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:44.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.288672+0000","last_change":"2024-01-11T07:13:18.288672+0000","last_active":"2024-01-11T07:13:18.288672+0000","last_peered":"2024-01-11T07:13:18.288672+0000","last_clean":"2024-01-11T07:13:18.288672+0000","last_became_active":"2024-01-11T07:13:18.288430+0000","last_became_peered":"2024-01-11T07:13:18.288430+0000","last_unstale":"2024-01-11T07:13:18.288672+0000","last_undegraded":"2024-01-11T07:13:18.288672+0000","last_fullsized":"2024-01-11T07:13:18.288672+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:33:29.540323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295490+0000","last_change":"2024-01-11T07:13:18.295490+0000","last_active":"2024-01-11T07:13:18.295490+0000","last_peered":"2024-01-11T07:13:18.295490+0000","last_clean":"2024-01-11T07:13:18.295490+0000","last_became_active":"2024-01-11T07:13:18.295203+0000","last_became_peered":"2024-01-11T07:13:18.295203+0000","last_unstale":"2024-01-11T07:13:18.295490+0000","last_undegraded":"2024-01-11T07:13:18.295490+0000","last_fullsized":"2024-01-11T07:13:18.295490+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:02:14.868338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.298547+0000","last_change":"2024-01-11T07:13:18.298547+0000","last_active":"2024-01-11T07:13:18.298547+0000","last_peered":"2024-01-11T07:13:18.298547+0000","last_clean":"2024-01-11T07:13:18.298547+0000","last_became_active":"2024-01-11T07:13:18.298311+0000","last_became_peered":"2024-01-11T07:13:18.298311+0000","last_unstale":"2024-01-11T07:13:18.298547+0000","last_undegraded":"2024-01-11T07:13:18.298547+0000","last_fullsized":"2024-01-11T07:13:18.298547+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:39:10.000850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292616+0000","last_change":"2024-01-11T07:13:18.292616+0000","last_active":"2024-01-11T07:13:18.292616+0000","last_peered":"2024-01-11T07:13:18.292616+0000","last_clean":"2024-01-11T07:13:18.292616+0000","last_became_active":"2024-01-11T07:13:18.292376+0000","last_became_peered":"2024-01-11T07:13:18.292376+0000","last_unstale":"2024-01-11T07:13:18.292616+0000","last_undegraded":"2024-01-11T07:13:18.292616+0000","last_fullsized":"2024-01-11T07:13:18.292616+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:00:46.176711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.296824+0000","last_change":"2024-01-11T07:13:18.296824+0000","last_active":"2024-01-11T07:13:18.296824+0000","last_peered":"2024-01-11T07:13:18.296824+0000","last_clean":"2024-01-11T07:13:18.296824+0000","last_became_active":"2024-01-11T07:13:18.296263+0000","last_became_peered":"2024-01-11T07:13:18.296263+0000","last_unstale":"2024-01-11T07:13:18.296824+0000","last_undegraded":"2024-01-11T07:13:18.296824+0000","last_fullsized":"2024-01-11T07:13:18.296824+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:07.274571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.305797+0000","last_change":"2024-01-11T07:13:18.305797+0000","last_active":"2024-01-11T07:13:18.305797+0000","last_peered":"2024-01-11T07:13:18.305797+0000","last_clean":"2024-01-11T07:13:18.305797+0000","last_became_active":"2024-01-11T07:13:18.305571+0000","last_became_peered":"2024-01-11T07:13:18.305571+0000","last_unstale":"2024-01-11T07:13:18.305797+0000","last_undegraded":"2024-01-11T07:13:18.305797+0000","last_fullsized":"2024-01-11T07:13:18.305797+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:12:18.804791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.302118+0000","last_change":"2024-01-11T07:13:18.302118+0000","last_active":"2024-01-11T07:13:18.302118+0000","last_peered":"2024-01-11T07:13:18.302118+0000","last_clean":"2024-01-11T07:13:18.302118+0000","last_became_active":"2024-01-11T07:13:18.301874+0000","last_became_peered":"2024-01-11T07:13:18.301874+0000","last_unstale":"2024-01-11T07:13:18.302118+0000","last_undegraded":"2024-01-11T07:13:18.302118+0000","last_fullsized":"2024-01-11T07:13:18.302118+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:19:30.069554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.296671+0000","last_change":"2024-01-11T07:13:18.296671+0000","last_active":"2024-01-11T07:13:18.296671+0000","last_peered":"2024-01-11T07:13:18.296671+0000","last_clean":"2024-01-11T07:13:18.296671+0000","last_became_active":"2024-01-11T07:13:18.295591+0000","last_became_peered":"2024-01-11T07:13:18.295591+0000","last_unstale":"2024-01-11T07:13:18.296671+0000","last_undegraded":"2024-01-11T07:13:18.296671+0000","last_fullsized":"2024-01-11T07:13:18.296671+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:59:19.451948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297225+0000","last_change":"2024-01-11T07:13:18.297225+0000","last_active":"2024-01-11T07:13:18.297225+0000","last_peered":"2024-01-11T07:13:18.297225+0000","last_clean":"2024-01-11T07:13:18.297225+0000","last_became_active":"2024-01-11T07:13:18.296970+0000","last_became_peered":"2024-01-11T07:13:18.296970+0000","last_unstale":"2024-01-11T07:13:18.297225+0000","last_undegraded":"2024-01-11T07:13:18.297225+0000","last_fullsized":"2024-01-11T07:13:18.297225+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:17:48.754294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297190+0000","last_change":"2024-01-11T07:13:18.297190+0000","last_active":"2024-01-11T07:13:18.297190+0000","last_peered":"2024-01-11T07:13:18.297190+0000","last_clean":"2024-01-11T07:13:18.297190+0000","last_became_active":"2024-01-11T07:13:18.296586+0000","last_became_peered":"2024-01-11T07:13:18.296586+0000","last_unstale":"2024-01-11T07:13:18.297190+0000","last_undegraded":"2024-01-11T07:13:18.297190+0000","last_fullsized":"2024-01-11T07:13:18.297190+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:56:56.035612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.313148+0000","last_change":"2024-01-11T07:13:18.313148+0000","last_active":"2024-01-11T07:13:18.313148+0000","last_peered":"2024-01-11T07:13:18.313148+0000","last_clean":"2024-01-11T07:13:18.313148+0000","last_became_active":"2024-01-11T07:13:18.312739+0000","last_became_peered":"2024-01-11T07:13:18.312739+0000","last_unstale":"2024-01-11T07:13:18.313148+0000","last_undegraded":"2024-01-11T07:13:18.313148+0000","last_fullsized":"2024-01-11T07:13:18.313148+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:13:37.840332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306714+0000","last_change":"2024-01-11T07:13:18.306714+0000","last_active":"2024-01-11T07:13:18.306714+0000","last_peered":"2024-01-11T07:13:18.306714+0000","last_clean":"2024-01-11T07:13:18.306714+0000","last_became_active":"2024-01-11T07:13:18.306119+0000","last_became_peered":"2024-01-11T07:13:18.306119+0000","last_unstale":"2024-01-11T07:13:18.306714+0000","last_undegraded":"2024-01-11T07:13:18.306714+0000","last_fullsized":"2024-01-11T07:13:18.306714+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:24:03.561483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.302050+0000","last_change":"2024-01-11T07:13:18.302050+0000","last_active":"2024-01-11T07:13:18.302050+0000","last_peered":"2024-01-11T07:13:18.302050+0000","last_clean":"2024-01-11T07:13:18.302050+0000","last_became_active":"2024-01-11T07:13:18.301782+0000","last_became_peered":"2024-01-11T07:13:18.301782+0000","last_unstale":"2024-01-11T07:13:18.302050+0000","last_undegraded":"2024-01-11T07:13:18.302050+0000","last_fullsized":"2024-01-11T07:13:18.302050+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:41.118932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295273+0000","last_change":"2024-01-11T07:13:18.295273+0000","last_active":"2024-01-11T07:13:18.295273+0000","last_peered":"2024-01-11T07:13:18.295273+0000","last_clean":"2024-01-11T07:13:18.295273+0000","last_became_active":"2024-01-11T07:13:18.294956+0000","last_became_peered":"2024-01-11T07:13:18.294956+0000","last_unstale":"2024-01-11T07:13:18.295273+0000","last_undegraded":"2024-01-11T07:13:18.295273+0000","last_fullsized":"2024-01-11T07:13:18.295273+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:51.391309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.310815+0000","last_change":"2024-01-11T07:13:18.310815+0000","last_active":"2024-01-11T07:13:18.310815+0000","last_peered":"2024-01-11T07:13:18.310815+0000","last_clean":"2024-01-11T07:13:18.310815+0000","last_became_active":"2024-01-11T07:13:18.310627+0000","last_became_peered":"2024-01-11T07:13:18.310627+0000","last_unstale":"2024-01-11T07:13:18.310815+0000","last_undegraded":"2024-01-11T07:13:18.310815+0000","last_fullsized":"2024-01-11T07:13:18.310815+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:24:03.561483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.312965+0000","last_change":"2024-01-11T07:13:18.312965+0000","last_active":"2024-01-11T07:13:18.312965+0000","last_peered":"2024-01-11T07:13:18.312965+0000","last_clean":"2024-01-11T07:13:18.312965+0000","last_became_active":"2024-01-11T07:13:18.312501+0000","last_became_peered":"2024-01-11T07:13:18.312501+0000","last_unstale":"2024-01-11T07:13:18.312965+0000","last_undegraded":"2024-01-11T07:13:18.312965+0000","last_fullsized":"2024-01-11T07:13:18.312965+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:05.275296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.307485+0000","last_change":"2024-01-11T07:13:18.307485+0000","last_active":"2024-01-11T07:13:18.307485+0000","last_peered":"2024-01-11T07:13:18.307485+0000","last_clean":"2024-01-11T07:13:18.307485+0000","last_became_active":"2024-01-11T07:13:18.307229+0000","last_became_peered":"2024-01-11T07:13:18.307229+0000","last_unstale":"2024-01-11T07:13:18.307485+0000","last_undegraded":"2024-01-11T07:13:18.307485+0000","last_fullsized":"2024-01-11T07:13:18.307485+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:09.898464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.300149+0000","last_change":"2024-01-11T07:13:18.300149+0000","last_active":"2024-01-11T07:13:18.300149+0000","last_peered":"2024-01-11T07:13:18.300149+0000","last_clean":"2024-01-11T07:13:18.300149+0000","last_became_active":"2024-01-11T07:13:18.299918+0000","last_became_peered":"2024-01-11T07:13:18.299918+0000","last_unstale":"2024-01-11T07:13:18.300149+0000","last_undegraded":"2024-01-11T07:13:18.300149+0000","last_fullsized":"2024-01-11T07:13:18.300149+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:01:53.801845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"134'4","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.308549+0000","last_change":"2024-01-11T07:13:17.296028+0000","last_active":"2024-01-11T07:13:18.308549+0000","last_peered":"2024-01-11T07:13:18.308549+0000","last_clean":"2024-01-11T07:13:18.308549+0000","last_became_active":"2024-01-11T07:13:17.295770+0000","last_became_peered":"2024-01-11T07:13:17.295770+0000","last_unstale":"2024-01-11T07:13:18.308549+0000","last_undegraded":"2024-01-11T07:13:18.308549+0000","last_fullsized":"2024-01-11T07:13:18.308549+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:36.657856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292265+0000","last_change":"2024-01-11T07:13:18.292265+0000","last_active":"2024-01-11T07:13:18.292265+0000","last_peered":"2024-01-11T07:13:18.292265+0000","last_clean":"2024-01-11T07:13:18.292265+0000","last_became_active":"2024-01-11T07:13:18.291800+0000","last_became_peered":"2024-01-11T07:13:18.291800+0000","last_unstale":"2024-01-11T07:13:18.292265+0000","last_undegraded":"2024-01-11T07:13:18.292265+0000","last_fullsized":"2024-01-11T07:13:18.292265+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:00:04.530196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.288845+0000","last_change":"2024-01-11T07:13:17.294772+0000","last_active":"2024-01-11T07:13:18.288845+0000","last_peered":"2024-01-11T07:13:18.288845+0000","last_clean":"2024-01-11T07:13:18.288845+0000","last_became_active":"2024-01-11T07:13:17.292791+0000","last_became_peered":"2024-01-11T07:13:17.292791+0000","last_unstale":"2024-01-11T07:13:18.288845+0000","last_undegraded":"2024-01-11T07:13:18.288845+0000","last_fullsized":"2024-01-11T07:13:18.288845+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:00:03.523470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306769+0000","last_change":"2024-01-11T07:13:18.306769+0000","last_active":"2024-01-11T07:13:18.306769+0000","last_peered":"2024-01-11T07:13:18.306769+0000","last_clean":"2024-01-11T07:13:18.306769+0000","last_became_active":"2024-01-11T07:13:18.305779+0000","last_became_peered":"2024-01-11T07:13:18.305779+0000","last_unstale":"2024-01-11T07:13:18.306769+0000","last_undegraded":"2024-01-11T07:13:18.306769+0000","last_fullsized":"2024-01-11T07:13:18.306769+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:42:43.114680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.317061+0000","last_change":"2024-01-11T07:13:17.296375+0000","last_active":"2024-01-11T07:13:18.317061+0000","last_peered":"2024-01-11T07:13:18.317061+0000","last_clean":"2024-01-11T07:13:18.317061+0000","last_became_active":"2024-01-11T07:13:17.295991+0000","last_became_peered":"2024-01-11T07:13:17.295991+0000","last_unstale":"2024-01-11T07:13:18.317061+0000","last_undegraded":"2024-01-11T07:13:18.317061+0000","last_fullsized":"2024-01-11T07:13:18.317061+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:38:42.695868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.293555+0000","last_change":"2024-01-11T07:13:18.293555+0000","last_active":"2024-01-11T07:13:18.293555+0000","last_peered":"2024-01-11T07:13:18.293555+0000","last_clean":"2024-01-11T07:13:18.293555+0000","last_became_active":"2024-01-11T07:13:18.293372+0000","last_became_peered":"2024-01-11T07:13:18.293372+0000","last_unstale":"2024-01-11T07:13:18.293555+0000","last_undegraded":"2024-01-11T07:13:18.293555+0000","last_fullsized":"2024-01-11T07:13:18.293555+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:59:19.451948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.314142+0000","last_change":"2024-01-11T07:13:17.300690+0000","last_active":"2024-01-11T07:13:18.314142+0000","last_peered":"2024-01-11T07:13:18.314142+0000","last_clean":"2024-01-11T07:13:18.314142+0000","last_became_active":"2024-01-11T07:13:17.300338+0000","last_became_peered":"2024-01-11T07:13:17.300338+0000","last_unstale":"2024-01-11T07:13:18.314142+0000","last_undegraded":"2024-01-11T07:13:18.314142+0000","last_fullsized":"2024-01-11T07:13:18.314142+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:09:25.512858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.305801+0000","last_change":"2024-01-11T07:13:18.305801+0000","last_active":"2024-01-11T07:13:18.305801+0000","last_peered":"2024-01-11T07:13:18.305801+0000","last_clean":"2024-01-11T07:13:18.305801+0000","last_became_active":"2024-01-11T07:13:18.305331+0000","last_became_peered":"2024-01-11T07:13:18.305331+0000","last_unstale":"2024-01-11T07:13:18.305801+0000","last_undegraded":"2024-01-11T07:13:18.305801+0000","last_fullsized":"2024-01-11T07:13:18.305801+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:04:46.248050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297009+0000","last_change":"2024-01-11T07:13:17.297043+0000","last_active":"2024-01-11T07:13:18.297009+0000","last_peered":"2024-01-11T07:13:18.297009+0000","last_clean":"2024-01-11T07:13:18.297009+0000","last_became_active":"2024-01-11T07:13:17.296564+0000","last_became_peered":"2024-01-11T07:13:17.296564+0000","last_unstale":"2024-01-11T07:13:18.297009+0000","last_undegraded":"2024-01-11T07:13:18.297009+0000","last_fullsized":"2024-01-11T07:13:18.297009+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:51:33.647768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.306446+0000","last_change":"2024-01-11T07:13:18.306446+0000","last_active":"2024-01-11T07:13:18.306446+0000","last_peered":"2024-01-11T07:13:18.306446+0000","last_clean":"2024-01-11T07:13:18.306446+0000","last_became_active":"2024-01-11T07:13:18.306239+0000","last_became_peered":"2024-01-11T07:13:18.306239+0000","last_unstale":"2024-01-11T07:13:18.306446+0000","last_undegraded":"2024-01-11T07:13:18.306446+0000","last_fullsized":"2024-01-11T07:13:18.306446+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:49:42.554340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295911+0000","last_change":"2024-01-11T07:13:17.297214+0000","last_active":"2024-01-11T07:13:18.295911+0000","last_peered":"2024-01-11T07:13:18.295911+0000","last_clean":"2024-01-11T07:13:18.295911+0000","last_became_active":"2024-01-11T07:13:17.296959+0000","last_became_peered":"2024-01-11T07:13:17.296959+0000","last_unstale":"2024-01-11T07:13:18.295911+0000","last_undegraded":"2024-01-11T07:13:18.295911+0000","last_fullsized":"2024-01-11T07:13:18.295911+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:38:32.291346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.304162+0000","last_change":"2024-01-11T07:13:18.304162+0000","last_active":"2024-01-11T07:13:18.304162+0000","last_peered":"2024-01-11T07:13:18.304162+0000","last_clean":"2024-01-11T07:13:18.304162+0000","last_became_active":"2024-01-11T07:13:18.303760+0000","last_became_peered":"2024-01-11T07:13:18.303760+0000","last_unstale":"2024-01-11T07:13:18.304162+0000","last_undegraded":"2024-01-11T07:13:18.304162+0000","last_fullsized":"2024-01-11T07:13:18.304162+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:21:34.087245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.295416+0000","last_change":"2024-01-11T07:13:17.295294+0000","last_active":"2024-01-11T07:13:17.295416+0000","last_peered":"2024-01-11T07:13:17.295416+0000","last_clean":"2024-01-11T07:13:17.295416+0000","last_became_active":"2024-01-11T07:13:17.295056+0000","last_became_peered":"2024-01-11T07:13:17.295056+0000","last_unstale":"2024-01-11T07:13:17.295416+0000","last_undegraded":"2024-01-11T07:13:17.295416+0000","last_fullsized":"2024-01-11T07:13:17.295416+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:14.944854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295509+0000","last_change":"2024-01-11T07:13:18.295509+0000","last_active":"2024-01-11T07:13:18.295509+0000","last_peered":"2024-01-11T07:13:18.295509+0000","last_clean":"2024-01-11T07:13:18.295509+0000","last_became_active":"2024-01-11T07:13:18.295278+0000","last_became_peered":"2024-01-11T07:13:18.295278+0000","last_unstale":"2024-01-11T07:13:18.295509+0000","last_undegraded":"2024-01-11T07:13:18.295509+0000","last_fullsized":"2024-01-11T07:13:18.295509+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:01:53.801845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.300446+0000","last_change":"2024-01-11T07:13:17.299335+0000","last_active":"2024-01-11T07:13:17.300446+0000","last_peered":"2024-01-11T07:13:17.300446+0000","last_clean":"2024-01-11T07:13:17.300446+0000","last_became_active":"2024-01-11T07:13:17.299089+0000","last_became_peered":"2024-01-11T07:13:17.299089+0000","last_unstale":"2024-01-11T07:13:17.300446+0000","last_undegraded":"2024-01-11T07:13:17.300446+0000","last_fullsized":"2024-01-11T07:13:17.300446+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:19:29.062828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.300962+0000","last_change":"2024-01-11T07:13:18.300962+0000","last_active":"2024-01-11T07:13:18.300962+0000","last_peered":"2024-01-11T07:13:18.300962+0000","last_clean":"2024-01-11T07:13:18.300962+0000","last_became_active":"2024-01-11T07:13:18.300795+0000","last_became_peered":"2024-01-11T07:13:18.300795+0000","last_unstale":"2024-01-11T07:13:18.300962+0000","last_undegraded":"2024-01-11T07:13:18.300962+0000","last_fullsized":"2024-01-11T07:13:18.300962+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:44.692375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.300614+0000","last_change":"2024-01-11T07:13:17.300549+0000","last_active":"2024-01-11T07:13:17.300614+0000","last_peered":"2024-01-11T07:13:17.300614+0000","last_clean":"2024-01-11T07:13:17.300614+0000","last_became_active":"2024-01-11T07:13:17.300007+0000","last_became_peered":"2024-01-11T07:13:17.300007+0000","last_unstale":"2024-01-11T07:13:17.300614+0000","last_undegraded":"2024-01-11T07:13:17.300614+0000","last_fullsized":"2024-01-11T07:13:17.300614+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:16:04.250535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.296696+0000","last_change":"2024-01-11T07:13:18.296696+0000","last_active":"2024-01-11T07:13:18.296696+0000","last_peered":"2024-01-11T07:13:18.296696+0000","last_clean":"2024-01-11T07:13:18.296696+0000","last_became_active":"2024-01-11T07:13:18.295962+0000","last_became_peered":"2024-01-11T07:13:18.295962+0000","last_unstale":"2024-01-11T07:13:18.296696+0000","last_undegraded":"2024-01-11T07:13:18.296696+0000","last_fullsized":"2024-01-11T07:13:18.296696+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:24:33.389471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.296625+0000","last_change":"2024-01-11T07:13:17.296548+0000","last_active":"2024-01-11T07:13:17.296625+0000","last_peered":"2024-01-11T07:13:17.296625+0000","last_clean":"2024-01-11T07:13:17.296625+0000","last_became_active":"2024-01-11T07:13:17.296286+0000","last_became_peered":"2024-01-11T07:13:17.296286+0000","last_unstale":"2024-01-11T07:13:17.296625+0000","last_undegraded":"2024-01-11T07:13:17.296625+0000","last_fullsized":"2024-01-11T07:13:17.296625+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:19:15.784688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.303867+0000","last_change":"2024-01-11T07:13:18.303867+0000","last_active":"2024-01-11T07:13:18.303867+0000","last_peered":"2024-01-11T07:13:18.303867+0000","last_clean":"2024-01-11T07:13:18.303867+0000","last_became_active":"2024-01-11T07:13:18.303605+0000","last_became_peered":"2024-01-11T07:13:18.303605+0000","last_unstale":"2024-01-11T07:13:18.303867+0000","last_undegraded":"2024-01-11T07:13:18.303867+0000","last_fullsized":"2024-01-11T07:13:18.303867+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:47:50.081608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"134'3","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.314161+0000","last_change":"2024-01-11T07:13:17.295227+0000","last_active":"2024-01-11T07:13:18.314161+0000","last_peered":"2024-01-11T07:13:18.314161+0000","last_clean":"2024-01-11T07:13:18.314161+0000","last_became_active":"2024-01-11T07:13:17.294983+0000","last_became_peered":"2024-01-11T07:13:17.294983+0000","last_unstale":"2024-01-11T07:13:18.314161+0000","last_undegraded":"2024-01-11T07:13:18.314161+0000","last_fullsized":"2024-01-11T07:13:18.314161+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:13:59.642885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.301514+0000","last_change":"2024-01-11T07:13:18.301514+0000","last_active":"2024-01-11T07:13:18.301514+0000","last_peered":"2024-01-11T07:13:18.301514+0000","last_clean":"2024-01-11T07:13:18.301514+0000","last_became_active":"2024-01-11T07:13:18.301322+0000","last_became_peered":"2024-01-11T07:13:18.301322+0000","last_unstale":"2024-01-11T07:13:18.301514+0000","last_undegraded":"2024-01-11T07:13:18.301514+0000","last_fullsized":"2024-01-11T07:13:18.301514+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:42:43.114680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"134'6","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.314521+0000","last_change":"2024-01-11T07:13:17.299287+0000","last_active":"2024-01-11T07:13:18.314521+0000","last_peered":"2024-01-11T07:13:18.314521+0000","last_clean":"2024-01-11T07:13:18.314521+0000","last_became_active":"2024-01-11T07:13:17.298550+0000","last_became_peered":"2024-01-11T07:13:17.298550+0000","last_unstale":"2024-01-11T07:13:18.314521+0000","last_undegraded":"2024-01-11T07:13:18.314521+0000","last_fullsized":"2024-01-11T07:13:18.314521+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:14:40.112206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.302865+0000","last_change":"2024-01-11T07:13:18.302865+0000","last_active":"2024-01-11T07:13:18.302865+0000","last_peered":"2024-01-11T07:13:18.302865+0000","last_clean":"2024-01-11T07:13:18.302865+0000","last_became_active":"2024-01-11T07:13:18.302633+0000","last_became_peered":"2024-01-11T07:13:18.302633+0000","last_unstale":"2024-01-11T07:13:18.302865+0000","last_undegraded":"2024-01-11T07:13:18.302865+0000","last_fullsized":"2024-01-11T07:13:18.302865+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:46:51.487989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.309309+0000","last_change":"2024-01-11T07:13:17.290909+0000","last_active":"2024-01-11T07:13:18.309309+0000","last_peered":"2024-01-11T07:13:18.309309+0000","last_clean":"2024-01-11T07:13:18.309309+0000","last_became_active":"2024-01-11T07:13:17.290606+0000","last_became_peered":"2024-01-11T07:13:17.290606+0000","last_unstale":"2024-01-11T07:13:18.309309+0000","last_undegraded":"2024-01-11T07:13:18.309309+0000","last_fullsized":"2024-01-11T07:13:18.309309+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:36.657856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.313675+0000","last_change":"2024-01-11T07:13:18.313675+0000","last_active":"2024-01-11T07:13:18.313675+0000","last_peered":"2024-01-11T07:13:18.313675+0000","last_clean":"2024-01-11T07:13:18.313675+0000","last_became_active":"2024-01-11T07:13:18.313203+0000","last_became_peered":"2024-01-11T07:13:18.313203+0000","last_unstale":"2024-01-11T07:13:18.313675+0000","last_undegraded":"2024-01-11T07:13:18.313675+0000","last_fullsized":"2024-01-11T07:13:18.313675+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:14:00.649612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.292232+0000","last_change":"2024-01-11T07:13:17.292098+0000","last_active":"2024-01-11T07:13:17.292232+0000","last_peered":"2024-01-11T07:13:17.292232+0000","last_clean":"2024-01-11T07:13:17.292232+0000","last_became_active":"2024-01-11T07:13:17.291852+0000","last_became_peered":"2024-01-11T07:13:17.291852+0000","last_unstale":"2024-01-11T07:13:17.292232+0000","last_undegraded":"2024-01-11T07:13:17.292232+0000","last_fullsized":"2024-01-11T07:13:17.292232+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:08.891738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.302745+0000","last_change":"2024-01-11T07:13:18.302745+0000","last_active":"2024-01-11T07:13:18.302745+0000","last_peered":"2024-01-11T07:13:18.302745+0000","last_clean":"2024-01-11T07:13:18.302745+0000","last_became_active":"2024-01-11T07:13:18.302468+0000","last_became_peered":"2024-01-11T07:13:18.302468+0000","last_unstale":"2024-01-11T07:13:18.302745+0000","last_undegraded":"2024-01-11T07:13:18.302745+0000","last_fullsized":"2024-01-11T07:13:18.302745+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:28:46.090843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"134'8","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.317649+0000","last_change":"2024-01-11T07:13:17.301276+0000","last_active":"2024-01-11T07:13:18.317649+0000","last_peered":"2024-01-11T07:13:18.317649+0000","last_clean":"2024-01-11T07:13:18.317649+0000","last_became_active":"2024-01-11T07:13:17.301027+0000","last_became_peered":"2024-01-11T07:13:17.301027+0000","last_unstale":"2024-01-11T07:13:18.317649+0000","last_undegraded":"2024-01-11T07:13:18.317649+0000","last_fullsized":"2024-01-11T07:13:18.317649+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:02:07.545480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.295022+0000","last_change":"2024-01-11T07:13:18.295022+0000","last_active":"2024-01-11T07:13:18.295022+0000","last_peered":"2024-01-11T07:13:18.295022+0000","last_clean":"2024-01-11T07:13:18.295022+0000","last_became_active":"2024-01-11T07:13:18.294598+0000","last_became_peered":"2024-01-11T07:13:18.294598+0000","last_unstale":"2024-01-11T07:13:18.295022+0000","last_undegraded":"2024-01-11T07:13:18.295022+0000","last_fullsized":"2024-01-11T07:13:18.295022+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:59:25.436030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.286364+0000","last_change":"2024-01-11T07:13:17.292532+0000","last_active":"2024-01-11T07:13:18.286364+0000","last_peered":"2024-01-11T07:13:18.286364+0000","last_clean":"2024-01-11T07:13:18.286364+0000","last_became_active":"2024-01-11T07:13:17.292231+0000","last_became_peered":"2024-01-11T07:13:17.292231+0000","last_unstale":"2024-01-11T07:13:18.286364+0000","last_undegraded":"2024-01-11T07:13:18.286364+0000","last_fullsized":"2024-01-11T07:13:18.286364+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:17:47.747568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.300616+0000","last_change":"2024-01-11T07:13:18.300616+0000","last_active":"2024-01-11T07:13:18.300616+0000","last_peered":"2024-01-11T07:13:18.300616+0000","last_clean":"2024-01-11T07:13:18.300616+0000","last_became_active":"2024-01-11T07:13:18.300388+0000","last_became_peered":"2024-01-11T07:13:18.300388+0000","last_unstale":"2024-01-11T07:13:18.300616+0000","last_undegraded":"2024-01-11T07:13:18.300616+0000","last_fullsized":"2024-01-11T07:13:18.300616+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:47:49.962092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.317190+0000","last_change":"2024-01-11T07:13:17.296239+0000","last_active":"2024-01-11T07:13:18.317190+0000","last_peered":"2024-01-11T07:13:18.317190+0000","last_clean":"2024-01-11T07:13:18.317190+0000","last_became_active":"2024-01-11T07:13:17.295632+0000","last_became_peered":"2024-01-11T07:13:17.295632+0000","last_unstale":"2024-01-11T07:13:18.317190+0000","last_undegraded":"2024-01-11T07:13:18.317190+0000","last_fullsized":"2024-01-11T07:13:18.317190+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:49:55.589246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.291734+0000","last_change":"2024-01-11T07:13:18.291734+0000","last_active":"2024-01-11T07:13:18.291734+0000","last_peered":"2024-01-11T07:13:18.291734+0000","last_clean":"2024-01-11T07:13:18.291734+0000","last_became_active":"2024-01-11T07:13:18.291447+0000","last_became_peered":"2024-01-11T07:13:18.291447+0000","last_unstale":"2024-01-11T07:13:18.291734+0000","last_undegraded":"2024-01-11T07:13:18.291734+0000","last_fullsized":"2024-01-11T07:13:18.291734+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:07.274571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"134'1","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.291641+0000","last_change":"2024-01-11T07:13:17.295147+0000","last_active":"2024-01-11T07:13:18.291641+0000","last_peered":"2024-01-11T07:13:18.291641+0000","last_clean":"2024-01-11T07:13:18.291641+0000","last_became_active":"2024-01-11T07:13:17.294847+0000","last_became_peered":"2024-01-11T07:13:17.294847+0000","last_unstale":"2024-01-11T07:13:18.291641+0000","last_undegraded":"2024-01-11T07:13:18.291641+0000","last_fullsized":"2024-01-11T07:13:18.291641+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:34:43.685649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.301780+0000","last_change":"2024-01-11T07:13:18.301780+0000","last_active":"2024-01-11T07:13:18.301780+0000","last_peered":"2024-01-11T07:13:18.301780+0000","last_clean":"2024-01-11T07:13:18.301780+0000","last_became_active":"2024-01-11T07:13:18.301558+0000","last_became_peered":"2024-01-11T07:13:18.301558+0000","last_unstale":"2024-01-11T07:13:18.301780+0000","last_undegraded":"2024-01-11T07:13:18.301780+0000","last_fullsized":"2024-01-11T07:13:18.301780+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:30:53.557140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.302973+0000","last_change":"2024-01-11T07:13:17.302902+0000","last_active":"2024-01-11T07:13:17.302973+0000","last_peered":"2024-01-11T07:13:17.302973+0000","last_clean":"2024-01-11T07:13:17.302973+0000","last_became_active":"2024-01-11T07:13:17.302669+0000","last_became_peered":"2024-01-11T07:13:17.302669+0000","last_unstale":"2024-01-11T07:13:17.302973+0000","last_undegraded":"2024-01-11T07:13:17.302973+0000","last_fullsized":"2024-01-11T07:13:17.302973+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:48:58.805168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.290298+0000","last_change":"2024-01-11T07:13:18.290298+0000","last_active":"2024-01-11T07:13:18.290298+0000","last_peered":"2024-01-11T07:13:18.290298+0000","last_clean":"2024-01-11T07:13:18.290298+0000","last_became_active":"2024-01-11T07:13:18.290002+0000","last_became_peered":"2024-01-11T07:13:18.290002+0000","last_unstale":"2024-01-11T07:13:18.290298+0000","last_undegraded":"2024-01-11T07:13:18.290298+0000","last_fullsized":"2024-01-11T07:13:18.290298+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:50:51.391309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.317821+0000","last_change":"2024-01-11T07:13:17.296046+0000","last_active":"2024-01-11T07:13:18.317821+0000","last_peered":"2024-01-11T07:13:18.317821+0000","last_clean":"2024-01-11T07:13:18.317821+0000","last_became_active":"2024-01-11T07:13:17.295737+0000","last_became_peered":"2024-01-11T07:13:17.295737+0000","last_unstale":"2024-01-11T07:13:18.317821+0000","last_undegraded":"2024-01-11T07:13:18.317821+0000","last_fullsized":"2024-01-11T07:13:18.317821+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:58.982357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.290137+0000","last_change":"2024-01-11T07:13:18.290137+0000","last_active":"2024-01-11T07:13:18.290137+0000","last_peered":"2024-01-11T07:13:18.290137+0000","last_clean":"2024-01-11T07:13:18.290137+0000","last_became_active":"2024-01-11T07:13:18.289875+0000","last_became_peered":"2024-01-11T07:13:18.289875+0000","last_unstale":"2024-01-11T07:13:18.290137+0000","last_undegraded":"2024-01-11T07:13:18.290137+0000","last_fullsized":"2024-01-11T07:13:18.290137+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:02.877332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.296433+0000","last_change":"2024-01-11T07:13:17.295777+0000","last_active":"2024-01-11T07:13:17.296433+0000","last_peered":"2024-01-11T07:13:17.296433+0000","last_clean":"2024-01-11T07:13:17.296433+0000","last_became_active":"2024-01-11T07:13:17.295453+0000","last_became_peered":"2024-01-11T07:13:17.295453+0000","last_unstale":"2024-01-11T07:13:17.296433+0000","last_undegraded":"2024-01-11T07:13:17.296433+0000","last_fullsized":"2024-01-11T07:13:17.296433+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:26:21.541884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292705+0000","last_change":"2024-01-11T07:13:18.292705+0000","last_active":"2024-01-11T07:13:18.292705+0000","last_peered":"2024-01-11T07:13:18.292705+0000","last_clean":"2024-01-11T07:13:18.292705+0000","last_became_active":"2024-01-11T07:13:18.292443+0000","last_became_peered":"2024-01-11T07:13:18.292443+0000","last_unstale":"2024-01-11T07:13:18.292705+0000","last_undegraded":"2024-01-11T07:13:18.292705+0000","last_fullsized":"2024-01-11T07:13:18.292705+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:57:57.007824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.299273+0000","last_change":"2024-01-11T07:13:17.299169+0000","last_active":"2024-01-11T07:13:17.299273+0000","last_peered":"2024-01-11T07:13:17.299273+0000","last_clean":"2024-01-11T07:13:17.299273+0000","last_became_active":"2024-01-11T07:13:17.298704+0000","last_became_peered":"2024-01-11T07:13:17.298704+0000","last_unstale":"2024-01-11T07:13:17.299273+0000","last_undegraded":"2024-01-11T07:13:17.299273+0000","last_fullsized":"2024-01-11T07:13:17.299273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:48:58.805168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.313500+0000","last_change":"2024-01-11T07:13:18.313500+0000","last_active":"2024-01-11T07:13:18.313500+0000","last_peered":"2024-01-11T07:13:18.313500+0000","last_clean":"2024-01-11T07:13:18.313500+0000","last_became_active":"2024-01-11T07:13:18.312722+0000","last_became_peered":"2024-01-11T07:13:18.312722+0000","last_unstale":"2024-01-11T07:13:18.313500+0000","last_undegraded":"2024-01-11T07:13:18.313500+0000","last_fullsized":"2024-01-11T07:13:18.313500+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:27:49.192067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.298010+0000","last_change":"2024-01-11T07:13:17.297670+0000","last_active":"2024-01-11T07:13:17.298010+0000","last_peered":"2024-01-11T07:13:17.298010+0000","last_clean":"2024-01-11T07:13:17.298010+0000","last_became_active":"2024-01-11T07:13:17.297143+0000","last_became_peered":"2024-01-11T07:13:17.297143+0000","last_unstale":"2024-01-11T07:13:17.298010+0000","last_undegraded":"2024-01-11T07:13:17.298010+0000","last_fullsized":"2024-01-11T07:13:17.298010+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:41:00.934039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292408+0000","last_change":"2024-01-11T07:13:18.292408+0000","last_active":"2024-01-11T07:13:18.292408+0000","last_peered":"2024-01-11T07:13:18.292408+0000","last_clean":"2024-01-11T07:13:18.292408+0000","last_became_active":"2024-01-11T07:13:18.292062+0000","last_became_peered":"2024-01-11T07:13:18.292062+0000","last_unstale":"2024-01-11T07:13:18.292408+0000","last_undegraded":"2024-01-11T07:13:18.292408+0000","last_fullsized":"2024-01-11T07:13:18.292408+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:11:47.850216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"134'2","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.315337+0000","last_change":"2024-01-11T07:13:17.303712+0000","last_active":"2024-01-11T07:13:18.315337+0000","last_peered":"2024-01-11T07:13:18.315337+0000","last_clean":"2024-01-11T07:13:18.315337+0000","last_became_active":"2024-01-11T07:13:17.303437+0000","last_became_peered":"2024-01-11T07:13:17.303437+0000","last_unstale":"2024-01-11T07:13:18.315337+0000","last_undegraded":"2024-01-11T07:13:18.315337+0000","last_fullsized":"2024-01-11T07:13:18.315337+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:06:11.543004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.298557+0000","last_change":"2024-01-11T07:13:18.298557+0000","last_active":"2024-01-11T07:13:18.298557+0000","last_peered":"2024-01-11T07:13:18.298557+0000","last_clean":"2024-01-11T07:13:18.298557+0000","last_became_active":"2024-01-11T07:13:18.298285+0000","last_became_peered":"2024-01-11T07:13:18.298285+0000","last_unstale":"2024-01-11T07:13:18.298557+0000","last_undegraded":"2024-01-11T07:13:18.298557+0000","last_fullsized":"2024-01-11T07:13:18.298557+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:41:22.502204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.294987+0000","last_change":"2024-01-11T07:13:17.294509+0000","last_active":"2024-01-11T07:13:17.294987+0000","last_peered":"2024-01-11T07:13:17.294987+0000","last_clean":"2024-01-11T07:13:17.294987+0000","last_became_active":"2024-01-11T07:13:17.293881+0000","last_became_peered":"2024-01-11T07:13:17.293881+0000","last_unstale":"2024-01-11T07:13:17.294987+0000","last_undegraded":"2024-01-11T07:13:17.294987+0000","last_fullsized":"2024-01-11T07:13:17.294987+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:08:08.531654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.304008+0000","last_change":"2024-01-11T07:13:18.304008+0000","last_active":"2024-01-11T07:13:18.304008+0000","last_peered":"2024-01-11T07:13:18.304008+0000","last_clean":"2024-01-11T07:13:18.304008+0000","last_became_active":"2024-01-11T07:13:18.303442+0000","last_became_peered":"2024-01-11T07:13:18.303442+0000","last_unstale":"2024-01-11T07:13:18.304008+0000","last_undegraded":"2024-01-11T07:13:18.304008+0000","last_fullsized":"2024-01-11T07:13:18.304008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:39:19.050764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.300412+0000","last_change":"2024-01-11T07:13:17.299139+0000","last_active":"2024-01-11T07:13:17.300412+0000","last_peered":"2024-01-11T07:13:17.300412+0000","last_clean":"2024-01-11T07:13:17.300412+0000","last_became_active":"2024-01-11T07:13:17.298176+0000","last_became_peered":"2024-01-11T07:13:17.298176+0000","last_unstale":"2024-01-11T07:13:17.300412+0000","last_undegraded":"2024-01-11T07:13:17.300412+0000","last_fullsized":"2024-01-11T07:13:17.300412+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:27:48.185341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.293069+0000","last_change":"2024-01-11T07:13:18.293069+0000","last_active":"2024-01-11T07:13:18.293069+0000","last_peered":"2024-01-11T07:13:18.293069+0000","last_clean":"2024-01-11T07:13:18.293069+0000","last_became_active":"2024-01-11T07:13:18.292641+0000","last_became_peered":"2024-01-11T07:13:18.292641+0000","last_unstale":"2024-01-11T07:13:18.293069+0000","last_undegraded":"2024-01-11T07:13:18.293069+0000","last_fullsized":"2024-01-11T07:13:18.293069+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:46:12.687312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.296680+0000","last_change":"2024-01-11T07:13:17.296602+0000","last_active":"2024-01-11T07:13:17.296680+0000","last_peered":"2024-01-11T07:13:17.296680+0000","last_clean":"2024-01-11T07:13:17.296680+0000","last_became_active":"2024-01-11T07:13:17.296329+0000","last_became_peered":"2024-01-11T07:13:17.296329+0000","last_unstale":"2024-01-11T07:13:17.296680+0000","last_undegraded":"2024-01-11T07:13:17.296680+0000","last_fullsized":"2024-01-11T07:13:17.296680+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:57:56.001098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.299494+0000","last_change":"2024-01-11T07:13:18.299494+0000","last_active":"2024-01-11T07:13:18.299494+0000","last_peered":"2024-01-11T07:13:18.299494+0000","last_clean":"2024-01-11T07:13:18.299494+0000","last_became_active":"2024-01-11T07:13:18.299086+0000","last_became_peered":"2024-01-11T07:13:18.299086+0000","last_unstale":"2024-01-11T07:13:18.299494+0000","last_undegraded":"2024-01-11T07:13:18.299494+0000","last_fullsized":"2024-01-11T07:13:18.299494+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:01:34.907281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.294313+0000","last_change":"2024-01-11T07:13:17.294195+0000","last_active":"2024-01-11T07:13:17.294313+0000","last_peered":"2024-01-11T07:13:17.294313+0000","last_clean":"2024-01-11T07:13:17.294313+0000","last_became_active":"2024-01-11T07:13:17.293589+0000","last_became_peered":"2024-01-11T07:13:17.293589+0000","last_unstale":"2024-01-11T07:13:17.294313+0000","last_undegraded":"2024-01-11T07:13:17.294313+0000","last_fullsized":"2024-01-11T07:13:17.294313+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:38:55.748896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.297297+0000","last_change":"2024-01-11T07:13:18.297297+0000","last_active":"2024-01-11T07:13:18.297297+0000","last_peered":"2024-01-11T07:13:18.297297+0000","last_clean":"2024-01-11T07:13:18.297297+0000","last_became_active":"2024-01-11T07:13:18.297083+0000","last_became_peered":"2024-01-11T07:13:18.297083+0000","last_unstale":"2024-01-11T07:13:18.297297+0000","last_undegraded":"2024-01-11T07:13:18.297297+0000","last_fullsized":"2024-01-11T07:13:18.297297+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:00:46.176711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.290204+0000","last_change":"2024-01-11T07:13:17.290129+0000","last_active":"2024-01-11T07:13:17.290204+0000","last_peered":"2024-01-11T07:13:17.290204+0000","last_clean":"2024-01-11T07:13:17.290204+0000","last_became_active":"2024-01-11T07:13:17.289878+0000","last_became_peered":"2024-01-11T07:13:17.289878+0000","last_unstale":"2024-01-11T07:13:17.290204+0000","last_undegraded":"2024-01-11T07:13:17.290204+0000","last_fullsized":"2024-01-11T07:13:17.290204+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:59:11.300237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.292085+0000","last_change":"2024-01-11T07:13:18.292085+0000","last_active":"2024-01-11T07:13:18.292085+0000","last_peered":"2024-01-11T07:13:18.292085+0000","last_clean":"2024-01-11T07:13:18.292085+0000","last_became_active":"2024-01-11T07:13:18.291824+0000","last_became_peered":"2024-01-11T07:13:18.291824+0000","last_unstale":"2024-01-11T07:13:18.292085+0000","last_undegraded":"2024-01-11T07:13:18.292085+0000","last_fullsized":"2024-01-11T07:13:18.292085+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:19:08.390105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.296449+0000","last_change":"2024-01-11T07:13:17.296023+0000","last_active":"2024-01-11T07:13:17.296449+0000","last_peered":"2024-01-11T07:13:17.296449+0000","last_clean":"2024-01-11T07:13:17.296449+0000","last_became_active":"2024-01-11T07:13:17.295201+0000","last_became_peered":"2024-01-11T07:13:17.295201+0000","last_unstale":"2024-01-11T07:13:17.296449+0000","last_undegraded":"2024-01-11T07:13:17.296449+0000","last_fullsized":"2024-01-11T07:13:17.296449+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:32:22.609129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.301984+0000","last_change":"2024-01-11T07:13:18.301984+0000","last_active":"2024-01-11T07:13:18.301984+0000","last_peered":"2024-01-11T07:13:18.301984+0000","last_clean":"2024-01-11T07:13:18.301984+0000","last_became_active":"2024-01-11T07:13:18.301711+0000","last_became_peered":"2024-01-11T07:13:18.301711+0000","last_unstale":"2024-01-11T07:13:18.301984+0000","last_undegraded":"2024-01-11T07:13:18.301984+0000","last_fullsized":"2024-01-11T07:13:18.301984+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:43:50.526144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.300390+0000","last_change":"2024-01-11T07:13:17.298989+0000","last_active":"2024-01-11T07:13:17.300390+0000","last_peered":"2024-01-11T07:13:17.300390+0000","last_clean":"2024-01-11T07:13:17.300390+0000","last_became_active":"2024-01-11T07:13:17.297773+0000","last_became_peered":"2024-01-11T07:13:17.297773+0000","last_unstale":"2024-01-11T07:13:17.300390+0000","last_undegraded":"2024-01-11T07:13:17.300390+0000","last_fullsized":"2024-01-11T07:13:17.300390+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:28:45.084117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":8,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-01-11T07:13:18.299562+0000","last_change":"2024-01-11T07:13:18.299562+0000","last_active":"2024-01-11T07:13:18.299562+0000","last_peered":"2024-01-11T07:13:18.299562+0000","last_clean":"2024-01-11T07:13:18.299562+0000","last_became_active":"2024-01-11T07:13:18.299317+0000","last_became_peered":"2024-01-11T07:13:18.299317+0000","last_unstale":"2024-01-11T07:13:18.299562+0000","last_undegraded":"2024-01-11T07:13:18.299562+0000","last_fullsized":"2024-01-11T07:13:18.299562+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:17.278894+0000","last_clean_scrub_stamp":"2024-01-11T07:13:17.278894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:41:01.940765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.293969+0000","last_change":"2024-01-11T07:13:17.293893+0000","last_active":"2024-01-11T07:13:17.293969+0000","last_peered":"2024-01-11T07:13:17.293969+0000","last_clean":"2024-01-11T07:13:17.293969+0000","last_became_active":"2024-01-11T07:13:17.293625+0000","last_became_peered":"2024-01-11T07:13:17.293625+0000","last_unstale":"2024-01-11T07:13:17.293969+0000","last_undegraded":"2024-01-11T07:13:17.293969+0000","last_fullsized":"2024-01-11T07:13:17.293969+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:13:16.272168+0000","last_clean_scrub_stamp":"2024-01-11T07:13:16.272168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:58.982357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":263,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-01-11T07:13:17.287495+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:13:17.287495+0000","last_peered":"2024-01-11T07:13:17.287495+0000","last_clean":"2024-01-11T07:13:17.287495+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:13:17.287495+0000","last_undegraded":"2024-01-11T07:13:17.287495+0000","last_fullsized":"2024-01-11T07:13:17.287495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1296,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194015,"statfs":{"total":200000000,"available":198672326,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.33200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68000000000000005,"5min":0.754,"15min":0.754},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72399999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57399999999999995,"15min":0.52400000000000002},"min":{"1min":0.44500000000000001,"5min":0.34000000000000002,"15min":0.311},"max":{"1min":0.753,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.69999999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58999999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56200000000000006,"15min":0.52400000000000002},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73699999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.55200000000000005},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.53600000000000003},"min":{"1min":0.40000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76600000000000001,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69799999999999995}]},{"osd":4,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.499,"15min":0.48699999999999999},"min":{"1min":0.29899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.747,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51500000000000001,"15min":0.48499999999999999},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.79900000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81499999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46700000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61799999999999999,"15min":0.58599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85399999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.57099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199263212,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.40500000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.81100000000000005,"15min":0.81399999999999995},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.46300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76000000000000001,"5min":0.81200000000000006,"15min":1.157},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.5720000000000001,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56999999999999995,"15min":0.55400000000000005},"min":{"1min":0.38700000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.85099999999999998},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72699999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.36599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70399999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.621,"5min":2.621,"15min":2.621},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45300000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.505,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.54800000000000004},"min":{"1min":0.32100000000000001,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.69899999999999995,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.745},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58199999999999996,"15min":0.61699999999999999},"min":{"1min":0.33500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.80500000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.53100000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.749,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.55700000000000005,"15min":0.57799999999999996},"min":{"1min":0.30199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.65000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199270045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.67600000000000005,"15min":0.505},"min":{"1min":0.25800000000000001,"5min":0.216,"15min":0.216},"max":{"1min":8.5150000000000006,"5min":20.41,"15min":20.41},"last":0.753},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.33000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.7029999999999998,"5min":3.7029999999999998,"15min":3.7029999999999998},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.63600000000000001,"15min":0.48799999999999999},"min":{"1min":0.29399999999999998,"5min":0.27000000000000002,"15min":0.26700000000000002},"max":{"1min":7.8099999999999996,"5min":15.611000000000001,"15min":15.611000000000001},"last":3.597},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.56599999999999995},"min":{"1min":0.29699999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":6.1390000000000002,"5min":20.199999999999999,"15min":20.199999999999999},"last":3.0710000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70999999999999996,"15min":0.55300000000000005},"min":{"1min":0.33100000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.6449999999999996,"5min":20.369,"15min":20.369},"last":3.8460000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.496,"15min":0.46000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":8.5790000000000006,"5min":8.5790000000000006,"15min":8.5790000000000006},"last":4.7569999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.76800000000000002,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.317,"15min":0.317},"max":{"1min":7.6749999999999998,"5min":9.9570000000000007,"15min":9.9570000000000007},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80100000000000005,"15min":0.67900000000000005},"min":{"1min":0.46300000000000002,"5min":0.44,"15min":0.41899999999999998},"max":{"1min":5.7830000000000004,"5min":12.449999999999999,"15min":12.449999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64600000000000002,"15min":0.61899999999999999},"min":{"1min":0.45400000000000001,"5min":0.379,"15min":0.376},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.78800000000000003,"15min":0.69399999999999995},"min":{"1min":0.46300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":7.7480000000000002,"5min":9.891,"15min":9.891},"last":1.177}]},{"osd":6,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.995,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.389},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.79300000000000004,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.7770000000000001,"5min":9.9900000000000002,"15min":9.9900000000000002},"last":2.9049999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.92100000000000004,"15min":0.78300000000000003},"min":{"1min":0.499,"5min":0.442,"15min":0.442},"max":{"1min":5.8449999999999998,"5min":20.324000000000002,"15min":20.324000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.69499999999999995,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46800000000000003},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.268}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1289,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194023,"statfs":{"total":200000000,"available":198679677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39900000000000002,"15min":0.46300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.224},"max":{"1min":0.56699999999999995,"5min":0.628,"15min":2.7429999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51500000000000001,"5min":0.627,"15min":0.72199999999999998},"last":0.26400000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.68600000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.73199999999999998,"15min":7.7960000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44600000000000001,"15min":0.52400000000000002},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63400000000000001,"5min":0.72099999999999997,"15min":2.5880000000000001},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.69699999999999995},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":8.0730000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.56599999999999995},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.755,"5min":0.77700000000000002,"15min":2.855},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.443},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.87},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40999999999999998},"max":{"1min":0.86599999999999999,"5min":0.94999999999999996,"15min":8.0399999999999991},"last":0.52500000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65700000000000003,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.872,"5min":0.97099999999999997,"15min":2.819},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.92300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":1.046,"15min":7.9539999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.48099999999999998,"15min":0.41799999999999998},"max":{"1min":0.91700000000000004,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.501}]},{"osd":7,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68100000000000005,"15min":0.66500000000000004},"min":{"1min":0.45200000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68700000000000006,"15min":0.91900000000000004},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.44900000000000001},"max":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":8.0030000000000001},"last":0.495}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194593,"statfs":{"total":200000000,"available":199263282,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41899999999999998},"min":{"1min":0.25800000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.505,"5min":1.472,"15min":1.472},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.312}]},{"osd":1,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.432,"15min":0.44},"min":{"1min":0.313,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.54000000000000004,"5min":1.0309999999999999,"15min":1.2430000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.39100000000000001},"min":{"1min":0.30499999999999999,"5min":0.27400000000000002,"15min":0.246},"max":{"1min":0.56999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.29299999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.48399999999999999},"min":{"1min":0.35799999999999998,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.65500000000000003,"5min":0.74399999999999999,"15min":0.76800000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.622,"5min":1.081,"15min":1.139},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"max":{"1min":0.92400000000000004,"5min":1.0169999999999999,"15min":1.0660000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.624},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97799999999999998,"5min":1.089,"15min":1.089},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68000000000000005,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89200000000000002,"5min":1.145,"15min":1.3100000000000001},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.499,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.052},"last":0.495}]},{"osd":7,"last update":"Thu Jan 11 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.47399999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.54500000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0409999999999999,"5min":1.042,"15min":1.042},"last":0.622}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194591,"statfs":{"total":200000000,"available":199261676,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39400000000000002,"15min":0.75},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.58699999999999997,"15min":11.481},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.34200000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.57299999999999995,"5min":0.65400000000000003,"15min":1.6200000000000001},"last":0.38}]},{"osd":1,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39500000000000002,"15min":0.92300000000000004},"min":{"1min":0.27200000000000002,"5min":0.252,"15min":0.23899999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":17.516999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.29199999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.59899999999999998,"5min":0.66500000000000004,"15min":0.67400000000000004},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42599999999999999,"15min":0.42699999999999999},"min":{"1min":0.30499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.76500000000000001},"last":0.433},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.46899999999999997,"15min":0.97999999999999998},"min":{"1min":0.35499999999999998,"5min":0.33400000000000002,"15min":0.30599999999999999},"max":{"1min":0.54300000000000004,"5min":0.79300000000000004,"15min":17.553000000000001},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40300000000000002},"max":{"1min":0.876,"5min":0.93899999999999995,"15min":0.94599999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68400000000000005,"15min":1.212},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84299999999999997,"5min":0.95499999999999996,"15min":17.474},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":1.149},"min":{"1min":0.42599999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":0.98799999999999999,"15min":17.393000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65700000000000003,"15min":0.98199999999999998},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.002,"5min":1.0149999999999999,"15min":11.394},"last":0.46400000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66300000000000003,"15min":0.997},"min":{"1min":0.46700000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95799999999999996,"5min":0.97299999999999998,"15min":11.436999999999999},"last":0.503},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:12:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94799999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68899999999999995,"15min":1.1870000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.42999999999999999},"max":{"1min":0.84099999999999997,"5min":1.1839999999999999,"15min":17.434000000000001},"last":0.53000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199270045,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.64800000000000002,"15min":1.0129999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.248},"max":{"1min":0.753,"5min":3.5579999999999998,"15min":8.484},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65200000000000002,"15min":1.0660000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.76700000000000002,"5min":3.5329999999999999,"15min":9.2599999999999998},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.59599999999999997,"15min":0.83999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.30199999999999999},"max":{"1min":0.92700000000000005,"5min":1.2250000000000001,"15min":8.5340000000000007},"last":0.505},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61299999999999999,"15min":0.96999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74099999999999999,"5min":4.2560000000000002,"15min":9.2279999999999998},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65900000000000003,"15min":0.94399999999999995},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":5.9770000000000003,"15min":9.5640000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.84999999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90600000000000003,"5min":0.93000000000000005,"15min":6.8639999999999999},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57899999999999996,"15min":1.004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81499999999999995,"5min":1.208,"15min":8.9779999999999998},"last":0.625},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65300000000000002,"15min":1.075},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73799999999999999,"5min":4.8680000000000003,"15min":8.7309999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46200000000000002,"15min":0.89800000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":11.388999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.82799999999999996},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.76800000000000002,"5min":1.0880000000000001,"15min":8.6509999999999998},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53100000000000003,"15min":0.877},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91800000000000004,"15min":9.1449999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.378,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.81200000000000006,"5min":0.86099999999999999,"15min":1.119},"last":0.69199999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.85999999999999999},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.83299999999999996,"5min":0.95999999999999996,"15min":8.6050000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.95499999999999996},"min":{"1min":0.40200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.80000000000000004,"5min":0.874,"15min":9.4640000000000004},"last":0.73599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":720,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199262706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.42899999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81200000000000006,"5min":0.82699999999999996,"15min":0.90000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.38,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.504}]},{"osd":1,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.56100000000000005},"min":{"1min":0.39900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.87},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.39600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76600000000000001,"5min":0.80300000000000005,"15min":0.84099999999999997},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.81599999999999995,"15min":0.82799999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.42299999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79500000000000004,"5min":0.874,"15min":0.93999999999999995},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76700000000000002,"5min":0.84199999999999997,"15min":1.0389999999999999},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.41799999999999998,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.42299999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71799999999999997,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.61699999999999999},"min":{"1min":0.40699999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79700000000000004,"5min":0.85199999999999998,"15min":0.88200000000000001},"last":0.45700000000000002}]}]}],"pool_statfs":[]}} 2024-01-11T07:13:22.899 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:13:22.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:13:23.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.077+0000 7f609a9bf700 1 Processor -- start 2024-01-11T07:13:23.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.077+0000 7f609a9bf700 1 -- start start 2024-01-11T07:13:23.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f609a9bf700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60940b0b40 0x7f60940b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f609a9bf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6094067de0 con 0x7f6094159c20 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f609a9bf700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60940b20e0 con 0x7f6094159ff0 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f609a9bf700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6094069570 con 0x7f60940b0b40 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f6093fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6094159c20 0x7f6094068270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59700/0 (socket says 172.21.15.7:59700) 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f6093fff700 1 -- 172.21.15.7:0/522147309 learned_addr learned my addr 172.21.15.7:0/522147309 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f6098f5c700 1 --2- 172.21.15.7:0/522147309 >> v2:172.21.15.113:3300/0 conn(0x7f60940b0b40 0x7f60940b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f6092ffd700 1 -- 172.21.15.7:0/522147309 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386965278 0 0) 0x7f6094067de0 con 0x7f6094159c20 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.078+0000 7f6092ffd700 1 -- 172.21.15.7:0/522147309 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c003570 con 0x7f6094159c20 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.079+0000 7f6098f5c700 1 -- 172.21.15.7:0/522147309 >> v1:172.21.15.113:6789/0 conn(0x7f6094159ff0 legacy=0x7f60940b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.079+0000 7f6098f5c700 1 -- 172.21.15.7:0/522147309 >> v1:172.21.15.7:6789/0 conn(0x7f6094159c20 legacy=0x7f6094068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.079+0000 7f6092ffd700 1 -- 172.21.15.7:0/522147309 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3070134559 0 0) 0x7f607c003570 con 0x7f6094159c20 2024-01-11T07:13:23.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.079+0000 7f6098f5c700 1 -- 172.21.15.7:0/522147309 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940b1e00 con 0x7f60940b0b40 2024-01-11T07:13:23.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.079+0000 7f6098f5c700 1 --2- 172.21.15.7:0/522147309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60940b0b40 0x7f60940b1820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f608c009e60 tx=0x7f608c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e9ae29f5abde41f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:23.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.079+0000 7f6092ffd700 1 -- 172.21.15.7:0/522147309 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c00e070 con 0x7f60940b0b40 2024-01-11T07:13:23.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.079+0000 7f6092ffd700 1 -- 172.21.15.7:0/522147309 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c00bb60 con 0x7f60940b0b40 2024-01-11T07:13:23.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.079+0000 7f6092ffd700 1 -- 172.21.15.7:0/522147309 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c012430 con 0x7f60940b0b40 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.080+0000 7f609a9bf700 1 -- 172.21.15.7:0/522147309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60940b0b40 msgr2=0x7f60940b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.080+0000 7f609a9bf700 1 --2- 172.21.15.7:0/522147309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60940b0b40 0x7f60940b1820 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f608c009e60 tx=0x7f608c009780 comp rx=0 tx=0).stop 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.080+0000 7f609a9bf700 1 -- 172.21.15.7:0/522147309 shutdown_connections 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.080+0000 7f609a9bf700 1 --2- 172.21.15.7:0/522147309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60940b0b40 0x7f60940b1820 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.080+0000 7f609a9bf700 1 -- 172.21.15.7:0/522147309 >> 172.21.15.7:0/522147309 conn(0x7f6094071160 msgr2=0x7f609406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.080+0000 7f609a9bf700 1 -- 172.21.15.7:0/522147309 shutdown_connections 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.080+0000 7f609a9bf700 1 -- 172.21.15.7:0/522147309 wait complete. 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.081+0000 7f609a9bf700 1 Processor -- start 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.081+0000 7f609a9bf700 1 -- start start 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f609a9bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60940b0b40 0x7f609415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f609a9bf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6094159c20 0x7f609415edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f609a9bf700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6094159ff0 0x7f609415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f609a9bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6094152f60 con 0x7f60940b0b40 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f609a9bf700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609416db50 con 0x7f6094159ff0 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f609a9bf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609415f9e0 con 0x7f6094159c20 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6093fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60940b0b40 0x7f609415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6093fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60940b0b40 0x7f609415e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46038/0 (socket says 172.21.15.7:46038) 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6093fff700 1 -- 172.21.15.7:0/3924194341 learned_addr learned my addr 172.21.15.7:0/3924194341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6098f5c700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6094159ff0 0x7f609415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f60937fe700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6094159c20 0x7f609415edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6093fff700 1 -- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6094159c20 msgr2=0x7f609415edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6093fff700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6094159c20 0x7f609415edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6093fff700 1 -- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6094159ff0 msgr2=0x7f609415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6093fff700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6094159ff0 0x7f609415c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.082+0000 7f6093fff700 1 -- 172.21.15.7:0/3924194341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c009a20 con 0x7f60940b0b40 2024-01-11T07:13:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.083+0000 7f6093fff700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60940b0b40 0x7f609415e8b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f608400c7c0 tx=0x7f6084007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.083+0000 7f607bfff700 1 -- 172.21.15.7:0/3924194341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084007360 con 0x7f60940b0b40 2024-01-11T07:13:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.083+0000 7f607bfff700 1 -- 172.21.15.7:0/3924194341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60840074c0 con 0x7f60940b0b40 2024-01-11T07:13:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.083+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6094152c30 con 0x7f60940b0b40 2024-01-11T07:13:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.083+0000 7f607bfff700 1 -- 172.21.15.7:0/3924194341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084004830 con 0x7f60940b0b40 2024-01-11T07:13:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.083+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609415f700 con 0x7f60940b0b40 2024-01-11T07:13:23.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.084+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60940cde80 con 0x7f60940b0b40 2024-01-11T07:13:23.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.084+0000 7f607bfff700 1 -- 172.21.15.7:0/3924194341 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6084004c00 con 0x7f60940b0b40 2024-01-11T07:13:23.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.085+0000 7f607bfff700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6080066f30 0x7f60800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.085+0000 7f607bfff700 1 -- 172.21.15.7:0/3924194341 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f60840842f0 con 0x7f60940b0b40 2024-01-11T07:13:23.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.085+0000 7f60937fe700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6080066f30 0x7f60800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.086+0000 7f60937fe700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6080066f30 0x7f60800693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f609416d7f0 tx=0x7f6088007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:23.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.089+0000 7f607bfff700 1 -- 172.21.15.7:0/3924194341 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6084051560 con 0x7f60940b0b40 2024-01-11T07:13:23.274 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-11T07:13:23.274+0000 7ff495202700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:13:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.376+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f609404c3c0 con 0x7f60940b0b40 2024-01-11T07:13:23.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.380+0000 7f607bfff700 1 -- 172.21.15.7:0/3924194341 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6084054b80 con 0x7f60940b0b40 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6080066f30 msgr2=0x7f60800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6080066f30 0x7f60800693e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f609416d7f0 tx=0x7f6088007ec0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60940b0b40 msgr2=0x7f609415e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60940b0b40 0x7f609415e8b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f608400c7c0 tx=0x7f6084007b90 comp rx=0 tx=0).stop 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 shutdown_connections 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6080066f30 0x7f60800693e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6094159ff0 0x7f609415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60940b0b40 0x7f609415e8b0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 --2- 172.21.15.7:0/3924194341 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6094159c20 0x7f609415edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 >> 172.21.15.7:0/3924194341 conn(0x7f6094071160 msgr2=0x7f6094156dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 shutdown_connections 2024-01-11T07:13:23.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.383+0000 7f609a9bf700 1 -- 172.21.15.7:0/3924194341 wait complete. 2024-01-11T07:13:23.384 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:13:23.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.528+0000 7fe310507700 1 Processor -- start 2024-01-11T07:13:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.528+0000 7fe310507700 1 -- start start 2024-01-11T07:13:23.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe310507700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe310507700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe308158150 0x7fe30815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe310507700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe308159130 con 0x7fe308152030 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe310507700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe308070a50 con 0x7fe308158150 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe310507700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe308158890 con 0x7fe308152400 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe30eaa4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe308158150 0x7fe30815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe30eaa4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe308158150 0x7fe30815bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46042/0 (socket says 172.21.15.7:46042) 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe30eaa4700 1 -- 172.21.15.7:0/2716900270 learned_addr learned my addr 172.21.15.7:0/2716900270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.529+0000 7fe30daa2700 1 --2- 172.21.15.7:0/2716900270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30eaa4700 1 -- 172.21.15.7:0/2716900270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 msgr2=0x7fe30806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30eaa4700 1 --2- 172.21.15.7:0/2716900270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30806d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30eaa4700 1 -- 172.21.15.7:0/2716900270 >> v1:172.21.15.113:6789/0 conn(0x7fe308152030 legacy=0x7fe308069610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30eaa4700 1 -- 172.21.15.7:0/2716900270 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308158b70 con 0x7fe308158150 2024-01-11T07:13:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30daa2700 1 --2- 172.21.15.7:0/2716900270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30806d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30eaa4700 1 --2- 172.21.15.7:0/2716900270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308158150 0x7fe30815bde0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe300009f20 tx=0x7fe300009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b5681710b7a8cb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30d2a1700 1 -- 172.21.15.7:0/2716900270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000e070 con 0x7fe308158150 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30d2a1700 1 -- 172.21.15.7:0/2716900270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30000ba80 con 0x7fe308158150 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.530+0000 7fe30d2a1700 1 -- 172.21.15.7:0/2716900270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000092e0 con 0x7fe308158150 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.531+0000 7fe310507700 1 -- 172.21.15.7:0/2716900270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308158150 msgr2=0x7fe30815bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.531+0000 7fe310507700 1 --2- 172.21.15.7:0/2716900270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308158150 0x7fe30815bde0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe300009f20 tx=0x7fe300009840 comp rx=0 tx=0).stop 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.531+0000 7fe310507700 1 -- 172.21.15.7:0/2716900270 shutdown_connections 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.531+0000 7fe310507700 1 --2- 172.21.15.7:0/2716900270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308158150 0x7fe30815bde0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.531+0000 7fe310507700 1 --2- 172.21.15.7:0/2716900270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30806d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.531+0000 7fe310507700 1 -- 172.21.15.7:0/2716900270 >> 172.21.15.7:0/2716900270 conn(0x7fe3080cd090 msgr2=0x7fe3080cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.531+0000 7fe310507700 1 -- 172.21.15.7:0/2716900270 shutdown_connections 2024-01-11T07:13:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.531+0000 7fe310507700 1 -- 172.21.15.7:0/2716900270 wait complete. 2024-01-11T07:13:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.532+0000 7fe310507700 1 Processor -- start 2024-01-11T07:13:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.532+0000 7fe310507700 1 -- start start 2024-01-11T07:13:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe310507700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308152030 0x7fe3080c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe310507700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30816e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe310507700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe308158150 0x7fe30814bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe310507700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080c7480 con 0x7fe308152030 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe310507700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080c4680 con 0x7fe308158150 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe310507700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30816ede0 con 0x7fe308152400 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308152030 0x7fe3080c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308152030 0x7fe3080c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46058/0 (socket says 172.21.15.7:46058) 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 -- 172.21.15.7:0/1183355705 learned_addr learned my addr 172.21.15.7:0/1183355705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30eaa4700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe308158150 0x7fe30814bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30daa2700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30816e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 -- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 msgr2=0x7fe30816e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30816e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 -- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe308158150 msgr2=0x7fe30814bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe308158150 0x7fe30814bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 -- 172.21.15.7:0/1183355705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe300009ae0 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.533+0000 7fe30e2a3700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308152030 0x7fe3080c4140 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c590 tx=0x7fe30400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.534+0000 7fe2fe7fc700 1 -- 172.21.15.7:0/1183355705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304011590 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.534+0000 7fe2fe7fc700 1 -- 172.21.15.7:0/1183355705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe304011710 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.534+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe308151150 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.534+0000 7fe2fe7fc700 1 -- 172.21.15.7:0/1183355705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304011ad0 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.534+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3080c71a0 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.535+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30804c3c0 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.535+0000 7fe2fe7fc700 1 -- 172.21.15.7:0/1183355705 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe30401b3b0 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.536+0000 7fe2fe7fc700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2ec066f30 0x7fe2ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.536+0000 7fe2fe7fc700 1 -- 172.21.15.7:0/1183355705 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe304083820 con 0x7fe308152030 2024-01-11T07:13:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.536+0000 7fe30daa2700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2ec066f30 0x7fe2ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.536+0000 7fe30daa2700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2ec066f30 0x7fe2ec0693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800adc0 tx=0x7fe2f8006f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.540+0000 7fe2fe7fc700 1 -- 172.21.15.7:0/1183355705 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3040509f0 con 0x7fe308152030 2024-01-11T07:13:23.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.788+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe308159130 con 0x7fe308152030 2024-01-11T07:13:23.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.788+0000 7fe2fe7fc700 1 -- 172.21.15.7:0/1183355705 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fe308159130 con 0x7fe308152030 2024-01-11T07:13:23.788 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:23.788 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:23.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2ec066f30 msgr2=0x7fe2ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2ec066f30 0x7fe2ec0693e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800adc0 tx=0x7fe2f8006f90 comp rx=0 tx=0).stop 2024-01-11T07:13:23.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308152030 msgr2=0x7fe3080c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308152030 0x7fe3080c4140 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c590 tx=0x7fe30400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 shutdown_connections 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe2ec066f30 0x7fe2ec0693e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe308158150 0x7fe30814bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe308152030 0x7fe3080c4140 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 --2- 172.21.15.7:0/1183355705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe308152400 0x7fe30816e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.790+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 >> 172.21.15.7:0/1183355705 conn(0x7fe3080cd090 msgr2=0x7fe30814d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.791+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 shutdown_connections 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:23.791+0000 7fe310507700 1 -- 172.21.15.7:0/1183355705 wait complete. 2024-01-11T07:13:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:24.207 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:13:17.725345+0000', 'modified': '2024-01-11T07:13:21.305111+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7068}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7068': {'gid': 7068, 'name': 'c', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/3803559903', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 3803559903}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3803559903}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:13:24.207 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:13:24.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-11T07:13:24.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.350+0000 7fa46de37700 1 Processor -- start 2024-01-11T07:13:24.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.350+0000 7fa46de37700 1 -- start start 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.351+0000 7fa46de37700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468157de0 0x7fa46815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.351+0000 7fa46de37700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4680ac9e0 con 0x7fa468157a10 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.351+0000 7fa46de37700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46806df50 con 0x7fa4680b2b70 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.351+0000 7fa46de37700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46806e230 con 0x7fa468157de0 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.351+0000 7fa4677fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa468157a10 0x7fa4680acd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59712/0 (socket says 172.21.15.7:59712) 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.351+0000 7fa4677fe700 1 -- 172.21.15.7:0/2775640 learned_addr learned my addr 172.21.15.7:0/2775640 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa466ffd700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468157de0 0x7fa46815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073609220 0 0) 0x7fa4680ac9e0 con 0x7fa468157a10 2024-01-11T07:13:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa450003570 con 0x7fa468157a10 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505219455 0 0) 0x7fa46806df50 con 0x7fa4680b2b70 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4680ac9e0 con 0x7fa4680b2b70 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2084435697 0 0) 0x7fa450003570 con 0x7fa468157a10 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468157de0 msgr2=0x7fa46815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468157de0 0x7fa46815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 >> v1:172.21.15.113:6789/0 conn(0x7fa4680b2b70 legacy=0x7fa468063610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46806e510 con 0x7fa468157a10 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa458001470 con 0x7fa468157a10 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 >> v1:172.21.15.7:6789/0 conn(0x7fa468157a10 legacy=0x7fa4680acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450003c20 0x7fa450004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa450004810 0x7fa450004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468157de0 0x7fa450005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46806e230 con 0x7fa450004810 2024-01-11T07:13:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45000bc20 con 0x7fa450003c20 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45000bde0 con 0x7fa468157de0 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa466ffd700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa450004810 0x7fa450004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa4677fe700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450003c20 0x7fa450004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.352+0000 7fa467fff700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468157de0 0x7fa450005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa466ffd700 1 -- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468157de0 msgr2=0x7fa450005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa466ffd700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468157de0 0x7fa450005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa466ffd700 1 -- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450003c20 msgr2=0x7fa450004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa466ffd700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450003c20 0x7fa450004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa466ffd700 1 -- 172.21.15.7:0/2775640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa450003570 con 0x7fa450004810 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa466ffd700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa450004810 0x7fa450004c80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009c00 tx=0x7fa45c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e896b3085c1cb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00e710 con 0x7fa450004810 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c00e870 con 0x7fa450004810 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.353+0000 7fa4667fc700 1 -- 172.21.15.7:0/2775640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00ec30 con 0x7fa450004810 2024-01-11T07:13:24.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.354+0000 7fa46de37700 1 -- 172.21.15.7:0/2775640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa450004810 msgr2=0x7fa450004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.354+0000 7fa46de37700 1 --2- 172.21.15.7:0/2775640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa450004810 0x7fa450004c80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009c00 tx=0x7fa45c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.354+0000 7fa467fff700 1 -- 172.21.15.7:0/2775640 reap_dead start 2024-01-11T07:13:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.354+0000 7fa46de37700 1 -- 172.21.15.7:0/2775640 shutdown_connections 2024-01-11T07:13:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.354+0000 7fa46de37700 1 -- 172.21.15.7:0/2775640 >> 172.21.15.7:0/2775640 conn(0x7fa4680cd910 msgr2=0x7fa4680cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.354+0000 7fa46de37700 1 -- 172.21.15.7:0/2775640 shutdown_connections 2024-01-11T07:13:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.354+0000 7fa46de37700 1 -- 172.21.15.7:0/2775640 wait complete. 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.355+0000 7fa46de37700 1 Processor -- start 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.355+0000 7fa46de37700 1 -- start start 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa46de37700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450004810 0x7fa4681523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa46de37700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468154820 0x7fa468154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa46de37700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa46816a9c0 0x7fa46816ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa46de37700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa468155e40 con 0x7fa46816a9c0 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa46de37700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46816a260 con 0x7fa450004810 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa46de37700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46816a530 con 0x7fa468154820 2024-01-11T07:13:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa467fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa46816a9c0 0x7fa46816ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa467fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa46816a9c0 0x7fa46816ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46074/0 (socket says 172.21.15.7:46074) 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa467fff700 1 -- 172.21.15.7:0/3962723195 learned_addr learned my addr 172.21.15.7:0/3962723195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa4677fe700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450004810 0x7fa4681523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.356+0000 7fa466ffd700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468154820 0x7fa468154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa467fff700 1 -- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468154820 msgr2=0x7fa468154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa467fff700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468154820 0x7fa468154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa467fff700 1 -- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450004810 msgr2=0x7fa4681523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa467fff700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450004810 0x7fa4681523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa467fff700 1 -- 172.21.15.7:0/3962723195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4681555a0 con 0x7fa46816a9c0 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa4677fe700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450004810 0x7fa4681523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa467fff700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa46816a9c0 0x7fa46816ae30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa4681581b0 tx=0x7fa4600017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa44f7fe700 1 -- 172.21.15.7:0/3962723195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460018070 con 0x7fa46816a9c0 2024-01-11T07:13:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa44f7fe700 1 -- 172.21.15.7:0/3962723195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa460006ba0 con 0x7fa46816a9c0 2024-01-11T07:13:24.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa46de37700 1 -- 172.21.15.7:0/3962723195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa468155880 con 0x7fa46816a9c0 2024-01-11T07:13:24.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa44f7fe700 1 -- 172.21.15.7:0/3962723195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46000c880 con 0x7fa46816a9c0 2024-01-11T07:13:24.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.357+0000 7fa46de37700 1 -- 172.21.15.7:0/3962723195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4681551d0 con 0x7fa46816a9c0 2024-01-11T07:13:24.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.358+0000 7fa44f7fe700 1 -- 172.21.15.7:0/3962723195 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa46000cbf0 con 0x7fa46816a9c0 2024-01-11T07:13:24.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.359+0000 7fa44f7fe700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa454066f30 0x7fa4540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.359+0000 7fa44f7fe700 1 -- 172.21.15.7:0/3962723195 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa460084400 con 0x7fa46816a9c0 2024-01-11T07:13:24.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.359+0000 7fa4677fe700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa454066f30 0x7fa4540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.360+0000 7fa4677fe700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa454066f30 0x7fa4540693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa458001120 tx=0x7fa458009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.361+0000 7fa477137b80 1 Processor -- start 2024-01-11T07:13:24.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.361+0000 7fa477137b80 1 -- start start 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa477137b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729dfa0 0x55d9f729e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa477137b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f7299c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa477137b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 0x55d9f72ba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa477137b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9f71cc3e0 con 0x55d9f729e8b0 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa477137b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9f71c9be0 con 0x55d9f729dfa0 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa477137b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9f71c59d0 con 0x55d9f729a170 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa466ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f7299c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa466ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f7299c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46086/0 (socket says 172.21.15.7:46086) 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa466ffd700 1 -- 172.21.15.7:0/1721523271 learned_addr learned my addr 172.21.15.7:0/1721523271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa4677fe700 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729dfa0 0x55d9f729e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa467fff700 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 0x55d9f72ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa466ffd700 1 -- 172.21.15.7:0/1721523271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 msgr2=0x55d9f72ba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa466ffd700 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 0x55d9f72ba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa466ffd700 1 -- 172.21.15.7:0/1721523271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729dfa0 msgr2=0x55d9f729e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa466ffd700 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729dfa0 0x55d9f729e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.362+0000 7fa466ffd700 1 -- 172.21.15.7:0/1721523271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9f7059360 con 0x55d9f729e8b0 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.363+0000 7fa466ffd700 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f7299c30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00e4b0 tx=0x7fa45c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5839681160da6aa7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.363+0000 7fa44cff9700 1 -- 172.21.15.7:0/1721523271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00e710 con 0x55d9f729e8b0 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.363+0000 7fa44cff9700 1 -- 172.21.15.7:0/1721523271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c023040 con 0x55d9f729e8b0 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.363+0000 7fa44cff9700 1 -- 172.21.15.7:0/1721523271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c0095b0 con 0x55d9f729e8b0 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 -- 172.21.15.7:0/1721523271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 msgr2=0x55d9f7299c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f7299c30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00e4b0 tx=0x7fa45c01a000 comp rx=0 tx=0).stop 2024-01-11T07:13:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 -- 172.21.15.7:0/1721523271 shutdown_connections 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 0x55d9f72ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729dfa0 0x55d9f729e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 --2- 172.21.15.7:0/1721523271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f7299c30 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 -- 172.21.15.7:0/1721523271 >> 172.21.15.7:0/1721523271 conn(0x55d9f7274440 msgr2=0x55d9f72bfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 -- 172.21.15.7:0/1721523271 shutdown_connections 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 -- 172.21.15.7:0/1721523271 wait complete. 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.364+0000 7fa477137b80 1 -- start start 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa477137b80 1 Processor -- start 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa477137b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 0x55d9f72cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa477137b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa477137b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa477137b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9f71721b0 con 0x55d9f729e8b0 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa477137b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9f7171e50 con 0x55d9f729a170 2024-01-11T07:13:24.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa477137b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9f70bee40 con 0x55d9f729dfa0 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa477137b80 10 client.0 _ll_register_callbacks cb 0x55d9f70c17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa467fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa467fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46098/0 (socket says 172.21.15.7:46098) 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.365+0000 7fa467fff700 1 -- 172.21.15.7:0/631680768 learned_addr learned my addr 172.21.15.7:0/631680768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa466ffd700 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa4677fe700 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 0x55d9f72cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa467fff700 1 -- 172.21.15.7:0/631680768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729dfa0 msgr2=0x55d9f72bfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa467fff700 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa467fff700 1 -- 172.21.15.7:0/631680768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 msgr2=0x55d9f72cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa467fff700 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 0x55d9f72cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa467fff700 1 -- 172.21.15.7:0/631680768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450003570 con 0x55d9f729e8b0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa467fff700 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa460001460 tx=0x7fa460009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ec2b96269628b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa445ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa445ffb700 1 -- 172.21.15.7:0/631680768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460058320 con 0x55d9f729e8b0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa445ffb700 1 -- 172.21.15.7:0/631680768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa460055290 con 0x55d9f729e8b0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa477137b80 1 -- 172.21.15.7:0/631680768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9f7246eb0 con 0x55d9f729e8b0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa445ffb700 1 -- 172.21.15.7:0/631680768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4600584e0 con 0x55d9f729e8b0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa477137b80 10 client.7197 fetch_fsmap learned FSMap version 65 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa477137b80 1 -- 172.21.15.7:0/631680768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d9f7005b10 con 0x55d9f729e8b0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.366+0000 7fa445ffb700 1 -- 172.21.15.7:0/631680768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d9f7246eb0 con 0x55d9f729e8b0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa445ffb700 1 -- 172.21.15.7:0/631680768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 65) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa4600554f0 con 0x55d9f729e8b0 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa477137b80 10 client.7197 fetch_fsmap finished waiting for FSMap version 65 2024-01-11T07:13:24.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa477137b80 10 client.7197 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa477137b80 20 client.7197 populate_metadata read hostname 'smithi007' 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa477137b80 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] conn(0x55d9f72b5ea0 0x55d9f72b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa477137b80 4 client.7197 mds_command: new command op to 7068 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa477137b80 1 -- 172.21.15.7:0/631680768 --> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9f719c880 con 0x55d9f72b5ea0 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa4677fe700 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] conn(0x55d9f72b5ea0 0x55d9f72b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa445ffb700 10 client.7197 ms_handle_connect on v2:172.21.15.7:6833/3803559903 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.367+0000 7fa4677fe700 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] conn(0x55d9f72b5ea0 0x55d9f72b6270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7088940e6f4d81 server_cookie=45ef12238d64b718 in_seq=0 out_seq=0 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa445ffb700 1 -- 172.21.15.7:0/631680768 <== mds.0 v2:172.21.15.7:6833/3803559903 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d9f719c880 con 0x55d9f72b5ea0 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa445ffb700 10 client.7197 handle_command_reply: tid=0 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 client.7197 shutdown 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 -- 172.21.15.7:0/631680768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 msgr2=0x55d9f72c0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa460001460 tx=0x7fa460009040 comp rx=0 tx=0).stop 2024-01-11T07:13:24.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 -- 172.21.15.7:0/631680768 shutdown_connections 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] conn(0x55d9f72b5ea0 0x55d9f72b6270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 0x55d9f72cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa445ffb700 0 client.7197 ms_handle_reset on v2:172.21.15.7:6833/3803559903 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 --2- 172.21.15.7:0/631680768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 -- 172.21.15.7:0/631680768 >> 172.21.15.7:0/631680768 conn(0x55d9f7274440 msgr2=0x55d9f7278c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 -- 172.21.15.7:0/631680768 shutdown_connections 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 1 -- 172.21.15.7:0/631680768 wait complete. 2024-01-11T07:13:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.368+0000 7fa477137b80 20 client.7197 trim_cache size 0 max 16384 2024-01-11T07:13:24.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.386+0000 7fa477137b80 1 Processor -- start 2024-01-11T07:13:24.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.386+0000 7fa477137b80 1 -- start start 2024-01-11T07:13:24.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa477137b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 0x55d9f72b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa477137b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f726bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa477137b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729e8b0 0x55d9f726c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa477137b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9f72748b0 con 0x55d9f729dfa0 2024-01-11T07:13:24.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa477137b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9f72731c0 con 0x55d9f729a170 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa477137b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9f7298b20 con 0x55d9f729e8b0 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f726bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f726bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46104/0 (socket says 172.21.15.7:46104) 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 -- 172.21.15.7:0/3164189847 learned_addr learned my addr 172.21.15.7:0/3164189847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa4677fe700 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 0x55d9f72b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa467fff700 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729e8b0 0x55d9f726c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 -- 172.21.15.7:0/3164189847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729e8b0 msgr2=0x55d9f726c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729e8b0 0x55d9f726c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 -- 172.21.15.7:0/3164189847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 msgr2=0x55d9f72b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 0x55d9f72b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 -- 172.21.15.7:0/3164189847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9f7194290 con 0x55d9f729dfa0 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.387+0000 7fa466ffd700 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f726bee0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x55d9f72bfd90 tx=0x7fa45c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f492810faa6809fc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa44d7fa700 1 -- 172.21.15.7:0/3164189847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c006e50 con 0x55d9f729dfa0 2024-01-11T07:13:24.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa44d7fa700 1 -- 172.21.15.7:0/3164189847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c023040 con 0x55d9f729dfa0 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa44d7fa700 1 -- 172.21.15.7:0/3164189847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00ebf0 con 0x55d9f729dfa0 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 -- 172.21.15.7:0/3164189847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 msgr2=0x55d9f726bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f726bee0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x55d9f72bfd90 tx=0x7fa45c024000 comp rx=0 tx=0).stop 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 -- 172.21.15.7:0/3164189847 shutdown_connections 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729e8b0 0x55d9f726c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729a170 0x55d9f72b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 --2- 172.21.15.7:0/3164189847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f726bee0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 -- 172.21.15.7:0/3164189847 >> 172.21.15.7:0/3164189847 conn(0x55d9f7274440 msgr2=0x55d9f72b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 -- 172.21.15.7:0/3164189847 shutdown_connections 2024-01-11T07:13:24.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.388+0000 7fa477137b80 1 -- 172.21.15.7:0/3164189847 wait complete. 2024-01-11T07:13:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.389+0000 7fa477137b80 1 -- start start 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.389+0000 7fa477137b80 1 Processor -- start 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa477137b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 0x55d9f73201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa477137b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa477137b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa477137b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9f726c490 con 0x55d9f729dfa0 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa477137b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9f7277a40 con 0x55d9f729e8b0 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa477137b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9f72a4cd0 con 0x55d9f729a170 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa477137b80 10 client.0 _ll_register_callbacks cb 0x55d9f70c17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa466ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa466ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46114/0 (socket says 172.21.15.7:46114) 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa466ffd700 1 -- 172.21.15.7:0/1635145653 learned_addr learned my addr 172.21.15.7:0/1635145653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa467fff700 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa4677fe700 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 0x55d9f73201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa466ffd700 1 -- 172.21.15.7:0/1635145653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 msgr2=0x55d9f73201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa466ffd700 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 0x55d9f73201b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa466ffd700 1 -- 172.21.15.7:0/1635145653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729e8b0 msgr2=0x55d9f72c0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa466ffd700 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.390+0000 7fa466ffd700 1 -- 172.21.15.7:0/1635145653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c02c040 con 0x55d9f729dfa0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa466ffd700 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa45c035040 tx=0x7fa45c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb6dfbc575df652 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa445ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa445ffb700 1 -- 172.21.15.7:0/1635145653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c00ebf0 con 0x55d9f729dfa0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa445ffb700 1 -- 172.21.15.7:0/1635145653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa45c013680 con 0x55d9f729dfa0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 1 -- 172.21.15.7:0/1635145653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9f726a490 con 0x55d9f729dfa0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa445ffb700 1 -- 172.21.15.7:0/1635145653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa45c013a50 con 0x55d9f729dfa0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 10 client.7203 fetch_fsmap learned FSMap version 65 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 1 -- 172.21.15.7:0/1635145653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d9f705dc80 con 0x55d9f729dfa0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa445ffb700 1 -- 172.21.15.7:0/1635145653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d9f726a490 con 0x55d9f729dfa0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa445ffb700 1 -- 172.21.15.7:0/1635145653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 65) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa45c02f070 con 0x55d9f729dfa0 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 10 client.7203 fetch_fsmap finished waiting for FSMap version 65 2024-01-11T07:13:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 10 client.7203 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 20 client.7203 populate_metadata read hostname 'smithi007' 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] conn(0x55d9f7276c60 0x55d9f7277030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 4 client.7203 mds_command: new command op to 7068 tid=0[{"prefix": "status"}] 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa477137b80 1 -- 172.21.15.7:0/1635145653 --> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d9f7168910 con 0x55d9f7276c60 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.391+0000 7fa4677fe700 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] conn(0x55d9f7276c60 0x55d9f7277030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa445ffb700 10 client.7203 ms_handle_connect on v2:172.21.15.7:6833/3803559903 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa4677fe700 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] conn(0x55d9f7276c60 0x55d9f7277030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349795fbc5920817 server_cookie=257dbba0ad5c07a6 in_seq=0 out_seq=0 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa445ffb700 1 -- 172.21.15.7:0/1635145653 <== mds.0 v2:172.21.15.7:6833/3803559903 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55d9f7168910 con 0x55d9f7276c60 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa445ffb700 10 client.7203 handle_command_reply: tid=0 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 client.7203 shutdown 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 -- 172.21.15.7:0/1635145653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 msgr2=0x55d9f72bfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa45c035040 tx=0x7fa45c02f000 comp rx=0 tx=0).stop 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 -- 172.21.15.7:0/1635145653 shutdown_connections 2024-01-11T07:13:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.7:6833/3803559903,v1:172.21.15.7:6835/3803559903] conn(0x55d9f7276c60 0x55d9f7277030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d9f729e8b0 0x55d9f72c0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d9f729a170 0x55d9f73201b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa445ffb700 0 client.7203 ms_handle_reset on v2:172.21.15.7:6833/3803559903 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 --2- 172.21.15.7:0/1635145653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9f729dfa0 0x55d9f72bfdc0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 -- 172.21.15.7:0/1635145653 >> 172.21.15.7:0/1635145653 conn(0x55d9f7274440 msgr2=0x55d9f72b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 -- 172.21.15.7:0/1635145653 shutdown_connections 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 1 -- 172.21.15.7:0/1635145653 wait complete. 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.392+0000 7fa477137b80 20 client.7203 trim_cache size 0 max 16384 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:13:24.398 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7068, 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.0960814860000001, 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 65, 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 135, 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 135, 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 12.090027750999999 2024-01-11T07:13:24.399 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 -- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa454066f30 msgr2=0x7fa4540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa454066f30 0x7fa4540693e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa458001120 tx=0x7fa458009040 comp rx=0 tx=0).stop 2024-01-11T07:13:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 -- 172.21.15.7:0/3962723195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa46816a9c0 msgr2=0x7fa46816ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa46816a9c0 0x7fa46816ae30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa4681581b0 tx=0x7fa4600017a0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 -- 172.21.15.7:0/3962723195 shutdown_connections 2024-01-11T07:13:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa454066f30 0x7fa4540693e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa46816a9c0 0x7fa46816ae30 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa450004810 0x7fa4681523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 --2- 172.21.15.7:0/3962723195 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa468154820 0x7fa468154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 -- 172.21.15.7:0/3962723195 >> 172.21.15.7:0/3962723195 conn(0x7fa4680cd910 msgr2=0x7fa4680ad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 -- 172.21.15.7:0/3962723195 shutdown_connections 2024-01-11T07:13:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.393+0000 7fa46de37700 1 -- 172.21.15.7:0/3962723195 wait complete. 2024-01-11T07:13:24.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:13:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.538+0000 7f84edde8700 1 Processor -- start 2024-01-11T07:13:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.538+0000 7f84edde8700 1 -- start start 2024-01-11T07:13:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.539+0000 7f84edde8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f84e80adf10 0x7f84e80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.539+0000 7f84edde8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e80cc2a0 con 0x7f84e80adb40 2024-01-11T07:13:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.539+0000 7f84edde8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e80b3b50 con 0x7f84e80b3590 2024-01-11T07:13:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.539+0000 7f84edde8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e80b3e30 con 0x7f84e80adf10 2024-01-11T07:13:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.539+0000 7f84e77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f84e80adb40 0x7f84e80b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59720/0 (socket says 172.21.15.7:59720) 2024-01-11T07:13:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.539+0000 7f84e77fe700 1 -- 172.21.15.7:0/1729344743 learned_addr learned my addr 172.21.15.7:0/1729344743 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e6ffd700 1 --2- 172.21.15.7:0/1729344743 >> v2:172.21.15.113:3300/0 conn(0x7f84e80adf10 0x7f84e80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e67fc700 1 -- 172.21.15.7:0/1729344743 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159157720 0 0) 0x7f84e80cc2a0 con 0x7f84e80adb40 2024-01-11T07:13:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e67fc700 1 -- 172.21.15.7:0/1729344743 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d0003570 con 0x7f84e80adb40 2024-01-11T07:13:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e6ffd700 1 -- 172.21.15.7:0/1729344743 >> v1:172.21.15.113:6789/0 conn(0x7f84e80b3590 legacy=0x7f84e80cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e67fc700 1 -- 172.21.15.7:0/1729344743 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411710009 0 0) 0x7f84e80b3b50 con 0x7f84e80b3590 2024-01-11T07:13:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e6ffd700 1 -- 172.21.15.7:0/1729344743 >> v1:172.21.15.7:6789/0 conn(0x7f84e80adb40 legacy=0x7f84e80b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e6ffd700 1 -- 172.21.15.7:0/1729344743 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e80cc540 con 0x7f84e80adf10 2024-01-11T07:13:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e6ffd700 1 --2- 172.21.15.7:0/1729344743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adf10 0x7f84e80b0430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0098e0 tx=0x7f84dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79587287d62e7807 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e67fc700 1 -- 172.21.15.7:0/1729344743 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc00b640 con 0x7f84e80adf10 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e67fc700 1 -- 172.21.15.7:0/1729344743 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc00b7a0 con 0x7f84e80adf10 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.540+0000 7f84e67fc700 1 -- 172.21.15.7:0/1729344743 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc00bb40 con 0x7f84e80adf10 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.541+0000 7f84edde8700 1 -- 172.21.15.7:0/1729344743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adf10 msgr2=0x7f84e80b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.541+0000 7f84edde8700 1 --2- 172.21.15.7:0/1729344743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adf10 0x7f84e80b0430 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0098e0 tx=0x7f84dc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.541+0000 7f84edde8700 1 -- 172.21.15.7:0/1729344743 shutdown_connections 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.541+0000 7f84edde8700 1 --2- 172.21.15.7:0/1729344743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adf10 0x7f84e80b0430 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.541+0000 7f84edde8700 1 -- 172.21.15.7:0/1729344743 >> 172.21.15.7:0/1729344743 conn(0x7f84e8071160 msgr2=0x7f84e806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.541+0000 7f84edde8700 1 -- 172.21.15.7:0/1729344743 shutdown_connections 2024-01-11T07:13:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.541+0000 7f84edde8700 1 -- 172.21.15.7:0/1729344743 wait complete. 2024-01-11T07:13:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.542+0000 7f84edde8700 1 Processor -- start 2024-01-11T07:13:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84edde8700 1 -- start start 2024-01-11T07:13:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84edde8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adb40 0x7f84e81428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84edde8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84e80adf10 0x7f84e80aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84edde8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84e80b3590 0x7f84e80aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84edde8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80c7c10 con 0x7f84e80b3590 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84edde8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80c81d0 con 0x7f84e80adb40 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84edde8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8069140 con 0x7f84e80adf10 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84e80b3590 0x7f84e80aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84e80b3590 0x7f84e80aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46122/0 (socket says 172.21.15.7:46122) 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.543+0000 7f84e7fff700 1 -- 172.21.15.7:0/4075779537 learned_addr learned my addr 172.21.15.7:0/4075779537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84e6ffd700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84e80adf10 0x7f84e80aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84e77fe700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adb40 0x7f84e81428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84e7fff700 1 -- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84e80adf10 msgr2=0x7f84e80aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84e7fff700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84e80adf10 0x7f84e80aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84e7fff700 1 -- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adb40 msgr2=0x7f84e81428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84e7fff700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adb40 0x7f84e81428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84e7fff700 1 -- 172.21.15.7:0/4075779537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc009420 con 0x7f84e80b3590 2024-01-11T07:13:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84e7fff700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84e80b3590 0x7f84e80aefd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c890 tx=0x7f84e0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84cf7fe700 1 -- 172.21.15.7:0/4075779537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0012630 con 0x7f84e80b3590 2024-01-11T07:13:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84cf7fe700 1 -- 172.21.15.7:0/4075779537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e0012790 con 0x7f84e80b3590 2024-01-11T07:13:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e8069420 con 0x7f84e80b3590 2024-01-11T07:13:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84cf7fe700 1 -- 172.21.15.7:0/4075779537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0012b50 con 0x7f84e80b3590 2024-01-11T07:13:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.544+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e8142e30 con 0x7f84e80b3590 2024-01-11T07:13:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.545+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e80c8340 con 0x7f84e80b3590 2024-01-11T07:13:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.546+0000 7f84cf7fe700 1 -- 172.21.15.7:0/4075779537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f84e001a3a0 con 0x7f84e80b3590 2024-01-11T07:13:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.546+0000 7f84cf7fe700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f84d4066f30 0x7f84d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.546+0000 7f84cf7fe700 1 -- 172.21.15.7:0/4075779537 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f84e0084650 con 0x7f84e80b3590 2024-01-11T07:13:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.547+0000 7f84e77fe700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f84d4066f30 0x7f84d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.547+0000 7f84e77fe700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f84d4066f30 0x7f84d40693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84e8156ec0 tx=0x7f84d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.551+0000 7f84cf7fe700 1 -- 172.21.15.7:0/4075779537 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84e00518c0 con 0x7f84e80b3590 2024-01-11T07:13:24.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.803+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f84e804c3c0 con 0x7f84e80b3590 2024-01-11T07:13:24.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.803+0000 7f84cf7fe700 1 -- 172.21.15.7:0/4075779537 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f84e0054ee0 con 0x7f84e80b3590 2024-01-11T07:13:24.803 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f84d4066f30 msgr2=0x7f84d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f84d4066f30 0x7f84d40693e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84e8156ec0 tx=0x7f84d8009040 comp rx=0 tx=0).stop 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84e80b3590 msgr2=0x7f84e80aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84e80b3590 0x7f84e80aefd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c890 tx=0x7f84e0001670 comp rx=0 tx=0).stop 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 shutdown_connections 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f84d4066f30 0x7f84d40693e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84e80b3590 0x7f84e80aefd0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f84e80adb40 0x7f84e81428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 --2- 172.21.15.7:0/4075779537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f84e80adf10 0x7f84e80aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 >> 172.21.15.7:0/4075779537 conn(0x7f84e8071160 msgr2=0x7f84e80b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 shutdown_connections 2024-01-11T07:13:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:24.806+0000 7f84edde8700 1 -- 172.21.15.7:0/4075779537 wait complete. 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:13:24.819 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:13:24.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:24.820 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:13:24.839 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:24.840 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42013sec preferred_lft 42013sec 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:13:24.841 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:13:24.842 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:13:24.842 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:24.842 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:13:24.842 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:13:24.842 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:13:24.842 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:24.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:24.842 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:13:24.911 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:13:24.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:24.913 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:13:24.978 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:13:24.978 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:13:24.978 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:13:24.978 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:13:24.978 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:13:24.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:24.979 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:13:24.979 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:13:24.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:13:24.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:13:24.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:13:24.979 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:13:25.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:25.101 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:13:25.122 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:13:25.122 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:13:25.122 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:13:25.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:25.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:13:25.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:13:25.198 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:13:25.260 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:13:25.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:13:25.260 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:13:25.321 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:13:25.322 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:13:25.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:13:25.415 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:13:25.415 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:13:25.415 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:13:25.425 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:13:25.425 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:13:25.425 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:13:25.425 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:13:25.425 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:13:25.442 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:13:25.443 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:13:25.443 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:13:25.443 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:13:25.443 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:13:25.443 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:13:25.443 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:13:25.443 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:13:25.448 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:13:25.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:25.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:13:25.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:13:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.659+0000 7fbd9c624700 1 Processor -- start 2024-01-11T07:13:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.659+0000 7fbd9c624700 1 -- start start 2024-01-11T07:13:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.660+0000 7fbd9c624700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbd941579b0 0x7fbd9415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.660+0000 7fbd9c624700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd94157d80 0x7fbd94063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.660+0000 7fbd9c624700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd94063b90 con 0x7fbd94063770 2024-01-11T07:13:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.660+0000 7fbd9c624700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd940aeab0 con 0x7fbd94157d80 2024-01-11T07:13:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.660+0000 7fbd9c624700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd940aed90 con 0x7fbd941579b0 2024-01-11T07:13:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd99bbf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd94157d80 0x7fbd94063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd99bbf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd94157d80 0x7fbd94063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46124/0 (socket says 172.21.15.7:46124) 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd99bbf700 1 -- 172.21.15.7:0/1293023099 learned_addr learned my addr 172.21.15.7:0/1293023099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd9a3c0700 1 --2- 172.21.15.7:0/1293023099 >> v2:172.21.15.113:3300/0 conn(0x7fbd941579b0 0x7fbd9415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd99bbf700 1 -- 172.21.15.7:0/1293023099 >> v2:172.21.15.113:3300/0 conn(0x7fbd941579b0 msgr2=0x7fbd9415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd99bbf700 1 --2- 172.21.15.7:0/1293023099 >> v2:172.21.15.113:3300/0 conn(0x7fbd941579b0 0x7fbd9415bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd99bbf700 1 -- 172.21.15.7:0/1293023099 >> v1:172.21.15.113:6789/0 conn(0x7fbd94063770 legacy=0x7fbd9406dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd99bbf700 1 -- 172.21.15.7:0/1293023099 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940af070 con 0x7fbd94157d80 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd99bbf700 1 --2- 172.21.15.7:0/1293023099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94157d80 0x7fbd94063230 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbd9406d6b0 tx=0x7fbd84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e40da2fee806c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd993be700 1 -- 172.21.15.7:0/1293023099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400b820 con 0x7fbd94157d80 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd993be700 1 -- 172.21.15.7:0/1293023099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8400b980 con 0x7fbd94157d80 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.661+0000 7fbd993be700 1 -- 172.21.15.7:0/1293023099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400bd20 con 0x7fbd94157d80 2024-01-11T07:13:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.662+0000 7fbd9c624700 1 -- 172.21.15.7:0/1293023099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94157d80 msgr2=0x7fbd94063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:25.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.662+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1293023099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94157d80 0x7fbd94063230 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbd9406d6b0 tx=0x7fbd84009250 comp rx=0 tx=0).stop 2024-01-11T07:13:25.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.662+0000 7fbd9c624700 1 -- 172.21.15.7:0/1293023099 shutdown_connections 2024-01-11T07:13:25.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.662+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1293023099 >> v2:172.21.15.113:3300/0 conn(0x7fbd941579b0 0x7fbd9415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.662+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1293023099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94157d80 0x7fbd94063230 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.662+0000 7fbd9c624700 1 -- 172.21.15.7:0/1293023099 >> 172.21.15.7:0/1293023099 conn(0x7fbd940680e0 msgr2=0x7fbd940cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:25.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.662+0000 7fbd9c624700 1 -- 172.21.15.7:0/1293023099 shutdown_connections 2024-01-11T07:13:25.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.662+0000 7fbd9c624700 1 -- 172.21.15.7:0/1293023099 wait complete. 2024-01-11T07:13:25.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.663+0000 7fbd9c624700 1 Processor -- start 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.663+0000 7fbd9c624700 1 -- start start 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9c624700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94063770 0x7fbd941559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9c624700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd941579b0 0x7fbd940b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9c624700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd94157d80 0x7fbd9415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9c624700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b0ad0 con 0x7fbd94063770 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9c624700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94155ee0 con 0x7fbd941579b0 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9c624700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd94158b80 con 0x7fbd94157d80 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9a3c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94063770 0x7fbd941559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9a3c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94063770 0x7fbd941559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46138/0 (socket says 172.21.15.7:46138) 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9a3c0700 1 -- 172.21.15.7:0/1604257980 learned_addr learned my addr 172.21.15.7:0/1604257980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9abc1700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd94157d80 0x7fbd9415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9a3c0700 1 -- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd94157d80 msgr2=0x7fbd9415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9a3c0700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd94157d80 0x7fbd9415a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd99bbf700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd941579b0 0x7fbd940b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.664+0000 7fbd9a3c0700 1 -- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd941579b0 msgr2=0x7fbd940b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd9a3c0700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd941579b0 0x7fbd940b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd9a3c0700 1 -- 172.21.15.7:0/1604257980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd840094f0 con 0x7fbd94063770 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd99bbf700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd941579b0 0x7fbd940b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd9a3c0700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94063770 0x7fbd941559a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000d890 tx=0x7fbd90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd8a7fc700 1 -- 172.21.15.7:0/1604257980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000b600 con 0x7fbd94063770 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd8a7fc700 1 -- 172.21.15.7:0/1604257980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9000b760 con 0x7fbd94063770 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd94158e60 con 0x7fbd94063770 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd8a7fc700 1 -- 172.21.15.7:0/1604257980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000bae0 con 0x7fbd94063770 2024-01-11T07:13:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.665+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd94159420 con 0x7fbd94063770 2024-01-11T07:13:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.666+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9404c3c0 con 0x7fbd94063770 2024-01-11T07:13:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.666+0000 7fbd8a7fc700 1 -- 172.21.15.7:0/1604257980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbd9001e460 con 0x7fbd94063770 2024-01-11T07:13:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.667+0000 7fbd8a7fc700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd78066f30 0x7fbd780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.667+0000 7fbd8a7fc700 1 -- 172.21.15.7:0/1604257980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbd90084be0 con 0x7fbd94063770 2024-01-11T07:13:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.667+0000 7fbd99bbf700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd78066f30 0x7fbd780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.668+0000 7fbd99bbf700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd78066f30 0x7fbd780693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbd9406d6b0 tx=0x7fbd84016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:25.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.671+0000 7fbd8a7fc700 1 -- 172.21.15.7:0/1604257980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd90051e50 con 0x7fbd94063770 2024-01-11T07:13:25.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.924+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbd940ce120 con 0x7fbd94063770 2024-01-11T07:13:25.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.925+0000 7fbd8a7fc700 1 -- 172.21.15.7:0/1604257980 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbd90055470 con 0x7fbd94063770 2024-01-11T07:13:25.925 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:13:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd78066f30 msgr2=0x7fbd780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd78066f30 0x7fbd780693e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbd9406d6b0 tx=0x7fbd84016000 comp rx=0 tx=0).stop 2024-01-11T07:13:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94063770 msgr2=0x7fbd941559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94063770 0x7fbd941559a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000d890 tx=0x7fbd90008040 comp rx=0 tx=0).stop 2024-01-11T07:13:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 shutdown_connections 2024-01-11T07:13:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd78066f30 0x7fbd780693e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbd94157d80 0x7fbd9415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd94063770 0x7fbd941559a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 --2- 172.21.15.7:0/1604257980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd941579b0 0x7fbd940b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:25.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 >> 172.21.15.7:0/1604257980 conn(0x7fbd940680e0 msgr2=0x7fbd940c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:25.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 shutdown_connections 2024-01-11T07:13:25.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:25.927+0000 7fbd9c624700 1 -- 172.21.15.7:0/1604257980 wait complete. 2024-01-11T07:13:25.940 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:13:25.940 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:13:25.940 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:13:25.941 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:13:25.941 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:13:25.941 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:13:25.941 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:13:25.941 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:13:25.941 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:13:25.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:25.941 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:13:25.963 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:13:25.963 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:13:25.963 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:13:25.963 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:25.963 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42054sec preferred_lft 42054sec 2024-01-11T07:13:25.964 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:13:25.965 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:13:25.966 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:13:25.966 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:13:25.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:25.966 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:13:26.033 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:13:26.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:26.035 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:13:26.096 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:13:26.096 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:13:26.096 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:13:26.096 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:13:26.096 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:13:26.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:26.097 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:13:26.097 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:13:26.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:13:26.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:13:26.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:13:26.097 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:13:26.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:26.208 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:13:26.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:13:26.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:13:26.227 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:13:26.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:26.227 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:13:26.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:13:26.295 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:13:26.352 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:13:26.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:13:26.353 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:13:26.409 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:13:26.410 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:13:26.410 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:13:26.492 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:13:26.492 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:13:26.492 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:13:26.500 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:13:26.500 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:13:26.500 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:13:26.500 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:13:26.500 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:13:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:13:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:13:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:13:26.519 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:13:26.520 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:13:26.520 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:13:26.520 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:13:26.520 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:13:26.522 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:13:26.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:26.522 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:13:26.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:26.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.730+0000 7f212d950700 1 Processor -- start 2024-01-11T07:13:26.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.730+0000 7f212d950700 1 -- start start 2024-01-11T07:13:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f212d950700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f212806ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f212d950700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f212806f1d0 0x7f21280b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:26.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f212d950700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f212815dbb0 con 0x7f2128159d10 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f212d950700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212815d9a0 con 0x7f212806f1d0 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f212d950700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212806f650 con 0x7f21280693a0 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f2126ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2128159d10 0x7f2128068c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59736/0 (socket says 172.21.15.7:59736) 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f2126ffd700 1 -- 172.21.15.7:0/3017965405 learned_addr learned my addr 172.21.15.7:0/3017965405 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f21267fc700 1 --2- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f212806ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f21277fe700 1 --2- 172.21.15.7:0/3017965405 >> v2:172.21.15.113:3300/0 conn(0x7f212806f1d0 0x7f21280b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f2125ffb700 1 -- 172.21.15.7:0/3017965405 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250483232 0 0) 0x7f212815dbb0 con 0x7f2128159d10 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f2125ffb700 1 -- 172.21.15.7:0/3017965405 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2110003570 con 0x7f2128159d10 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f21277fe700 1 -- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 msgr2=0x7f212806ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f21277fe700 1 --2- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f212806ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f21277fe700 1 -- 172.21.15.7:0/3017965405 >> v1:172.21.15.7:6789/0 conn(0x7f2128159d10 legacy=0x7f2128068c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f21277fe700 1 -- 172.21.15.7:0/3017965405 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212815de90 con 0x7f212806f1d0 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.731+0000 7f21267fc700 1 --2- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f212806ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:26.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f21277fe700 1 --2- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 0x7f21280b3b80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2120009e60 tx=0x7f2120009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1dbdb454ac46429 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:26.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f2125ffb700 1 -- 172.21.15.7:0/3017965405 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000e070 con 0x7f212806f1d0 2024-01-11T07:13:26.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f2125ffb700 1 -- 172.21.15.7:0/3017965405 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212000bab0 con 0x7f212806f1d0 2024-01-11T07:13:26.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f2125ffb700 1 -- 172.21.15.7:0/3017965405 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000be70 con 0x7f212806f1d0 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f212d950700 1 -- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 msgr2=0x7f21280b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f212d950700 1 --2- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 0x7f21280b3b80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2120009e60 tx=0x7f2120009780 comp rx=0 tx=0).stop 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f212d950700 1 -- 172.21.15.7:0/3017965405 shutdown_connections 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f212d950700 1 --2- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 0x7f21280b3b80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f212d950700 1 --2- 172.21.15.7:0/3017965405 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f212806ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f212d950700 1 -- 172.21.15.7:0/3017965405 >> 172.21.15.7:0/3017965405 conn(0x7f21280ce9f0 msgr2=0x7f21280cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f212d950700 1 -- 172.21.15.7:0/3017965405 shutdown_connections 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.732+0000 7f212d950700 1 -- 172.21.15.7:0/3017965405 wait complete. 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.734+0000 7f212d950700 1 Processor -- start 2024-01-11T07:13:26.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.734+0000 7f212d950700 1 -- start start 2024-01-11T07:13:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f212d950700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f21280aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f212d950700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 0x7f212815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f212d950700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2128159d10 0x7f21280c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f212d950700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280c39e0 con 0x7f2128159d10 2024-01-11T07:13:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f212d950700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280c3cc0 con 0x7f212806f1d0 2024-01-11T07:13:26.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f212d950700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21280c3fa0 con 0x7f21280693a0 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2128159d10 0x7f21280c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21267fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 0x7f212815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2128159d10 0x7f21280c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46148/0 (socket says 172.21.15.7:46148) 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21277fe700 1 -- 172.21.15.7:0/276604028 learned_addr learned my addr 172.21.15.7:0/276604028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f2126ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f21280aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21267fc700 1 -- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 msgr2=0x7f21280aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21267fc700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f21280aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21267fc700 1 -- 172.21.15.7:0/276604028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2128159d10 msgr2=0x7f21280c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21267fc700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2128159d10 0x7f21280c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21267fc700 1 -- 172.21.15.7:0/276604028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c009420 con 0x7f212806f1d0 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.735+0000 7f21267fc700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 0x7f212815ad40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f211c00ec80 tx=0x7f211c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.736+0000 7f210effd700 1 -- 172.21.15.7:0/276604028 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00c850 con 0x7f212806f1d0 2024-01-11T07:13:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.736+0000 7f210effd700 1 -- 172.21.15.7:0/276604028 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c00c9b0 con 0x7f212806f1d0 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.736+0000 7f210effd700 1 -- 172.21.15.7:0/276604028 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00cd70 con 0x7f212806f1d0 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.736+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2120009a20 con 0x7f212806f1d0 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.736+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21280c4560 con 0x7f212806f1d0 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.737+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2128069770 con 0x7f212806f1d0 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.737+0000 7f210effd700 1 -- 172.21.15.7:0/276604028 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f211c013680 con 0x7f212806f1d0 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.737+0000 7f210effd700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2114066f30 0x7f21140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.738+0000 7f210effd700 1 -- 172.21.15.7:0/276604028 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f211c0847f0 con 0x7f212806f1d0 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.738+0000 7f2126ffd700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2114066f30 0x7f21140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.738+0000 7f2126ffd700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2114066f30 0x7f21140693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2118007fd0 tx=0x7f2118009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:26.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.742+0000 7f210effd700 1 -- 172.21.15.7:0/276604028 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f211c0519c0 con 0x7f212806f1d0 2024-01-11T07:13:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.992+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f212804c3c0 con 0x7f212806f1d0 2024-01-11T07:13:26.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.992+0000 7f210effd700 1 -- 172.21.15.7:0/276604028 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f211c054fe0 con 0x7f212806f1d0 2024-01-11T07:13:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:26.992 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2114066f30 msgr2=0x7f21140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2114066f30 0x7f21140693e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2118007fd0 tx=0x7f2118009040 comp rx=0 tx=0).stop 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 msgr2=0x7f212815ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 0x7f212815ad40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f211c00ec80 tx=0x7f211c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 shutdown_connections 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2114066f30 0x7f21140693e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2128159d10 0x7f21280c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21280693a0 0x7f21280aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 --2- 172.21.15.7:0/276604028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f212806f1d0 0x7f212815ad40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 >> 172.21.15.7:0/276604028 conn(0x7f21280ce9f0 msgr2=0x7f2128159010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 shutdown_connections 2024-01-11T07:13:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:26.995+0000 7f212d950700 1 -- 172.21.15.7:0/276604028 wait complete. 2024-01-11T07:13:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:27.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:13:27.160 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":7068,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8536650720000001,"mdsmap_epoch":65,"osdmap_epoch":135,"osdmap_epoch_barrier":135,"uptime":14.847613782} 2024-01-11T07:13:27.161 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 7068, "mdsmap_epoch": 65, "osdmap_epoch": 135, "osdmap_epoch_barrier": 135, "rank_uptime": 8.853665072, "state": "up:active", "uptime": 14.847613782, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:13:27.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-11T07:13:27.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.298+0000 7fb8e4b1a700 1 Processor -- start 2024-01-11T07:13:27.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.298+0000 7fb8e4b1a700 1 -- start start 2024-01-11T07:13:27.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8e4b1a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e00acf80 0x7fb8e00c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8e4b1a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8e00c8580 0x7fb8e0069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8e4b1a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00ca060 con 0x7fb8e00acbb0 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8e4b1a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e015ad70 con 0x7fb8e00c8580 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8e4b1a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015b050 con 0x7fb8e00acf80 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8de59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8e00acbb0 0x7fb8e00c7e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59752/0 (socket says 172.21.15.7:59752) 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8de59c700 1 -- 172.21.15.7:0/736806498 learned_addr learned my addr 172.21.15.7:0/736806498 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8ddd9b700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e00acf80 0x7fb8e00c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8ded9d700 1 --2- 172.21.15.7:0/736806498 >> v2:172.21.15.113:3300/0 conn(0x7fb8e00c8580 0x7fb8e0069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20673728 0 0) 0x7fb8e00ca060 con 0x7fb8e00acbb0 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8c8003570 con 0x7fb8e00acbb0 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 975485436 0 0) 0x7fb8c8003570 con 0x7fb8e00acbb0 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.299+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e00acf80 msgr2=0x7fb8e00c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e00acf80 0x7fb8e00c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 >> v2:172.21.15.113:3300/0 conn(0x7fb8e00c8580 msgr2=0x7fb8e0069fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 --2- 172.21.15.7:0/736806498 >> v2:172.21.15.113:3300/0 conn(0x7fb8e00c8580 0x7fb8e0069fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e015b330 con 0x7fb8e00acbb0 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fb8d00015f0 con 0x7fb8e00acbb0 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 >> v1:172.21.15.7:6789/0 conn(0x7fb8e00acbb0 legacy=0x7fb8e00c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c8003a60 0x7fb8c8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e00acf80 0x7fb8c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c8004d50 0x7fb8c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015ad70 con 0x7fb8c8003a60 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015b050 con 0x7fb8c8004d50 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c800ba60 con 0x7fb8e00acf80 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8de59c700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c8003a60 0x7fb8c8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8ddd9b700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e00acf80 0x7fb8c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8ded9d700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c8004d50 0x7fb8c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8de59c700 1 -- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e00acf80 msgr2=0x7fb8c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8de59c700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e00acf80 0x7fb8c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8de59c700 1 -- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c8004d50 msgr2=0x7fb8c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8de59c700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8c8004d50 0x7fb8c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8de59c700 1 -- 172.21.15.7:0/736806498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8003570 con 0x7fb8c8003a60 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8de59c700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c8003a60 0x7fb8c8003e70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000c3c0 tx=0x7fb8d0007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51420d4149e1b362 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000cc30 con 0x7fb8c8003a60 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d000cdb0 con 0x7fb8c8003a60 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.300+0000 7fb8dd59a700 1 -- 172.21.15.7:0/736806498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0007320 con 0x7fb8c8003a60 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.301+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/736806498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c8003a60 msgr2=0x7fb8c8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.301+0000 7fb8e4b1a700 1 --2- 172.21.15.7:0/736806498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c8003a60 0x7fb8c8003e70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000c3c0 tx=0x7fb8d0007920 comp rx=0 tx=0).stop 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.301+0000 7fb8ded9d700 1 -- 172.21.15.7:0/736806498 reap_dead start 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.301+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/736806498 shutdown_connections 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.301+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/736806498 >> 172.21.15.7:0/736806498 conn(0x7fb8e0068080 msgr2=0x7fb8e00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.301+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/736806498 shutdown_connections 2024-01-11T07:13:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.301+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/736806498 wait complete. 2024-01-11T07:13:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.302+0000 7fb8e4b1a700 1 Processor -- start 2024-01-11T07:13:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8e4b1a700 1 -- start start 2024-01-11T07:13:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8e4b1a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c8003a60 0x7fb8e0168240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8e4b1a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e015dff0 0x7fb8e015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8e4b1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00b2b90 0x7fb8e015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8e4b1a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00b33c0 con 0x7fb8e00b2b90 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8e4b1a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e0168780 con 0x7fb8e015dff0 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8e4b1a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015db60 con 0x7fb8c8003a60 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8ded9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00b2b90 0x7fb8e015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8ded9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00b2b90 0x7fb8e015bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46166/0 (socket says 172.21.15.7:46166) 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.303+0000 7fb8ded9d700 1 -- 172.21.15.7:0/1729741096 learned_addr learned my addr 172.21.15.7:0/1729741096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ddd9b700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e015dff0 0x7fb8e015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8de59c700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c8003a60 0x7fb8e0168240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ded9d700 1 -- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c8003a60 msgr2=0x7fb8e0168240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ded9d700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c8003a60 0x7fb8e0168240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ded9d700 1 -- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e015dff0 msgr2=0x7fb8e015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ded9d700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e015dff0 0x7fb8e015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ded9d700 1 -- 172.21.15.7:0/1729741096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0155a90 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8de59c700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c8003a60 0x7fb8e0168240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ded9d700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00b2b90 0x7fb8e015bfe0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006d940 tx=0x7fb8d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ce7fc700 1 -- 172.21.15.7:0/1729741096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800e040 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ce7fc700 1 -- 172.21.15.7:0/1729741096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d800f040 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e00b30e0 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8ce7fc700 1 -- 172.21.15.7:0/1729741096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d800fe20 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.304+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e015d7f0 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.305+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e015d350 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.306+0000 7fb8ce7fc700 1 -- 172.21.15.7:0/1729741096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb8d8015640 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.306+0000 7fb8ce7fc700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c0066f30 0x7fb8c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.306+0000 7fb8ce7fc700 1 -- 172.21.15.7:0/1729741096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb8d8084d20 con 0x7fb8e00b2b90 2024-01-11T07:13:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.306+0000 7fb8de59c700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c0066f30 0x7fb8c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:27.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.307+0000 7fb8de59c700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c0066f30 0x7fb8c00693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0008fd0 tx=0x7fb8d0015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:27.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.311+0000 7fb8ce7fc700 1 -- 172.21.15.7:0/1729741096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8d8051f90 con 0x7fb8e00b2b90 2024-01-11T07:13:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:27.514+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fb8e004c3c0 con 0x7fb8e00b2b90 2024-01-11T07:13:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.398+0000 7fb8ce7fc700 1 -- 172.21.15.7:0/1729741096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v136) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb8d80555b0 con 0x7fb8e00b2b90 2024-01-11T07:13:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c0066f30 msgr2=0x7fb8c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c0066f30 0x7fb8c00693e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0008fd0 tx=0x7fb8d0015000 comp rx=0 tx=0).stop 2024-01-11T07:13:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00b2b90 msgr2=0x7fb8e015bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00b2b90 0x7fb8e015bfe0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb8e006d940 tx=0x7fb8d800b040 comp rx=0 tx=0).stop 2024-01-11T07:13:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 shutdown_connections 2024-01-11T07:13:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c0066f30 0x7fb8c00693e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00b2b90 0x7fb8e015bfe0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8c8003a60 0x7fb8e0168240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 --2- 172.21.15.7:0/1729741096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e015dff0 0x7fb8e015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.401+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 >> 172.21.15.7:0/1729741096 conn(0x7fb8e0068080 msgr2=0x7fb8e0156300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.402+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 shutdown_connections 2024-01-11T07:13:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.402+0000 7fb8e4b1a700 1 -- 172.21.15.7:0/1729741096 wait complete. 2024-01-11T07:13:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-11T07:13:28.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-11T07:13:28.569 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-11T07:13:28.569 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-11T07:13:28.569 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-01-11T07:13:28.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:28.570 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-01-11T07:13:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:16+0 records in 2024-01-11T07:13:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:16+0 records out 2024-01-11T07:13:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.101701 s, 165 MB/s 2024-01-11T07:13:28.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:28.695 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-01-11T07:13:28.832 INFO:teuthology.orchestra.run.smithi007.stderr:17+0 records in 2024-01-11T07:13:28.832 INFO:teuthology.orchestra.run.smithi007.stderr:17+0 records out 2024-01-11T07:13:28.832 INFO:teuthology.orchestra.run.smithi007.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.11619 s, 153 MB/s 2024-01-11T07:13:28.833 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-01-11T07:13:28.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:13:28.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.966+0000 7f645edce700 1 Processor -- start 2024-01-11T07:13:28.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.967+0000 7f645edce700 1 -- start start 2024-01-11T07:13:28.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.967+0000 7f645edce700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64580d0170 0x7f64580cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:28.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.967+0000 7f645edce700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645815acf0 con 0x7f64580cd6d0 2024-01-11T07:13:28.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.967+0000 7f645edce700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645815afd0 con 0x7f645815dfb0 2024-01-11T07:13:28.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.967+0000 7f645edce700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64580ce980 con 0x7f64580d0170 2024-01-11T07:13:28.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f6457fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64580d0170 0x7f64580cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f645cb6a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f64580cd6d0 0x7f645815d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43138/0 (socket says 172.21.15.7:43138) 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f645cb6a700 1 -- 172.21.15.7:0/2361217987 learned_addr learned my addr 172.21.15.7:0/2361217987 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f6457fff700 1 -- 172.21.15.7:0/2361217987 >> v1:172.21.15.113:6789/0 conn(0x7f645815dfb0 legacy=0x7f6458159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f6457fff700 1 -- 172.21.15.7:0/2361217987 >> v1:172.21.15.7:6789/0 conn(0x7f64580cd6d0 legacy=0x7f645815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f6457fff700 1 -- 172.21.15.7:0/2361217987 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580cec60 con 0x7f64580d0170 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f64577fe700 1 -- 172.21.15.7:0/2361217987 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937008861 0 0) 0x7f645815acf0 con 0x7f64580cd6d0 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f6457fff700 1 --2- 172.21.15.7:0/2361217987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580d0170 0x7f64580cdaa0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f64440098e0 tx=0x7f64440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48f6201709cd584 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f64577fe700 1 -- 172.21.15.7:0/2361217987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444009e80 con 0x7f64580d0170 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f64577fe700 1 -- 172.21.15.7:0/2361217987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644400b860 con 0x7f64580d0170 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.968+0000 7f64577fe700 1 -- 172.21.15.7:0/2361217987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644400bbe0 con 0x7f64580d0170 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.969+0000 7f645edce700 1 -- 172.21.15.7:0/2361217987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580d0170 msgr2=0x7f64580cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.969+0000 7f645edce700 1 --2- 172.21.15.7:0/2361217987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580d0170 0x7f64580cdaa0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f64440098e0 tx=0x7f64440091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.969+0000 7f645edce700 1 -- 172.21.15.7:0/2361217987 shutdown_connections 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.969+0000 7f645edce700 1 --2- 172.21.15.7:0/2361217987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580d0170 0x7f64580cdaa0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:28.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.969+0000 7f645edce700 1 -- 172.21.15.7:0/2361217987 >> 172.21.15.7:0/2361217987 conn(0x7f6458071120 msgr2=0x7f645806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:28.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.969+0000 7f645edce700 1 -- 172.21.15.7:0/2361217987 shutdown_connections 2024-01-11T07:13:28.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.969+0000 7f645edce700 1 -- 172.21.15.7:0/2361217987 wait complete. 2024-01-11T07:13:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.970+0000 7f645edce700 1 Processor -- start 2024-01-11T07:13:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.970+0000 7f645edce700 1 -- start start 2024-01-11T07:13:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645edce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580cd6d0 0x7f64580b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645edce700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64580d0170 0x7f645814e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645edce700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645815dfb0 0x7f64580b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645edce700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64580b41b0 con 0x7f64580cd6d0 2024-01-11T07:13:28.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645edce700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64580b67e0 con 0x7f64580d0170 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645edce700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6458150ff0 con 0x7f645815dfb0 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645cb6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580cd6d0 0x7f64580b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645cb6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580cd6d0 0x7f64580b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48800/0 (socket says 172.21.15.7:48800) 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.971+0000 7f645cb6a700 1 -- 172.21.15.7:0/609433445 learned_addr learned my addr 172.21.15.7:0/609433445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f6457fff700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64580d0170 0x7f645814e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645d36b700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645815dfb0 0x7f64580b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645cb6a700 1 -- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645815dfb0 msgr2=0x7f64580b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645cb6a700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645815dfb0 0x7f64580b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645cb6a700 1 -- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64580d0170 msgr2=0x7f645814e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645cb6a700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64580d0170 0x7f645814e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645cb6a700 1 -- 172.21.15.7:0/609433445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444009420 con 0x7f64580cd6d0 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645d36b700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645815dfb0 0x7f64580b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:28.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645cb6a700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580cd6d0 0x7f64580b62a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c810 tx=0x7f644c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f6454ff9700 1 -- 172.21.15.7:0/609433445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c005c00 con 0x7f64580cd6d0 2024-01-11T07:13:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f6454ff9700 1 -- 172.21.15.7:0/609433445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c005d60 con 0x7f64580cd6d0 2024-01-11T07:13:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f6454ff9700 1 -- 172.21.15.7:0/609433445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c007340 con 0x7f64580cd6d0 2024-01-11T07:13:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64581512d0 con 0x7f64580cd6d0 2024-01-11T07:13:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.972+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645814e830 con 0x7f64580cd6d0 2024-01-11T07:13:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.973+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64581514c0 con 0x7f64580cd6d0 2024-01-11T07:13:28.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.974+0000 7f6454ff9700 1 -- 172.21.15.7:0/609433445 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f644c01b400 con 0x7f64580cd6d0 2024-01-11T07:13:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.974+0000 7f6454ff9700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f643c066f30 0x7f643c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.974+0000 7f6454ff9700 1 -- 172.21.15.7:0/609433445 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f644c083fe0 con 0x7f64580cd6d0 2024-01-11T07:13:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.974+0000 7f6457fff700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f643c066f30 0x7f643c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.975+0000 7f6457fff700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f643c066f30 0x7f643c0693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64440093f0 tx=0x7f6444017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:28.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:28.979+0000 7f6454ff9700 1 -- 172.21.15.7:0/609433445 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f644c0512d0 con 0x7f64580cd6d0 2024-01-11T07:13:29.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.187+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f645804c3c0 con 0x7f64580cd6d0 2024-01-11T07:13:29.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.188+0000 7f6454ff9700 1 -- 172.21.15.7:0/609433445 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v136) v1 ==== 81+0+32234 (secure 0 0 0) 0x7f644c0548f0 con 0x7f64580cd6d0 2024-01-11T07:13:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:29.188 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 136, 2024-01-11T07:13:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:13:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:13:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:13:28.389948+0000", 2024-01-11T07:13:29.188 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:29.189 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:13:29.190 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:29.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:13:29.192 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:29.193 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:29.194 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:29.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:13:29.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:15.625657+0000", 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:29.197 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:29.198 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:29.201 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:29.202 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:13:29.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:13:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:13:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:13:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-11T07:13:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:13:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:16.691544+0000", 2024-01-11T07:13:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:29.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:13:29.206 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "136", 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:29.207 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:29.208 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:29.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:29.211 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:13:29.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:29.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:29.216 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:29.217 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:13:29.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:13:29.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:13:29.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.222 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:29.223 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.224 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.225 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.227 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:29.228 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:29.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:29.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:29.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:13:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:13:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:13:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:13:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:29.234 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:29.235 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:29.236 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:13:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:13:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:29.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:13:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:13:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:13:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:29.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:13:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:13:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:13:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:29.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:13:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:13:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:13:29.246 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:13:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:29.248 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:29.249 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:13:29.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:13:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:29.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:13:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:13:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.254 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:13:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:29.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:13:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:13:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:13:29.258 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:13:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:13:29.259 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:13:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:13:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:29.261 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:29.262 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:13:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:13:29.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:13:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:13:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.267 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:13:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:29.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:13:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:13:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:29.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:13:29.271 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:29.272 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:13:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:29.280 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:13:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:29.282 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:13:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:29.283 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:13:29.284 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:29.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:13:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:29.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:29.287 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:13:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:29.288 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:29.289 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:13:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:29.290 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:13:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:29.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:29.292 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:13:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:29.293 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:13:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:13:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:13:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:13:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:13:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:13:29.294 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:13:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:13:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:13:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:13:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:13:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:13:29.295 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:13:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:13:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:13:29.296 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:13:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:13:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:13:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:13:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:13:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:13:29.297 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:13:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:13:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:13:29.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:29.298 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f643c066f30 msgr2=0x7f643c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f643c066f30 0x7f643c0693e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64440093f0 tx=0x7f6444017000 comp rx=0 tx=0).stop 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580cd6d0 msgr2=0x7f64580b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580cd6d0 0x7f64580b62a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c810 tx=0x7f644c007d90 comp rx=0 tx=0).stop 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 shutdown_connections 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f643c066f30 0x7f643c0693e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645815dfb0 0x7f64580b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64580cd6d0 0x7f64580b62a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 --2- 172.21.15.7:0/609433445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64580d0170 0x7f645814e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:29.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 >> 172.21.15.7:0/609433445 conn(0x7f6458071120 msgr2=0x7f64580c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:29.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 shutdown_connections 2024-01-11T07:13:29.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:29.190+0000 7f645edce700 1 -- 172.21.15.7:0/609433445 wait complete. 2024-01-11T07:13:29.301 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:13:31.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:13:31.222 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:13:34.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:13:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.445+0000 7f98b1fc3700 1 Processor -- start 2024-01-11T07:13:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.445+0000 7f98b1fc3700 1 -- start start 2024-01-11T07:13:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98b1fc3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac159ba0 0x7f98ac06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98b1fc3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac067c60 con 0x7f98ac159f70 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98b1fc3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac069230 con 0x7f98ac0680f0 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98b1fc3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac069510 con 0x7f98ac159ba0 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98aaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f98ac159f70 0x7f98ac0c9030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43140/0 (socket says 172.21.15.7:43140) 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98aaffd700 1 -- 172.21.15.7:0/3309508499 learned_addr learned my addr 172.21.15.7:0/3309508499 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98ab7fe700 1 --2- 172.21.15.7:0/3309508499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac159ba0 0x7f98ac06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98aa7fc700 1 -- 172.21.15.7:0/3309508499 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858741927 0 0) 0x7f98ac067c60 con 0x7f98ac159f70 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98aa7fc700 1 -- 172.21.15.7:0/3309508499 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9894003570 con 0x7f98ac159f70 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.446+0000 7f98aa7fc700 1 -- 172.21.15.7:0/3309508499 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305326085 0 0) 0x7f98ac069230 con 0x7f98ac0680f0 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98aa7fc700 1 -- 172.21.15.7:0/3309508499 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98ac067c60 con 0x7f98ac0680f0 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98aa7fc700 1 -- 172.21.15.7:0/3309508499 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3054440375 0 0) 0x7f9894003570 con 0x7f98ac159f70 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98ab7fe700 1 -- 172.21.15.7:0/3309508499 >> v1:172.21.15.113:6789/0 conn(0x7f98ac0680f0 legacy=0x7f98ac0cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98ab7fe700 1 -- 172.21.15.7:0/3309508499 >> v1:172.21.15.7:6789/0 conn(0x7f98ac159f70 legacy=0x7f98ac0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98ab7fe700 1 -- 172.21.15.7:0/3309508499 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac067610 con 0x7f98ac159ba0 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98ab7fe700 1 --2- 172.21.15.7:0/3309508499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac159ba0 0x7f98ac06d020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f989c009910 tx=0x7f989c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9c925c3027a6eb7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98aa7fc700 1 -- 172.21.15.7:0/3309508499 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0001470 con 0x7f98ac159ba0 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98aa7fc700 1 -- 172.21.15.7:0/3309508499 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c011420 con 0x7f98ac159ba0 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.447+0000 7f98aa7fc700 1 -- 172.21.15.7:0/3309508499 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c0117c0 con 0x7f98ac159ba0 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.448+0000 7f98b1fc3700 1 -- 172.21.15.7:0/3309508499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac159ba0 msgr2=0x7f98ac06d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.448+0000 7f98b1fc3700 1 --2- 172.21.15.7:0/3309508499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac159ba0 0x7f98ac06d020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f989c009910 tx=0x7f989c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.448+0000 7f98b1fc3700 1 -- 172.21.15.7:0/3309508499 shutdown_connections 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.448+0000 7f98b1fc3700 1 --2- 172.21.15.7:0/3309508499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac159ba0 0x7f98ac06d020 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.448+0000 7f98b1fc3700 1 -- 172.21.15.7:0/3309508499 >> 172.21.15.7:0/3309508499 conn(0x7f98ac06ae00 msgr2=0x7f98ac06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.448+0000 7f98b1fc3700 1 -- 172.21.15.7:0/3309508499 shutdown_connections 2024-01-11T07:13:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.448+0000 7f98b1fc3700 1 -- 172.21.15.7:0/3309508499 wait complete. 2024-01-11T07:13:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.449+0000 7f98b1fc3700 1 Processor -- start 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.449+0000 7f98b1fc3700 1 -- start start 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98b1fc3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac0680f0 0x7f98ac158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98b1fc3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98ac159ba0 0x7f98ac15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98b1fc3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac159f70 0x7f98ac15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98b1fc3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac154f00 con 0x7f98ac159f70 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98b1fc3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac155ad0 con 0x7f98ac159ba0 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98b1fc3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c5b40 con 0x7f98ac0680f0 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac159f70 0x7f98ac15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98abfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac159f70 0x7f98ac15a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48812/0 (socket says 172.21.15.7:48812) 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98abfff700 1 -- 172.21.15.7:0/2968761417 learned_addr learned my addr 172.21.15.7:0/2968761417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98ab7fe700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac0680f0 0x7f98ac158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98aaffd700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98ac159ba0 0x7f98ac15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98abfff700 1 -- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac0680f0 msgr2=0x7f98ac158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98abfff700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac0680f0 0x7f98ac158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98abfff700 1 -- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98ac159ba0 msgr2=0x7f98ac15c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98abfff700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98ac159ba0 0x7f98ac15c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.450+0000 7f98abfff700 1 -- 172.21.15.7:0/2968761417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009420 con 0x7f98ac159f70 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.451+0000 7f98abfff700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac159f70 0x7f98ac15a3a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c910 tx=0x7f98a40027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.451+0000 7f98937fe700 1 -- 172.21.15.7:0/2968761417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4012630 con 0x7f98ac159f70 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.451+0000 7f98937fe700 1 -- 172.21.15.7:0/2968761417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a40127b0 con 0x7f98ac159f70 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.451+0000 7f98937fe700 1 -- 172.21.15.7:0/2968761417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a4012b70 con 0x7f98ac159f70 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.451+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac0c5e20 con 0x7f98ac159f70 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.451+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac0c63e0 con 0x7f98ac159f70 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.452+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac0c65a0 con 0x7f98ac159f70 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.452+0000 7f98937fe700 1 -- 172.21.15.7:0/2968761417 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f98a401b400 con 0x7f98ac159f70 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.453+0000 7f98937fe700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9898066f30 0x7f98980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.453+0000 7f98937fe700 1 -- 172.21.15.7:0/2968761417 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f98a40848f0 con 0x7f98ac159f70 2024-01-11T07:13:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.453+0000 7f98ab7fe700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9898066f30 0x7f98980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.453+0000 7f98937fe700 1 -- 172.21.15.7:0/2968761417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=138}) v3 -- 0x7f989806ea50 con 0x7f98ac159f70 2024-01-11T07:13:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.454+0000 7f98ab7fe700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9898066f30 0x7f98980693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f989c0030c0 tx=0x7f989c016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.457+0000 7f98937fe700 1 -- 172.21.15.7:0/2968761417 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98a4016020 con 0x7f98ac159f70 2024-01-11T07:13:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.660+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f98ac04c3c0 con 0x7f98ac159f70 2024-01-11T07:13:34.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.661+0000 7f98937fe700 1 -- 172.21.15.7:0/2968761417 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f98a4051c10 con 0x7f98ac159f70 2024-01-11T07:13:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:34.662 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 137, 2024-01-11T07:13:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:13:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:13:32.234651+0000", 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:13:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:13:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:13:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:13:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:15.625657+0000", 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:13:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:16.691544+0000", 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "137", 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:13:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:13:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:13:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:13:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:13:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:13:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:13:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:13:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:13:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:13:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:13:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:13:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:13:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:13:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:13:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:13:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:13:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:13:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:13:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:34.732 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9898066f30 msgr2=0x7f98980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9898066f30 0x7f98980693e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f989c0030c0 tx=0x7f989c016000 comp rx=0 tx=0).stop 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac159f70 msgr2=0x7f98ac15a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac159f70 0x7f98ac15a3a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c910 tx=0x7f98a40027a0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 shutdown_connections 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9898066f30 0x7f98980693e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac159f70 0x7f98ac15a3a0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98ac0680f0 0x7f98ac158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 --2- 172.21.15.7:0/2968761417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98ac159ba0 0x7f98ac15c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.663+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 >> 172.21.15.7:0/2968761417 conn(0x7f98ac06ae00 msgr2=0x7f98ac0cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:34.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.664+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 shutdown_connections 2024-01-11T07:13:34.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:34.664+0000 7f98b1fc3700 1 -- 172.21.15.7:0/2968761417 wait complete. 2024-01-11T07:13:34.735 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-01-11T07:13:34.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:34.735 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-01-11T07:13:35.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:13:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-01-11T07:13:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-11T07:13:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-11T07:13:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.259316 s, 202 MB/s 2024-01-11T07:13:35.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:13:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.166+0000 7fbf38d48700 1 Processor -- start 2024-01-11T07:13:35.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.166+0000 7fbf38d48700 1 -- start start 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf38d48700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf34159c90 0x7fbf340b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf38d48700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf3415a060 0x7fbf340b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf38d48700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf34069320 0x7fbf340b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf38d48700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf340ce9b0 con 0x7fbf34159c90 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf38d48700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf340cf530 con 0x7fbf3415a060 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf38d48700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340cf9c0 con 0x7fbf34069320 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf3259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf34159c90 0x7fbf340b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf3259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf34159c90 0x7fbf340b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48820/0 (socket says 172.21.15.7:48820) 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf3259c700 1 -- 172.21.15.7:0/1123345964 learned_addr learned my addr 172.21.15.7:0/1123345964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf31d9b700 1 --2- 172.21.15.7:0/1123345964 >> v2:172.21.15.113:3300/0 conn(0x7fbf3415a060 0x7fbf340b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.167+0000 7fbf32d9d700 1 --2- 172.21.15.7:0/1123345964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf34069320 0x7fbf340b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3259c700 1 -- 172.21.15.7:0/1123345964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf34069320 msgr2=0x7fbf340b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3259c700 1 --2- 172.21.15.7:0/1123345964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf34069320 0x7fbf340b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3259c700 1 -- 172.21.15.7:0/1123345964 >> v2:172.21.15.113:3300/0 conn(0x7fbf3415a060 msgr2=0x7fbf340b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3259c700 1 --2- 172.21.15.7:0/1123345964 >> v2:172.21.15.113:3300/0 conn(0x7fbf3415a060 0x7fbf340b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3259c700 1 -- 172.21.15.7:0/1123345964 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340cec00 con 0x7fbf34159c90 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf31d9b700 1 --2- 172.21.15.7:0/1123345964 >> v2:172.21.15.113:3300/0 conn(0x7fbf3415a060 0x7fbf340b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3259c700 1 --2- 172.21.15.7:0/1123345964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34159c90 0x7fbf340b1820 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fbf340699e0 tx=0x7fbf1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac476352549a434 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3159a700 1 -- 172.21.15.7:0/1123345964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00b860 con 0x7fbf34159c90 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3159a700 1 -- 172.21.15.7:0/1123345964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c00b9c0 con 0x7fbf34159c90 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf3159a700 1 -- 172.21.15.7:0/1123345964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00bd60 con 0x7fbf34159c90 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf38d48700 1 -- 172.21.15.7:0/1123345964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34159c90 msgr2=0x7fbf340b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.168+0000 7fbf38d48700 1 --2- 172.21.15.7:0/1123345964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34159c90 0x7fbf340b1820 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fbf340699e0 tx=0x7fbf1c009250 comp rx=0 tx=0).stop 2024-01-11T07:13:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.169+0000 7fbf38d48700 1 -- 172.21.15.7:0/1123345964 shutdown_connections 2024-01-11T07:13:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.169+0000 7fbf38d48700 1 --2- 172.21.15.7:0/1123345964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34159c90 0x7fbf340b1820 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.169+0000 7fbf38d48700 1 --2- 172.21.15.7:0/1123345964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf34069320 0x7fbf340b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.169+0000 7fbf38d48700 1 --2- 172.21.15.7:0/1123345964 >> v2:172.21.15.113:3300/0 conn(0x7fbf3415a060 0x7fbf340b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.169+0000 7fbf38d48700 1 -- 172.21.15.7:0/1123345964 >> 172.21.15.7:0/1123345964 conn(0x7fbf34071100 msgr2=0x7fbf3406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.169+0000 7fbf38d48700 1 -- 172.21.15.7:0/1123345964 shutdown_connections 2024-01-11T07:13:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.169+0000 7fbf38d48700 1 -- 172.21.15.7:0/1123345964 wait complete. 2024-01-11T07:13:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.170+0000 7fbf38d48700 1 Processor -- start 2024-01-11T07:13:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.170+0000 7fbf38d48700 1 -- start start 2024-01-11T07:13:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf38d48700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34069320 0x7fbf341545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf38d48700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34159c90 0x7fbf341475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf38d48700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf3415a060 0x7fbf340af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf38d48700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34147ae0 con 0x7fbf34069320 2024-01-11T07:13:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf38d48700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340b08c0 con 0x7fbf34159c90 2024-01-11T07:13:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf38d48700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340b0ba0 con 0x7fbf3415a060 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34069320 0x7fbf341545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34069320 0x7fbf341545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48832/0 (socket says 172.21.15.7:48832) 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 -- 172.21.15.7:0/3287965540 learned_addr learned my addr 172.21.15.7:0/3287965540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf31d9b700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34159c90 0x7fbf341475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf32d9d700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf3415a060 0x7fbf340af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 -- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf3415a060 msgr2=0x7fbf340af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf3415a060 0x7fbf340af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 -- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34159c90 msgr2=0x7fbf341475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34159c90 0x7fbf341475a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 -- 172.21.15.7:0/3287965540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0094f0 con 0x7fbf34069320 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf3259c700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34069320 0x7fbf341545a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009290 tx=0x7fbf1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf2a7fc700 1 -- 172.21.15.7:0/3287965540 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c011e40 con 0x7fbf34069320 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf2a7fc700 1 -- 172.21.15.7:0/3287965540 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c01f070 con 0x7fbf34069320 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf340c88c0 con 0x7fbf34069320 2024-01-11T07:13:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.171+0000 7fbf2a7fc700 1 -- 172.21.15.7:0/3287965540 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0266d0 con 0x7fbf34069320 2024-01-11T07:13:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.172+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf340b0400 con 0x7fbf34069320 2024-01-11T07:13:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.173+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3404c3c0 con 0x7fbf34069320 2024-01-11T07:13:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.173+0000 7fbf2a7fc700 1 -- 172.21.15.7:0/3287965540 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbf1c026a40 con 0x7fbf34069320 2024-01-11T07:13:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.173+0000 7fbf2a7fc700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbf14066f30 0x7fbf140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.173+0000 7fbf2a7fc700 1 -- 172.21.15.7:0/3287965540 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbf1c021070 con 0x7fbf34069320 2024-01-11T07:13:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.174+0000 7fbf31d9b700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbf14066f30 0x7fbf140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.174+0000 7fbf2a7fc700 1 -- 172.21.15.7:0/3287965540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=138}) v3 -- 0x7fbf1406ea50 con 0x7fbf34069320 2024-01-11T07:13:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.174+0000 7fbf31d9b700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbf14066f30 0x7fbf140693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbf34154f10 tx=0x7fbf24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.178+0000 7fbf2a7fc700 1 -- 172.21.15.7:0/3287965540 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf1c015060 con 0x7fbf34069320 2024-01-11T07:13:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.383+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbf340ae830 con 0x7fbf34069320 2024-01-11T07:13:35.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.384+0000 7fbf2a7fc700 1 -- 172.21.15.7:0/3287965540 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+32253 (secure 0 0 0) 0x7fbf1c0534f0 con 0x7fbf34069320 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 137, 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:13:32.234651+0000", 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:13:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:13:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:35.388 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:35.388 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:35.388 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:35.389 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:35.390 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:35.391 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:15.625657+0000", 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:35.392 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:35.393 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:35.394 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:35.395 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-11T07:13:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:13:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:16.691544+0000", 2024-01-11T07:13:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-11T07:13:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-11T07:13:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:13:35.399 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "137", 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:35.400 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:35.402 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:35.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:35.404 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:35.410 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:35.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:13:35.414 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:35.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:13:35.416 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:13:35.417 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:13:35.418 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:35.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:13:35.425 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:35.426 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:13:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:13:35.435 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:35.436 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:35.437 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:35.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:35.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:35.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:35.442 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:35.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:13:35.447 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:35.448 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.449 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:35.450 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.451 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:35.452 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.453 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:13:35.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:35.455 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbf14066f30 msgr2=0x7fbf140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbf14066f30 0x7fbf140693e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbf34154f10 tx=0x7fbf24008040 comp rx=0 tx=0).stop 2024-01-11T07:13:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34069320 msgr2=0x7fbf341545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34069320 0x7fbf341545a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009290 tx=0x7fbf1c017000 comp rx=0 tx=0).stop 2024-01-11T07:13:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 shutdown_connections 2024-01-11T07:13:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbf14066f30 0x7fbf140693e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf3415a060 0x7fbf340af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf34069320 0x7fbf341545a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 --2- 172.21.15.7:0/3287965540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf34159c90 0x7fbf341475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 >> 172.21.15.7:0/3287965540 conn(0x7fbf34071100 msgr2=0x7fbf340b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 shutdown_connections 2024-01-11T07:13:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.386+0000 7fbf38d48700 1 -- 172.21.15.7:0/3287965540 wait complete. 2024-01-11T07:13:35.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.593+0000 7f6c062f6700 1 Processor -- start 2024-01-11T07:13:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.593+0000 7f6c062f6700 1 -- start start 2024-01-11T07:13:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.594+0000 7f6c062f6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c00159c20 0x7f6c0015af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.594+0000 7f6c062f6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0015aa80 con 0x7f6c00063940 2024-01-11T07:13:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.594+0000 7f6c062f6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c000ad010 con 0x7f6c000ac450 2024-01-11T07:13:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.594+0000 7f6c062f6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c000ad2f0 con 0x7f6c00159c20 2024-01-11T07:13:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.594+0000 7f6c04893700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6c00063940 0x7f6c0015d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43154/0 (socket says 172.21.15.7:43154) 2024-01-11T07:13:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.594+0000 7f6c04893700 1 -- 172.21.15.7:0/2741212861 learned_addr learned my addr 172.21.15.7:0/2741212861 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfffff700 1 --2- 172.21.15.7:0/2741212861 >> v2:172.21.15.113:3300/0 conn(0x7f6c00159c20 0x7f6c0015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfeffd700 1 -- 172.21.15.7:0/2741212861 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712746132 0 0) 0x7f6c0015aa80 con 0x7f6c00063940 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfeffd700 1 -- 172.21.15.7:0/2741212861 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6be8003570 con 0x7f6c00063940 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfeffd700 1 -- 172.21.15.7:0/2741212861 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205498782 0 0) 0x7f6c000ad010 con 0x7f6c000ac450 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfeffd700 1 -- 172.21.15.7:0/2741212861 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c0015aa80 con 0x7f6c000ac450 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfffff700 1 -- 172.21.15.7:0/2741212861 >> v1:172.21.15.113:6789/0 conn(0x7f6c000ac450 legacy=0x7f6c00063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfffff700 1 -- 172.21.15.7:0/2741212861 >> v1:172.21.15.7:6789/0 conn(0x7f6c00063940 legacy=0x7f6c0015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfeffd700 1 -- 172.21.15.7:0/2741212861 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2329968911 0 0) 0x7f6be8003570 con 0x7f6c00063940 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfffff700 1 -- 172.21.15.7:0/2741212861 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0015a430 con 0x7f6c00159c20 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfffff700 1 --2- 172.21.15.7:0/2741212861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c00159c20 0x7f6c0015af10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00098e0 tx=0x7f6bf00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f441f125d979f3ab server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.595+0000 7f6bfeffd700 1 -- 172.21.15.7:0/2741212861 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80017f0 con 0x7f6c00159c20 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6bfeffd700 1 -- 172.21.15.7:0/2741212861 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf0011420 con 0x7f6c00159c20 2024-01-11T07:13:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6bfeffd700 1 -- 172.21.15.7:0/2741212861 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf00117c0 con 0x7f6c00159c20 2024-01-11T07:13:35.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6c062f6700 1 -- 172.21.15.7:0/2741212861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c00159c20 msgr2=0x7f6c0015af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6c062f6700 1 --2- 172.21.15.7:0/2741212861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c00159c20 0x7f6c0015af10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00098e0 tx=0x7f6bf00091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6c062f6700 1 -- 172.21.15.7:0/2741212861 shutdown_connections 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6c062f6700 1 --2- 172.21.15.7:0/2741212861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c00159c20 0x7f6c0015af10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6c062f6700 1 -- 172.21.15.7:0/2741212861 >> 172.21.15.7:0/2741212861 conn(0x7f6c000cd910 msgr2=0x7f6c000cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6c062f6700 1 -- 172.21.15.7:0/2741212861 shutdown_connections 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.596+0000 7f6c062f6700 1 -- 172.21.15.7:0/2741212861 wait complete. 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.597+0000 7f6c062f6700 1 Processor -- start 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.598+0000 7f6c062f6700 1 -- start start 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.598+0000 7f6c062f6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c00063940 0x7f6c00145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.598+0000 7f6c062f6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c000ac450 0x7f6c000aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.598+0000 7f6c062f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c00159c20 0x7f6c000af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.598+0000 7f6c062f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0015bf80 con 0x7f6c00159c20 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.598+0000 7f6c062f6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c00146410 con 0x7f6c000ac450 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.598+0000 7f6c062f6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c000c3540 con 0x7f6c00063940 2024-01-11T07:13:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c00159c20 0x7f6c000af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c00159c20 0x7f6c000af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48848/0 (socket says 172.21.15.7:48848) 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 -- 172.21.15.7:0/3140908132 learned_addr learned my addr 172.21.15.7:0/3140908132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6bfffff700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c00063940 0x7f6c00145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6bff7fe700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c000ac450 0x7f6c000aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 -- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c00063940 msgr2=0x7f6c00145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c00063940 0x7f6c00145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 -- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c000ac450 msgr2=0x7f6c000aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c000ac450 0x7f6c000aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 -- 172.21.15.7:0/3140908132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf0009420 con 0x7f6c00159c20 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6bff7fe700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c000ac450 0x7f6c000aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c04893700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c00159c20 0x7f6c000af1f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800ccc0 tx=0x7f6bf80018e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6be7fff700 1 -- 172.21.15.7:0/3140908132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8007e70 con 0x7f6c00159c20 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0015b650 con 0x7f6c00159c20 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6be7fff700 1 -- 172.21.15.7:0/3140908132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf8004dc0 con 0x7f6c00159c20 2024-01-11T07:13:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6be7fff700 1 -- 172.21.15.7:0/3140908132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800ab50 con 0x7f6c00159c20 2024-01-11T07:13:35.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.599+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0015bc10 con 0x7f6c00159c20 2024-01-11T07:13:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.600+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0015c130 con 0x7f6c00159c20 2024-01-11T07:13:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.601+0000 7f6be7fff700 1 -- 172.21.15.7:0/3140908132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6bf8007150 con 0x7f6c00159c20 2024-01-11T07:13:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.602+0000 7f6be7fff700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6bec066f30 0x7f6bec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.602+0000 7f6be7fff700 1 -- 172.21.15.7:0/3140908132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6bf8083e30 con 0x7f6c00159c20 2024-01-11T07:13:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.602+0000 7f6be7fff700 1 -- 172.21.15.7:0/3140908132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=138}) v3 -- 0x7f6bec06ea50 con 0x7f6c00159c20 2024-01-11T07:13:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.602+0000 7f6bfffff700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6bec066f30 0x7f6bec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.602+0000 7f6bfffff700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6bec066f30 0x7f6bec0693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00093f0 tx=0x7f6bf0016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:35.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.605+0000 7f6be7fff700 1 -- 172.21.15.7:0/3140908132 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bf8018060 con 0x7f6c00159c20 2024-01-11T07:13:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.855+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c0004c3c0 con 0x7f6c00159c20 2024-01-11T07:13:35.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.855+0000 7f6be7fff700 1 -- 172.21.15.7:0/3140908132 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f6bf8051120 con 0x7f6c00159c20 2024-01-11T07:13:35.855 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:35.855 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.857+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6bec066f30 msgr2=0x7f6bec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.857+0000 7f6c062f6700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6bec066f30 0x7f6bec0693e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00093f0 tx=0x7f6bf0016000 comp rx=0 tx=0).stop 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.857+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c00159c20 msgr2=0x7f6c000af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c00159c20 0x7f6c000af1f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800ccc0 tx=0x7f6bf80018e0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 shutdown_connections 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6bec066f30 0x7f6bec0693e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c00159c20 0x7f6c000af1f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c00063940 0x7f6c00145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 --2- 172.21.15.7:0/3140908132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c000ac450 0x7f6c000aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 >> 172.21.15.7:0/3140908132 conn(0x7f6c000cd910 msgr2=0x7f6c00159150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 shutdown_connections 2024-01-11T07:13:35.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:35.858+0000 7f6c062f6700 1 -- 172.21.15.7:0/3140908132 wait complete. 2024-01-11T07:13:35.859 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:35.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:13:36.023 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":7068,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.7165085,"mdsmap_epoch":65,"osdmap_epoch":137,"osdmap_epoch_barrier":137,"uptime":23.710458174999999} 2024-01-11T07:13:36.024 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 7068, "mdsmap_epoch": 65, "osdmap_epoch": 137, "osdmap_epoch_barrier": 137, "rank_uptime": 17.7165085, "state": "up:active", "uptime": 23.710458175, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:13:36.024 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:13:36.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:36.024 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-01-11T07:13:36.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:36.045 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-01-11T07:13:36.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:13:36.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.289+0000 7f4d69008700 1 Processor -- start 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.289+0000 7f4d69008700 1 -- start start 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d69008700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4d640cd6d0 0x7f4d640cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d69008700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d6415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d69008700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6415e6c0 con 0x7f4d6415e280 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d69008700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6415acf0 con 0x7f4d640cd6d0 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d69008700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6415afd0 con 0x7f4d6415d8a0 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d6359e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4d6415e280 0x7f4d640cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43170/0 (socket says 172.21.15.7:43170) 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d6359e700 1 -- 172.21.15.7:0/4057949865 learned_addr learned my addr 172.21.15.7:0/4057949865 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d6259c700 1 --2- 172.21.15.7:0/4057949865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d6415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.290+0000 7f4d62d9d700 1 --2- 172.21.15.7:0/4057949865 >> v2:172.21.15.113:3300/0 conn(0x7f4d640cd6d0 0x7f4d640cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d61d9b700 1 -- 172.21.15.7:0/4057949865 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905751790 0 0) 0x7f4d6415e6c0 con 0x7f4d6415e280 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d61d9b700 1 -- 172.21.15.7:0/4057949865 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d4c003570 con 0x7f4d6415e280 2024-01-11T07:13:36.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d6259c700 1 -- 172.21.15.7:0/4057949865 >> v2:172.21.15.113:3300/0 conn(0x7f4d640cd6d0 msgr2=0x7f4d640cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d6259c700 1 --2- 172.21.15.7:0/4057949865 >> v2:172.21.15.113:3300/0 conn(0x7f4d640cd6d0 0x7f4d640cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d6259c700 1 -- 172.21.15.7:0/4057949865 >> v1:172.21.15.7:6789/0 conn(0x7f4d6415e280 legacy=0x7f4d640cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d6259c700 1 -- 172.21.15.7:0/4057949865 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640ce980 con 0x7f4d6415d8a0 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d6259c700 1 --2- 172.21.15.7:0/4057949865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d6415dd40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4d580098e0 tx=0x7f4d580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da8bb8a22be1f723 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d61d9b700 1 -- 172.21.15.7:0/4057949865 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58011630 con 0x7f4d6415d8a0 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d61d9b700 1 -- 172.21.15.7:0/4057949865 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58011790 con 0x7f4d6415d8a0 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.291+0000 7f4d61d9b700 1 -- 172.21.15.7:0/4057949865 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58011b30 con 0x7f4d6415d8a0 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.292+0000 7f4d69008700 1 -- 172.21.15.7:0/4057949865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 msgr2=0x7f4d6415dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.292+0000 7f4d69008700 1 --2- 172.21.15.7:0/4057949865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d6415dd40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4d580098e0 tx=0x7f4d580091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.292+0000 7f4d69008700 1 -- 172.21.15.7:0/4057949865 shutdown_connections 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.292+0000 7f4d69008700 1 --2- 172.21.15.7:0/4057949865 >> v2:172.21.15.113:3300/0 conn(0x7f4d640cd6d0 0x7f4d640cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.292+0000 7f4d69008700 1 --2- 172.21.15.7:0/4057949865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d6415dd40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.292+0000 7f4d69008700 1 -- 172.21.15.7:0/4057949865 >> 172.21.15.7:0/4057949865 conn(0x7f4d64071120 msgr2=0x7f4d6406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.292+0000 7f4d69008700 1 -- 172.21.15.7:0/4057949865 shutdown_connections 2024-01-11T07:13:36.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.292+0000 7f4d69008700 1 -- 172.21.15.7:0/4057949865 wait complete. 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.293+0000 7f4d69008700 1 Processor -- start 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.293+0000 7f4d69008700 1 -- start start 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d69008700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d640cd6d0 0x7f4d640c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d69008700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d640cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d69008700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d6415e280 0x7f4d640cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d69008700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64156e30 con 0x7f4d640cd6d0 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d69008700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6416a960 con 0x7f4d6415e280 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d69008700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6416a0a0 con 0x7f4d6415d8a0 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d640cd6d0 0x7f4d640c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d640cd6d0 0x7f4d640c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48860/0 (socket says 172.21.15.7:48860) 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.294+0000 7f4d62d9d700 1 -- 172.21.15.7:0/4091770962 learned_addr learned my addr 172.21.15.7:0/4091770962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d6259c700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d640cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d6359e700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d6415e280 0x7f4d640cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:36.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d62d9d700 1 -- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 msgr2=0x7f4d640cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d62d9d700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d640cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d62d9d700 1 -- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d6415e280 msgr2=0x7f4d640cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d62d9d700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d6415e280 0x7f4d640cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d62d9d700 1 -- 172.21.15.7:0/4091770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d58009420 con 0x7f4d640cd6d0 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d62d9d700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d640cd6d0 0x7f4d640c7cc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400ecb0 tx=0x7f4d540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d4affd700 1 -- 172.21.15.7:0/4091770962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400c850 con 0x7f4d640cd6d0 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d4affd700 1 -- 172.21.15.7:0/4091770962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5400c9b0 con 0x7f4d640cd6d0 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d4affd700 1 -- 172.21.15.7:0/4091770962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400cd50 con 0x7f4d640cd6d0 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d640c8200 con 0x7f4d640cd6d0 2024-01-11T07:13:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.295+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6416a310 con 0x7f4d640cd6d0 2024-01-11T07:13:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.297+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6415c4e0 con 0x7f4d640cd6d0 2024-01-11T07:13:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.297+0000 7f4d4affd700 1 -- 172.21.15.7:0/4091770962 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4d54013680 con 0x7f4d640cd6d0 2024-01-11T07:13:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.297+0000 7f4d4affd700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.297+0000 7f4d4affd700 1 -- 172.21.15.7:0/4091770962 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d54084790 con 0x7f4d640cd6d0 2024-01-11T07:13:36.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.297+0000 7f4d4affd700 1 -- 172.21.15.7:0/4091770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=138}) v3 -- 0x7f4d5006ea50 con 0x7f4d640cd6d0 2024-01-11T07:13:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.297+0000 7f4d6259c700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.298+0000 7f4d6259c700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d50066f30 0x7f4d500693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4d580091e0 tx=0x7f4d58017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:36.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.302+0000 7f4d4affd700 1 -- 172.21.15.7:0/4091770962 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d54017060 con 0x7f4d640cd6d0 2024-01-11T07:13:36.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.507+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4d6404c3c0 con 0x7f4d640cd6d0 2024-01-11T07:13:36.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.508+0000 7f4d4affd700 1 -- 172.21.15.7:0/4091770962 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f4d54051a00 con 0x7f4d640cd6d0 2024-01-11T07:13:36.508 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:36.508 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:36.508 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 137, 2024-01-11T07:13:36.508 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:13:36.508 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:13:36.508 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:13:32.234651+0000", 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:13:36.509 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:36.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:13:36.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:36.512 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:36.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:13:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:15.625657+0000", 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:36.518 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:36.519 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:36.520 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:13:36.521 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:16.691544+0000", 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:13:36.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "137", 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:36.523 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:36.524 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:36.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:13:36.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:36.533 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.535 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.537 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:36.538 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:36.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:36.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:36.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:36.544 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.545 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:36.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:36.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:13:36.553 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:36.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:36.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:36.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:36.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:13:36.560 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:36.561 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:13:36.563 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:13:36.564 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:13:36.565 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:36.566 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.572 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:36.573 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:13:36.574 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:36.575 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:36.576 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:36.577 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:36.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:13:36.579 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:13:36.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:36.581 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d50066f30 msgr2=0x7f4d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d50066f30 0x7f4d500693e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4d580091e0 tx=0x7f4d58017000 comp rx=0 tx=0).stop 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d640cd6d0 msgr2=0x7f4d640c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d640cd6d0 0x7f4d640c7cc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400ecb0 tx=0x7f4d540091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 shutdown_connections 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4d50066f30 0x7f4d500693e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d6415e280 0x7f4d640cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d640cd6d0 0x7f4d640c7cc0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 --2- 172.21.15.7:0/4091770962 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d6415d8a0 0x7f4d640cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 >> 172.21.15.7:0/4091770962 conn(0x7f4d64071120 msgr2=0x7f4d640b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 shutdown_connections 2024-01-11T07:13:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:36.510+0000 7f4d69008700 1 -- 172.21.15.7:0/4091770962 wait complete. 2024-01-11T07:13:36.584 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:13:41.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:13:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.725+0000 7fb042c0c700 1 Processor -- start 2024-01-11T07:13:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.725+0000 7fb042c0c700 1 -- start start 2024-01-11T07:13:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb042c0c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb03c159e20 0x7fb03c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb042c0c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c06bc50 0x7fb03c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb042c0c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c15e010 con 0x7fb03c159a50 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb042c0c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c06c610 con 0x7fb03c159e20 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb042c0c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0acf80 con 0x7fb03c06bc50 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb03bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb03c159e20 0x7fb03c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb03bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb03c159e20 0x7fb03c15e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45750/0 (socket says 172.21.15.7:45750) 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb03bfff700 1 -- 172.21.15.7:0/3042588900 learned_addr learned my addr 172.21.15.7:0/3042588900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.726+0000 7fb0411a9700 1 --2- 172.21.15.7:0/3042588900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c06bc50 0x7fb03c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb03bfff700 1 -- 172.21.15.7:0/3042588900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c06bc50 msgr2=0x7fb03c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb03bfff700 1 --2- 172.21.15.7:0/3042588900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c06bc50 0x7fb03c06c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb03bfff700 1 -- 172.21.15.7:0/3042588900 >> v1:172.21.15.113:6789/0 conn(0x7fb03c159a50 legacy=0x7fb03c0ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb03bfff700 1 -- 172.21.15.7:0/3042588900 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c15d9c0 con 0x7fb03c159e20 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb03bfff700 1 --2- 172.21.15.7:0/3042588900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c159e20 0x7fb03c15e4a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb0280099b0 tx=0x7fb028009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a557623582aba9d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb03b7fe700 1 -- 172.21.15.7:0/3042588900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028009ea0 con 0x7fb03c159e20 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb03b7fe700 1 -- 172.21.15.7:0/3042588900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02800b860 con 0x7fb03c159e20 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb03b7fe700 1 -- 172.21.15.7:0/3042588900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02800bc00 con 0x7fb03c159e20 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb042c0c700 1 -- 172.21.15.7:0/3042588900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c159e20 msgr2=0x7fb03c15e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.727+0000 7fb042c0c700 1 --2- 172.21.15.7:0/3042588900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c159e20 0x7fb03c15e4a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb0280099b0 tx=0x7fb028009250 comp rx=0 tx=0).stop 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.728+0000 7fb042c0c700 1 -- 172.21.15.7:0/3042588900 shutdown_connections 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.728+0000 7fb042c0c700 1 --2- 172.21.15.7:0/3042588900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c06bc50 0x7fb03c06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.728+0000 7fb042c0c700 1 --2- 172.21.15.7:0/3042588900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c159e20 0x7fb03c15e4a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.728+0000 7fb042c0c700 1 -- 172.21.15.7:0/3042588900 >> 172.21.15.7:0/3042588900 conn(0x7fb03c0cd910 msgr2=0x7fb03c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.728+0000 7fb042c0c700 1 -- 172.21.15.7:0/3042588900 shutdown_connections 2024-01-11T07:13:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.728+0000 7fb042c0c700 1 -- 172.21.15.7:0/3042588900 wait complete. 2024-01-11T07:13:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.729+0000 7fb042c0c700 1 Processor -- start 2024-01-11T07:13:41.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.729+0000 7fb042c0c700 1 -- start start 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb042c0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c06bc50 0x7fb03c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb042c0c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c159a50 0x7fb03c156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb042c0c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c159e20 0x7fb03c0af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb042c0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0af080 con 0x7fb03c06bc50 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb042c0c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0ae470 con 0x7fb03c159e20 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb042c0c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0ae750 con 0x7fb03c159a50 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c06bc50 0x7fb03c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c06bc50 0x7fb03c1530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45756/0 (socket says 172.21.15.7:45756) 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 -- 172.21.15.7:0/1101781995 learned_addr learned my addr 172.21.15.7:0/1101781995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb03bfff700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c159a50 0x7fb03c156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0411a9700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c159e20 0x7fb03c0af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 -- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c159a50 msgr2=0x7fb03c156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c159a50 0x7fb03c156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 -- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c159e20 msgr2=0x7fb03c0af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c159e20 0x7fb03c0af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 -- 172.21.15.7:0/1101781995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280094f0 con 0x7fb03c06bc50 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.730+0000 7fb0409a8700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c06bc50 0x7fb03c1530d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb03000c590 tx=0x7fb03000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.731+0000 7fb038ff9700 1 -- 172.21.15.7:0/1101781995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030011590 con 0x7fb03c06bc50 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.731+0000 7fb038ff9700 1 -- 172.21.15.7:0/1101781995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb030011710 con 0x7fb03c06bc50 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.731+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c0aea30 con 0x7fb03c06bc50 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.731+0000 7fb038ff9700 1 -- 172.21.15.7:0/1101781995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030011ad0 con 0x7fb03c06bc50 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.731+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c155b20 con 0x7fb03c06bc50 2024-01-11T07:13:41.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.732+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c04c3c0 con 0x7fb03c06bc50 2024-01-11T07:13:41.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.733+0000 7fb038ff9700 1 -- 172.21.15.7:0/1101781995 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb03001b3b0 con 0x7fb03c06bc50 2024-01-11T07:13:41.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.733+0000 7fb038ff9700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb020066f30 0x7fb0200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:41.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.733+0000 7fb038ff9700 1 -- 172.21.15.7:0/1101781995 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb030083da0 con 0x7fb03c06bc50 2024-01-11T07:13:41.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.733+0000 7fb038ff9700 1 -- 172.21.15.7:0/1101781995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=138}) v3 -- 0x7fb02006ea50 con 0x7fb03c06bc50 2024-01-11T07:13:41.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.733+0000 7fb03bfff700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb020066f30 0x7fb0200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:41.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.734+0000 7fb03bfff700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb020066f30 0x7fb0200693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb0280099b0 tx=0x7fb028017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:41.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.737+0000 7fb038ff9700 1 -- 172.21.15.7:0/1101781995 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb030051090 con 0x7fb03c06bc50 2024-01-11T07:13:41.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.945+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb03c0af360 con 0x7fb03c06bc50 2024-01-11T07:13:41.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.946+0000 7fb038ff9700 1 -- 172.21.15.7:0/1101781995 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v137) v1 ==== 81+0+32253 (secure 0 0 0) 0x7fb030017030 con 0x7fb03c06bc50 2024-01-11T07:13:41.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:41.946 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:41.946 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 137, 2024-01-11T07:13:41.946 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:13:41.946 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:13:32.234651+0000", 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:13:41.947 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:13:41.948 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:41.949 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:41.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:41.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:41.952 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:41.953 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:41.954 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:41.955 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:15.625657+0000", 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:41.956 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:13:41.957 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:41.960 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:13:41.961 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:13:41.962 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:16.691544+0000", 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:41.963 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:41.964 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "137", 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:41.965 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:41.966 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:41.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.968 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:41.969 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:41.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:41.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:41.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.974 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:41.977 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:41.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.982 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:41.983 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:41.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:41.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:41.990 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:41.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:41.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.996 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:41.997 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:41.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:13:42.002 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:42.003 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.005 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:13:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:42.010 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.016 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:13:42.017 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:13:42.019 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:13:42.023 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:13:42.025 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:13:42.026 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:42.027 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:42.028 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:42.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:42.032 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:42.033 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:13:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:42.035 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:42.036 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb020066f30 msgr2=0x7fb0200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb020066f30 0x7fb0200693e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb0280099b0 tx=0x7fb028017000 comp rx=0 tx=0).stop 2024-01-11T07:13:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c06bc50 msgr2=0x7fb03c1530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c06bc50 0x7fb03c1530d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb03000c590 tx=0x7fb03000c8a0 comp rx=0 tx=0).stop 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 shutdown_connections 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb020066f30 0x7fb0200693e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb03c159e20 0x7fb03c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c06bc50 0x7fb03c1530d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 --2- 172.21.15.7:0/1101781995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb03c159a50 0x7fb03c156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 >> 172.21.15.7:0/1101781995 conn(0x7fb03c0cd910 msgr2=0x7fb03c0b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 shutdown_connections 2024-01-11T07:13:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:41.948+0000 7fb042c0c700 1 -- 172.21.15.7:0/1101781995 wait complete. 2024-01-11T07:13:42.040 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:13:47.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:13:47.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.182+0000 7fde936dc700 1 Processor -- start 2024-01-11T07:13:47.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.183+0000 7fde936dc700 1 -- start start 2024-01-11T07:13:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.183+0000 7fde936dc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c159ca0 0x7fde8c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde936dc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde8c15a070 0x7fde8c06e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde936dc700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8c0d18c0 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde936dc700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c0d0010 con 0x7fde8c15a070 2024-01-11T07:13:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde936dc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0cf400 con 0x7fde8c159ca0 2024-01-11T07:13:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde91478700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c159ca0 0x7fde8c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde90c77700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde8c15a070 0x7fde8c06e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde91c79700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fde8c0ce7f0 0x7fde8c0cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:51740/0 (socket says 172.21.15.7:51740) 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde91c79700 1 -- 172.21.15.7:0/3875308497 learned_addr learned my addr 172.21.15.7:0/3875308497 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde83fff700 1 -- 172.21.15.7:0/3875308497 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361910105 0 0) 0x7fde8c0d18c0 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde83fff700 1 -- 172.21.15.7:0/3875308497 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde78003570 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde91478700 1 -- 172.21.15.7:0/3875308497 >> v2:172.21.15.113:3300/0 conn(0x7fde8c15a070 msgr2=0x7fde8c06e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde91478700 1 --2- 172.21.15.7:0/3875308497 >> v2:172.21.15.113:3300/0 conn(0x7fde8c15a070 0x7fde8c06e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde91478700 1 -- 172.21.15.7:0/3875308497 >> v1:172.21.15.113:6789/0 conn(0x7fde8c0ce7f0 legacy=0x7fde8c0cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde91478700 1 -- 172.21.15.7:0/3875308497 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c0cf6e0 con 0x7fde8c159ca0 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.184+0000 7fde90c77700 1 --2- 172.21.15.7:0/3875308497 >> v2:172.21.15.113:3300/0 conn(0x7fde8c15a070 0x7fde8c06e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde91478700 1 --2- 172.21.15.7:0/3875308497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c159ca0 0x7fde8c06dc30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde880098e0 tx=0x7fde880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a571d63c938b99e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde83fff700 1 -- 172.21.15.7:0/3875308497 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88011630 con 0x7fde8c159ca0 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde83fff700 1 -- 172.21.15.7:0/3875308497 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde88011790 con 0x7fde8c159ca0 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde83fff700 1 -- 172.21.15.7:0/3875308497 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88011b30 con 0x7fde8c159ca0 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde936dc700 1 -- 172.21.15.7:0/3875308497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c159ca0 msgr2=0x7fde8c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde936dc700 1 --2- 172.21.15.7:0/3875308497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c159ca0 0x7fde8c06dc30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde880098e0 tx=0x7fde880091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde936dc700 1 -- 172.21.15.7:0/3875308497 shutdown_connections 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde936dc700 1 --2- 172.21.15.7:0/3875308497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c159ca0 0x7fde8c06dc30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde936dc700 1 --2- 172.21.15.7:0/3875308497 >> v2:172.21.15.113:3300/0 conn(0x7fde8c15a070 0x7fde8c06e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.185+0000 7fde936dc700 1 -- 172.21.15.7:0/3875308497 >> 172.21.15.7:0/3875308497 conn(0x7fde8c063cc0 msgr2=0x7fde8c0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.186+0000 7fde936dc700 1 -- 172.21.15.7:0/3875308497 shutdown_connections 2024-01-11T07:13:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.186+0000 7fde936dc700 1 -- 172.21.15.7:0/3875308497 wait complete. 2024-01-11T07:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.187+0000 7fde936dc700 1 Processor -- start 2024-01-11T07:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.187+0000 7fde936dc700 1 -- start start 2024-01-11T07:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde936dc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde8c0ce7f0 0x7fde8c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde936dc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde8c159ca0 0x7fde8c0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde936dc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c15a070 0x7fde8c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde936dc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0b2210 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde936dc700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0b24f0 con 0x7fde8c159ca0 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde936dc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0b27d0 con 0x7fde8c15a070 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde8c0ce7f0 0x7fde8c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde8c0ce7f0 0x7fde8c14ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45762/0 (socket says 172.21.15.7:45762) 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 -- 172.21.15.7:0/984435870 learned_addr learned my addr 172.21.15.7:0/984435870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde90c77700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde8c159ca0 0x7fde8c0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91c79700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c15a070 0x7fde8c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 -- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c15a070 msgr2=0x7fde8c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c15a070 0x7fde8c0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 -- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde8c159ca0 msgr2=0x7fde8c0b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde8c159ca0 0x7fde8c0b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 -- 172.21.15.7:0/984435870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c009420 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.188+0000 7fde91478700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde8c0ce7f0 0x7fde8c14ee50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0cf540 tx=0x7fde88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.189+0000 7fde817fa700 1 -- 172.21.15.7:0/984435870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88000d10 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.189+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde88009420 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.189+0000 7fde817fa700 1 -- 172.21.15.7:0/984435870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde88011b10 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.189+0000 7fde817fa700 1 -- 172.21.15.7:0/984435870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88025660 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.189+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c0b2d90 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.190+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c04c3c0 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.190+0000 7fde817fa700 1 -- 172.21.15.7:0/984435870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fde88025a50 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.191+0000 7fde817fa700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde70066f30 0x7fde700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.191+0000 7fde817fa700 1 -- 172.21.15.7:0/984435870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde88085d50 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.191+0000 7fde90c77700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde70066f30 0x7fde700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.191+0000 7fde90c77700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde70066f30 0x7fde700693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fde8c155850 tx=0x7fde7c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.195+0000 7fde817fa700 1 -- 172.21.15.7:0/984435870 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde88089030 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.399+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fde8c0b2fb0 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.400+0000 7fde817fa700 1 -- 172.21.15.7:0/984435870 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v138) v1 ==== 81+0+32234 (secure 0 0 0) 0x7fde88053040 con 0x7fde8c0ce7f0 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 138, 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:13:42.236945+0000", 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:13:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:13:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:15.625657+0000", 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:16.691544+0000", 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "138", 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:13:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:13:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:13:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:13:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:13:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:13:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:13:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:13:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:13:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.451 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.451 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.451 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:13:47.451 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:13:47.452 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.453 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.457 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.458 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.460 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.462 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:47.465 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.466 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.467 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.468 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:13:47.471 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.472 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.473 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.402+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde70066f30 msgr2=0x7fde700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.402+0000 7fde936dc700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde70066f30 0x7fde700693e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fde8c155850 tx=0x7fde7c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.402+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde8c0ce7f0 msgr2=0x7fde8c14ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.402+0000 7fde936dc700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde8c0ce7f0 0x7fde8c14ee50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0cf540 tx=0x7fde88017000 comp rx=0 tx=0).stop 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.403+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 shutdown_connections 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.403+0000 7fde936dc700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fde70066f30 0x7fde700693e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.403+0000 7fde936dc700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde8c15a070 0x7fde8c0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.403+0000 7fde936dc700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde8c0ce7f0 0x7fde8c14ee50 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.403+0000 7fde936dc700 1 --2- 172.21.15.7:0/984435870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde8c159ca0 0x7fde8c0b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.403+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 >> 172.21.15.7:0/984435870 conn(0x7fde8c063cc0 msgr2=0x7fde8c0c59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.403+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 shutdown_connections 2024-01-11T07:13:47.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.403+0000 7fde936dc700 1 -- 172.21.15.7:0/984435870 wait complete. 2024-01-11T07:13:47.477 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-01-11T07:13:47.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:13:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.612+0000 7fb3869bf700 1 Processor -- start 2024-01-11T07:13:47.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.612+0000 7fb3869bf700 1 -- start start 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.613+0000 7fb3869bf700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3800b0b40 0x7fb380162770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.613+0000 7fb3869bf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb380159300 con 0x7fb3800cafb0 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.613+0000 7fb3869bf700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3800cf000 con 0x7fb380158280 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.613+0000 7fb3869bf700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3800cf260 con 0x7fb3800b0b40 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.613+0000 7fb37ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb3800cafb0 0x7fb3800ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59778/0 (socket says 172.21.15.7:59778) 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.613+0000 7fb37ffff700 1 -- 172.21.15.7:0/3426990112 learned_addr learned my addr 172.21.15.7:0/3426990112 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.613+0000 7fb384f5c700 1 --2- 172.21.15.7:0/3426990112 >> v2:172.21.15.113:3300/0 conn(0x7fb3800b0b40 0x7fb380162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb37effd700 1 -- 172.21.15.7:0/3426990112 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153063425 0 0) 0x7fb380159300 con 0x7fb3800cafb0 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb37effd700 1 -- 172.21.15.7:0/3426990112 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb368003570 con 0x7fb3800cafb0 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb384f5c700 1 -- 172.21.15.7:0/3426990112 >> v1:172.21.15.113:6789/0 conn(0x7fb380158280 legacy=0x7fb3800b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb384f5c700 1 -- 172.21.15.7:0/3426990112 >> v1:172.21.15.7:6789/0 conn(0x7fb3800cafb0 legacy=0x7fb3800ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb384f5c700 1 -- 172.21.15.7:0/3426990112 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3800cf540 con 0x7fb3800b0b40 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb384f5c700 1 --2- 172.21.15.7:0/3426990112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3800b0b40 0x7fb380162770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb378009e60 tx=0x7fb378009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee0b374a8c8ce158 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:47.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb37effd700 1 -- 172.21.15.7:0/3426990112 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37800e070 con 0x7fb3800b0b40 2024-01-11T07:13:47.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb37effd700 1 -- 172.21.15.7:0/3426990112 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37800bb60 con 0x7fb3800b0b40 2024-01-11T07:13:47.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.614+0000 7fb37effd700 1 -- 172.21.15.7:0/3426990112 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378012430 con 0x7fb3800b0b40 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.615+0000 7fb3869bf700 1 -- 172.21.15.7:0/3426990112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3800b0b40 msgr2=0x7fb380162770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.615+0000 7fb3869bf700 1 --2- 172.21.15.7:0/3426990112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3800b0b40 0x7fb380162770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb378009e60 tx=0x7fb378009780 comp rx=0 tx=0).stop 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.615+0000 7fb3869bf700 1 -- 172.21.15.7:0/3426990112 shutdown_connections 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.615+0000 7fb3869bf700 1 --2- 172.21.15.7:0/3426990112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3800b0b40 0x7fb380162770 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.615+0000 7fb3869bf700 1 -- 172.21.15.7:0/3426990112 >> 172.21.15.7:0/3426990112 conn(0x7fb380071100 msgr2=0x7fb38006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.615+0000 7fb3869bf700 1 -- 172.21.15.7:0/3426990112 shutdown_connections 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.615+0000 7fb3869bf700 1 -- 172.21.15.7:0/3426990112 wait complete. 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.616+0000 7fb3869bf700 1 Processor -- start 2024-01-11T07:13:47.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.616+0000 7fb3869bf700 1 -- start start 2024-01-11T07:13:47.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb3869bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3800b0b40 0x7fb380068ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb3869bf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3800cafb0 0x7fb3801552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb3869bf700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb380158280 0x7fb3800b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb3869bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800c7c10 con 0x7fb3800b0b40 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb3869bf700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb380068600 con 0x7fb380158280 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb3869bf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb380069420 con 0x7fb3800cafb0 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb37ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3800b0b40 0x7fb380068ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb37ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3800b0b40 0x7fb380068ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41346/0 (socket says 172.21.15.7:41346) 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb37ffff700 1 -- 172.21.15.7:0/3157095264 learned_addr learned my addr 172.21.15.7:0/3157095264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb384f5c700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb380158280 0x7fb3800b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb37f7fe700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3800cafb0 0x7fb3801552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb37ffff700 1 -- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3800cafb0 msgr2=0x7fb3801552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.617+0000 7fb37ffff700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3800cafb0 0x7fb3801552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb37ffff700 1 -- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb380158280 msgr2=0x7fb3800b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb37ffff700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb380158280 0x7fb3800b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb37ffff700 1 -- 172.21.15.7:0/3157095264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378009a20 con 0x7fb3800b0b40 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb37f7fe700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3800cafb0 0x7fb3801552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb384f5c700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb380158280 0x7fb3800b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb37ffff700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3800b0b40 0x7fb380068ee0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c7c0 tx=0x7fb370007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb367fff700 1 -- 172.21.15.7:0/3157095264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370005b30 con 0x7fb3800b0b40 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb367fff700 1 -- 172.21.15.7:0/3157095264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370005c90 con 0x7fb3800b0b40 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3800679f0 con 0x7fb3800b0b40 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb367fff700 1 -- 172.21.15.7:0/3157095264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370007340 con 0x7fb3800b0b40 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.618+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb380067fb0 con 0x7fb3800b0b40 2024-01-11T07:13:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.619+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38015c660 con 0x7fb3800b0b40 2024-01-11T07:13:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.619+0000 7fb367fff700 1 -- 172.21.15.7:0/3157095264 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb37001b400 con 0x7fb3800b0b40 2024-01-11T07:13:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.620+0000 7fb367fff700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb36c066f30 0x7fb36c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.620+0000 7fb367fff700 1 -- 172.21.15.7:0/3157095264 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb370083fd0 con 0x7fb3800b0b40 2024-01-11T07:13:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.620+0000 7fb37f7fe700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb36c066f30 0x7fb36c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:47.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.621+0000 7fb37f7fe700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb36c066f30 0x7fb36c0693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb380068e90 tx=0x7fb374009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:47.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.624+0000 7fb367fff700 1 -- 172.21.15.7:0/3157095264 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3700512c0 con 0x7fb3800b0b40 2024-01-11T07:13:47.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.831+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb38004c3c0 con 0x7fb3800b0b40 2024-01-11T07:13:47.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.832+0000 7fb367fff700 1 -- 172.21.15.7:0/3157095264 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v138) v1 ==== 81+0+32234 (secure 0 0 0) 0x7fb370064e00 con 0x7fb3800b0b40 2024-01-11T07:13:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:47.833 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:13:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 138, 2024-01-11T07:13:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:13:47.833 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:13:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:13:42.236945+0000", 2024-01-11T07:13:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:13:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:13:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:13:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:13:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:13:47.834 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:13:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:13:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:13:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:13:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:13:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:13:47.835 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:13:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:13:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:13:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 17, 2024-01-11T07:13:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:13:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:13:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:13:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:13:47.836 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:47.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:13:47.838 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:47.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:47.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:47.841 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:47.842 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 16, 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:15.625657+0000", 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:47.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:13:47.844 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "134", 2024-01-11T07:13:47.845 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:47.846 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:47.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:47.848 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:13:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:13:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:13:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:47.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 17, 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:13:16.691544+0000", 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:13:47.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:13:47.851 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "138", 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:13:47.852 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:13:47.853 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:13:47.854 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.858 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:13:47.861 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:13:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.863 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.864 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.867 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.869 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.874 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.875 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.877 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:47.880 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.885 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.885 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.885 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:13:47.886 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:13:47.887 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:13:47.887 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.887 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.887 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.887 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.888 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:47.888 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.888 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.888 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.889 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.889 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.889 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.889 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.889 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.889 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.894 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.895 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.898 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:13:47.899 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:13:47.900 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 133, 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.904 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:13:47.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:13:47.906 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.907 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.908 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.909 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.910 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.911 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.912 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.916 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:13:47.917 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:13:47.918 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:13:47.919 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:13:47.920 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:13:47.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb36c066f30 msgr2=0x7fb36c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb36c066f30 0x7fb36c0693e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb380068e90 tx=0x7fb374009040 comp rx=0 tx=0).stop 2024-01-11T07:13:47.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3800b0b40 msgr2=0x7fb380068ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3800b0b40 0x7fb380068ee0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c7c0 tx=0x7fb370007e00 comp rx=0 tx=0).stop 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 shutdown_connections 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb36c066f30 0x7fb36c0693e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb380158280 0x7fb3800b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3800b0b40 0x7fb380068ee0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 --2- 172.21.15.7:0/3157095264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3800cafb0 0x7fb3801552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 >> 172.21.15.7:0/3157095264 conn(0x7fb380071100 msgr2=0x7fb380156470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 shutdown_connections 2024-01-11T07:13:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:47.835+0000 7fb3869bf700 1 -- 172.21.15.7:0/3157095264 wait complete. 2024-01-11T07:13:47.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.060+0000 7f01c455d700 1 Processor -- start 2024-01-11T07:13:48.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.060+0000 7f01c455d700 1 -- start start 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.061+0000 7f01c455d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01bc06e0d0 0x7f01bc159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.061+0000 7f01c455d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc15b770 con 0x7f01bc159ff0 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.061+0000 7f01c455d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc15abf0 con 0x7f01bc06dd00 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.061+0000 7f01c455d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc15aed0 con 0x7f01bc06e0d0 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.061+0000 7f01c2afa700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f01bc159ff0 0x7f01bc069fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59790/0 (socket says 172.21.15.7:59790) 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.061+0000 7f01c2afa700 1 -- 172.21.15.7:0/2763778622 learned_addr learned my addr 172.21.15.7:0/2763778622 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.061+0000 7f01c1af8700 1 --2- 172.21.15.7:0/2763778622 >> v2:172.21.15.113:3300/0 conn(0x7f01bc06e0d0 0x7f01bc159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c12f7700 1 -- 172.21.15.7:0/2763778622 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787757181 0 0) 0x7f01bc15b770 con 0x7f01bc159ff0 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c12f7700 1 -- 172.21.15.7:0/2763778622 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01a8003570 con 0x7f01bc159ff0 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c1af8700 1 -- 172.21.15.7:0/2763778622 >> v1:172.21.15.113:6789/0 conn(0x7f01bc06dd00 legacy=0x7f01bc0c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:48.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c12f7700 1 -- 172.21.15.7:0/2763778622 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238980626 0 0) 0x7f01bc15abf0 con 0x7f01bc06dd00 2024-01-11T07:13:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c1af8700 1 -- 172.21.15.7:0/2763778622 >> v1:172.21.15.7:6789/0 conn(0x7f01bc159ff0 legacy=0x7f01bc069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c1af8700 1 -- 172.21.15.7:0/2763778622 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc15b1b0 con 0x7f01bc06e0d0 2024-01-11T07:13:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c1af8700 1 --2- 172.21.15.7:0/2763778622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 0x7f01bc159ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0098e0 tx=0x7f01ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59f388819f436bb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c12f7700 1 -- 172.21.15.7:0/2763778622 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00b640 con 0x7f01bc06e0d0 2024-01-11T07:13:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c12f7700 1 -- 172.21.15.7:0/2763778622 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00b7a0 con 0x7f01bc06e0d0 2024-01-11T07:13:48.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.062+0000 7f01c12f7700 1 -- 172.21.15.7:0/2763778622 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00bb40 con 0x7f01bc06e0d0 2024-01-11T07:13:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.063+0000 7f01c455d700 1 -- 172.21.15.7:0/2763778622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 msgr2=0x7f01bc159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.063+0000 7f01c455d700 1 --2- 172.21.15.7:0/2763778622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 0x7f01bc159ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0098e0 tx=0x7f01ac0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.063+0000 7f01c455d700 1 -- 172.21.15.7:0/2763778622 shutdown_connections 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.063+0000 7f01c455d700 1 --2- 172.21.15.7:0/2763778622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 0x7f01bc159ab0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.063+0000 7f01c455d700 1 -- 172.21.15.7:0/2763778622 >> 172.21.15.7:0/2763778622 conn(0x7f01bc0680e0 msgr2=0x7f01bc0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.063+0000 7f01c455d700 1 -- 172.21.15.7:0/2763778622 shutdown_connections 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.063+0000 7f01c455d700 1 -- 172.21.15.7:0/2763778622 wait complete. 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.064+0000 7f01c455d700 1 Processor -- start 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.064+0000 7f01c455d700 1 -- start start 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c455d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01bc06dd00 0x7f01bc168430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c455d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 0x7f01bc167560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c455d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01bc159ff0 0x7f01bc0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c455d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc155ce0 con 0x7f01bc159ff0 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c455d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0c7b80 con 0x7f01bc06e0d0 2024-01-11T07:13:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c455d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0c78a0 con 0x7f01bc06dd00 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c2afa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01bc159ff0 0x7f01bc0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c2afa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01bc159ff0 0x7f01bc0c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41356/0 (socket says 172.21.15.7:41356) 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c2afa700 1 -- 172.21.15.7:0/1804737549 learned_addr learned my addr 172.21.15.7:0/1804737549 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c1af8700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 0x7f01bc167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.065+0000 7f01c22f9700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01bc06dd00 0x7f01bc168430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01c2afa700 1 -- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01bc06dd00 msgr2=0x7f01bc168430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01c2afa700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01bc06dd00 0x7f01bc168430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01c2afa700 1 -- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 msgr2=0x7f01bc167560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01c2afa700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 0x7f01bc167560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01c2afa700 1 -- 172.21.15.7:0/1804737549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac009420 con 0x7f01bc159ff0 2024-01-11T07:13:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01c2afa700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01bc159ff0 0x7f01bc0c6ec0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f01b4001460 tx=0x7f01b4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01b27fc700 1 -- 172.21.15.7:0/1804737549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b400a870 con 0x7f01bc159ff0 2024-01-11T07:13:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01b27fc700 1 -- 172.21.15.7:0/1804737549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b400a9d0 con 0x7f01bc159ff0 2024-01-11T07:13:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc0ad7c0 con 0x7f01bc159ff0 2024-01-11T07:13:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01b27fc700 1 -- 172.21.15.7:0/1804737549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b400ad50 con 0x7f01bc159ff0 2024-01-11T07:13:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.066+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc0add80 con 0x7f01bc159ff0 2024-01-11T07:13:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.067+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc04c3c0 con 0x7f01bc159ff0 2024-01-11T07:13:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.068+0000 7f01b27fc700 1 -- 172.21.15.7:0/1804737549 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f01b4007300 con 0x7f01bc159ff0 2024-01-11T07:13:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.068+0000 7f01b27fc700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f01a0066f30 0x7f01a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.068+0000 7f01b27fc700 1 -- 172.21.15.7:0/1804737549 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f01b4084790 con 0x7f01bc159ff0 2024-01-11T07:13:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.068+0000 7f01c22f9700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f01a0066f30 0x7f01a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.069+0000 7f01c22f9700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f01a0066f30 0x7f01a00693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f01bc15b110 tx=0x7f01b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.073+0000 7f01b27fc700 1 -- 172.21.15.7:0/1804737549 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01b4051a80 con 0x7f01bc159ff0 2024-01-11T07:13:48.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.323+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01bc15b490 con 0x7f01bc159ff0 2024-01-11T07:13:48.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.323+0000 7f01b27fc700 1 -- 172.21.15.7:0/1804737549 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f01b40550a0 con 0x7f01bc159ff0 2024-01-11T07:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:48.324 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f01a0066f30 msgr2=0x7f01a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f01a0066f30 0x7f01a00693e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f01bc15b110 tx=0x7f01b8009040 comp rx=0 tx=0).stop 2024-01-11T07:13:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01bc159ff0 msgr2=0x7f01bc0c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:48.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01bc159ff0 0x7f01bc0c6ec0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f01b4001460 tx=0x7f01b4001a40 comp rx=0 tx=0).stop 2024-01-11T07:13:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 shutdown_connections 2024-01-11T07:13:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f01a0066f30 0x7f01a00693e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01bc159ff0 0x7f01bc0c6ec0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01bc06dd00 0x7f01bc168430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:48.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 --2- 172.21.15.7:0/1804737549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01bc06e0d0 0x7f01bc167560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 >> 172.21.15.7:0/1804737549 conn(0x7f01bc0680e0 msgr2=0x7f01bc0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 shutdown_connections 2024-01-11T07:13:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:48.326+0000 7f01c455d700 1 -- 172.21.15.7:0/1804737549 wait complete. 2024-01-11T07:13:48.329 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:48.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:13:48.491 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":7068,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":30.184627150000001,"mdsmap_epoch":65,"osdmap_epoch":138,"osdmap_epoch_barrier":137,"uptime":36.178576206000002} 2024-01-11T07:13:48.492 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 7068, "mdsmap_epoch": 65, "osdmap_epoch": 138, "osdmap_epoch_barrier": 137, "rank_uptime": 30.18462715, "state": "up:active", "uptime": 36.178576206, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:13:48.492 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:13:48.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:48.492 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-01-11T07:13:48.804 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-11T07:13:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-11T07:13:48.805 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.292552 s, 179 MB/s 2024-01-11T07:13:48.806 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:13:48.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:13:48.849 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:13:48.895 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:13:48.895 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:13:48.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:48.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:13:48.978 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:13:48.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:48.979 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:13:49.004 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:13:49.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:13:49.004 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:13:49.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:13:49.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:13:49.042 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:13:49.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.240+0000 7f0c0c7d6700 1 Processor -- start 2024-01-11T07:13:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.240+0000 7f0c0c7d6700 1 -- start start 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c0c7d6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c040cafb0 0x7f0c040ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c0c7d6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0c04158280 0x7f0c040b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c0c7d6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c04159300 con 0x7f0c040b0970 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c0c7d6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c040b0dc0 con 0x7f0c040cafb0 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c0c7d6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c040cf260 con 0x7f0c04158280 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c0a572700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c040cafb0 0x7f0c040ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c0a572700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c040cafb0 0x7f0c040ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41362/0 (socket says 172.21.15.7:41362) 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c0a572700 1 -- 172.21.15.7:0/697137408 learned_addr learned my addr 172.21.15.7:0/697137408 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.241+0000 7f0c09d71700 1 --2- 172.21.15.7:0/697137408 >> v2:172.21.15.113:3300/0 conn(0x7f0c04158280 0x7f0c040b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c0a572700 1 -- 172.21.15.7:0/697137408 >> v2:172.21.15.113:3300/0 conn(0x7f0c04158280 msgr2=0x7f0c040b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c0a572700 1 --2- 172.21.15.7:0/697137408 >> v2:172.21.15.113:3300/0 conn(0x7f0c04158280 0x7f0c040b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c0a572700 1 -- 172.21.15.7:0/697137408 >> v1:172.21.15.113:6789/0 conn(0x7f0c040b0970 legacy=0x7f0c04159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c0a572700 1 -- 172.21.15.7:0/697137408 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040cf540 con 0x7f0c040cafb0 2024-01-11T07:13:49.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c0a572700 1 --2- 172.21.15.7:0/697137408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c040cafb0 0x7f0c040ce8f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0c0406a660 tx=0x7f0bf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925c52b08d029a31 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:49.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c09570700 1 -- 172.21.15.7:0/697137408 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4009e70 con 0x7f0c040cafb0 2024-01-11T07:13:49.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c09570700 1 -- 172.21.15.7:0/697137408 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf400b860 con 0x7f0c040cafb0 2024-01-11T07:13:49.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c09570700 1 -- 172.21.15.7:0/697137408 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400bc00 con 0x7f0c040cafb0 2024-01-11T07:13:49.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.242+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/697137408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c040cafb0 msgr2=0x7f0c040ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.243+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/697137408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c040cafb0 0x7f0c040ce8f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0c0406a660 tx=0x7f0bf4009250 comp rx=0 tx=0).stop 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.243+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/697137408 shutdown_connections 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.243+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/697137408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c040cafb0 0x7f0c040ce8f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.243+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/697137408 >> v2:172.21.15.113:3300/0 conn(0x7f0c04158280 0x7f0c040b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.243+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/697137408 >> 172.21.15.7:0/697137408 conn(0x7f0c04071100 msgr2=0x7f0c0406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.243+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/697137408 shutdown_connections 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.243+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/697137408 wait complete. 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.244+0000 7f0c0c7d6700 1 Processor -- start 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.244+0000 7f0c0c7d6700 1 -- start start 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0c7d6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040b0970 0x7f0c0414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0c7d6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c040cafb0 0x7f0c040c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0c7d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04158280 0x7f0c040c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0c7d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040c9390 con 0x7f0c04158280 2024-01-11T07:13:49.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0c7d6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040c9670 con 0x7f0c040b0970 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0c7d6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040c9950 con 0x7f0c040cafb0 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0a572700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040b0970 0x7f0c0414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c09d71700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c040cafb0 0x7f0c040c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0ad73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04158280 0x7f0c040c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0a572700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040b0970 0x7f0c0414e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:48640/0 (socket says 172.21.15.7:48640) 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c0a572700 1 -- 172.21.15.7:0/826180099 learned_addr learned my addr 172.21.15.7:0/826180099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c09d71700 1 -- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040b0970 msgr2=0x7f0c0414e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c09d71700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040b0970 0x7f0c0414e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c09d71700 1 -- 172.21.15.7:0/826180099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04158280 msgr2=0x7f0c040c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c09d71700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04158280 0x7f0c040c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c09d71700 1 -- 172.21.15.7:0/826180099 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40094f0 con 0x7f0c040cafb0 2024-01-11T07:13:49.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.245+0000 7f0c09d71700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c040cafb0 0x7f0c040c8840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000e8d0 tx=0x7f0c000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:49.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.249+0000 7f0bfa7fc700 1 -- 172.21.15.7:0/826180099 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00013680 con 0x7f0c040cafb0 2024-01-11T07:13:49.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.249+0000 7f0bfa7fc700 1 -- 172.21.15.7:0/826180099 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c000137e0 con 0x7f0c040cafb0 2024-01-11T07:13:49.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.249+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c040c9c30 con 0x7f0c040cafb0 2024-01-11T07:13:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.249+0000 7f0bfa7fc700 1 -- 172.21.15.7:0/826180099 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00013ba0 con 0x7f0c040cafb0 2024-01-11T07:13:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.250+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0414e800 con 0x7f0c040cafb0 2024-01-11T07:13:49.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.251+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0404c3c0 con 0x7f0c040cafb0 2024-01-11T07:13:49.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.252+0000 7f0bfa7fc700 1 -- 172.21.15.7:0/826180099 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0c000253d0 con 0x7f0c040cafb0 2024-01-11T07:13:49.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.252+0000 7f0bfa7fc700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0be8067080 0x7f0be8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.252+0000 7f0bfa7fc700 1 -- 172.21.15.7:0/826180099 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f0c00084df0 con 0x7f0c040cafb0 2024-01-11T07:13:49.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.253+0000 7f0c0a572700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0be8067080 0x7f0be8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.253+0000 7f0c0a572700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0be8067080 0x7f0be8069530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009290 tx=0x7f0bf4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:49.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.258+0000 7f0bfa7fc700 1 -- 172.21.15.7:0/826180099 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c00051f80 con 0x7f0c040cafb0 2024-01-11T07:13:49.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.515+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c040c9a90 con 0x7f0c040cafb0 2024-01-11T07:13:49.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.516+0000 7f0bfa7fc700 1 -- 172.21.15.7:0/826180099 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f0c040c9a90 con 0x7f0c040cafb0 2024-01-11T07:13:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:49.516 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:49.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0be8067080 msgr2=0x7f0be8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0be8067080 0x7f0be8069530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009290 tx=0x7f0bf4017000 comp rx=0 tx=0).stop 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c040cafb0 msgr2=0x7f0c040c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c040cafb0 0x7f0c040c8840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000e8d0 tx=0x7f0c000091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 shutdown_connections 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0be8067080 0x7f0be8069530 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c04158280 0x7f0c040c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c040b0970 0x7f0c0414e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 --2- 172.21.15.7:0/826180099 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c040cafb0 0x7f0c040c8840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 >> 172.21.15.7:0/826180099 conn(0x7f0c04071100 msgr2=0x7f0c0415ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 shutdown_connections 2024-01-11T07:13:49.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.518+0000 7f0c0c7d6700 1 -- 172.21.15.7:0/826180099 wait complete. 2024-01-11T07:13:49.520 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:49.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.663+0000 7f68552a0700 1 Processor -- start 2024-01-11T07:13:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.664+0000 7f68552a0700 1 -- start start 2024-01-11T07:13:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.664+0000 7f68552a0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 0x7f68500ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.664+0000 7f68552a0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f685015a070 0x7f68500ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.664+0000 7f68552a0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500d18c0 con 0x7f6850159ca0 2024-01-11T07:13:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.664+0000 7f68552a0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500d0010 con 0x7f685015a070 2024-01-11T07:13:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.664+0000 7f68552a0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500cf400 con 0x7f685006e340 2024-01-11T07:13:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f685015a070 0x7f68500ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f685015a070 0x7f68500ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41378/0 (socket says 172.21.15.7:41378) 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684f7fe700 1 -- 172.21.15.7:0/2761419473 learned_addr learned my addr 172.21.15.7:0/2761419473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684e7fc700 1 --2- 172.21.15.7:0/2761419473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 0x7f68500ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684f7fe700 1 -- 172.21.15.7:0/2761419473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 msgr2=0x7f68500ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684f7fe700 1 --2- 172.21.15.7:0/2761419473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 0x7f68500ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684f7fe700 1 -- 172.21.15.7:0/2761419473 >> v1:172.21.15.113:6789/0 conn(0x7f6850159ca0 legacy=0x7f685006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684f7fe700 1 -- 172.21.15.7:0/2761419473 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500cf6e0 con 0x7f685015a070 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684dffb700 1 -- 172.21.15.7:0/2761419473 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457606292 0 0) 0x7f68500d18c0 con 0x7f6850159ca0 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684f7fe700 1 --2- 172.21.15.7:0/2761419473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500ced30 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6848009f20 tx=0x7f6848009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c8a8bbb85b108c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684dffb700 1 -- 172.21.15.7:0/2761419473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684800e070 con 0x7f685015a070 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684dffb700 1 -- 172.21.15.7:0/2761419473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684800b9d0 con 0x7f685015a070 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.665+0000 7f684dffb700 1 -- 172.21.15.7:0/2761419473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480092e0 con 0x7f685015a070 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.666+0000 7f68552a0700 1 -- 172.21.15.7:0/2761419473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 msgr2=0x7f68500ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.666+0000 7f68552a0700 1 --2- 172.21.15.7:0/2761419473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500ced30 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6848009f20 tx=0x7f6848009840 comp rx=0 tx=0).stop 2024-01-11T07:13:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.666+0000 7f68552a0700 1 -- 172.21.15.7:0/2761419473 shutdown_connections 2024-01-11T07:13:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.666+0000 7f68552a0700 1 --2- 172.21.15.7:0/2761419473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500ced30 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.666+0000 7f68552a0700 1 --2- 172.21.15.7:0/2761419473 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 0x7f68500ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.666+0000 7f68552a0700 1 -- 172.21.15.7:0/2761419473 >> 172.21.15.7:0/2761419473 conn(0x7f6850063cc0 msgr2=0x7f68500d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.666+0000 7f68552a0700 1 -- 172.21.15.7:0/2761419473 shutdown_connections 2024-01-11T07:13:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.666+0000 7f68552a0700 1 -- 172.21.15.7:0/2761419473 wait complete. 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.668+0000 7f68552a0700 1 Processor -- start 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.668+0000 7f68552a0700 1 -- start start 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f68552a0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 0x7f685016d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f68552a0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6850159ca0 0x7f68500b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f68552a0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f68552a0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685016d7d0 con 0x7f685015a070 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f68552a0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c7cd0 con 0x7f6850159ca0 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f68552a0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c8b30 con 0x7f685006e340 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f684f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f684f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41386/0 (socket says 172.21.15.7:41386) 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f684f7fe700 1 -- 172.21.15.7:0/2792040814 learned_addr learned my addr 172.21.15.7:0/2792040814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f684effd700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 0x7f685016d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.669+0000 7f684e7fc700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6850159ca0 0x7f68500b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f684f7fe700 1 -- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 msgr2=0x7f685016d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f684f7fe700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 0x7f685016d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f684f7fe700 1 -- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6850159ca0 msgr2=0x7f68500b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f684f7fe700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6850159ca0 0x7f68500b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f684f7fe700 1 -- 172.21.15.7:0/2792040814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009ae0 con 0x7f685015a070 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f684f7fe700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500b5780 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6850068ad0 tx=0x7f6848018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f6846ffd700 1 -- 172.21.15.7:0/2792040814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848008c70 con 0x7f685015a070 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f6846ffd700 1 -- 172.21.15.7:0/2792040814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6848008df0 con 0x7f685015a070 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500c7f20 con 0x7f685015a070 2024-01-11T07:13:49.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f6846ffd700 1 -- 172.21.15.7:0/2792040814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684801c660 con 0x7f685015a070 2024-01-11T07:13:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.670+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68500c84e0 con 0x7f685015a070 2024-01-11T07:13:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.671+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68500c8ca0 con 0x7f685015a070 2024-01-11T07:13:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.671+0000 7f6846ffd700 1 -- 172.21.15.7:0/2792040814 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6848003040 con 0x7f685015a070 2024-01-11T07:13:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.672+0000 7f6846ffd700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6830066f30 0x7f68300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.672+0000 7f6846ffd700 1 -- 172.21.15.7:0/2792040814 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6848085ed0 con 0x7f685015a070 2024-01-11T07:13:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.672+0000 7f684effd700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6830066f30 0x7f68300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.673+0000 7f684effd700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6830066f30 0x7f68300693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f68380011e0 tx=0x7f6838009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:49.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.676+0000 7f6846ffd700 1 -- 172.21.15.7:0/2792040814 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6848053060 con 0x7f685015a070 2024-01-11T07:13:49.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.882+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f685004c3c0 con 0x7f685015a070 2024-01-11T07:13:49.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.883+0000 7f6846ffd700 1 -- 172.21.15.7:0/2792040814 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+15420 (secure 0 0 0) 0x7f6848056680 con 0x7f685015a070 2024-01-11T07:13:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:49.883 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":139,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:13:49.245052+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:13:15.625657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-11T07:13:16.691544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.f","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6830066f30 msgr2=0x7f68300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6830066f30 0x7f68300693e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f68380011e0 tx=0x7f6838009040 comp rx=0 tx=0).stop 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 msgr2=0x7f68500b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500b5780 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6850068ad0 tx=0x7f6848018000 comp rx=0 tx=0).stop 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 shutdown_connections 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6830066f30 0x7f68300693e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685015a070 0x7f68500b5780 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f685006e340 0x7f685016d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 --2- 172.21.15.7:0/2792040814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6850159ca0 0x7f68500b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 >> 172.21.15.7:0/2792040814 conn(0x7f6850063cc0 msgr2=0x7f68501570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:49.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 shutdown_connections 2024-01-11T07:13:49.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:49.885+0000 7f68552a0700 1 -- 172.21.15.7:0/2792040814 wait complete. 2024-01-11T07:13:49.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:13:15.625657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:13:16.691544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:13:49.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.033+0000 7fb0701c3700 1 Processor -- start 2024-01-11T07:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.033+0000 7fb0701c3700 1 -- start start 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb0701c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0680ad9b0 0x7fb06806f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb0701c3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb068068830 0x7fb068068c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb0701c3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06806b490 con 0x7fb0680c18a0 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb0701c3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb068161f30 con 0x7fb0680ad9b0 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb0701c3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06815e860 con 0x7fb068068830 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb06d75e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0680c18a0 0x7fb0681626a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59802/0 (socket says 172.21.15.7:59802) 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb06df5f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0680ad9b0 0x7fb06806f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb06d75e700 1 -- 172.21.15.7:0/2421841160 learned_addr learned my addr 172.21.15.7:0/2421841160 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.034+0000 7fb06e760700 1 --2- 172.21.15.7:0/2421841160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb068068830 0x7fb068068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06cf5d700 1 -- 172.21.15.7:0/2421841160 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630209884 0 0) 0x7fb06806b490 con 0x7fb0680c18a0 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06cf5d700 1 -- 172.21.15.7:0/2421841160 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb054003570 con 0x7fb0680c18a0 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06df5f700 1 -- 172.21.15.7:0/2421841160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb068068830 msgr2=0x7fb068068c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06df5f700 1 --2- 172.21.15.7:0/2421841160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb068068830 0x7fb068068c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06df5f700 1 -- 172.21.15.7:0/2421841160 >> v1:172.21.15.7:6789/0 conn(0x7fb0680c18a0 legacy=0x7fb0681626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06df5f700 1 -- 172.21.15.7:0/2421841160 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680691b0 con 0x7fb0680ad9b0 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06df5f700 1 --2- 172.21.15.7:0/2421841160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb06806f6c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fb0640098e0 tx=0x7fb0640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5f0cc18d011229 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06cf5d700 1 -- 172.21.15.7:0/2421841160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06400b640 con 0x7fb0680ad9b0 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06cf5d700 1 -- 172.21.15.7:0/2421841160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06400b7a0 con 0x7fb0680ad9b0 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.035+0000 7fb06cf5d700 1 -- 172.21.15.7:0/2421841160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06400bb40 con 0x7fb0680ad9b0 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.036+0000 7fb0701c3700 1 -- 172.21.15.7:0/2421841160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 msgr2=0x7fb06806f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.036+0000 7fb0701c3700 1 --2- 172.21.15.7:0/2421841160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb06806f6c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fb0640098e0 tx=0x7fb0640091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.036+0000 7fb0701c3700 1 -- 172.21.15.7:0/2421841160 shutdown_connections 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.036+0000 7fb0701c3700 1 --2- 172.21.15.7:0/2421841160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb068068830 0x7fb068068c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.036+0000 7fb0701c3700 1 --2- 172.21.15.7:0/2421841160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb06806f6c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.036+0000 7fb0701c3700 1 -- 172.21.15.7:0/2421841160 >> 172.21.15.7:0/2421841160 conn(0x7fb068070690 msgr2=0x7fb068070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.036+0000 7fb0701c3700 1 -- 172.21.15.7:0/2421841160 shutdown_connections 2024-01-11T07:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.036+0000 7fb0701c3700 1 -- 172.21.15.7:0/2421841160 wait complete. 2024-01-11T07:13:50.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.037+0000 7fb0701c3700 1 Processor -- start 2024-01-11T07:13:50.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.037+0000 7fb0701c3700 1 -- start start 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb0701c3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb068068830 0x7fb0680cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb0701c3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb0680aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb0701c3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0680c18a0 0x7fb0680af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb0701c3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c2b10 con 0x7fb0680ad9b0 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb0701c3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c4f80 con 0x7fb068068830 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb0701c3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c5260 con 0x7fb0680c18a0 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb0680aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb0680aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41400/0 (socket says 172.21.15.7:41400) 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 -- 172.21.15.7:0/1589518114 learned_addr learned my addr 172.21.15.7:0/1589518114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06e760700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0680c18a0 0x7fb0680af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06df5f700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb068068830 0x7fb0680cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 -- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0680c18a0 msgr2=0x7fb0680af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0680c18a0 0x7fb0680af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 -- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb068068830 msgr2=0x7fb0680cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb068068830 0x7fb0680cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 -- 172.21.15.7:0/1589518114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb064009420 con 0x7fb0680ad9b0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06e760700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0680c18a0 0x7fb0680af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06df5f700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb068068830 0x7fb0680cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.038+0000 7fb06d75e700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb0680aecf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fb05800c790 tx=0x7fb058007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.039+0000 7fb05dffb700 1 -- 172.21.15.7:0/1589518114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058007300 con 0x7fb0680ad9b0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.039+0000 7fb05dffb700 1 -- 172.21.15.7:0/1589518114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058007460 con 0x7fb0680ad9b0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.039+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0680c24c0 con 0x7fb0680ad9b0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.039+0000 7fb05dffb700 1 -- 172.21.15.7:0/1589518114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580057f0 con 0x7fb0680ad9b0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.039+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06815c770 con 0x7fb0680ad9b0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.040+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06804c3c0 con 0x7fb0680ad9b0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.040+0000 7fb05dffb700 1 -- 172.21.15.7:0/1589518114 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb058005c20 con 0x7fb0680ad9b0 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.041+0000 7fb05dffb700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb04c066f30 0x7fb04c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.041+0000 7fb05dffb700 1 -- 172.21.15.7:0/1589518114 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb058084030 con 0x7fb0680ad9b0 2024-01-11T07:13:50.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.041+0000 7fb06df5f700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb04c066f30 0x7fb04c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.042+0000 7fb06df5f700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb04c066f30 0x7fb04c0693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb0640093f0 tx=0x7fb064017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.045+0000 7fb05dffb700 1 -- 172.21.15.7:0/1589518114 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0580512e0 con 0x7fb0680ad9b0 2024-01-11T07:13:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.247+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb068162db0 con 0x7fb0680ad9b0 2024-01-11T07:13:50.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.249+0000 7fb05dffb700 1 -- 172.21.15.7:0/1589518114 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb058054900 con 0x7fb0680ad9b0 2024-01-11T07:13:50.250 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:13:50.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb04c066f30 msgr2=0x7fb04c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb04c066f30 0x7fb04c0693e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb0640093f0 tx=0x7fb064017000 comp rx=0 tx=0).stop 2024-01-11T07:13:50.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 msgr2=0x7fb0680aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb0680aecf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fb05800c790 tx=0x7fb058007c80 comp rx=0 tx=0).stop 2024-01-11T07:13:50.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 shutdown_connections 2024-01-11T07:13:50.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb04c066f30 0x7fb04c0693e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb0680c18a0 0x7fb0680af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb068068830 0x7fb0680cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 --2- 172.21.15.7:0/1589518114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680ad9b0 0x7fb0680aecf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 >> 172.21.15.7:0/1589518114 conn(0x7fb068070690 msgr2=0x7fb06806ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:50.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 shutdown_connections 2024-01-11T07:13:50.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.253+0000 7fb0701c3700 1 -- 172.21.15.7:0/1589518114 wait complete. 2024-01-11T07:13:50.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:13:50.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.404+0000 7f58e4bc7700 1 Processor -- start 2024-01-11T07:13:50.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.405+0000 7f58e4bc7700 1 -- start start 2024-01-11T07:13:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58e4bc7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58e015db90 0x7f58e015df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58e4bc7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e00c3690 0x7f58e006e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58e4bc7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f58e006ec70 0x7f58e00c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58e4bc7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e015e4a0 con 0x7f58e015db90 2024-01-11T07:13:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58e4bc7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e006d3b0 con 0x7f58e006ec70 2024-01-11T07:13:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58e4bc7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e006d690 con 0x7f58e00c3690 2024-01-11T07:13:50.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58e015db90 0x7f58e015df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58e015db90 0x7f58e015df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41416/0 (socket says 172.21.15.7:41416) 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 -- 172.21.15.7:0/2418471337 learned_addr learned my addr 172.21.15.7:0/2418471337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58ded9d700 1 --2- 172.21.15.7:0/2418471337 >> v2:172.21.15.113:3300/0 conn(0x7f58e006ec70 0x7f58e00c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58ddd9b700 1 --2- 172.21.15.7:0/2418471337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e00c3690 0x7f58e006e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 -- 172.21.15.7:0/2418471337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e00c3690 msgr2=0x7f58e006e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 --2- 172.21.15.7:0/2418471337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e00c3690 0x7f58e006e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 -- 172.21.15.7:0/2418471337 >> v2:172.21.15.113:3300/0 conn(0x7f58e006ec70 msgr2=0x7f58e00c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 --2- 172.21.15.7:0/2418471337 >> v2:172.21.15.113:3300/0 conn(0x7f58e006ec70 0x7f58e00c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 -- 172.21.15.7:0/2418471337 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e006d970 con 0x7f58e015db90 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.406+0000 7f58de59c700 1 --2- 172.21.15.7:0/2418471337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e015db90 0x7f58e015df60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f58e00c3ea0 tx=0x7f58c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315384607ee25b4a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58dd59a700 1 -- 172.21.15.7:0/2418471337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c800b640 con 0x7f58e015db90 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58dd59a700 1 -- 172.21.15.7:0/2418471337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c800b7a0 con 0x7f58e015db90 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58dd59a700 1 -- 172.21.15.7:0/2418471337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c800bb60 con 0x7f58e015db90 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 -- 172.21.15.7:0/2418471337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e015db90 msgr2=0x7f58e015df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/2418471337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e015db90 0x7f58e015df60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f58e00c3ea0 tx=0x7f58c8009250 comp rx=0 tx=0).stop 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 -- 172.21.15.7:0/2418471337 shutdown_connections 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/2418471337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e015db90 0x7f58e015df60 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/2418471337 >> v2:172.21.15.113:3300/0 conn(0x7f58e006ec70 0x7f58e00c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/2418471337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e00c3690 0x7f58e006e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 -- 172.21.15.7:0/2418471337 >> 172.21.15.7:0/2418471337 conn(0x7f58e00cd070 msgr2=0x7f58e00cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 -- 172.21.15.7:0/2418471337 shutdown_connections 2024-01-11T07:13:50.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.407+0000 7f58e4bc7700 1 -- 172.21.15.7:0/2418471337 wait complete. 2024-01-11T07:13:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58e4bc7700 1 Processor -- start 2024-01-11T07:13:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58e4bc7700 1 -- start start 2024-01-11T07:13:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58e4bc7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58e006ec70 0x7f58e00b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58e4bc7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e00c3690 0x7f58e0099f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58e4bc7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e015db90 0x7f58e0098bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58e4bc7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00b19d0 con 0x7f58e00c3690 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58e4bc7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e009a4c0 con 0x7f58e006ec70 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58e4bc7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00b0dc0 con 0x7f58e015db90 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.409+0000 7f58ddd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e00c3690 0x7f58e0099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ddd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e00c3690 0x7f58e0099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41422/0 (socket says 172.21.15.7:41422) 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ddd9b700 1 -- 172.21.15.7:0/504365048 learned_addr learned my addr 172.21.15.7:0/504365048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58de59c700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58e006ec70 0x7f58e00b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ded9d700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e015db90 0x7f58e0098bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ddd9b700 1 -- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e015db90 msgr2=0x7f58e0098bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ddd9b700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e015db90 0x7f58e0098bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ddd9b700 1 -- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58e006ec70 msgr2=0x7f58e00b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ddd9b700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58e006ec70 0x7f58e00b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ddd9b700 1 -- 172.21.15.7:0/504365048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c80094f0 con 0x7f58e00c3690 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58ddd9b700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e00c3690 0x7f58e0099f80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f58d000ec80 tx=0x7f58d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58d67fc700 1 -- 172.21.15.7:0/504365048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000c850 con 0x7f58e00c3690 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58d67fc700 1 -- 172.21.15.7:0/504365048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d000c9b0 con 0x7f58e00c3690 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e00b10a0 con 0x7f58e00c3690 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58d67fc700 1 -- 172.21.15.7:0/504365048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000cd50 con 0x7f58e00c3690 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.410+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e00b1660 con 0x7f58e00c3690 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.411+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e01579d0 con 0x7f58e00c3690 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.412+0000 7f58d67fc700 1 -- 172.21.15.7:0/504365048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f58d0013680 con 0x7f58e00c3690 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.412+0000 7f58d67fc700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58c0066f30 0x7f58c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.412+0000 7f58d67fc700 1 -- 172.21.15.7:0/504365048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f58d00847d0 con 0x7f58e00c3690 2024-01-11T07:13:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.413+0000 7f58de59c700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58c0066f30 0x7f58c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.413+0000 7f58de59c700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58c0066f30 0x7f58c00693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f58e00c3ea0 tx=0x7f58c8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.417+0000 7f58d67fc700 1 -- 172.21.15.7:0/504365048 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58d0051a00 con 0x7f58e00c3690 2024-01-11T07:13:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.620+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58e00b1b10 con 0x7f58e00c3690 2024-01-11T07:13:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.621+0000 7f58d67fc700 1 -- 172.21.15.7:0/504365048 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58e00b1b10 con 0x7f58e00c3690 2024-01-11T07:13:50.621 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:13:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58c0066f30 msgr2=0x7f58c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58c0066f30 0x7f58c00693e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f58e00c3ea0 tx=0x7f58c8017000 comp rx=0 tx=0).stop 2024-01-11T07:13:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e00c3690 msgr2=0x7f58e0099f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e00c3690 0x7f58e0099f80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f58d000ec80 tx=0x7f58d00091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 shutdown_connections 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f58c0066f30 0x7f58c00693e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58e015db90 0x7f58e0098bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58e006ec70 0x7f58e00b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 --2- 172.21.15.7:0/504365048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e00c3690 0x7f58e0099f80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 >> 172.21.15.7:0/504365048 conn(0x7f58e00cd070 msgr2=0x7f58e00c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 shutdown_connections 2024-01-11T07:13:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.623+0000 7f58e4bc7700 1 -- 172.21.15.7:0/504365048 wait complete. 2024-01-11T07:13:50.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:13:50.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.771+0000 7fa49e0b8700 1 Processor -- start 2024-01-11T07:13:50.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.771+0000 7fa49e0b8700 1 -- start start 2024-01-11T07:13:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa49e0b8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa49815a280 0x7fa4980aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa49e0b8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49815d880 con 0x7fa4980ac5b0 2024-01-11T07:13:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa49e0b8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49806c5b0 con 0x7fa498063230 2024-01-11T07:13:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa49e0b8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49815a7f0 con 0x7fa49815a280 2024-01-11T07:13:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa497fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa4980ac5b0 0x7fa4980aca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59812/0 (socket says 172.21.15.7:59812) 2024-01-11T07:13:50.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa497fff700 1 -- 172.21.15.7:0/2567289905 learned_addr learned my addr 172.21.15.7:0/2567289905 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa496ffd700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa49815a280 0x7fa4980aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211543037 0 0) 0x7fa49815d880 con 0x7fa4980ac5b0 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.772+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa480003570 con 0x7fa4980ac5b0 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606384799 0 0) 0x7fa49806c5b0 con 0x7fa498063230 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa49815d880 con 0x7fa498063230 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1062436720 0 0) 0x7fa480003570 con 0x7fa4980ac5b0 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa49815a280 msgr2=0x7fa4980aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa49815a280 0x7fa4980aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 >> v1:172.21.15.113:6789/0 conn(0x7fa498063230 legacy=0x7fa498063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49815aad0 con 0x7fa4980ac5b0 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa4900017f0 con 0x7fa4980ac5b0 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 >> v1:172.21.15.7:6789/0 conn(0x7fa4980ac5b0 legacy=0x7fa4980aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa49815a280 0x7fa480003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa480004180 0x7fa480004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004d50 0x7fa4800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49815a7f0 con 0x7fa480004180 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48000bba0 con 0x7fa480004d50 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48000bd60 con 0x7fa49815a280 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa496ffd700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa480004180 0x7fa480004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa4977fe700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa49815a280 0x7fa480003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa497fff700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004d50 0x7fa4800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa496ffd700 1 -- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa49815a280 msgr2=0x7fa480003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa496ffd700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa49815a280 0x7fa480003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa496ffd700 1 -- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004d50 msgr2=0x7fa4800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa496ffd700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004d50 0x7fa4800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa496ffd700 1 -- 172.21.15.7:0/2567289905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa480003570 con 0x7fa480004180 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa497fff700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004d50 0x7fa4800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.773+0000 7fa496ffd700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa480004180 0x7fa480004810 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009c00 tx=0x7fa48c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e7a8e49e661786 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00e710 con 0x7fa480004180 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c00e870 con 0x7fa480004180 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa4967fc700 1 -- 172.21.15.7:0/2567289905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00ec30 con 0x7fa480004180 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa49e0b8700 1 -- 172.21.15.7:0/2567289905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa480004180 msgr2=0x7fa480004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa49e0b8700 1 --2- 172.21.15.7:0/2567289905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa480004180 0x7fa480004810 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009c00 tx=0x7fa48c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa497fff700 1 -- 172.21.15.7:0/2567289905 reap_dead start 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa49e0b8700 1 -- 172.21.15.7:0/2567289905 shutdown_connections 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa49e0b8700 1 -- 172.21.15.7:0/2567289905 >> 172.21.15.7:0/2567289905 conn(0x7fa4980680e0 msgr2=0x7fa4980cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa49e0b8700 1 -- 172.21.15.7:0/2567289905 shutdown_connections 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.774+0000 7fa49e0b8700 1 -- 172.21.15.7:0/2567289905 wait complete. 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa49e0b8700 1 Processor -- start 2024-01-11T07:13:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa49e0b8700 1 -- start start 2024-01-11T07:13:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa49e0b8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004180 0x7fa498153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa49e0b8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4980b2080 0x7fa498152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa49e0b8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4980c3220 0x7fa4980c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa49e0b8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980c3c00 con 0x7fa4980c3220 2024-01-11T07:13:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa49e0b8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980ad9e0 con 0x7fa480004180 2024-01-11T07:13:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa49e0b8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa498153730 con 0x7fa4980b2080 2024-01-11T07:13:50.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.776+0000 7fa497fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4980c3220 0x7fa4980c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa497fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4980c3220 0x7fa4980c36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41442/0 (socket says 172.21.15.7:41442) 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa497fff700 1 -- 172.21.15.7:0/412169580 learned_addr learned my addr 172.21.15.7:0/412169580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa4977fe700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004180 0x7fa498153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa496ffd700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4980b2080 0x7fa498152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa497fff700 1 -- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4980b2080 msgr2=0x7fa498152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa497fff700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4980b2080 0x7fa498152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa497fff700 1 -- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004180 msgr2=0x7fa498153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa497fff700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004180 0x7fa498153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa497fff700 1 -- 172.21.15.7:0/412169580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980ad390 con 0x7fa4980c3220 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa496ffd700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4980b2080 0x7fa498152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa497fff700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4980c3220 0x7fa4980c36c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa49806d820 tx=0x7fa490001fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa47f7fe700 1 -- 172.21.15.7:0/412169580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4900012e0 con 0x7fa4980c3220 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa47f7fe700 1 -- 172.21.15.7:0/412169580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49000ebc0 con 0x7fa4980c3220 2024-01-11T07:13:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4980ad670 con 0x7fa4980c3220 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa47f7fe700 1 -- 172.21.15.7:0/412169580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490013420 con 0x7fa4980c3220 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.777+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49815ddf0 con 0x7fa4980c3220 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.778+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4980ad1f0 con 0x7fa4980c3220 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.779+0000 7fa47f7fe700 1 -- 172.21.15.7:0/412169580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa490013810 con 0x7fa4980c3220 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.779+0000 7fa47f7fe700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa484066f30 0x7fa4840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.779+0000 7fa47f7fe700 1 -- 172.21.15.7:0/412169580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fa490084ae0 con 0x7fa4980c3220 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.779+0000 7fa4977fe700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa484066f30 0x7fa4840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.780+0000 7fa4977fe700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa484066f30 0x7fa4840693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa488003d30 tx=0x7fa488009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.784+0000 7fa47f7fe700 1 -- 172.21.15.7:0/412169580 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4980ad1f0 con 0x7fa4980c3220 2024-01-11T07:13:50.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.988+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4980ad1f0 con 0x7fa4980c3220 2024-01-11T07:13:50.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.988+0000 7fa47f7fe700 1 -- 172.21.15.7:0/412169580 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4980ad1f0 con 0x7fa4980c3220 2024-01-11T07:13:50.989 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:13:50.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa484066f30 msgr2=0x7fa4840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa484066f30 0x7fa4840693e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa488003d30 tx=0x7fa488009040 comp rx=0 tx=0).stop 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4980c3220 msgr2=0x7fa4980c36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4980c3220 0x7fa4980c36c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa49806d820 tx=0x7fa490001fc0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 shutdown_connections 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa484066f30 0x7fa4840693e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4980c3220 0x7fa4980c36c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa480004180 0x7fa498153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 --2- 172.21.15.7:0/412169580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4980b2080 0x7fa498152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.990+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 >> 172.21.15.7:0/412169580 conn(0x7fa4980680e0 msgr2=0x7fa4980c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.991+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 shutdown_connections 2024-01-11T07:13:50.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:50.991+0000 7fa49e0b8700 1 -- 172.21.15.7:0/412169580 wait complete. 2024-01-11T07:13:51.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.136+0000 7fb6c5ab3700 1 Processor -- start 2024-01-11T07:13:51.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.136+0000 7fb6c5ab3700 1 -- start start 2024-01-11T07:13:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.136+0000 7fb6c5ab3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6c006dd00 0x7fb6c00c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6c5ab3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c015b770 con 0x7fb6c006e0d0 2024-01-11T07:13:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6c5ab3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c015a620 con 0x7fb6c015a1c0 2024-01-11T07:13:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6c5ab3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c015abf0 con 0x7fb6c006dd00 2024-01-11T07:13:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6bf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6c006dd00 0x7fb6c00c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6beffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb6c006e0d0 0x7fb6c0159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59820/0 (socket says 172.21.15.7:59820) 2024-01-11T07:13:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6beffd700 1 -- 172.21.15.7:0/1287961085 learned_addr learned my addr 172.21.15.7:0/1287961085 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6bf7fe700 1 -- 172.21.15.7:0/1287961085 >> v1:172.21.15.113:6789/0 conn(0x7fb6c015a1c0 legacy=0x7fb6c0069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6bf7fe700 1 -- 172.21.15.7:0/1287961085 >> v1:172.21.15.7:6789/0 conn(0x7fb6c006e0d0 legacy=0x7fb6c0159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6bf7fe700 1 -- 172.21.15.7:0/1287961085 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c015aed0 con 0x7fb6c006dd00 2024-01-11T07:13:51.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6bf7fe700 1 --2- 172.21.15.7:0/1287961085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006dd00 0x7fb6c00c98d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009e60 tx=0x7fb6b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86038dc78e96672d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6be7fc700 1 -- 172.21.15.7:0/1287961085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000e070 con 0x7fb6c006dd00 2024-01-11T07:13:51.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6be7fc700 1 -- 172.21.15.7:0/1287961085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b000bc30 con 0x7fb6c006dd00 2024-01-11T07:13:51.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.137+0000 7fb6be7fc700 1 -- 172.21.15.7:0/1287961085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0012430 con 0x7fb6c006dd00 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.138+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/1287961085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006dd00 msgr2=0x7fb6c00c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.138+0000 7fb6c5ab3700 1 --2- 172.21.15.7:0/1287961085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006dd00 0x7fb6c00c98d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009e60 tx=0x7fb6b0009780 comp rx=0 tx=0).stop 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.138+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/1287961085 shutdown_connections 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.138+0000 7fb6c5ab3700 1 --2- 172.21.15.7:0/1287961085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006dd00 0x7fb6c00c98d0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.138+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/1287961085 >> 172.21.15.7:0/1287961085 conn(0x7fb6c00680e0 msgr2=0x7fb6c00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.138+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/1287961085 shutdown_connections 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.138+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/1287961085 wait complete. 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.139+0000 7fb6c5ab3700 1 Processor -- start 2024-01-11T07:13:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6c5ab3700 1 -- start start 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6c5ab3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c006dd00 0x7fb6c00c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6c5ab3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006e0d0 0x7fb6c00adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6c5ab3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c015a1c0 0x7fb6c0156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6c5ab3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c016daf0 con 0x7fb6c006e0d0 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6c5ab3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00ae520 con 0x7fb6c015a1c0 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6c5ab3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c016dd40 con 0x7fb6c006dd00 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6beffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006e0d0 0x7fb6c00adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6beffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006e0d0 0x7fb6c00adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41464/0 (socket says 172.21.15.7:41464) 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.140+0000 7fb6beffd700 1 -- 172.21.15.7:0/2206786553 learned_addr learned my addr 172.21.15.7:0/2206786553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6bffff700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c015a1c0 0x7fb6c0156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6bf7fe700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c006dd00 0x7fb6c00c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6beffd700 1 -- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c006dd00 msgr2=0x7fb6c00c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6beffd700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c006dd00 0x7fb6c00c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6beffd700 1 -- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c015a1c0 msgr2=0x7fb6c0156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6beffd700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c015a1c0 0x7fb6c0156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6beffd700 1 -- 172.21.15.7:0/2206786553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0009a20 con 0x7fb6c006e0d0 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6bf7fe700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c006dd00 0x7fb6c00c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6beffd700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006e0d0 0x7fb6c00adfe0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c7c0 tx=0x7fb6b4007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6a77fe700 1 -- 172.21.15.7:0/2206786553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4005b30 con 0x7fb6c006e0d0 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6a77fe700 1 -- 172.21.15.7:0/2206786553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4005c90 con 0x7fb6c006e0d0 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c016e020 con 0x7fb6c006e0d0 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6a77fe700 1 -- 172.21.15.7:0/2206786553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4007320 con 0x7fb6c006e0d0 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.141+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c016e5e0 con 0x7fb6c006e0d0 2024-01-11T07:13:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.142+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c004c3c0 con 0x7fb6c006e0d0 2024-01-11T07:13:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.143+0000 7fb6a77fe700 1 -- 172.21.15.7:0/2206786553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb6b4007690 con 0x7fb6c006e0d0 2024-01-11T07:13:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.143+0000 7fb6a77fe700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6ac066f30 0x7fb6ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.143+0000 7fb6a77fe700 1 -- 172.21.15.7:0/2206786553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb6b4083b50 con 0x7fb6c006e0d0 2024-01-11T07:13:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.143+0000 7fb6bf7fe700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6ac066f30 0x7fb6ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.144+0000 7fb6bf7fe700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6ac066f30 0x7fb6ac0693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009e60 tx=0x7fb6b0019000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.148+0000 7fb6a77fe700 1 -- 172.21.15.7:0/2206786553 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6b4050ce0 con 0x7fb6c006e0d0 2024-01-11T07:13:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.397+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6c015b1b0 con 0x7fb6c006e0d0 2024-01-11T07:13:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.398+0000 7fb6a77fe700 1 -- 172.21.15.7:0/2206786553 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fb6b4017020 con 0x7fb6c006e0d0 2024-01-11T07:13:51.399 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:51.399 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6ac066f30 msgr2=0x7fb6ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6ac066f30 0x7fb6ac0693e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009e60 tx=0x7fb6b0019000 comp rx=0 tx=0).stop 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006e0d0 msgr2=0x7fb6c00adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006e0d0 0x7fb6c00adfe0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c7c0 tx=0x7fb6b4007f30 comp rx=0 tx=0).stop 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 shutdown_connections 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb6ac066f30 0x7fb6ac0693e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6c015a1c0 0x7fb6c0156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6c006dd00 0x7fb6c00c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 --2- 172.21.15.7:0/2206786553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6c006e0d0 0x7fb6c00adfe0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 >> 172.21.15.7:0/2206786553 conn(0x7fb6c00680e0 msgr2=0x7fb6c00cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:51.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 shutdown_connections 2024-01-11T07:13:51.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.400+0000 7fb6c5ab3700 1 -- 172.21.15.7:0/2206786553 wait complete. 2024-01-11T07:13:51.402 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:51.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:51.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.547+0000 7f067e2c2700 1 Processor -- start 2024-01-11T07:13:51.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.547+0000 7f067e2c2700 1 -- start start 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.548+0000 7f067e2c2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780cafb0 0x7f06780ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.548+0000 7f067e2c2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0678158280 0x7f06780b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.548+0000 7f067e2c2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0678159300 con 0x7f06780b0970 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.548+0000 7f067e2c2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780b0df0 con 0x7f0678158280 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.548+0000 7f067e2c2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06780cf260 con 0x7f06780cafb0 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f0677fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780cafb0 0x7f06780ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f06777fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0678158280 0x7f06780b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f067c85f700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f06780b0970 0x7f0678159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:58878/0 (socket says 172.21.15.7:58878) 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f06777fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0678158280 0x7f06780b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:48696/0 (socket says 172.21.15.7:48696) 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f067c85f700 1 -- 172.21.15.7:0/2382603062 learned_addr learned my addr 172.21.15.7:0/2382603062 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f0677fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780cafb0 0x7f06780ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:53202/0 (socket says 172.21.15.7:53202) 2024-01-11T07:13:51.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f06777fe700 1 -- 172.21.15.7:0/2382603062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780cafb0 msgr2=0x7f06780ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f06777fe700 1 --2- 172.21.15.7:0/2382603062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780cafb0 0x7f06780ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f06777fe700 1 -- 172.21.15.7:0/2382603062 >> v1:172.21.15.113:6789/0 conn(0x7f06780b0970 legacy=0x7f0678159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f06777fe700 1 -- 172.21.15.7:0/2382603062 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780cf540 con 0x7f0678158280 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f0676ffd700 1 -- 172.21.15.7:0/2382603062 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888553433 0 0) 0x7f0678159300 con 0x7f06780b0970 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f06777fe700 1 --2- 172.21.15.7:0/2382603062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 0x7f06780b0430 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f066c0098e0 tx=0x7f066c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba311658f0bae7e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f0676ffd700 1 -- 172.21.15.7:0/2382603062 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00b640 con 0x7f0678158280 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.549+0000 7f0676ffd700 1 -- 172.21.15.7:0/2382603062 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c00b7a0 con 0x7f0678158280 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f0676ffd700 1 -- 172.21.15.7:0/2382603062 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00bb40 con 0x7f0678158280 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f067e2c2700 1 -- 172.21.15.7:0/2382603062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 msgr2=0x7f06780b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2382603062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 0x7f06780b0430 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f066c0098e0 tx=0x7f066c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f067e2c2700 1 -- 172.21.15.7:0/2382603062 shutdown_connections 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2382603062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780cafb0 0x7f06780ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2382603062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 0x7f06780b0430 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f067e2c2700 1 -- 172.21.15.7:0/2382603062 >> 172.21.15.7:0/2382603062 conn(0x7f0678071100 msgr2=0x7f067806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f067e2c2700 1 -- 172.21.15.7:0/2382603062 shutdown_connections 2024-01-11T07:13:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.550+0000 7f067e2c2700 1 -- 172.21.15.7:0/2382603062 wait complete. 2024-01-11T07:13:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.551+0000 7f067e2c2700 1 Processor -- start 2024-01-11T07:13:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.552+0000 7f067e2c2700 1 -- start start 2024-01-11T07:13:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.552+0000 7f067e2c2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780b0970 0x7f067814a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.552+0000 7f067e2c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06780cafb0 0x7f06780a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.552+0000 7f067e2c2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 0x7f06780af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.552+0000 7f067e2c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06780a8570 con 0x7f06780cafb0 2024-01-11T07:13:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.552+0000 7f067e2c2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06780a8850 con 0x7f0678158280 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.552+0000 7f067e2c2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06780a8b30 con 0x7f06780b0970 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06780cafb0 0x7f06780a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06780cafb0 0x7f06780a9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41480/0 (socket says 172.21.15.7:41480) 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 -- 172.21.15.7:0/2418025499 learned_addr learned my addr 172.21.15.7:0/2418025499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f067c85f700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 0x7f06780af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f0677fff700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780b0970 0x7f067814a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 -- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780b0970 msgr2=0x7f067814a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780b0970 0x7f067814a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 -- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 msgr2=0x7f06780af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 0x7f06780af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 -- 172.21.15.7:0/2418025499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c009420 con 0x7f06780cafb0 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f067c85f700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 0x7f06780af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f06777fe700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06780cafb0 0x7f06780a9ca0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f066c013000 tx=0x7f066c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f065ffff700 1 -- 172.21.15.7:0/2418025499 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00bb20 con 0x7f06780cafb0 2024-01-11T07:13:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f065ffff700 1 -- 172.21.15.7:0/2418025499 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c011420 con 0x7f06780cafb0 2024-01-11T07:13:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06780a8e10 con 0x7f06780cafb0 2024-01-11T07:13:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f065ffff700 1 -- 172.21.15.7:0/2418025499 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0117c0 con 0x7f06780cafb0 2024-01-11T07:13:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.553+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067814a950 con 0x7f06780cafb0 2024-01-11T07:13:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.554+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06780abce0 con 0x7f06780cafb0 2024-01-11T07:13:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.555+0000 7f065ffff700 1 -- 172.21.15.7:0/2418025499 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f066c011b70 con 0x7f06780cafb0 2024-01-11T07:13:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.555+0000 7f065ffff700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0664066f30 0x7f06640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.555+0000 7f065ffff700 1 -- 172.21.15.7:0/2418025499 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f066c085b20 con 0x7f06780cafb0 2024-01-11T07:13:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.555+0000 7f0677fff700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0664066f30 0x7f06640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.556+0000 7f0677fff700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0664066f30 0x7f06640693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0668009d40 tx=0x7f0668008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.560+0000 7f065ffff700 1 -- 172.21.15.7:0/2418025499 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f066c052d50 con 0x7f06780cafb0 2024-01-11T07:13:51.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.763+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06780ab980 con 0x7f06780cafb0 2024-01-11T07:13:51.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.764+0000 7f065ffff700 1 -- 172.21.15.7:0/2418025499 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+15420 (secure 0 0 0) 0x7f06780ab980 con 0x7f06780cafb0 2024-01-11T07:13:51.765 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:51.765 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:13:50.246126+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:13:15.625657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-11T07:13:16.691544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.f","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:51.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0664066f30 msgr2=0x7f06640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0664066f30 0x7f06640693e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0668009d40 tx=0x7f0668008040 comp rx=0 tx=0).stop 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06780cafb0 msgr2=0x7f06780a9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06780cafb0 0x7f06780a9ca0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f066c013000 tx=0x7f066c017000 comp rx=0 tx=0).stop 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 shutdown_connections 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0664066f30 0x7f06640693e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0678158280 0x7f06780af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06780b0970 0x7f067814a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 --2- 172.21.15.7:0/2418025499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06780cafb0 0x7f06780a9ca0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.766+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 >> 172.21.15.7:0/2418025499 conn(0x7f0678071100 msgr2=0x7f0678155440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.767+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 shutdown_connections 2024-01-11T07:13:51.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.767+0000 7f067e2c2700 1 -- 172.21.15.7:0/2418025499 wait complete. 2024-01-11T07:13:51.782 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:13:51.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.914+0000 7fbe57260700 1 Processor -- start 2024-01-11T07:13:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.914+0000 7fbe57260700 1 -- start start 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.914+0000 7fbe57260700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe50159ca0 0x7fbe5006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.914+0000 7fbe57260700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe5015a070 0x7fbe5006e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.914+0000 7fbe57260700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe500ce7f0 0x7fbe500cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.914+0000 7fbe57260700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe500d18c0 con 0x7fbe5015a070 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.914+0000 7fbe57260700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe500d0010 con 0x7fbe50159ca0 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.914+0000 7fbe57260700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500cf150 con 0x7fbe500ce7f0 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe5015a070 0x7fbe5006e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe5015a070 0x7fbe5006e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41496/0 (socket says 172.21.15.7:41496) 2024-01-11T07:13:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 -- 172.21.15.7:0/2846974514 learned_addr learned my addr 172.21.15.7:0/2846974514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe54ffc700 1 --2- 172.21.15.7:0/2846974514 >> v2:172.21.15.113:3300/0 conn(0x7fbe50159ca0 0x7fbe5006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe557fd700 1 --2- 172.21.15.7:0/2846974514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe500ce7f0 0x7fbe500cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 -- 172.21.15.7:0/2846974514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe500ce7f0 msgr2=0x7fbe500cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 --2- 172.21.15.7:0/2846974514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe500ce7f0 0x7fbe500cec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 -- 172.21.15.7:0/2846974514 >> v2:172.21.15.113:3300/0 conn(0x7fbe50159ca0 msgr2=0x7fbe5006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 --2- 172.21.15.7:0/2846974514 >> v2:172.21.15.113:3300/0 conn(0x7fbe50159ca0 0x7fbe5006dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 -- 172.21.15.7:0/2846974514 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500cf400 con 0x7fbe5015a070 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4ffff700 1 --2- 172.21.15.7:0/2846974514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5015a070 0x7fbe5006e170 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbe50068aa0 tx=0x7fbe3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc9de1e3f064d19 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4f7fe700 1 -- 172.21.15.7:0/2846974514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00b860 con 0x7fbe5015a070 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4f7fe700 1 -- 172.21.15.7:0/2846974514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c00b9c0 con 0x7fbe5015a070 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.915+0000 7fbe4f7fe700 1 -- 172.21.15.7:0/2846974514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00bd60 con 0x7fbe5015a070 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 -- 172.21.15.7:0/2846974514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5015a070 msgr2=0x7fbe5006e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 --2- 172.21.15.7:0/2846974514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5015a070 0x7fbe5006e170 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbe50068aa0 tx=0x7fbe3c009250 comp rx=0 tx=0).stop 2024-01-11T07:13:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 -- 172.21.15.7:0/2846974514 shutdown_connections 2024-01-11T07:13:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 --2- 172.21.15.7:0/2846974514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe500ce7f0 0x7fbe500cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 --2- 172.21.15.7:0/2846974514 >> v2:172.21.15.113:3300/0 conn(0x7fbe50159ca0 0x7fbe5006dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 --2- 172.21.15.7:0/2846974514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5015a070 0x7fbe5006e170 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 -- 172.21.15.7:0/2846974514 >> 172.21.15.7:0/2846974514 conn(0x7fbe50063cc0 msgr2=0x7fbe500d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 -- 172.21.15.7:0/2846974514 shutdown_connections 2024-01-11T07:13:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.916+0000 7fbe57260700 1 -- 172.21.15.7:0/2846974514 wait complete. 2024-01-11T07:13:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.917+0000 7fbe57260700 1 Processor -- start 2024-01-11T07:13:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.918+0000 7fbe57260700 1 -- start start 2024-01-11T07:13:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.918+0000 7fbe57260700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe500ce7f0 0x7fbe5015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.918+0000 7fbe57260700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe50159ca0 0x7fbe5016a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.918+0000 7fbe57260700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe5015a070 0x7fbe500b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.918+0000 7fbe57260700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5016ba80 con 0x7fbe50159ca0 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.918+0000 7fbe57260700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500b1200 con 0x7fbe500ce7f0 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.918+0000 7fbe57260700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500c8840 con 0x7fbe5015a070 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe50159ca0 0x7fbe5016a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe50159ca0 0x7fbe5016a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41500/0 (socket says 172.21.15.7:41500) 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 -- 172.21.15.7:0/2954862157 learned_addr learned my addr 172.21.15.7:0/2954862157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe54ffc700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe500ce7f0 0x7fbe5015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe557fd700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe5015a070 0x7fbe500b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 -- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe5015a070 msgr2=0x7fbe500b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe5015a070 0x7fbe500b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 -- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe500ce7f0 msgr2=0x7fbe5015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe500ce7f0 0x7fbe5015cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 -- 172.21.15.7:0/2954862157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0094f0 con 0x7fbe50159ca0 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4ffff700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe50159ca0 0x7fbe5016a870 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c006fd0 tx=0x7fbe3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4cff9700 1 -- 172.21.15.7:0/2954862157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c024440 con 0x7fbe50159ca0 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4cff9700 1 -- 172.21.15.7:0/2954862157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c0245a0 con 0x7fbe50159ca0 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe4cff9700 1 -- 172.21.15.7:0/2954862157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c024960 con 0x7fbe50159ca0 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5016b1e0 con 0x7fbe50159ca0 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.919+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5016b7a0 con 0x7fbe50159ca0 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.920+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe50089960 con 0x7fbe50159ca0 2024-01-11T07:13:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.921+0000 7fbe4cff9700 1 -- 172.21.15.7:0/2954862157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbe3c01a420 con 0x7fbe50159ca0 2024-01-11T07:13:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.921+0000 7fbe4cff9700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbe34066f30 0x7fbe340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.921+0000 7fbe4cff9700 1 -- 172.21.15.7:0/2954862157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fbe3c0863a0 con 0x7fbe50159ca0 2024-01-11T07:13:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.921+0000 7fbe54ffc700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbe34066f30 0x7fbe340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.922+0000 7fbe54ffc700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbe34066f30 0x7fbe340693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbe44006fd0 tx=0x7fbe44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:51.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:51.926+0000 7fbe4cff9700 1 -- 172.21.15.7:0/2954862157 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe3c0535d0 con 0x7fbe50159ca0 2024-01-11T07:13:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.180+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe5004c3c0 con 0x7fbe50159ca0 2024-01-11T07:13:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.181+0000 7fbe4cff9700 1 -- 172.21.15.7:0/2954862157 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fbe3c056bf0 con 0x7fbe50159ca0 2024-01-11T07:13:52.181 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:52.182 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbe34066f30 msgr2=0x7fbe340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbe34066f30 0x7fbe340693e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbe44006fd0 tx=0x7fbe44008040 comp rx=0 tx=0).stop 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe50159ca0 msgr2=0x7fbe5016a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe50159ca0 0x7fbe5016a870 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c006fd0 tx=0x7fbe3c017000 comp rx=0 tx=0).stop 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 shutdown_connections 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbe34066f30 0x7fbe340693e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe5015a070 0x7fbe500b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe500ce7f0 0x7fbe5015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 --2- 172.21.15.7:0/2954862157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe50159ca0 0x7fbe5016a870 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 >> 172.21.15.7:0/2954862157 conn(0x7fbe50063cc0 msgr2=0x7fbe50155da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:52.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 shutdown_connections 2024-01-11T07:13:52.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.183+0000 7fbe57260700 1 -- 172.21.15.7:0/2954862157 wait complete. 2024-01-11T07:13:52.185 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:52.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:52.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.330+0000 7f4c59b7e700 1 Processor -- start 2024-01-11T07:13:52.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.330+0000 7f4c59b7e700 1 -- start start 2024-01-11T07:13:52.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c59b7e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c5415c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c59b7e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4c540ce3f0 0x7f4c540ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c59b7e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5406d270 con 0x7f4c54157da0 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c59b7e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c540cda40 con 0x7f4c540ce3f0 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c59b7e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540cd5b0 con 0x7f4c541579d0 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c537fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c5415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c53fff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4c540ce3f0 0x7f4c540ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c537fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c5415c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:53242/0 (socket says 172.21.15.7:53242) 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c537fe700 1 -- 172.21.15.7:0/546374749 learned_addr learned my addr 172.21.15.7:0/546374749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.331+0000 7f4c52ffd700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4c54157da0 0x7f4c540cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:58892/0 (socket says 172.21.15.7:58892) 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c53fff700 1 -- 172.21.15.7:0/546374749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 msgr2=0x7f4c5415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c53fff700 1 --2- 172.21.15.7:0/546374749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c5415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c53fff700 1 -- 172.21.15.7:0/546374749 >> v1:172.21.15.113:6789/0 conn(0x7f4c54157da0 legacy=0x7f4c540cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c527fc700 1 -- 172.21.15.7:0/546374749 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107187166 0 0) 0x7f4c5406d270 con 0x7f4c54157da0 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c53fff700 1 -- 172.21.15.7:0/546374749 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540cc9a0 con 0x7f4c540ce3f0 2024-01-11T07:13:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c537fe700 1 --2- 172.21.15.7:0/546374749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c5415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c53fff700 1 --2- 172.21.15.7:0/546374749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c540ce3f0 0x7f4c540ca500 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009e60 tx=0x7f4c4c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c4eba3e382cc81 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c527fc700 1 -- 172.21.15.7:0/546374749 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00e070 con 0x7f4c540ce3f0 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c527fc700 1 -- 172.21.15.7:0/546374749 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c00bb60 con 0x7f4c540ce3f0 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.332+0000 7f4c527fc700 1 -- 172.21.15.7:0/546374749 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c012430 con 0x7f4c540ce3f0 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.333+0000 7f4c59b7e700 1 -- 172.21.15.7:0/546374749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c540ce3f0 msgr2=0x7f4c540ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.333+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/546374749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c540ce3f0 0x7f4c540ca500 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009e60 tx=0x7f4c4c009780 comp rx=0 tx=0).stop 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.333+0000 7f4c59b7e700 1 -- 172.21.15.7:0/546374749 shutdown_connections 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.333+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/546374749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c540ce3f0 0x7f4c540ca500 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.333+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/546374749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c5415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.333+0000 7f4c59b7e700 1 -- 172.21.15.7:0/546374749 >> 172.21.15.7:0/546374749 conn(0x7f4c5406ae00 msgr2=0x7f4c5406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.333+0000 7f4c59b7e700 1 -- 172.21.15.7:0/546374749 shutdown_connections 2024-01-11T07:13:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.333+0000 7f4c59b7e700 1 -- 172.21.15.7:0/546374749 wait complete. 2024-01-11T07:13:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.334+0000 7f4c59b7e700 1 Processor -- start 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.334+0000 7f4c59b7e700 1 -- start start 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c59b7e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c540ce3f0 0x7f4c54158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c59b7e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c54152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c59b7e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54157da0 0x7f4c54156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c59b7e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c541586b0 con 0x7f4c540ce3f0 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c59b7e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540b3750 con 0x7f4c54157da0 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c59b7e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540694e0 con 0x7f4c541579d0 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c540ce3f0 0x7f4c54158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c540ce3f0 0x7f4c54158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41514/0 (socket says 172.21.15.7:41514) 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c537fe700 1 -- 172.21.15.7:0/1506077933 learned_addr learned my addr 172.21.15.7:0/1506077933 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c53fff700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54157da0 0x7f4c54156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c52ffd700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c54152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c537fe700 1 -- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 msgr2=0x7f4c54152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c537fe700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c54152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c537fe700 1 -- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54157da0 msgr2=0x7f4c54156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.335+0000 7f4c537fe700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54157da0 0x7f4c54156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.336+0000 7f4c537fe700 1 -- 172.21.15.7:0/1506077933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440094f0 con 0x7f4c540ce3f0 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.336+0000 7f4c52ffd700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c54152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.336+0000 7f4c537fe700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c540ce3f0 0x7f4c54158170 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400ec50 tx=0x7f4c44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.336+0000 7f4c3b7fe700 1 -- 172.21.15.7:0/1506077933 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400c920 con 0x7f4c540ce3f0 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.336+0000 7f4c3b7fe700 1 -- 172.21.15.7:0/1506077933 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4400ca80 con 0x7f4c540ce3f0 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.336+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c009a20 con 0x7f4c540ce3f0 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.336+0000 7f4c3b7fe700 1 -- 172.21.15.7:0/1506077933 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400ce00 con 0x7f4c540ce3f0 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.336+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c540b2b40 con 0x7f4c540ce3f0 2024-01-11T07:13:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.337+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c54159c40 con 0x7f4c540ce3f0 2024-01-11T07:13:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.337+0000 7f4c3b7fe700 1 -- 172.21.15.7:0/1506077933 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4c44013680 con 0x7f4c540ce3f0 2024-01-11T07:13:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.338+0000 7f4c3b7fe700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c40066f60 0x7f4c40069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.338+0000 7f4c3b7fe700 1 -- 172.21.15.7:0/1506077933 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4c440848c0 con 0x7f4c540ce3f0 2024-01-11T07:13:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.338+0000 7f4c3b7fe700 1 -- 172.21.15.7:0/1506077933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f4c4006eb40 con 0x7f4c540ce3f0 2024-01-11T07:13:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.338+0000 7f4c52ffd700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c40066f60 0x7f4c40069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.339+0000 7f4c52ffd700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c40066f60 0x7f4c40069410 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4c48007fd0 tx=0x7f4c48007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.342+0000 7f4c3b7fe700 1 -- 172.21.15.7:0/1506077933 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c54159c40 con 0x7f4c540ce3f0 2024-01-11T07:13:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.595+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c54159c40 con 0x7f4c540ce3f0 2024-01-11T07:13:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.595+0000 7f4c3b7fe700 1 -- 172.21.15.7:0/1506077933 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f4c54159c40 con 0x7f4c540ce3f0 2024-01-11T07:13:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:52.596 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:13:17.725345+0000","modified":"2024-01-11T07:13:21.305111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"c","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3803559903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3803559903},{"type":"v1","addr":"172.21.15.7:6835","nonce":3803559903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2024-01-11T07:13:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.597+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c40066f60 msgr2=0x7f4c40069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.597+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c40066f60 0x7f4c40069410 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4c48007fd0 tx=0x7f4c48007e30 comp rx=0 tx=0).stop 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.597+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c540ce3f0 msgr2=0x7f4c54158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.597+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c540ce3f0 0x7f4c54158170 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400ec50 tx=0x7f4c44009250 comp rx=0 tx=0).stop 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.598+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 shutdown_connections 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.598+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4c40066f60 0x7f4c40069410 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.598+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c54157da0 0x7f4c54156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.598+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c540ce3f0 0x7f4c54158170 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.598+0000 7f4c59b7e700 1 --2- 172.21.15.7:0/1506077933 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4c541579d0 0x7f4c54152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.598+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 >> 172.21.15.7:0/1506077933 conn(0x7f4c5406ae00 msgr2=0x7f4c540c3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.598+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 shutdown_connections 2024-01-11T07:13:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.598+0000 7f4c59b7e700 1 -- 172.21.15.7:0/1506077933 wait complete. 2024-01-11T07:13:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2024-01-11T07:13:52.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:13:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.744+0000 7f0f2e6c6700 1 Processor -- start 2024-01-11T07:13:52.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.744+0000 7f0f2e6c6700 1 -- start start 2024-01-11T07:13:52.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f2e6c6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f281579b0 0x7f0f2815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f2e6c6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f28063940 0x7f0f28069c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f2e6c6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f280aeab0 con 0x7f0f28157d80 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f2e6c6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f280aed90 con 0x7f0f281579b0 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f2e6c6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280af070 con 0x7f0f28063940 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f27fff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f281579b0 0x7f0f2815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f2cc63700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f28063940 0x7f0f28069c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f277fe700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0f28157d80 0x7f0f28063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:58900/0 (socket says 172.21.15.7:58900) 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f277fe700 1 -- 172.21.15.7:0/4081909581 learned_addr learned my addr 172.21.15.7:0/4081909581 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f26ffd700 1 -- 172.21.15.7:0/4081909581 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898640585 0 0) 0x7f0f280aeab0 con 0x7f0f28157d80 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f26ffd700 1 -- 172.21.15.7:0/4081909581 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f10003570 con 0x7f0f28157d80 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f27fff700 1 -- 172.21.15.7:0/4081909581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f28063940 msgr2=0x7f0f28069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f27fff700 1 --2- 172.21.15.7:0/4081909581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f28063940 0x7f0f28069c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f27fff700 1 -- 172.21.15.7:0/4081909581 >> v1:172.21.15.113:6789/0 conn(0x7f0f28157d80 legacy=0x7f0f28063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f27fff700 1 -- 172.21.15.7:0/4081909581 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280af350 con 0x7f0f281579b0 2024-01-11T07:13:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.745+0000 7f0f2cc63700 1 --2- 172.21.15.7:0/4081909581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f28063940 0x7f0f28069c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.746+0000 7f0f27fff700 1 --2- 172.21.15.7:0/4081909581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f281579b0 0x7f0f2815bf50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0098e0 tx=0x7f0f1c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=106e9bb359a62ee5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.746+0000 7f0f26ffd700 1 -- 172.21.15.7:0/4081909581 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c011630 con 0x7f0f281579b0 2024-01-11T07:13:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.746+0000 7f0f26ffd700 1 -- 172.21.15.7:0/4081909581 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c011790 con 0x7f0f281579b0 2024-01-11T07:13:52.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.746+0000 7f0f26ffd700 1 -- 172.21.15.7:0/4081909581 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c011b10 con 0x7f0f281579b0 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.746+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/4081909581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f281579b0 msgr2=0x7f0f2815bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.746+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/4081909581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f281579b0 0x7f0f2815bf50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0098e0 tx=0x7f0f1c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.747+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/4081909581 shutdown_connections 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.747+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/4081909581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f28063940 0x7f0f28069c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.747+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/4081909581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f281579b0 0x7f0f2815bf50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.747+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/4081909581 >> 172.21.15.7:0/4081909581 conn(0x7f0f280680e0 msgr2=0x7f0f280cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.747+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/4081909581 shutdown_connections 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.747+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/4081909581 wait complete. 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.748+0000 7f0f2e6c6700 1 Processor -- start 2024-01-11T07:13:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.749+0000 7f0f2e6c6700 1 -- start start 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.749+0000 7f0f2e6c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f28063940 0x7f0f280c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.749+0000 7f0f2e6c6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f281579b0 0x7f0f28154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.749+0000 7f0f2e6c6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f28157d80 0x7f0f2816b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.749+0000 7f0f2e6c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280c5c60 con 0x7f0f28063940 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.749+0000 7f0f2e6c6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2816c3d0 con 0x7f0f28157d80 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.749+0000 7f0f2e6c6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280c48e0 con 0x7f0f281579b0 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f28063940 0x7f0f280c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f28063940 0x7f0f280c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41526/0 (socket says 172.21.15.7:41526) 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 -- 172.21.15.7:0/166763936 learned_addr learned my addr 172.21.15.7:0/166763936 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f277fe700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f281579b0 0x7f0f28154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f2cc63700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f28157d80 0x7f0f2816b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 -- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f281579b0 msgr2=0x7f0f28154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f281579b0 0x7f0f28154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 -- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f28157d80 msgr2=0x7f0f2816b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f28157d80 0x7f0f2816b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 -- 172.21.15.7:0/166763936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20009a20 con 0x7f0f28063940 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f27fff700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f28063940 0x7f0f280c6e30 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0f2806d820 tx=0x7f0f1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f0ffff700 1 -- 172.21.15.7:0/166763936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c011900 con 0x7f0f28063940 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1c009420 con 0x7f0f28063940 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f0ffff700 1 -- 172.21.15.7:0/166763936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c011a80 con 0x7f0f28063940 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f0ffff700 1 -- 172.21.15.7:0/166763936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c01a680 con 0x7f0f28063940 2024-01-11T07:13:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.750+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f280c4ea0 con 0x7f0f28063940 2024-01-11T07:13:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.751+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2804c3c0 con 0x7f0f28063940 2024-01-11T07:13:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.752+0000 7f0f0ffff700 1 -- 172.21.15.7:0/166763936 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0f1c01aa70 con 0x7f0f28063940 2024-01-11T07:13:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.752+0000 7f0f0ffff700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f14066f30 0x7f0f140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.753+0000 7f0f0ffff700 1 -- 172.21.15.7:0/166763936 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f0f1c0861c0 con 0x7f0f28063940 2024-01-11T07:13:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.753+0000 7f0f277fe700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f14066f30 0x7f0f140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.753+0000 7f0f0ffff700 1 -- 172.21.15.7:0/166763936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f0f1406eb10 con 0x7f0f28063940 2024-01-11T07:13:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.753+0000 7f0f277fe700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f14066f30 0x7f0f140693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0f2815b610 tx=0x7f0f18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.757+0000 7f0f0ffff700 1 -- 172.21.15.7:0/166763936 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f1c0534b0 con 0x7f0f28063940 2024-01-11T07:13:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.961+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f280af630 con 0x7f0f28063940 2024-01-11T07:13:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.962+0000 7f0f0ffff700 1 -- 172.21.15.7:0/166763936 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+15439 (secure 0 0 0) 0x7f0f1c015020 con 0x7f0f28063940 2024-01-11T07:13:52.963 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:52.963 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":141,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:13:52.238928+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:13:15.625657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-11T07:13:16.691544+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.f","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:13:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f14066f30 msgr2=0x7f0f140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f14066f30 0x7f0f140693e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0f2815b610 tx=0x7f0f18009040 comp rx=0 tx=0).stop 2024-01-11T07:13:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f28063940 msgr2=0x7f0f280c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f28063940 0x7f0f280c6e30 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0f2806d820 tx=0x7f0f1c017000 comp rx=0 tx=0).stop 2024-01-11T07:13:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 shutdown_connections 2024-01-11T07:13:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0f14066f30 0x7f0f140693e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f28157d80 0x7f0f2816b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f28063940 0x7f0f280c6e30 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 --2- 172.21.15.7:0/166763936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f281579b0 0x7f0f28154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 >> 172.21.15.7:0/166763936 conn(0x7f0f280680e0 msgr2=0x7f0f280cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 shutdown_connections 2024-01-11T07:13:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:52.964+0000 7f0f2e6c6700 1 -- 172.21.15.7:0/166763936 wait complete. 2024-01-11T07:13:52.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:13:53.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.110+0000 7f5378b3c700 1 Processor -- start 2024-01-11T07:13:53.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.110+0000 7f5378b3c700 1 -- start start 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.111+0000 7f5378b3c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406dd00 0x7f53740c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.111+0000 7f5378b3c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f537415a1c0 0x7f5374069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.111+0000 7f5378b3c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537415a640 con 0x7f537406e0d0 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.111+0000 7f5378b3c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537415b770 con 0x7f537415a1c0 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.111+0000 7f5378b3c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537415abf0 con 0x7f537406dd00 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537259c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406dd00 0x7f53740c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f5372d9d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f537415a1c0 0x7f5374069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f5372d9d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f537415a1c0 0x7f5374069fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:48780/0 (socket says 172.21.15.7:48780) 2024-01-11T07:13:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f5371d9b700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f537406e0d0 0x7f5374159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:58908/0 (socket says 172.21.15.7:58908) 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f5372d9d700 1 -- 172.21.15.7:0/3874090873 learned_addr learned my addr 172.21.15.7:0/3874090873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537159a700 1 -- 172.21.15.7:0/3874090873 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924350721 0 0) 0x7f535c001110 con 0x7f537406e0d0 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537259c700 1 -- 172.21.15.7:0/3874090873 >> v2:172.21.15.113:3300/0 conn(0x7f537415a1c0 msgr2=0x7f5374069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537259c700 1 --2- 172.21.15.7:0/3874090873 >> v2:172.21.15.113:3300/0 conn(0x7f537415a1c0 0x7f5374069fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537259c700 1 -- 172.21.15.7:0/3874090873 >> v1:172.21.15.113:6789/0 conn(0x7f537406e0d0 legacy=0x7f5374159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537259c700 1 -- 172.21.15.7:0/3874090873 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537415aed0 con 0x7f537406dd00 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f5372d9d700 1 --2- 172.21.15.7:0/3874090873 >> v2:172.21.15.113:3300/0 conn(0x7f537415a1c0 0x7f5374069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537259c700 1 --2- 172.21.15.7:0/3874090873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406dd00 0x7f53740c98d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f53640098e0 tx=0x7f53640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10f382328203cd23 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537159a700 1 -- 172.21.15.7:0/3874090873 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536400b640 con 0x7f537406dd00 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537159a700 1 -- 172.21.15.7:0/3874090873 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536400b7a0 con 0x7f537406dd00 2024-01-11T07:13:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.112+0000 7f537159a700 1 -- 172.21.15.7:0/3874090873 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536400bb40 con 0x7f537406dd00 2024-01-11T07:13:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.113+0000 7f5378b3c700 1 -- 172.21.15.7:0/3874090873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406dd00 msgr2=0x7f53740c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.113+0000 7f5378b3c700 1 --2- 172.21.15.7:0/3874090873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406dd00 0x7f53740c98d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f53640098e0 tx=0x7f53640091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.113+0000 7f5378b3c700 1 -- 172.21.15.7:0/3874090873 shutdown_connections 2024-01-11T07:13:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.113+0000 7f5378b3c700 1 --2- 172.21.15.7:0/3874090873 >> v2:172.21.15.113:3300/0 conn(0x7f537415a1c0 0x7f5374069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.113+0000 7f5378b3c700 1 --2- 172.21.15.7:0/3874090873 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406dd00 0x7f53740c98d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.113+0000 7f5378b3c700 1 -- 172.21.15.7:0/3874090873 >> 172.21.15.7:0/3874090873 conn(0x7f53740680e0 msgr2=0x7f53740cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.113+0000 7f5378b3c700 1 -- 172.21.15.7:0/3874090873 shutdown_connections 2024-01-11T07:13:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.113+0000 7f5378b3c700 1 -- 172.21.15.7:0/3874090873 wait complete. 2024-01-11T07:13:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.114+0000 7f5378b3c700 1 Processor -- start 2024-01-11T07:13:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.114+0000 7f5378b3c700 1 -- start start 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5378b3c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537406dd00 0x7f53740c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5378b3c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406e0d0 0x7f5374168470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5378b3c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415a1c0 0x7f53741695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5378b3c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740c64e0 con 0x7f537415a1c0 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5378b3c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53741689b0 con 0x7f537406dd00 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5378b3c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5374168c10 con 0x7f537406e0d0 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5372d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415a1c0 0x7f53741695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5372d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415a1c0 0x7f53741695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41528/0 (socket says 172.21.15.7:41528) 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.115+0000 7f5372d9d700 1 -- 172.21.15.7:0/2087188155 learned_addr learned my addr 172.21.15.7:0/2087188155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f537259c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537406dd00 0x7f53740c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5371d9b700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406e0d0 0x7f5374168470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5372d9d700 1 -- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406e0d0 msgr2=0x7f5374168470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5372d9d700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406e0d0 0x7f5374168470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5372d9d700 1 -- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537406dd00 msgr2=0x7f53740c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5372d9d700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537406dd00 0x7f53740c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5372d9d700 1 -- 172.21.15.7:0/2087188155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c009ae0 con 0x7f537415a1c0 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5372d9d700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415a1c0 0x7f53741695c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f536c009f30 tx=0x7f536c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c018070 con 0x7f537415a1c0 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c00f040 con 0x7f537415a1c0 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5364009420 con 0x7f537415a1c0 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00cb70 con 0x7f537415a1c0 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.116+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53741691d0 con 0x7f537415a1c0 2024-01-11T07:13:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.117+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537404c3c0 con 0x7f537415a1c0 2024-01-11T07:13:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.118+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f536c003020 con 0x7f537415a1c0 2024-01-11T07:13:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.118+0000 7f536a7fc700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5354066f30 0x7f53540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.118+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f536c084420 con 0x7f537415a1c0 2024-01-11T07:13:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.118+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=142}) v3 -- 0x7f535406eb10 con 0x7f537415a1c0 2024-01-11T07:13:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.118+0000 7f537259c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5354066f30 0x7f53540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.119+0000 7f537259c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5354066f30 0x7f53540693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f53640093f0 tx=0x7f5364017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.123+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f536c051650 con 0x7f537415a1c0 2024-01-11T07:13:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.374+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f537415b1b0 con 0x7f537415a1c0 2024-01-11T07:13:53.375 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:13:53.374+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:13:53.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.376+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(142..142 src has 1..142) v4 ==== 366+0+0 (secure 0 0 0) 0x7f536c055a10 con 0x7f537415a1c0 2024-01-11T07:13:53.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.376+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7f535406f460 con 0x7f537415a1c0 2024-01-11T07:13:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.381+0000 7f536a7fc700 1 -- 172.21.15.7:0/2087188155 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f536c054c70 con 0x7f537415a1c0 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5354066f30 msgr2=0x7f53540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5354066f30 0x7f53540693e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f53640093f0 tx=0x7f5364017000 comp rx=0 tx=0).stop 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415a1c0 msgr2=0x7f53741695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415a1c0 0x7f53741695c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f536c009f30 tx=0x7f536c00b040 comp rx=0 tx=0).stop 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 shutdown_connections 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5354066f30 0x7f53540693e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415a1c0 0x7f53741695c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537406dd00 0x7f53740c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 --2- 172.21.15.7:0/2087188155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f537406e0d0 0x7f5374168470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 >> 172.21.15.7:0/2087188155 conn(0x7f53740680e0 msgr2=0x7f537406a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 shutdown_connections 2024-01-11T07:13:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.383+0000 7f5378b3c700 1 -- 172.21.15.7:0/2087188155 wait complete. 2024-01-11T07:13:53.385 INFO:tasks.ceph.mds.c.smithi007.stderr: -7601> 2024-01-11T07:13:23.274+0000 7ff495202700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:13:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:13:53.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:13:53.416 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:13:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.532+0000 7fe11d588700 1 Processor -- start 2024-01-11T07:13:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.532+0000 7fe11d588700 1 -- start start 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe11d588700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe1180aee00 0x7fe118156e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe11d588700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1180c2e50 0x7fe1180cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe11d588700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1181573c0 con 0x7fe11815af90 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe11d588700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe118158200 con 0x7fe1180aee00 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe11d588700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1181575f0 con 0x7fe1180c2e50 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe116ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe1180aee00 0x7fe118156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe116ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe1180aee00 0x7fe118156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41560/0 (socket says 172.21.15.7:41560) 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe116ffd700 1 -- 172.21.15.7:0/1991309079 learned_addr learned my addr 172.21.15.7:0/1991309079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe1177fe700 1 --2- 172.21.15.7:0/1991309079 >> v2:172.21.15.113:3300/0 conn(0x7fe1180c2e50 0x7fe1180cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe116ffd700 1 -- 172.21.15.7:0/1991309079 >> v2:172.21.15.113:3300/0 conn(0x7fe1180c2e50 msgr2=0x7fe1180cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe116ffd700 1 --2- 172.21.15.7:0/1991309079 >> v2:172.21.15.113:3300/0 conn(0x7fe1180c2e50 0x7fe1180cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe116ffd700 1 -- 172.21.15.7:0/1991309079 >> v1:172.21.15.113:6789/0 conn(0x7fe11815af90 legacy=0x7fe1180c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.533+0000 7fe116ffd700 1 -- 172.21.15.7:0/1991309079 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1181578d0 con 0x7fe1180aee00 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe116ffd700 1 --2- 172.21.15.7:0/1991309079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180aee00 0x7fe118156e80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe118158170 tx=0x7fe10800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f094a5d39bdb904 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe115ffb700 1 -- 172.21.15.7:0/1991309079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800dc50 con 0x7fe1180aee00 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe115ffb700 1 -- 172.21.15.7:0/1991309079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10800ddb0 con 0x7fe1180aee00 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe115ffb700 1 -- 172.21.15.7:0/1991309079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080196d0 con 0x7fe1180aee00 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe11d588700 1 -- 172.21.15.7:0/1991309079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180aee00 msgr2=0x7fe118156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe11d588700 1 --2- 172.21.15.7:0/1991309079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180aee00 0x7fe118156e80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe118158170 tx=0x7fe10800b4f0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe11d588700 1 -- 172.21.15.7:0/1991309079 shutdown_connections 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe11d588700 1 --2- 172.21.15.7:0/1991309079 >> v2:172.21.15.113:3300/0 conn(0x7fe1180c2e50 0x7fe1180cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe11d588700 1 --2- 172.21.15.7:0/1991309079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180aee00 0x7fe118156e80 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.534+0000 7fe11d588700 1 -- 172.21.15.7:0/1991309079 >> 172.21.15.7:0/1991309079 conn(0x7fe118063230 msgr2=0x7fe118063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.535+0000 7fe11d588700 1 -- 172.21.15.7:0/1991309079 shutdown_connections 2024-01-11T07:13:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.535+0000 7fe11d588700 1 -- 172.21.15.7:0/1991309079 wait complete. 2024-01-11T07:13:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.536+0000 7fe11d588700 1 Processor -- start 2024-01-11T07:13:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.536+0000 7fe11d588700 1 -- start start 2024-01-11T07:13:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe11d588700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1180aee00 0x7fe1181522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe11d588700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1180c2e50 0x7fe1180b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe11d588700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815af90 0x7fe118169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe11d588700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180b1020 con 0x7fe11815af90 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe11d588700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180b03e0 con 0x7fe1180c2e50 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe11d588700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180b0680 con 0x7fe1180aee00 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815af90 0x7fe118169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815af90 0x7fe118169780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41576/0 (socket says 172.21.15.7:41576) 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 -- 172.21.15.7:0/1442328737 learned_addr learned my addr 172.21.15.7:0/1442328737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe116ffd700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1180aee00 0x7fe1181522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1167fc700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1180c2e50 0x7fe1180b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 -- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1180aee00 msgr2=0x7fe1181522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1180aee00 0x7fe1181522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 -- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1180c2e50 msgr2=0x7fe1180b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1180c2e50 0x7fe1180b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 -- 172.21.15.7:0/1442328737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10800b790 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.537+0000 7fe1177fe700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815af90 0x7fe118169780 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe110007f30 tx=0x7fe1100098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.538+0000 7fe0feffd700 1 -- 172.21.15.7:0/1442328737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11000ecf0 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.538+0000 7fe0feffd700 1 -- 172.21.15.7:0/1442328737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11000ee50 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.538+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1180b0960 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.538+0000 7fe0feffd700 1 -- 172.21.15.7:0/1442328737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110014670 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.538+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1181527f0 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.539+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11804c3c0 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.539+0000 7fe0feffd700 1 -- 172.21.15.7:0/1442328737 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe1100149e0 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.540+0000 7fe0feffd700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe104066f30 0x7fe1040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.540+0000 7fe0feffd700 1 -- 172.21.15.7:0/1442328737 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5849+0+0 (secure 0 0 0) 0x7fe110084d00 con 0x7fe11815af90 2024-01-11T07:13:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.540+0000 7fe0feffd700 1 -- 172.21.15.7:0/1442328737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7fe10406ebb0 con 0x7fe11815af90 2024-01-11T07:13:53.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.540+0000 7fe116ffd700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe104066f30 0x7fe1040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:53.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.541+0000 7fe116ffd700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe104066f30 0x7fe1040693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe118158170 tx=0x7fe10801b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:53.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.544+0000 7fe0feffd700 1 -- 172.21.15.7:0/1442328737 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe110055970 con 0x7fe11815af90 2024-01-11T07:13:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:53.796+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe118065450 con 0x7fe11815af90 2024-01-11T07:13:54.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.383+0000 7fe0feffd700 1 -- 172.21.15.7:0/1442328737 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe110009a20 con 0x7fe11815af90 2024-01-11T07:13:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe104066f30 msgr2=0x7fe1040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe104066f30 0x7fe1040693e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe118158170 tx=0x7fe10801b000 comp rx=0 tx=0).stop 2024-01-11T07:13:54.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815af90 msgr2=0x7fe118169780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815af90 0x7fe118169780 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe110007f30 tx=0x7fe1100098e0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 shutdown_connections 2024-01-11T07:13:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe104066f30 0x7fe1040693e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815af90 0x7fe118169780 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe1180aee00 0x7fe1181522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 --2- 172.21.15.7:0/1442328737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1180c2e50 0x7fe1180b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 >> 172.21.15.7:0/1442328737 conn(0x7fe118063230 msgr2=0x7fe11806e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 shutdown_connections 2024-01-11T07:13:54.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.386+0000 7fe11d588700 1 -- 172.21.15.7:0/1442328737 wait complete. 2024-01-11T07:13:54.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:13:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.587+0000 7f723ddda700 1 Processor -- start 2024-01-11T07:13:54.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.587+0000 7f723ddda700 1 -- start start 2024-01-11T07:13:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.587+0000 7f723ddda700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7238159bf0 0x7f72380c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.587+0000 7f723ddda700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f7238063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.587+0000 7f723ddda700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7238069c90 0x7f7238063770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.587+0000 7f723ddda700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723815dd40 con 0x7f7238069c90 2024-01-11T07:13:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.587+0000 7f723ddda700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723815e020 con 0x7f7238159bf0 2024-01-11T07:13:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.587+0000 7f723ddda700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723815e300 con 0x7f7238159fc0 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7238069c90 0x7f7238063770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7238069c90 0x7f7238063770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41578/0 (socket says 172.21.15.7:41578) 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 -- 172.21.15.7:0/99973105 learned_addr learned my addr 172.21.15.7:0/99973105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f72377fe700 1 --2- 172.21.15.7:0/99973105 >> v2:172.21.15.113:3300/0 conn(0x7f7238159bf0 0x7f72380c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7236ffd700 1 --2- 172.21.15.7:0/99973105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f7238063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 -- 172.21.15.7:0/99973105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 msgr2=0x7f7238063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 --2- 172.21.15.7:0/99973105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f7238063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 -- 172.21.15.7:0/99973105 >> v2:172.21.15.113:3300/0 conn(0x7f7238159bf0 msgr2=0x7f72380c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 --2- 172.21.15.7:0/99973105 >> v2:172.21.15.113:3300/0 conn(0x7f7238159bf0 0x7f72380c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 -- 172.21.15.7:0/99973105 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723815e5e0 con 0x7f7238069c90 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f72377fe700 1 --2- 172.21.15.7:0/99973105 >> v2:172.21.15.113:3300/0 conn(0x7f7238159bf0 0x7f72380c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f7237fff700 1 --2- 172.21.15.7:0/99973105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238069c90 0x7f7238063770 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f723806d490 tx=0x7f7230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9967894fd3340a06 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f72367fc700 1 -- 172.21.15.7:0/99973105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000b860 con 0x7f7238069c90 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.588+0000 7f72367fc700 1 -- 172.21.15.7:0/99973105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723000b9c0 con 0x7f7238069c90 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f72367fc700 1 -- 172.21.15.7:0/99973105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723000bd80 con 0x7f7238069c90 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 -- 172.21.15.7:0/99973105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238069c90 msgr2=0x7f7238063770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 --2- 172.21.15.7:0/99973105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238069c90 0x7f7238063770 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f723806d490 tx=0x7f7230009250 comp rx=0 tx=0).stop 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 -- 172.21.15.7:0/99973105 shutdown_connections 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 --2- 172.21.15.7:0/99973105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238069c90 0x7f7238063770 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 --2- 172.21.15.7:0/99973105 >> v2:172.21.15.113:3300/0 conn(0x7f7238159bf0 0x7f72380c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 --2- 172.21.15.7:0/99973105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f7238063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 -- 172.21.15.7:0/99973105 >> 172.21.15.7:0/99973105 conn(0x7f72380680a0 msgr2=0x7f72380cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 -- 172.21.15.7:0/99973105 shutdown_connections 2024-01-11T07:13:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.589+0000 7f723ddda700 1 -- 172.21.15.7:0/99973105 wait complete. 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.590+0000 7f723ddda700 1 Processor -- start 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f723ddda700 1 -- start start 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f723ddda700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7238069c90 0x7f72380b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f723ddda700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238159bf0 0x7f72380c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f723ddda700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f72380c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f723ddda700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c78e0 con 0x7f7238159bf0 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f723ddda700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c4990 con 0x7f7238069c90 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f723ddda700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c5510 con 0x7f7238159fc0 2024-01-11T07:13:54.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f7236ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238159bf0 0x7f72380c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f7236ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238159bf0 0x7f72380c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41586/0 (socket says 172.21.15.7:41586) 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.591+0000 7f7236ffd700 1 -- 172.21.15.7:0/2988209969 learned_addr learned my addr 172.21.15.7:0/2988209969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f72377fe700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7238069c90 0x7f72380b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f7237fff700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f72380c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f7236ffd700 1 -- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 msgr2=0x7f72380c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f7236ffd700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f72380c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f7236ffd700 1 -- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7238069c90 msgr2=0x7f72380b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f7236ffd700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7238069c90 0x7f72380b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f7236ffd700 1 -- 172.21.15.7:0/2988209969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72300094f0 con 0x7f7238159bf0 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f7237fff700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f72380c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f7236ffd700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238159bf0 0x7f72380c59a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f722000d890 tx=0x7f7220008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722000b600 con 0x7f7238159bf0 2024-01-11T07:13:54.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722000b760 con 0x7f7238159bf0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72380c4be0 con 0x7f7238159bf0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722000bae0 con 0x7f7238159bf0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.592+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72380c51a0 con 0x7f7238159bf0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.593+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72380cb940 con 0x7f7238159bf0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.594+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f722001e460 con 0x7f7238159bf0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.594+0000 7f722f7fe700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7218066f30 0x7f72180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.594+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f7220084c50 con 0x7f7238159bf0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.594+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=143}) v3 -- 0x7f721806ebb0 con 0x7f7238159bf0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.594+0000 7f72377fe700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7218066f30 0x7f72180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.595+0000 7f72377fe700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7218066f30 0x7f72180693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f72280076a0 tx=0x7f72280098f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:54.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.599+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f722008a020 con 0x7f7238159bf0 2024-01-11T07:13:54.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:54.801+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f723804c3c0 con 0x7f7238159bf0 2024-01-11T07:13:55.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.383+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(143..143 src has 1..143) v4 ==== 288+0+0 (secure 0 0 0) 0x7f722008a070 con 0x7f7238159bf0 2024-01-11T07:13:55.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.383+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=144}) v3 -- 0x7f721806cfe0 con 0x7f7238159bf0 2024-01-11T07:13:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.393+0000 7f722f7fe700 1 -- 172.21.15.7:0/2988209969 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v143) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7220051e30 con 0x7f7238159bf0 2024-01-11T07:13:55.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7218066f30 msgr2=0x7f72180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7218066f30 0x7f72180693e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f72280076a0 tx=0x7f72280098f0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238159bf0 msgr2=0x7f72380c59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238159bf0 0x7f72380c59a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f722000d890 tx=0x7f7220008040 comp rx=0 tx=0).stop 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 shutdown_connections 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7218066f30 0x7f72180693e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7238159fc0 0x7f72380c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7238069c90 0x7f72380b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 --2- 172.21.15.7:0/2988209969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7238159bf0 0x7f72380c59a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 >> 172.21.15.7:0/2988209969 conn(0x7f72380680a0 msgr2=0x7f723806db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 shutdown_connections 2024-01-11T07:13:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.397+0000 7f723ddda700 1 -- 172.21.15.7:0/2988209969 wait complete. 2024-01-11T07:13:55.400 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:13:55.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:13:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.554+0000 7f77ae4f5700 1 Processor -- start 2024-01-11T07:13:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.554+0000 7f77ae4f5700 1 -- start start 2024-01-11T07:13:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77ae4f5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f77a815db70 0x7f77a80cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77ae4f5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a80cdd70 con 0x7f77a815cc60 2024-01-11T07:13:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77ae4f5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a80ce050 con 0x7f77a815df40 2024-01-11T07:13:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77ae4f5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a80ce330 con 0x7f77a815db70 2024-01-11T07:13:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77a7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f77a815db70 0x7f77a80cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77aca92700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f77a815cc60 0x7f77a815d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59826/0 (socket says 172.21.15.7:59826) 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77aca92700 1 -- 172.21.15.7:0/407966470 learned_addr learned my addr 172.21.15.7:0/407966470 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77a7fff700 1 -- 172.21.15.7:0/407966470 >> v1:172.21.15.113:6789/0 conn(0x7f77a815df40 legacy=0x7f77a806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77a7fff700 1 -- 172.21.15.7:0/407966470 >> v1:172.21.15.7:6789/0 conn(0x7f77a815cc60 legacy=0x7f77a815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77a7fff700 1 -- 172.21.15.7:0/407966470 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80ce610 con 0x7f77a815db70 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.555+0000 7f77a7fff700 1 --2- 172.21.15.7:0/407966470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a80cee10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7798009910 tx=0x7f77980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af46bdcddd960887 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77a6ffd700 1 -- 172.21.15.7:0/407966470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800b820 con 0x7f77a815db70 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77a6ffd700 1 -- 172.21.15.7:0/407966470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779800b980 con 0x7f77a815db70 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77a6ffd700 1 -- 172.21.15.7:0/407966470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800bd20 con 0x7f77a815db70 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77ae4f5700 1 -- 172.21.15.7:0/407966470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 msgr2=0x7f77a80cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77ae4f5700 1 --2- 172.21.15.7:0/407966470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a80cee10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7798009910 tx=0x7f77980091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77ae4f5700 1 -- 172.21.15.7:0/407966470 shutdown_connections 2024-01-11T07:13:55.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77ae4f5700 1 --2- 172.21.15.7:0/407966470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a80cee10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77ae4f5700 1 -- 172.21.15.7:0/407966470 >> 172.21.15.7:0/407966470 conn(0x7f77a8063230 msgr2=0x7f77a8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:55.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77ae4f5700 1 -- 172.21.15.7:0/407966470 shutdown_connections 2024-01-11T07:13:55.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.556+0000 7f77ae4f5700 1 -- 172.21.15.7:0/407966470 wait complete. 2024-01-11T07:13:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.557+0000 7f77ae4f5700 1 Processor -- start 2024-01-11T07:13:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.558+0000 7f77ae4f5700 1 -- start start 2024-01-11T07:13:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.558+0000 7f77ae4f5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77a815cc60 0x7f77a8154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.558+0000 7f77ae4f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.558+0000 7f77ae4f5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a815df40 0x7f77a815be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.558+0000 7f77ae4f5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80acfb0 con 0x7f77a815db70 2024-01-11T07:13:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.558+0000 7f77ae4f5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80accd0 con 0x7f77a815df40 2024-01-11T07:13:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.558+0000 7f77ae4f5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8158c40 con 0x7f77a815cc60 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a815b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41612/0 (socket says 172.21.15.7:41612) 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 -- 172.21.15.7:0/3072288570 learned_addr learned my addr 172.21.15.7:0/3072288570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a7fff700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77a815cc60 0x7f77a8154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77aca92700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a815df40 0x7f77a815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 -- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77a815cc60 msgr2=0x7f77a8154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77a815cc60 0x7f77a8154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 -- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a815df40 msgr2=0x7f77a815be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a815df40 0x7f77a815be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 -- 172.21.15.7:0/3072288570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798009420 con 0x7f77a815db70 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77a77fe700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a815b940 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c790 tx=0x7f779c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:55.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c011590 con 0x7f77a815db70 2024-01-11T07:13:55.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c011710 con 0x7f77a815db70 2024-01-11T07:13:55.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a8158f20 con 0x7f77a815db70 2024-01-11T07:13:55.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c011ad0 con 0x7f77a815db70 2024-01-11T07:13:55.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.559+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a81549b0 con 0x7f77a815db70 2024-01-11T07:13:55.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.561+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a81593c0 con 0x7f77a815db70 2024-01-11T07:13:55.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.561+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f779c01b3b0 con 0x7f77a815db70 2024-01-11T07:13:55.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.562+0000 7f778ffff700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7794066f30 0x7f77940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:55.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.562+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5411+0+0 (secure 0 0 0) 0x7f779c083be0 con 0x7f77a815db70 2024-01-11T07:13:55.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.562+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=144}) v3 -- 0x7f779406ddb0 con 0x7f77a815db70 2024-01-11T07:13:55.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.562+0000 7f77a7fff700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7794066f30 0x7f77940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:55.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.563+0000 7f77a7fff700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7794066f30 0x7f77940693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f77980093f0 tx=0x7f7798017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:55.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.566+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f779c017020 con 0x7f77a815db70 2024-01-11T07:13:55.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:55.768+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77a804c3c0 con 0x7f77a815db70 2024-01-11T07:13:56.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.388+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(144..144 src has 1..144) v4 ==== 322+0+0 (secure 0 0 0) 0x7f779c054cb0 con 0x7f77a815db70 2024-01-11T07:13:56.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.388+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=145}) v3 -- 0x7f779406cb40 con 0x7f77a815db70 2024-01-11T07:13:56.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.392+0000 7f778ffff700 1 -- 172.21.15.7:0/3072288570 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v144) v1 ==== 169+0+0 (secure 0 0 0) 0x7f779c050ff0 con 0x7f77a815db70 2024-01-11T07:13:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7794066f30 msgr2=0x7f77940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7794066f30 0x7f77940693e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f77980093f0 tx=0x7f7798017000 comp rx=0 tx=0).stop 2024-01-11T07:13:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 msgr2=0x7f77a815b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a815b940 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c790 tx=0x7f779c007f40 comp rx=0 tx=0).stop 2024-01-11T07:13:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 shutdown_connections 2024-01-11T07:13:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7794066f30 0x7f77940693e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77a815df40 0x7f77a815be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f77a815cc60 0x7f77a8154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 --2- 172.21.15.7:0/3072288570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a815db70 0x7f77a815b940 secure :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c790 tx=0x7f779c007f40 comp rx=0 tx=0).stop 2024-01-11T07:13:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.401+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 >> 172.21.15.7:0/3072288570 conn(0x7f77a8063230 msgr2=0x7f77a80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.402+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 shutdown_connections 2024-01-11T07:13:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.402+0000 7f77ae4f5700 1 -- 172.21.15.7:0/3072288570 wait complete. 2024-01-11T07:13:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:13:56.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:13:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.556+0000 7f5b9a280700 1 Processor -- start 2024-01-11T07:13:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.556+0000 7f5b9a280700 1 -- start start 2024-01-11T07:13:56.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.556+0000 7f5b9a280700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b940ace00 0x7f5b94164c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.556+0000 7f5b9a280700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940aad50 con 0x7f5b940aa8c0 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.556+0000 7f5b9a280700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940ab030 con 0x7f5b9406a850 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b9a280700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b940ab310 con 0x7f5b940ace00 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b937fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5b940aa8c0 0x7f5b94159760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59830/0 (socket says 172.21.15.7:59830) 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b937fe700 1 -- 172.21.15.7:0/634788871 learned_addr learned my addr 172.21.15.7:0/634788871 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b9881d700 1 --2- 172.21.15.7:0/634788871 >> v2:172.21.15.113:3300/0 conn(0x7f5b940ace00 0x7f5b94164c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b92ffd700 1 -- 172.21.15.7:0/634788871 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917957502 0 0) 0x7f5b940aad50 con 0x7f5b940aa8c0 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b92ffd700 1 -- 172.21.15.7:0/634788871 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c003570 con 0x7f5b940aa8c0 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b92ffd700 1 -- 172.21.15.7:0/634788871 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821964462 0 0) 0x7f5b940ab030 con 0x7f5b9406a850 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b92ffd700 1 -- 172.21.15.7:0/634788871 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b940aad50 con 0x7f5b9406a850 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b92ffd700 1 -- 172.21.15.7:0/634788871 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127298478 0 0) 0x7f5b7c003570 con 0x7f5b940aa8c0 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b9881d700 1 -- 172.21.15.7:0/634788871 >> v1:172.21.15.113:6789/0 conn(0x7f5b9406a850 legacy=0x7f5b94070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b9881d700 1 -- 172.21.15.7:0/634788871 >> v1:172.21.15.7:6789/0 conn(0x7f5b940aa8c0 legacy=0x7f5b94159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.557+0000 7f5b9881d700 1 -- 172.21.15.7:0/634788871 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940ab5f0 con 0x7f5b940ace00 2024-01-11T07:13:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b9881d700 1 --2- 172.21.15.7:0/634788871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940ace00 0x7f5b94164c90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009e60 tx=0x7f5b8c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a6ab92e1fa38bd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b92ffd700 1 -- 172.21.15.7:0/634788871 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00e070 con 0x7f5b940ace00 2024-01-11T07:13:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b92ffd700 1 -- 172.21.15.7:0/634788871 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c00b7a0 con 0x7f5b940ace00 2024-01-11T07:13:56.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b92ffd700 1 -- 172.21.15.7:0/634788871 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88001470 con 0x7f5b940ace00 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b9a280700 1 -- 172.21.15.7:0/634788871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940ace00 msgr2=0x7f5b94164c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b9a280700 1 --2- 172.21.15.7:0/634788871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940ace00 0x7f5b94164c90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009e60 tx=0x7f5b8c009780 comp rx=0 tx=0).stop 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b9a280700 1 -- 172.21.15.7:0/634788871 shutdown_connections 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b9a280700 1 --2- 172.21.15.7:0/634788871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940ace00 0x7f5b94164c90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b9a280700 1 -- 172.21.15.7:0/634788871 >> 172.21.15.7:0/634788871 conn(0x7f5b940cd070 msgr2=0x7f5b940cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b9a280700 1 -- 172.21.15.7:0/634788871 shutdown_connections 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.558+0000 7f5b9a280700 1 -- 172.21.15.7:0/634788871 wait complete. 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.560+0000 7f5b9a280700 1 Processor -- start 2024-01-11T07:13:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.560+0000 7f5b9a280700 1 -- start start 2024-01-11T07:13:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b9a280700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b9406a850 0x7f5b9414db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b9a280700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940aa8c0 0x7f5b94152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b9a280700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b940ace00 0x7f5b94153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b9a280700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9416af40 con 0x7f5b9406a850 2024-01-11T07:13:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b9a280700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9416a330 con 0x7f5b940aa8c0 2024-01-11T07:13:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b9a280700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9416a610 con 0x7f5b940ace00 2024-01-11T07:13:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b9406a850 0x7f5b9414db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b9406a850 0x7f5b9414db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41628/0 (socket says 172.21.15.7:41628) 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 -- 172.21.15.7:0/4051369079 learned_addr learned my addr 172.21.15.7:0/4051369079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b937fe700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940aa8c0 0x7f5b94152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b9881d700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b940ace00 0x7f5b94153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 -- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b940ace00 msgr2=0x7f5b94153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b940ace00 0x7f5b94153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 -- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940aa8c0 msgr2=0x7f5b94152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940aa8c0 0x7f5b94152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 -- 172.21.15.7:0/4051369079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c009a20 con 0x7f5b9406a850 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b9881d700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b940ace00 0x7f5b94153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b937fe700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940aa8c0 0x7f5b94152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b93fff700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b9406a850 0x7f5b9414db30 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400c590 tx=0x7f5b84007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b7bfff700 1 -- 172.21.15.7:0/4051369079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84010510 con 0x7f5b9406a850 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.561+0000 7f5b7bfff700 1 -- 172.21.15.7:0/4051369079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84010670 con 0x7f5b9406a850 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.562+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9416a8f0 con 0x7f5b9406a850 2024-01-11T07:13:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.562+0000 7f5b7bfff700 1 -- 172.21.15.7:0/4051369079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84010a30 con 0x7f5b9406a850 2024-01-11T07:13:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.562+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9416a040 con 0x7f5b9406a850 2024-01-11T07:13:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.563+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9404c3c0 con 0x7f5b9406a850 2024-01-11T07:13:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.563+0000 7f5b7bfff700 1 -- 172.21.15.7:0/4051369079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5b8401b400 con 0x7f5b9406a850 2024-01-11T07:13:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.563+0000 7f5b7bfff700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b80066f30 0x7f5b800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.563+0000 7f5b7bfff700 1 -- 172.21.15.7:0/4051369079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5b84083e60 con 0x7f5b9406a850 2024-01-11T07:13:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.564+0000 7f5b937fe700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b80066f30 0x7f5b800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.564+0000 7f5b937fe700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b80066f30 0x7f5b800693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5b94153970 tx=0x7f5b880019d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:56.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.568+0000 7f5b7bfff700 1 -- 172.21.15.7:0/4051369079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b840513d0 con 0x7f5b9406a850 2024-01-11T07:13:56.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.771+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5b9406cff0 con 0x7f5b9406a850 2024-01-11T07:13:56.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.775+0000 7f5b7bfff700 1 -- 172.21.15.7:0/4051369079 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5b840549f0 con 0x7f5b9406a850 2024-01-11T07:13:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b80066f30 msgr2=0x7f5b800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b80066f30 0x7f5b800693e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5b94153970 tx=0x7f5b880019d0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b9406a850 msgr2=0x7f5b9414db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b9406a850 0x7f5b9414db30 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400c590 tx=0x7f5b84007dc0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 shutdown_connections 2024-01-11T07:13:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5b80066f30 0x7f5b800693e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b940ace00 0x7f5b94153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b9406a850 0x7f5b9414db30 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 --2- 172.21.15.7:0/4051369079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940aa8c0 0x7f5b94152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 >> 172.21.15.7:0/4051369079 conn(0x7f5b940cd070 msgr2=0x7f5b94156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:56.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 shutdown_connections 2024-01-11T07:13:56.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.777+0000 7f5b9a280700 1 -- 172.21.15.7:0/4051369079 wait complete. 2024-01-11T07:13:56.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-01-11T07:13:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.922+0000 7f79035a2700 1 Processor -- start 2024-01-11T07:13:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.922+0000 7f79035a2700 1 -- start start 2024-01-11T07:13:56.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.923+0000 7f79035a2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc156760 0x7f78fc06e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.923+0000 7f79035a2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78fc15b260 0x7f78fc0b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.923+0000 7f79035a2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc06e3f0 con 0x7f78fc069c80 2024-01-11T07:13:56.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.923+0000 7f79035a2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc157b40 con 0x7f78fc15b260 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.923+0000 7f79035a2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0b1f10 con 0x7f78fc156760 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.923+0000 7f7901b3f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f78fc069c80 0x7f78fc06b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59842/0 (socket says 172.21.15.7:59842) 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.923+0000 7f7901b3f700 1 -- 172.21.15.7:0/3588010079 learned_addr learned my addr 172.21.15.7:0/3588010079 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f7900b3d700 1 --2- 172.21.15.7:0/3588010079 >> v2:172.21.15.113:3300/0 conn(0x7f78fc15b260 0x7f78fc0b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f790133e700 1 --2- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc156760 0x7f78fc06e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f78f3fff700 1 -- 172.21.15.7:0/3588010079 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327552531 0 0) 0x7f78fc06e3f0 con 0x7f78fc069c80 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f78f3fff700 1 -- 172.21.15.7:0/3588010079 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e8003570 con 0x7f78fc069c80 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f7900b3d700 1 -- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc156760 msgr2=0x7f78fc06e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f7900b3d700 1 --2- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc156760 0x7f78fc06e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f7900b3d700 1 -- 172.21.15.7:0/3588010079 >> v1:172.21.15.7:6789/0 conn(0x7f78fc069c80 legacy=0x7f78fc06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f7900b3d700 1 -- 172.21.15.7:0/3588010079 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc06dda0 con 0x7f78fc15b260 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f790133e700 1 --2- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc156760 0x7f78fc06e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f7900b3d700 1 --2- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc15b260 0x7f78fc0b1610 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f78f80098e0 tx=0x7f78f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6822f43e7d1848a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:56.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f78f3fff700 1 -- 172.21.15.7:0/3588010079 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8011630 con 0x7f78fc15b260 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f78f3fff700 1 -- 172.21.15.7:0/3588010079 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f8011790 con 0x7f78fc15b260 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.924+0000 7f78f3fff700 1 -- 172.21.15.7:0/3588010079 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8011b30 con 0x7f78fc15b260 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.925+0000 7f79035a2700 1 -- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc15b260 msgr2=0x7f78fc0b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.925+0000 7f79035a2700 1 --2- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc15b260 0x7f78fc0b1610 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f78f80098e0 tx=0x7f78f80091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.925+0000 7f79035a2700 1 -- 172.21.15.7:0/3588010079 shutdown_connections 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.925+0000 7f79035a2700 1 --2- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc156760 0x7f78fc06e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.925+0000 7f79035a2700 1 --2- 172.21.15.7:0/3588010079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc15b260 0x7f78fc0b1610 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.925+0000 7f79035a2700 1 -- 172.21.15.7:0/3588010079 >> 172.21.15.7:0/3588010079 conn(0x7f78fc0cd910 msgr2=0x7f78fc0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.925+0000 7f79035a2700 1 -- 172.21.15.7:0/3588010079 shutdown_connections 2024-01-11T07:13:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.925+0000 7f79035a2700 1 -- 172.21.15.7:0/3588010079 wait complete. 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.926+0000 7f79035a2700 1 Processor -- start 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.926+0000 7f79035a2700 1 -- start start 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.927+0000 7f79035a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc069c80 0x7f78fc0ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.927+0000 7f79035a2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc156760 0x7f78fc0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.927+0000 7f79035a2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc15b260 0x7f78fc152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.927+0000 7f79035a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc14f170 con 0x7f78fc069c80 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.927+0000 7f79035a2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c8890 con 0x7f78fc156760 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.927+0000 7f79035a2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c7c80 con 0x7f78fc15b260 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.927+0000 7f790133e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc069c80 0x7f78fc0ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f790133e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc069c80 0x7f78fc0ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41634/0 (socket says 172.21.15.7:41634) 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f790133e700 1 -- 172.21.15.7:0/1009667532 learned_addr learned my addr 172.21.15.7:0/1009667532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f7900b3d700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc156760 0x7f78fc0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f7901b3f700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc15b260 0x7f78fc152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f790133e700 1 -- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc15b260 msgr2=0x7f78fc152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f790133e700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc15b260 0x7f78fc152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f790133e700 1 -- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc156760 msgr2=0x7f78fc0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f790133e700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc156760 0x7f78fc0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f790133e700 1 -- 172.21.15.7:0/1009667532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009420 con 0x7f78fc069c80 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f7901b3f700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc15b260 0x7f78fc152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f790133e700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc069c80 0x7f78fc0ccaa0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00ec80 tx=0x7f78ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f78f17fa700 1 -- 172.21.15.7:0/1009667532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00c9d0 con 0x7f78fc069c80 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f78f17fa700 1 -- 172.21.15.7:0/1009667532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00cb30 con 0x7f78fc069c80 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f8009420 con 0x7f78fc069c80 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f78f17fa700 1 -- 172.21.15.7:0/1009667532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00ceb0 con 0x7f78fc069c80 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.928+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0c82c0 con 0x7f78fc069c80 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.929+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc0c8b70 con 0x7f78fc069c80 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.930+0000 7f78f17fa700 1 -- 172.21.15.7:0/1009667532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f78ec013680 con 0x7f78fc069c80 2024-01-11T07:13:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.930+0000 7f78f17fa700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f78e0066f30 0x7f78e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.930+0000 7f78f17fa700 1 -- 172.21.15.7:0/1009667532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f78ec0846d0 con 0x7f78fc069c80 2024-01-11T07:13:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.930+0000 7f7900b3d700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f78e0066f30 0x7f78e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.931+0000 7f7900b3d700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f78e0066f30 0x7f78e00693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f78f80091e0 tx=0x7f78f8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:56.935+0000 7f78f17fa700 1 -- 172.21.15.7:0/1009667532 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78ec051ba0 con 0x7f78fc069c80 2024-01-11T07:13:57.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.220+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f78fc04c420 con 0x7f78fc069c80 2024-01-11T07:13:57.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.392+0000 7f78f17fa700 1 -- 172.21.15.7:0/1009667532 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v360) v1 ==== 140+0+0 (secure 0 0 0) 0x7f78ec0551c0 con 0x7f78fc069c80 2024-01-11T07:13:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f78e0066f30 msgr2=0x7f78e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f78e0066f30 0x7f78e00693e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f78f80091e0 tx=0x7f78f8017000 comp rx=0 tx=0).stop 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc069c80 msgr2=0x7f78fc0ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc069c80 0x7f78fc0ccaa0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00ec80 tx=0x7f78ec0091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 shutdown_connections 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f78e0066f30 0x7f78e00693e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78fc15b260 0x7f78fc152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc069c80 0x7f78fc0ccaa0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 --2- 172.21.15.7:0/1009667532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78fc156760 0x7f78fc0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.394+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 >> 172.21.15.7:0/1009667532 conn(0x7f78fc0cd910 msgr2=0x7f78fc157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.395+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 shutdown_connections 2024-01-11T07:13:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.395+0000 7f79035a2700 1 -- 172.21.15.7:0/1009667532 wait complete. 2024-01-11T07:13:57.411 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-01-11T07:13:57.412 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2024-01-11T07:13:57.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-01-11T07:13:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.548+0000 7f1f119e0700 1 Processor -- start 2024-01-11T07:13:57.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.548+0000 7f1f119e0700 1 -- start start 2024-01-11T07:13:57.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.548+0000 7f1f119e0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c0b5d50 0x7f1f0c165070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:57.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.548+0000 7f1f119e0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c0d0730 con 0x7f1f0c06de80 2024-01-11T07:13:57.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f119e0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c0cfb20 con 0x7f1f0c06e250 2024-01-11T07:13:57.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f119e0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0cfe00 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f0affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f0c06de80 0x7f1f0c0afee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34162/0 (socket says 172.21.15.7:34162) 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f0affd700 1 -- 172.21.15.7:0/709189875 learned_addr learned my addr 172.21.15.7:0/709189875 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f0b7fe700 1 --2- 172.21.15.7:0/709189875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c0b5d50 0x7f1f0c165070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f09ffb700 1 -- 172.21.15.7:0/709189875 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905961618 0 0) 0x7f1f0c0d0730 con 0x7f1f0c06de80 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f09ffb700 1 -- 172.21.15.7:0/709189875 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef4003570 con 0x7f1f0c06de80 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f09ffb700 1 -- 172.21.15.7:0/709189875 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075070844 0 0) 0x7f1f0c0cfb20 con 0x7f1f0c06e250 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f09ffb700 1 -- 172.21.15.7:0/709189875 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f0c0d0730 con 0x7f1f0c06e250 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f0b7fe700 1 -- 172.21.15.7:0/709189875 >> v1:172.21.15.113:6789/0 conn(0x7f1f0c06e250 legacy=0x7f1f0c159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f0b7fe700 1 -- 172.21.15.7:0/709189875 >> v1:172.21.15.7:6789/0 conn(0x7f1f0c06de80 legacy=0x7f1f0c0afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f0b7fe700 1 -- 172.21.15.7:0/709189875 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0d00e0 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.549+0000 7f1f0b7fe700 1 --2- 172.21.15.7:0/709189875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c0b5d50 0x7f1f0c165070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009ec0 tx=0x7f1f04009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9816fb963965248e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f09ffb700 1 -- 172.21.15.7:0/709189875 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400e070 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f09ffb700 1 -- 172.21.15.7:0/709189875 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0400bb40 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f09ffb700 1 -- 172.21.15.7:0/709189875 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04012430 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f119e0700 1 -- 172.21.15.7:0/709189875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c0b5d50 msgr2=0x7f1f0c165070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f119e0700 1 --2- 172.21.15.7:0/709189875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c0b5d50 0x7f1f0c165070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009ec0 tx=0x7f1f04009780 comp rx=0 tx=0).stop 2024-01-11T07:13:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f119e0700 1 -- 172.21.15.7:0/709189875 shutdown_connections 2024-01-11T07:13:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f119e0700 1 --2- 172.21.15.7:0/709189875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c0b5d50 0x7f1f0c165070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f119e0700 1 -- 172.21.15.7:0/709189875 >> 172.21.15.7:0/709189875 conn(0x7f1f0c071120 msgr2=0x7f1f0c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f119e0700 1 -- 172.21.15.7:0/709189875 shutdown_connections 2024-01-11T07:13:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.550+0000 7f1f119e0700 1 -- 172.21.15.7:0/709189875 wait complete. 2024-01-11T07:13:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.552+0000 7f1f119e0700 1 Processor -- start 2024-01-11T07:13:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.552+0000 7f1f119e0700 1 -- start start 2024-01-11T07:13:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f119e0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c06de80 0x7f1f0c16ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f119e0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c06e250 0x7f1f0c151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f119e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f0c0b5d50 0x7f1f0c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f119e0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c16b160 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f119e0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c15c1f0 con 0x7f1f0c06e250 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f119e0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c15a850 con 0x7f1f0c06de80 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f0c0b5d50 0x7f1f0c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f0c0b5d50 0x7f1f0c151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52770/0 (socket says 172.21.15.7:52770) 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 -- 172.21.15.7:0/994711010 learned_addr learned my addr 172.21.15.7:0/994711010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0affd700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c06de80 0x7f1f0c16ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0a7fc700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c06e250 0x7f1f0c151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 -- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c06de80 msgr2=0x7f1f0c16ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c06de80 0x7f1f0c16ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 -- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c06e250 msgr2=0x7f1f0c151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c06e250 0x7f1f0c151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 -- 172.21.15.7:0/994711010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f04009a20 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0a7fc700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c06e250 0x7f1f0c151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:13:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f0b7fe700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f0c0b5d50 0x7f1f0c151ba0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1f04013000 tx=0x7f1f04019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1ef2ffd700 1 -- 172.21.15.7:0/994711010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04012e00 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c15a290 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1ef2ffd700 1 -- 172.21.15.7:0/994711010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0401b630 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.553+0000 7f1ef2ffd700 1 -- 172.21.15.7:0/994711010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0401b9f0 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.554+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c15be80 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.555+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c15c3a0 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.555+0000 7f1ef2ffd700 1 -- 172.21.15.7:0/994711010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f0401c400 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.556+0000 7f1ef2ffd700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1ef8066f30 0x7f1ef80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.556+0000 7f1ef2ffd700 1 -- 172.21.15.7:0/994711010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1f04085cc0 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.556+0000 7f1f0affd700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1ef8066f30 0x7f1ef80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.556+0000 7f1f0affd700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1ef8066f30 0x7f1ef80693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c1555e0 tx=0x7f1efc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.560+0000 7f1ef2ffd700 1 -- 172.21.15.7:0/994711010 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f0c15c3a0 con 0x7f1f0c0b5d50 2024-01-11T07:13:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:57.851+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f1f0c089350 con 0x7f1f0c0b5d50 2024-01-11T07:13:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.396+0000 7f1ef2ffd700 1 -- 172.21.15.7:0/994711010 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v361) v1 ==== 135+0+0 (secure 0 0 0) 0x7f1f04053230 con 0x7f1f0c0b5d50 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1ef8066f30 msgr2=0x7f1ef80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1ef8066f30 0x7f1ef80693e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c1555e0 tx=0x7f1efc009040 comp rx=0 tx=0).stop 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f0c0b5d50 msgr2=0x7f1f0c151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f0c0b5d50 0x7f1f0c151ba0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1f04013000 tx=0x7f1f04019000 comp rx=0 tx=0).stop 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 shutdown_connections 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1ef8066f30 0x7f1ef80693e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f0c0b5d50 0x7f1f0c151ba0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f0c06de80 0x7f1f0c16ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 --2- 172.21.15.7:0/994711010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f0c06e250 0x7f1f0c151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 >> 172.21.15.7:0/994711010 conn(0x7f1f0c071120 msgr2=0x7f1f0c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 shutdown_connections 2024-01-11T07:13:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.399+0000 7f1f119e0700 1 -- 172.21.15.7:0/994711010 wait complete. 2024-01-11T07:13:58.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:13:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.553+0000 7f1258ba0700 1 Processor -- start 2024-01-11T07:13:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.553+0000 7f1258ba0700 1 -- start start 2024-01-11T07:13:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1258ba0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12540af7a0 0x7f125406a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1258ba0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 0x7f125406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1258ba0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125415b860 con 0x7f12540ada70 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1258ba0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125415a9d0 con 0x7f12540af7a0 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1258ba0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125415b530 con 0x7f12540ade40 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1251d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12540af7a0 0x7f125406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1251d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12540af7a0 0x7f125406a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52782/0 (socket says 172.21.15.7:52782) 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1251d9b700 1 -- 172.21.15.7:0/1186201434 learned_addr learned my addr 172.21.15.7:0/1186201434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1252d9d700 1 --2- 172.21.15.7:0/1186201434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 0x7f125406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1251d9b700 1 -- 172.21.15.7:0/1186201434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 msgr2=0x7f125406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1251d9b700 1 --2- 172.21.15.7:0/1186201434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 0x7f125406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1251d9b700 1 -- 172.21.15.7:0/1186201434 >> v1:172.21.15.113:6789/0 conn(0x7f12540ada70 legacy=0x7f125415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.554+0000 7f1251d9b700 1 -- 172.21.15.7:0/1186201434 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125415ac30 con 0x7f12540af7a0 2024-01-11T07:13:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f1251d9b700 1 --2- 172.21.15.7:0/1186201434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f125406a600 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f12480099b0 tx=0x7f1248009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfd6372b524b3c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f125159a700 1 -- 172.21.15.7:0/1186201434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124800b820 con 0x7f12540af7a0 2024-01-11T07:13:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f125159a700 1 -- 172.21.15.7:0/1186201434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124800b980 con 0x7f12540af7a0 2024-01-11T07:13:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f125159a700 1 -- 172.21.15.7:0/1186201434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124800bd20 con 0x7f12540af7a0 2024-01-11T07:13:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f1258ba0700 1 -- 172.21.15.7:0/1186201434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 msgr2=0x7f125406a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f1258ba0700 1 --2- 172.21.15.7:0/1186201434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f125406a600 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f12480099b0 tx=0x7f1248009250 comp rx=0 tx=0).stop 2024-01-11T07:13:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f1258ba0700 1 -- 172.21.15.7:0/1186201434 shutdown_connections 2024-01-11T07:13:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f1258ba0700 1 --2- 172.21.15.7:0/1186201434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 0x7f125406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f1258ba0700 1 --2- 172.21.15.7:0/1186201434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f125406a600 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.555+0000 7f1258ba0700 1 -- 172.21.15.7:0/1186201434 >> 172.21.15.7:0/1186201434 conn(0x7f12540cd070 msgr2=0x7f12540cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.556+0000 7f1258ba0700 1 -- 172.21.15.7:0/1186201434 shutdown_connections 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.556+0000 7f1258ba0700 1 -- 172.21.15.7:0/1186201434 wait complete. 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.557+0000 7f1258ba0700 1 Processor -- start 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.557+0000 7f1258ba0700 1 -- start start 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.557+0000 7f1258ba0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540ada70 0x7f1254144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.557+0000 7f1258ba0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 0x7f12540b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.557+0000 7f1258ba0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f12541589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.557+0000 7f1258ba0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12541450c0 con 0x7f12540af7a0 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.557+0000 7f1258ba0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12540b2bf0 con 0x7f12540ada70 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1258ba0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540b0590 con 0x7f12540ade40 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f12541589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f12541589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52790/0 (socket says 172.21.15.7:52790) 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 -- 172.21.15.7:0/2608268789 learned_addr learned my addr 172.21.15.7:0/2608268789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1251d9b700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 0x7f12540b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f125259c700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540ada70 0x7f1254144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 -- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 msgr2=0x7f12540b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 0x7f12540b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 -- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540ada70 msgr2=0x7f1254144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540ada70 0x7f1254144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 -- 172.21.15.7:0/2608268789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480094f0 con 0x7f12540af7a0 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1252d9d700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f12541589f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f124c009fd0 tx=0x7f124c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f12427fc700 1 -- 172.21.15.7:0/2608268789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00ebf0 con 0x7f12540af7a0 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f12427fc700 1 -- 172.21.15.7:0/2608268789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c00ed50 con 0x7f12540af7a0 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125416b620 con 0x7f12540af7a0 2024-01-11T07:13:58.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f12427fc700 1 -- 172.21.15.7:0/2608268789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c014670 con 0x7f12540af7a0 2024-01-11T07:13:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.558+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12540aff40 con 0x7f12540af7a0 2024-01-11T07:13:58.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.559+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12540b0870 con 0x7f12540af7a0 2024-01-11T07:13:58.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.560+0000 7f12427fc700 1 -- 172.21.15.7:0/2608268789 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f124c0149e0 con 0x7f12540af7a0 2024-01-11T07:13:58.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.560+0000 7f12427fc700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1234066f30 0x7f12340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:58.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.561+0000 7f12427fc700 1 -- 172.21.15.7:0/2608268789 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f124c0848a0 con 0x7f12540af7a0 2024-01-11T07:13:58.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.561+0000 7f125259c700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1234066f30 0x7f12340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:58.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.561+0000 7f125259c700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1234066f30 0x7f12340693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f125406f0d0 tx=0x7f1244007ea0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:58.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.565+0000 7f12427fc700 1 -- 172.21.15.7:0/2608268789 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f124c051e10 con 0x7f12540af7a0 2024-01-11T07:13:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.771+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f125404c3c0 con 0x7f12540af7a0 2024-01-11T07:13:58.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.775+0000 7f12427fc700 1 -- 172.21.15.7:0/2608268789 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f124c055430 con 0x7f12540af7a0 2024-01-11T07:13:58.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1234066f30 msgr2=0x7f12340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:58.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1234066f30 0x7f12340693e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f125406f0d0 tx=0x7f1244007ea0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 msgr2=0x7f12541589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f12541589f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f124c009fd0 tx=0x7f124c009900 comp rx=0 tx=0).stop 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 shutdown_connections 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1234066f30 0x7f12340693e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540af7a0 0x7f12541589f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12540ada70 0x7f1254144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.777+0000 7f1258ba0700 1 --2- 172.21.15.7:0/2608268789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12540ade40 0x7f12540b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.778+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 >> 172.21.15.7:0/2608268789 conn(0x7f12540cd070 msgr2=0x7f12540aaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.778+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 shutdown_connections 2024-01-11T07:13:58.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:58.778+0000 7f1258ba0700 1 -- 172.21.15.7:0/2608268789 wait complete. 2024-01-11T07:13:58.791 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:13:58.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:13:58.837 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:13:58.888 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:13:58.888 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:13:58.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:13:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.026+0000 7f5ee5795700 1 Processor -- start 2024-01-11T07:13:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.026+0000 7f5ee5795700 1 -- start start 2024-01-11T07:13:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.026+0000 7f5ee5795700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ee00cd6d0 0x7f5ee00cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.026+0000 7f5ee5795700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015e450 0x7f5ee0159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.026+0000 7f5ee5795700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee015acf0 con 0x7f5ee015d8a0 2024-01-11T07:13:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.026+0000 7f5ee5795700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee015afd0 con 0x7f5ee00cd6d0 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.026+0000 7f5ee5795700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00ce980 con 0x7f5ee015e450 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5ede7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5ee015d8a0 0x7f5ee015dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34178/0 (socket says 172.21.15.7:34178) 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5ede7fc700 1 -- 172.21.15.7:0/1753689898 learned_addr learned my addr 172.21.15.7:0/1753689898 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5edeffd700 1 --2- 172.21.15.7:0/1753689898 >> v2:172.21.15.113:3300/0 conn(0x7f5ee00cd6d0 0x7f5ee00cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5edf7fe700 1 --2- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015e450 0x7f5ee0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5eddffb700 1 -- 172.21.15.7:0/1753689898 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705984627 0 0) 0x7f5ee015acf0 con 0x7f5ee015d8a0 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5eddffb700 1 -- 172.21.15.7:0/1753689898 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ecc003570 con 0x7f5ee015d8a0 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5edeffd700 1 -- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015e450 msgr2=0x7f5ee0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5edeffd700 1 --2- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015e450 0x7f5ee0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5edeffd700 1 -- 172.21.15.7:0/1753689898 >> v1:172.21.15.7:6789/0 conn(0x7f5ee015d8a0 legacy=0x7f5ee015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5edeffd700 1 -- 172.21.15.7:0/1753689898 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00cec60 con 0x7f5ee00cd6d0 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5edf7fe700 1 --2- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015e450 0x7f5ee0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.027+0000 7f5edeffd700 1 --2- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 0x7f5ee00cdaa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80098e0 tx=0x7f5ec80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eca01ab6540f7f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5eddffb700 1 -- 172.21.15.7:0/1753689898 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8011630 con 0x7f5ee00cd6d0 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5eddffb700 1 -- 172.21.15.7:0/1753689898 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec8011790 con 0x7f5ee00cd6d0 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5eddffb700 1 -- 172.21.15.7:0/1753689898 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8011b30 con 0x7f5ee00cd6d0 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5ee5795700 1 -- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 msgr2=0x7f5ee00cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5ee5795700 1 --2- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 0x7f5ee00cdaa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80098e0 tx=0x7f5ec80091a0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5ee5795700 1 -- 172.21.15.7:0/1753689898 shutdown_connections 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5ee5795700 1 --2- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015e450 0x7f5ee0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5ee5795700 1 --2- 172.21.15.7:0/1753689898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 0x7f5ee00cdaa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5ee5795700 1 -- 172.21.15.7:0/1753689898 >> 172.21.15.7:0/1753689898 conn(0x7f5ee0071120 msgr2=0x7f5ee006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5ee5795700 1 -- 172.21.15.7:0/1753689898 shutdown_connections 2024-01-11T07:13:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.028+0000 7f5ee5795700 1 -- 172.21.15.7:0/1753689898 wait complete. 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.030+0000 7f5ee5795700 1 Processor -- start 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.030+0000 7f5ee5795700 1 -- start start 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.030+0000 7f5ee5795700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 0x7f5ee015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.030+0000 7f5ee5795700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015d8a0 0x7f5ee0154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.030+0000 7f5ee5795700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee015e450 0x7f5ee0152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.030+0000 7f5ee5795700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee015d4e0 con 0x7f5ee015e450 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.030+0000 7f5ee5795700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee0155a20 con 0x7f5ee00cd6d0 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.030+0000 7f5ee5795700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee0154bc0 con 0x7f5ee015d8a0 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee015e450 0x7f5ee0152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee015e450 0x7f5ee0152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52800/0 (socket says 172.21.15.7:52800) 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 -- 172.21.15.7:0/207890193 learned_addr learned my addr 172.21.15.7:0/207890193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edeffd700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 0x7f5ee015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5ede7fc700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015d8a0 0x7f5ee0154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 -- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015d8a0 msgr2=0x7f5ee0154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015d8a0 0x7f5ee0154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 -- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 msgr2=0x7f5ee015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 0x7f5ee015cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 -- 172.21.15.7:0/207890193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8009a20 con 0x7f5ee015e450 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5edf7fe700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee015e450 0x7f5ee0152c60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009f50 tx=0x7f5ed800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5ed6ffd700 1 -- 172.21.15.7:0/207890193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed800cb50 con 0x7f5ee015e450 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5ed6ffd700 1 -- 172.21.15.7:0/207890193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed800f040 con 0x7f5ee015e450 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec8009420 con 0x7f5ee015e450 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5ed6ffd700 1 -- 172.21.15.7:0/207890193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8014480 con 0x7f5ee015e450 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.031+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee01550f0 con 0x7f5ee015e450 2024-01-11T07:13:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.032+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee004c3c0 con 0x7f5ee015e450 2024-01-11T07:13:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.033+0000 7f5ed6ffd700 1 -- 172.21.15.7:0/207890193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5ed8003020 con 0x7f5ee015e450 2024-01-11T07:13:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.033+0000 7f5ed6ffd700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ec0066f30 0x7f5ec00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.033+0000 7f5ed6ffd700 1 -- 172.21.15.7:0/207890193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5ed8084bd0 con 0x7f5ee015e450 2024-01-11T07:13:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.034+0000 7f5edeffd700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ec0066f30 0x7f5ec00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.034+0000 7f5edeffd700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ec0066f30 0x7f5ec00693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80091e0 tx=0x7f5ec8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.038+0000 7f5ed6ffd700 1 -- 172.21.15.7:0/207890193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ed80520a0 con 0x7f5ee015e450 2024-01-11T07:13:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.292+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ee00ce490 con 0x7f5ee015e450 2024-01-11T07:13:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.292+0000 7f5ed6ffd700 1 -- 172.21.15.7:0/207890193 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f5ed8018dd0 con 0x7f5ee015e450 2024-01-11T07:13:59.293 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:13:59.293 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6140,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4119556214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4119556214},{"type":"v1","addr":"172.21.15.113:6837","nonce":4119556214}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/563869587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":563869587},{"type":"v1","addr":"172.21.15.7:6834","nonce":563869587}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6808,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2013725052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2013725052},{"type":"v1","addr":"172.21.15.113:6836","nonce":2013725052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":7350,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/71264649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":71264649},{"type":"v1","addr":"172.21.15.7:6835","nonce":71264649}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[]} 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ec0066f30 msgr2=0x7f5ec00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ec0066f30 0x7f5ec00693e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80091e0 tx=0x7f5ec8017000 comp rx=0 tx=0).stop 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee015e450 msgr2=0x7f5ee0152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee015e450 0x7f5ee0152c60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009f50 tx=0x7f5ed800b040 comp rx=0 tx=0).stop 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 shutdown_connections 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ec0066f30 0x7f5ec00693e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee015e450 0x7f5ee0152c60 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00cd6d0 0x7f5ee015cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 --2- 172.21.15.7:0/207890193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ee015d8a0 0x7f5ee0154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 >> 172.21.15.7:0/207890193 conn(0x7f5ee0071120 msgr2=0x7f5ee015b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 shutdown_connections 2024-01-11T07:13:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.295+0000 7f5ee5795700 1 -- 172.21.15.7:0/207890193 wait complete. 2024-01-11T07:13:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2024-01-11T07:13:59.309 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:13:59.310 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:13:59.310 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:13:59.310 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:13:59.310 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:13:59.310 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:13:59.310 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:13:59.311 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:13:59.311 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:13:59.311 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:13:59.311 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:13:59.311 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:13:59.311 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:13:59.311 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:13:59.312 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:13:59.312 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:13:59.319 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:13:59.319 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:13:59.320 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:13:59.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:13:59.323 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:13:59.323 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:13:59.324 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:13:59.324 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:13:59.362 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:13:59.363 DEBUG:teuthology.parallel:result is None 2024-01-11T07:13:59.363 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:13:59.363 DEBUG:teuthology.parallel:result is None 2024-01-11T07:13:59.366 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:13:59.366 DEBUG:teuthology.parallel:result is None 2024-01-11T07:13:59.367 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:13:59.367 DEBUG:teuthology.parallel:result is None 2024-01-11T07:13:59.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:13:59.452 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:13:59.453 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:13:59.453 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:13:59.453 INFO:tasks.ceph.mds.d.smithi113.stderr:2024-01-11T07:13:59.454+0000 7f19c355c700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-01-11T07:13:59.454 INFO:tasks.ceph.mds.b.smithi113.stderr:2024-01-11T07:13:59.455+0000 7fc85e861700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-01-11T07:13:59.462 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:13:59.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.513+0000 7f18ded66700 1 Processor -- start 2024-01-11T07:13:59.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.514+0000 7f18ded66700 1 -- start start 2024-01-11T07:13:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.514+0000 7f18ded66700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18d80cd6d0 0x7f18d80cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.514+0000 7f18ded66700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18d815d8a0 0x7f18d815dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.514+0000 7f18ded66700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d815e280 0x7f18d80cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.514+0000 7f18ded66700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d815e700 con 0x7f18d80cd6d0 2024-01-11T07:13:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.514+0000 7f18ded66700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d815acf0 con 0x7f18d815d8a0 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.514+0000 7f18ded66700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d815afd0 con 0x7f18d815e280 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18d80cd6d0 0x7f18d80cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18d80cd6d0 0x7f18d80cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52848/0 (socket says 172.21.15.7:52848) 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 -- 172.21.15.7:0/1345053358 learned_addr learned my addr 172.21.15.7:0/1345053358 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18d7fff700 1 --2- 172.21.15.7:0/1345053358 >> v2:172.21.15.113:3300/0 conn(0x7f18d815d8a0 0x7f18d815dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dd303700 1 --2- 172.21.15.7:0/1345053358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d815e280 0x7f18d80cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 -- 172.21.15.7:0/1345053358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d815e280 msgr2=0x7f18d80cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 --2- 172.21.15.7:0/1345053358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d815e280 0x7f18d80cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 -- 172.21.15.7:0/1345053358 >> v2:172.21.15.113:3300/0 conn(0x7f18d815d8a0 msgr2=0x7f18d815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 --2- 172.21.15.7:0/1345053358 >> v2:172.21.15.113:3300/0 conn(0x7f18d815d8a0 0x7f18d815dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 -- 172.21.15.7:0/1345053358 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d80ce980 con 0x7f18d80cd6d0 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18d7fff700 1 --2- 172.21.15.7:0/1345053358 >> v2:172.21.15.113:3300/0 conn(0x7f18d815d8a0 0x7f18d815dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18dcb02700 1 --2- 172.21.15.7:0/1345053358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d80cd6d0 0x7f18d80cdaa0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f18d80cc620 tx=0x7f18d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f19f2ba44be2fc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18d77fe700 1 -- 172.21.15.7:0/1345053358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d000e070 con 0x7f18d80cd6d0 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18d77fe700 1 -- 172.21.15.7:0/1345053358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d000b970 con 0x7f18d80cd6d0 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.515+0000 7f18d77fe700 1 -- 172.21.15.7:0/1345053358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00092e0 con 0x7f18d80cd6d0 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 -- 172.21.15.7:0/1345053358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d80cd6d0 msgr2=0x7f18d80cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 --2- 172.21.15.7:0/1345053358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d80cd6d0 0x7f18d80cdaa0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f18d80cc620 tx=0x7f18d0009840 comp rx=0 tx=0).stop 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 -- 172.21.15.7:0/1345053358 shutdown_connections 2024-01-11T07:13:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 --2- 172.21.15.7:0/1345053358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d80cd6d0 0x7f18d80cdaa0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 --2- 172.21.15.7:0/1345053358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d815e280 0x7f18d80cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 --2- 172.21.15.7:0/1345053358 >> v2:172.21.15.113:3300/0 conn(0x7f18d815d8a0 0x7f18d815dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 -- 172.21.15.7:0/1345053358 >> 172.21.15.7:0/1345053358 conn(0x7f18d8071120 msgr2=0x7f18d806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:13:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 -- 172.21.15.7:0/1345053358 shutdown_connections 2024-01-11T07:13:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.516+0000 7f18ded66700 1 -- 172.21.15.7:0/1345053358 wait complete. 2024-01-11T07:13:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.517+0000 7f18ded66700 1 Processor -- start 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.517+0000 7f18ded66700 1 -- start start 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18ded66700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d80cd6d0 0x7f18d80b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18ded66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d815d8a0 0x7f18d815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18ded66700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d815e280 0x7f18d8148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18ded66700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d815d550 con 0x7f18d815d8a0 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18ded66700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d8169e90 con 0x7f18d815e280 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18ded66700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80b4020 con 0x7f18d80cd6d0 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d815d8a0 0x7f18d815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d815d8a0 0x7f18d815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52854/0 (socket says 172.21.15.7:52854) 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.518+0000 7f18d7fff700 1 -- 172.21.15.7:0/3451052492 learned_addr learned my addr 172.21.15.7:0/3451052492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18dd303700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d815e280 0x7f18d8148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18dcb02700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d80cd6d0 0x7f18d80b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d7fff700 1 -- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d80cd6d0 msgr2=0x7f18d80b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d7fff700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d80cd6d0 0x7f18d80b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d7fff700 1 -- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d815e280 msgr2=0x7f18d8148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d7fff700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d815e280 0x7f18d8148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d7fff700 1 -- 172.21.15.7:0/3451052492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009420 con 0x7f18d815d8a0 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18dd303700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d815e280 0x7f18d8148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d7fff700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d815d8a0 0x7f18d815d010 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f18c000ec80 tx=0x7f18c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d4ff9700 1 -- 172.21.15.7:0/3451052492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000c850 con 0x7f18d815d8a0 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d4ff9700 1 -- 172.21.15.7:0/3451052492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c000c9b0 con 0x7f18d815d8a0 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d0009ae0 con 0x7f18d815d8a0 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18d4ff9700 1 -- 172.21.15.7:0/3451052492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000cd30 con 0x7f18d815d8a0 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.519+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d80b45e0 con 0x7f18d815d8a0 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.520+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d804c3c0 con 0x7f18d815d8a0 2024-01-11T07:13:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.521+0000 7f18d4ff9700 1 -- 172.21.15.7:0/3451052492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f18c0013680 con 0x7f18d815d8a0 2024-01-11T07:13:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.522+0000 7f18d4ff9700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18b8066f30 0x7f18b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:13:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.522+0000 7f18d4ff9700 1 -- 172.21.15.7:0/3451052492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f18c0084d20 con 0x7f18d815d8a0 2024-01-11T07:13:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.522+0000 7f18dcb02700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18b8066f30 0x7f18b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:13:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.522+0000 7f18dcb02700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18b8066f30 0x7f18b80693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009880 tx=0x7f18d0018000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:13:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.526+0000 7f18d4ff9700 1 -- 172.21.15.7:0/3451052492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18c0055da0 con 0x7f18d815d8a0 2024-01-11T07:13:59.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:13:59.746+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f18d816a000 con 0x7f18d815d8a0 2024-01-11T07:14:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.416+0000 7f18d4ff9700 1 -- 172.21.15.7:0/3451052492 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v146) v1 ==== 97+0+0 (secure 0 0 0) 0x7f18c0055930 con 0x7f18d815d8a0 2024-01-11T07:14:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18b8066f30 msgr2=0x7f18b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18b8066f30 0x7f18b80693e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009880 tx=0x7f18d0018000 comp rx=0 tx=0).stop 2024-01-11T07:14:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d815d8a0 msgr2=0x7f18d815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d815d8a0 0x7f18d815d010 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f18c000ec80 tx=0x7f18c00091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 shutdown_connections 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f18b8066f30 0x7f18b80693e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d815e280 0x7f18d8148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18d80cd6d0 0x7f18d80b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 --2- 172.21.15.7:0/3451052492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18d815d8a0 0x7f18d815d010 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 >> 172.21.15.7:0/3451052492 conn(0x7f18d8071120 msgr2=0x7f18d80cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 shutdown_connections 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.419+0000 7f18ded66700 1 -- 172.21.15.7:0/3451052492 wait complete. 2024-01-11T07:14:00.421 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:14:00.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:14:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.572+0000 7fb88421c700 1 Processor -- start 2024-01-11T07:14:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.572+0000 7fb88421c700 1 -- start start 2024-01-11T07:14:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb88421c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb87c0c7b30 0x7fb87c1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb88421c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87c0cb6c0 0x7fb87c153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb88421c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c0cbb20 con 0x7fb87c0c7f00 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb88421c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87c157880 con 0x7fb87c0c7b30 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb88421c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c069a20 con 0x7fb87c0cb6c0 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb8817b7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb87c0c7f00 0x7fb87c0cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34204/0 (socket says 172.21.15.7:34204) 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb8817b7700 1 -- 172.21.15.7:0/2178339883 learned_addr learned my addr 172.21.15.7:0/2178339883 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb8827b9700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87c0cb6c0 0x7fb87c153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb881fb8700 1 --2- 172.21.15.7:0/2178339883 >> v2:172.21.15.113:3300/0 conn(0x7fb87c0c7b30 0x7fb87c1537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644540122 0 0) 0x7fb87c0cbb20 con 0x7fb87c0c7f00 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.573+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb868004680 con 0x7fb87c0c7f00 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813229908 0 0) 0x7fb868004680 con 0x7fb87c0c7f00 2024-01-11T07:14:00.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87c0cb6c0 msgr2=0x7fb87c153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87c0cb6c0 0x7fb87c153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 >> v2:172.21.15.113:3300/0 conn(0x7fb87c0c7b30 msgr2=0x7fb87c1537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 --2- 172.21.15.7:0/2178339883 >> v2:172.21.15.113:3300/0 conn(0x7fb87c0c7b30 0x7fb87c1537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0cbd50 con 0x7fb87c0c7f00 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fb86c001470 con 0x7fb87c0c7f00 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 >> v1:172.21.15.7:6789/0 conn(0x7fb87c0c7f00 legacy=0x7fb87c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb868004b70 0x7fb868004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868005920 0x7fb868005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87c0cb6c0 0x7fb8680062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c157880 con 0x7fb868005920 2024-01-11T07:14:00.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c069a20 con 0x7fb868004b70 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86800cb70 con 0x7fb87c0cb6c0 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb8817b7700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868005920 0x7fb868005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb881fb8700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb868004b70 0x7fb868004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb8827b9700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87c0cb6c0 0x7fb8680062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb881fb8700 1 -- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87c0cb6c0 msgr2=0x7fb8680062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb881fb8700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87c0cb6c0 0x7fb8680062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb881fb8700 1 -- 172.21.15.7:0/2178339883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868005920 msgr2=0x7fb868005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb881fb8700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868005920 0x7fb868005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb881fb8700 1 -- 172.21.15.7:0/2178339883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb868004680 con 0x7fb868004b70 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb8817b7700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb868005920 0x7fb868005d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.574+0000 7fb881fb8700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb868004b70 0x7fb868004f80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb878009c30 tx=0x7fb8780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9fa90773a5e5920 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:00.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87800e710 con 0x7fb868004b70 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87800e870 con 0x7fb868004b70 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb880fb6700 1 -- 172.21.15.7:0/2178339883 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87800ec30 con 0x7fb868004b70 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb88421c700 1 -- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb868004b70 msgr2=0x7fb868004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb88421c700 1 --2- 172.21.15.7:0/2178339883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb868004b70 0x7fb868004f80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb878009c30 tx=0x7fb8780091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb8827b9700 1 -- 172.21.15.7:0/2178339883 reap_dead start 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb88421c700 1 -- 172.21.15.7:0/2178339883 shutdown_connections 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb88421c700 1 -- 172.21.15.7:0/2178339883 >> 172.21.15.7:0/2178339883 conn(0x7fb87c0cd090 msgr2=0x7fb87c0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb88421c700 1 -- 172.21.15.7:0/2178339883 shutdown_connections 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.575+0000 7fb88421c700 1 -- 172.21.15.7:0/2178339883 wait complete. 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.577+0000 7fb88421c700 1 Processor -- start 2024-01-11T07:14:00.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.577+0000 7fb88421c700 1 -- start start 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.577+0000 7fb88421c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb868004b70 0x7fb87c0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.577+0000 7fb88421c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0c55c0 0x7fb87c0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.577+0000 7fb88421c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c151b30 0x7fb87c151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.577+0000 7fb88421c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c1524f0 con 0x7fb87c151b30 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.577+0000 7fb88421c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c16afc0 con 0x7fb87c0c55c0 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.577+0000 7fb88421c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c16a690 con 0x7fb868004b70 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c151b30 0x7fb87c151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c151b30 0x7fb87c151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52874/0 (socket says 172.21.15.7:52874) 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 -- 172.21.15.7:0/3520390908 learned_addr learned my addr 172.21.15.7:0/3520390908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8817b7700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0c55c0 0x7fb87c0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb881fb8700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb868004b70 0x7fb87c0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 -- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb868004b70 msgr2=0x7fb87c0b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb868004b70 0x7fb87c0b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 -- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0c55c0 msgr2=0x7fb87c0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0c55c0 0x7fb87c0aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 -- 172.21.15.7:0/3520390908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c16a970 con 0x7fb87c151b30 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8817b7700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0c55c0 0x7fb87c0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb8827b9700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c151b30 0x7fb87c151fb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb8740094d0 tx=0x7fb87400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb871ffb700 1 -- 172.21.15.7:0/3520390908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400e040 con 0x7fb87c151b30 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb871ffb700 1 -- 172.21.15.7:0/3520390908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87400f040 con 0x7fb87c151b30 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb871ffb700 1 -- 172.21.15.7:0/3520390908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400fe50 con 0x7fb87c151b30 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.578+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c16ac50 con 0x7fb87c151b30 2024-01-11T07:14:00.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.579+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c16a370 con 0x7fb87c151b30 2024-01-11T07:14:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.580+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c04c3c0 con 0x7fb87c151b30 2024-01-11T07:14:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.580+0000 7fb871ffb700 1 -- 172.21.15.7:0/3520390908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb874015640 con 0x7fb87c151b30 2024-01-11T07:14:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.580+0000 7fb871ffb700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb860066f30 0x7fb8600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.580+0000 7fb871ffb700 1 -- 172.21.15.7:0/3520390908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb874084c40 con 0x7fb87c151b30 2024-01-11T07:14:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.581+0000 7fb881fb8700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb860066f30 0x7fb8600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.581+0000 7fb881fb8700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb860066f30 0x7fb8600693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb8780091e0 tx=0x7fb87801a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:00.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.585+0000 7fb871ffb700 1 -- 172.21.15.7:0/3520390908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb874052200 con 0x7fb87c151b30 2024-01-11T07:14:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.873+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb87c16b100 con 0x7fb87c151b30 2024-01-11T07:14:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.874+0000 7fb871ffb700 1 -- 172.21.15.7:0/3520390908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb874055820 con 0x7fb87c151b30 2024-01-11T07:14:00.875 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:14:00.876 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.878 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:14:00.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:14:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:14:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:14:00.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.883 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:14:00.884 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:14:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:14:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:14:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:14:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:14:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:14:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:14:00.886 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:14:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:14:00.887 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:14:00.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:14:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:14:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:14:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.889 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:14:00.890 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:14:00.891 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:14:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:14:00.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:14:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:14:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:00.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:14:00.894 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb860066f30 msgr2=0x7fb8600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb860066f30 0x7fb8600693e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb8780091e0 tx=0x7fb87801a000 comp rx=0 tx=0).stop 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c151b30 msgr2=0x7fb87c151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c151b30 0x7fb87c151fb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb8740094d0 tx=0x7fb87400b040 comp rx=0 tx=0).stop 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 shutdown_connections 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb860066f30 0x7fb8600693e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c151b30 0x7fb87c151fb0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb868004b70 0x7fb87c0b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.876+0000 7fb88421c700 1 --2- 172.21.15.7:0/3520390908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0c55c0 0x7fb87c0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.877+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 >> 172.21.15.7:0/3520390908 conn(0x7fb87c0cd090 msgr2=0x7fb87c0cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.877+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 shutdown_connections 2024-01-11T07:14:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:00.877+0000 7fb88421c700 1 -- 172.21.15.7:0/3520390908 wait complete. 2024-01-11T07:14:00.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f414b841700 1 Processor -- start 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f414b841700 1 -- start start 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f414b841700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41441617c0 0x7f41440cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f414b841700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41440b0430 0x7f41440b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f414b841700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4144161b90 con 0x7f41440b1820 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f414b841700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41440b0df0 con 0x7f41440b0430 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f414b841700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41440b36c0 con 0x7f41441617c0 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f4148ddc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f41440b1820 0x7f41440b4410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34210/0 (socket says 172.21.15.7:34210) 2024-01-11T07:14:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f4148ddc700 1 -- 172.21.15.7:0/4195483501 learned_addr learned my addr 172.21.15.7:0/4195483501 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f4149dde700 1 --2- 172.21.15.7:0/4195483501 >> v2:172.21.15.7:3300/0 conn(0x7f41440b0430 0x7f41440b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.029+0000 7f41495dd700 1 --2- 172.21.15.7:0/4195483501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41441617c0 0x7f41440cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4137fff700 1 -- 172.21.15.7:0/4195483501 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274778183 0 0) 0x7f4144161b90 con 0x7f41440b1820 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4137fff700 1 -- 172.21.15.7:0/4195483501 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4130003570 con 0x7f41440b1820 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4149dde700 1 -- 172.21.15.7:0/4195483501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41441617c0 msgr2=0x7f41440cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4149dde700 1 --2- 172.21.15.7:0/4195483501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41441617c0 0x7f41440cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4149dde700 1 -- 172.21.15.7:0/4195483501 >> v1:172.21.15.7:6789/0 conn(0x7f41440b1820 legacy=0x7f41440b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4149dde700 1 -- 172.21.15.7:0/4195483501 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41440b3930 con 0x7f41440b0430 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4149dde700 1 --2- 172.21.15.7:0/4195483501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b0430 0x7f41440b08b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f413c009e90 tx=0x7f413c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49570147b8cf46fe server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4137fff700 1 -- 172.21.15.7:0/4195483501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c00e070 con 0x7f41440b0430 2024-01-11T07:14:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4137fff700 1 -- 172.21.15.7:0/4195483501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c00bb40 con 0x7f41440b0430 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.030+0000 7f4137fff700 1 -- 172.21.15.7:0/4195483501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c012430 con 0x7f41440b0430 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.031+0000 7f414b841700 1 -- 172.21.15.7:0/4195483501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b0430 msgr2=0x7f41440b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.031+0000 7f414b841700 1 --2- 172.21.15.7:0/4195483501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b0430 0x7f41440b08b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f413c009e90 tx=0x7f413c009780 comp rx=0 tx=0).stop 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.031+0000 7f414b841700 1 -- 172.21.15.7:0/4195483501 shutdown_connections 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.031+0000 7f414b841700 1 --2- 172.21.15.7:0/4195483501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b0430 0x7f41440b08b0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.031+0000 7f414b841700 1 --2- 172.21.15.7:0/4195483501 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41441617c0 0x7f41440cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.031+0000 7f414b841700 1 -- 172.21.15.7:0/4195483501 >> 172.21.15.7:0/4195483501 conn(0x7f4144071100 msgr2=0x7f414406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.031+0000 7f414b841700 1 -- 172.21.15.7:0/4195483501 shutdown_connections 2024-01-11T07:14:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.031+0000 7f414b841700 1 -- 172.21.15.7:0/4195483501 wait complete. 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.032+0000 7f414b841700 1 Processor -- start 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.032+0000 7f414b841700 1 -- start start 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f414b841700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41440b0430 0x7f414414cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f414b841700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b1820 0x7f414416f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f414b841700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41441617c0 0x7f414415fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f414b841700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41440ac530 con 0x7f41440b1820 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f414b841700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414415a3f0 con 0x7f41441617c0 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f414b841700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414416f090 con 0x7f41440b0430 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f4148ddc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b1820 0x7f414416f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f4148ddc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b1820 0x7f414416f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52882/0 (socket says 172.21.15.7:52882) 2024-01-11T07:14:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f4148ddc700 1 -- 172.21.15.7:0/3156343856 learned_addr learned my addr 172.21.15.7:0/3156343856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f41495dd700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41440b0430 0x7f414414cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.033+0000 7f4149dde700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41441617c0 0x7f414415fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f4148ddc700 1 -- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41440b0430 msgr2=0x7f414414cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f4148ddc700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41440b0430 0x7f414414cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f4148ddc700 1 -- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41441617c0 msgr2=0x7f414415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f4148ddc700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41441617c0 0x7f414415fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f4148ddc700 1 -- 172.21.15.7:0/3156343856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c009a20 con 0x7f41440b1820 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f4148ddc700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b1820 0x7f414416f650 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f413800c7c0 tx=0x7f4138007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f41357fa700 1 -- 172.21.15.7:0/3156343856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138005b30 con 0x7f41440b1820 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f41357fa700 1 -- 172.21.15.7:0/3156343856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4138005c90 con 0x7f41440b1820 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f41357fa700 1 -- 172.21.15.7:0/3156343856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138007340 con 0x7f41440b1820 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414416f370 con 0x7f41440b1820 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.034+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41440ac1c0 con 0x7f41440b1820 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.035+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41440abd40 con 0x7f41440b1820 2024-01-11T07:14:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.035+0000 7f41357fa700 1 -- 172.21.15.7:0/3156343856 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f413801b400 con 0x7f41440b1820 2024-01-11T07:14:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.036+0000 7f41357fa700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4128066f30 0x7f41280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.036+0000 7f41357fa700 1 -- 172.21.15.7:0/3156343856 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4138083cf0 con 0x7f41440b1820 2024-01-11T07:14:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.036+0000 7f41495dd700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4128066f30 0x7f41280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.037+0000 7f41495dd700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4128066f30 0x7f41280693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f41440af800 tx=0x7f4140008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.040+0000 7f41357fa700 1 -- 172.21.15.7:0/3156343856 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41440abd40 con 0x7f41440b1820 2024-01-11T07:14:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.243+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41440abd40 con 0x7f41440b1820 2024-01-11T07:14:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.243+0000 7f41357fa700 1 -- 172.21.15.7:0/3156343856 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+12097 (secure 0 0 0) 0x7f41440abd40 con 0x7f41440b1820 2024-01-11T07:14:01.244 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:01.244 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":148,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:14:00.423142+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:14:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4128066f30 msgr2=0x7f41280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4128066f30 0x7f41280693e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f41440af800 tx=0x7f4140008040 comp rx=0 tx=0).stop 2024-01-11T07:14:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b1820 msgr2=0x7f414416f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b1820 0x7f414416f650 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f413800c7c0 tx=0x7f4138007e00 comp rx=0 tx=0).stop 2024-01-11T07:14:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 shutdown_connections 2024-01-11T07:14:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4128066f30 0x7f41280693e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41441617c0 0x7f414415fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f41440b0430 0x7f414414cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 --2- 172.21.15.7:0/3156343856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41440b1820 0x7f414416f650 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 >> 172.21.15.7:0/3156343856 conn(0x7f4144071100 msgr2=0x7f414415ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 shutdown_connections 2024-01-11T07:14:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.246+0000 7f414b841700 1 -- 172.21.15.7:0/3156343856 wait complete. 2024-01-11T07:14:01.261 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:14:01.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:14:01.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:14:01.334 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:14:01.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.398+0000 7f25e31db700 1 Processor -- start 2024-01-11T07:14:01.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.398+0000 7f25e31db700 1 -- start start 2024-01-11T07:14:01.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e31db700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25dc06baa0 0x7f25dc1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e31db700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25dc0cb680 con 0x7f25dc155720 2024-01-11T07:14:01.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e31db700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25dc155af0 con 0x7f25dc06b5e0 2024-01-11T07:14:01.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e31db700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25dc0caa70 con 0x7f25dc06baa0 2024-01-11T07:14:01.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e1778700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25dc06baa0 0x7f25dc1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e0f77700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f25dc155720 0x7f25dc06aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34226/0 (socket says 172.21.15.7:34226) 2024-01-11T07:14:01.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e0f77700 1 -- 172.21.15.7:0/3812367425 learned_addr learned my addr 172.21.15.7:0/3812367425 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e1778700 1 -- 172.21.15.7:0/3812367425 >> v1:172.21.15.113:6789/0 conn(0x7f25dc06b5e0 legacy=0x7f25dc1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e1778700 1 -- 172.21.15.7:0/3812367425 >> v1:172.21.15.7:6789/0 conn(0x7f25dc155720 legacy=0x7f25dc06aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.399+0000 7f25e1778700 1 -- 172.21.15.7:0/3812367425 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc0cad50 con 0x7f25dc06baa0 2024-01-11T07:14:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.400+0000 7f25e1778700 1 --2- 172.21.15.7:0/3812367425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06baa0 0x7f25dc1580e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009e60 tx=0x7f25d4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c9939f28d16d1f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25d37fe700 1 -- 172.21.15.7:0/3812367425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d400e070 con 0x7f25dc06baa0 2024-01-11T07:14:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25d37fe700 1 -- 172.21.15.7:0/3812367425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d400bc30 con 0x7f25dc06baa0 2024-01-11T07:14:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25d37fe700 1 -- 172.21.15.7:0/3812367425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4012430 con 0x7f25dc06baa0 2024-01-11T07:14:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25e31db700 1 -- 172.21.15.7:0/3812367425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06baa0 msgr2=0x7f25dc1580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25e31db700 1 --2- 172.21.15.7:0/3812367425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06baa0 0x7f25dc1580e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009e60 tx=0x7f25d4009780 comp rx=0 tx=0).stop 2024-01-11T07:14:01.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25e31db700 1 -- 172.21.15.7:0/3812367425 shutdown_connections 2024-01-11T07:14:01.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25e31db700 1 --2- 172.21.15.7:0/3812367425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06baa0 0x7f25dc1580e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25e31db700 1 -- 172.21.15.7:0/3812367425 >> 172.21.15.7:0/3812367425 conn(0x7f25dc06ed90 msgr2=0x7f25dc06f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:01.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25e31db700 1 -- 172.21.15.7:0/3812367425 shutdown_connections 2024-01-11T07:14:01.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.401+0000 7f25e31db700 1 -- 172.21.15.7:0/3812367425 wait complete. 2024-01-11T07:14:01.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.403+0000 7f25e31db700 1 Processor -- start 2024-01-11T07:14:01.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.403+0000 7f25e31db700 1 -- start start 2024-01-11T07:14:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e31db700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06b5e0 0x7f25dc0c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e31db700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25dc06baa0 0x7f25dc167ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e31db700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25dc155720 0x7f25dc168fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e31db700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc0c7c40 con 0x7f25dc06b5e0 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e31db700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc168400 con 0x7f25dc155720 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e31db700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc1694e0 con 0x7f25dc06baa0 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06b5e0 0x7f25dc0c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06b5e0 0x7f25dc0c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52902/0 (socket says 172.21.15.7:52902) 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 -- 172.21.15.7:0/914205926 learned_addr learned my addr 172.21.15.7:0/914205926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25d3fff700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25dc06baa0 0x7f25dc167ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e1778700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25dc155720 0x7f25dc168fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 -- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25dc06baa0 msgr2=0x7f25dc167ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25dc06baa0 0x7f25dc167ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 -- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25dc155720 msgr2=0x7f25dc168fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25dc155720 0x7f25dc168fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 -- 172.21.15.7:0/914205926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d4009a20 con 0x7f25dc06b5e0 2024-01-11T07:14:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e0f77700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06b5e0 0x7f25dc0c80d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c560 tx=0x7f25d8001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25d0ff9700 1 -- 172.21.15.7:0/914205926 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8005b30 con 0x7f25dc06b5e0 2024-01-11T07:14:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25d0ff9700 1 -- 172.21.15.7:0/914205926 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8005c90 con 0x7f25dc06b5e0 2024-01-11T07:14:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25d0ff9700 1 -- 172.21.15.7:0/914205926 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8007320 con 0x7f25dc06b5e0 2024-01-11T07:14:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.404+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc0c7030 con 0x7f25dc06b5e0 2024-01-11T07:14:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.405+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc0c75f0 con 0x7f25dc06b5e0 2024-01-11T07:14:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.406+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc0c7f20 con 0x7f25dc06b5e0 2024-01-11T07:14:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.407+0000 7f25d0ff9700 1 -- 172.21.15.7:0/914205926 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f25d801b3b0 con 0x7f25dc06b5e0 2024-01-11T07:14:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.407+0000 7f25d0ff9700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25c8066f30 0x7f25c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.407+0000 7f25d0ff9700 1 -- 172.21.15.7:0/914205926 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f25d8083980 con 0x7f25dc06b5e0 2024-01-11T07:14:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.407+0000 7f25d3fff700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25c8066f30 0x7f25c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.408+0000 7f25d3fff700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25c8066f30 0x7f25c80693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0ce6b0 tx=0x7f25cc000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.411+0000 7f25d0ff9700 1 -- 172.21.15.7:0/914205926 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25d8050fc0 con 0x7f25dc06b5e0 2024-01-11T07:14:01.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.615+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25dc04c3c0 con 0x7f25dc06b5e0 2024-01-11T07:14:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.616+0000 7f25d0ff9700 1 -- 172.21.15.7:0/914205926 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25d80170a0 con 0x7f25dc06b5e0 2024-01-11T07:14:01.617 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:14:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25c8066f30 msgr2=0x7f25c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25c8066f30 0x7f25c80693e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0ce6b0 tx=0x7f25cc000bc0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06b5e0 msgr2=0x7f25dc0c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06b5e0 0x7f25dc0c80d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c560 tx=0x7f25d8001020 comp rx=0 tx=0).stop 2024-01-11T07:14:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 shutdown_connections 2024-01-11T07:14:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f25c8066f30 0x7f25c80693e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25dc155720 0x7f25dc168fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25dc06b5e0 0x7f25dc0c80d0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 --2- 172.21.15.7:0/914205926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25dc06baa0 0x7f25dc167ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 >> 172.21.15.7:0/914205926 conn(0x7f25dc06ed90 msgr2=0x7f25dc15ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 shutdown_connections 2024-01-11T07:14:01.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.618+0000 7f25e31db700 1 -- 172.21.15.7:0/914205926 wait complete. 2024-01-11T07:14:01.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:14:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9adeba700 1 Processor -- start 2024-01-11T07:14:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9adeba700 1 -- start start 2024-01-11T07:14:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9adeba700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a81579d0 0x7ff9a815c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9adeba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9a8157da0 0x7ff9a80cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9adeba700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a806d270 con 0x7ff9a80ce220 2024-01-11T07:14:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9adeba700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a80ce6a0 con 0x7ff9a8157da0 2024-01-11T07:14:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9adeba700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80cda40 con 0x7ff9a81579d0 2024-01-11T07:14:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9a6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9a8157da0 0x7ff9a80cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9a6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9a8157da0 0x7ff9a80cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52904/0 (socket says 172.21.15.7:52904) 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/245039815 learned_addr learned my addr 172.21.15.7:0/245039815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9a77fe700 1 --2- 172.21.15.7:0/245039815 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a81579d0 0x7ff9a815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/245039815 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a81579d0 msgr2=0x7ff9a815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9a6ffd700 1 --2- 172.21.15.7:0/245039815 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a81579d0 0x7ff9a815c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/245039815 >> v1:172.21.15.113:6789/0 conn(0x7ff9a80ce220 legacy=0x7ff9a806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.807+0000 7ff9a6ffd700 1 -- 172.21.15.7:0/245039815 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80cd5b0 con 0x7ff9a8157da0 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9a6ffd700 1 --2- 172.21.15.7:0/245039815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a80cdce0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff9900099b0 tx=0x7ff990009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409020e3ea7f3515 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9a67fc700 1 -- 172.21.15.7:0/245039815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99000b860 con 0x7ff9a8157da0 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9a67fc700 1 -- 172.21.15.7:0/245039815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99000b9c0 con 0x7ff9a8157da0 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9a67fc700 1 -- 172.21.15.7:0/245039815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99000bd60 con 0x7ff9a8157da0 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9adeba700 1 -- 172.21.15.7:0/245039815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 msgr2=0x7ff9a80cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9adeba700 1 --2- 172.21.15.7:0/245039815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a80cdce0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff9900099b0 tx=0x7ff990009250 comp rx=0 tx=0).stop 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9adeba700 1 -- 172.21.15.7:0/245039815 shutdown_connections 2024-01-11T07:14:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9adeba700 1 --2- 172.21.15.7:0/245039815 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a81579d0 0x7ff9a815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9adeba700 1 --2- 172.21.15.7:0/245039815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a80cdce0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9adeba700 1 -- 172.21.15.7:0/245039815 >> 172.21.15.7:0/245039815 conn(0x7ff9a806ae00 msgr2=0x7ff9a806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9adeba700 1 -- 172.21.15.7:0/245039815 shutdown_connections 2024-01-11T07:14:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.808+0000 7ff9adeba700 1 -- 172.21.15.7:0/245039815 wait complete. 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.810+0000 7ff9adeba700 1 Processor -- start 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.810+0000 7ff9adeba700 1 -- start start 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9adeba700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a80ce220 0x7ff9a8151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9adeba700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a81579d0 0x7ff9a8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9adeba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a8159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9adeba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80b0110 con 0x7ff9a8157da0 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9adeba700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a8159180 con 0x7ff9a81579d0 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9adeba700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a8158570 con 0x7ff9a80ce220 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a8159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a8159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52912/0 (socket says 172.21.15.7:52912) 2024-01-11T07:14:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a77fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a80ce220 0x7ff9a8151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 -- 172.21.15.7:0/3357028993 learned_addr learned my addr 172.21.15.7:0/3357028993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a6ffd700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a81579d0 0x7ff9a8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 -- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a80ce220 msgr2=0x7ff9a8151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a80ce220 0x7ff9a8151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 -- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a81579d0 msgr2=0x7ff9a8068b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a81579d0 0x7ff9a8068b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 -- 172.21.15.7:0/3357028993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9900094f0 con 0x7ff9a8157da0 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9a7fff700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a8159370 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c910 tx=0x7ff9a0001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff99f7fe700 1 -- 172.21.15.7:0/3357028993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0012630 con 0x7ff9a8157da0 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff99f7fe700 1 -- 172.21.15.7:0/3357028993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0012790 con 0x7ff9a8157da0 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff99f7fe700 1 -- 172.21.15.7:0/3357028993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0012b50 con 0x7ff9a8157da0 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a8158850 con 0x7ff9a8157da0 2024-01-11T07:14:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.812+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a8158e10 con 0x7ff9a8157da0 2024-01-11T07:14:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.813+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a804c3c0 con 0x7ff9a8157da0 2024-01-11T07:14:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.813+0000 7ff99f7fe700 1 -- 172.21.15.7:0/3357028993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff9a001a3a0 con 0x7ff9a8157da0 2024-01-11T07:14:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.813+0000 7ff99f7fe700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff988066f30 0x7ff9880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.814+0000 7ff99f7fe700 1 -- 172.21.15.7:0/3357028993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff9a00843d0 con 0x7ff9a8157da0 2024-01-11T07:14:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.814+0000 7ff9a77fe700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff988066f30 0x7ff9880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.814+0000 7ff9a77fe700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff988066f30 0x7ff9880693e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff998006fd0 tx=0x7ff998008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:01.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:01.818+0000 7ff99f7fe700 1 -- 172.21.15.7:0/3357028993 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9a0055420 con 0x7ff9a8157da0 2024-01-11T07:14:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.024+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff9a80cc9a0 con 0x7ff9a8157da0 2024-01-11T07:14:02.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.024+0000 7ff99f7fe700 1 -- 172.21.15.7:0/3357028993 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v148) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff9a0054fb0 con 0x7ff9a8157da0 2024-01-11T07:14:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:02.024 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:14:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:14:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:14:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:14:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:14:02.024 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:14:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff988066f30 msgr2=0x7ff9880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff988066f30 0x7ff9880693e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff998006fd0 tx=0x7ff998008040 comp rx=0 tx=0).stop 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 msgr2=0x7ff9a8159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a8159370 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c910 tx=0x7ff9a0001710 comp rx=0 tx=0).stop 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 shutdown_connections 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff988066f30 0x7ff9880693e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9a8157da0 0x7ff9a8159370 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9a80ce220 0x7ff9a8151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 --2- 172.21.15.7:0/3357028993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a81579d0 0x7ff9a8068b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 >> 172.21.15.7:0/3357028993 conn(0x7ff9a806ae00 msgr2=0x7ff9a80c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 shutdown_connections 2024-01-11T07:14:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.026+0000 7ff9adeba700 1 -- 172.21.15.7:0/3357028993 wait complete. 2024-01-11T07:14:02.039 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:14:02.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:14:02.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.176+0000 7f38a61aa700 1 Processor -- start 2024-01-11T07:14:02.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.176+0000 7f38a61aa700 1 -- start start 2024-01-11T07:14:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f38a61aa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38a0154540 0x7f38a006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f38a61aa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0063850 0x7f38a0069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f38a61aa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a006ac80 con 0x7f38a0063480 2024-01-11T07:14:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f38a61aa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a006e630 con 0x7f38a0154540 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f38a61aa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a006a9a0 con 0x7f38a0063850 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f38a0154540 0x7f38a006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f38a0063480 0x7f38a0153e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34234/0 (socket says 172.21.15.7:34234) 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389f7fe700 1 -- 172.21.15.7:0/2907963917 learned_addr learned my addr 172.21.15.7:0/2907963917 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389ffff700 1 --2- 172.21.15.7:0/2907963917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0063850 0x7f38a0069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389effd700 1 -- 172.21.15.7:0/2907963917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0063850 msgr2=0x7f38a0069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389effd700 1 --2- 172.21.15.7:0/2907963917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0063850 0x7f38a0069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389effd700 1 -- 172.21.15.7:0/2907963917 >> v1:172.21.15.7:6789/0 conn(0x7f38a0063480 legacy=0x7f38a0153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389effd700 1 -- 172.21.15.7:0/2907963917 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a006dfe0 con 0x7f38a0154540 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.177+0000 7f389e7fc700 1 -- 172.21.15.7:0/2907963917 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262363580 0 0) 0x7f38a006ac80 con 0x7f38a0063480 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f389effd700 1 --2- 172.21.15.7:0/2907963917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0154540 0x7f38a006eac0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f38880098e0 tx=0x7f38880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d5e3a56553e080 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f389e7fc700 1 -- 172.21.15.7:0/2907963917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888009e80 con 0x7f38a0154540 2024-01-11T07:14:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f389e7fc700 1 -- 172.21.15.7:0/2907963917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800b860 con 0x7f38a0154540 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f389e7fc700 1 -- 172.21.15.7:0/2907963917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388800bbe0 con 0x7f38a0154540 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f38a61aa700 1 -- 172.21.15.7:0/2907963917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0154540 msgr2=0x7f38a006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f38a61aa700 1 --2- 172.21.15.7:0/2907963917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0154540 0x7f38a006eac0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f38880098e0 tx=0x7f38880091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f38a61aa700 1 -- 172.21.15.7:0/2907963917 shutdown_connections 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f38a61aa700 1 --2- 172.21.15.7:0/2907963917 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0063850 0x7f38a0069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f38a61aa700 1 --2- 172.21.15.7:0/2907963917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0154540 0x7f38a006eac0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f38a61aa700 1 -- 172.21.15.7:0/2907963917 >> 172.21.15.7:0/2907963917 conn(0x7f38a006bce0 msgr2=0x7f38a006c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f38a61aa700 1 -- 172.21.15.7:0/2907963917 shutdown_connections 2024-01-11T07:14:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.178+0000 7f38a61aa700 1 -- 172.21.15.7:0/2907963917 wait complete. 2024-01-11T07:14:02.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.180+0000 7f38a61aa700 1 Processor -- start 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.180+0000 7f38a61aa700 1 -- start start 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f38a61aa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a0063480 0x7f38a00c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f38a61aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0063850 0x7f38a00c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f38a61aa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0154540 0x7f38a00c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f38a61aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00c9510 con 0x7f38a0063850 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f38a61aa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00c97f0 con 0x7f38a0063480 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f38a61aa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00c9ad0 con 0x7f38a0154540 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0063850 0x7f38a00c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0063850 0x7f38a00c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52920/0 (socket says 172.21.15.7:52920) 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 -- 172.21.15.7:0/3919135944 learned_addr learned my addr 172.21.15.7:0/3919135944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389ffff700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0154540 0x7f38a00c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389f7fe700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a0063480 0x7f38a00c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 -- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0154540 msgr2=0x7f38a00c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0154540 0x7f38a00c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 -- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a0063480 msgr2=0x7f38a00c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a0063480 0x7f38a00c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 -- 172.21.15.7:0/3919135944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888009420 con 0x7f38a0063850 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389f7fe700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a0063480 0x7f38a00c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.181+0000 7f389effd700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0063850 0x7f38a00c6af0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3888013000 tx=0x7f3888017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.182+0000 7f38977fe700 1 -- 172.21.15.7:0/3919135944 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388800b9d0 con 0x7f38a0063850 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.182+0000 7f38977fe700 1 -- 172.21.15.7:0/3919135944 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800bb70 con 0x7f38a0063850 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.182+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a00c9db0 con 0x7f38a0063850 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.182+0000 7f38977fe700 1 -- 172.21.15.7:0/3919135944 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888025670 con 0x7f38a0063850 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.182+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a00ca5b0 con 0x7f38a0063850 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.183+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a004c3c0 con 0x7f38a0063850 2024-01-11T07:14:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.183+0000 7f38977fe700 1 -- 172.21.15.7:0/3919135944 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3888023070 con 0x7f38a0063850 2024-01-11T07:14:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.184+0000 7f38977fe700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.184+0000 7f38977fe700 1 -- 172.21.15.7:0/3919135944 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3888085760 con 0x7f38a0063850 2024-01-11T07:14:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.184+0000 7f389f7fe700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:02.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.184+0000 7f389f7fe700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f38900011e0 tx=0x7f3890009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:02.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.188+0000 7f38977fe700 1 -- 172.21.15.7:0/3919135944 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3888052dd0 con 0x7f38a0063850 2024-01-11T07:14:02.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:02.395+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f38a00ca2d0 con 0x7f38a0063850 2024-01-11T07:14:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.247+0000 7f38977fe700 1 -- 172.21.15.7:0/3919135944 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v149) v1 ==== 138+0+0 (secure 0 0 0) 0x7f38a00ca2d0 con 0x7f38a0063850 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3880066f30 msgr2=0x7f38800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f38900011e0 tx=0x7f3890009040 comp rx=0 tx=0).stop 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0063850 msgr2=0x7f38a00c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0063850 0x7f38a00c6af0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3888013000 tx=0x7f3888017000 comp rx=0 tx=0).stop 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 shutdown_connections 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f38900011e0 tx=0x7f3890009040 comp rx=0 tx=0).stop 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38a0154540 0x7f38a00c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a0063480 0x7f38a00c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 --2- 172.21.15.7:0/3919135944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38a0063850 0x7f38a00c6af0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 >> 172.21.15.7:0/3919135944 conn(0x7f38a006bce0 msgr2=0x7f38a00b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 shutdown_connections 2024-01-11T07:14:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.251+0000 7f38a61aa700 1 -- 172.21.15.7:0/3919135944 wait complete. 2024-01-11T07:14:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:14:03.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:14:03.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.411+0000 7fd061f47700 1 Processor -- start 2024-01-11T07:14:03.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.411+0000 7fd061f47700 1 -- start start 2024-01-11T07:14:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd061f47700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd05c0ac3a0 0x7fd05c0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd061f47700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ab8d0 0x7fd05c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd061f47700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd05c15e2d0 con 0x7fd05c15d920 2024-01-11T07:14:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd061f47700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd05c06ac30 con 0x7fd05c0ac3a0 2024-01-11T07:14:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd061f47700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0ac770 con 0x7fd05c0ab8d0 2024-01-11T07:14:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd05affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd05c15d920 0x7fd05c15a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34236/0 (socket says 172.21.15.7:34236) 2024-01-11T07:14:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd05affd700 1 -- 172.21.15.7:0/872233750 learned_addr learned my addr 172.21.15.7:0/872233750 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd05b7fe700 1 --2- 172.21.15.7:0/872233750 >> v2:172.21.15.113:3300/0 conn(0x7fd05c0ac3a0 0x7fd05c0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd05bfff700 1 --2- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ab8d0 0x7fd05c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.412+0000 7fd05a7fc700 1 -- 172.21.15.7:0/872233750 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884921952 0 0) 0x7fd05c15e2d0 con 0x7fd05c15d920 2024-01-11T07:14:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05a7fc700 1 -- 172.21.15.7:0/872233750 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd044003570 con 0x7fd05c15d920 2024-01-11T07:14:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05b7fe700 1 -- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ab8d0 msgr2=0x7fd05c15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05b7fe700 1 --2- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ab8d0 0x7fd05c15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05b7fe700 1 -- 172.21.15.7:0/872233750 >> v1:172.21.15.7:6789/0 conn(0x7fd05c15d920 legacy=0x7fd05c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05b7fe700 1 -- 172.21.15.7:0/872233750 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c06a300 con 0x7fd05c0ac3a0 2024-01-11T07:14:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05bfff700 1 --2- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ab8d0 0x7fd05c15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:14:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05b7fe700 1 --2- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ac3a0 0x7fd05c0cac60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0098e0 tx=0x7fd04c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa9eb6e1a97a568a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05a7fc700 1 -- 172.21.15.7:0/872233750 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c011630 con 0x7fd05c0ac3a0 2024-01-11T07:14:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05a7fc700 1 -- 172.21.15.7:0/872233750 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c011790 con 0x7fd05c0ac3a0 2024-01-11T07:14:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.413+0000 7fd05a7fc700 1 -- 172.21.15.7:0/872233750 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c011b30 con 0x7fd05c0ac3a0 2024-01-11T07:14:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.414+0000 7fd061f47700 1 -- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ac3a0 msgr2=0x7fd05c0cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:03.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.414+0000 7fd061f47700 1 --2- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ac3a0 0x7fd05c0cac60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0098e0 tx=0x7fd04c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.414+0000 7fd061f47700 1 -- 172.21.15.7:0/872233750 shutdown_connections 2024-01-11T07:14:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.414+0000 7fd061f47700 1 --2- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ab8d0 0x7fd05c15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.414+0000 7fd061f47700 1 --2- 172.21.15.7:0/872233750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ac3a0 0x7fd05c0cac60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.414+0000 7fd061f47700 1 -- 172.21.15.7:0/872233750 >> 172.21.15.7:0/872233750 conn(0x7fd05c0cd090 msgr2=0x7fd05c0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.414+0000 7fd061f47700 1 -- 172.21.15.7:0/872233750 shutdown_connections 2024-01-11T07:14:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.414+0000 7fd061f47700 1 -- 172.21.15.7:0/872233750 wait complete. 2024-01-11T07:14:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.415+0000 7fd061f47700 1 Processor -- start 2024-01-11T07:14:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.415+0000 7fd061f47700 1 -- start start 2024-01-11T07:14:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd061f47700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ab8d0 0x7fd05c13dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd061f47700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ac3a0 0x7fd05c0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd061f47700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05c15d920 0x7fd05c15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd061f47700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0afe90 con 0x7fd05c15d920 2024-01-11T07:14:03.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd061f47700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0b00e0 con 0x7fd05c0ab8d0 2024-01-11T07:14:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd061f47700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c0b03c0 con 0x7fd05c0ac3a0 2024-01-11T07:14:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05c15d920 0x7fd05c15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05c15d920 0x7fd05c15acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52928/0 (socket says 172.21.15.7:52928) 2024-01-11T07:14:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05bfff700 1 -- 172.21.15.7:0/4027096363 learned_addr learned my addr 172.21.15.7:0/4027096363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05affd700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ac3a0 0x7fd05c0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:03.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05b7fe700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ab8d0 0x7fd05c13dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05bfff700 1 -- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ac3a0 msgr2=0x7fd05c0cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05bfff700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ac3a0 0x7fd05c0cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05bfff700 1 -- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ab8d0 msgr2=0x7fd05c13dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05bfff700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ab8d0 0x7fd05c13dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05bfff700 1 -- 172.21.15.7:0/4027096363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054009a20 con 0x7fd05c15d920 2024-01-11T07:14:03.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.416+0000 7fd05affd700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ac3a0 0x7fd05c0cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.417+0000 7fd05bfff700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05c15d920 0x7fd05c15acf0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd054009ec0 tx=0x7fd05400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.417+0000 7fd0437fe700 1 -- 172.21.15.7:0/4027096363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05400cc20 con 0x7fd05c15d920 2024-01-11T07:14:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.417+0000 7fd0437fe700 1 -- 172.21.15.7:0/4027096363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05400f040 con 0x7fd05c15d920 2024-01-11T07:14:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.417+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c009420 con 0x7fd05c15d920 2024-01-11T07:14:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.417+0000 7fd0437fe700 1 -- 172.21.15.7:0/4027096363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054014480 con 0x7fd05c15d920 2024-01-11T07:14:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.417+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05c0b0980 con 0x7fd05c15d920 2024-01-11T07:14:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.418+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05c04c3c0 con 0x7fd05c15d920 2024-01-11T07:14:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.418+0000 7fd0437fe700 1 -- 172.21.15.7:0/4027096363 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd054003020 con 0x7fd05c15d920 2024-01-11T07:14:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.419+0000 7fd0437fe700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd048066f30 0x7fd0480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.419+0000 7fd0437fe700 1 -- 172.21.15.7:0/4027096363 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fd054084cc0 con 0x7fd05c15d920 2024-01-11T07:14:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.419+0000 7fd05b7fe700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd048066f30 0x7fd0480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.420+0000 7fd05b7fe700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd048066f30 0x7fd0480693e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0091e0 tx=0x7fd04c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.423+0000 7fd0437fe700 1 -- 172.21.15.7:0/4027096363 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd054052110 con 0x7fd05c15d920 2024-01-11T07:14:03.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:03.632+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd05c0b0b70 con 0x7fd05c15d920 2024-01-11T07:14:04.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.249+0000 7fd0437fe700 1 -- 172.21.15.7:0/4027096363 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v150) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd054055730 con 0x7fd05c15d920 2024-01-11T07:14:04.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.255+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd048066f30 msgr2=0x7fd0480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:04.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.255+0000 7fd061f47700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd048066f30 0x7fd0480693e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0091e0 tx=0x7fd04c017000 comp rx=0 tx=0).stop 2024-01-11T07:14:04.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.255+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05c15d920 msgr2=0x7fd05c15acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:04.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.255+0000 7fd061f47700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05c15d920 0x7fd05c15acf0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd054009ec0 tx=0x7fd05400b040 comp rx=0 tx=0).stop 2024-01-11T07:14:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.257+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 shutdown_connections 2024-01-11T07:14:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.257+0000 7fd061f47700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd048066f30 0x7fd0480693e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.257+0000 7fd061f47700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05c15d920 0x7fd05c15acf0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.257+0000 7fd061f47700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05c0ab8d0 0x7fd05c13dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.257+0000 7fd061f47700 1 --2- 172.21.15.7:0/4027096363 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd05c0ac3a0 0x7fd05c0cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.257+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 >> 172.21.15.7:0/4027096363 conn(0x7fd05c0cd090 msgr2=0x7fd05c06e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:04.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.258+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 shutdown_connections 2024-01-11T07:14:04.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.259+0000 7fd061f47700 1 -- 172.21.15.7:0/4027096363 wait complete. 2024-01-11T07:14:04.263 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:14:04.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:14:04.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.416+0000 7f954bf31700 1 Processor -- start 2024-01-11T07:14:04.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.416+0000 7f954bf31700 1 -- start start 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.417+0000 7f954bf31700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f95440ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.417+0000 7f954bf31700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95440ace80 con 0x7f9544157ed0 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.417+0000 7f954bf31700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95440ad160 con 0x7f9544063480 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.417+0000 7f954bf31700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95440ad440 con 0x7f9544069c60 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.417+0000 7f954a4ce700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9544157ed0 0x7f954406dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34252/0 (socket says 172.21.15.7:34252) 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.417+0000 7f954a4ce700 1 -- 172.21.15.7:0/2435675072 learned_addr learned my addr 172.21.15.7:0/2435675072 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.417+0000 7f95494cc700 1 --2- 172.21.15.7:0/2435675072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f95440ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f9548ccb700 1 -- 172.21.15.7:0/2435675072 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153000033 0 0) 0x7f95440ace80 con 0x7f9544157ed0 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f9548ccb700 1 -- 172.21.15.7:0/2435675072 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9530003570 con 0x7f9544157ed0 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f9548ccb700 1 -- 172.21.15.7:0/2435675072 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360840395 0 0) 0x7f95440ad160 con 0x7f9544063480 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f9548ccb700 1 -- 172.21.15.7:0/2435675072 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95440ace80 con 0x7f9544063480 2024-01-11T07:14:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f95494cc700 1 -- 172.21.15.7:0/2435675072 >> v1:172.21.15.113:6789/0 conn(0x7f9544063480 legacy=0x7f95441577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:04.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f95494cc700 1 -- 172.21.15.7:0/2435675072 >> v1:172.21.15.7:6789/0 conn(0x7f9544157ed0 legacy=0x7f954406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:04.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f95494cc700 1 -- 172.21.15.7:0/2435675072 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440aeeb0 con 0x7f9544069c60 2024-01-11T07:14:04.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f95494cc700 1 --2- 172.21.15.7:0/2435675072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f95440ac7c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9534009910 tx=0x7f95340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1d65a831849f679 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:04.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f9548ccb700 1 -- 172.21.15.7:0/2435675072 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534011630 con 0x7f9544069c60 2024-01-11T07:14:04.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f9548ccb700 1 -- 172.21.15.7:0/2435675072 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9534011790 con 0x7f9544069c60 2024-01-11T07:14:04.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.418+0000 7f9548ccb700 1 -- 172.21.15.7:0/2435675072 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534011b10 con 0x7f9544069c60 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.419+0000 7f954bf31700 1 -- 172.21.15.7:0/2435675072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 msgr2=0x7f95440ac7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.419+0000 7f954bf31700 1 --2- 172.21.15.7:0/2435675072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f95440ac7c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9534009910 tx=0x7f95340091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.419+0000 7f954bf31700 1 -- 172.21.15.7:0/2435675072 shutdown_connections 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.419+0000 7f954bf31700 1 --2- 172.21.15.7:0/2435675072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f95440ac7c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.419+0000 7f954bf31700 1 -- 172.21.15.7:0/2435675072 >> 172.21.15.7:0/2435675072 conn(0x7f95440cd8f0 msgr2=0x7f95440cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.419+0000 7f954bf31700 1 -- 172.21.15.7:0/2435675072 shutdown_connections 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.419+0000 7f954bf31700 1 -- 172.21.15.7:0/2435675072 wait complete. 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.420+0000 7f954bf31700 1 Processor -- start 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.420+0000 7f954bf31700 1 -- start start 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954bf31700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9544063480 0x7f954416aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954bf31700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f9544158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954bf31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9544157ed0 0x7f95441588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954bf31700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954416b530 con 0x7f9544157ed0 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954bf31700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954415b280 con 0x7f9544063480 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954bf31700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954415b560 con 0x7f9544069c60 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954a4ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9544157ed0 0x7f95441588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954a4ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9544157ed0 0x7f95441588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52932/0 (socket says 172.21.15.7:52932) 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954a4ce700 1 -- 172.21.15.7:0/2748267160 learned_addr learned my addr 172.21.15.7:0/2748267160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f9549ccd700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9544063480 0x7f954416aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f95494cc700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f9544158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954a4ce700 1 -- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 msgr2=0x7f9544158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954a4ce700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f9544158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954a4ce700 1 -- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9544063480 msgr2=0x7f954416aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954a4ce700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9544063480 0x7f954416aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.421+0000 7f954a4ce700 1 -- 172.21.15.7:0/2748267160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534009420 con 0x7f9544157ed0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.422+0000 7f95494cc700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f9544158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.422+0000 7f954a4ce700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9544157ed0 0x7f95441588c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f953c001460 tx=0x7f953c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.422+0000 7f9539ffb700 1 -- 172.21.15.7:0/2748267160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00a8b0 con 0x7f9544157ed0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.422+0000 7f9539ffb700 1 -- 172.21.15.7:0/2748267160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c00aa10 con 0x7f9544157ed0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.422+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95440b1830 con 0x7f9544157ed0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.422+0000 7f9539ffb700 1 -- 172.21.15.7:0/2748267160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00add0 con 0x7f9544157ed0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.422+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95440b1df0 con 0x7f9544157ed0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.423+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95440b1fb0 con 0x7f9544157ed0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.424+0000 7f9539ffb700 1 -- 172.21.15.7:0/2748267160 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f953c007300 con 0x7f9544157ed0 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.424+0000 7f9539ffb700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9528066f30 0x7f95280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.424+0000 7f9539ffb700 1 -- 172.21.15.7:0/2748267160 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f953c084810 con 0x7f9544157ed0 2024-01-11T07:14:04.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.424+0000 7f9549ccd700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9528066f30 0x7f95280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:04.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.425+0000 7f9549ccd700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9528066f30 0x7f95280693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9540007fd0 tx=0x7f9540007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:04.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.429+0000 7f9539ffb700 1 -- 172.21.15.7:0/2748267160 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f953c051b60 con 0x7f9544157ed0 2024-01-11T07:14:04.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:04.677+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9544064a40 con 0x7f9544157ed0 2024-01-11T07:14:05.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.446+0000 7f9539ffb700 1 -- 172.21.15.7:0/2748267160 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7f953c055180 con 0x7f9544157ed0 2024-01-11T07:14:05.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.451+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9528066f30 msgr2=0x7f95280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.451+0000 7f954bf31700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9528066f30 0x7f95280693e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9540007fd0 tx=0x7f9540007d50 comp rx=0 tx=0).stop 2024-01-11T07:14:05.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.451+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9544157ed0 msgr2=0x7f95441588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.451+0000 7f954bf31700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9544157ed0 0x7f95441588c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f953c001460 tx=0x7f953c001a20 comp rx=0 tx=0).stop 2024-01-11T07:14:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.452+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 shutdown_connections 2024-01-11T07:14:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.452+0000 7f954bf31700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9528066f30 0x7f95280693e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.452+0000 7f954bf31700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9544157ed0 0x7f95441588c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.452+0000 7f954bf31700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9544063480 0x7f954416aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.452+0000 7f954bf31700 1 --2- 172.21.15.7:0/2748267160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9544069c60 0x7f9544158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.452+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 >> 172.21.15.7:0/2748267160 conn(0x7f95440cd8f0 msgr2=0x7f95440af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.452+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 shutdown_connections 2024-01-11T07:14:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.452+0000 7f954bf31700 1 -- 172.21.15.7:0/2748267160 wait complete. 2024-01-11T07:14:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 18 and data pool 19 2024-01-11T07:14:05.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:14:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.614+0000 7fc121bbe700 1 Processor -- start 2024-01-11T07:14:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.614+0000 7fc121bbe700 1 -- start start 2024-01-11T07:14:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc121bbe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc11c159f70 0x7fc11c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc121bbe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 0x7fc11c0680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc121bbe700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c067c60 con 0x7fc11c159ba0 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc121bbe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c06d730 con 0x7fc11c159f70 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc121bbe700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c069230 con 0x7fc11c0c9570 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc11affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc11c159f70 0x7fc11c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc11b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc11c159ba0 0x7fc11c06d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34268/0 (socket says 172.21.15.7:34268) 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc11b7fe700 1 -- 172.21.15.7:0/1198842285 learned_addr learned my addr 172.21.15.7:0/1198842285 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc11bfff700 1 --2- 172.21.15.7:0/1198842285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 0x7fc11c0680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc11affd700 1 -- 172.21.15.7:0/1198842285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 msgr2=0x7fc11c0680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc11affd700 1 --2- 172.21.15.7:0/1198842285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 0x7fc11c0680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc11affd700 1 -- 172.21.15.7:0/1198842285 >> v1:172.21.15.7:6789/0 conn(0x7fc11c159ba0 legacy=0x7fc11c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.615+0000 7fc11a7fc700 1 -- 172.21.15.7:0/1198842285 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841213001 0 0) 0x7fc11c067c60 con 0x7fc11c159ba0 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc11affd700 1 -- 172.21.15.7:0/1198842285 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c069510 con 0x7fc11c159f70 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc11affd700 1 --2- 172.21.15.7:0/1198842285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c9030 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc1100098e0 tx=0x7fc1100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96de823878f6f421 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc11a7fc700 1 -- 172.21.15.7:0/1198842285 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000b640 con 0x7fc11c159f70 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc11a7fc700 1 -- 172.21.15.7:0/1198842285 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000b7a0 con 0x7fc11c159f70 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc11a7fc700 1 -- 172.21.15.7:0/1198842285 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000bb40 con 0x7fc11c159f70 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc121bbe700 1 -- 172.21.15.7:0/1198842285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 msgr2=0x7fc11c0c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1198842285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c9030 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc1100098e0 tx=0x7fc1100091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc121bbe700 1 -- 172.21.15.7:0/1198842285 shutdown_connections 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1198842285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 0x7fc11c0680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1198842285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c9030 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.616+0000 7fc121bbe700 1 -- 172.21.15.7:0/1198842285 >> 172.21.15.7:0/1198842285 conn(0x7fc11c06ae00 msgr2=0x7fc11c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.617+0000 7fc121bbe700 1 -- 172.21.15.7:0/1198842285 shutdown_connections 2024-01-11T07:14:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.617+0000 7fc121bbe700 1 -- 172.21.15.7:0/1198842285 wait complete. 2024-01-11T07:14:05.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.618+0000 7fc121bbe700 1 Processor -- start 2024-01-11T07:14:05.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.618+0000 7fc121bbe700 1 -- start start 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc121bbe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 0x7fc11c1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc121bbe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c159ba0 0x7fc11c0c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc121bbe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc121bbe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15c390 con 0x7fc11c159f70 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc121bbe700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15b030 con 0x7fc11c159ba0 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc121bbe700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15b310 con 0x7fc11c0c9570 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52948/0 (socket says 172.21.15.7:52948) 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11bfff700 1 -- 172.21.15.7:0/1469684168 learned_addr learned my addr 172.21.15.7:0/1469684168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11b7fe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 0x7fc11c1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11affd700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c159ba0 0x7fc11c0c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11bfff700 1 -- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 msgr2=0x7fc11c1511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11bfff700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 0x7fc11c1511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11bfff700 1 -- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c159ba0 msgr2=0x7fc11c0c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11bfff700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c159ba0 0x7fc11c0c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11bfff700 1 -- 172.21.15.7:0/1469684168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110009420 con 0x7fc11c159f70 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.619+0000 7fc11affd700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c159ba0 0x7fc11c0c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.620+0000 7fc11bfff700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c3ca0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc114009df0 tx=0x7fc11400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.620+0000 7fc1037fe700 1 -- 172.21.15.7:0/1469684168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400cb50 con 0x7fc11c159f70 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.620+0000 7fc1037fe700 1 -- 172.21.15.7:0/1469684168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400f040 con 0x7fc11c159f70 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.620+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c0b1910 con 0x7fc11c159f70 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.620+0000 7fc1037fe700 1 -- 172.21.15.7:0/1469684168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114014480 con 0x7fc11c159f70 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.620+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c151710 con 0x7fc11c159f70 2024-01-11T07:14:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.621+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0b1db0 con 0x7fc11c159f70 2024-01-11T07:14:05.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.621+0000 7fc1037fe700 1 -- 172.21.15.7:0/1469684168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc114014830 con 0x7fc11c159f70 2024-01-11T07:14:05.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.622+0000 7fc1037fe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc108066f30 0x7fc1080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.622+0000 7fc1037fe700 1 -- 172.21.15.7:0/1469684168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc114084ee0 con 0x7fc11c159f70 2024-01-11T07:14:05.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.622+0000 7fc11b7fe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc108066f30 0x7fc1080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.623+0000 7fc11b7fe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc108066f30 0x7fc1080693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc10c007fd0 tx=0x7fc10c007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.626+0000 7fc1037fe700 1 -- 172.21.15.7:0/1469684168 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1140520b0 con 0x7fc11c159f70 2024-01-11T07:14:05.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.830+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc11c04c3c0 con 0x7fc11c159f70 2024-01-11T07:14:05.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.831+0000 7fc1037fe700 1 -- 172.21.15.7:0/1469684168 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+15330 (secure 0 0 0) 0x7fc1140556d0 con 0x7fc11c159f70 2024-01-11T07:14:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:05.831 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:14:04.678801+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:14:02.428153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-11T07:14:03.665002+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:14:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc108066f30 msgr2=0x7fc1080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc108066f30 0x7fc1080693e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc10c007fd0 tx=0x7fc10c007d30 comp rx=0 tx=0).stop 2024-01-11T07:14:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 msgr2=0x7fc11c0c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c3ca0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc114009df0 tx=0x7fc11400b040 comp rx=0 tx=0).stop 2024-01-11T07:14:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 shutdown_connections 2024-01-11T07:14:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc108066f30 0x7fc1080693e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c159f70 0x7fc11c0c3ca0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc11c0c9570 0x7fc11c1511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 --2- 172.21.15.7:0/1469684168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c159ba0 0x7fc11c0c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 >> 172.21.15.7:0/1469684168 conn(0x7fc11c06ae00 msgr2=0x7fc11c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 shutdown_connections 2024-01-11T07:14:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.833+0000 7fc121bbe700 1 -- 172.21.15.7:0/1469684168 wait complete. 2024-01-11T07:14:05.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:14:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.979+0000 7f9c14a7c700 1 Processor -- start 2024-01-11T07:14:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.979+0000 7f9c14a7c700 1 -- start start 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c14a7c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c10159c90 0x7f9c100b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c14a7c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c10069320 0x7f9c100b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c14a7c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c100ce9b0 con 0x7f9c1015a060 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c14a7c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c100cf530 con 0x7f9c10159c90 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c14a7c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c100cf9c0 con 0x7f9c10069320 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c0e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c10159c90 0x7f9c100b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c0e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c10159c90 0x7f9c100b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52960/0 (socket says 172.21.15.7:52960) 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c0e59c700 1 -- 172.21.15.7:0/3037663682 learned_addr learned my addr 172.21.15.7:0/3037663682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.980+0000 7f9c0ed9d700 1 --2- 172.21.15.7:0/3037663682 >> v2:172.21.15.113:3300/0 conn(0x7f9c10069320 0x7f9c100b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.981+0000 7f9c0e59c700 1 -- 172.21.15.7:0/3037663682 >> v2:172.21.15.113:3300/0 conn(0x7f9c10069320 msgr2=0x7f9c100b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.981+0000 7f9c0e59c700 1 --2- 172.21.15.7:0/3037663682 >> v2:172.21.15.113:3300/0 conn(0x7f9c10069320 0x7f9c100b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.981+0000 7f9c0e59c700 1 -- 172.21.15.7:0/3037663682 >> v1:172.21.15.113:6789/0 conn(0x7f9c1015a060 legacy=0x7f9c100b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.981+0000 7f9c0e59c700 1 -- 172.21.15.7:0/3037663682 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100cec00 con 0x7f9c10159c90 2024-01-11T07:14:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.981+0000 7f9c0e59c700 1 --2- 172.21.15.7:0/3037663682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10159c90 0x7f9c100b1820 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9c100cf4a0 tx=0x7f9c00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba5e87ec3ef4442 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.981+0000 7f9c0d59a700 1 -- 172.21.15.7:0/3037663682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000b860 con 0x7f9c10159c90 2024-01-11T07:14:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.981+0000 7f9c0d59a700 1 -- 172.21.15.7:0/3037663682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000b9c0 con 0x7f9c10159c90 2024-01-11T07:14:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.981+0000 7f9c0d59a700 1 -- 172.21.15.7:0/3037663682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000bd60 con 0x7f9c10159c90 2024-01-11T07:14:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.982+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3037663682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10159c90 msgr2=0x7f9c100b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.982+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3037663682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10159c90 0x7f9c100b1820 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9c100cf4a0 tx=0x7f9c00009250 comp rx=0 tx=0).stop 2024-01-11T07:14:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.982+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3037663682 shutdown_connections 2024-01-11T07:14:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.982+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3037663682 >> v2:172.21.15.113:3300/0 conn(0x7f9c10069320 0x7f9c100b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.982+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3037663682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10159c90 0x7f9c100b1820 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.982+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3037663682 >> 172.21.15.7:0/3037663682 conn(0x7f9c10071100 msgr2=0x7f9c1006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.982+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3037663682 shutdown_connections 2024-01-11T07:14:05.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.982+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3037663682 wait complete. 2024-01-11T07:14:05.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.983+0000 7f9c14a7c700 1 Processor -- start 2024-01-11T07:14:05.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.983+0000 7f9c14a7c700 1 -- start start 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.984+0000 7f9c14a7c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10069320 0x7f9c100c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.984+0000 7f9c14a7c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c10159c90 0x7f9c10155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.984+0000 7f9c14a7c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1015a060 0x7f9c100cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.984+0000 7f9c14a7c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100c91a0 con 0x7f9c10069320 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.984+0000 7f9c14a7c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100c82d0 con 0x7f9c1015a060 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.984+0000 7f9c14a7c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c100c8590 con 0x7f9c10159c90 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10069320 0x7f9c100c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10069320 0x7f9c100c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52964/0 (socket says 172.21.15.7:52964) 2024-01-11T07:14:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 -- 172.21.15.7:0/3391491828 learned_addr learned my addr 172.21.15.7:0/3391491828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0ed9d700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1015a060 0x7f9c100cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0dd9b700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c10159c90 0x7f9c10155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 -- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c10159c90 msgr2=0x7f9c10155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c10159c90 0x7f9c10155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 -- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1015a060 msgr2=0x7f9c100cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1015a060 0x7f9c100cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 -- 172.21.15.7:0/3391491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000094f0 con 0x7f9c10069320 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0ed9d700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1015a060 0x7f9c100cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c0e59c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10069320 0x7f9c100c9630 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9c1006a1d0 tx=0x7f9c00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c067fc700 1 -- 172.21.15.7:0/3391491828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00011e60 con 0x7f9c10069320 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c067fc700 1 -- 172.21.15.7:0/3391491828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0001a630 con 0x7f9c10069320 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c067fc700 1 -- 172.21.15.7:0/3391491828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0001a9f0 con 0x7f9c10069320 2024-01-11T07:14:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c100c8870 con 0x7f9c10069320 2024-01-11T07:14:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.985+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c100c8e30 con 0x7f9c10069320 2024-01-11T07:14:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.986+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c100c9480 con 0x7f9c10069320 2024-01-11T07:14:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.987+0000 7f9c067fc700 1 -- 172.21.15.7:0/3391491828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9c0002d440 con 0x7f9c10069320 2024-01-11T07:14:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.987+0000 7f9c067fc700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bf0066f60 0x7f9bf0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.987+0000 7f9c067fc700 1 -- 172.21.15.7:0/3391491828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c00086060 con 0x7f9c10069320 2024-01-11T07:14:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.988+0000 7f9c0dd9b700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bf0066f60 0x7f9bf0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.988+0000 7f9c0dd9b700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bf0066f60 0x7f9bf0069410 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9c100cc6b0 tx=0x7f9bf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:05.992+0000 7f9c067fc700 1 -- 172.21.15.7:0/3391491828 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c000b75a0 con 0x7f9c10069320 2024-01-11T07:14:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.200+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c1004c3c0 con 0x7f9c10069320 2024-01-11T07:14:06.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.201+0000 7f9c067fc700 1 -- 172.21.15.7:0/3391491828 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f9c000532b0 con 0x7f9c10069320 2024-01-11T07:14:06.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:06.201 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:14:04.678801+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:14:02.428153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-11T07:14:03.665002+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:14:06.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bf0066f60 msgr2=0x7f9bf0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:06.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bf0066f60 0x7f9bf0069410 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9c100cc6b0 tx=0x7f9bf8009040 comp rx=0 tx=0).stop 2024-01-11T07:14:06.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10069320 msgr2=0x7f9c100c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:06.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10069320 0x7f9c100c9630 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9c1006a1d0 tx=0x7f9c00017000 comp rx=0 tx=0).stop 2024-01-11T07:14:06.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 shutdown_connections 2024-01-11T07:14:06.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9bf0066f60 0x7f9bf0069410 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c1015a060 0x7f9c100cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c10069320 0x7f9c100c9630 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 --2- 172.21.15.7:0/3391491828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c10159c90 0x7f9c10155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 >> 172.21.15.7:0/3391491828 conn(0x7f9c10071100 msgr2=0x7f9c100b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:06.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 shutdown_connections 2024-01-11T07:14:06.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.204+0000 7f9c14a7c700 1 -- 172.21.15.7:0/3391491828 wait complete. 2024-01-11T07:14:06.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:14:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.353+0000 7fccc0032700 1 Processor -- start 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.354+0000 7fccc0032700 1 -- start start 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.354+0000 7fccc0032700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb8159e50 0x7fccb80cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.354+0000 7fccc0032700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fccb815d8e0 0x7fccb815dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.354+0000 7fccc0032700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb80cdb90 con 0x7fccb8159a80 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.354+0000 7fccc0032700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb80cf3a0 con 0x7fccb815d8e0 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.354+0000 7fccc0032700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb815e260 con 0x7fccb8159e50 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbd5cd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb8159e50 0x7fccb80cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbe5cf700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fccb815d8e0 0x7fccb815dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbd5cd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb8159e50 0x7fccb80cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:58034/0 (socket says 172.21.15.7:58034) 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbd5cd700 1 -- 172.21.15.7:0/943429773 learned_addr learned my addr 172.21.15.7:0/943429773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbd5cd700 1 -- 172.21.15.7:0/943429773 >> v2:172.21.15.113:3300/0 conn(0x7fccb815d8e0 msgr2=0x7fccb815dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbd5cd700 1 --2- 172.21.15.7:0/943429773 >> v2:172.21.15.113:3300/0 conn(0x7fccb815d8e0 0x7fccb815dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbd5cd700 1 -- 172.21.15.7:0/943429773 >> v1:172.21.15.113:6789/0 conn(0x7fccb8159a80 legacy=0x7fccb806cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbd5cd700 1 -- 172.21.15.7:0/943429773 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb80cddc0 con 0x7fccb8159e50 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.355+0000 7fccbd5cd700 1 --2- 172.21.15.7:0/943429773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb8159e50 0x7fccb80cee60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fccb40099b0 tx=0x7fccb4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19464b5146dcd5a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccbcdcc700 1 -- 172.21.15.7:0/943429773 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4009ea0 con 0x7fccb8159e50 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccbcdcc700 1 -- 172.21.15.7:0/943429773 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb400b860 con 0x7fccb8159e50 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccbcdcc700 1 -- 172.21.15.7:0/943429773 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb400bc00 con 0x7fccb8159e50 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccc0032700 1 -- 172.21.15.7:0/943429773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb8159e50 msgr2=0x7fccb80cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccc0032700 1 --2- 172.21.15.7:0/943429773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb8159e50 0x7fccb80cee60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fccb40099b0 tx=0x7fccb4009250 comp rx=0 tx=0).stop 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccc0032700 1 -- 172.21.15.7:0/943429773 shutdown_connections 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccc0032700 1 --2- 172.21.15.7:0/943429773 >> v2:172.21.15.113:3300/0 conn(0x7fccb815d8e0 0x7fccb815dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccc0032700 1 --2- 172.21.15.7:0/943429773 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb8159e50 0x7fccb80cee60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccc0032700 1 -- 172.21.15.7:0/943429773 >> 172.21.15.7:0/943429773 conn(0x7fccb806ade0 msgr2=0x7fccb806b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccc0032700 1 -- 172.21.15.7:0/943429773 shutdown_connections 2024-01-11T07:14:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.356+0000 7fccc0032700 1 -- 172.21.15.7:0/943429773 wait complete. 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.358+0000 7fccc0032700 1 Processor -- start 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.358+0000 7fccc0032700 1 -- start start 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.358+0000 7fccc0032700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb8159a80 0x7fccb814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.358+0000 7fccc0032700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb8159e50 0x7fccb814b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.358+0000 7fccc0032700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb815d8e0 0x7fccb8153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.358+0000 7fccc0032700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb814bed0 con 0x7fccb8159a80 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.358+0000 7fccc0032700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb814c190 con 0x7fccb8159e50 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.358+0000 7fccc0032700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80b4860 con 0x7fccb815d8e0 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb8159a80 0x7fccb814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb8159a80 0x7fccb814f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52978/0 (socket says 172.21.15.7:52978) 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 -- 172.21.15.7:0/1756696284 learned_addr learned my addr 172.21.15.7:0/1756696284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbd5cd700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb8159e50 0x7fccb814b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbe5cf700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb815d8e0 0x7fccb8153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 -- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb815d8e0 msgr2=0x7fccb8153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb815d8e0 0x7fccb8153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 -- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb8159e50 msgr2=0x7fccb814b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb8159e50 0x7fccb814b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 -- 172.21.15.7:0/1756696284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb40094f0 con 0x7fccb8159a80 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccbddce700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb8159a80 0x7fccb814f2c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fccac00c5c0 tx=0x7fccac00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fcca9ffb700 1 -- 172.21.15.7:0/1756696284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac011590 con 0x7fccb8159a80 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fcca9ffb700 1 -- 172.21.15.7:0/1756696284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccac011710 con 0x7fccb8159a80 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb814f9c0 con 0x7fccb8159a80 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.359+0000 7fcca9ffb700 1 -- 172.21.15.7:0/1756696284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac011ad0 con 0x7fccb8159a80 2024-01-11T07:14:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.360+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb814ff80 con 0x7fccb8159a80 2024-01-11T07:14:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.361+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb81502c0 con 0x7fccb8159a80 2024-01-11T07:14:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.361+0000 7fcca9ffb700 1 -- 172.21.15.7:0/1756696284 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fccac01b3b0 con 0x7fccb8159a80 2024-01-11T07:14:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.361+0000 7fcca9ffb700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcc9c066f30 0x7fcc9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.361+0000 7fcca9ffb700 1 -- 172.21.15.7:0/1756696284 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fccac083b40 con 0x7fccb8159a80 2024-01-11T07:14:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.362+0000 7fccbd5cd700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcc9c066f30 0x7fcc9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.362+0000 7fccbd5cd700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcc9c066f30 0x7fcc9c0693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fccb40099b0 tx=0x7fccb4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.366+0000 7fcca9ffb700 1 -- 172.21.15.7:0/1756696284 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccac050db0 con 0x7fccb8159a80 2024-01-11T07:14:06.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:06.623+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fccb804c3c0 con 0x7fccb8159a80 2024-01-11T07:14:07.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.473+0000 7fcca9ffb700 1 -- 172.21.15.7:0/1756696284 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) v1 ==== 118+0+0 (secure 0 0 0) 0x7fccac017070 con 0x7fccb8159a80 2024-01-11T07:14:07.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcc9c066f30 msgr2=0x7fcc9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:07.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcc9c066f30 0x7fcc9c0693e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fccb40099b0 tx=0x7fccb4017000 comp rx=0 tx=0).stop 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb8159a80 msgr2=0x7fccb814f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb8159a80 0x7fccb814f2c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fccac00c5c0 tx=0x7fccac00c980 comp rx=0 tx=0).stop 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 shutdown_connections 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcc9c066f30 0x7fcc9c0693e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fccb815d8e0 0x7fccb8153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccb8159a80 0x7fccb814f2c0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 --2- 172.21.15.7:0/1756696284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccb8159e50 0x7fccb814b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 >> 172.21.15.7:0/1756696284 conn(0x7fccb806ade0 msgr2=0x7fccb8155330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 shutdown_connections 2024-01-11T07:14:07.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.476+0000 7fccc0032700 1 -- 172.21.15.7:0/1756696284 wait complete. 2024-01-11T07:14:07.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:14:07.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.631+0000 7faa28226700 1 Processor -- start 2024-01-11T07:14:07.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.632+0000 7faa28226700 1 -- start start 2024-01-11T07:14:07.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa28226700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faa2006e350 0x7faa200c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:07.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa28226700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2006a940 con 0x7faa200c85f0 2024-01-11T07:14:07.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa28226700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa200c8dc0 con 0x7faa20156a80 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa28226700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa200c90a0 con 0x7faa2006e350 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa267c3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faa200c85f0 0x7faa2006adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37032/0 (socket says 172.21.15.7:37032) 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa267c3700 1 -- 172.21.15.7:0/2572561167 learned_addr learned my addr 172.21.15.7:0/2572561167 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa257c1700 1 --2- 172.21.15.7:0/2572561167 >> v2:172.21.15.113:3300/0 conn(0x7faa2006e350 0x7faa200c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa24fc0700 1 -- 172.21.15.7:0/2572561167 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806757731 0 0) 0x7faa2006a940 con 0x7faa200c85f0 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa24fc0700 1 -- 172.21.15.7:0/2572561167 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa0c003570 con 0x7faa200c85f0 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa24fc0700 1 -- 172.21.15.7:0/2572561167 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402481521 0 0) 0x7faa200c8dc0 con 0x7faa20156a80 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa24fc0700 1 -- 172.21.15.7:0/2572561167 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa2006a940 con 0x7faa20156a80 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa24fc0700 1 -- 172.21.15.7:0/2572561167 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 962631171 0 0) 0x7faa0c003570 con 0x7faa200c85f0 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa257c1700 1 -- 172.21.15.7:0/2572561167 >> v1:172.21.15.113:6789/0 conn(0x7faa20156a80 legacy=0x7faa2006dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa257c1700 1 -- 172.21.15.7:0/2572561167 >> v1:172.21.15.7:6789/0 conn(0x7faa200c85f0 legacy=0x7faa2006adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.633+0000 7faa257c1700 1 -- 172.21.15.7:0/2572561167 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa200c9380 con 0x7faa2006e350 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.634+0000 7faa257c1700 1 --2- 172.21.15.7:0/2572561167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2006e350 0x7faa200c80b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faa140098e0 tx=0x7faa140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf03e177b38265d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.634+0000 7faa24fc0700 1 -- 172.21.15.7:0/2572561167 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180017f0 con 0x7faa2006e350 2024-01-11T07:14:07.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.634+0000 7faa24fc0700 1 -- 172.21.15.7:0/2572561167 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14011420 con 0x7faa2006e350 2024-01-11T07:14:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.634+0000 7faa24fc0700 1 -- 172.21.15.7:0/2572561167 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140117c0 con 0x7faa2006e350 2024-01-11T07:14:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.634+0000 7faa28226700 1 -- 172.21.15.7:0/2572561167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2006e350 msgr2=0x7faa200c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.634+0000 7faa28226700 1 --2- 172.21.15.7:0/2572561167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2006e350 0x7faa200c80b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faa140098e0 tx=0x7faa140091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.634+0000 7faa28226700 1 -- 172.21.15.7:0/2572561167 shutdown_connections 2024-01-11T07:14:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.634+0000 7faa28226700 1 --2- 172.21.15.7:0/2572561167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa2006e350 0x7faa200c80b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.635+0000 7faa28226700 1 -- 172.21.15.7:0/2572561167 >> 172.21.15.7:0/2572561167 conn(0x7faa200cd860 msgr2=0x7faa200cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.635+0000 7faa28226700 1 -- 172.21.15.7:0/2572561167 shutdown_connections 2024-01-11T07:14:07.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.635+0000 7faa28226700 1 -- 172.21.15.7:0/2572561167 wait complete. 2024-01-11T07:14:07.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.636+0000 7faa28226700 1 Processor -- start 2024-01-11T07:14:07.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.636+0000 7faa28226700 1 -- start start 2024-01-11T07:14:07.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa28226700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2006e350 0x7faa20145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:07.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa28226700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa200c85f0 0x7faa201684c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa28226700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa20156a80 0x7faa200ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa28226700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200acef0 con 0x7faa2006e350 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa28226700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa20145840 con 0x7faa20156a80 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa28226700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa20168e10 con 0x7faa200c85f0 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2006e350 0x7faa20145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2006e350 0x7faa20145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56480/0 (socket says 172.21.15.7:56480) 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 -- 172.21.15.7:0/4131401524 learned_addr learned my addr 172.21.15.7:0/4131401524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa257c1700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa200c85f0 0x7faa201684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa267c3700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa20156a80 0x7faa200ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 -- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa200c85f0 msgr2=0x7faa201684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa200c85f0 0x7faa201684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 -- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa20156a80 msgr2=0x7faa200ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:07.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa20156a80 0x7faa200ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:07.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 -- 172.21.15.7:0/4131401524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14009420 con 0x7faa2006e350 2024-01-11T07:14:07.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.637+0000 7faa25fc2700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2006e350 0x7faa20145300 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c790 tx=0x7faa1c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:07.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.638+0000 7faa11ffb700 1 -- 172.21.15.7:0/4131401524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c010510 con 0x7faa2006e350 2024-01-11T07:14:07.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.638+0000 7faa11ffb700 1 -- 172.21.15.7:0/4131401524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c010670 con 0x7faa2006e350 2024-01-11T07:14:07.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.638+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa201690f0 con 0x7faa2006e350 2024-01-11T07:14:07.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.638+0000 7faa11ffb700 1 -- 172.21.15.7:0/4131401524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c010a10 con 0x7faa2006e350 2024-01-11T07:14:07.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.638+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa200acb80 con 0x7faa2006e350 2024-01-11T07:14:07.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.639+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2004c3c0 con 0x7faa2006e350 2024-01-11T07:14:07.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.639+0000 7faa11ffb700 1 -- 172.21.15.7:0/4131401524 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faa1c01b3b0 con 0x7faa2006e350 2024-01-11T07:14:07.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.640+0000 7faa11ffb700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa04066f30 0x7faa040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:07.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.640+0000 7faa11ffb700 1 -- 172.21.15.7:0/4131401524 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa1c083f00 con 0x7faa2006e350 2024-01-11T07:14:07.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.640+0000 7faa257c1700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa04066f30 0x7faa040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:07.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.640+0000 7faa257c1700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa04066f30 0x7faa040693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7faa140093f0 tx=0x7faa14016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:07.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.644+0000 7faa11ffb700 1 -- 172.21.15.7:0/4131401524 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa1c0510d0 con 0x7faa2006e350 2024-01-11T07:14:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:07.898+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faa2006ee40 con 0x7faa2006e350 2024-01-11T07:14:08.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.479+0000 7faa11ffb700 1 -- 172.21.15.7:0/4131401524 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa1c017020 con 0x7faa2006e350 2024-01-11T07:14:08.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa04066f30 msgr2=0x7faa040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa04066f30 0x7faa040693e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7faa140093f0 tx=0x7faa14016000 comp rx=0 tx=0).stop 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2006e350 msgr2=0x7faa20145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2006e350 0x7faa20145300 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c790 tx=0x7faa1c007e10 comp rx=0 tx=0).stop 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 shutdown_connections 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faa04066f30 0x7faa040693e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa20156a80 0x7faa200ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa2006e350 0x7faa20145300 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 --2- 172.21.15.7:0/4131401524 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa200c85f0 0x7faa201684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 >> 172.21.15.7:0/4131401524 conn(0x7faa200cd860 msgr2=0x7faa2006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 shutdown_connections 2024-01-11T07:14:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.482+0000 7faa28226700 1 -- 172.21.15.7:0/4131401524 wait complete. 2024-01-11T07:14:08.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:14:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.632+0000 7f5e7c524700 1 Processor -- start 2024-01-11T07:14:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.633+0000 7f5e7c524700 1 -- start start 2024-01-11T07:14:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e7c524700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406e0d0 0x7f5e74159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e7c524700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7415b770 con 0x7f5e7406dd00 2024-01-11T07:14:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e7c524700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7415abf0 con 0x7f5e74159ff0 2024-01-11T07:14:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e7c524700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7415aed0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e7a2c0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5e7406dd00 0x7f5e740c98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37040/0 (socket says 172.21.15.7:37040) 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e7a2c0700 1 -- 172.21.15.7:0/1793185905 learned_addr learned my addr 172.21.15.7:0/1793185905 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e79abf700 1 --2- 172.21.15.7:0/1793185905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406e0d0 0x7f5e74159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e792be700 1 -- 172.21.15.7:0/1793185905 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945187520 0 0) 0x7f5e7415b770 con 0x7f5e7406dd00 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e792be700 1 -- 172.21.15.7:0/1793185905 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e60003570 con 0x7f5e7406dd00 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e792be700 1 -- 172.21.15.7:0/1793185905 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352070853 0 0) 0x7f5e7415abf0 con 0x7f5e74159ff0 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e792be700 1 -- 172.21.15.7:0/1793185905 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e7415b770 con 0x7f5e74159ff0 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e79abf700 1 -- 172.21.15.7:0/1793185905 >> v1:172.21.15.113:6789/0 conn(0x7f5e74159ff0 legacy=0x7f5e74069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e79abf700 1 -- 172.21.15.7:0/1793185905 >> v1:172.21.15.7:6789/0 conn(0x7f5e7406dd00 legacy=0x7f5e740c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e792be700 1 -- 172.21.15.7:0/1793185905 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 83723521 0 0) 0x7f5e60003570 con 0x7f5e7406dd00 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.634+0000 7f5e79abf700 1 -- 172.21.15.7:0/1793185905 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7415b1b0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.635+0000 7f5e79abf700 1 --2- 172.21.15.7:0/1793185905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406e0d0 0x7f5e74159ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009910 tx=0x7f5e640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=534af6ce6ff85f3a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.635+0000 7f5e792be700 1 -- 172.21.15.7:0/1793185905 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70001470 con 0x7f5e7406e0d0 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.635+0000 7f5e792be700 1 -- 172.21.15.7:0/1793185905 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e64011420 con 0x7f5e7406e0d0 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.635+0000 7f5e792be700 1 -- 172.21.15.7:0/1793185905 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e640117c0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.635+0000 7f5e7c524700 1 -- 172.21.15.7:0/1793185905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406e0d0 msgr2=0x7f5e74159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.635+0000 7f5e7c524700 1 --2- 172.21.15.7:0/1793185905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406e0d0 0x7f5e74159ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009910 tx=0x7f5e640091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.636+0000 7f5e7c524700 1 -- 172.21.15.7:0/1793185905 shutdown_connections 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.636+0000 7f5e7c524700 1 --2- 172.21.15.7:0/1793185905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406e0d0 0x7f5e74159ab0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.636+0000 7f5e7c524700 1 -- 172.21.15.7:0/1793185905 >> 172.21.15.7:0/1793185905 conn(0x7f5e740680e0 msgr2=0x7f5e740cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.636+0000 7f5e7c524700 1 -- 172.21.15.7:0/1793185905 shutdown_connections 2024-01-11T07:14:08.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.636+0000 7f5e7c524700 1 -- 172.21.15.7:0/1793185905 wait complete. 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.637+0000 7f5e7c524700 1 Processor -- start 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.637+0000 7f5e7c524700 1 -- start start 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.637+0000 7f5e7c524700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406dd00 0x7f5e74167a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7c524700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e7406e0d0 0x7f5e74166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7c524700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e74159ff0 0x7f5e740b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7c524700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740c48e0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7c524700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740c5460 con 0x7f5e74159ff0 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7c524700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740c4b30 con 0x7f5e7406dd00 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e7406e0d0 0x7f5e74166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e7406e0d0 0x7f5e74166fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56484/0 (socket says 172.21.15.7:56484) 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 -- 172.21.15.7:0/3104441335 learned_addr learned my addr 172.21.15.7:0/3104441335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7a2c0700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406dd00 0x7f5e74167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7aac1700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e74159ff0 0x7f5e740b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 -- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406dd00 msgr2=0x7f5e74167a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406dd00 0x7f5e74167a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 -- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e74159ff0 msgr2=0x7f5e740b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e74159ff0 0x7f5e740b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 -- 172.21.15.7:0/3104441335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64009420 con 0x7f5e7406e0d0 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7aac1700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e74159ff0 0x7f5e740b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e79abf700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e7406e0d0 0x7f5e74166fe0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009860 tx=0x7f5e64016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e6a7fc700 1 -- 172.21.15.7:0/3104441335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64021070 con 0x7f5e7406e0d0 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e6a7fc700 1 -- 172.21.15.7:0/3104441335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6401c320 con 0x7f5e7406e0d0 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e740c4e10 con 0x7f5e7406e0d0 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e6a7fc700 1 -- 172.21.15.7:0/3104441335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6401cdb0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.638+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e74167520 con 0x7f5e7406e0d0 2024-01-11T07:14:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.639+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e740c52b0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.640+0000 7f5e6a7fc700 1 -- 172.21.15.7:0/3104441335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5e6401a770 con 0x7f5e7406e0d0 2024-01-11T07:14:08.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.640+0000 7f5e6a7fc700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e58066f30 0x7f5e580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:08.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.640+0000 7f5e6a7fc700 1 -- 172.21.15.7:0/3104441335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e64087130 con 0x7f5e7406e0d0 2024-01-11T07:14:08.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.641+0000 7f5e7a2c0700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e58066f30 0x7f5e580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:08.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.641+0000 7f5e7a2c0700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e58066f30 0x7f5e580693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5e70001ed0 tx=0x7f5e70000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:08.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.645+0000 7f5e6a7fc700 1 -- 172.21.15.7:0/3104441335 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e640543a0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.892+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e740c55d0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.893+0000 7f5e6a7fc700 1 -- 172.21.15.7:0/3104441335 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f5e640579c0 con 0x7f5e7406e0d0 2024-01-11T07:14:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:08.892 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1458931008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1458931008},{"type":"v1","addr":"172.21.15.113:6836","nonce":1458931008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3377988669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3377988669},{"type":"v1","addr":"172.21.15.113:6837","nonce":3377988669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1420951490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1420951490},{"type":"v1","addr":"172.21.15.7:6833","nonce":1420951490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:14:04.679133+0000","modified":"2024-01-11T07:14:08.473548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7425":{"gid":7425,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/187487340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":187487340},{"type":"v1","addr":"172.21.15.7:6835","nonce":187487340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-11T07:14:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e58066f30 msgr2=0x7f5e580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e58066f30 0x7f5e580693e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5e70001ed0 tx=0x7f5e70000bc0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e7406e0d0 msgr2=0x7f5e74166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e7406e0d0 0x7f5e74166fe0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009860 tx=0x7f5e64016000 comp rx=0 tx=0).stop 2024-01-11T07:14:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 shutdown_connections 2024-01-11T07:14:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5e58066f30 0x7f5e580693e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e74159ff0 0x7f5e740b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e7406dd00 0x7f5e74167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 --2- 172.21.15.7:0/3104441335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e7406e0d0 0x7f5e74166fe0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 >> 172.21.15.7:0/3104441335 conn(0x7f5e740680e0 msgr2=0x7f5e740c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 shutdown_connections 2024-01-11T07:14:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:08.895+0000 7f5e7c524700 1 -- 172.21.15.7:0/3104441335 wait complete. 2024-01-11T07:14:08.896 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2024-01-11T07:14:08.908 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:14:09.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.041+0000 7fba2235b700 1 Processor -- start 2024-01-11T07:14:09.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.041+0000 7fba2235b700 1 -- start start 2024-01-11T07:14:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.041+0000 7fba2235b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba1c15b850 0x7fba1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.041+0000 7fba2235b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c15bc20 0x7fba1c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.041+0000 7fba2235b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c0c87a0 con 0x7fba1c06e1b0 2024-01-11T07:14:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.041+0000 7fba2235b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c0c9cb0 con 0x7fba1c15b850 2024-01-11T07:14:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.041+0000 7fba2235b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c069280 con 0x7fba1c15bc20 2024-01-11T07:14:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba1c15b850 0x7fba1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba1c15b850 0x7fba1c068bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56492/0 (socket says 172.21.15.7:56492) 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1bfff700 1 -- 172.21.15.7:0/1945462180 learned_addr learned my addr 172.21.15.7:0/1945462180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1b7fe700 1 --2- 172.21.15.7:0/1945462180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c15bc20 0x7fba1c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1bfff700 1 -- 172.21.15.7:0/1945462180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c15bc20 msgr2=0x7fba1c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1bfff700 1 --2- 172.21.15.7:0/1945462180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c15bc20 0x7fba1c15c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1bfff700 1 -- 172.21.15.7:0/1945462180 >> v1:172.21.15.113:6789/0 conn(0x7fba1c06e1b0 legacy=0x7fba1c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1bfff700 1 -- 172.21.15.7:0/1945462180 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c069560 con 0x7fba1c15b850 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1b7fe700 1 --2- 172.21.15.7:0/1945462180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c15bc20 0x7fba1c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1bfff700 1 --2- 172.21.15.7:0/1945462180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c068bf0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0099b0 tx=0x7fba0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2609c5f7ff5dcdc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.042+0000 7fba1affd700 1 -- 172.21.15.7:0/1945462180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00b860 con 0x7fba1c15b850 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba1affd700 1 -- 172.21.15.7:0/1945462180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c00b9c0 con 0x7fba1c15b850 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba1affd700 1 -- 172.21.15.7:0/1945462180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00bd60 con 0x7fba1c15b850 2024-01-11T07:14:09.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba2235b700 1 -- 172.21.15.7:0/1945462180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 msgr2=0x7fba1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba2235b700 1 --2- 172.21.15.7:0/1945462180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c068bf0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0099b0 tx=0x7fba0c009250 comp rx=0 tx=0).stop 2024-01-11T07:14:09.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba2235b700 1 -- 172.21.15.7:0/1945462180 shutdown_connections 2024-01-11T07:14:09.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba2235b700 1 --2- 172.21.15.7:0/1945462180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c068bf0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba2235b700 1 --2- 172.21.15.7:0/1945462180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c15bc20 0x7fba1c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba2235b700 1 -- 172.21.15.7:0/1945462180 >> 172.21.15.7:0/1945462180 conn(0x7fba1c063230 msgr2=0x7fba1c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:09.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba2235b700 1 -- 172.21.15.7:0/1945462180 shutdown_connections 2024-01-11T07:14:09.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.043+0000 7fba2235b700 1 -- 172.21.15.7:0/1945462180 wait complete. 2024-01-11T07:14:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.044+0000 7fba2235b700 1 Processor -- start 2024-01-11T07:14:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.045+0000 7fba2235b700 1 -- start start 2024-01-11T07:14:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.045+0000 7fba2235b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c06e1b0 0x7fba1c0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.045+0000 7fba2235b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.045+0000 7fba2235b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba1c15bc20 0x7fba1c0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.045+0000 7fba2235b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c169f30 con 0x7fba1c15b850 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.045+0000 7fba2235b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c15dba0 con 0x7fba1c15bc20 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba2235b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c15de80 con 0x7fba1c06e1b0 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56504/0 (socket says 172.21.15.7:56504) 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 -- 172.21.15.7:0/6596844 learned_addr learned my addr 172.21.15.7:0/6596844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1bfff700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c06e1b0 0x7fba1c0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba208f8700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba1c15bc20 0x7fba1c0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 -- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c06e1b0 msgr2=0x7fba1c0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c06e1b0 0x7fba1c0ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 -- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba1c15bc20 msgr2=0x7fba1c0b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba1c15bc20 0x7fba1c0b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 -- 172.21.15.7:0/6596844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c0094f0 con 0x7fba1c15b850 2024-01-11T07:14:09.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba208f8700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba1c15bc20 0x7fba1c0b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:09.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba1b7fe700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c154f20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fba0400d860 tx=0x7fba04008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba13fff700 1 -- 172.21.15.7:0/6596844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba040125f0 con 0x7fba1c15b850 2024-01-11T07:14:09.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba13fff700 1 -- 172.21.15.7:0/6596844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04012750 con 0x7fba1c15b850 2024-01-11T07:14:09.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c15e160 con 0x7fba1c15b850 2024-01-11T07:14:09.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba13fff700 1 -- 172.21.15.7:0/6596844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04012af0 con 0x7fba1c15b850 2024-01-11T07:14:09.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.046+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c15e720 con 0x7fba1c15b850 2024-01-11T07:14:09.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.047+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c04c3c0 con 0x7fba1c15b850 2024-01-11T07:14:09.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.048+0000 7fba13fff700 1 -- 172.21.15.7:0/6596844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fba04014430 con 0x7fba1c15b850 2024-01-11T07:14:09.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.048+0000 7fba13fff700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9fc066f30 0x7fb9fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.048+0000 7fba13fff700 1 -- 172.21.15.7:0/6596844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba040848d0 con 0x7fba1c15b850 2024-01-11T07:14:09.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.049+0000 7fba1bfff700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9fc066f30 0x7fb9fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.049+0000 7fba1bfff700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9fc066f30 0x7fb9fc0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0099b0 tx=0x7fba0c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.053+0000 7fba13fff700 1 -- 172.21.15.7:0/6596844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba04051b40 con 0x7fba1c15b850 2024-01-11T07:14:09.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.258+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba1c15e940 con 0x7fba1c15b850 2024-01-11T07:14:09.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.259+0000 7fba13fff700 1 -- 172.21.15.7:0/6596844 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+15317 (secure 0 0 0) 0x7fba04055160 con 0x7fba1c15b850 2024-01-11T07:14:09.259 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:09.259 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:14:06.443060+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:14:02.428153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-11T07:14:03.665002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:14:09.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9fc066f30 msgr2=0x7fb9fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9fc066f30 0x7fb9fc0693e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0099b0 tx=0x7fba0c017000 comp rx=0 tx=0).stop 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 msgr2=0x7fba1c154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c154f20 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fba0400d860 tx=0x7fba04008040 comp rx=0 tx=0).stop 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 shutdown_connections 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb9fc066f30 0x7fb9fc0693e0 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0099b0 tx=0x7fba0c017000 comp rx=0 tx=0).stop 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba1c15bc20 0x7fba1c0b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba1c06e1b0 0x7fba1c0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 --2- 172.21.15.7:0/6596844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba1c15b850 0x7fba1c154f20 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.261+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 >> 172.21.15.7:0/6596844 conn(0x7fba1c063230 msgr2=0x7fba1c0c1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.262+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 shutdown_connections 2024-01-11T07:14:09.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.262+0000 7fba2235b700 1 -- 172.21.15.7:0/6596844 wait complete. 2024-01-11T07:14:09.275 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:14:09.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:14:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.408+0000 7fdd8701c700 1 Processor -- start 2024-01-11T07:14:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.408+0000 7fdd8701c700 1 -- start start 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.409+0000 7fdd8701c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800cafb0 0x7fdd800ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.409+0000 7fdd8701c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd80159300 con 0x7fdd80158280 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.409+0000 7fdd8701c700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800cf260 con 0x7fdd800b0b40 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.409+0000 7fdd8701c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800cf540 con 0x7fdd800cafb0 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.409+0000 7fdd7ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdd80158280 0x7fdd800b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37050/0 (socket says 172.21.15.7:37050) 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.409+0000 7fdd7ffff700 1 -- 172.21.15.7:0/1828336600 learned_addr learned my addr 172.21.15.7:0/1828336600 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.409+0000 7fdd84db8700 1 --2- 172.21.15.7:0/1828336600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800cafb0 0x7fdd800ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd7f7fe700 1 -- 172.21.15.7:0/1828336600 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984326329 0 0) 0x7fdd80159300 con 0x7fdd80158280 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd7f7fe700 1 -- 172.21.15.7:0/1828336600 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd70003570 con 0x7fdd80158280 2024-01-11T07:14:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd7f7fe700 1 -- 172.21.15.7:0/1828336600 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374535313 0 0) 0x7fdd800cf260 con 0x7fdd800b0b40 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd7f7fe700 1 -- 172.21.15.7:0/1828336600 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd80159300 con 0x7fdd800b0b40 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd84db8700 1 -- 172.21.15.7:0/1828336600 >> v1:172.21.15.113:6789/0 conn(0x7fdd800b0b40 legacy=0x7fdd80161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd7f7fe700 1 -- 172.21.15.7:0/1828336600 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1880321475 0 0) 0x7fdd70003570 con 0x7fdd80158280 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd84db8700 1 -- 172.21.15.7:0/1828336600 >> v1:172.21.15.7:6789/0 conn(0x7fdd80158280 legacy=0x7fdd800b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd84db8700 1 -- 172.21.15.7:0/1828336600 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800cf820 con 0x7fdd800cafb0 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd84db8700 1 --2- 172.21.15.7:0/1828336600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800cafb0 0x7fdd800ce8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009910 tx=0x7fdd740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee32f9367046b3b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd7f7fe700 1 -- 172.21.15.7:0/1828336600 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c001470 con 0x7fdd800cafb0 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd7f7fe700 1 -- 172.21.15.7:0/1828336600 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd74011420 con 0x7fdd800cafb0 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.410+0000 7fdd7f7fe700 1 -- 172.21.15.7:0/1828336600 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740117c0 con 0x7fdd800cafb0 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.411+0000 7fdd8701c700 1 -- 172.21.15.7:0/1828336600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800cafb0 msgr2=0x7fdd800ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.411+0000 7fdd8701c700 1 --2- 172.21.15.7:0/1828336600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800cafb0 0x7fdd800ce8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009910 tx=0x7fdd740091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.411+0000 7fdd8701c700 1 -- 172.21.15.7:0/1828336600 shutdown_connections 2024-01-11T07:14:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.411+0000 7fdd8701c700 1 --2- 172.21.15.7:0/1828336600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800cafb0 0x7fdd800ce8f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.411+0000 7fdd8701c700 1 -- 172.21.15.7:0/1828336600 >> 172.21.15.7:0/1828336600 conn(0x7fdd80071100 msgr2=0x7fdd8006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.411+0000 7fdd8701c700 1 -- 172.21.15.7:0/1828336600 shutdown_connections 2024-01-11T07:14:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.411+0000 7fdd8701c700 1 -- 172.21.15.7:0/1828336600 wait complete. 2024-01-11T07:14:09.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.412+0000 7fdd8701c700 1 Processor -- start 2024-01-11T07:14:09.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.412+0000 7fdd8701c700 1 -- start start 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd8701c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800b0b40 0x7fdd80148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd8701c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800cafb0 0x7fdd800af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd8701c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80158280 0x7fdd800af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd8701c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801678f0 con 0x7fdd80158280 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd8701c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800c4ed0 con 0x7fdd800cafb0 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd8701c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800c51b0 con 0x7fdd800b0b40 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd855b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80158280 0x7fdd800af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd84db8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800b0b40 0x7fdd80148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd855b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80158280 0x7fdd800af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56520/0 (socket says 172.21.15.7:56520) 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd855b9700 1 -- 172.21.15.7:0/1107208607 learned_addr learned my addr 172.21.15.7:0/1107208607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.413+0000 7fdd7ffff700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800cafb0 0x7fdd800af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd855b9700 1 -- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800b0b40 msgr2=0x7fdd80148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd855b9700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800b0b40 0x7fdd80148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd855b9700 1 -- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800cafb0 msgr2=0x7fdd800af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd855b9700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800cafb0 0x7fdd800af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd855b9700 1 -- 172.21.15.7:0/1107208607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd74009420 con 0x7fdd80158280 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd84db8700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800b0b40 0x7fdd80148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd855b9700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80158280 0x7fdd800af8b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c910 tx=0x7fdd78007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd7cff9700 1 -- 172.21.15.7:0/1107208607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78012630 con 0x7fdd80158280 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd7cff9700 1 -- 172.21.15.7:0/1107208607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd780127b0 con 0x7fdd80158280 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd800c5490 con 0x7fdd80158280 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd7cff9700 1 -- 172.21.15.7:0/1107208607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78012b70 con 0x7fdd80158280 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.414+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd801489d0 con 0x7fdd80158280 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.415+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd80167b10 con 0x7fdd80158280 2024-01-11T07:14:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.416+0000 7fdd7cff9700 1 -- 172.21.15.7:0/1107208607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdd7801b400 con 0x7fdd80158280 2024-01-11T07:14:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.416+0000 7fdd7cff9700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.416+0000 7fdd7cff9700 1 -- 172.21.15.7:0/1107208607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd78084850 con 0x7fdd80158280 2024-01-11T07:14:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.416+0000 7fdd84db8700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.417+0000 7fdd84db8700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd64066f30 0x7fdd640693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd740030c0 tx=0x7fdd74016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.421+0000 7fdd7cff9700 1 -- 172.21.15.7:0/1107208607 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd80167b10 con 0x7fdd80158280 2024-01-11T07:14:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.621+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd80167b10 con 0x7fdd64066f30 2024-01-11T07:14:09.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.630+0000 7fdd7cff9700 1 -- 172.21.15.7:0/1107208607 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276330 (secure 0 0 0) 0x7fdd80167b10 con 0x7fdd64066f30 2024-01-11T07:14:09.633 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd64066f30 msgr2=0x7fdd640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd64066f30 0x7fdd640693e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd740030c0 tx=0x7fdd74016000 comp rx=0 tx=0).stop 2024-01-11T07:14:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80158280 msgr2=0x7fdd800af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80158280 0x7fdd800af8b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c910 tx=0x7fdd78007f90 comp rx=0 tx=0).stop 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 shutdown_connections 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80158280 0x7fdd800af8b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd800b0b40 0x7fdd80148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 --2- 172.21.15.7:0/1107208607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800cafb0 0x7fdd800af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 >> 172.21.15.7:0/1107208607 conn(0x7fdd80071100 msgr2=0x7fdd80155310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 shutdown_connections 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.633+0000 7fdd8701c700 1 -- 172.21.15.7:0/1107208607 wait complete. 2024-01-11T07:14:09.637 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:14:09.646 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-01-11T07:14:07.940803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7585,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554914,"statfs":{"total":1600000000,"available":1592232237,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.695981"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.249345+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:23:18.764571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.247448+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:44:36.489439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.249042+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:09:08.648826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.247639+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:23:38.301919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248460+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:56:40.994670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.246028+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:18:39.179579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.246698+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:08.983522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.246992+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:56:40.378672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.246696+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:40:05.014059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.246154+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:36:01.427098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248240+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:56:40.378672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.247295+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:23:38.301919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.245503+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:01:30.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248099+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:44:07.700264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.246750+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:15:00.686776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.247012+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:42:08.465499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248575+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:49:21.912004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.247144+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:46:59.994274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.246823+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:22:20.050540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.247794+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:43.890558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.245963+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:34.472672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.245987+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:32:24.133588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.249549+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:31:35.645852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.245494+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:13:57.653530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248419+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:51:37.315739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.246907+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:23:43.890558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248177+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:50:47.757427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.247111+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:15:00.686776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248357+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:15:01.956220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.245641+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:19.675512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248070+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:46.911503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-11T07:14:04.243945+0000","last_change":"2024-01-11T07:14:04.243945+0000","last_active":"2024-01-11T07:14:04.243945+0000","last_peered":"2024-01-11T07:14:04.243945+0000","last_clean":"2024-01-11T07:14:04.243945+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:04.243945+0000","last_undegraded":"2024-01-11T07:14:04.243945+0000","last_fullsized":"2024-01-11T07:14:04.243945+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-01-11T07:14:03.242189+0000","last_change":"2024-01-11T07:14:03.248790+0000","last_active":"2024-01-11T07:14:03.242189+0000","last_peered":"2024-01-11T07:14:03.242189+0000","last_clean":"2024-01-11T07:14:03.242189+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-11T07:14:03.242189+0000","last_undegraded":"2024-01-11T07:14:03.242189+0000","last_fullsized":"2024-01-11T07:14:03.242189+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:04:30.735281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":276,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-01-11T07:14:00.599823+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:14:00.599823+0000","last_peered":"2024-01-11T07:14:00.599823+0000","last_clean":"2024-01-11T07:14:00.599823+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:14:00.599823+0000","last_undegraded":"2024-01-11T07:14:00.599823+0000","last_fullsized":"2024-01-11T07:14:00.599823+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1381,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193931,"statfs":{"total":200000000,"available":198585528,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.34499999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57499999999999996,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.62}]},{"osd":1,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58599999999999997,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57799999999999996,"15min":0.55000000000000004},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.66300000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.751,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.49399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74299999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.57299999999999995},"min":{"1min":0.499,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73199999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.48599999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70899999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51300000000000001,"15min":0.48799999999999999},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.308},"max":{"1min":0.755,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.52400000000000002},"min":{"1min":0.33100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.34100000000000003},"max":{"1min":0.748,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60999999999999999,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.35999999999999999},"max":{"1min":0.81000000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.63500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.81100000000000005,"15min":0.81399999999999995},"last":0.496},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":1.157},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.56899999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79500000000000004,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.33800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":0.74099999999999999,"15min":0.80500000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.47799999999999998,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.46999999999999997}]},{"osd":3,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.29599999999999999},"max":{"1min":0.84599999999999997,"5min":2.621,"15min":2.621},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.85799999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48899999999999999,"15min":0.53600000000000003},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68500000000000005,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.54800000000000004},"min":{"1min":0.39000000000000001,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.72299999999999998,"5min":0.72399999999999998,"15min":0.78800000000000003},"last":0.42199999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.40799999999999997,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.745,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.61399999999999999},"min":{"1min":0.36299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.755,"5min":0.83399999999999996,"15min":0.89100000000000001},"last":0.36099999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68600000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70899999999999996,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.38}]}]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.47199999999999998,"15min":0.504},"min":{"1min":0.26800000000000002,"5min":0.25800000000000001,"15min":0.216},"max":{"1min":0.753,"5min":8.5150000000000006,"15min":20.41},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48199999999999998,"15min":0.46100000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.72099999999999997,"5min":3.7029999999999998,"15min":3.7029999999999998},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.597,"5min":7.8099999999999996,"15min":15.611000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.0710000000000002,"5min":6.1390000000000002,"15min":20.199999999999999},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53200000000000003,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.29999999999999999},"max":{"1min":3.8460000000000001,"5min":5.6449999999999996,"15min":20.369},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.54000000000000004,"15min":0.47199999999999998},"min":{"1min":0.33400000000000002,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":4.7569999999999997,"5min":8.5790000000000006,"15min":8.5790000000000006},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.40999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.3300000000000001,"5min":7.6749999999999998,"15min":9.9570000000000007},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.51100000000000001,"5min":0.44,"15min":0.41899999999999998},"max":{"1min":1.2030000000000001,"5min":5.7830000000000004,"15min":12.449999999999999},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65600000000000003,"15min":0.624},"min":{"1min":0.48099999999999998,"5min":0.437,"15min":0.376},"max":{"1min":1.3540000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":1.177,"5min":7.7480000000000002,"15min":9.891},"last":0.56399999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":1.389,"5min":1.389,"15min":1.5569999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.504,"5min":0.47499999999999998,"15min":0.41699999999999998},"max":{"1min":2.9049999999999998,"5min":7.7770000000000001,"15min":9.9900000000000002},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.78200000000000003},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.442},"max":{"1min":1.1599999999999999,"5min":5.8449999999999998,"15min":20.324000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46800000000000003},"max":{"1min":1.268,"5min":2.548,"15min":2.548},"last":0.63200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1375,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193936,"statfs":{"total":200000000,"available":198591333,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39900000000000002,"15min":0.46300000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.224},"max":{"1min":0.56699999999999995,"5min":0.628,"15min":2.7429999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.51500000000000001,"5min":0.627,"15min":0.72199999999999998},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.68600000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.23999999999999999},"max":{"1min":0.59199999999999997,"5min":0.73199999999999998,"15min":7.7960000000000003},"last":0.433},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44600000000000001,"15min":0.52400000000000002},"min":{"1min":0.29099999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63400000000000001,"5min":0.72099999999999997,"15min":2.5880000000000001},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.69699999999999995},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":8.0730000000000004},"last":0.31},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48599999999999999,"15min":0.56599999999999995},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.755,"5min":0.77700000000000002,"15min":2.855},"last":0.46700000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64200000000000002,"15min":0.87},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40999999999999998},"max":{"1min":0.86599999999999999,"5min":0.94999999999999996,"15min":8.0399999999999991},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65700000000000003,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.872,"5min":0.97099999999999997,"15min":2.819},"last":0.628},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.92300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":1.046,"15min":7.9539999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.48099999999999998,"15min":0.41799999999999998},"max":{"1min":0.91700000000000004,"5min":7.4690000000000003,"15min":7.4690000000000003},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:13:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68100000000000005,"15min":0.66500000000000004},"min":{"1min":0.45200000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68700000000000006,"15min":0.91900000000000004},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.44900000000000001},"max":{"1min":0.89500000000000002,"5min":1.0429999999999999,"15min":8.0030000000000001},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40100000000000002,"15min":0.41499999999999998},"min":{"1min":0.27000000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48999999999999999,"5min":1.472,"15min":1.472},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.53900000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.30499999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61899999999999999,"5min":1.0309999999999999,"15min":1.2430000000000001},"last":0.375},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.41299999999999998,"15min":0.39100000000000001},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.246},"max":{"1min":0.47899999999999998,"5min":0.59499999999999997,"15min":0.60899999999999999},"last":0.312}]},{"osd":3,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.64500000000000002,"5min":0.74399999999999999,"15min":0.76800000000000002},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.37,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59499999999999997,"5min":1.081,"15min":1.139},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63500000000000001,"15min":0.65900000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79000000000000004,"5min":1.0169999999999999,"15min":1.0660000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":1.089,"15min":1.089},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65600000000000003,"15min":0.64700000000000002},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.41999999999999998},"max":{"1min":0.86599999999999999,"5min":1.099,"15min":1.099},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90500000000000003,"5min":1.145,"15min":1.3100000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44900000000000001},"max":{"1min":0.88800000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89400000000000002,"5min":1.006,"15min":1.052},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84999999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81799999999999995,"5min":1.042,"15min":1.042},"last":0.63600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39300000000000002,"15min":0.71299999999999997},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53100000000000003,"5min":0.58699999999999997,"15min":11.481},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46100000000000002,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.52200000000000002,"5min":0.65400000000000003,"15min":1.6200000000000001},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39500000000000002,"15min":0.86899999999999999},"min":{"1min":0.253,"5min":0.252,"15min":0.23899999999999999},"max":{"1min":0.51700000000000002,"5min":0.69999999999999996,"15min":17.516999999999999},"last":0.379},{"interface":"front","average":{"1min":0.434,"5min":0.438,"15min":0.42199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.20200000000000001},"max":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.67400000000000004},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.31,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.31},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.92700000000000005},"min":{"1min":0.35699999999999998,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60499999999999998,"5min":0.79300000000000004,"15min":17.553000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65300000000000002,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40300000000000002},"max":{"1min":0.81200000000000006,"5min":0.93899999999999995,"15min":0.94599999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67400000000000004,"15min":1.157},"min":{"1min":0.56100000000000005,"5min":0.38500000000000001,"15min":0.36199999999999999},"max":{"1min":0.84399999999999997,"5min":0.95499999999999996,"15min":17.474},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.626,"15min":1.0960000000000001},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.98799999999999999,"15min":17.393000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65000000000000002,"15min":0.94699999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.82299999999999995,"5min":1.0149999999999999,"15min":11.394},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.66000000000000003,"15min":0.96099999999999997},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.34699999999999998},"max":{"1min":0.88,"5min":0.97299999999999998,"15min":11.436999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.47199999999999998,"5min":0.46000000000000002,"15min":0.44400000000000001},"max":{"1min":0.81599999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46300000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.53100000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79900000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68000000000000005,"15min":1.135},"min":{"1min":0.51300000000000001,"5min":0.45700000000000002,"15min":0.42999999999999999},"max":{"1min":0.77300000000000002,"5min":1.1839999999999999,"15min":17.434000000000001},"last":0.625}]}]},{"osd":4,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.64800000000000002,"15min":1.0129999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.248},"max":{"1min":0.753,"5min":3.5579999999999998,"15min":8.484},"last":0.502},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65200000000000002,"15min":1.0660000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.76700000000000002,"5min":3.5329999999999999,"15min":9.2599999999999998},"last":0.49099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.59599999999999997,"15min":0.83999999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.30199999999999999},"max":{"1min":0.92700000000000005,"5min":1.2250000000000001,"15min":8.5340000000000007},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.61299999999999999,"15min":0.96999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.74099999999999999,"5min":4.2560000000000002,"15min":9.2279999999999998},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.65900000000000003,"15min":0.94399999999999995},"min":{"1min":0.42399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70299999999999996,"5min":5.9770000000000003,"15min":9.5640000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.84999999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.90600000000000003,"5min":0.93000000000000005,"15min":6.8639999999999999},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57899999999999996,"15min":1.004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81499999999999995,"5min":1.208,"15min":8.9779999999999998},"last":0.432},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.65300000000000002,"15min":1.075},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73799999999999999,"5min":4.8680000000000003,"15min":8.7309999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46200000000000002,"15min":0.89800000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":11.388999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.82799999999999996},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.76800000000000002,"5min":1.0880000000000001,"15min":8.6509999999999998},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53100000000000003,"15min":0.877},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80100000000000005,"5min":0.91800000000000004,"15min":9.1449999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.378,"5min":0.377,"15min":0.35799999999999998},"max":{"1min":0.81200000000000006,"5min":0.86099999999999999,"15min":1.119},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:13:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.85999999999999999},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.83299999999999996,"5min":0.95999999999999996,"15min":8.6050000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.95499999999999996},"min":{"1min":0.40200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.80000000000000004,"5min":0.874,"15min":9.4640000000000004},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":804,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194507,"statfs":{"total":200000000,"available":199175896,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73499999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.56399999999999995},"min":{"1min":0.36699999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76700000000000002,"5min":0.80300000000000005,"15min":0.84099999999999997},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.56399999999999995},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.35399999999999998},"max":{"1min":0.77100000000000002,"5min":0.79700000000000004,"15min":0.82799999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.42199999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.93999999999999995},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.624,"15min":0.62},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.497,"15min":0.504},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63400000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":0.69099999999999995,"5min":0.73999999999999999,"15min":0.76100000000000001},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52100000000000002,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.313,"15min":0.313},"max":{"1min":0.69599999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73499999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.499}]},{"osd":7,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77700000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.85199999999999998,"15min":0.88200000000000001},"last":0.53000000000000003}]}]}],"pool_statfs":[]}} 2024-01-11T07:14:09.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:14:09.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.822+0000 7fe025121700 1 Processor -- start 2024-01-11T07:14:09.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.822+0000 7fe025121700 1 -- start start 2024-01-11T07:14:09.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe025121700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 0x7fe020063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe025121700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe02015a280 0x7fe0200aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe025121700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0200ac5b0 0x7fe0200aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe025121700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02015d880 con 0x7fe02015a280 2024-01-11T07:14:09.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe025121700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02006c5b0 con 0x7fe0200ac5b0 2024-01-11T07:14:09.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe025121700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe020063b40 con 0x7fe020063230 2024-01-11T07:14:09.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe01e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe02015a280 0x7fe0200aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe01e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe02015a280 0x7fe0200aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56526/0 (socket says 172.21.15.7:56526) 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe01e59c700 1 -- 172.21.15.7:0/3076396373 learned_addr learned my addr 172.21.15.7:0/3076396373 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe01ed9d700 1 --2- 172.21.15.7:0/3076396373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 0x7fe020063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.823+0000 7fe01f59e700 1 --2- 172.21.15.7:0/3076396373 >> v2:172.21.15.113:3300/0 conn(0x7fe0200ac5b0 0x7fe0200aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01e59c700 1 -- 172.21.15.7:0/3076396373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 msgr2=0x7fe020063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01e59c700 1 --2- 172.21.15.7:0/3076396373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 0x7fe020063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01e59c700 1 -- 172.21.15.7:0/3076396373 >> v2:172.21.15.113:3300/0 conn(0x7fe0200ac5b0 msgr2=0x7fe0200aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01e59c700 1 --2- 172.21.15.7:0/3076396373 >> v2:172.21.15.113:3300/0 conn(0x7fe0200ac5b0 0x7fe0200aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01e59c700 1 -- 172.21.15.7:0/3076396373 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02015a7f0 con 0x7fe02015a280 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01e59c700 1 --2- 172.21.15.7:0/3076396373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a280 0x7fe0200aa9f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe02006f050 tx=0x7fe008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcbf1c7f9ca14 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01dd9b700 1 -- 172.21.15.7:0/3076396373 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800b640 con 0x7fe02015a280 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01dd9b700 1 -- 172.21.15.7:0/3076396373 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00800b7a0 con 0x7fe02015a280 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.824+0000 7fe01dd9b700 1 -- 172.21.15.7:0/3076396373 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800bb60 con 0x7fe02015a280 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 -- 172.21.15.7:0/3076396373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a280 msgr2=0x7fe0200aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 --2- 172.21.15.7:0/3076396373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a280 0x7fe0200aa9f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe02006f050 tx=0x7fe008009250 comp rx=0 tx=0).stop 2024-01-11T07:14:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 -- 172.21.15.7:0/3076396373 shutdown_connections 2024-01-11T07:14:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 --2- 172.21.15.7:0/3076396373 >> v2:172.21.15.113:3300/0 conn(0x7fe0200ac5b0 0x7fe0200aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 --2- 172.21.15.7:0/3076396373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 0x7fe020063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 --2- 172.21.15.7:0/3076396373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02015a280 0x7fe0200aa9f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 -- 172.21.15.7:0/3076396373 >> 172.21.15.7:0/3076396373 conn(0x7fe0200680e0 msgr2=0x7fe0200cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 -- 172.21.15.7:0/3076396373 shutdown_connections 2024-01-11T07:14:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.825+0000 7fe025121700 1 -- 172.21.15.7:0/3076396373 wait complete. 2024-01-11T07:14:09.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.826+0000 7fe025121700 1 Processor -- start 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.826+0000 7fe025121700 1 -- start start 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe025121700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 0x7fe020153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe025121700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0200ac5b0 0x7fe0200b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe025121700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe02015a280 0x7fe0200b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe025121700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200af690 con 0x7fe0200ac5b0 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe025121700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200aea80 con 0x7fe02015a280 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe025121700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200aed60 con 0x7fe020063230 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe01e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0200ac5b0 0x7fe0200b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe01e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0200ac5b0 0x7fe0200b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56532/0 (socket says 172.21.15.7:56532) 2024-01-11T07:14:09.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe01e59c700 1 -- 172.21.15.7:0/234260360 learned_addr learned my addr 172.21.15.7:0/234260360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe01ed9d700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 0x7fe020153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.827+0000 7fe01f59e700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe02015a280 0x7fe0200b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe01e59c700 1 -- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 msgr2=0x7fe020153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe01e59c700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 0x7fe020153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe01e59c700 1 -- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe02015a280 msgr2=0x7fe0200b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe01e59c700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe02015a280 0x7fe0200b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe01e59c700 1 -- 172.21.15.7:0/234260360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080094f0 con 0x7fe0200ac5b0 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe01e59c700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0200ac5b0 0x7fe0200b1b00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe02006d820 tx=0x7fe008017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe016ffd700 1 -- 172.21.15.7:0/234260360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800bb20 con 0x7fe0200ac5b0 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe016ffd700 1 -- 172.21.15.7:0/234260360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe008000c90 con 0x7fe0200ac5b0 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0200af040 con 0x7fe0200ac5b0 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe016ffd700 1 -- 172.21.15.7:0/234260360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008011670 con 0x7fe0200ac5b0 2024-01-11T07:14:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.828+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0201540e0 con 0x7fe0200ac5b0 2024-01-11T07:14:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.829+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe020154300 con 0x7fe0200ac5b0 2024-01-11T07:14:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.829+0000 7fe016ffd700 1 -- 172.21.15.7:0/234260360 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe0080119e0 con 0x7fe0200ac5b0 2024-01-11T07:14:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.830+0000 7fe016ffd700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe000066f30 0x7fe0000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.830+0000 7fe016ffd700 1 -- 172.21.15.7:0/234260360 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe008085ce0 con 0x7fe0200ac5b0 2024-01-11T07:14:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.830+0000 7fe01ed9d700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe000066f30 0x7fe0000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.831+0000 7fe01ed9d700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe000066f30 0x7fe0000693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe010009d10 tx=0x7fe0100091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:09.834+0000 7fe016ffd700 1 -- 172.21.15.7:0/234260360 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe008052f50 con 0x7fe0200ac5b0 2024-01-11T07:14:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.038+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe02004c3c0 con 0x7fe000066f30 2024-01-11T07:14:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.047+0000 7fe016ffd700 1 -- 172.21.15.7:0/234260360 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285615 (secure 0 0 0) 0x7fe02004c3c0 con 0x7fe000066f30 2024-01-11T07:14:10.051 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe000066f30 msgr2=0x7fe0000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe000066f30 0x7fe0000693e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe010009d10 tx=0x7fe0100091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0200ac5b0 msgr2=0x7fe0200b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0200ac5b0 0x7fe0200b1b00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe02006d820 tx=0x7fe008017000 comp rx=0 tx=0).stop 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 shutdown_connections 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe000066f30 0x7fe0000693e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe02015a280 0x7fe0200b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe020063230 0x7fe020153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 --2- 172.21.15.7:0/234260360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0200ac5b0 0x7fe0200b1b00 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 >> 172.21.15.7:0/234260360 conn(0x7fe0200680e0 msgr2=0x7fe02015cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.050+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 shutdown_connections 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.051+0000 7fe025121700 1 -- 172.21.15.7:0/234260360 wait complete. 2024-01-11T07:14:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:14:10.064 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-01-11T07:14:09.941312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554753,"statfs":{"total":1600000000,"available":1592067308,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001381"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.468131+0000","last_change":"2024-01-11T07:14:05.465763+0000","last_active":"2024-01-11T07:14:05.468131+0000","last_peered":"2024-01-11T07:14:05.468131+0000","last_clean":"2024-01-11T07:14:05.468131+0000","last_became_active":"2024-01-11T07:14:05.464860+0000","last_became_peered":"2024-01-11T07:14:05.464860+0000","last_unstale":"2024-01-11T07:14:05.468131+0000","last_undegraded":"2024-01-11T07:14:05.468131+0000","last_fullsized":"2024-01-11T07:14:05.468131+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:40:06.015816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.459926+0000","last_change":"2024-01-11T07:14:05.459294+0000","last_active":"2024-01-11T07:14:05.459926+0000","last_peered":"2024-01-11T07:14:05.459926+0000","last_clean":"2024-01-11T07:14:05.459926+0000","last_became_active":"2024-01-11T07:14:05.459010+0000","last_became_peered":"2024-01-11T07:14:05.459010+0000","last_unstale":"2024-01-11T07:14:05.459926+0000","last_undegraded":"2024-01-11T07:14:05.459926+0000","last_fullsized":"2024-01-11T07:14:05.459926+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:51:17.756057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.451130+0000","last_change":"2024-01-11T07:14:05.450707+0000","last_active":"2024-01-11T07:14:05.451130+0000","last_peered":"2024-01-11T07:14:05.451130+0000","last_clean":"2024-01-11T07:14:05.451130+0000","last_became_active":"2024-01-11T07:14:05.449956+0000","last_became_peered":"2024-01-11T07:14:05.449956+0000","last_unstale":"2024-01-11T07:14:05.451130+0000","last_undegraded":"2024-01-11T07:14:05.451130+0000","last_fullsized":"2024-01-11T07:14:05.451130+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:09.985279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.463051+0000","last_change":"2024-01-11T07:14:05.462575+0000","last_active":"2024-01-11T07:14:05.463051+0000","last_peered":"2024-01-11T07:14:05.463051+0000","last_clean":"2024-01-11T07:14:05.463051+0000","last_became_active":"2024-01-11T07:14:05.462271+0000","last_became_peered":"2024-01-11T07:14:05.462271+0000","last_unstale":"2024-01-11T07:14:05.463051+0000","last_undegraded":"2024-01-11T07:14:05.463051+0000","last_fullsized":"2024-01-11T07:14:05.463051+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:14:24.805384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.469977+0000","last_change":"2024-01-11T07:14:05.468364+0000","last_active":"2024-01-11T07:14:05.469977+0000","last_peered":"2024-01-11T07:14:05.469977+0000","last_clean":"2024-01-11T07:14:05.469977+0000","last_became_active":"2024-01-11T07:14:05.467108+0000","last_became_peered":"2024-01-11T07:14:05.467108+0000","last_unstale":"2024-01-11T07:14:05.469977+0000","last_undegraded":"2024-01-11T07:14:05.469977+0000","last_fullsized":"2024-01-11T07:14:05.469977+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:58:59.702023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.465297+0000","last_change":"2024-01-11T07:14:05.464519+0000","last_active":"2024-01-11T07:14:05.465297+0000","last_peered":"2024-01-11T07:14:05.465297+0000","last_clean":"2024-01-11T07:14:05.465297+0000","last_became_active":"2024-01-11T07:14:05.464255+0000","last_became_peered":"2024-01-11T07:14:05.464255+0000","last_unstale":"2024-01-11T07:14:05.465297+0000","last_undegraded":"2024-01-11T07:14:05.465297+0000","last_fullsized":"2024-01-11T07:14:05.465297+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:50:18.747387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460752+0000","last_change":"2024-01-11T07:14:05.458133+0000","last_active":"2024-01-11T07:14:05.460752+0000","last_peered":"2024-01-11T07:14:05.460752+0000","last_clean":"2024-01-11T07:14:05.460752+0000","last_became_active":"2024-01-11T07:14:05.457900+0000","last_became_peered":"2024-01-11T07:14:05.457900+0000","last_unstale":"2024-01-11T07:14:05.460752+0000","last_undegraded":"2024-01-11T07:14:05.460752+0000","last_fullsized":"2024-01-11T07:14:05.460752+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:18:40.181336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460972+0000","last_change":"2024-01-11T07:14:05.460359+0000","last_active":"2024-01-11T07:14:05.460972+0000","last_peered":"2024-01-11T07:14:05.460972+0000","last_clean":"2024-01-11T07:14:05.460972+0000","last_became_active":"2024-01-11T07:14:05.459759+0000","last_became_peered":"2024-01-11T07:14:05.459759+0000","last_unstale":"2024-01-11T07:14:05.460972+0000","last_undegraded":"2024-01-11T07:14:05.460972+0000","last_fullsized":"2024-01-11T07:14:05.460972+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:38:29.965187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.456291+0000","last_change":"2024-01-11T07:14:05.455196+0000","last_active":"2024-01-11T07:14:05.456291+0000","last_peered":"2024-01-11T07:14:05.456291+0000","last_clean":"2024-01-11T07:14:05.456291+0000","last_became_active":"2024-01-11T07:14:05.454382+0000","last_became_peered":"2024-01-11T07:14:05.454382+0000","last_unstale":"2024-01-11T07:14:05.456291+0000","last_undegraded":"2024-01-11T07:14:05.456291+0000","last_fullsized":"2024-01-11T07:14:05.456291+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:56:15.397612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.462966+0000","last_change":"2024-01-11T07:14:05.462531+0000","last_active":"2024-01-11T07:14:05.462966+0000","last_peered":"2024-01-11T07:14:05.462966+0000","last_clean":"2024-01-11T07:14:05.462966+0000","last_became_active":"2024-01-11T07:14:05.461065+0000","last_became_peered":"2024-01-11T07:14:05.461065+0000","last_unstale":"2024-01-11T07:14:05.462966+0000","last_undegraded":"2024-01-11T07:14:05.462966+0000","last_fullsized":"2024-01-11T07:14:05.462966+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:51:38.317496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.451398+0000","last_change":"2024-01-11T07:14:05.451199+0000","last_active":"2024-01-11T07:14:05.451398+0000","last_peered":"2024-01-11T07:14:05.451398+0000","last_clean":"2024-01-11T07:14:05.451398+0000","last_became_active":"2024-01-11T07:14:05.450879+0000","last_became_peered":"2024-01-11T07:14:05.450879+0000","last_unstale":"2024-01-11T07:14:05.451398+0000","last_undegraded":"2024-01-11T07:14:05.451398+0000","last_fullsized":"2024-01-11T07:14:05.451398+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:59:41.511814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.456309+0000","last_change":"2024-01-11T07:14:05.455668+0000","last_active":"2024-01-11T07:14:05.456309+0000","last_peered":"2024-01-11T07:14:05.456309+0000","last_clean":"2024-01-11T07:14:05.456309+0000","last_became_active":"2024-01-11T07:14:05.455000+0000","last_became_peered":"2024-01-11T07:14:05.455000+0000","last_unstale":"2024-01-11T07:14:05.456309+0000","last_undegraded":"2024-01-11T07:14:05.456309+0000","last_fullsized":"2024-01-11T07:14:05.456309+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:48.912839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.464529+0000","last_change":"2024-01-11T07:14:05.464272+0000","last_active":"2024-01-11T07:14:05.464529+0000","last_peered":"2024-01-11T07:14:05.464529+0000","last_clean":"2024-01-11T07:14:05.464529+0000","last_became_active":"2024-01-11T07:14:05.463730+0000","last_became_peered":"2024-01-11T07:14:05.463730+0000","last_unstale":"2024-01-11T07:14:05.464529+0000","last_undegraded":"2024-01-11T07:14:05.464529+0000","last_fullsized":"2024-01-11T07:14:05.464529+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:08:12.014703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.464435+0000","last_change":"2024-01-11T07:14:05.464183+0000","last_active":"2024-01-11T07:14:05.464435+0000","last_peered":"2024-01-11T07:14:05.464435+0000","last_clean":"2024-01-11T07:14:05.464435+0000","last_became_active":"2024-01-11T07:14:05.463981+0000","last_became_peered":"2024-01-11T07:14:05.463981+0000","last_unstale":"2024-01-11T07:14:05.464435+0000","last_undegraded":"2024-01-11T07:14:05.464435+0000","last_fullsized":"2024-01-11T07:14:05.464435+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:20.677269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.462549+0000","last_change":"2024-01-11T07:14:05.462193+0000","last_active":"2024-01-11T07:14:05.462549+0000","last_peered":"2024-01-11T07:14:05.462549+0000","last_clean":"2024-01-11T07:14:05.462549+0000","last_became_active":"2024-01-11T07:14:05.454922+0000","last_became_peered":"2024-01-11T07:14:05.454922+0000","last_unstale":"2024-01-11T07:14:05.462549+0000","last_undegraded":"2024-01-11T07:14:05.462549+0000","last_fullsized":"2024-01-11T07:14:05.462549+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:33:18.031272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.470951+0000","last_change":"2024-01-11T07:14:05.469524+0000","last_active":"2024-01-11T07:14:05.470951+0000","last_peered":"2024-01-11T07:14:05.470951+0000","last_clean":"2024-01-11T07:14:05.470951+0000","last_became_active":"2024-01-11T07:14:05.469265+0000","last_became_peered":"2024-01-11T07:14:05.469265+0000","last_unstale":"2024-01-11T07:14:05.470951+0000","last_undegraded":"2024-01-11T07:14:05.470951+0000","last_fullsized":"2024-01-11T07:14:05.470951+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:15:56.495437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460715+0000","last_change":"2024-01-11T07:14:05.457488+0000","last_active":"2024-01-11T07:14:05.460715+0000","last_peered":"2024-01-11T07:14:05.460715+0000","last_clean":"2024-01-11T07:14:05.460715+0000","last_became_active":"2024-01-11T07:14:05.456867+0000","last_became_peered":"2024-01-11T07:14:05.456867+0000","last_unstale":"2024-01-11T07:14:05.460715+0000","last_undegraded":"2024-01-11T07:14:05.460715+0000","last_fullsized":"2024-01-11T07:14:05.460715+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:32:25.135344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.463252+0000","last_change":"2024-01-11T07:14:05.462998+0000","last_active":"2024-01-11T07:14:05.463252+0000","last_peered":"2024-01-11T07:14:05.463252+0000","last_clean":"2024-01-11T07:14:05.463252+0000","last_became_active":"2024-01-11T07:14:05.462551+0000","last_became_peered":"2024-01-11T07:14:05.462551+0000","last_unstale":"2024-01-11T07:14:05.463252+0000","last_undegraded":"2024-01-11T07:14:05.463252+0000","last_fullsized":"2024-01-11T07:14:05.463252+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:06:23.988529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.458385+0000","last_change":"2024-01-11T07:14:05.458308+0000","last_active":"2024-01-11T07:14:05.458385+0000","last_peered":"2024-01-11T07:14:05.458385+0000","last_clean":"2024-01-11T07:14:05.458385+0000","last_became_active":"2024-01-11T07:14:05.457270+0000","last_became_peered":"2024-01-11T07:14:05.457270+0000","last_unstale":"2024-01-11T07:14:05.458385+0000","last_undegraded":"2024-01-11T07:14:05.458385+0000","last_fullsized":"2024-01-11T07:14:05.458385+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:59:59.100697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.471000+0000","last_change":"2024-01-11T07:14:05.469664+0000","last_active":"2024-01-11T07:14:05.471000+0000","last_peered":"2024-01-11T07:14:05.471000+0000","last_clean":"2024-01-11T07:14:05.471000+0000","last_became_active":"2024-01-11T07:14:05.468858+0000","last_became_peered":"2024-01-11T07:14:05.468858+0000","last_unstale":"2024-01-11T07:14:05.471000+0000","last_undegraded":"2024-01-11T07:14:05.471000+0000","last_fullsized":"2024-01-11T07:14:05.471000+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:51:30.774196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.458127+0000","last_change":"2024-01-11T07:14:05.457983+0000","last_active":"2024-01-11T07:14:05.458127+0000","last_peered":"2024-01-11T07:14:05.458127+0000","last_clean":"2024-01-11T07:14:05.458127+0000","last_became_active":"2024-01-11T07:14:05.457732+0000","last_became_peered":"2024-01-11T07:14:05.457732+0000","last_unstale":"2024-01-11T07:14:05.458127+0000","last_undegraded":"2024-01-11T07:14:05.458127+0000","last_fullsized":"2024-01-11T07:14:05.458127+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:51:30.774196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.452613+0000","last_change":"2024-01-11T07:14:05.452541+0000","last_active":"2024-01-11T07:14:05.452613+0000","last_peered":"2024-01-11T07:14:05.452613+0000","last_clean":"2024-01-11T07:14:05.452613+0000","last_became_active":"2024-01-11T07:14:05.452327+0000","last_became_peered":"2024-01-11T07:14:05.452327+0000","last_unstale":"2024-01-11T07:14:05.452613+0000","last_undegraded":"2024-01-11T07:14:05.452613+0000","last_fullsized":"2024-01-11T07:14:05.452613+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:24:19.856282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460991+0000","last_change":"2024-01-11T07:14:05.460509+0000","last_active":"2024-01-11T07:14:05.460991+0000","last_peered":"2024-01-11T07:14:05.460991+0000","last_clean":"2024-01-11T07:14:05.460991+0000","last_became_active":"2024-01-11T07:14:05.460065+0000","last_became_peered":"2024-01-11T07:14:05.460065+0000","last_unstale":"2024-01-11T07:14:05.460991+0000","last_undegraded":"2024-01-11T07:14:05.460991+0000","last_fullsized":"2024-01-11T07:14:05.460991+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:03:01.174268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.458676+0000","last_change":"2024-01-11T07:14:05.458602+0000","last_active":"2024-01-11T07:14:05.458676+0000","last_peered":"2024-01-11T07:14:05.458676+0000","last_clean":"2024-01-11T07:14:05.458676+0000","last_became_active":"2024-01-11T07:14:05.458062+0000","last_became_peered":"2024-01-11T07:14:05.458062+0000","last_unstale":"2024-01-11T07:14:05.458676+0000","last_undegraded":"2024-01-11T07:14:05.458676+0000","last_fullsized":"2024-01-11T07:14:05.458676+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:55:11.733649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.468405+0000","last_change":"2024-01-11T07:14:05.468230+0000","last_active":"2024-01-11T07:14:05.468405+0000","last_peered":"2024-01-11T07:14:05.468405+0000","last_clean":"2024-01-11T07:14:05.468405+0000","last_became_active":"2024-01-11T07:14:05.467859+0000","last_became_peered":"2024-01-11T07:14:05.467859+0000","last_unstale":"2024-01-11T07:14:05.468405+0000","last_undegraded":"2024-01-11T07:14:05.468405+0000","last_fullsized":"2024-01-11T07:14:05.468405+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:33:00.414698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.470930+0000","last_change":"2024-01-11T07:14:05.469440+0000","last_active":"2024-01-11T07:14:05.470930+0000","last_peered":"2024-01-11T07:14:05.470930+0000","last_clean":"2024-01-11T07:14:05.470930+0000","last_became_active":"2024-01-11T07:14:05.469007+0000","last_became_peered":"2024-01-11T07:14:05.469007+0000","last_unstale":"2024-01-11T07:14:05.470930+0000","last_undegraded":"2024-01-11T07:14:05.470930+0000","last_fullsized":"2024-01-11T07:14:05.470930+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:19:18.955469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.468177+0000","last_change":"2024-01-11T07:14:05.466720+0000","last_active":"2024-01-11T07:14:05.468177+0000","last_peered":"2024-01-11T07:14:05.468177+0000","last_clean":"2024-01-11T07:14:05.468177+0000","last_became_active":"2024-01-11T07:14:05.466021+0000","last_became_peered":"2024-01-11T07:14:05.466021+0000","last_unstale":"2024-01-11T07:14:05.468177+0000","last_undegraded":"2024-01-11T07:14:05.468177+0000","last_fullsized":"2024-01-11T07:14:05.468177+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:22:21.052297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.464259+0000","last_change":"2024-01-11T07:14:05.462866+0000","last_active":"2024-01-11T07:14:05.464259+0000","last_peered":"2024-01-11T07:14:05.464259+0000","last_clean":"2024-01-11T07:14:05.464259+0000","last_became_active":"2024-01-11T07:14:05.462457+0000","last_became_peered":"2024-01-11T07:14:05.462457+0000","last_unstale":"2024-01-11T07:14:05.464259+0000","last_undegraded":"2024-01-11T07:14:05.464259+0000","last_fullsized":"2024-01-11T07:14:05.464259+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:01:28.517624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.468195+0000","last_change":"2024-01-11T07:14:05.467817+0000","last_active":"2024-01-11T07:14:05.468195+0000","last_peered":"2024-01-11T07:14:05.468195+0000","last_clean":"2024-01-11T07:14:05.468195+0000","last_became_active":"2024-01-11T07:14:05.467048+0000","last_became_peered":"2024-01-11T07:14:05.467048+0000","last_unstale":"2024-01-11T07:14:05.468195+0000","last_undegraded":"2024-01-11T07:14:05.468195+0000","last_fullsized":"2024-01-11T07:14:05.468195+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:44:37.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.462567+0000","last_change":"2024-01-11T07:14:05.462333+0000","last_active":"2024-01-11T07:14:05.462567+0000","last_peered":"2024-01-11T07:14:05.462567+0000","last_clean":"2024-01-11T07:14:05.462567+0000","last_became_active":"2024-01-11T07:14:05.461168+0000","last_became_peered":"2024-01-11T07:14:05.461168+0000","last_unstale":"2024-01-11T07:14:05.462567+0000","last_undegraded":"2024-01-11T07:14:05.462567+0000","last_fullsized":"2024-01-11T07:14:05.462567+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:05:33.213102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.466475+0000","last_change":"2024-01-11T07:14:05.466434+0000","last_active":"2024-01-11T07:14:05.466475+0000","last_peered":"2024-01-11T07:14:05.466475+0000","last_clean":"2024-01-11T07:14:05.466475+0000","last_became_active":"2024-01-11T07:14:05.465769+0000","last_became_peered":"2024-01-11T07:14:05.465769+0000","last_unstale":"2024-01-11T07:14:05.466475+0000","last_undegraded":"2024-01-11T07:14:05.466475+0000","last_fullsized":"2024-01-11T07:14:05.466475+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:42:05.443357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460737+0000","last_change":"2024-01-11T07:14:05.457646+0000","last_active":"2024-01-11T07:14:05.460737+0000","last_peered":"2024-01-11T07:14:05.460737+0000","last_clean":"2024-01-11T07:14:05.460737+0000","last_became_active":"2024-01-11T07:14:05.457257+0000","last_became_peered":"2024-01-11T07:14:05.457257+0000","last_unstale":"2024-01-11T07:14:05.460737+0000","last_undegraded":"2024-01-11T07:14:05.460737+0000","last_fullsized":"2024-01-11T07:14:05.460737+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:29:56.885308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.463954+0000","last_change":"2024-01-11T07:14:05.463879+0000","last_active":"2024-01-11T07:14:05.463954+0000","last_peered":"2024-01-11T07:14:05.463954+0000","last_clean":"2024-01-11T07:14:05.463954+0000","last_became_active":"2024-01-11T07:14:05.463585+0000","last_became_peered":"2024-01-11T07:14:05.463585+0000","last_unstale":"2024-01-11T07:14:05.463954+0000","last_undegraded":"2024-01-11T07:14:05.463954+0000","last_fullsized":"2024-01-11T07:14:05.463954+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:42:05.443357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"151'4","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.470382+0000","last_change":"2024-01-11T07:14:04.260729+0000","last_active":"2024-01-11T07:14:05.470382+0000","last_peered":"2024-01-11T07:14:05.470382+0000","last_clean":"2024-01-11T07:14:05.470382+0000","last_became_active":"2024-01-11T07:14:04.260485+0000","last_became_peered":"2024-01-11T07:14:04.260485+0000","last_unstale":"2024-01-11T07:14:05.470382+0000","last_undegraded":"2024-01-11T07:14:05.470382+0000","last_fullsized":"2024-01-11T07:14:05.470382+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:19.675512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.457194+0000","last_change":"2024-01-11T07:14:05.457119+0000","last_active":"2024-01-11T07:14:05.457194+0000","last_peered":"2024-01-11T07:14:05.457194+0000","last_clean":"2024-01-11T07:14:05.457194+0000","last_became_active":"2024-01-11T07:14:05.456384+0000","last_became_peered":"2024-01-11T07:14:05.456384+0000","last_unstale":"2024-01-11T07:14:05.457194+0000","last_undegraded":"2024-01-11T07:14:05.457194+0000","last_fullsized":"2024-01-11T07:14:05.457194+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:14.961575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.461254+0000","last_change":"2024-01-11T07:14:04.254754+0000","last_active":"2024-01-11T07:14:05.461254+0000","last_peered":"2024-01-11T07:14:05.461254+0000","last_clean":"2024-01-11T07:14:05.461254+0000","last_became_active":"2024-01-11T07:14:04.254518+0000","last_became_peered":"2024-01-11T07:14:04.254518+0000","last_unstale":"2024-01-11T07:14:05.461254+0000","last_undegraded":"2024-01-11T07:14:05.461254+0000","last_fullsized":"2024-01-11T07:14:05.461254+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:57:49.515592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.466378+0000","last_change":"2024-01-11T07:14:05.466281+0000","last_active":"2024-01-11T07:14:05.466378+0000","last_peered":"2024-01-11T07:14:05.466378+0000","last_clean":"2024-01-11T07:14:05.466378+0000","last_became_active":"2024-01-11T07:14:05.465024+0000","last_became_peered":"2024-01-11T07:14:05.465024+0000","last_unstale":"2024-01-11T07:14:05.466378+0000","last_undegraded":"2024-01-11T07:14:05.466378+0000","last_fullsized":"2024-01-11T07:14:05.466378+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:24:29.858141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.462856+0000","last_change":"2024-01-11T07:14:04.261649+0000","last_active":"2024-01-11T07:14:05.462856+0000","last_peered":"2024-01-11T07:14:05.462856+0000","last_clean":"2024-01-11T07:14:05.462856+0000","last_became_active":"2024-01-11T07:14:04.261404+0000","last_became_peered":"2024-01-11T07:14:04.261404+0000","last_unstale":"2024-01-11T07:14:05.462856+0000","last_undegraded":"2024-01-11T07:14:05.462856+0000","last_fullsized":"2024-01-11T07:14:05.462856+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:12:17.113999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.469996+0000","last_change":"2024-01-11T07:14:05.468539+0000","last_active":"2024-01-11T07:14:05.469996+0000","last_peered":"2024-01-11T07:14:05.469996+0000","last_clean":"2024-01-11T07:14:05.469996+0000","last_became_active":"2024-01-11T07:14:05.467696+0000","last_became_peered":"2024-01-11T07:14:05.467696+0000","last_unstale":"2024-01-11T07:14:05.469996+0000","last_undegraded":"2024-01-11T07:14:05.469996+0000","last_fullsized":"2024-01-11T07:14:05.469996+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:13:26.788464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.464796+0000","last_change":"2024-01-11T07:14:04.260655+0000","last_active":"2024-01-11T07:14:05.464796+0000","last_peered":"2024-01-11T07:14:05.464796+0000","last_clean":"2024-01-11T07:14:05.464796+0000","last_became_active":"2024-01-11T07:14:04.260181+0000","last_became_peered":"2024-01-11T07:14:04.260181+0000","last_unstale":"2024-01-11T07:14:05.464796+0000","last_undegraded":"2024-01-11T07:14:05.464796+0000","last_fullsized":"2024-01-11T07:14:05.464796+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:19:14.916831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.456631+0000","last_change":"2024-01-11T07:14:05.456561+0000","last_active":"2024-01-11T07:14:05.456631+0000","last_peered":"2024-01-11T07:14:05.456631+0000","last_clean":"2024-01-11T07:14:05.456631+0000","last_became_active":"2024-01-11T07:14:05.456238+0000","last_became_peered":"2024-01-11T07:14:05.456238+0000","last_unstale":"2024-01-11T07:14:05.456631+0000","last_undegraded":"2024-01-11T07:14:05.456631+0000","last_fullsized":"2024-01-11T07:14:05.456631+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:56:53.029019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.468294+0000","last_change":"2024-01-11T07:14:04.258184+0000","last_active":"2024-01-11T07:14:05.468294+0000","last_peered":"2024-01-11T07:14:05.468294+0000","last_clean":"2024-01-11T07:14:05.468294+0000","last_became_active":"2024-01-11T07:14:04.257924+0000","last_became_peered":"2024-01-11T07:14:04.257924+0000","last_unstale":"2024-01-11T07:14:05.468294+0000","last_undegraded":"2024-01-11T07:14:05.468294+0000","last_fullsized":"2024-01-11T07:14:05.468294+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:57:49.515592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.453630+0000","last_change":"2024-01-11T07:14:05.453385+0000","last_active":"2024-01-11T07:14:05.453630+0000","last_peered":"2024-01-11T07:14:05.453630+0000","last_clean":"2024-01-11T07:14:05.453630+0000","last_became_active":"2024-01-11T07:14:05.452525+0000","last_became_peered":"2024-01-11T07:14:05.452525+0000","last_unstale":"2024-01-11T07:14:05.453630+0000","last_undegraded":"2024-01-11T07:14:05.453630+0000","last_fullsized":"2024-01-11T07:14:05.453630+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:20.677269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.461902+0000","last_change":"2024-01-11T07:14:04.266227+0000","last_active":"2024-01-11T07:14:05.461902+0000","last_peered":"2024-01-11T07:14:05.461902+0000","last_clean":"2024-01-11T07:14:05.461902+0000","last_became_active":"2024-01-11T07:14:04.265006+0000","last_became_peered":"2024-01-11T07:14:04.265006+0000","last_unstale":"2024-01-11T07:14:05.461902+0000","last_undegraded":"2024-01-11T07:14:05.461902+0000","last_fullsized":"2024-01-11T07:14:05.461902+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:29:55.883552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.458687+0000","last_change":"2024-01-11T07:14:05.458516+0000","last_active":"2024-01-11T07:14:05.458687+0000","last_peered":"2024-01-11T07:14:05.458687+0000","last_clean":"2024-01-11T07:14:05.458687+0000","last_became_active":"2024-01-11T07:14:05.458215+0000","last_became_peered":"2024-01-11T07:14:05.458215+0000","last_unstale":"2024-01-11T07:14:05.458687+0000","last_undegraded":"2024-01-11T07:14:05.458687+0000","last_fullsized":"2024-01-11T07:14:05.458687+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:51:17.756057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.450964+0000","last_change":"2024-01-11T07:14:04.257302+0000","last_active":"2024-01-11T07:14:05.450964+0000","last_peered":"2024-01-11T07:14:05.450964+0000","last_clean":"2024-01-11T07:14:05.450964+0000","last_became_active":"2024-01-11T07:14:04.257049+0000","last_became_peered":"2024-01-11T07:14:04.257049+0000","last_unstale":"2024-01-11T07:14:05.450964+0000","last_undegraded":"2024-01-11T07:14:05.450964+0000","last_fullsized":"2024-01-11T07:14:05.450964+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:22:13.959818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460561+0000","last_change":"2024-01-11T07:14:05.460244+0000","last_active":"2024-01-11T07:14:05.460561+0000","last_peered":"2024-01-11T07:14:05.460561+0000","last_clean":"2024-01-11T07:14:05.460561+0000","last_became_active":"2024-01-11T07:14:05.459981+0000","last_became_peered":"2024-01-11T07:14:05.459981+0000","last_unstale":"2024-01-11T07:14:05.460561+0000","last_undegraded":"2024-01-11T07:14:05.460561+0000","last_fullsized":"2024-01-11T07:14:05.460561+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:38:29.965187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.450901+0000","last_change":"2024-01-11T07:14:04.261380+0000","last_active":"2024-01-11T07:14:05.450901+0000","last_peered":"2024-01-11T07:14:05.450901+0000","last_clean":"2024-01-11T07:14:05.450901+0000","last_became_active":"2024-01-11T07:14:04.258955+0000","last_became_peered":"2024-01-11T07:14:04.258955+0000","last_unstale":"2024-01-11T07:14:05.450901+0000","last_undegraded":"2024-01-11T07:14:05.450901+0000","last_fullsized":"2024-01-11T07:14:05.450901+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:25:19.946181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460613+0000","last_change":"2024-01-11T07:14:05.460526+0000","last_active":"2024-01-11T07:14:05.460613+0000","last_peered":"2024-01-11T07:14:05.460613+0000","last_clean":"2024-01-11T07:14:05.460613+0000","last_became_active":"2024-01-11T07:14:05.460287+0000","last_became_peered":"2024-01-11T07:14:05.460287+0000","last_unstale":"2024-01-11T07:14:05.460613+0000","last_undegraded":"2024-01-11T07:14:05.460613+0000","last_fullsized":"2024-01-11T07:14:05.460613+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:04:49.842384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.454318+0000","last_change":"2024-01-11T07:14:04.260950+0000","last_active":"2024-01-11T07:14:05.454318+0000","last_peered":"2024-01-11T07:14:05.454318+0000","last_clean":"2024-01-11T07:14:05.454318+0000","last_became_active":"2024-01-11T07:14:04.260449+0000","last_became_peered":"2024-01-11T07:14:04.260449+0000","last_unstale":"2024-01-11T07:14:05.454318+0000","last_undegraded":"2024-01-11T07:14:05.454318+0000","last_fullsized":"2024-01-11T07:14:05.454318+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:44:07.700264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.459961+0000","last_change":"2024-01-11T07:14:05.457640+0000","last_active":"2024-01-11T07:14:05.459961+0000","last_peered":"2024-01-11T07:14:05.459961+0000","last_clean":"2024-01-11T07:14:05.459961+0000","last_became_active":"2024-01-11T07:14:05.457380+0000","last_became_peered":"2024-01-11T07:14:05.457380+0000","last_unstale":"2024-01-11T07:14:05.459961+0000","last_undegraded":"2024-01-11T07:14:05.459961+0000","last_fullsized":"2024-01-11T07:14:05.459961+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:00:10.763805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.452885+0000","last_change":"2024-01-11T07:14:04.264338+0000","last_active":"2024-01-11T07:14:05.452885+0000","last_peered":"2024-01-11T07:14:05.452885+0000","last_clean":"2024-01-11T07:14:05.452885+0000","last_became_active":"2024-01-11T07:14:04.264110+0000","last_became_peered":"2024-01-11T07:14:04.264110+0000","last_unstale":"2024-01-11T07:14:05.452885+0000","last_undegraded":"2024-01-11T07:14:05.452885+0000","last_fullsized":"2024-01-11T07:14:05.452885+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:17:11.707362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.462751+0000","last_change":"2024-01-11T07:14:05.462649+0000","last_active":"2024-01-11T07:14:05.462751+0000","last_peered":"2024-01-11T07:14:05.462751+0000","last_clean":"2024-01-11T07:14:05.462751+0000","last_became_active":"2024-01-11T07:14:05.462250+0000","last_became_peered":"2024-01-11T07:14:05.462250+0000","last_unstale":"2024-01-11T07:14:05.462751+0000","last_undegraded":"2024-01-11T07:14:05.462751+0000","last_fullsized":"2024-01-11T07:14:05.462751+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:19:18.955469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"151'3","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.464742+0000","last_change":"2024-01-11T07:14:04.263173+0000","last_active":"2024-01-11T07:14:05.464742+0000","last_peered":"2024-01-11T07:14:05.464742+0000","last_clean":"2024-01-11T07:14:05.464742+0000","last_became_active":"2024-01-11T07:14:04.262882+0000","last_became_peered":"2024-01-11T07:14:04.262882+0000","last_unstale":"2024-01-11T07:14:05.464742+0000","last_undegraded":"2024-01-11T07:14:05.464742+0000","last_fullsized":"2024-01-11T07:14:05.464742+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:23:38.301919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.461534+0000","last_change":"2024-01-11T07:14:05.461438+0000","last_active":"2024-01-11T07:14:05.461534+0000","last_peered":"2024-01-11T07:14:05.461534+0000","last_clean":"2024-01-11T07:14:05.461534+0000","last_became_active":"2024-01-11T07:14:05.461059+0000","last_became_peered":"2024-01-11T07:14:05.461059+0000","last_unstale":"2024-01-11T07:14:05.461534+0000","last_undegraded":"2024-01-11T07:14:05.461534+0000","last_fullsized":"2024-01-11T07:14:05.461534+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:36:22.868464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"151'6","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.470592+0000","last_change":"2024-01-11T07:14:04.257129+0000","last_active":"2024-01-11T07:14:05.470592+0000","last_peered":"2024-01-11T07:14:05.470592+0000","last_clean":"2024-01-11T07:14:05.470592+0000","last_became_active":"2024-01-11T07:14:04.256829+0000","last_became_peered":"2024-01-11T07:14:04.256829+0000","last_unstale":"2024-01-11T07:14:05.470592+0000","last_undegraded":"2024-01-11T07:14:05.470592+0000","last_fullsized":"2024-01-11T07:14:05.470592+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:44:07.700264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.462583+0000","last_change":"2024-01-11T07:14:05.462476+0000","last_active":"2024-01-11T07:14:05.462583+0000","last_peered":"2024-01-11T07:14:05.462583+0000","last_clean":"2024-01-11T07:14:05.462583+0000","last_became_active":"2024-01-11T07:14:05.461495+0000","last_became_peered":"2024-01-11T07:14:05.461495+0000","last_unstale":"2024-01-11T07:14:05.462583+0000","last_undegraded":"2024-01-11T07:14:05.462583+0000","last_fullsized":"2024-01-11T07:14:05.462583+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:34:16.505375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.450821+0000","last_change":"2024-01-11T07:14:04.262023+0000","last_active":"2024-01-11T07:14:05.450821+0000","last_peered":"2024-01-11T07:14:05.450821+0000","last_clean":"2024-01-11T07:14:05.450821+0000","last_became_active":"2024-01-11T07:14:04.260375+0000","last_became_peered":"2024-01-11T07:14:04.260375+0000","last_unstale":"2024-01-11T07:14:05.450821+0000","last_undegraded":"2024-01-11T07:14:05.450821+0000","last_fullsized":"2024-01-11T07:14:05.450821+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:48:14.455861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.459150+0000","last_change":"2024-01-11T07:14:05.459046+0000","last_active":"2024-01-11T07:14:05.459150+0000","last_peered":"2024-01-11T07:14:05.459150+0000","last_clean":"2024-01-11T07:14:05.459150+0000","last_became_active":"2024-01-11T07:14:05.453220+0000","last_became_peered":"2024-01-11T07:14:05.453220+0000","last_unstale":"2024-01-11T07:14:05.459150+0000","last_undegraded":"2024-01-11T07:14:05.459150+0000","last_fullsized":"2024-01-11T07:14:05.459150+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:23:19.766328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.450300+0000","last_change":"2024-01-11T07:14:04.260779+0000","last_active":"2024-01-11T07:14:05.450300+0000","last_peered":"2024-01-11T07:14:05.450300+0000","last_clean":"2024-01-11T07:14:05.450300+0000","last_became_active":"2024-01-11T07:14:04.259031+0000","last_became_peered":"2024-01-11T07:14:04.259031+0000","last_unstale":"2024-01-11T07:14:05.450300+0000","last_undegraded":"2024-01-11T07:14:05.450300+0000","last_fullsized":"2024-01-11T07:14:05.450300+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:06:29.202478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.456325+0000","last_change":"2024-01-11T07:14:05.455812+0000","last_active":"2024-01-11T07:14:05.456325+0000","last_peered":"2024-01-11T07:14:05.456325+0000","last_clean":"2024-01-11T07:14:05.456325+0000","last_became_active":"2024-01-11T07:14:05.455442+0000","last_became_peered":"2024-01-11T07:14:05.455442+0000","last_unstale":"2024-01-11T07:14:05.456325+0000","last_undegraded":"2024-01-11T07:14:05.456325+0000","last_fullsized":"2024-01-11T07:14:05.456325+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:50:31.112318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"151'8","reported_seq":18,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.469994+0000","last_change":"2024-01-11T07:14:04.262629+0000","last_active":"2024-01-11T07:14:05.469994+0000","last_peered":"2024-01-11T07:14:05.469994+0000","last_clean":"2024-01-11T07:14:05.469994+0000","last_became_active":"2024-01-11T07:14:04.262082+0000","last_became_peered":"2024-01-11T07:14:04.262082+0000","last_unstale":"2024-01-11T07:14:05.469994+0000","last_undegraded":"2024-01-11T07:14:05.469994+0000","last_fullsized":"2024-01-11T07:14:05.469994+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:56:40.994670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.458070+0000","last_change":"2024-01-11T07:14:05.457794+0000","last_active":"2024-01-11T07:14:05.458070+0000","last_peered":"2024-01-11T07:14:05.458070+0000","last_clean":"2024-01-11T07:14:05.458070+0000","last_became_active":"2024-01-11T07:14:05.456175+0000","last_became_peered":"2024-01-11T07:14:05.456175+0000","last_unstale":"2024-01-11T07:14:05.458070+0000","last_undegraded":"2024-01-11T07:14:05.458070+0000","last_fullsized":"2024-01-11T07:14:05.458070+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:47:00.996030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.462149+0000","last_change":"2024-01-11T07:14:04.258746+0000","last_active":"2024-01-11T07:14:05.462149+0000","last_peered":"2024-01-11T07:14:05.462149+0000","last_clean":"2024-01-11T07:14:05.462149+0000","last_became_active":"2024-01-11T07:14:04.258452+0000","last_became_peered":"2024-01-11T07:14:04.258452+0000","last_unstale":"2024-01-11T07:14:05.462149+0000","last_undegraded":"2024-01-11T07:14:05.462149+0000","last_fullsized":"2024-01-11T07:14:05.462149+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:15:00.686776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.461375+0000","last_change":"2024-01-11T07:14:05.461157+0000","last_active":"2024-01-11T07:14:05.461375+0000","last_peered":"2024-01-11T07:14:05.461375+0000","last_clean":"2024-01-11T07:14:05.461375+0000","last_became_active":"2024-01-11T07:14:05.460265+0000","last_became_peered":"2024-01-11T07:14:05.460265+0000","last_unstale":"2024-01-11T07:14:05.461375+0000","last_undegraded":"2024-01-11T07:14:05.461375+0000","last_fullsized":"2024-01-11T07:14:05.461375+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:11:20.910385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.462980+0000","last_change":"2024-01-11T07:14:04.254153+0000","last_active":"2024-01-11T07:14:05.462980+0000","last_peered":"2024-01-11T07:14:05.462980+0000","last_clean":"2024-01-11T07:14:05.462980+0000","last_became_active":"2024-01-11T07:14:04.253831+0000","last_became_peered":"2024-01-11T07:14:04.253831+0000","last_unstale":"2024-01-11T07:14:05.462980+0000","last_undegraded":"2024-01-11T07:14:05.462980+0000","last_fullsized":"2024-01-11T07:14:05.462980+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:53:57.849375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.472291+0000","last_change":"2024-01-11T07:14:05.472214+0000","last_active":"2024-01-11T07:14:05.472291+0000","last_peered":"2024-01-11T07:14:05.472291+0000","last_clean":"2024-01-11T07:14:05.472291+0000","last_became_active":"2024-01-11T07:14:05.471863+0000","last_became_peered":"2024-01-11T07:14:05.471863+0000","last_unstale":"2024-01-11T07:14:05.472291+0000","last_undegraded":"2024-01-11T07:14:05.472291+0000","last_fullsized":"2024-01-11T07:14:05.472291+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:21:37.506093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.466347+0000","last_change":"2024-01-11T07:14:04.258221+0000","last_active":"2024-01-11T07:14:05.466347+0000","last_peered":"2024-01-11T07:14:05.466347+0000","last_clean":"2024-01-11T07:14:05.466347+0000","last_became_active":"2024-01-11T07:14:04.257954+0000","last_became_peered":"2024-01-11T07:14:04.257954+0000","last_unstale":"2024-01-11T07:14:05.466347+0000","last_undegraded":"2024-01-11T07:14:05.466347+0000","last_fullsized":"2024-01-11T07:14:05.466347+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:51:16.754301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.461209+0000","last_change":"2024-01-11T07:14:05.461130+0000","last_active":"2024-01-11T07:14:05.461209+0000","last_peered":"2024-01-11T07:14:05.461209+0000","last_clean":"2024-01-11T07:14:05.461209+0000","last_became_active":"2024-01-11T07:14:05.460789+0000","last_became_peered":"2024-01-11T07:14:05.460789+0000","last_unstale":"2024-01-11T07:14:05.461209+0000","last_undegraded":"2024-01-11T07:14:05.461209+0000","last_fullsized":"2024-01-11T07:14:05.461209+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:15:02.957977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.454383+0000","last_change":"2024-01-11T07:14:04.253514+0000","last_active":"2024-01-11T07:14:05.454383+0000","last_peered":"2024-01-11T07:14:05.454383+0000","last_clean":"2024-01-11T07:14:05.454383+0000","last_became_active":"2024-01-11T07:14:04.253321+0000","last_became_peered":"2024-01-11T07:14:04.253321+0000","last_unstale":"2024-01-11T07:14:05.454383+0000","last_undegraded":"2024-01-11T07:14:05.454383+0000","last_fullsized":"2024-01-11T07:14:05.454383+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:56:40.994670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.468214+0000","last_change":"2024-01-11T07:14:05.467969+0000","last_active":"2024-01-11T07:14:05.468214+0000","last_peered":"2024-01-11T07:14:05.468214+0000","last_clean":"2024-01-11T07:14:05.468214+0000","last_became_active":"2024-01-11T07:14:05.467502+0000","last_became_peered":"2024-01-11T07:14:05.467502+0000","last_unstale":"2024-01-11T07:14:05.468214+0000","last_undegraded":"2024-01-11T07:14:05.468214+0000","last_fullsized":"2024-01-11T07:14:05.468214+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:42:09.467255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.464954+0000","last_change":"2024-01-11T07:14:04.259974+0000","last_active":"2024-01-11T07:14:05.464954+0000","last_peered":"2024-01-11T07:14:05.464954+0000","last_clean":"2024-01-11T07:14:05.464954+0000","last_became_active":"2024-01-11T07:14:04.259743+0000","last_became_peered":"2024-01-11T07:14:04.259743+0000","last_unstale":"2024-01-11T07:14:05.464954+0000","last_undegraded":"2024-01-11T07:14:05.464954+0000","last_fullsized":"2024-01-11T07:14:05.464954+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:58:50.815963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460800+0000","last_change":"2024-01-11T07:14:05.460600+0000","last_active":"2024-01-11T07:14:05.460800+0000","last_peered":"2024-01-11T07:14:05.460800+0000","last_clean":"2024-01-11T07:14:05.460800+0000","last_became_active":"2024-01-11T07:14:05.460215+0000","last_became_peered":"2024-01-11T07:14:05.460215+0000","last_unstale":"2024-01-11T07:14:05.460800+0000","last_undegraded":"2024-01-11T07:14:05.460800+0000","last_fullsized":"2024-01-11T07:14:05.460800+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:17:12.709119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.457475+0000","last_change":"2024-01-11T07:14:04.263576+0000","last_active":"2024-01-11T07:14:05.457475+0000","last_peered":"2024-01-11T07:14:05.457475+0000","last_clean":"2024-01-11T07:14:05.457475+0000","last_became_active":"2024-01-11T07:14:04.263089+0000","last_became_peered":"2024-01-11T07:14:04.263089+0000","last_unstale":"2024-01-11T07:14:05.457475+0000","last_undegraded":"2024-01-11T07:14:05.457475+0000","last_fullsized":"2024-01-11T07:14:05.457475+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:50:17.745630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.452125+0000","last_change":"2024-01-11T07:14:05.452050+0000","last_active":"2024-01-11T07:14:05.452125+0000","last_peered":"2024-01-11T07:14:05.452125+0000","last_clean":"2024-01-11T07:14:05.452125+0000","last_became_active":"2024-01-11T07:14:05.451722+0000","last_became_peered":"2024-01-11T07:14:05.451722+0000","last_unstale":"2024-01-11T07:14:05.452125+0000","last_undegraded":"2024-01-11T07:14:05.452125+0000","last_fullsized":"2024-01-11T07:14:05.452125+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:19:43.540786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.456424+0000","last_change":"2024-01-11T07:14:04.263308+0000","last_active":"2024-01-11T07:14:05.456424+0000","last_peered":"2024-01-11T07:14:05.456424+0000","last_clean":"2024-01-11T07:14:05.456424+0000","last_became_active":"2024-01-11T07:14:04.262976+0000","last_became_peered":"2024-01-11T07:14:04.262976+0000","last_unstale":"2024-01-11T07:14:05.456424+0000","last_undegraded":"2024-01-11T07:14:05.456424+0000","last_fullsized":"2024-01-11T07:14:05.456424+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:26:20.746054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.470016+0000","last_change":"2024-01-11T07:14:05.468794+0000","last_active":"2024-01-11T07:14:05.470016+0000","last_peered":"2024-01-11T07:14:05.470016+0000","last_clean":"2024-01-11T07:14:05.470016+0000","last_became_active":"2024-01-11T07:14:05.468138+0000","last_became_peered":"2024-01-11T07:14:05.468138+0000","last_unstale":"2024-01-11T07:14:05.470016+0000","last_undegraded":"2024-01-11T07:14:05.470016+0000","last_fullsized":"2024-01-11T07:14:05.470016+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:21:37.506093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.445119+0000","last_change":"2024-01-11T07:14:04.253958+0000","last_active":"2024-01-11T07:14:05.445119+0000","last_peered":"2024-01-11T07:14:05.445119+0000","last_clean":"2024-01-11T07:14:05.445119+0000","last_became_active":"2024-01-11T07:14:04.253690+0000","last_became_peered":"2024-01-11T07:14:04.253690+0000","last_unstale":"2024-01-11T07:14:05.445119+0000","last_undegraded":"2024-01-11T07:14:05.445119+0000","last_fullsized":"2024-01-11T07:14:05.445119+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:09:50.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460786+0000","last_change":"2024-01-11T07:14:05.460431+0000","last_active":"2024-01-11T07:14:05.460786+0000","last_peered":"2024-01-11T07:14:05.460786+0000","last_clean":"2024-01-11T07:14:05.460786+0000","last_became_active":"2024-01-11T07:14:05.459687+0000","last_became_peered":"2024-01-11T07:14:05.459687+0000","last_unstale":"2024-01-11T07:14:05.460786+0000","last_undegraded":"2024-01-11T07:14:05.460786+0000","last_fullsized":"2024-01-11T07:14:05.460786+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:26:21.747811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.471382+0000","last_change":"2024-01-11T07:14:04.261461+0000","last_active":"2024-01-11T07:14:05.471382+0000","last_peered":"2024-01-11T07:14:05.471382+0000","last_clean":"2024-01-11T07:14:05.471382+0000","last_became_active":"2024-01-11T07:14:04.260965+0000","last_became_peered":"2024-01-11T07:14:04.260965+0000","last_unstale":"2024-01-11T07:14:05.471382+0000","last_undegraded":"2024-01-11T07:14:05.471382+0000","last_fullsized":"2024-01-11T07:14:05.471382+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:55:43.550103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.454576+0000","last_change":"2024-01-11T07:14:05.454428+0000","last_active":"2024-01-11T07:14:05.454576+0000","last_peered":"2024-01-11T07:14:05.454576+0000","last_clean":"2024-01-11T07:14:05.454576+0000","last_became_active":"2024-01-11T07:14:05.454124+0000","last_became_peered":"2024-01-11T07:14:05.454124+0000","last_unstale":"2024-01-11T07:14:05.454576+0000","last_undegraded":"2024-01-11T07:14:05.454576+0000","last_fullsized":"2024-01-11T07:14:05.454576+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:52.240347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.453890+0000","last_change":"2024-01-11T07:14:04.256189+0000","last_active":"2024-01-11T07:14:05.453890+0000","last_peered":"2024-01-11T07:14:05.453890+0000","last_clean":"2024-01-11T07:14:05.453890+0000","last_became_active":"2024-01-11T07:14:04.255491+0000","last_became_peered":"2024-01-11T07:14:04.255491+0000","last_unstale":"2024-01-11T07:14:05.453890+0000","last_undegraded":"2024-01-11T07:14:05.453890+0000","last_fullsized":"2024-01-11T07:14:05.453890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:19:42.539029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.463006+0000","last_change":"2024-01-11T07:14:05.462698+0000","last_active":"2024-01-11T07:14:05.463006+0000","last_peered":"2024-01-11T07:14:05.463006+0000","last_clean":"2024-01-11T07:14:05.463006+0000","last_became_active":"2024-01-11T07:14:05.462307+0000","last_became_peered":"2024-01-11T07:14:05.462307+0000","last_unstale":"2024-01-11T07:14:05.463006+0000","last_undegraded":"2024-01-11T07:14:05.463006+0000","last_fullsized":"2024-01-11T07:14:05.463006+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:35:04.984288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.449358+0000","last_change":"2024-01-11T07:14:04.257505+0000","last_active":"2024-01-11T07:14:05.449358+0000","last_peered":"2024-01-11T07:14:05.449358+0000","last_clean":"2024-01-11T07:14:05.449358+0000","last_became_active":"2024-01-11T07:14:04.257270+0000","last_became_peered":"2024-01-11T07:14:04.257270+0000","last_unstale":"2024-01-11T07:14:05.449358+0000","last_undegraded":"2024-01-11T07:14:05.449358+0000","last_fullsized":"2024-01-11T07:14:05.449358+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:58:50.815963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.466074+0000","last_change":"2024-01-11T07:14:05.466008+0000","last_active":"2024-01-11T07:14:05.466074+0000","last_peered":"2024-01-11T07:14:05.466074+0000","last_clean":"2024-01-11T07:14:05.466074+0000","last_became_active":"2024-01-11T07:14:05.465757+0000","last_became_peered":"2024-01-11T07:14:05.465757+0000","last_unstale":"2024-01-11T07:14:05.466074+0000","last_undegraded":"2024-01-11T07:14:05.466074+0000","last_fullsized":"2024-01-11T07:14:05.466074+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:09:38.669408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.444982+0000","last_change":"2024-01-11T07:14:04.254693+0000","last_active":"2024-01-11T07:14:05.444982+0000","last_peered":"2024-01-11T07:14:05.444982+0000","last_clean":"2024-01-11T07:14:05.444982+0000","last_became_active":"2024-01-11T07:14:04.254446+0000","last_became_peered":"2024-01-11T07:14:04.254446+0000","last_unstale":"2024-01-11T07:14:05.444982+0000","last_undegraded":"2024-01-11T07:14:05.444982+0000","last_fullsized":"2024-01-11T07:14:05.444982+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:07:50.149313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.458107+0000","last_change":"2024-01-11T07:14:05.457413+0000","last_active":"2024-01-11T07:14:05.458107+0000","last_peered":"2024-01-11T07:14:05.458107+0000","last_clean":"2024-01-11T07:14:05.458107+0000","last_became_active":"2024-01-11T07:14:05.455352+0000","last_became_peered":"2024-01-11T07:14:05.455352+0000","last_unstale":"2024-01-11T07:14:05.458107+0000","last_undegraded":"2024-01-11T07:14:05.458107+0000","last_fullsized":"2024-01-11T07:14:05.458107+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:55:45.189248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.449388+0000","last_change":"2024-01-11T07:14:04.263335+0000","last_active":"2024-01-11T07:14:05.449388+0000","last_peered":"2024-01-11T07:14:05.449388+0000","last_clean":"2024-01-11T07:14:05.449388+0000","last_became_active":"2024-01-11T07:14:04.263109+0000","last_became_peered":"2024-01-11T07:14:04.263109+0000","last_unstale":"2024-01-11T07:14:05.449388+0000","last_undegraded":"2024-01-11T07:14:05.449388+0000","last_fullsized":"2024-01-11T07:14:05.449388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:38:00.241764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.469257+0000","last_change":"2024-01-11T07:14:05.469158+0000","last_active":"2024-01-11T07:14:05.469257+0000","last_peered":"2024-01-11T07:14:05.469257+0000","last_clean":"2024-01-11T07:14:05.469257+0000","last_became_active":"2024-01-11T07:14:05.468919+0000","last_became_peered":"2024-01-11T07:14:05.468919+0000","last_unstale":"2024-01-11T07:14:05.469257+0000","last_undegraded":"2024-01-11T07:14:05.469257+0000","last_fullsized":"2024-01-11T07:14:05.469257+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:48.912839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.453958+0000","last_change":"2024-01-11T07:14:04.259915+0000","last_active":"2024-01-11T07:14:05.453958+0000","last_peered":"2024-01-11T07:14:05.453958+0000","last_clean":"2024-01-11T07:14:05.453958+0000","last_became_active":"2024-01-11T07:14:04.259655+0000","last_became_peered":"2024-01-11T07:14:04.259655+0000","last_unstale":"2024-01-11T07:14:05.453958+0000","last_undegraded":"2024-01-11T07:14:05.453958+0000","last_fullsized":"2024-01-11T07:14:05.453958+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:35:03.982531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460312+0000","last_change":"2024-01-11T07:14:05.460244+0000","last_active":"2024-01-11T07:14:05.460312+0000","last_peered":"2024-01-11T07:14:05.460312+0000","last_clean":"2024-01-11T07:14:05.460312+0000","last_became_active":"2024-01-11T07:14:05.460028+0000","last_became_peered":"2024-01-11T07:14:05.460028+0000","last_unstale":"2024-01-11T07:14:05.460312+0000","last_undegraded":"2024-01-11T07:14:05.460312+0000","last_fullsized":"2024-01-11T07:14:05.460312+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:18:34.022111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.454724+0000","last_change":"2024-01-11T07:14:04.266474+0000","last_active":"2024-01-11T07:14:05.454724+0000","last_peered":"2024-01-11T07:14:05.454724+0000","last_clean":"2024-01-11T07:14:05.454724+0000","last_became_active":"2024-01-11T07:14:04.266011+0000","last_became_peered":"2024-01-11T07:14:04.266011+0000","last_unstale":"2024-01-11T07:14:05.454724+0000","last_undegraded":"2024-01-11T07:14:05.454724+0000","last_fullsized":"2024-01-11T07:14:05.454724+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:09:37.667652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.460653+0000","last_change":"2024-01-11T07:14:05.460570+0000","last_active":"2024-01-11T07:14:05.460653+0000","last_peered":"2024-01-11T07:14:05.460653+0000","last_clean":"2024-01-11T07:14:05.460653+0000","last_became_active":"2024-01-11T07:14:05.460292+0000","last_became_peered":"2024-01-11T07:14:05.460292+0000","last_unstale":"2024-01-11T07:14:05.460653+0000","last_undegraded":"2024-01-11T07:14:05.460653+0000","last_fullsized":"2024-01-11T07:14:05.460653+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:04:31.737037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.445863+0000","last_change":"2024-01-11T07:14:04.249279+0000","last_active":"2024-01-11T07:14:05.445863+0000","last_peered":"2024-01-11T07:14:05.445863+0000","last_clean":"2024-01-11T07:14:05.445863+0000","last_became_active":"2024-01-11T07:14:04.249074+0000","last_became_peered":"2024-01-11T07:14:04.249074+0000","last_unstale":"2024-01-11T07:14:05.445863+0000","last_undegraded":"2024-01-11T07:14:05.445863+0000","last_fullsized":"2024-01-11T07:14:05.445863+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:19:18.141986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.466549+0000","last_change":"2024-01-11T07:14:05.466513+0000","last_active":"2024-01-11T07:14:05.466549+0000","last_peered":"2024-01-11T07:14:05.466549+0000","last_clean":"2024-01-11T07:14:05.466549+0000","last_became_active":"2024-01-11T07:14:05.466064+0000","last_became_peered":"2024-01-11T07:14:05.466064+0000","last_unstale":"2024-01-11T07:14:05.466549+0000","last_undegraded":"2024-01-11T07:14:05.466549+0000","last_fullsized":"2024-01-11T07:14:05.466549+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:04.243945+0000","last_clean_scrub_stamp":"2024-01-11T07:14:04.243945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:18:34.022111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.470905+0000","last_change":"2024-01-11T07:14:04.261204+0000","last_active":"2024-01-11T07:14:05.470905+0000","last_peered":"2024-01-11T07:14:05.470905+0000","last_clean":"2024-01-11T07:14:05.470905+0000","last_became_active":"2024-01-11T07:14:04.260716+0000","last_became_peered":"2024-01-11T07:14:04.260716+0000","last_unstale":"2024-01-11T07:14:05.470905+0000","last_undegraded":"2024-01-11T07:14:05.470905+0000","last_fullsized":"2024-01-11T07:14:05.470905+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:14:03.242189+0000","last_clean_scrub_stamp":"2024-01-11T07:14:03.242189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:32:59.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":279,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-01-11T07:14:05.453104+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:14:05.453104+0000","last_peered":"2024-01-11T07:14:05.453104+0000","last_clean":"2024-01-11T07:14:05.453104+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:14:05.453104+0000","last_undegraded":"2024-01-11T07:14:05.453104+0000","last_fullsized":"2024-01-11T07:14:05.453104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574938,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193910,"statfs":{"total":200000000,"available":198564068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.34499999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.57499999999999996,"15min":0.53500000000000003},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69799999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.625}]},{"osd":1,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58599999999999997,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.34000000000000002,"15min":0.311},"max":{"1min":0.75900000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57799999999999996,"15min":0.55000000000000004},"min":{"1min":0.45200000000000001,"5min":0.377,"15min":0.36499999999999999},"max":{"1min":0.66300000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77500000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.435,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.751,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.54200000000000004},"min":{"1min":0.49399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74299999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.57299999999999995},"min":{"1min":0.499,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73199999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.48599999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70899999999999996,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51300000000000001,"15min":0.48799999999999999},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.308},"max":{"1min":0.755,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.497}]},{"osd":5,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54700000000000004,"15min":0.52400000000000002},"min":{"1min":0.33100000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.73699999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.34100000000000003},"max":{"1min":0.748,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60999999999999999,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60499999999999998,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.379,"15min":0.35999999999999999},"max":{"1min":0.81000000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194487,"statfs":{"total":200000000,"available":199155046,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77500000000000002,"5min":0.81100000000000005,"15min":0.81399999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":1.157},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.56899999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79500000000000004,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.33800000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.41799999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":0.74099999999999999,"15min":0.80500000000000005},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.47799999999999998,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.29599999999999999},"max":{"1min":0.84599999999999997,"5min":2.621,"15min":2.621},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.433,"5min":0.373,"15min":0.373},"max":{"1min":0.85799999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48899999999999999,"15min":0.53600000000000003},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68500000000000005,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.54800000000000004},"min":{"1min":0.39000000000000001,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.72299999999999998,"5min":0.72399999999999998,"15min":0.78800000000000003},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.503,"15min":0.51500000000000001},"min":{"1min":0.40799999999999997,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.745,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57299999999999995,"15min":0.61399999999999999},"min":{"1min":0.36299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.755,"5min":0.83399999999999996,"15min":0.89100000000000001},"last":0.754}]},{"osd":7,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68600000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.622},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.57499999999999996},"min":{"1min":0.39200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70899999999999996,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.67000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199156582,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.47199999999999998,"15min":0.504},"min":{"1min":0.26800000000000002,"5min":0.25800000000000001,"15min":0.216},"max":{"1min":0.753,"5min":8.5150000000000006,"15min":20.41},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48199999999999998,"15min":0.46100000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.72099999999999997,"5min":3.7029999999999998,"15min":3.7029999999999998},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.597,"5min":7.8099999999999996,"15min":15.611000000000001},"last":0.379},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.0710000000000002,"5min":6.1390000000000002,"15min":20.199999999999999},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53200000000000003,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.30199999999999999,"15min":0.29999999999999999},"max":{"1min":3.8460000000000001,"5min":5.6449999999999996,"15min":20.369},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.54000000000000004,"15min":0.47199999999999998},"min":{"1min":0.33400000000000002,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":4.7569999999999997,"5min":8.5790000000000006,"15min":8.5790000000000006},"last":0.39600000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.40999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.3300000000000001,"5min":7.6749999999999998,"15min":9.9570000000000007},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.51100000000000001,"5min":0.44,"15min":0.41899999999999998},"max":{"1min":1.2030000000000001,"5min":5.7830000000000004,"15min":12.449999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65600000000000003,"15min":0.624},"min":{"1min":0.48099999999999998,"5min":0.437,"15min":0.376},"max":{"1min":1.3540000000000001,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.497},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":1.177,"5min":7.7480000000000002,"15min":9.891},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66600000000000004,"15min":0.65700000000000003},"min":{"1min":0.55900000000000005,"5min":0.42899999999999999,"15min":0.41899999999999998},"max":{"1min":1.389,"5min":1.389,"15min":1.5569999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.504,"5min":0.47499999999999998,"15min":0.41699999999999998},"max":{"1min":2.9049999999999998,"5min":7.7770000000000001,"15min":9.9900000000000002},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:13:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.78200000000000003},"min":{"1min":0.48899999999999999,"5min":0.48199999999999998,"15min":0.442},"max":{"1min":1.1599999999999999,"5min":5.8449999999999998,"15min":20.324000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46800000000000003},"max":{"1min":1.268,"5min":2.548,"15min":2.548},"last":0.55300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574938,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1400,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193912,"statfs":{"total":200000000,"available":198566214,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:14:04 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.45700000000000002},"min":{"1min":0.26600000000000001,"5min":0.22700000000000001,"15min":0.224},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:14:04 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44,"15min":0.66400000000000003},"min":{"1min":0.28199999999999997,"5min":0.28000000000000003,"15min":0.23999999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.51600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29099999999999998,"15min":0.27300000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:14:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.46600000000000003,"15min":0.68400000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.435},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49399999999999999,"15min":0.55800000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.33500000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":2.855},"last":0.46500000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:14:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90900000000000003,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.68700000000000006,"15min":0.85899999999999999},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.40999999999999998},"max":{"1min":4.9279999999999999,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:14:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.70199999999999996},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":1.032,"5min":1.032,"15min":2.819},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.66700000000000004,"15min":0.64900000000000002},"min":{"1min":0.433,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85399999999999998,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:14:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.70499999999999996,"15min":0.90900000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.9850000000000003,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70499999999999996,"15min":0.76500000000000001},"min":{"1min":0.501,"5min":0.48099999999999998,"15min":0.41799999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":0.63}]},{"osd":7,"last update":"Thu Jan 11 07:14:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68899999999999995,"15min":0.66500000000000004},"min":{"1min":0.48299999999999998,"5min":0.45200000000000001,"15min":0.42499999999999999},"max":{"1min":0.96699999999999997,"5min":1.276,"15min":1.276},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.72999999999999998,"15min":0.90800000000000003},"min":{"1min":0.495,"5min":0.45900000000000002,"15min":0.44900000000000001},"max":{"1min":5.0720000000000001,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574938,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":824,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199155994,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40100000000000002,"15min":0.41499999999999998},"min":{"1min":0.27000000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48999999999999999,"5min":1.472,"15min":1.472},"last":0.251},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.53900000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.37}]},{"osd":1,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.42599999999999999,"15min":0.437},"min":{"1min":0.30499999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.61899999999999999,"5min":1.0309999999999999,"15min":1.2430000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.41299999999999998,"15min":0.39100000000000001},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.246},"max":{"1min":0.47899999999999998,"5min":0.59499999999999997,"15min":0.60899999999999999},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.64500000000000002,"5min":0.74399999999999999,"15min":0.76800000000000002},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.37,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59499999999999997,"5min":1.081,"15min":1.139},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63500000000000001,"15min":0.65900000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79000000000000004,"5min":1.0169999999999999,"15min":1.0660000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64400000000000002,"15min":0.625},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.40100000000000002},"max":{"1min":0.876,"5min":1.089,"15min":1.089},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65600000000000003,"15min":0.64700000000000002},"min":{"1min":0.48299999999999998,"5min":0.437,"15min":0.41999999999999998},"max":{"1min":0.86599999999999999,"5min":1.099,"15min":1.099},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90500000000000003,"5min":1.145,"15min":1.3100000000000001},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44900000000000001},"max":{"1min":0.88800000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.495,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89400000000000002,"5min":1.006,"15min":1.052},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:14:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.52700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84999999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81799999999999995,"5min":1.042,"15min":1.042},"last":0.61499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199156422,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39300000000000002,"15min":0.71299999999999997},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53100000000000003,"5min":0.58699999999999997,"15min":11.481},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.46100000000000002,"15min":0.47699999999999998},"min":{"1min":0.374,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.52200000000000002,"5min":0.65400000000000003,"15min":1.6200000000000001},"last":0.375}]},{"osd":1,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39500000000000002,"15min":0.86899999999999999},"min":{"1min":0.253,"5min":0.252,"15min":0.23899999999999999},"max":{"1min":0.51700000000000002,"5min":0.69999999999999996,"15min":17.516999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.438,"15min":0.42199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.20200000000000001},"max":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.67400000000000004},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42299999999999999,"15min":0.42499999999999999},"min":{"1min":0.31,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.76500000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.92700000000000005},"min":{"1min":0.35699999999999998,"5min":0.34599999999999997,"15min":0.30599999999999999},"max":{"1min":0.60499999999999998,"5min":0.79300000000000004,"15min":17.553000000000001},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65300000000000002,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40300000000000002},"max":{"1min":0.81200000000000006,"5min":0.93899999999999995,"15min":0.94599999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67400000000000004,"15min":1.157},"min":{"1min":0.56100000000000005,"5min":0.38500000000000001,"15min":0.36199999999999999},"max":{"1min":0.84399999999999997,"5min":0.95499999999999996,"15min":17.474},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.626,"15min":1.0960000000000001},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.98799999999999999,"15min":17.393000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65000000000000002,"15min":0.94699999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.82299999999999995,"5min":1.0149999999999999,"15min":11.394},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.66000000000000003,"15min":0.96099999999999997},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.34699999999999998},"max":{"1min":0.88,"5min":0.97299999999999998,"15min":11.436999999999999},"last":0.505},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.47199999999999998,"5min":0.46000000000000002,"15min":0.44400000000000001},"max":{"1min":0.81599999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:13:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.53100000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79900000000000004,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68000000000000005,"15min":1.135},"min":{"1min":0.51300000000000001,"5min":0.45700000000000002,"15min":0.42999999999999999},"max":{"1min":0.77300000000000002,"5min":1.1839999999999999,"15min":17.434000000000001},"last":0.497}]}]},{"osd":4,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199156512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:14:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64100000000000001,"15min":0.97099999999999997},"min":{"1min":0.31900000000000001,"5min":0.30299999999999999,"15min":0.248},"max":{"1min":0.78900000000000003,"5min":3.5579999999999998,"15min":8.484},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":1.0189999999999999},"min":{"1min":0.41899999999999998,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.753,"5min":3.5329999999999999,"15min":9.2599999999999998},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:14:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59199999999999997,"15min":0.81399999999999995},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.30199999999999999},"max":{"1min":0.76800000000000002,"5min":1.2250000000000001,"15min":8.5340000000000007},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61199999999999999,"15min":0.93100000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76500000000000001,"5min":4.2560000000000002,"15min":9.2279999999999998},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:14:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.66300000000000003,"15min":0.91300000000000003},"min":{"1min":0.434,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82399999999999995,"5min":5.9770000000000003,"15min":9.5640000000000001},"last":0.502},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.82099999999999995},"min":{"1min":0.36399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82099999999999995,"5min":0.93000000000000005,"15min":6.8639999999999999},"last":0.629}]},{"osd":3,"last update":"Thu Jan 11 07:14:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.95899999999999996},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73899999999999999,"5min":1.208,"15min":8.9779999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.65200000000000002,"15min":1.028},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80000000000000004,"5min":4.8680000000000003,"15min":8.7309999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:14:03 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46500000000000002,"15min":0.85499999999999998},"min":{"1min":0.33200000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.68200000000000005,"5min":0.77700000000000002,"15min":11.388999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.5,"15min":0.79300000000000004},"min":{"1min":0.33900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.68100000000000005,"5min":1.0880000000000001,"15min":8.6509999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:14:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52900000000000003,"15min":0.84299999999999997},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69999999999999996,"5min":0.91800000000000004,"15min":9.1449999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.35799999999999998},"max":{"1min":0.752,"5min":0.86099999999999999,"15min":1.119},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:14:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.82799999999999996},"min":{"1min":0.40699999999999997,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.71599999999999997,"5min":0.95999999999999996,"15min":8.6050000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61699999999999999,"15min":0.92100000000000004},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.73599999999999999,"5min":0.874,"15min":9.4640000000000004},"last":0.64300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":823,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194488,"statfs":{"total":200000000,"available":199156470,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.44400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73499999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.435}]},{"osd":1,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.56399999999999995},"min":{"1min":0.36699999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76700000000000002,"5min":0.80300000000000005,"15min":0.84099999999999997},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.56399999999999995},"min":{"1min":0.39200000000000002,"5min":0.376,"15min":0.35399999999999998},"max":{"1min":0.77100000000000002,"5min":0.79700000000000004,"15min":0.82799999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.42199999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80900000000000005,"15min":0.93999999999999995},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.44500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.624,"15min":0.62},"min":{"1min":0.42199999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.497,"15min":0.504},"min":{"1min":0.35299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63400000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52000000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":0.69099999999999995,"5min":0.73999999999999999,"15min":0.76100000000000001},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52100000000000002,"15min":0.52800000000000002},"min":{"1min":0.38,"5min":0.313,"15min":0.313},"max":{"1min":0.69599999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73499999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:13:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77700000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.85199999999999998,"15min":0.88200000000000001},"last":0.71099999999999997}]}]}],"pool_statfs":[]}} 2024-01-11T07:14:10.068 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:14:10.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:14:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.248+0000 7ff525b7a700 1 Processor -- start 2024-01-11T07:14:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.248+0000 7ff525b7a700 1 -- start start 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.249+0000 7ff525b7a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520159f50 0x7ff5200cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.249+0000 7ff525b7a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff52015e9c0 0x7ff5200cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.249+0000 7ff525b7a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52015e530 con 0x7ff520071130 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.249+0000 7ff525b7a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200cc970 con 0x7ff52015e9c0 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.249+0000 7ff525b7a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200cc690 con 0x7ff520159f50 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.249+0000 7ff51effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff52015e9c0 0x7ff5200cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.249+0000 7ff51effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff52015e9c0 0x7ff5200cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56544/0 (socket says 172.21.15.7:56544) 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.249+0000 7ff51effd700 1 -- 172.21.15.7:0/861074890 learned_addr learned my addr 172.21.15.7:0/861074890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51f7fe700 1 --2- 172.21.15.7:0/861074890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520159f50 0x7ff5200cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51effd700 1 -- 172.21.15.7:0/861074890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520159f50 msgr2=0x7ff5200cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:10.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51effd700 1 --2- 172.21.15.7:0/861074890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520159f50 0x7ff5200cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51effd700 1 -- 172.21.15.7:0/861074890 >> v1:172.21.15.113:6789/0 conn(0x7ff520071130 legacy=0x7ff52015b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51effd700 1 -- 172.21.15.7:0/861074890 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52015dee0 con 0x7ff52015e9c0 2024-01-11T07:14:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51effd700 1 --2- 172.21.15.7:0/861074890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52015e9c0 0x7ff5200cd630 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff5140099b0 tx=0x7ff514009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae88d3c87808f4e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51e7fc700 1 -- 172.21.15.7:0/861074890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400b860 con 0x7ff52015e9c0 2024-01-11T07:14:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51e7fc700 1 -- 172.21.15.7:0/861074890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51400b9c0 con 0x7ff52015e9c0 2024-01-11T07:14:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.250+0000 7ff51e7fc700 1 -- 172.21.15.7:0/861074890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400bd60 con 0x7ff52015e9c0 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.251+0000 7ff525b7a700 1 -- 172.21.15.7:0/861074890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52015e9c0 msgr2=0x7ff5200cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.251+0000 7ff525b7a700 1 --2- 172.21.15.7:0/861074890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52015e9c0 0x7ff5200cd630 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff5140099b0 tx=0x7ff514009250 comp rx=0 tx=0).stop 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.251+0000 7ff525b7a700 1 -- 172.21.15.7:0/861074890 shutdown_connections 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.251+0000 7ff525b7a700 1 --2- 172.21.15.7:0/861074890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520159f50 0x7ff5200cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.251+0000 7ff525b7a700 1 --2- 172.21.15.7:0/861074890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52015e9c0 0x7ff5200cd630 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.251+0000 7ff525b7a700 1 -- 172.21.15.7:0/861074890 >> 172.21.15.7:0/861074890 conn(0x7ff52006e6b0 msgr2=0x7ff520070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.251+0000 7ff525b7a700 1 -- 172.21.15.7:0/861074890 shutdown_connections 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.251+0000 7ff525b7a700 1 -- 172.21.15.7:0/861074890 wait complete. 2024-01-11T07:14:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.252+0000 7ff525b7a700 1 Processor -- start 2024-01-11T07:14:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.252+0000 7ff525b7a700 1 -- start start 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff525b7a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520071130 0x7ff52016ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff525b7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff520159f50 0x7ff520156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff525b7a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff52015e9c0 0x7ff5200c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff525b7a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520156850 con 0x7ff520159f50 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff525b7a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200c99e0 con 0x7ff52015e9c0 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff525b7a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff520156a60 con 0x7ff520071130 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff51effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff520159f50 0x7ff520156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff51effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff520159f50 0x7ff520156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56558/0 (socket says 172.21.15.7:56558) 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff51effd700 1 -- 172.21.15.7:0/2536805590 learned_addr learned my addr 172.21.15.7:0/2536805590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff51f7fe700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520071130 0x7ff52016ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.253+0000 7ff51ffff700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff52015e9c0 0x7ff5200c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff51effd700 1 -- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520071130 msgr2=0x7ff52016ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff51effd700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520071130 0x7ff52016ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff51effd700 1 -- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff52015e9c0 msgr2=0x7ff5200c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff51effd700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff52015e9c0 0x7ff5200c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff51effd700 1 -- 172.21.15.7:0/2536805590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5140094f0 con 0x7ff520159f50 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff51effd700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff520159f50 0x7ff520156310 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff520067f20 tx=0x7ff514017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff5077fe700 1 -- 172.21.15.7:0/2536805590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514011e40 con 0x7ff520159f50 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff5077fe700 1 -- 172.21.15.7:0/2536805590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51401a420 con 0x7ff520159f50 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff520156d40 con 0x7ff520159f50 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff5077fe700 1 -- 172.21.15.7:0/2536805590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51401a7e0 con 0x7ff520159f50 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.254+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5200c7d80 con 0x7ff520159f50 2024-01-11T07:14:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.255+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520089350 con 0x7ff520159f50 2024-01-11T07:14:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.256+0000 7ff5077fe700 1 -- 172.21.15.7:0/2536805590 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff51401ab50 con 0x7ff520159f50 2024-01-11T07:14:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.257+0000 7ff5077fe700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff50c066f30 0x7ff50c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.257+0000 7ff5077fe700 1 -- 172.21.15.7:0/2536805590 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff514086660 con 0x7ff520159f50 2024-01-11T07:14:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.257+0000 7ff51f7fe700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff50c066f30 0x7ff50c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.257+0000 7ff51f7fe700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff50c066f30 0x7ff50c0693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff510006fd0 tx=0x7ff510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:10.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.261+0000 7ff5077fe700 1 -- 172.21.15.7:0/2536805590 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5140538f0 con 0x7ff520159f50 2024-01-11T07:14:10.413 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-11T07:14:10.413+0000 7efddfe86700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:14:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.550+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff5200890f0 con 0x7ff520159f50 2024-01-11T07:14:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.554+0000 7ff5077fe700 1 -- 172.21.15.7:0/2536805590 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff514056ed0 con 0x7ff520159f50 2024-01-11T07:14:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff50c066f30 msgr2=0x7ff50c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff50c066f30 0x7ff50c0693e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff510006fd0 tx=0x7ff510008040 comp rx=0 tx=0).stop 2024-01-11T07:14:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff520159f50 msgr2=0x7ff520156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff520159f50 0x7ff520156310 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff520067f20 tx=0x7ff514017000 comp rx=0 tx=0).stop 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 shutdown_connections 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff50c066f30 0x7ff50c0693e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff52015e9c0 0x7ff5200c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff520071130 0x7ff52016ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 --2- 172.21.15.7:0/2536805590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff520159f50 0x7ff520156310 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 >> 172.21.15.7:0/2536805590 conn(0x7ff52006e6b0 msgr2=0x7ff520153a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 shutdown_connections 2024-01-11T07:14:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.556+0000 7ff525b7a700 1 -- 172.21.15.7:0/2536805590 wait complete. 2024-01-11T07:14:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:14:10.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:14:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.702+0000 7f559482f700 1 Processor -- start 2024-01-11T07:14:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.702+0000 7f559482f700 1 -- start start 2024-01-11T07:14:10.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f559482f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f55900cafb0 0x7f55900ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f559482f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5590159300 con 0x7f55900b0b40 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f559482f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55900cf260 con 0x7f5590158280 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f559482f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55900cf540 con 0x7f55900cafb0 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f558e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f55900cafb0 0x7f55900ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f558ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f55900b0b40 0x7f5590161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37064/0 (socket says 172.21.15.7:37064) 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f558ed9d700 1 -- 172.21.15.7:0/4124943374 learned_addr learned my addr 172.21.15.7:0/4124943374 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f558e59c700 1 -- 172.21.15.7:0/4124943374 >> v1:172.21.15.113:6789/0 conn(0x7f5590158280 legacy=0x7f55900b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f558e59c700 1 -- 172.21.15.7:0/4124943374 >> v1:172.21.15.7:6789/0 conn(0x7f55900b0b40 legacy=0x7f5590161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f558e59c700 1 -- 172.21.15.7:0/4124943374 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55900cf820 con 0x7f55900cafb0 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.703+0000 7f558d59a700 1 -- 172.21.15.7:0/4124943374 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365171435 0 0) 0x7f5590159300 con 0x7f55900b0b40 2024-01-11T07:14:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.704+0000 7f558e59c700 1 --2- 172.21.15.7:0/4124943374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55900cafb0 0x7f55900ce8f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f55800098e0 tx=0x7f55800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6d9acb99e73adc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:10.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.704+0000 7f558d59a700 1 -- 172.21.15.7:0/4124943374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580009e80 con 0x7f55900cafb0 2024-01-11T07:14:10.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.704+0000 7f558d59a700 1 -- 172.21.15.7:0/4124943374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558000b860 con 0x7f55900cafb0 2024-01-11T07:14:10.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.704+0000 7f558d59a700 1 -- 172.21.15.7:0/4124943374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558000bbe0 con 0x7f55900cafb0 2024-01-11T07:14:10.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.704+0000 7f559482f700 1 -- 172.21.15.7:0/4124943374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55900cafb0 msgr2=0x7f55900ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.704+0000 7f559482f700 1 --2- 172.21.15.7:0/4124943374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55900cafb0 0x7f55900ce8f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f55800098e0 tx=0x7f55800091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.704+0000 7f559482f700 1 -- 172.21.15.7:0/4124943374 shutdown_connections 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.704+0000 7f559482f700 1 --2- 172.21.15.7:0/4124943374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55900cafb0 0x7f55900ce8f0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.705+0000 7f559482f700 1 -- 172.21.15.7:0/4124943374 >> 172.21.15.7:0/4124943374 conn(0x7f5590071100 msgr2=0x7f559006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.705+0000 7f559482f700 1 -- 172.21.15.7:0/4124943374 shutdown_connections 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.705+0000 7f559482f700 1 -- 172.21.15.7:0/4124943374 wait complete. 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.706+0000 7f559482f700 1 Processor -- start 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.706+0000 7f559482f700 1 -- start start 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f559482f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55900b0b40 0x7f55900cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f559482f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55900cafb0 0x7f55900b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f559482f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5590158280 0x7f5590067610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f559482f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55900b4210 con 0x7f5590158280 2024-01-11T07:14:10.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f559482f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590147a50 con 0x7f55900b0b40 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f559482f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5590154db0 con 0x7f55900cafb0 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5590158280 0x7f5590067610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5590158280 0x7f5590067610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56566/0 (socket says 172.21.15.7:56566) 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 -- 172.21.15.7:0/2271720499 learned_addr learned my addr 172.21.15.7:0/2271720499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558dd9b700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55900cafb0 0x7f55900b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558e59c700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55900b0b40 0x7f55900cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 -- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55900cafb0 msgr2=0x7f55900b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55900cafb0 0x7f55900b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 -- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55900b0b40 msgr2=0x7f55900cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55900b0b40 0x7f55900cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 -- 172.21.15.7:0/2271720499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5580009420 con 0x7f5590158280 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558e59c700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55900b0b40 0x7f55900cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:10.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f558ed9d700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5590158280 0x7f5590067610 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f558800cc90 tx=0x7f5588001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f557e7fc700 1 -- 172.21.15.7:0/2271720499 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558800cea0 con 0x7f5590158280 2024-01-11T07:14:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f557e7fc700 1 -- 172.21.15.7:0/2271720499 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558800ab70 con 0x7f5590158280 2024-01-11T07:14:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55900b3c50 con 0x7f5590158280 2024-01-11T07:14:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.707+0000 7f557e7fc700 1 -- 172.21.15.7:0/2271720499 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55880070f0 con 0x7f5590158280 2024-01-11T07:14:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.708+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55900b27f0 con 0x7f5590158280 2024-01-11T07:14:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.708+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55900b2e40 con 0x7f5590158280 2024-01-11T07:14:10.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.709+0000 7f557e7fc700 1 -- 172.21.15.7:0/2271720499 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f55880074e0 con 0x7f5590158280 2024-01-11T07:14:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.709+0000 7f557e7fc700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5570066f30 0x7f55700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.709+0000 7f557e7fc700 1 -- 172.21.15.7:0/2271720499 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5588084920 con 0x7f5590158280 2024-01-11T07:14:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.710+0000 7f558e59c700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5570066f30 0x7f55700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:10.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.710+0000 7f558e59c700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5570066f30 0x7f55700693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f55800093f0 tx=0x7f5580017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:10.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.714+0000 7f557e7fc700 1 -- 172.21.15.7:0/2271720499 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5588051c10 con 0x7f5590158280 2024-01-11T07:14:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.966+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f559004c3c0 con 0x7f5590158280 2024-01-11T07:14:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.967+0000 7f557e7fc700 1 -- 172.21.15.7:0/2271720499 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f5588055230 con 0x7f5590158280 2024-01-11T07:14:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:10.967 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1458931008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1458931008},{"type":"v1","addr":"172.21.15.113:6836","nonce":1458931008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3377988669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3377988669},{"type":"v1","addr":"172.21.15.113:6837","nonce":3377988669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1420951490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1420951490},{"type":"v1","addr":"172.21.15.7:6833","nonce":1420951490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:14:04.679133+0000","modified":"2024-01-11T07:14:08.473548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7425":{"gid":7425,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/187487340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":187487340},{"type":"v1","addr":"172.21.15.7:6835","nonce":187487340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-11T07:14:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5570066f30 msgr2=0x7f55700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5570066f30 0x7f55700693e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f55800093f0 tx=0x7f5580017000 comp rx=0 tx=0).stop 2024-01-11T07:14:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5590158280 msgr2=0x7f5590067610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5590158280 0x7f5590067610 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f558800cc90 tx=0x7f5588001a80 comp rx=0 tx=0).stop 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 shutdown_connections 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5570066f30 0x7f55700693e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5590158280 0x7f5590067610 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55900b0b40 0x7f55900cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 --2- 172.21.15.7:0/2271720499 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55900cafb0 0x7f55900b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 >> 172.21.15.7:0/2271720499 conn(0x7f5590071100 msgr2=0x7f55900ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 shutdown_connections 2024-01-11T07:14:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:10.969+0000 7f559482f700 1 -- 172.21.15.7:0/2271720499 wait complete. 2024-01-11T07:14:10.971 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2024-01-11T07:14:11.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:14:04.679133+0000', 'modified': '2024-01-11T07:14:08.473548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7425': {'gid': 7425, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/187487340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 187487340}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 187487340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:14:11.386 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:14:11.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-11T07:14:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.529+0000 7f25249fe700 1 Processor -- start 2024-01-11T07:14:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.529+0000 7f25249fe700 1 -- start start 2024-01-11T07:14:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f25249fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25200cafb0 0x7f25200ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f25249fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2520158280 0x7f25200b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f25249fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25200b0970 0x7f2520159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f25249fe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2520159300 con 0x7f2520158280 2024-01-11T07:14:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f25249fe700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25200b0dc0 con 0x7f25200cafb0 2024-01-11T07:14:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f25249fe700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25200cf260 con 0x7f25200b0970 2024-01-11T07:14:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f251dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2520158280 0x7f25200b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f251dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2520158280 0x7f25200b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56574/0 (socket says 172.21.15.7:56574) 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.530+0000 7f251dd9b700 1 -- 172.21.15.7:0/3865586536 learned_addr learned my addr 172.21.15.7:0/3865586536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251ed9d700 1 --2- 172.21.15.7:0/3865586536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25200b0970 0x7f2520159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251e59c700 1 --2- 172.21.15.7:0/3865586536 >> v2:172.21.15.113:3300/0 conn(0x7f25200cafb0 0x7f25200ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251dd9b700 1 -- 172.21.15.7:0/3865586536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25200b0970 msgr2=0x7f2520159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251dd9b700 1 --2- 172.21.15.7:0/3865586536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25200b0970 0x7f2520159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251dd9b700 1 -- 172.21.15.7:0/3865586536 >> v2:172.21.15.113:3300/0 conn(0x7f25200cafb0 msgr2=0x7f25200ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251dd9b700 1 --2- 172.21.15.7:0/3865586536 >> v2:172.21.15.113:3300/0 conn(0x7f25200cafb0 0x7f25200ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251dd9b700 1 -- 172.21.15.7:0/3865586536 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25200cf540 con 0x7f2520158280 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251ed9d700 1 --2- 172.21.15.7:0/3865586536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25200b0970 0x7f2520159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251e59c700 1 --2- 172.21.15.7:0/3865586536 >> v2:172.21.15.113:3300/0 conn(0x7f25200cafb0 0x7f25200ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251dd9b700 1 --2- 172.21.15.7:0/3865586536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520158280 0x7f25200b0430 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f25200cfb70 tx=0x7f2508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d704c395978cb27 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251d59a700 1 -- 172.21.15.7:0/3865586536 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250800b860 con 0x7f2520158280 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251d59a700 1 -- 172.21.15.7:0/3865586536 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250800b9c0 con 0x7f2520158280 2024-01-11T07:14:11.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.531+0000 7f251d59a700 1 -- 172.21.15.7:0/3865586536 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250800bd60 con 0x7f2520158280 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 -- 172.21.15.7:0/3865586536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520158280 msgr2=0x7f25200b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 --2- 172.21.15.7:0/3865586536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520158280 0x7f25200b0430 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f25200cfb70 tx=0x7f2508009250 comp rx=0 tx=0).stop 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 -- 172.21.15.7:0/3865586536 shutdown_connections 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 --2- 172.21.15.7:0/3865586536 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f25200b0970 0x7f2520159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 --2- 172.21.15.7:0/3865586536 >> v2:172.21.15.113:3300/0 conn(0x7f25200cafb0 0x7f25200ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 --2- 172.21.15.7:0/3865586536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2520158280 0x7f25200b0430 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 -- 172.21.15.7:0/3865586536 >> 172.21.15.7:0/3865586536 conn(0x7f2520071100 msgr2=0x7f252006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 -- 172.21.15.7:0/3865586536 shutdown_connections 2024-01-11T07:14:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.532+0000 7f25249fe700 1 -- 172.21.15.7:0/3865586536 wait complete. 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.533+0000 7f25249fe700 1 Processor -- start 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.533+0000 7f25249fe700 1 -- start start 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f25249fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25200b0970 0x7f25201445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f25249fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25200cafb0 0x7f2520159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f25249fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2520158280 0x7f25200a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f25249fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252015b410 con 0x7f25200cafb0 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f25249fe700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252015a800 con 0x7f25200b0970 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f25249fe700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252015aae0 con 0x7f2520158280 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25200cafb0 0x7f2520159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25200cafb0 0x7f2520159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56586/0 (socket says 172.21.15.7:56586) 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251dd9b700 1 -- 172.21.15.7:0/2114500706 learned_addr learned my addr 172.21.15.7:0/2114500706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251ed9d700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2520158280 0x7f25200a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251e59c700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25200b0970 0x7f25201445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251dd9b700 1 -- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2520158280 msgr2=0x7f25200a6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251dd9b700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2520158280 0x7f25200a6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251dd9b700 1 -- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25200b0970 msgr2=0x7f25201445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251dd9b700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25200b0970 0x7f25201445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251dd9b700 1 -- 172.21.15.7:0/2114500706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25080094f0 con 0x7f25200cafb0 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.534+0000 7f251e59c700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25200b0970 0x7f25201445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.535+0000 7f251dd9b700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25200cafb0 0x7f2520159d80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f252006a1d0 tx=0x7f2508017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.535+0000 7f25167fc700 1 -- 172.21.15.7:0/2114500706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508024440 con 0x7f25200cafb0 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.535+0000 7f25167fc700 1 -- 172.21.15.7:0/2114500706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25080245a0 con 0x7f25200cafb0 2024-01-11T07:14:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.535+0000 7f25167fc700 1 -- 172.21.15.7:0/2114500706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508024960 con 0x7f25200cafb0 2024-01-11T07:14:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.535+0000 7f25249fe700 1 -- 172.21.15.7:0/2114500706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252015adc0 con 0x7f25200cafb0 2024-01-11T07:14:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.535+0000 7f25249fe700 1 -- 172.21.15.7:0/2114500706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25200a66d0 con 0x7f25200cafb0 2024-01-11T07:14:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.536+0000 7f25167fc700 1 -- 172.21.15.7:0/2114500706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f250801a420 con 0x7f25200cafb0 2024-01-11T07:14:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.537+0000 7f25167fc700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2500066f30 0x7f25000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.537+0000 7f25167fc700 1 -- 172.21.15.7:0/2114500706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2508086290 con 0x7f25200cafb0 2024-01-11T07:14:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.537+0000 7f251e59c700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2500066f30 0x7f25000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.538+0000 7f251e59c700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2500066f30 0x7f25000693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2510006fd0 tx=0x7f2510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.539+0000 7f252dcfeb80 1 Processor -- start 2024-01-11T07:14:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.539+0000 7f252dcfeb80 1 -- start start 2024-01-11T07:14:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21bfa0 0x56361c21c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 0x56361c217c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c238360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c14a3e0 con 0x56361c218170 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c147be0 con 0x56361c21c8b0 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56361c1439d0 con 0x56361c21bfa0 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c238360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c238360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56594/0 (socket says 172.21.15.7:56594) 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 -- 172.21.15.7:0/2901765053 learned_addr learned my addr 172.21.15.7:0/2901765053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251e59c700 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21bfa0 0x56361c21c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251dd9b700 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 0x56361c217c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 -- 172.21.15.7:0/2901765053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21bfa0 msgr2=0x56361c21c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21bfa0 0x56361c21c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 -- 172.21.15.7:0/2901765053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 msgr2=0x56361c217c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 0x56361c217c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 -- 172.21.15.7:0/2901765053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361bfd7360 con 0x56361c218170 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.540+0000 7f251ed9d700 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c238360 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2518007fc0 tx=0x7f2518007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b1dceba979b9d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f24ff7fe700 1 -- 172.21.15.7:0/2901765053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518013450 con 0x56361c218170 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f24ff7fe700 1 -- 172.21.15.7:0/2901765053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2518017070 con 0x56361c218170 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f24ff7fe700 1 -- 172.21.15.7:0/2901765053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518013800 con 0x56361c218170 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 -- 172.21.15.7:0/2901765053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 msgr2=0x56361c238360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c238360 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2518007fc0 tx=0x7f2518007a40 comp rx=0 tx=0).stop 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 -- 172.21.15.7:0/2901765053 shutdown_connections 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c238360 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21bfa0 0x56361c21c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/2901765053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 0x56361c217c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 -- 172.21.15.7:0/2901765053 >> 172.21.15.7:0/2901765053 conn(0x56361c1f2440 msgr2=0x56361c23ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 -- 172.21.15.7:0/2901765053 shutdown_connections 2024-01-11T07:14:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.541+0000 7f252dcfeb80 1 -- 172.21.15.7:0/2901765053 wait complete. 2024-01-11T07:14:11.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.542+0000 7f252dcfeb80 1 -- start start 2024-01-11T07:14:11.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.542+0000 7f252dcfeb80 1 Processor -- start 2024-01-11T07:14:11.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c2171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 0x56361c2344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21c8b0 0x56361c21f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c0f01b0 con 0x56361c218170 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c0efe50 con 0x56361c21bfa0 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56361c03ce40 con 0x56361c21c8b0 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f252dcfeb80 10 client.0 _ll_register_callbacks cb 0x56361c03f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f251e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c2171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f251e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c2171d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56604/0 (socket says 172.21.15.7:56604) 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f251e59c700 1 -- 172.21.15.7:0/1062347550 learned_addr learned my addr 172.21.15.7:0/1062347550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f251ed9d700 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21c8b0 0x56361c21f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.543+0000 7f251dd9b700 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 0x56361c2344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f251e59c700 1 -- 172.21.15.7:0/1062347550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21c8b0 msgr2=0x56361c21f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f251e59c700 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21c8b0 0x56361c21f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f251e59c700 1 -- 172.21.15.7:0/1062347550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 msgr2=0x56361c2344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f251e59c700 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 0x56361c2344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f251e59c700 1 -- 172.21.15.7:0/1062347550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2518007d20 con 0x56361c218170 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f251e59c700 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c2171d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2510012850 tx=0x7f2510016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7c89e414929b29 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f24fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f24fcff9700 1 -- 172.21.15.7:0/1062347550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251000b050 con 0x56361c218170 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f24fcff9700 1 -- 172.21.15.7:0/1062347550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f251001f630 con 0x56361c218170 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1062347550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56361c1c4eb0 con 0x56361c218170 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f24fcff9700 1 -- 172.21.15.7:0/1062347550 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f251001fa00 con 0x56361c218170 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f252dcfeb80 10 client.7554 fetch_fsmap learned FSMap version 74 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1062347550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x56361bf83b10 con 0x56361c218170 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f24fcff9700 1 -- 172.21.15.7:0/1062347550 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56361c1c4eb0 con 0x56361c218170 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.544+0000 7f24fcff9700 1 -- 172.21.15.7:0/1062347550 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 74) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f251001fb60 con 0x56361c218170 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f252dcfeb80 10 client.7554 fetch_fsmap finished waiting for FSMap version 74 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f252dcfeb80 10 client.7554 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f252dcfeb80 20 client.7554 populate_metadata read hostname 'smithi007' 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] conn(0x56361c231e40 0x56361c232210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f252dcfeb80 4 client.7554 mds_command: new command op to 7425 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1062347550 --> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56361c11a880 con 0x56361c231e40 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f251dd9b700 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] conn(0x56361c231e40 0x56361c232210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f251dd9b700 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] conn(0x56361c231e40 0x56361c232210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f077f30be40f2e44 server_cookie=6344118e86aaacc2 in_seq=0 out_seq=0 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.545+0000 7f24fcff9700 10 client.7554 ms_handle_connect on v2:172.21.15.7:6834/187487340 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f24fcff9700 1 -- 172.21.15.7:0/1062347550 <== mds.0 v2:172.21.15.7:6834/187487340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56361c11a880 con 0x56361c231e40 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f24fcff9700 10 client.7554 handle_command_reply: tid=0 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 client.7554 shutdown 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1062347550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 msgr2=0x56361c2171d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c2171d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2510012850 tx=0x7f2510016040 comp rx=0 tx=0).stop 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1062347550 shutdown_connections 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] conn(0x56361c231e40 0x56361c232210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c21c8b0 0x56361c21f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c218170 0x56361c2171d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f24fcff9700 0 client.7554 ms_handle_reset on v2:172.21.15.7:6834/187487340 2024-01-11T07:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1062347550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 0x56361c2344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1062347550 >> 172.21.15.7:0/1062347550 conn(0x56361c1f2440 msgr2=0x56361c1f2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1062347550 shutdown_connections 2024-01-11T07:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1062347550 wait complete. 2024-01-11T07:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.546+0000 7f252dcfeb80 20 client.7554 trim_cache size 0 max 16384 2024-01-11T07:14:11.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.564+0000 7f252dcfeb80 1 Processor -- start 2024-01-11T07:14:11.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.564+0000 7f252dcfeb80 1 -- start start 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c233ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21bfa0 0x56361c234420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 0x56361c2171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c223a00 con 0x56361c21bfa0 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c1f65a0 con 0x56361c21c8b0 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56361c23daa0 con 0x56361c218170 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21bfa0 0x56361c234420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21bfa0 0x56361c234420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56606/0 (socket says 172.21.15.7:56606) 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 -- 172.21.15.7:0/1098688974 learned_addr learned my addr 172.21.15.7:0/1098688974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251e59c700 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c233ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251ed9d700 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 0x56361c2171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 -- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 msgr2=0x56361c233ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c233ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 -- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 msgr2=0x56361c2171d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 0x56361c2171d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 -- 172.21.15.7:0/1098688974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361c112290 con 0x56361c21bfa0 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251e59c700 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c233ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.565+0000 7f251dd9b700 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21bfa0 0x56361c234420 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2508057150 tx=0x7f250806b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817529886020e9a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f24fffff700 1 -- 172.21.15.7:0/1098688974 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080578c0 con 0x56361c21bfa0 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f24fffff700 1 -- 172.21.15.7:0/1098688974 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25080571c0 con 0x56361c21bfa0 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f24fffff700 1 -- 172.21.15.7:0/1098688974 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250805a410 con 0x56361c21bfa0 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1098688974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21bfa0 msgr2=0x56361c234420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21bfa0 0x56361c234420 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2508057150 tx=0x7f250806b040 comp rx=0 tx=0).stop 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1098688974 shutdown_connections 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21c8b0 0x56361c2171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c233ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/1098688974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21bfa0 0x56361c234420 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1098688974 >> 172.21.15.7:0/1098688974 conn(0x56361c1f2440 msgr2=0x56361c1e9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1098688974 shutdown_connections 2024-01-11T07:14:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.566+0000 7f252dcfeb80 1 -- 172.21.15.7:0/1098688974 wait complete. 2024-01-11T07:14:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.567+0000 7f252dcfeb80 1 -- start start 2024-01-11T07:14:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.567+0000 7f252dcfeb80 1 Processor -- start 2024-01-11T07:14:11.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c288720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 0x56361c21fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f252dcfeb80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21c8b0 0x56361c253950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c220860 con 0x56361c21c8b0 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c216210 con 0x56361c21bfa0 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f252dcfeb80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x56361c252f00 con 0x56361c218170 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f252dcfeb80 10 client.0 _ll_register_callbacks cb 0x7f251000dc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21c8b0 0x56361c253950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21c8b0 0x56361c253950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56616/0 (socket says 172.21.15.7:56616) 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251ed9d700 1 -- 172.21.15.7:0/3463732164 learned_addr learned my addr 172.21.15.7:0/3463732164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251e59c700 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c288720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251dd9b700 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 0x56361c21fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251ed9d700 1 -- 172.21.15.7:0/3463732164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 msgr2=0x56361c288720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251ed9d700 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c288720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251ed9d700 1 -- 172.21.15.7:0/3463732164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 msgr2=0x56361c21fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251ed9d700 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 0x56361c21fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.568+0000 7f251ed9d700 1 -- 172.21.15.7:0/3463732164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25200a66d0 con 0x56361c21c8b0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f251ed9d700 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21c8b0 0x56361c253950 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2518007f90 tx=0x7f251801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b73475d6cd7d756 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f24fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f24fcff9700 1 -- 172.21.15.7:0/3463732164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518013660 con 0x56361c21c8b0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f24fcff9700 1 -- 172.21.15.7:0/3463732164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f25180137c0 con 0x56361c21c8b0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 1 -- 172.21.15.7:0/3463732164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56361c1e8490 con 0x56361c21c8b0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f24fcff9700 1 -- 172.21.15.7:0/3463732164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2518031cd0 con 0x56361c21c8b0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 10 client.7560 fetch_fsmap learned FSMap version 74 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 1 -- 172.21.15.7:0/3463732164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x56361bfdbc80 con 0x56361c21c8b0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f24fcff9700 1 -- 172.21.15.7:0/3463732164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56361c1e8490 con 0x56361c21c8b0 2024-01-11T07:14:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f24fcff9700 1 -- 172.21.15.7:0/3463732164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 74) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f251802e020 con 0x56361c21c8b0 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 10 client.7560 fetch_fsmap finished waiting for FSMap version 74 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 10 client.7560 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 20 client.7560 populate_metadata read hostname 'smithi007' 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] conn(0x56361c2171d0 0x56361c2175a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 4 client.7560 mds_command: new command op to 7425 tid=0[{"prefix": "status"}] 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.569+0000 7f252dcfeb80 1 -- 172.21.15.7:0/3463732164 --> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56361c0e6910 con 0x56361c2171d0 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f251e59c700 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] conn(0x56361c2171d0 0x56361c2175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f24fcff9700 10 client.7560 ms_handle_connect on v2:172.21.15.7:6834/187487340 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f251e59c700 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] conn(0x56361c2171d0 0x56361c2175a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=931cca2041c37157 server_cookie=da28ce6c7170a121 in_seq=0 out_seq=0 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f24fcff9700 1 -- 172.21.15.7:0/3463732164 <== mds.0 v2:172.21.15.7:6834/187487340 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56361c0e6910 con 0x56361c2171d0 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f24fcff9700 10 client.7560 handle_command_reply: tid=0 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 client.7560 shutdown 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 -- 172.21.15.7:0/3463732164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21c8b0 msgr2=0x56361c253950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21c8b0 0x56361c253950 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2518007f90 tx=0x7f251801d000 comp rx=0 tx=0).stop 2024-01-11T07:14:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 -- 172.21.15.7:0/3463732164 shutdown_connections 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.7:6834/187487340,v1:172.21.15.7:6835/187487340] conn(0x56361c2171d0 0x56361c2175a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56361c21c8b0 0x56361c253950 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56361c218170 0x56361c288720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f24fcff9700 0 client.7560 ms_handle_reset on v2:172.21.15.7:6834/187487340 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 --2- 172.21.15.7:0/3463732164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56361c21bfa0 0x56361c21fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.570+0000 7f252dcfeb80 1 -- 172.21.15.7:0/3463732164 >> 172.21.15.7:0/3463732164 conn(0x56361c1f2440 msgr2=0x56361c233df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f252dcfeb80 1 -- 172.21.15.7:0/3463732164 shutdown_connections 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f252dcfeb80 1 -- 172.21.15.7:0/3463732164 wait complete. 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f252dcfeb80 20 client.7560 trim_cache size 0 max 16384 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:14:11.576 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7425, 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1277271510000002, 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 74, 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 152, 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 152, 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 12.106319697 2024-01-11T07:14:11.577 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:14:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 -- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2500066f30 msgr2=0x7f25000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2500066f30 0x7f25000693e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2510006fd0 tx=0x7f2510008040 comp rx=0 tx=0).stop 2024-01-11T07:14:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 -- 172.21.15.7:0/2114500706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25200cafb0 msgr2=0x7f2520159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25200cafb0 0x7f2520159d80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f252006a1d0 tx=0x7f2508017000 comp rx=0 tx=0).stop 2024-01-11T07:14:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 -- 172.21.15.7:0/2114500706 shutdown_connections 2024-01-11T07:14:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2500066f30 0x7f25000693e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2520158280 0x7f25200a6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25200b0970 0x7f25201445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 --2- 172.21.15.7:0/2114500706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25200cafb0 0x7f2520159d80 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 -- 172.21.15.7:0/2114500706 >> 172.21.15.7:0/2114500706 conn(0x7f2520071100 msgr2=0x7f2520156810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 -- 172.21.15.7:0/2114500706 shutdown_connections 2024-01-11T07:14:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.571+0000 7f25249fe700 1 -- 172.21.15.7:0/2114500706 wait complete. 2024-01-11T07:14:11.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:14:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.715+0000 7f2bd3b39700 1 Processor -- start 2024-01-11T07:14:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.716+0000 7f2bd3b39700 1 -- start start 2024-01-11T07:14:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd3b39700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bcc063230 0x7f2bcc15e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd3b39700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bcc15aa40 0x7f2bcc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd3b39700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bcc15df80 con 0x7f2bcc063600 2024-01-11T07:14:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd3b39700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bcc15e950 con 0x7f2bcc15aa40 2024-01-11T07:14:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd3b39700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc0aab70 con 0x7f2bcc063230 2024-01-11T07:14:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd10d4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2bcc063600 0x7f2bcc15a330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37074/0 (socket says 172.21.15.7:37074) 2024-01-11T07:14:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd10d4700 1 -- 172.21.15.7:0/650607894 learned_addr learned my addr 172.21.15.7:0/650607894 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bcc063230 0x7f2bcc15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd20d6700 1 --2- 172.21.15.7:0/650607894 >> v2:172.21.15.113:3300/0 conn(0x7f2bcc15aa40 0x7f2bcc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283712906 0 0) 0x7f2bcc15df80 con 0x7f2bcc063600 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb8003570 con 0x7f2bcc063600 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4286590052 0 0) 0x7f2bb8003570 con 0x7f2bcc063600 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bcc063230 msgr2=0x7f2bcc15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bcc063230 0x7f2bcc15e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 >> v2:172.21.15.113:3300/0 conn(0x7f2bcc15aa40 msgr2=0x7f2bcc06b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 --2- 172.21.15.7:0/650607894 >> v2:172.21.15.113:3300/0 conn(0x7f2bcc15aa40 0x7f2bcc06b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc15af10 con 0x7f2bcc063600 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f2bbc001470 con 0x7f2bcc063600 2024-01-11T07:14:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 >> v1:172.21.15.7:6789/0 conn(0x7f2bcc063600 legacy=0x7f2bcc15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.717+0000 7f2bd08d3700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8003a60 0x7f2bb8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd08d3700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bcc063230 0x7f2bb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd08d3700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb8004d50 0x7f2bb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc15e950 con 0x7f2bb8003a60 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc0aab70 con 0x7f2bb8004d50 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb800ba60 con 0x7f2bcc063230 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8003a60 0x7f2bb8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd10d4700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bcc063230 0x7f2bb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd20d6700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb8004d50 0x7f2bb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd18d5700 1 -- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bcc063230 msgr2=0x7f2bb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bcc063230 0x7f2bb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd18d5700 1 -- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb8004d50 msgr2=0x7f2bb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb8004d50 0x7f2bb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd18d5700 1 -- 172.21.15.7:0/650607894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8003570 con 0x7f2bb8003a60 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8003a60 0x7f2bb8003e70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009c00 tx=0x7f2bc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0a0666682d0bf6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800e6b0 con 0x7f2bb8003a60 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800e810 con 0x7f2bb8003a60 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.718+0000 7f2bd08d3700 1 -- 172.21.15.7:0/650607894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ebd0 con 0x7f2bb8003a60 2024-01-11T07:14:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.719+0000 7f2bd3b39700 1 -- 172.21.15.7:0/650607894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8003a60 msgr2=0x7f2bb8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.719+0000 7f2bd3b39700 1 --2- 172.21.15.7:0/650607894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8003a60 0x7f2bb8003e70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009c00 tx=0x7f2bc80091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.719+0000 7f2bd20d6700 1 -- 172.21.15.7:0/650607894 reap_dead start 2024-01-11T07:14:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.719+0000 7f2bd3b39700 1 -- 172.21.15.7:0/650607894 shutdown_connections 2024-01-11T07:14:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.719+0000 7f2bd3b39700 1 -- 172.21.15.7:0/650607894 >> 172.21.15.7:0/650607894 conn(0x7f2bcc0cd910 msgr2=0x7f2bcc0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.719+0000 7f2bd3b39700 1 -- 172.21.15.7:0/650607894 shutdown_connections 2024-01-11T07:14:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.719+0000 7f2bd3b39700 1 -- 172.21.15.7:0/650607894 wait complete. 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.720+0000 7f2bd3b39700 1 Processor -- start 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.720+0000 7f2bd3b39700 1 -- start start 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd3b39700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb8003a60 0x7f2bcc14a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd3b39700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc0cb4e0 0x7f2bcc0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd3b39700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bcc15b8e0 0x7f2bcc15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd3b39700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc15c2c0 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd3b39700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc14a9b0 con 0x7f2bcc15b8e0 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd3b39700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc16b0e0 con 0x7f2bb8003a60 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd10d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc0cb4e0 0x7f2bcc0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd10d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc0cb4e0 0x7f2bcc0afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56632/0 (socket says 172.21.15.7:56632) 2024-01-11T07:14:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd10d4700 1 -- 172.21.15.7:0/3415519958 learned_addr learned my addr 172.21.15.7:0/3415519958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb8003a60 0x7f2bcc14a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd20d6700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bcc15b8e0 0x7f2bcc15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd10d4700 1 -- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb8003a60 msgr2=0x7f2bcc14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd10d4700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb8003a60 0x7f2bcc14a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd10d4700 1 -- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bcc15b8e0 msgr2=0x7f2bcc15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd10d4700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bcc15b8e0 0x7f2bcc15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.721+0000 7f2bd10d4700 1 -- 172.21.15.7:0/3415519958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc0cbc80 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.722+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb8003a60 0x7f2bcc14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.722+0000 7f2bd10d4700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc0cb4e0 0x7f2bcc0afb40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc007450 tx=0x7f2bcc15def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.722+0000 7f2bc1ffb700 1 -- 172.21.15.7:0/3415519958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00f840 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.722+0000 7f2bc1ffb700 1 -- 172.21.15.7:0/3415519958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc00f9c0 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.722+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc16b570 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.722+0000 7f2bc1ffb700 1 -- 172.21.15.7:0/3415519958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00fd80 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.722+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc0b0320 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.723+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc04c3c0 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.724+0000 7f2bc1ffb700 1 -- 172.21.15.7:0/3415519958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2bbc0135c0 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.724+0000 7f2bc1ffb700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2bb0066f30 0x7f2bb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.724+0000 7f2bc1ffb700 1 -- 172.21.15.7:0/3415519958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2bbc0843a0 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.724+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2bb0066f30 0x7f2bb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.725+0000 7f2bd18d5700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2bb0066f30 0x7f2bb00693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0ad450 tx=0x7f2bc801a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.729+0000 7f2bc1ffb700 1 -- 172.21.15.7:0/3415519958 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bbc051610 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.980+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2bcc16b220 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.980+0000 7f2bc1ffb700 1 -- 172.21.15.7:0/3415519958 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2bbc054c30 con 0x7f2bcc0cb4e0 2024-01-11T07:14:11.980 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:14:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2bb0066f30 msgr2=0x7f2bb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2bb0066f30 0x7f2bb00693e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0ad450 tx=0x7f2bc801a000 comp rx=0 tx=0).stop 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc0cb4e0 msgr2=0x7f2bcc0afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc0cb4e0 0x7f2bcc0afb40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc007450 tx=0x7f2bcc15def0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 shutdown_connections 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2bb0066f30 0x7f2bb00693e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bcc15b8e0 0x7f2bcc15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2bb8003a60 0x7f2bcc14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 --2- 172.21.15.7:0/3415519958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc0cb4e0 0x7f2bcc0afb40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.982+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 >> 172.21.15.7:0/3415519958 conn(0x7f2bcc0cd910 msgr2=0x7f2bcc157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.983+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 shutdown_connections 2024-01-11T07:14:11.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:11.983+0000 7f2bd3b39700 1 -- 172.21.15.7:0/3415519958 wait complete. 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:14:11.996 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:14:11.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:11.997 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:14:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41966sec preferred_lft 41966sec 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:14:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:12.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:12.018 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:14:12.087 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:14:12.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:12.089 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:14:12.152 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:14:12.153 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:14:12.153 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:14:12.153 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:14:12.153 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:14:12.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:12.153 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:14:12.153 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:14:12.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:14:12.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:14:12.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:14:12.154 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:14:12.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:12.278 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:14:12.297 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:14:12.298 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:14:12.298 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:14:12.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:12.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:14:12.372 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:14:12.372 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:14:12.434 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:14:12.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:14:12.435 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:14:12.496 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:14:12.496 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:14:12.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:14:12.590 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:14:12.591 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:14:12.591 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:14:12.601 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:14:12.601 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:14:12.601 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:14:12.601 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:14:12.601 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:14:12.618 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:14:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:14:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:14:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:14:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:14:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:14:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:14:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:14:12.621 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:14:12.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:12.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:14:12.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:14:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.831+0000 7f6ca2700700 1 Processor -- start 2024-01-11T07:14:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.831+0000 7f6ca2700700 1 -- start start 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca2700700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c9c0adb40 0x7f6c9c0b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca2700700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0adf10 0x7f6c9c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca2700700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c9c0b3590 0x7f6c9c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca2700700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c0cc2a0 con 0x7f6c9c0b3590 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca2700700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c0b2930 con 0x7f6c9c0adb40 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca2700700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0b3b50 con 0x7f6c9c0adf10 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca0c9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c9c0b3590 0x7f6c9c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca0c9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c9c0b3590 0x7f6c9c0b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56634/0 (socket says 172.21.15.7:56634) 2024-01-11T07:14:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.832+0000 7f6ca0c9d700 1 -- 172.21.15.7:0/2032891772 learned_addr learned my addr 172.21.15.7:0/2032891772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/2032891772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0adf10 0x7f6c9c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/2032891772 >> v2:172.21.15.113:3300/0 conn(0x7f6c9c0adb40 0x7f6c9c0b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6ca0c9d700 1 -- 172.21.15.7:0/2032891772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0adf10 msgr2=0x7f6c9c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6ca0c9d700 1 --2- 172.21.15.7:0/2032891772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0adf10 0x7f6c9c0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6ca0c9d700 1 -- 172.21.15.7:0/2032891772 >> v2:172.21.15.113:3300/0 conn(0x7f6c9c0adb40 msgr2=0x7f6c9c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6ca0c9d700 1 --2- 172.21.15.7:0/2032891772 >> v2:172.21.15.113:3300/0 conn(0x7f6c9c0adb40 0x7f6c9c0b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6ca0c9d700 1 -- 172.21.15.7:0/2032891772 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0b3e30 con 0x7f6c9c0b3590 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/2032891772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0adf10 0x7f6c9c0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/2032891772 >> v2:172.21.15.113:3300/0 conn(0x7f6c9c0adb40 0x7f6c9c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6ca0c9d700 1 --2- 172.21.15.7:0/2032891772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0b3590 0x7f6c9c0b0970 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0cfb70 tx=0x7f6c94009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf9417dac933774 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6c9affd700 1 -- 172.21.15.7:0/2032891772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400e070 con 0x7f6c9c0b3590 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6c9affd700 1 -- 172.21.15.7:0/2032891772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9400ba50 con 0x7f6c9c0b3590 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.833+0000 7f6c9affd700 1 -- 172.21.15.7:0/2032891772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c940092e0 con 0x7f6c9c0b3590 2024-01-11T07:14:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 -- 172.21.15.7:0/2032891772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0b3590 msgr2=0x7f6c9c0b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 --2- 172.21.15.7:0/2032891772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0b3590 0x7f6c9c0b0970 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0cfb70 tx=0x7f6c94009840 comp rx=0 tx=0).stop 2024-01-11T07:14:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 -- 172.21.15.7:0/2032891772 shutdown_connections 2024-01-11T07:14:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 --2- 172.21.15.7:0/2032891772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0b3590 0x7f6c9c0b0970 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 --2- 172.21.15.7:0/2032891772 >> v2:172.21.15.113:3300/0 conn(0x7f6c9c0adb40 0x7f6c9c0b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 --2- 172.21.15.7:0/2032891772 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0adf10 0x7f6c9c0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 -- 172.21.15.7:0/2032891772 >> 172.21.15.7:0/2032891772 conn(0x7f6c9c071160 msgr2=0x7f6c9c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 -- 172.21.15.7:0/2032891772 shutdown_connections 2024-01-11T07:14:12.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.834+0000 7f6ca2700700 1 -- 172.21.15.7:0/2032891772 wait complete. 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.835+0000 7f6ca2700700 1 Processor -- start 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.835+0000 7f6ca2700700 1 -- start start 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6ca2700700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9c0adb40 0x7f6c9c16d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6ca2700700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0adf10 0x7f6c9c15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6ca2700700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0b3590 0x7f6c9c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6ca2700700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c16d720 con 0x7f6c9c0adf10 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6ca2700700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c069060 con 0x7f6c9c0adb40 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6ca2700700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c068450 con 0x7f6c9c0b3590 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6c9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0adf10 0x7f6c9c15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6c9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0adf10 0x7f6c9c15ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56644/0 (socket says 172.21.15.7:56644) 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/172775320 learned_addr learned my addr 172.21.15.7:0/172775320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6ca0c9d700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0b3590 0x7f6c9c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.836+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9c0adb40 0x7f6c9c16d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0b3590 msgr2=0x7f6c9c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0b3590 0x7f6c9c15b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9c0adb40 msgr2=0x7f6c9c16d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9c0adb40 0x7f6c9c16d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c9b7fe700 1 -- 172.21.15.7:0/172775320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c94009ae0 con 0x7f6c9c0adf10 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c9b7fe700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0adf10 0x7f6c9c15ab40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c830 tx=0x7f6c90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c83fff700 1 -- 172.21.15.7:0/172775320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90011600 con 0x7f6c9c0adf10 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c83fff700 1 -- 172.21.15.7:0/172775320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c90011760 con 0x7f6c9c0adf10 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c068730 con 0x7f6c9c0adf10 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6c83fff700 1 -- 172.21.15.7:0/172775320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90011b20 con 0x7f6c9c0adf10 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.837+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c068cf0 con 0x7f6c9c0adf10 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.838+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c16a370 con 0x7f6c9c0adf10 2024-01-11T07:14:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.839+0000 7f6c83fff700 1 -- 172.21.15.7:0/172775320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6c90018360 con 0x7f6c9c0adf10 2024-01-11T07:14:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.839+0000 7f6c83fff700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c88066f30 0x7f6c880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.839+0000 7f6c83fff700 1 -- 172.21.15.7:0/172775320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6c900845d0 con 0x7f6c9c0adf10 2024-01-11T07:14:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.840+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c88066f30 0x7f6c880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:12.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.840+0000 7f6c9bfff700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c88066f30 0x7f6c880693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c15d850 tx=0x7f6c8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:12.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:12.844+0000 7f6c83fff700 1 -- 172.21.15.7:0/172775320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c9c16a370 con 0x7f6c9c0adf10 2024-01-11T07:14:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.094+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6c9c16a370 con 0x7f6c9c0adf10 2024-01-11T07:14:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.095+0000 7f6c83fff700 1 -- 172.21.15.7:0/172775320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6c9c16a370 con 0x7f6c9c0adf10 2024-01-11T07:14:13.095 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:14:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c88066f30 msgr2=0x7f6c880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c88066f30 0x7f6c880693e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c15d850 tx=0x7f6c8c008040 comp rx=0 tx=0).stop 2024-01-11T07:14:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0adf10 msgr2=0x7f6c9c15ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0adf10 0x7f6c9c15ab40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c830 tx=0x7f6c90008040 comp rx=0 tx=0).stop 2024-01-11T07:14:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 shutdown_connections 2024-01-11T07:14:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6c88066f30 0x7f6c880693e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c9c0b3590 0x7f6c9c15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:13.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c9c0adb40 0x7f6c9c16d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 --2- 172.21.15.7:0/172775320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c9c0adf10 0x7f6c9c15ab40 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 >> 172.21.15.7:0/172775320 conn(0x7f6c9c071160 msgr2=0x7f6c9c150680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 shutdown_connections 2024-01-11T07:14:13.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.097+0000 7f6ca2700700 1 -- 172.21.15.7:0/172775320 wait complete. 2024-01-11T07:14:13.110 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:14:13.110 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:14:13.110 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:14:13.110 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:14:13.110 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:14:13.111 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:14:13.111 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:14:13.111 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:14:13.111 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:14:13.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:13.111 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:14:13.133 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:14:13.133 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:14:13.133 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:14:13.133 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42007sec preferred_lft 42007sec 2024-01-11T07:14:13.134 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:14:13.135 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:14:13.136 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:14:13.136 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:14:13.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:13.136 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:14:13.203 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:14:13.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:13.205 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:14:13.267 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:14:13.268 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:14:13.268 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:14:13.268 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:14:13.268 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:14:13.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:13.268 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:14:13.269 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:14:13.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:14:13.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:14:13.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:14:13.269 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:14:13.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:13.379 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:14:13.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:14:13.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:14:13.396 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:14:13.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:13.397 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:14:13.465 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:14:13.466 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:14:13.523 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:14:13.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:14:13.524 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:14:13.580 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:14:13.580 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:14:13.580 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:14:13.664 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:14:13.664 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:14:13.664 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:14:13.672 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:14:13.672 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:14:13.672 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:14:13.672 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:14:13.672 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:14:13.690 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:14:13.690 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:14:13.690 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:14:13.690 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:14:13.690 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:14:13.690 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:14:13.690 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:14:13.690 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:14:13.692 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:14:13.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:14:13.692 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:14:13.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:14:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.900+0000 7f570335d700 1 Processor -- start 2024-01-11T07:14:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.901+0000 7f570335d700 1 -- start start 2024-01-11T07:14:13.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f570335d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56fc157760 0x7f56fc15a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:13.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f570335d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56fc0ccea0 con 0x7f56fc157b30 2024-01-11T07:14:13.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f570335d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56fc0caeb0 con 0x7f56fc0c76f0 2024-01-11T07:14:13.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f570335d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56fc15b450 con 0x7f56fc157760 2024-01-11T07:14:13.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f57018fa700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f56fc157b30 0x7f56fc0cc180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37076/0 (socket says 172.21.15.7:37076) 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f57018fa700 1 -- 172.21.15.7:0/1834919948 learned_addr learned my addr 172.21.15.7:0/1834919948 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f57010f9700 1 --2- 172.21.15.7:0/1834919948 >> v2:172.21.15.113:3300/0 conn(0x7f56fc157760 0x7f56fc15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f56f3fff700 1 -- 172.21.15.7:0/1834919948 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243832509 0 0) 0x7f56fc0ccea0 con 0x7f56fc157b30 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f56f3fff700 1 -- 172.21.15.7:0/1834919948 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e8003570 con 0x7f56fc157b30 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f56f3fff700 1 -- 172.21.15.7:0/1834919948 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131227685 0 0) 0x7f56fc0caeb0 con 0x7f56fc0c76f0 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f56f3fff700 1 -- 172.21.15.7:0/1834919948 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56fc0ccea0 con 0x7f56fc0c76f0 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f57010f9700 1 -- 172.21.15.7:0/1834919948 >> v1:172.21.15.113:6789/0 conn(0x7f56fc0c76f0 legacy=0x7f56fc0c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f57010f9700 1 -- 172.21.15.7:0/1834919948 >> v1:172.21.15.7:6789/0 conn(0x7f56fc157b30 legacy=0x7f56fc0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.902+0000 7f57010f9700 1 -- 172.21.15.7:0/1834919948 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc15abb0 con 0x7f56fc157760 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f57010f9700 1 --2- 172.21.15.7:0/1834919948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157760 0x7f56fc15a520 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f56f80098e0 tx=0x7f56f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=702eb540d442631b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f56f3fff700 1 -- 172.21.15.7:0/1834919948 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8011630 con 0x7f56fc157760 2024-01-11T07:14:13.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f56f3fff700 1 -- 172.21.15.7:0/1834919948 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f8011790 con 0x7f56fc157760 2024-01-11T07:14:13.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f56f3fff700 1 -- 172.21.15.7:0/1834919948 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8011b30 con 0x7f56fc157760 2024-01-11T07:14:13.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f570335d700 1 -- 172.21.15.7:0/1834919948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157760 msgr2=0x7f56fc15a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:13.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f570335d700 1 --2- 172.21.15.7:0/1834919948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157760 0x7f56fc15a520 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f56f80098e0 tx=0x7f56f80091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:13.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f570335d700 1 -- 172.21.15.7:0/1834919948 shutdown_connections 2024-01-11T07:14:13.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f570335d700 1 --2- 172.21.15.7:0/1834919948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157760 0x7f56fc15a520 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:13.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f570335d700 1 -- 172.21.15.7:0/1834919948 >> 172.21.15.7:0/1834919948 conn(0x7f56fc0cd0b0 msgr2=0x7f56fc0cd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:13.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f570335d700 1 -- 172.21.15.7:0/1834919948 shutdown_connections 2024-01-11T07:14:13.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.903+0000 7f570335d700 1 -- 172.21.15.7:0/1834919948 wait complete. 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.905+0000 7f570335d700 1 Processor -- start 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.905+0000 7f570335d700 1 -- start start 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f570335d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56fc0c76f0 0x7f56fc099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f570335d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56fc157760 0x7f56fc156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f570335d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157b30 0x7f56fc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f570335d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc0995e0 con 0x7f56fc157760 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f570335d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc098cb0 con 0x7f56fc157b30 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f570335d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc098f90 con 0x7f56fc0c76f0 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57008f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56fc157760 0x7f56fc156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56fc0c76f0 0x7f56fc099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57018fa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157b30 0x7f56fc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:13.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56fc0c76f0 0x7f56fc099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:37210/0 (socket says 172.21.15.7:37210) 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 -- 172.21.15.7:0/4245578786 learned_addr learned my addr 172.21.15.7:0/4245578786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57008f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56fc157760 0x7f56fc156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56660/0 (socket says 172.21.15.7:56660) 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 -- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157b30 msgr2=0x7f56fc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157b30 0x7f56fc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 -- 172.21.15.7:0/4245578786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56fc157760 msgr2=0x7f56fc156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56fc157760 0x7f56fc156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 -- 172.21.15.7:0/4245578786 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8009420 con 0x7f56fc0c76f0 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.906+0000 7f57010f9700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56fc0c76f0 0x7f56fc099a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f56fc06fcb0 tx=0x7f56f8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.907+0000 7f56f17fa700 1 -- 172.21.15.7:0/4245578786 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f801f070 con 0x7f56fc0c76f0 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.907+0000 7f56f17fa700 1 -- 172.21.15.7:0/4245578786 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f8011900 con 0x7f56fc0c76f0 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.907+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc099270 con 0x7f56fc0c76f0 2024-01-11T07:14:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.907+0000 7f56f17fa700 1 -- 172.21.15.7:0/4245578786 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8023410 con 0x7f56fc0c76f0 2024-01-11T07:14:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.907+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc1573c0 con 0x7f56fc0c76f0 2024-01-11T07:14:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.908+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc06bdd0 con 0x7f56fc0c76f0 2024-01-11T07:14:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.908+0000 7f56f17fa700 1 -- 172.21.15.7:0/4245578786 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f56f802c3b0 con 0x7f56fc0c76f0 2024-01-11T07:14:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.908+0000 7f56f17fa700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f56e0066f30 0x7f56e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.909+0000 7f56f17fa700 1 -- 172.21.15.7:0/4245578786 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f56f8085700 con 0x7f56fc0c76f0 2024-01-11T07:14:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.909+0000 7f57008f8700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f56e0066f30 0x7f56e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.909+0000 7f57008f8700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f56e0066f30 0x7f56e00693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f56fc159c70 tx=0x7f56ec007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:13.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:13.913+0000 7f56f17fa700 1 -- 172.21.15.7:0/4245578786 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56f8052970 con 0x7f56fc0c76f0 2024-01-11T07:14:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.165+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56fc0998c0 con 0x7f56fc0c76f0 2024-01-11T07:14:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.166+0000 7f56f17fa700 1 -- 172.21.15.7:0/4245578786 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f56f8055f90 con 0x7f56fc0c76f0 2024-01-11T07:14:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:14:14.166 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1458931008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1458931008},{"type":"v1","addr":"172.21.15.113:6836","nonce":1458931008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3377988669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3377988669},{"type":"v1","addr":"172.21.15.113:6837","nonce":3377988669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1420951490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1420951490},{"type":"v1","addr":"172.21.15.7:6833","nonce":1420951490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:14:04.679133+0000","modified":"2024-01-11T07:14:08.473548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7425":{"gid":7425,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/187487340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":187487340},{"type":"v1","addr":"172.21.15.7:6835","nonce":187487340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-11T07:14:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f56e0066f30 msgr2=0x7f56e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f56e0066f30 0x7f56e00693e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f56fc159c70 tx=0x7f56ec007d50 comp rx=0 tx=0).stop 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56fc0c76f0 msgr2=0x7f56fc099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56fc0c76f0 0x7f56fc099a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f56fc06fcb0 tx=0x7f56f8017000 comp rx=0 tx=0).stop 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 shutdown_connections 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f56e0066f30 0x7f56e00693e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56fc157b30 0x7f56fc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56fc0c76f0 0x7f56fc099a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 --2- 172.21.15.7:0/4245578786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56fc157760 0x7f56fc156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 >> 172.21.15.7:0/4245578786 conn(0x7f56fc0cd0b0 msgr2=0x7f56fc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 shutdown_connections 2024-01-11T07:14:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.168+0000 7f570335d700 1 -- 172.21.15.7:0/4245578786 wait complete. 2024-01-11T07:14:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2024-01-11T07:14:14.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-11T07:14:14.334 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":7425,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8802708750000008,"mdsmap_epoch":74,"osdmap_epoch":152,"osdmap_epoch_barrier":152,"uptime":14.858867254} 2024-01-11T07:14:14.334 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 7425, "mdsmap_epoch": 74, "osdmap_epoch": 152, "osdmap_epoch_barrier": 152, "rank_uptime": 8.880270875, "state": "up:active", "uptime": 14.858867254, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:14:14.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-11T07:14:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.470+0000 7fd338ac8700 1 Processor -- start 2024-01-11T07:14:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.470+0000 7fd338ac8700 1 -- start start 2024-01-11T07:14:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.471+0000 7fd338ac8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd33415a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd338ac8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3340b39f0 con 0x7fd33415db10 2024-01-11T07:14:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd338ac8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3340d0600 con 0x7fd334159cb0 2024-01-11T07:14:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd338ac8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3340cf9f0 con 0x7fd33406e340 2024-01-11T07:14:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd332d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd33415db10 0x7fd3340ccc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37086/0 (socket says 172.21.15.7:37086) 2024-01-11T07:14:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd332d9d700 1 -- 172.21.15.7:0/11056998 learned_addr learned my addr 172.21.15.7:0/11056998 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd331d9b700 1 --2- 172.21.15.7:0/11056998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd33415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd33159a700 1 -- 172.21.15.7:0/11056998 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276843087 0 0) 0x7fd3340b39f0 con 0x7fd33415db10 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd33159a700 1 -- 172.21.15.7:0/11056998 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd31c003570 con 0x7fd33415db10 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd33159a700 1 -- 172.21.15.7:0/11056998 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190129188 0 0) 0x7fd3340d0600 con 0x7fd334159cb0 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd33159a700 1 -- 172.21.15.7:0/11056998 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3340b39f0 con 0x7fd334159cb0 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd331d9b700 1 -- 172.21.15.7:0/11056998 >> v1:172.21.15.113:6789/0 conn(0x7fd334159cb0 legacy=0x7fd33406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd331d9b700 1 -- 172.21.15.7:0/11056998 >> v1:172.21.15.7:6789/0 conn(0x7fd33415db10 legacy=0x7fd3340ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd331d9b700 1 -- 172.21.15.7:0/11056998 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3340cfcd0 con 0x7fd33406e340 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.472+0000 7fd331d9b700 1 --2- 172.21.15.7:0/11056998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd33415a080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd328009910 tx=0x7fd3280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6127539f3564ee6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd33159a700 1 -- 172.21.15.7:0/11056998 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328011630 con 0x7fd33406e340 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd33159a700 1 -- 172.21.15.7:0/11056998 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd328011790 con 0x7fd33406e340 2024-01-11T07:14:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd33159a700 1 -- 172.21.15.7:0/11056998 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328011b30 con 0x7fd33406e340 2024-01-11T07:14:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd338ac8700 1 -- 172.21.15.7:0/11056998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 msgr2=0x7fd33415a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd338ac8700 1 --2- 172.21.15.7:0/11056998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd33415a080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd328009910 tx=0x7fd3280091a0 comp rx=0 tx=0).stop 2024-01-11T07:14:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd338ac8700 1 -- 172.21.15.7:0/11056998 shutdown_connections 2024-01-11T07:14:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd338ac8700 1 --2- 172.21.15.7:0/11056998 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd33415a080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd338ac8700 1 -- 172.21.15.7:0/11056998 >> 172.21.15.7:0/11056998 conn(0x7fd334063cc0 msgr2=0x7fd3340b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd338ac8700 1 -- 172.21.15.7:0/11056998 shutdown_connections 2024-01-11T07:14:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.473+0000 7fd338ac8700 1 -- 172.21.15.7:0/11056998 wait complete. 2024-01-11T07:14:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd338ac8700 1 Processor -- start 2024-01-11T07:14:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd338ac8700 1 -- start start 2024-01-11T07:14:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd338ac8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd3341495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd338ac8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd334159cb0 0x7fd3340b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd338ac8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33415db10 0x7fd3340b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd338ac8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd334157090 con 0x7fd334159cb0 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd338ac8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd334149af0 con 0x7fd33415db10 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd338ac8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33415d0d0 con 0x7fd33406e340 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.475+0000 7fd331d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd334159cb0 0x7fd3340b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd331d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd334159cb0 0x7fd3340b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56670/0 (socket says 172.21.15.7:56670) 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd331d9b700 1 -- 172.21.15.7:0/1628379365 learned_addr learned my addr 172.21.15.7:0/1628379365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd332d9d700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33415db10 0x7fd3340b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd33259c700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd3341495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd331d9b700 1 -- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 msgr2=0x7fd3341495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd331d9b700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd3341495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd331d9b700 1 -- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33415db10 msgr2=0x7fd3340b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd331d9b700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33415db10 0x7fd3340b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd331d9b700 1 -- 172.21.15.7:0/1628379365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328009420 con 0x7fd334159cb0 2024-01-11T07:14:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd33259c700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd3341495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd332d9d700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33415db10 0x7fd3340b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd331d9b700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd334159cb0 0x7fd3340b5660 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd328009f00 tx=0x7fd328017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd3227fc700 1 -- 172.21.15.7:0/1628379365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32801f070 con 0x7fd334159cb0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd3227fc700 1 -- 172.21.15.7:0/1628379365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd328002f70 con 0x7fd334159cb0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd338ac8700 1 -- 172.21.15.7:0/1628379365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33415d3b0 con 0x7fd334159cb0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd3227fc700 1 -- 172.21.15.7:0/1628379365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328011900 con 0x7fd334159cb0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.476+0000 7fd338ac8700 1 -- 172.21.15.7:0/1628379365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd334156d20 con 0x7fd334159cb0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.478+0000 7fd313fff700 1 -- 172.21.15.7:0/1628379365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c0052a0 con 0x7fd334159cb0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.478+0000 7fd3227fc700 1 -- 172.21.15.7:0/1628379365 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd328023070 con 0x7fd334159cb0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.478+0000 7fd3227fc700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd314066f30 0x7fd3140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.479+0000 7fd3227fc700 1 -- 172.21.15.7:0/1628379365 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd328085820 con 0x7fd334159cb0 2024-01-11T07:14:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.479+0000 7fd33259c700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd314066f30 0x7fd3140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:14:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.479+0000 7fd33259c700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd314066f30 0x7fd3140693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd324007fd0 tx=0x7fd324007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:14:14.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.483+0000 7fd3227fc700 1 -- 172.21.15.7:0/1628379365 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd328052a90 con 0x7fd334159cb0 2024-01-11T07:14:14.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:14.687+0000 7fd313fff700 1 -- 172.21.15.7:0/1628379365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fd30c005820 con 0x7fd334159cb0 2024-01-11T07:14:15.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.505+0000 7fd3227fc700 1 -- 172.21.15.7:0/1628379365 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v153) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd32802a0c0 con 0x7fd334159cb0 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 -- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd314066f30 msgr2=0x7fd3140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd314066f30 0x7fd3140693e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd324007fd0 tx=0x7fd324007d50 comp rx=0 tx=0).stop 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 -- 172.21.15.7:0/1628379365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd334159cb0 msgr2=0x7fd3340b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd334159cb0 0x7fd3340b5660 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fd328009f00 tx=0x7fd328017000 comp rx=0 tx=0).stop 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 -- 172.21.15.7:0/1628379365 shutdown_connections 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd314066f30 0x7fd3140693e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33415db10 0x7fd3340b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd33406e340 0x7fd3341495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 --2- 172.21.15.7:0/1628379365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd334159cb0 0x7fd3340b5660 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:14:15.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 -- 172.21.15.7:0/1628379365 >> 172.21.15.7:0/1628379365 conn(0x7fd334063cc0 msgr2=0x7fd3340b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:14:15.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 -- 172.21.15.7:0/1628379365 shutdown_connections 2024-01-11T07:14:15.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:14:15.508+0000 7fd313fff700 1 -- 172.21.15.7:0/1628379365 wait complete. 2024-01-11T07:14:15.509 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-11T07:14:15.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-11T07:14:15.677 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-11T07:14:15.678 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-11T07:14:15.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-01-11T07:14:15.819 INFO:teuthology.orchestra.run.smithi007.stdout:{"mon_tick_interval":"5"} 2024-01-11T07:14:15.820 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-01-11T07:14:15.820 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-01-11T07:14:15.820 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:14:15.820 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-01-11T07:14:15.820 DEBUG:teuthology.orchestra.run.smithi007:> import datetime 2024-01-11T07:14:15.820 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-11T07:14:15.820 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:14:15.820 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.820 DEBUG:teuthology.orchestra.run.smithi007:> # Write some buffered data through before going full, all should be well 2024-01-11T07:14:15.820 DEBUG:teuthology.orchestra.run.smithi007:> print("writing some data through which we expect to succeed") 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> bytes = 0 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> # buffered IO, the error should always only appear from fsync and not 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> # from write 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> full = False 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(32 * 0.9)): 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-11T07:14:15.821 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> print("done writing {0} bytes".format(bytes)) 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> # OK, now we should sneak in under the full condition 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> # due to the time it takes the OSDs to report to the 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> # mons, and get a successful fsync on our full-making data 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> # buffered write, add more dirty data to the buffer 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> print("starting buffered write") 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(32 * 0.2)): 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-01-11T07:14:15.822 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(15.0) 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> except OSError: 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> pass; 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> # Wait long enough for a background flush that should fail 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(30) 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> # ...and check that the failed background flush is reflected in fclose 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> except OSError: 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> print("close() returned an error as expected") 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-11T07:14:15.823 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("close() failed to raise error") 2024-01-11T07:14:15.824 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-11T07:14:15.824 DEBUG:teuthology.orchestra.run.smithi007:> # The kernel cephfs client does not raise errors on fclose 2024-01-11T07:14:15.824 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-11T07:14:15.824 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:14:15.824 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-01-11T07:14:15.824 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:14:31.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:14:31.421 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:15:01.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:15:01.492 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:15:31.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:15:31.563 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:16:01.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:16:01.632 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:16:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:writing some data through which we expect to succeed 2024-01-11T07:16:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-01-11T07:16:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-01-11T07:16:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-01-11T07:16:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-01-11T07:16:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-01-11T07:16:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-01-11T07:16:19.632 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-01-11T07:16:19.633 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:done writing 29884416 bytes 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:successfully fsync'ed prior to getting full state reported 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:starting buffered write 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:16:19.634 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:16:19.635 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:16:19.635 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:16:19.635 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:16:19.635 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:16:19.635 INFO:teuthology.orchestra.run.smithi007.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-01-11T07:16:19.672 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:16:19.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:16:19.721 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:16:19.768 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:16:19.769 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:16:19.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:16:19.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:16:19.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:16:19.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:19.834 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:16:19.856 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:16:19.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:16:19.856 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:16:19.899 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:16:19.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:19.899 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:16:19.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:16:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.099+0000 7fde1c21e700 1 Processor -- start 2024-01-11T07:16:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.099+0000 7fde1c21e700 1 -- start start 2024-01-11T07:16:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.100+0000 7fde1c21e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde1406e340 0x7fde140ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.100+0000 7fde1c21e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde140d18c0 con 0x7fde14159ca0 2024-01-11T07:16:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.100+0000 7fde1c21e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde140d0010 con 0x7fde1415a070 2024-01-11T07:16:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.100+0000 7fde1c21e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde140cf400 con 0x7fde1406e340 2024-01-11T07:16:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.100+0000 7fde19fba700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fde14159ca0 0x7fde1406dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44022/0 (socket says 172.21.15.7:44022) 2024-01-11T07:16:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.100+0000 7fde19fba700 1 -- 172.21.15.7:0/2520906070 learned_addr learned my addr 172.21.15.7:0/2520906070 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde197b9700 1 --2- 172.21.15.7:0/2520906070 >> v2:172.21.15.113:3300/0 conn(0x7fde1406e340 0x7fde140ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde18fb8700 1 -- 172.21.15.7:0/2520906070 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968986396 0 0) 0x7fde140d18c0 con 0x7fde14159ca0 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde18fb8700 1 -- 172.21.15.7:0/2520906070 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde00003570 con 0x7fde14159ca0 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde18fb8700 1 -- 172.21.15.7:0/2520906070 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829708156 0 0) 0x7fde140d0010 con 0x7fde1415a070 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde18fb8700 1 -- 172.21.15.7:0/2520906070 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde140d18c0 con 0x7fde1415a070 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde197b9700 1 -- 172.21.15.7:0/2520906070 >> v1:172.21.15.113:6789/0 conn(0x7fde1415a070 legacy=0x7fde140b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde197b9700 1 -- 172.21.15.7:0/2520906070 >> v1:172.21.15.7:6789/0 conn(0x7fde14159ca0 legacy=0x7fde1406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde197b9700 1 -- 172.21.15.7:0/2520906070 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde140cf6e0 con 0x7fde1406e340 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde197b9700 1 --2- 172.21.15.7:0/2520906070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1406e340 0x7fde140ce7f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fde04009910 tx=0x7fde040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d557bff6929c0c14 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde18fb8700 1 -- 172.21.15.7:0/2520906070 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04011630 con 0x7fde1406e340 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde18fb8700 1 -- 172.21.15.7:0/2520906070 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04011790 con 0x7fde1406e340 2024-01-11T07:16:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.101+0000 7fde18fb8700 1 -- 172.21.15.7:0/2520906070 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04011b30 con 0x7fde1406e340 2024-01-11T07:16:20.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.102+0000 7fde1c21e700 1 -- 172.21.15.7:0/2520906070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1406e340 msgr2=0x7fde140ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.102+0000 7fde1c21e700 1 --2- 172.21.15.7:0/2520906070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1406e340 0x7fde140ce7f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fde04009910 tx=0x7fde040091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.102+0000 7fde1c21e700 1 -- 172.21.15.7:0/2520906070 shutdown_connections 2024-01-11T07:16:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.102+0000 7fde1c21e700 1 --2- 172.21.15.7:0/2520906070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1406e340 0x7fde140ce7f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.102+0000 7fde1c21e700 1 -- 172.21.15.7:0/2520906070 >> 172.21.15.7:0/2520906070 conn(0x7fde14063cc0 msgr2=0x7fde140d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.102+0000 7fde1c21e700 1 -- 172.21.15.7:0/2520906070 shutdown_connections 2024-01-11T07:16:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.102+0000 7fde1c21e700 1 -- 172.21.15.7:0/2520906070 wait complete. 2024-01-11T07:16:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.103+0000 7fde1c21e700 1 Processor -- start 2024-01-11T07:16:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.103+0000 7fde1c21e700 1 -- start start 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde1c21e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde1406e340 0x7fde1414f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde1c21e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14159ca0 0x7fde140ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde1c21e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1415a070 0x7fde140cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde1c21e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde140b43d0 con 0x7fde1406e340 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde1c21e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde140b4f10 con 0x7fde14159ca0 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde1c21e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde140b4670 con 0x7fde1415a070 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde19fba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde1406e340 0x7fde1414f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde19fba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde1406e340 0x7fde1414f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53820/0 (socket says 172.21.15.7:53820) 2024-01-11T07:16:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.104+0000 7fde19fba700 1 -- 172.21.15.7:0/1529294089 learned_addr learned my addr 172.21.15.7:0/1529294089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde197b9700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14159ca0 0x7fde140ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde1a7bb700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1415a070 0x7fde140cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde19fba700 1 -- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1415a070 msgr2=0x7fde140cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde19fba700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1415a070 0x7fde140cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde19fba700 1 -- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14159ca0 msgr2=0x7fde140ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde19fba700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14159ca0 0x7fde140ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde19fba700 1 -- 172.21.15.7:0/1529294089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde04009420 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde1a7bb700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1415a070 0x7fde140cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde197b9700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14159ca0 0x7fde140ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde19fba700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde1406e340 0x7fde1414f6c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c790 tx=0x7fde10007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde09ffb700 1 -- 172.21.15.7:0/1529294089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10005b30 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde09ffb700 1 -- 172.21.15.7:0/1529294089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10005c90 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde140b4950 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde09ffb700 1 -- 172.21.15.7:0/1529294089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10007300 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.105+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde140c8d40 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.106+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1404c3c0 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.106+0000 7fde09ffb700 1 -- 172.21.15.7:0/1529294089 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fde1001b400 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.107+0000 7fde09ffb700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fddf8066f30 0x7fddf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.107+0000 7fde09ffb700 1 -- 172.21.15.7:0/1529294089 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fde100840b0 con 0x7fde1406e340 2024-01-11T07:16:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.107+0000 7fde09ffb700 1 -- 172.21.15.7:0/1529294089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fddf806eb10 con 0x7fde1406e340 2024-01-11T07:16:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.107+0000 7fde197b9700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fddf8066f30 0x7fddf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.108+0000 7fde197b9700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fddf8066f30 0x7fddf80693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fde1406e740 tx=0x7fde04017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.112+0000 7fde09ffb700 1 -- 172.21.15.7:0/1529294089 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde10051410 con 0x7fde1406e340 2024-01-11T07:16:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.363+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde140c8f30 con 0x7fde1406e340 2024-01-11T07:16:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.363+0000 7fde09ffb700 1 -- 172.21.15.7:0/1529294089 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4596 (secure 0 0 0) 0x7fde100076f0 con 0x7fde1406e340 2024-01-11T07:16:20.363 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:20.364 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1458931008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1458931008},{"type":"v1","addr":"172.21.15.113:6836","nonce":1458931008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3377988669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3377988669},{"type":"v1","addr":"172.21.15.113:6837","nonce":3377988669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1420951490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1420951490},{"type":"v1","addr":"172.21.15.7:6833","nonce":1420951490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:14:04.679133+0000","modified":"2024-01-11T07:14:08.473548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7425":{"gid":7425,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/187487340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":187487340},{"type":"v1","addr":"172.21.15.7:6835","nonce":187487340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fddf8066f30 msgr2=0x7fddf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fddf8066f30 0x7fddf80693e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fde1406e740 tx=0x7fde04017000 comp rx=0 tx=0).stop 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde1406e340 msgr2=0x7fde1414f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde1406e340 0x7fde1414f6c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c790 tx=0x7fde10007ea0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 shutdown_connections 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fddf8066f30 0x7fddf80693e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1415a070 0x7fde140cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde1406e340 0x7fde1414f6c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 --2- 172.21.15.7:0/1529294089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde14159ca0 0x7fde140ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 >> 172.21.15.7:0/1529294089 conn(0x7fde14063cc0 msgr2=0x7fde140b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 shutdown_connections 2024-01-11T07:16:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.366+0000 7fde1c21e700 1 -- 172.21.15.7:0/1529294089 wait complete. 2024-01-11T07:16:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2024-01-11T07:16:20.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:16:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.516+0000 7f0cd5fe4700 1 Processor -- start 2024-01-11T07:16:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.517+0000 7f0cd5fe4700 1 -- start start 2024-01-11T07:16:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.518+0000 7f0cd5fe4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cd0063d40 0x7f0cd00cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.518+0000 7f0cd5fe4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd00ca3a0 con 0x7f0cd00cbd10 2024-01-11T07:16:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.518+0000 7f0cd5fe4700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd00cb200 con 0x7f0cd00c6f30 2024-01-11T07:16:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.518+0000 7f0cd5fe4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd00cc150 con 0x7f0cd0063d40 2024-01-11T07:16:20.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.518+0000 7f0cc6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cd0063d40 0x7f0cd00cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.518+0000 7f0ccffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0cd00cbd10 0x7f0cd0155830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44028/0 (socket says 172.21.15.7:44028) 2024-01-11T07:16:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.518+0000 7f0ccf7fe700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0cd00c6f30 0x7f0cd015a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:51752/0 (socket says 172.21.15.7:51752) 2024-01-11T07:16:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.518+0000 7f0ccffff700 1 -- 172.21.15.7:0/375876771 learned_addr learned my addr 172.21.15.7:0/375876771 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.519+0000 7f0cc6ffd700 1 -- 172.21.15.7:0/375876771 >> v1:172.21.15.113:6789/0 conn(0x7f0cd00c6f30 legacy=0x7f0cd015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.519+0000 7f0cc6ffd700 1 -- 172.21.15.7:0/375876771 >> v1:172.21.15.7:6789/0 conn(0x7f0cd00cbd10 legacy=0x7f0cd0155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.519+0000 7f0cc6ffd700 1 -- 172.21.15.7:0/375876771 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd00ca8d0 con 0x7f0cd0063d40 2024-01-11T07:16:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.519+0000 7f0cc6ffd700 1 --2- 172.21.15.7:0/375876771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd00cb7d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80098e0 tx=0x7f0cb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9469b1defcca008 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.519+0000 7f0cceffd700 1 -- 172.21.15.7:0/375876771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8009e80 con 0x7f0cd0063d40 2024-01-11T07:16:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.519+0000 7f0cceffd700 1 -- 172.21.15.7:0/375876771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb800b8f0 con 0x7f0cd0063d40 2024-01-11T07:16:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.519+0000 7f0cceffd700 1 -- 172.21.15.7:0/375876771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb800bc70 con 0x7f0cd0063d40 2024-01-11T07:16:20.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.520+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/375876771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 msgr2=0x7f0cd00cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.520+0000 7f0cd5fe4700 1 --2- 172.21.15.7:0/375876771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd00cb7d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80098e0 tx=0x7f0cb80091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.520+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/375876771 shutdown_connections 2024-01-11T07:16:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.520+0000 7f0cd5fe4700 1 --2- 172.21.15.7:0/375876771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd00cb7d0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.520+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/375876771 >> 172.21.15.7:0/375876771 conn(0x7f0cd0063230 msgr2=0x7f0cd0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.520+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/375876771 shutdown_connections 2024-01-11T07:16:20.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.520+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/375876771 wait complete. 2024-01-11T07:16:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.521+0000 7f0cd5fe4700 1 Processor -- start 2024-01-11T07:16:20.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.522+0000 7f0cd5fe4700 1 -- start start 2024-01-11T07:16:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0cd5fe4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd0143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0cd5fe4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cd00c6f30 0x7f0cd006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0cd5fe4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00cbd10 0x7f0cd0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0cd5fe4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd006ef60 con 0x7f0cd0063d40 2024-01-11T07:16:20.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0cd5fe4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd0168620 con 0x7f0cd00cbd10 2024-01-11T07:16:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0cd5fe4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd006e6c0 con 0x7f0cd00c6f30 2024-01-11T07:16:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd0143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd0143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53840/0 (socket says 172.21.15.7:53840) 2024-01-11T07:16:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccf7fe700 1 -- 172.21.15.7:0/2364445007 learned_addr learned my addr 172.21.15.7:0/2364445007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0cc6ffd700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cd00c6f30 0x7f0cd006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccffff700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00cbd10 0x7f0cd0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccf7fe700 1 -- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cd00c6f30 msgr2=0x7f0cd006dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccf7fe700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cd00c6f30 0x7f0cd006dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccf7fe700 1 -- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00cbd10 msgr2=0x7f0cd0068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccf7fe700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00cbd10 0x7f0cd0068cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccf7fe700 1 -- 172.21.15.7:0/2364445007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb8009420 con 0x7f0cd0063d40 2024-01-11T07:16:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.523+0000 7f0ccffff700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00cbd10 0x7f0cd0068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.524+0000 7f0ccf7fe700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd0143690 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000cd60 tx=0x7f0cc0001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.524+0000 7f0cc7fff700 1 -- 172.21.15.7:0/2364445007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0005df0 con 0x7f0cd0063d40 2024-01-11T07:16:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.524+0000 7f0cc7fff700 1 -- 172.21.15.7:0/2364445007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc000a920 con 0x7f0cd0063d40 2024-01-11T07:16:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.524+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd006e9a0 con 0x7f0cd0063d40 2024-01-11T07:16:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.524+0000 7f0cc7fff700 1 -- 172.21.15.7:0/2364445007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc000ace0 con 0x7f0cd0063d40 2024-01-11T07:16:20.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.524+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd00b04b0 con 0x7f0cd0063d40 2024-01-11T07:16:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.525+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd004c3c0 con 0x7f0cd0063d40 2024-01-11T07:16:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.525+0000 7f0cc7fff700 1 -- 172.21.15.7:0/2364445007 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0cc0007300 con 0x7f0cd0063d40 2024-01-11T07:16:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.526+0000 7f0cc7fff700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cb0066f30 0x7f0cb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.526+0000 7f0cc7fff700 1 -- 172.21.15.7:0/2364445007 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f0cc0084670 con 0x7f0cd0063d40 2024-01-11T07:16:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.526+0000 7f0cc7fff700 1 -- 172.21.15.7:0/2364445007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f0cb006eb10 con 0x7f0cd0063d40 2024-01-11T07:16:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.526+0000 7f0cc6ffd700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cb0066f30 0x7f0cb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.527+0000 7f0cc6ffd700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cb0066f30 0x7f0cb00693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0064160 tx=0x7f0cb8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.530+0000 7f0cc7fff700 1 -- 172.21.15.7:0/2364445007 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cc0051920 con 0x7f0cd0063d40 2024-01-11T07:16:20.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.735+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cd00cabb0 con 0x7f0cd0063d40 2024-01-11T07:16:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.736+0000 7f0cc7fff700 1 -- 172.21.15.7:0/2364445007 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+15440 (secure 0 0 0) 0x7f0cc0054f40 con 0x7f0cd0063d40 2024-01-11T07:16:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:20.736 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:15:42.262261+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:14:02.428153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-11T07:14:03.665002+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.c","mappings":[{"from":2,"to":0}]},{"pgid":"19.21","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:16:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cb0066f30 msgr2=0x7f0cb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cb0066f30 0x7f0cb00693e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0064160 tx=0x7f0cb8017000 comp rx=0 tx=0).stop 2024-01-11T07:16:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 msgr2=0x7f0cd0143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd0143690 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000cd60 tx=0x7f0cc0001b40 comp rx=0 tx=0).stop 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 shutdown_connections 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0cb0066f30 0x7f0cb00693e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cd00cbd10 0x7f0cd0068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd0063d40 0x7f0cd0143690 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 --2- 172.21.15.7:0/2364445007 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cd00c6f30 0x7f0cd006dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 >> 172.21.15.7:0/2364445007 conn(0x7f0cd0063230 msgr2=0x7f0cd0146770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 shutdown_connections 2024-01-11T07:16:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.738+0000 7f0cd5fe4700 1 -- 172.21.15.7:0/2364445007 wait complete. 2024-01-11T07:16:20.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:14:02.428153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:14:03.665002+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:16:20.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.888+0000 7fabf68aa700 1 Processor -- start 2024-01-11T07:16:20.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.888+0000 7fabf68aa700 1 -- start start 2024-01-11T07:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabf68aa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabf0159d70 0x7fabf015a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabf68aa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf0067840 con 0x7fabf00cf540 2024-01-11T07:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabf68aa700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf00ca820 con 0x7fabf00cf0d0 2024-01-11T07:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabf68aa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf00cb0c0 con 0x7fabf0159d70 2024-01-11T07:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabeffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabf0159d70 0x7fabf015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabf4e47700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fabf00cf540 0x7fabf00c9fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44042/0 (socket says 172.21.15.7:44042) 2024-01-11T07:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabf4e47700 1 -- 172.21.15.7:0/3795945101 learned_addr learned my addr 172.21.15.7:0/3795945101 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:20.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabeffff700 1 -- 172.21.15.7:0/3795945101 >> v1:172.21.15.113:6789/0 conn(0x7fabf00cf0d0 legacy=0x7fabf00c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabeffff700 1 -- 172.21.15.7:0/3795945101 >> v1:172.21.15.7:6789/0 conn(0x7fabf00cf540 legacy=0x7fabf00c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabeffff700 1 -- 172.21.15.7:0/3795945101 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00cab00 con 0x7fabf0159d70 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.889+0000 7fabeffff700 1 --2- 172.21.15.7:0/3795945101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf0159d70 0x7fabf015a140 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fabe00098d0 tx=0x7fabe0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6bf4419f337572 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabeeffd700 1 -- 172.21.15.7:0/3795945101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0009e70 con 0x7fabf0159d70 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabeeffd700 1 -- 172.21.15.7:0/3795945101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe0005c10 con 0x7fabf0159d70 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabeeffd700 1 -- 172.21.15.7:0/3795945101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe000b650 con 0x7fabf0159d70 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabf68aa700 1 -- 172.21.15.7:0/3795945101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf0159d70 msgr2=0x7fabf015a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabf68aa700 1 --2- 172.21.15.7:0/3795945101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf0159d70 0x7fabf015a140 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fabe00098d0 tx=0x7fabe0009120 comp rx=0 tx=0).stop 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabf68aa700 1 -- 172.21.15.7:0/3795945101 shutdown_connections 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabf68aa700 1 --2- 172.21.15.7:0/3795945101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf0159d70 0x7fabf015a140 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabf68aa700 1 -- 172.21.15.7:0/3795945101 >> 172.21.15.7:0/3795945101 conn(0x7fabf0097a50 msgr2=0x7fabf00ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:20.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabf68aa700 1 -- 172.21.15.7:0/3795945101 shutdown_connections 2024-01-11T07:16:20.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.890+0000 7fabf68aa700 1 -- 172.21.15.7:0/3795945101 wait complete. 2024-01-11T07:16:20.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.891+0000 7fabf68aa700 1 Processor -- start 2024-01-11T07:16:20.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.892+0000 7fabf68aa700 1 -- start start 2024-01-11T07:16:20.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.892+0000 7fabf68aa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf00cf0d0 0x7fabf0098620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.892+0000 7fabf68aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00cf540 0x7fabf0156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.892+0000 7fabf68aa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf0159d70 0x7fabf0098f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.892+0000 7fabf68aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf009b7f0 con 0x7fabf00cf540 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.892+0000 7fabf68aa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf0098b60 con 0x7fabf00cf0d0 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.892+0000 7fabf68aa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf009abe0 con 0x7fabf0159d70 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00cf540 0x7fabf0156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00cf540 0x7fabf0156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53858/0 (socket says 172.21.15.7:53858) 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 -- 172.21.15.7:0/3836534719 learned_addr learned my addr 172.21.15.7:0/3836534719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabeffff700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf00cf0d0 0x7fabf0098620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabf4e47700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf0159d70 0x7fabf0098f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 -- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf0159d70 msgr2=0x7fabf0098f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf0159d70 0x7fabf0098f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 -- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf00cf0d0 msgr2=0x7fabf0098620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf00cf0d0 0x7fabf0098620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 -- 172.21.15.7:0/3836534719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe00093c0 con 0x7fabf00cf540 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabeffff700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf00cf0d0 0x7fabf0098620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabef7fe700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00cf540 0x7fabf0156600 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fabd800c7c0 tx=0x7fabd8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabe7fff700 1 -- 172.21.15.7:0/3836534719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8011590 con 0x7fabf00cf540 2024-01-11T07:16:20.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabe7fff700 1 -- 172.21.15.7:0/3836534719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd8011710 con 0x7fabf00cf540 2024-01-11T07:16:20.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf009aec0 con 0x7fabf00cf540 2024-01-11T07:16:20.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabe7fff700 1 -- 172.21.15.7:0/3836534719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8011ad0 con 0x7fabf00cf540 2024-01-11T07:16:20.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.893+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf009b480 con 0x7fabf00cf540 2024-01-11T07:16:20.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.895+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf004c3c0 con 0x7fabf00cf540 2024-01-11T07:16:20.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.895+0000 7fabe7fff700 1 -- 172.21.15.7:0/3836534719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fabd801a360 con 0x7fabf00cf540 2024-01-11T07:16:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.895+0000 7fabe7fff700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.895+0000 7fabe7fff700 1 -- 172.21.15.7:0/3836534719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fabd80835c0 con 0x7fabf00cf540 2024-01-11T07:16:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.895+0000 7fabe7fff700 1 -- 172.21.15.7:0/3836534719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fabd006eb10 con 0x7fabf00cf540 2024-01-11T07:16:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.895+0000 7fabeffff700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.896+0000 7fabeffff700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabd0066f30 0x7fabd00693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009160 tx=0x7fabe0016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:20.900+0000 7fabe7fff700 1 -- 172.21.15.7:0/3836534719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabd80507f0 con 0x7fabf00cf540 2024-01-11T07:16:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.102+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabf00cade0 con 0x7fabf00cf540 2024-01-11T07:16:21.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.103+0000 7fabe7fff700 1 -- 172.21.15.7:0/3836534719 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabd8017020 con 0x7fabf00cf540 2024-01-11T07:16:21.103 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabd0066f30 msgr2=0x7fabd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabd0066f30 0x7fabd00693e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009160 tx=0x7fabe0016000 comp rx=0 tx=0).stop 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00cf540 msgr2=0x7fabf0156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00cf540 0x7fabf0156600 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fabd800c7c0 tx=0x7fabd8007f50 comp rx=0 tx=0).stop 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 shutdown_connections 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabf0159d70 0x7fabf0098f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf00cf0d0 0x7fabf0098620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 --2- 172.21.15.7:0/3836534719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00cf540 0x7fabf0156600 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.105+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 >> 172.21.15.7:0/3836534719 conn(0x7fabf0097a50 msgr2=0x7fabf00cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.106+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 shutdown_connections 2024-01-11T07:16:21.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.106+0000 7fabf68aa700 1 -- 172.21.15.7:0/3836534719 wait complete. 2024-01-11T07:16:21.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:16:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.254+0000 7fb301159700 1 Processor -- start 2024-01-11T07:16:21.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.254+0000 7fb301159700 1 -- start start 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.255+0000 7fb301159700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2fc1511b0 0x7fb2fc0ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.255+0000 7fb301159700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2fc06bf20 0x7fb2fc063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.255+0000 7fb301159700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2fc0acc30 con 0x7fb2fc150de0 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.255+0000 7fb301159700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc0ab260 con 0x7fb2fc06bf20 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.255+0000 7fb301159700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc0acf10 con 0x7fb2fc1511b0 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fb59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2fc06bf20 0x7fb2fc063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb2fc150de0 0x7fb2fc06a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44044/0 (socket says 172.21.15.7:44044) 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fad9d700 1 -- 172.21.15.7:0/1041705762 learned_addr learned my addr 172.21.15.7:0/1041705762 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fa59c700 1 --2- 172.21.15.7:0/1041705762 >> v2:172.21.15.113:3300/0 conn(0x7fb2fc1511b0 0x7fb2fc0ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fb59e700 1 -- 172.21.15.7:0/1041705762 >> v2:172.21.15.113:3300/0 conn(0x7fb2fc1511b0 msgr2=0x7fb2fc0ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fb59e700 1 --2- 172.21.15.7:0/1041705762 >> v2:172.21.15.113:3300/0 conn(0x7fb2fc1511b0 0x7fb2fc0ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fb59e700 1 -- 172.21.15.7:0/1041705762 >> v1:172.21.15.7:6789/0 conn(0x7fb2fc150de0 legacy=0x7fb2fc06a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fb59e700 1 -- 172.21.15.7:0/1041705762 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc0ad1f0 con 0x7fb2fc06bf20 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2fb59e700 1 --2- 172.21.15.7:0/1041705762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc063230 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc15a4f0 tx=0x7fb2f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e75e0b2200fb297 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2f9d9b700 1 -- 172.21.15.7:0/1041705762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f400e070 con 0x7fb2fc06bf20 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2f9d9b700 1 -- 172.21.15.7:0/1041705762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f400bb60 con 0x7fb2fc06bf20 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.256+0000 7fb2f9d9b700 1 -- 172.21.15.7:0/1041705762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4012430 con 0x7fb2fc06bf20 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.257+0000 7fb301159700 1 -- 172.21.15.7:0/1041705762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 msgr2=0x7fb2fc063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.257+0000 7fb301159700 1 --2- 172.21.15.7:0/1041705762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc063230 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc15a4f0 tx=0x7fb2f4009780 comp rx=0 tx=0).stop 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.257+0000 7fb301159700 1 -- 172.21.15.7:0/1041705762 shutdown_connections 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.257+0000 7fb301159700 1 --2- 172.21.15.7:0/1041705762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc063230 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.257+0000 7fb301159700 1 --2- 172.21.15.7:0/1041705762 >> v2:172.21.15.113:3300/0 conn(0x7fb2fc1511b0 0x7fb2fc0ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.257+0000 7fb301159700 1 -- 172.21.15.7:0/1041705762 >> 172.21.15.7:0/1041705762 conn(0x7fb2fc0cd910 msgr2=0x7fb2fc0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:21.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.257+0000 7fb301159700 1 -- 172.21.15.7:0/1041705762 shutdown_connections 2024-01-11T07:16:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.257+0000 7fb301159700 1 -- 172.21.15.7:0/1041705762 wait complete. 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.258+0000 7fb301159700 1 Processor -- start 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.258+0000 7fb301159700 1 -- start start 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb301159700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb301159700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc150de0 0x7fb2fc0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb301159700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2fc1511b0 0x7fb2fc0addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb301159700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0cbf60 con 0x7fb2fc06bf20 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb301159700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0cc930 con 0x7fb2fc150de0 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb301159700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0cb350 con 0x7fb2fc1511b0 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb2fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb2fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc0cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53878/0 (socket says 172.21.15.7:53878) 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.259+0000 7fb2fad9d700 1 -- 172.21.15.7:0/2995311638 learned_addr learned my addr 172.21.15.7:0/2995311638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2fb59e700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2fc1511b0 0x7fb2fc0addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2fa59c700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc150de0 0x7fb2fc0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2fad9d700 1 -- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2fc1511b0 msgr2=0x7fb2fc0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2fad9d700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2fc1511b0 0x7fb2fc0addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2fad9d700 1 -- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc150de0 msgr2=0x7fb2fc0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2fad9d700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc150de0 0x7fb2fc0cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2fad9d700 1 -- 172.21.15.7:0/2995311638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f4009a20 con 0x7fb2fc06bf20 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2fad9d700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc0cace0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c7c0 tx=0x7fb2e4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2f2ffd700 1 -- 172.21.15.7:0/2995311638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4005b30 con 0x7fb2fc06bf20 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2f2ffd700 1 -- 172.21.15.7:0/2995311638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4005cb0 con 0x7fb2fc06bf20 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc0cb630 con 0x7fb2fc06bf20 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb2f2ffd700 1 -- 172.21.15.7:0/2995311638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4004a80 con 0x7fb2fc06bf20 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.260+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc0cbbf0 con 0x7fb2fc06bf20 2024-01-11T07:16:21.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.261+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc064a40 con 0x7fb2fc06bf20 2024-01-11T07:16:21.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.262+0000 7fb2f2ffd700 1 -- 172.21.15.7:0/2995311638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb2e40070f0 con 0x7fb2fc06bf20 2024-01-11T07:16:21.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.262+0000 7fb2f2ffd700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb2dc066f30 0x7fb2dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.262+0000 7fb2f2ffd700 1 -- 172.21.15.7:0/2995311638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fb2e4083af0 con 0x7fb2fc06bf20 2024-01-11T07:16:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.262+0000 7fb2f2ffd700 1 -- 172.21.15.7:0/2995311638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fb2dc06eb10 con 0x7fb2fc06bf20 2024-01-11T07:16:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.262+0000 7fb2fa59c700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb2dc066f30 0x7fb2dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.263+0000 7fb2fa59c700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb2dc066f30 0x7fb2dc0693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc15d1c0 tx=0x7fb2ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.267+0000 7fb2f2ffd700 1 -- 172.21.15.7:0/2995311638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2e4018070 con 0x7fb2fc06bf20 2024-01-11T07:16:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.475+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2fc0cc240 con 0x7fb2fc06bf20 2024-01-11T07:16:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.475+0000 7fb2f2ffd700 1 -- 172.21.15.7:0/2995311638 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2e4050c80 con 0x7fb2fc06bf20 2024-01-11T07:16:21.475 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb2dc066f30 msgr2=0x7fb2dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb2dc066f30 0x7fb2dc0693e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc15d1c0 tx=0x7fb2ec008040 comp rx=0 tx=0).stop 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 msgr2=0x7fb2fc0cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc0cace0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c7c0 tx=0x7fb2e4007f70 comp rx=0 tx=0).stop 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 shutdown_connections 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb2dc066f30 0x7fb2dc0693e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb2fc1511b0 0x7fb2fc0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc06bf20 0x7fb2fc0cace0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 --2- 172.21.15.7:0/2995311638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc150de0 0x7fb2fc0cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 >> 172.21.15.7:0/2995311638 conn(0x7fb2fc0cd910 msgr2=0x7fb2fc0b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 shutdown_connections 2024-01-11T07:16:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.478+0000 7fb301159700 1 -- 172.21.15.7:0/2995311638 wait complete. 2024-01-11T07:16:21.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:16:21.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.626+0000 7fe44be48700 1 Processor -- start 2024-01-11T07:16:21.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.626+0000 7fe44be48700 1 -- start start 2024-01-11T07:16:21.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.627+0000 7fe44be48700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070690 0x7fe44406a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.627+0000 7fe44be48700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4441568c0 con 0x7fe444070a60 2024-01-11T07:16:21.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.627+0000 7fe44be48700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe444156ba0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.627+0000 7fe44be48700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe444157540 con 0x7fe444070690 2024-01-11T07:16:21.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.627+0000 7fe4493e3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe444070a60 0x7fe4440b25b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44050/0 (socket says 172.21.15.7:44050) 2024-01-11T07:16:21.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.627+0000 7fe4493e3700 1 -- 172.21.15.7:0/1577977388 learned_addr learned my addr 172.21.15.7:0/1577977388 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:21.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe449be4700 1 --2- 172.21.15.7:0/1577977388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070690 0x7fe44406a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe448be2700 1 -- 172.21.15.7:0/1577977388 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505652890 0 0) 0x7fe4441568c0 con 0x7fe444070a60 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe448be2700 1 -- 172.21.15.7:0/1577977388 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe430003570 con 0x7fe444070a60 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe448be2700 1 -- 172.21.15.7:0/1577977388 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272905490 0 0) 0x7fe444156ba0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe448be2700 1 -- 172.21.15.7:0/1577977388 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4441568c0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe449be4700 1 -- 172.21.15.7:0/1577977388 >> v1:172.21.15.113:6789/0 conn(0x7fe4440b2cc0 legacy=0x7fe4440b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe449be4700 1 -- 172.21.15.7:0/1577977388 >> v1:172.21.15.7:6789/0 conn(0x7fe444070a60 legacy=0x7fe4440b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe448be2700 1 -- 172.21.15.7:0/1577977388 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2989395311 0 0) 0x7fe430003570 con 0x7fe444070a60 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe449be4700 1 -- 172.21.15.7:0/1577977388 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe444156e80 con 0x7fe444070690 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe449be4700 1 --2- 172.21.15.7:0/1577977388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070690 0x7fe44406a850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe440009910 tx=0x7fe4400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9b982177d37f3b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.628+0000 7fe448be2700 1 -- 172.21.15.7:0/1577977388 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438001470 con 0x7fe444070690 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe448be2700 1 -- 172.21.15.7:0/1577977388 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440011420 con 0x7fe444070690 2024-01-11T07:16:21.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe448be2700 1 -- 172.21.15.7:0/1577977388 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400117c0 con 0x7fe444070690 2024-01-11T07:16:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe44be48700 1 -- 172.21.15.7:0/1577977388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070690 msgr2=0x7fe44406a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe44be48700 1 --2- 172.21.15.7:0/1577977388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070690 0x7fe44406a850 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe440009910 tx=0x7fe4400091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe44be48700 1 -- 172.21.15.7:0/1577977388 shutdown_connections 2024-01-11T07:16:21.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe44be48700 1 --2- 172.21.15.7:0/1577977388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070690 0x7fe44406a850 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe44be48700 1 -- 172.21.15.7:0/1577977388 >> 172.21.15.7:0/1577977388 conn(0x7fe4440cd070 msgr2=0x7fe4440cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:21.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe44be48700 1 -- 172.21.15.7:0/1577977388 shutdown_connections 2024-01-11T07:16:21.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.629+0000 7fe44be48700 1 -- 172.21.15.7:0/1577977388 wait complete. 2024-01-11T07:16:21.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.630+0000 7fe44be48700 1 Processor -- start 2024-01-11T07:16:21.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44be48700 1 -- start start 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44be48700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe444070690 0x7fe444141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44be48700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070a60 0x7fe444152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44be48700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4440b2cc0 0x7fe444153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44be48700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44416acb0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44be48700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe444142020 con 0x7fe444070690 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44be48700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44415bcb0 con 0x7fe444070a60 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44a3e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4440b2cc0 0x7fe444153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44a3e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4440b2cc0 0x7fe444153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53880/0 (socket says 172.21.15.7:53880) 2024-01-11T07:16:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.631+0000 7fe44a3e5700 1 -- 172.21.15.7:0/306199015 learned_addr learned my addr 172.21.15.7:0/306199015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe4493e3700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070a60 0x7fe444152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe449be4700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe444070690 0x7fe444141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe44a3e5700 1 -- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070a60 msgr2=0x7fe444152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe44a3e5700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070a60 0x7fe444152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe44a3e5700 1 -- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe444070690 msgr2=0x7fe444141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe44a3e5700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe444070690 0x7fe444141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe44a3e5700 1 -- 172.21.15.7:0/306199015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe440009420 con 0x7fe4440b2cc0 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe44a3e5700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4440b2cc0 0x7fe444153470 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00c860 tx=0x7fe43c002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe435ffb700 1 -- 172.21.15.7:0/306199015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c002030 con 0x7fe4440b2cc0 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe435ffb700 1 -- 172.21.15.7:0/306199015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c0021b0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe435ffb700 1 -- 172.21.15.7:0/306199015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c012630 con 0x7fe4440b2cc0 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44415bf90 con 0x7fe4440b2cc0 2024-01-11T07:16:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.632+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44415c550 con 0x7fe4440b2cc0 2024-01-11T07:16:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.633+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44415c8c0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.634+0000 7fe435ffb700 1 -- 172.21.15.7:0/306199015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe43c0129e0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.634+0000 7fe435ffb700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe428066f30 0x7fe4280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.634+0000 7fe435ffb700 1 -- 172.21.15.7:0/306199015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fe43c0844a0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.634+0000 7fe435ffb700 1 -- 172.21.15.7:0/306199015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fe42806eb10 con 0x7fe4440b2cc0 2024-01-11T07:16:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.634+0000 7fe449be4700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe428066f30 0x7fe4280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.635+0000 7fe449be4700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe428066f30 0x7fe4280693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe4400030c0 tx=0x7fe440016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.639+0000 7fe435ffb700 1 -- 172.21.15.7:0/306199015 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe43c0516d0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.841+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe44404c3c0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.842+0000 7fe435ffb700 1 -- 172.21.15.7:0/306199015 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe43c054cf0 con 0x7fe4440b2cc0 2024-01-11T07:16:21.842 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe428066f30 msgr2=0x7fe4280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe428066f30 0x7fe4280693e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe4400030c0 tx=0x7fe440016000 comp rx=0 tx=0).stop 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4440b2cc0 msgr2=0x7fe444153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4440b2cc0 0x7fe444153470 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00c860 tx=0x7fe43c002700 comp rx=0 tx=0).stop 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 shutdown_connections 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe428066f30 0x7fe4280693e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4440b2cc0 0x7fe444153470 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe444070690 0x7fe444141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 --2- 172.21.15.7:0/306199015 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe444070a60 0x7fe444152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 >> 172.21.15.7:0/306199015 conn(0x7fe4440cd070 msgr2=0x7fe44406ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:21.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 shutdown_connections 2024-01-11T07:16:21.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.844+0000 7fe44be48700 1 -- 172.21.15.7:0/306199015 wait complete. 2024-01-11T07:16:21.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:16:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.990+0000 7f66875ea700 1 Processor -- start 2024-01-11T07:16:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.990+0000 7f66875ea700 1 -- start start 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.991+0000 7f66875ea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6680063230 0x7f6680063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.991+0000 7f66875ea700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f668015a280 0x7f66800aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.991+0000 7f66875ea700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668015d880 con 0x7f66800ac5b0 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.991+0000 7f66875ea700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668006c5b0 con 0x7f6680063230 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.991+0000 7f66875ea700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6680063b40 con 0x7f668015a280 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6685386700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6680063230 0x7f6680063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6685b87700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f66800ac5b0 0x7f66800ac9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44058/0 (socket says 172.21.15.7:44058) 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6685b87700 1 -- 172.21.15.7:0/1601136885 learned_addr learned my addr 172.21.15.7:0/1601136885 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6684b85700 1 --2- 172.21.15.7:0/1601136885 >> v2:172.21.15.113:3300/0 conn(0x7f668015a280 0x7f66800aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6685386700 1 -- 172.21.15.7:0/1601136885 >> v2:172.21.15.113:3300/0 conn(0x7f668015a280 msgr2=0x7f66800aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6685386700 1 --2- 172.21.15.7:0/1601136885 >> v2:172.21.15.113:3300/0 conn(0x7f668015a280 0x7f66800aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6685386700 1 -- 172.21.15.7:0/1601136885 >> v1:172.21.15.7:6789/0 conn(0x7f66800ac5b0 legacy=0x7f66800ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6685386700 1 -- 172.21.15.7:0/1601136885 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668015a7f0 con 0x7f6680063230 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6677fff700 1 -- 172.21.15.7:0/1601136885 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333350113 0 0) 0x7f668015d880 con 0x7f66800ac5b0 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6685386700 1 --2- 172.21.15.7:0/1601136885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680063600 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f66800b39b0 tx=0x7f667c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18473508f4b94a28 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6677fff700 1 -- 172.21.15.7:0/1601136885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00b6b0 con 0x7f6680063230 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6677fff700 1 -- 172.21.15.7:0/1601136885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c00b810 con 0x7f6680063230 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.992+0000 7f6677fff700 1 -- 172.21.15.7:0/1601136885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00bbb0 con 0x7f6680063230 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.993+0000 7f66875ea700 1 -- 172.21.15.7:0/1601136885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 msgr2=0x7f6680063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.993+0000 7f66875ea700 1 --2- 172.21.15.7:0/1601136885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680063600 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f66800b39b0 tx=0x7f667c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.993+0000 7f66875ea700 1 -- 172.21.15.7:0/1601136885 shutdown_connections 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.993+0000 7f66875ea700 1 --2- 172.21.15.7:0/1601136885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680063600 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.993+0000 7f66875ea700 1 --2- 172.21.15.7:0/1601136885 >> v2:172.21.15.113:3300/0 conn(0x7f668015a280 0x7f66800aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.993+0000 7f66875ea700 1 -- 172.21.15.7:0/1601136885 >> 172.21.15.7:0/1601136885 conn(0x7f66800680e0 msgr2=0x7f66800cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.993+0000 7f66875ea700 1 -- 172.21.15.7:0/1601136885 shutdown_connections 2024-01-11T07:16:21.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.993+0000 7f66875ea700 1 -- 172.21.15.7:0/1601136885 wait complete. 2024-01-11T07:16:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.994+0000 7f66875ea700 1 Processor -- start 2024-01-11T07:16:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.994+0000 7f66875ea700 1 -- start start 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f66875ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f66875ea700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800ac5b0 0x7f66800b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f66875ea700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f668015a280 0x7f66800b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f66875ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66800affd0 con 0x7f6680063230 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f66875ea700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6680160a90 con 0x7f668015a280 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f66875ea700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66800af350 con 0x7f66800ac5b0 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f6685386700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f6685386700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53904/0 (socket says 172.21.15.7:53904) 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.995+0000 7f6685386700 1 -- 172.21.15.7:0/1244551842 learned_addr learned my addr 172.21.15.7:0/1244551842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f6684b85700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800ac5b0 0x7f66800b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f6685b87700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f668015a280 0x7f66800b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f6685386700 1 -- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800ac5b0 msgr2=0x7f66800b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f6685386700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800ac5b0 0x7f66800b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f6685386700 1 -- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f668015a280 msgr2=0x7f66800b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f6685386700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f668015a280 0x7f66800b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f6685386700 1 -- 172.21.15.7:0/1244551842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c009420 con 0x7f6680063230 2024-01-11T07:16:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f6685386700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680153ca0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f668006d820 tx=0x7f667c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f66757fa700 1 -- 172.21.15.7:0/1244551842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00bb90 con 0x7f6680063230 2024-01-11T07:16:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f66757fa700 1 -- 172.21.15.7:0/1244551842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c011420 con 0x7f6680063230 2024-01-11T07:16:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f66757fa700 1 -- 172.21.15.7:0/1244551842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0117c0 con 0x7f6680063230 2024-01-11T07:16:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66800af630 con 0x7f6680063230 2024-01-11T07:16:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.996+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66800afbf0 con 0x7f6680063230 2024-01-11T07:16:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.998+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668004c3c0 con 0x7f6680063230 2024-01-11T07:16:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.998+0000 7f66757fa700 1 -- 172.21.15.7:0/1244551842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f667c011bb0 con 0x7f6680063230 2024-01-11T07:16:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.998+0000 7f66757fa700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6664066f30 0x7f66640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.998+0000 7f66757fa700 1 -- 172.21.15.7:0/1244551842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f667c085cd0 con 0x7f6680063230 2024-01-11T07:16:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.998+0000 7f66757fa700 1 -- 172.21.15.7:0/1244551842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f666406eb10 con 0x7f6680063230 2024-01-11T07:16:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.999+0000 7f6684b85700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6664066f30 0x7f66640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:21.999+0000 7f6684b85700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6664066f30 0x7f66640693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6680160e00 tx=0x7f6670008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.003+0000 7f66757fa700 1 -- 172.21.15.7:0/1244551842 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f667c011e60 con 0x7f6680063230 2024-01-11T07:16:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.252+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f668015aad0 con 0x7f6680063230 2024-01-11T07:16:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.252+0000 7f66757fa700 1 -- 172.21.15.7:0/1244551842 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f667c015090 con 0x7f6680063230 2024-01-11T07:16:22.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:22.253 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1458931008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1458931008},{"type":"v1","addr":"172.21.15.113:6836","nonce":1458931008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3377988669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3377988669},{"type":"v1","addr":"172.21.15.113:6837","nonce":3377988669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1420951490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1420951490},{"type":"v1","addr":"172.21.15.7:6833","nonce":1420951490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:14:04.679133+0000","modified":"2024-01-11T07:14:08.473548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7425":{"gid":7425,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/187487340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":187487340},{"type":"v1","addr":"172.21.15.7:6835","nonce":187487340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6664066f30 msgr2=0x7f66640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6664066f30 0x7f66640693e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6680160e00 tx=0x7f6670008040 comp rx=0 tx=0).stop 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 msgr2=0x7f6680153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680153ca0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f668006d820 tx=0x7f667c017000 comp rx=0 tx=0).stop 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 shutdown_connections 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6664066f30 0x7f66640693e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f668015a280 0x7f66800b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6680063230 0x7f6680153ca0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 --2- 172.21.15.7:0/1244551842 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f66800ac5b0 0x7f66800b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 >> 172.21.15.7:0/1244551842 conn(0x7f66800680e0 msgr2=0x7f66800cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 shutdown_connections 2024-01-11T07:16:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.255+0000 7f66875ea700 1 -- 172.21.15.7:0/1244551842 wait complete. 2024-01-11T07:16:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2024-01-11T07:16:22.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:16:22.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.399+0000 7fbc37d8e700 1 Processor -- start 2024-01-11T07:16:22.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.400+0000 7fbc37d8e700 1 -- start start 2024-01-11T07:16:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc37d8e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc3006dd00 0x7fbc300c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc37d8e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc3006e0d0 0x7fbc30159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc37d8e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc30159ff0 0x7fbc300c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc37d8e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3015b770 con 0x7fbc3006dd00 2024-01-11T07:16:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc37d8e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3015abf0 con 0x7fbc3006e0d0 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc37d8e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3015aed0 con 0x7fbc30159ff0 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc3006dd00 0x7fbc300c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc3006dd00 0x7fbc300c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53914/0 (socket says 172.21.15.7:53914) 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 -- 172.21.15.7:0/675473479 learned_addr learned my addr 172.21.15.7:0/675473479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 -- 172.21.15.7:0/675473479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc30159ff0 msgr2=0x7fbc300c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc3632b700 1 --2- 172.21.15.7:0/675473479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc30159ff0 0x7fbc300c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35329700 1 --2- 172.21.15.7:0/675473479 >> v2:172.21.15.113:3300/0 conn(0x7fbc3006e0d0 0x7fbc30159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 --2- 172.21.15.7:0/675473479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc30159ff0 0x7fbc300c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 -- 172.21.15.7:0/675473479 >> v2:172.21.15.113:3300/0 conn(0x7fbc3006e0d0 msgr2=0x7fbc30159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 --2- 172.21.15.7:0/675473479 >> v2:172.21.15.113:3300/0 conn(0x7fbc3006e0d0 0x7fbc30159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 -- 172.21.15.7:0/675473479 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3015b1b0 con 0x7fbc3006dd00 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35329700 1 --2- 172.21.15.7:0/675473479 >> v2:172.21.15.113:3300/0 conn(0x7fbc3006e0d0 0x7fbc30159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc3632b700 1 --2- 172.21.15.7:0/675473479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc30159ff0 0x7fbc300c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc35b2a700 1 --2- 172.21.15.7:0/675473479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc300c98d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbc3006d2e0 tx=0x7fbc2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618c3669726121a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.401+0000 7fbc34b28700 1 -- 172.21.15.7:0/675473479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00b860 con 0x7fbc3006dd00 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc34b28700 1 -- 172.21.15.7:0/675473479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c00b9c0 con 0x7fbc3006dd00 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc34b28700 1 -- 172.21.15.7:0/675473479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00bd60 con 0x7fbc3006dd00 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 -- 172.21.15.7:0/675473479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 msgr2=0x7fbc300c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/675473479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc300c98d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbc3006d2e0 tx=0x7fbc2c009250 comp rx=0 tx=0).stop 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 -- 172.21.15.7:0/675473479 shutdown_connections 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/675473479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc300c98d0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/675473479 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc30159ff0 0x7fbc300c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/675473479 >> v2:172.21.15.113:3300/0 conn(0x7fbc3006e0d0 0x7fbc30159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 -- 172.21.15.7:0/675473479 >> 172.21.15.7:0/675473479 conn(0x7fbc300680e0 msgr2=0x7fbc300cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 -- 172.21.15.7:0/675473479 shutdown_connections 2024-01-11T07:16:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.402+0000 7fbc37d8e700 1 -- 172.21.15.7:0/675473479 wait complete. 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.403+0000 7fbc37d8e700 1 Processor -- start 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.403+0000 7fbc37d8e700 1 -- start start 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc37d8e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc30156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc37d8e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc3006e0d0 0x7fbc3016ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc37d8e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc30159ff0 0x7fbc30157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc37d8e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3016a6e0 con 0x7fbc3006dd00 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc37d8e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300c6210 con 0x7fbc30159ff0 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc37d8e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300c64b0 con 0x7fbc3006e0d0 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc35b2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc30156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc35b2a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc30156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53926/0 (socket says 172.21.15.7:53926) 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc35b2a700 1 -- 172.21.15.7:0/2625909432 learned_addr learned my addr 172.21.15.7:0/2625909432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc35329700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc3006e0d0 0x7fbc3016ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.404+0000 7fbc3632b700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc30159ff0 0x7fbc30157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc35b2a700 1 -- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc3006e0d0 msgr2=0x7fbc3016ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc35b2a700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc3006e0d0 0x7fbc3016ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc35b2a700 1 -- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc30159ff0 msgr2=0x7fbc30157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc35b2a700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc30159ff0 0x7fbc30157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc35b2a700 1 -- 172.21.15.7:0/2625909432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0094f0 con 0x7fbc3006dd00 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc35b2a700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc30156fa0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fbc3006d820 tx=0x7fbc2c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc25ffb700 1 -- 172.21.15.7:0/2625909432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c020070 con 0x7fbc3006dd00 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc25ffb700 1 -- 172.21.15.7:0/2625909432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c01b5c0 con 0x7fbc3006dd00 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc300c6790 con 0x7fbc3006dd00 2024-01-11T07:16:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc25ffb700 1 -- 172.21.15.7:0/2625909432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c01b980 con 0x7fbc3006dd00 2024-01-11T07:16:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.405+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc300c6d50 con 0x7fbc3006dd00 2024-01-11T07:16:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.406+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3004c3c0 con 0x7fbc3006dd00 2024-01-11T07:16:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.407+0000 7fbc25ffb700 1 -- 172.21.15.7:0/2625909432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbc2c02d3b0 con 0x7fbc3006dd00 2024-01-11T07:16:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.407+0000 7fbc25ffb700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.407+0000 7fbc25ffb700 1 -- 172.21.15.7:0/2625909432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fbc2c0879e0 con 0x7fbc3006dd00 2024-01-11T07:16:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.407+0000 7fbc25ffb700 1 -- 172.21.15.7:0/2625909432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7fbc1406eb10 con 0x7fbc3006dd00 2024-01-11T07:16:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.407+0000 7fbc35329700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.408+0000 7fbc35329700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc14066f30 0x7fbc140693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbc300cbf10 tx=0x7fbc20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:22.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.412+0000 7fbc25ffb700 1 -- 172.21.15.7:0/2625909432 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc2c054c90 con 0x7fbc3006dd00 2024-01-11T07:16:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.618+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc3015b490 con 0x7fbc3006dd00 2024-01-11T07:16:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.619+0000 7fbc25ffb700 1 -- 172.21.15.7:0/2625909432 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+15440 (secure 0 0 0) 0x7fbc2c029030 con 0x7fbc3006dd00 2024-01-11T07:16:22.620 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:22.621 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:15:42.262261+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:14:02.428153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-11T07:14:03.665002+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.c","mappings":[{"from":2,"to":0}]},{"pgid":"19.21","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:16:22.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.621+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc14066f30 msgr2=0x7fbc140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc14066f30 0x7fbc140693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbc300cbf10 tx=0x7fbc20008040 comp rx=0 tx=0).stop 2024-01-11T07:16:22.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 msgr2=0x7fbc30156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc30156fa0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fbc3006d820 tx=0x7fbc2c017000 comp rx=0 tx=0).stop 2024-01-11T07:16:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 shutdown_connections 2024-01-11T07:16:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc30159ff0 0x7fbc30157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc3006dd00 0x7fbc30156fa0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 --2- 172.21.15.7:0/2625909432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc3006e0d0 0x7fbc3016ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 >> 172.21.15.7:0/2625909432 conn(0x7fbc300680e0 msgr2=0x7fbc300cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 shutdown_connections 2024-01-11T07:16:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.622+0000 7fbc37d8e700 1 -- 172.21.15.7:0/2625909432 wait complete. 2024-01-11T07:16:22.636 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:16:22.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:16:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.769+0000 7f229e114700 1 Processor -- start 2024-01-11T07:16:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.769+0000 7f229e114700 1 -- start start 2024-01-11T07:16:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.770+0000 7f229e114700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.770+0000 7f229e114700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2298069320 0x7f22980b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.770+0000 7f229e114700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22980ce9b0 con 0x7f229815a060 2024-01-11T07:16:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.770+0000 7f229e114700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22980cf530 con 0x7f2298069320 2024-01-11T07:16:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.770+0000 7f229e114700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22980cf9c0 con 0x7f2298159c90 2024-01-11T07:16:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.770+0000 7f2296ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f229815a060 0x7f22980b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44072/0 (socket says 172.21.15.7:44072) 2024-01-11T07:16:22.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.770+0000 7f2296ffd700 1 -- 172.21.15.7:0/121247004 learned_addr learned my addr 172.21.15.7:0/121247004 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f2297fff700 1 --2- 172.21.15.7:0/121247004 >> v2:172.21.15.113:3300/0 conn(0x7f2298069320 0x7f22980b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f22977fe700 1 --2- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f22967fc700 1 -- 172.21.15.7:0/121247004 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775007935 0 0) 0x7f22980ce9b0 con 0x7f229815a060 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f22967fc700 1 -- 172.21.15.7:0/121247004 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2280003570 con 0x7f229815a060 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f2297fff700 1 -- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 msgr2=0x7f22980b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f2297fff700 1 --2- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f2297fff700 1 -- 172.21.15.7:0/121247004 >> v1:172.21.15.7:6789/0 conn(0x7f229815a060 legacy=0x7f22980b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f2297fff700 1 -- 172.21.15.7:0/121247004 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22980cec00 con 0x7f2298069320 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f22977fe700 1 --2- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f2297fff700 1 --2- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2298069320 0x7f22980b3520 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2290009e60 tx=0x7f2290009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13b4efb7c8f8c90e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f22967fc700 1 -- 172.21.15.7:0/121247004 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000e070 con 0x7f2298069320 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f22967fc700 1 -- 172.21.15.7:0/121247004 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229000bab0 con 0x7f2298069320 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.771+0000 7f22967fc700 1 -- 172.21.15.7:0/121247004 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000be30 con 0x7f2298069320 2024-01-11T07:16:22.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.772+0000 7f229e114700 1 -- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2298069320 msgr2=0x7f22980b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.772+0000 7f229e114700 1 --2- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2298069320 0x7f22980b3520 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2290009e60 tx=0x7f2290009780 comp rx=0 tx=0).stop 2024-01-11T07:16:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.772+0000 7f229e114700 1 -- 172.21.15.7:0/121247004 shutdown_connections 2024-01-11T07:16:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.772+0000 7f229e114700 1 --2- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2298069320 0x7f22980b3520 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.772+0000 7f229e114700 1 --2- 172.21.15.7:0/121247004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.772+0000 7f229e114700 1 -- 172.21.15.7:0/121247004 >> 172.21.15.7:0/121247004 conn(0x7f2298071100 msgr2=0x7f229806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.772+0000 7f229e114700 1 -- 172.21.15.7:0/121247004 shutdown_connections 2024-01-11T07:16:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.772+0000 7f229e114700 1 -- 172.21.15.7:0/121247004 wait complete. 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.773+0000 7f229e114700 1 Processor -- start 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.773+0000 7f229e114700 1 -- start start 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f229e114700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2298069320 0x7f2298144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f229e114700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f229e114700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229815a060 0x7f22980cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f229e114700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229815c230 con 0x7f2298069320 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f229e114700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229816bae0 con 0x7f229815a060 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f229e114700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229815b990 con 0x7f2298159c90 2024-01-11T07:16:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f22977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2298069320 0x7f2298144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f22977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2298069320 0x7f2298144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53928/0 (socket says 172.21.15.7:53928) 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.774+0000 7f22977fe700 1 -- 172.21.15.7:0/4271281741 learned_addr learned my addr 172.21.15.7:0/4271281741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f2296ffd700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f22977fe700 1 -- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 msgr2=0x7f22980cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f2297fff700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229815a060 0x7f22980cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f22977fe700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f22977fe700 1 -- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229815a060 msgr2=0x7f22980cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f22977fe700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229815a060 0x7f22980cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f22977fe700 1 -- 172.21.15.7:0/4271281741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288009420 con 0x7f2298069320 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f2296ffd700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f2297fff700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229815a060 0x7f22980cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f22977fe700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2298069320 0x7f2298144680 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f228800ec80 tx=0x7f22880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f227f7fe700 1 -- 172.21.15.7:0/4271281741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800c9d0 con 0x7f2298069320 2024-01-11T07:16:22.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f227f7fe700 1 -- 172.21.15.7:0/4271281741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228800cb30 con 0x7f2298069320 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2290009a20 con 0x7f2298069320 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f227f7fe700 1 -- 172.21.15.7:0/4271281741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800ceb0 con 0x7f2298069320 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.775+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229815bf50 con 0x7f2298069320 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.776+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229816bdc0 con 0x7f2298069320 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.777+0000 7f227f7fe700 1 -- 172.21.15.7:0/4271281741 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2288013680 con 0x7f2298069320 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.777+0000 7f227f7fe700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.777+0000 7f227f7fe700 1 -- 172.21.15.7:0/4271281741 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f2288084a30 con 0x7f2298069320 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.777+0000 7f227f7fe700 1 -- 172.21.15.7:0/4271281741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f228406eb10 con 0x7f2298069320 2024-01-11T07:16:22.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.777+0000 7f2296ffd700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.778+0000 7f2296ffd700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2284066f30 0x7f22840693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f228c0058d0 tx=0x7f228c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:22.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:22.782+0000 7f227f7fe700 1 -- 172.21.15.7:0/4271281741 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2288017060 con 0x7f2298069320 2024-01-11T07:16:23.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.040+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2298064a50 con 0x7f2298069320 2024-01-11T07:16:23.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.040+0000 7f227f7fe700 1 -- 172.21.15.7:0/4271281741 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f2288051c60 con 0x7f2298069320 2024-01-11T07:16:23.040 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:23.040 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1458931008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1458931008},{"type":"v1","addr":"172.21.15.113:6836","nonce":1458931008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3377988669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3377988669},{"type":"v1","addr":"172.21.15.113:6837","nonce":3377988669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1420951490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1420951490},{"type":"v1","addr":"172.21.15.7:6833","nonce":1420951490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:14:04.679133+0000","modified":"2024-01-11T07:14:08.473548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7425":{"gid":7425,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/187487340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":187487340},{"type":"v1","addr":"172.21.15.7:6835","nonce":187487340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.042+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2284066f30 msgr2=0x7f22840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.042+0000 7f229e114700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2284066f30 0x7f22840693e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f228c0058d0 tx=0x7f228c009040 comp rx=0 tx=0).stop 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.042+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2298069320 msgr2=0x7f2298144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.042+0000 7f229e114700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2298069320 0x7f2298144680 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f228800ec80 tx=0x7f22880091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.042+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 shutdown_connections 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.043+0000 7f229e114700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2284066f30 0x7f22840693e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.043+0000 7f229e114700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229815a060 0x7f22980cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.043+0000 7f229e114700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2298069320 0x7f2298144680 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.043+0000 7f229e114700 1 --2- 172.21.15.7:0/4271281741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2298159c90 0x7f22980cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.043+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 >> 172.21.15.7:0/4271281741 conn(0x7f2298071100 msgr2=0x7f2298068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.043+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 shutdown_connections 2024-01-11T07:16:23.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.043+0000 7f229e114700 1 -- 172.21.15.7:0/4271281741 wait complete. 2024-01-11T07:16:23.044 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2024-01-11T07:16:23.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:16:23.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.188+0000 7f17346d9700 1 Processor -- start 2024-01-11T07:16:23.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.188+0000 7f17346d9700 1 -- start start 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.189+0000 7f17346d9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f172c157f20 0x7f172c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.189+0000 7f17346d9700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f172c15b4b0 0x7f172c0b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.189+0000 7f17346d9700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172c06a580 con 0x7f172c0cce70 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.189+0000 7f17346d9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172c06a860 con 0x7f172c157f20 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.189+0000 7f17346d9700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172c06a390 con 0x7f172c15b4b0 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.189+0000 7f1731c74700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f172c157f20 0x7f172c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.189+0000 7f1731c74700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f172c157f20 0x7f172c0cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53934/0 (socket says 172.21.15.7:53934) 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.189+0000 7f1731c74700 1 -- 172.21.15.7:0/376508123 learned_addr learned my addr 172.21.15.7:0/376508123 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1732c76700 1 --2- 172.21.15.7:0/376508123 >> v2:172.21.15.113:3300/0 conn(0x7f172c15b4b0 0x7f172c0b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1731c74700 1 -- 172.21.15.7:0/376508123 >> v2:172.21.15.113:3300/0 conn(0x7f172c15b4b0 msgr2=0x7f172c0b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1731c74700 1 --2- 172.21.15.7:0/376508123 >> v2:172.21.15.113:3300/0 conn(0x7f172c15b4b0 0x7f172c0b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1731c74700 1 -- 172.21.15.7:0/376508123 >> v1:172.21.15.113:6789/0 conn(0x7f172c0cce70 legacy=0x7f172c069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1731c74700 1 -- 172.21.15.7:0/376508123 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c15a9d0 con 0x7f172c157f20 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1731c74700 1 --2- 172.21.15.7:0/376508123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c157f20 0x7f172c0cd240 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f172c0b24a0 tx=0x7f1728009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b7bd3218f09fb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1731473700 1 -- 172.21.15.7:0/376508123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172800b820 con 0x7f172c157f20 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1731473700 1 -- 172.21.15.7:0/376508123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172800b980 con 0x7f172c157f20 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.190+0000 7f1731473700 1 -- 172.21.15.7:0/376508123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172800bd20 con 0x7f172c157f20 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.191+0000 7f17346d9700 1 -- 172.21.15.7:0/376508123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c157f20 msgr2=0x7f172c0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.191+0000 7f17346d9700 1 --2- 172.21.15.7:0/376508123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c157f20 0x7f172c0cd240 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f172c0b24a0 tx=0x7f1728009250 comp rx=0 tx=0).stop 2024-01-11T07:16:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.191+0000 7f17346d9700 1 -- 172.21.15.7:0/376508123 shutdown_connections 2024-01-11T07:16:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.191+0000 7f17346d9700 1 --2- 172.21.15.7:0/376508123 >> v2:172.21.15.113:3300/0 conn(0x7f172c15b4b0 0x7f172c0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.191+0000 7f17346d9700 1 --2- 172.21.15.7:0/376508123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c157f20 0x7f172c0cd240 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.191+0000 7f17346d9700 1 -- 172.21.15.7:0/376508123 >> 172.21.15.7:0/376508123 conn(0x7f172c0cd910 msgr2=0x7f172c0cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.191+0000 7f17346d9700 1 -- 172.21.15.7:0/376508123 shutdown_connections 2024-01-11T07:16:23.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.191+0000 7f17346d9700 1 -- 172.21.15.7:0/376508123 wait complete. 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.192+0000 7f17346d9700 1 Processor -- start 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.192+0000 7f17346d9700 1 -- start start 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f17346d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c0cce70 0x7f172c0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f17346d9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f172c157f20 0x7f172c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f17346d9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f172c15b4b0 0x7f172c16a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f17346d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c15c860 con 0x7f172c0cce70 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f17346d9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c15cb30 con 0x7f172c15b4b0 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f17346d9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c0ad070 con 0x7f172c157f20 2024-01-11T07:16:23.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732475700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c0cce70 0x7f172c0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732475700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c0cce70 0x7f172c0afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53936/0 (socket says 172.21.15.7:53936) 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732475700 1 -- 172.21.15.7:0/69597765 learned_addr learned my addr 172.21.15.7:0/69597765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1731c74700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f172c157f20 0x7f172c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732c76700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f172c15b4b0 0x7f172c16a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732475700 1 -- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f172c157f20 msgr2=0x7f172c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732475700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f172c157f20 0x7f172c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732475700 1 -- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f172c15b4b0 msgr2=0x7f172c16a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732475700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f172c15b4b0 0x7f172c16a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.193+0000 7f1732475700 1 -- 172.21.15.7:0/69597765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280094f0 con 0x7f172c0cce70 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.194+0000 7f1732475700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c0cce70 0x7f172c0afcb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f172000c5d0 tx=0x7f1720007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.194+0000 7f171e7fc700 1 -- 172.21.15.7:0/69597765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720011590 con 0x7f172c0cce70 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.194+0000 7f171e7fc700 1 -- 172.21.15.7:0/69597765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1720011710 con 0x7f172c0cce70 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.194+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c0ad350 con 0x7f172c0cce70 2024-01-11T07:16:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.194+0000 7f171e7fc700 1 -- 172.21.15.7:0/69597765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720011ad0 con 0x7f172c0cce70 2024-01-11T07:16:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.194+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c0ad910 con 0x7f172c0cce70 2024-01-11T07:16:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.195+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c0adc80 con 0x7f172c0cce70 2024-01-11T07:16:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.195+0000 7f171e7fc700 1 -- 172.21.15.7:0/69597765 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f172001c460 con 0x7f172c0cce70 2024-01-11T07:16:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.196+0000 7f171e7fc700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1710066f30 0x7f17100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.196+0000 7f171e7fc700 1 -- 172.21.15.7:0/69597765 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f1720084230 con 0x7f172c0cce70 2024-01-11T07:16:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.196+0000 7f171e7fc700 1 -- 172.21.15.7:0/69597765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f171006eb10 con 0x7f172c0cce70 2024-01-11T07:16:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.196+0000 7f1731c74700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1710066f30 0x7f17100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.196+0000 7f1731c74700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1710066f30 0x7f17100693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f172c0b24a0 tx=0x7f1728016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.200+0000 7f171e7fc700 1 -- 172.21.15.7:0/69597765 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1720051460 con 0x7f172c0cce70 2024-01-11T07:16:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.454+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f172c064a40 con 0x7f172c0cce70 2024-01-11T07:16:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.455+0000 7f171e7fc700 1 -- 172.21.15.7:0/69597765 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f1720054a80 con 0x7f172c0cce70 2024-01-11T07:16:23.455 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:23.455 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1458931008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1458931008},{"type":"v1","addr":"172.21.15.113:6836","nonce":1458931008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3377988669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3377988669},{"type":"v1","addr":"172.21.15.113:6837","nonce":3377988669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1420951490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1420951490},{"type":"v1","addr":"172.21.15.7:6833","nonce":1420951490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:14:04.679133+0000","modified":"2024-01-11T07:14:08.473548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7425":{"gid":7425,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/187487340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":187487340},{"type":"v1","addr":"172.21.15.7:6835","nonce":187487340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2024-01-11T07:16:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1710066f30 msgr2=0x7f17100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1710066f30 0x7f17100693e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f172c0b24a0 tx=0x7f1728016000 comp rx=0 tx=0).stop 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c0cce70 msgr2=0x7f172c0afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c0cce70 0x7f172c0afcb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f172000c5d0 tx=0x7f1720007f60 comp rx=0 tx=0).stop 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 shutdown_connections 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1710066f30 0x7f17100693e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f172c15b4b0 0x7f172c16a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f172c0cce70 0x7f172c0afcb0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 --2- 172.21.15.7:0/69597765 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f172c157f20 0x7f172c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 >> 172.21.15.7:0/69597765 conn(0x7f172c0cd910 msgr2=0x7f172c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 shutdown_connections 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.457+0000 7f17346d9700 1 -- 172.21.15.7:0/69597765 wait complete. 2024-01-11T07:16:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2024-01-11T07:16:23.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:16:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.602+0000 7f3b62d9f700 1 Processor -- start 2024-01-11T07:16:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.603+0000 7f3b62d9f700 1 -- start start 2024-01-11T07:16:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.603+0000 7f3b62d9f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3b5c0cafb0 0x7f3b5c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.603+0000 7f3b62d9f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5c159300 con 0x7f3b5c0b0b40 2024-01-11T07:16:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.603+0000 7f3b62d9f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5c0cf260 con 0x7f3b5c158280 2024-01-11T07:16:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.603+0000 7f3b62d9f700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5c0cf540 con 0x7f3b5c0cafb0 2024-01-11T07:16:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b6133c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3b5c0b0b40 0x7f3b5c161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44078/0 (socket says 172.21.15.7:44078) 2024-01-11T07:16:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b6133c700 1 -- 172.21.15.7:0/680634103 learned_addr learned my addr 172.21.15.7:0/680634103 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b60b3b700 1 --2- 172.21.15.7:0/680634103 >> v2:172.21.15.113:3300/0 conn(0x7f3b5c0cafb0 0x7f3b5c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183568881 0 0) 0x7f3b5c159300 con 0x7f3b5c0b0b40 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b4c003570 con 0x7f3b5c0b0b40 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132698941 0 0) 0x7f3b5c0cf260 con 0x7f3b5c158280 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b5c159300 con 0x7f3b5c158280 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3550158219 0 0) 0x7f3b4c003570 con 0x7f3b5c0b0b40 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 >> v2:172.21.15.113:3300/0 conn(0x7f3b5c0cafb0 msgr2=0x7f3b5c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 --2- 172.21.15.7:0/680634103 >> v2:172.21.15.113:3300/0 conn(0x7f3b5c0cafb0 0x7f3b5c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 >> v1:172.21.15.113:6789/0 conn(0x7f3b5c158280 legacy=0x7f3b5c0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0cf820 con 0x7f3b5c0b0b40 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f3b540017f0 con 0x7f3b5c0b0b40 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 >> v1:172.21.15.7:6789/0 conn(0x7f3b5c0b0b40 legacy=0x7f3b5c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b4c003bf0 0x7f3b4c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b4c004810 0x7f3b4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0cf540 con 0x7f3b4c0051c0 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c00c0b0 con 0x7f3b4c004810 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c00c270 con 0x7f3b4c003bf0 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b6133c700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.604+0000 7f3b60b3b700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b4c003bf0 0x7f3b4c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b5bfff700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b4c004810 0x7f3b4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b6133c700 1 -- 172.21.15.7:0/680634103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b4c003bf0 msgr2=0x7f3b4c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b6133c700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b4c003bf0 0x7f3b4c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b6133c700 1 -- 172.21.15.7:0/680634103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b4c004810 msgr2=0x7f3b4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b6133c700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b4c004810 0x7f3b4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b6133c700 1 -- 172.21.15.7:0/680634103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c003570 con 0x7f3b4c0051c0 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b6133c700 1 -- 172.21.15.7:0/680634103 reap_dead start 2024-01-11T07:16:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b6133c700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00ba60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400a8f0 tx=0x7f3b5c0cfb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b6b147fb896769 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b540070d0 con 0x7f3b4c0051c0 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b54007290 con 0x7f3b4c0051c0 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.605+0000 7f3b5b7fe700 1 -- 172.21.15.7:0/680634103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54007650 con 0x7f3b4c0051c0 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.606+0000 7f3b62d9f700 1 -- 172.21.15.7:0/680634103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 msgr2=0x7f3b4c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.606+0000 7f3b62d9f700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00ba60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400a8f0 tx=0x7f3b5c0cfb00 comp rx=0 tx=0).stop 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.606+0000 7f3b62d9f700 1 -- 172.21.15.7:0/680634103 shutdown_connections 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.606+0000 7f3b62d9f700 1 --2- 172.21.15.7:0/680634103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b4c00ba60 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.606+0000 7f3b62d9f700 1 -- 172.21.15.7:0/680634103 >> 172.21.15.7:0/680634103 conn(0x7f3b5c071100 msgr2=0x7f3b5c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.606+0000 7f3b62d9f700 1 -- 172.21.15.7:0/680634103 shutdown_connections 2024-01-11T07:16:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.606+0000 7f3b62d9f700 1 -- 172.21.15.7:0/680634103 wait complete. 2024-01-11T07:16:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.607+0000 7f3b62d9f700 1 Processor -- start 2024-01-11T07:16:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.607+0000 7f3b62d9f700 1 -- start start 2024-01-11T07:16:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b62d9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b5c0c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b62d9f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b5c0af510 0x7f3b5c155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b62d9f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b5c0af980 0x7f3b5c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b62d9f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0c51c0 con 0x7f3b4c0051c0 2024-01-11T07:16:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b62d9f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c155b80 con 0x7f3b5c0af510 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b62d9f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0c3ec0 con 0x7f3b5c0af980 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b60b3b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b5c0c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b60b3b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b5c0c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53952/0 (socket says 172.21.15.7:53952) 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b60b3b700 1 -- 172.21.15.7:0/2444242730 learned_addr learned my addr 172.21.15.7:0/2444242730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b5bfff700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b5c0af510 0x7f3b5c155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b6133c700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b5c0af980 0x7f3b5c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b60b3b700 1 -- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b5c0af980 msgr2=0x7f3b5c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b60b3b700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b5c0af980 0x7f3b5c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b60b3b700 1 -- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b5c0af510 msgr2=0x7f3b5c155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.608+0000 7f3b60b3b700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b5c0af510 0x7f3b5c155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.609+0000 7f3b60b3b700 1 -- 172.21.15.7:0/2444242730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0c41a0 con 0x7f3b4c0051c0 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.609+0000 7f3b60b3b700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b5c0c4c80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3b50006ac0 tx=0x7f3b500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.609+0000 7f3b58ff9700 1 -- 172.21.15.7:0/2444242730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50011ec0 con 0x7f3b4c0051c0 2024-01-11T07:16:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.609+0000 7f3b58ff9700 1 -- 172.21.15.7:0/2444242730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5000f940 con 0x7f3b4c0051c0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.609+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c0c4480 con 0x7f3b4c0051c0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.609+0000 7f3b58ff9700 1 -- 172.21.15.7:0/2444242730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5000fcc0 con 0x7f3b4c0051c0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.609+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c167c80 con 0x7f3b4c0051c0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.610+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c0c4000 con 0x7f3b4c0051c0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.611+0000 7f3b58ff9700 1 -- 172.21.15.7:0/2444242730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3b50014650 con 0x7f3b4c0051c0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.611+0000 7f3b58ff9700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3b40066f30 0x7f3b400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.611+0000 7f3b58ff9700 1 -- 172.21.15.7:0/2444242730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f3b50018030 con 0x7f3b4c0051c0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.611+0000 7f3b5bfff700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3b40066f30 0x7f3b400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.611+0000 7f3b58ff9700 1 -- 172.21.15.7:0/2444242730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f3b4006eb10 con 0x7f3b4c0051c0 2024-01-11T07:16:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.612+0000 7f3b5bfff700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3b40066f30 0x7f3b400693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3b48000c00 tx=0x7f3b48007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.616+0000 7f3b58ff9700 1 -- 172.21.15.7:0/2444242730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b50022060 con 0x7f3b4c0051c0 2024-01-11T07:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.822+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b5c04c3c0 con 0x7f3b4c0051c0 2024-01-11T07:16:23.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.823+0000 7f3b58ff9700 1 -- 172.21.15.7:0/2444242730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+15440 (secure 0 0 0) 0x7f3b5c0c4000 con 0x7f3b4c0051c0 2024-01-11T07:16:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:23.823 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:15:42.262261+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:14:02.428153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-01-11T07:14:03.665002+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.c","mappings":[{"from":2,"to":0}]},{"pgid":"19.21","mappings":[{"from":2,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:16:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3b40066f30 msgr2=0x7f3b400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3b40066f30 0x7f3b400693e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3b48000c00 tx=0x7f3b48007d30 comp rx=0 tx=0).stop 2024-01-11T07:16:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 msgr2=0x7f3b5c0c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b5c0c4c80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3b50006ac0 tx=0x7f3b500091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 shutdown_connections 2024-01-11T07:16:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3b40066f30 0x7f3b400693e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3b5c0af980 0x7f3b5c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c0051c0 0x7f3b5c0c4c80 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 --2- 172.21.15.7:0/2444242730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b5c0af510 0x7f3b5c155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 >> 172.21.15.7:0/2444242730 conn(0x7f3b5c071100 msgr2=0x7f3b5c156760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 shutdown_connections 2024-01-11T07:16:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.825+0000 7f3b62d9f700 1 -- 172.21.15.7:0/2444242730 wait complete. 2024-01-11T07:16:23.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:16:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.972+0000 7f9c06fa1700 1 Processor -- start 2024-01-11T07:16:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.972+0000 7f9c06fa1700 1 -- start start 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9c06fa1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c0015d920 0x7f9c0015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9c06fa1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0015e2d0 con 0x7f9c000aa8c0 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9c06fa1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0006ac30 con 0x7f9c000ac3a0 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9c06fa1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c000cb370 con 0x7f9c0015d920 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9c0553e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9c000aa8c0 0x7f9c000aad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44084/0 (socket says 172.21.15.7:44084) 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9c0553e700 1 -- 172.21.15.7:0/1799180086 learned_addr learned my addr 172.21.15.7:0/1799180086 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9bfffff700 1 --2- 172.21.15.7:0/1799180086 >> v2:172.21.15.113:3300/0 conn(0x7f9c0015d920 0x7f9c0015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9bff7fe700 1 -- 172.21.15.7:0/1799180086 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181666934 0 0) 0x7f9c0015e2d0 con 0x7f9c000aa8c0 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.973+0000 7f9bff7fe700 1 -- 172.21.15.7:0/1799180086 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf0003570 con 0x7f9c000aa8c0 2024-01-11T07:16:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.974+0000 7f9bfffff700 1 -- 172.21.15.7:0/1799180086 >> v1:172.21.15.113:6789/0 conn(0x7f9c000ac3a0 legacy=0x7f9c000cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.974+0000 7f9bfffff700 1 -- 172.21.15.7:0/1799180086 >> v1:172.21.15.7:6789/0 conn(0x7f9c000aa8c0 legacy=0x7f9c000aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.974+0000 7f9bff7fe700 1 -- 172.21.15.7:0/1799180086 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494196897 0 0) 0x7f9bf0003570 con 0x7f9c000aa8c0 2024-01-11T07:16:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.974+0000 7f9bfffff700 1 -- 172.21.15.7:0/1799180086 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000ac770 con 0x7f9c0015d920 2024-01-11T07:16:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.974+0000 7f9bfffff700 1 --2- 172.21.15.7:0/1799180086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c0015d920 0x7f9c0015dd90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0098e0 tx=0x7f9bec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0cf110311933d03 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.974+0000 7f9bff7fe700 1 -- 172.21.15.7:0/1799180086 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf80017f0 con 0x7f9c0015d920 2024-01-11T07:16:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.974+0000 7f9bff7fe700 1 -- 172.21.15.7:0/1799180086 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec011420 con 0x7f9c0015d920 2024-01-11T07:16:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.974+0000 7f9bff7fe700 1 -- 172.21.15.7:0/1799180086 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0117c0 con 0x7f9c0015d920 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.975+0000 7f9c06fa1700 1 -- 172.21.15.7:0/1799180086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c0015d920 msgr2=0x7f9c0015dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.975+0000 7f9c06fa1700 1 --2- 172.21.15.7:0/1799180086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c0015d920 0x7f9c0015dd90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0098e0 tx=0x7f9bec0091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.975+0000 7f9c06fa1700 1 -- 172.21.15.7:0/1799180086 shutdown_connections 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.975+0000 7f9c06fa1700 1 --2- 172.21.15.7:0/1799180086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c0015d920 0x7f9c0015dd90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.975+0000 7f9c06fa1700 1 -- 172.21.15.7:0/1799180086 >> 172.21.15.7:0/1799180086 conn(0x7f9c000cd090 msgr2=0x7f9c000cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.975+0000 7f9c06fa1700 1 -- 172.21.15.7:0/1799180086 shutdown_connections 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.975+0000 7f9c06fa1700 1 -- 172.21.15.7:0/1799180086 wait complete. 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.976+0000 7f9c06fa1700 1 Processor -- start 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.976+0000 7f9c06fa1700 1 -- start start 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c06fa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000aa8c0 0x7f9c000c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c06fa1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c000ac3a0 0x7f9c00169520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c06fa1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c0015d920 0x7f9c000ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c06fa1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b2c90 con 0x7f9c000aa8c0 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c06fa1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b0050 con 0x7f9c000ac3a0 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c06fa1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c00169a60 con 0x7f9c0015d920 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c04d3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000aa8c0 0x7f9c000c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c04d3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000aa8c0 0x7f9c000c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53962/0 (socket says 172.21.15.7:53962) 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c04d3d700 1 -- 172.21.15.7:0/2271408384 learned_addr learned my addr 172.21.15.7:0/2271408384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c0553e700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c0015d920 0x7f9c000ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9bfffff700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c000ac3a0 0x7f9c00169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c04d3d700 1 -- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c0015d920 msgr2=0x7f9c000ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c04d3d700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c0015d920 0x7f9c000ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c04d3d700 1 -- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c000ac3a0 msgr2=0x7f9c00169520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c04d3d700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c000ac3a0 0x7f9c00169520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.977+0000 7f9c04d3d700 1 -- 172.21.15.7:0/2271408384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec009420 con 0x7f9c000aa8c0 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.978+0000 7f9c04d3d700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000aa8c0 0x7f9c000c8af0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c590 tx=0x7f9bf400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.978+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4011590 con 0x7f9c000aa8c0 2024-01-11T07:16:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.978+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf4011710 con 0x7f9c000aa8c0 2024-01-11T07:16:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.978+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c000ace40 con 0x7f9c000aa8c0 2024-01-11T07:16:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.978+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4011ad0 con 0x7f9c000aa8c0 2024-01-11T07:16:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.978+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c000b26d0 con 0x7f9c000aa8c0 2024-01-11T07:16:23.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.979+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0004c3c0 con 0x7f9c000aa8c0 2024-01-11T07:16:23.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.979+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9bf401b3b0 con 0x7f9c000aa8c0 2024-01-11T07:16:23.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.980+0000 7f9bfcff9700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:23.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.980+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f9bf40839e0 con 0x7f9c000aa8c0 2024-01-11T07:16:23.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.980+0000 7f9bfffff700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:23.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.980+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f9be406eb10 con 0x7f9c000aa8c0 2024-01-11T07:16:23.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.981+0000 7f9bfffff700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9be4066f30 0x7f9be40693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0091e0 tx=0x7f9bec016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:23.984+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bf4017070 con 0x7f9c000aa8c0 2024-01-11T07:16:24.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.235+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9c0006a300 con 0x7f9c000aa8c0 2024-01-11T07:16:24.236 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:16:24.236+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:16:24.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.238+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(157..157 src has 1..157) v4 ==== 366+0+0 (secure 0 0 0) 0x7f9bf4054f30 con 0x7f9c000aa8c0 2024-01-11T07:16:24.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.238+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=158}) v3 -- 0x7f9be406f460 con 0x7f9c000aa8c0 2024-01-11T07:16:24.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.242+0000 7f9bfcff9700 1 -- 172.21.15.7:0/2271408384 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9bf4054600 con 0x7f9c000aa8c0 2024-01-11T07:16:24.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9be4066f30 msgr2=0x7f9be40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9be4066f30 0x7f9be40693e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0091e0 tx=0x7f9bec016000 comp rx=0 tx=0).stop 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000aa8c0 msgr2=0x7f9c000c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000aa8c0 0x7f9c000c8af0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c590 tx=0x7f9bf400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 shutdown_connections 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9be4066f30 0x7f9be40693e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c0015d920 0x7f9c000ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c000aa8c0 0x7f9c000c8af0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 --2- 172.21.15.7:0/2271408384 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c000ac3a0 0x7f9c00169520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 >> 172.21.15.7:0/2271408384 conn(0x7f9c000cd090 msgr2=0x7f9c00159790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 shutdown_connections 2024-01-11T07:16:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.244+0000 7f9c06fa1700 1 -- 172.21.15.7:0/2271408384 wait complete. 2024-01-11T07:16:24.246 INFO:tasks.ceph.mds.a.smithi007.stderr: -9298> 2024-01-11T07:14:10.413+0000 7efddfe86700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:16:24.246 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:16:24.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:16:24.277 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:16:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.392+0000 7f92fc205700 1 Processor -- start 2024-01-11T07:16:24.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.393+0000 7f92fc205700 1 -- start start 2024-01-11T07:16:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.393+0000 7f92fc205700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40add00 0x7f92f4158c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.393+0000 7f92fc205700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92f40b35e0 0x7f92f40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fc205700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f40cee70 con 0x7f92f40b0430 2024-01-11T07:16:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fc205700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f40cfa10 con 0x7f92f40b35e0 2024-01-11T07:16:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fc205700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40ce840 con 0x7f92f40add00 2024-01-11T07:16:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92f9fa1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40add00 0x7f92f4158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92f97a0700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f92f40b0430 0x7f92f40cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:51820/0 (socket says 172.21.15.7:51820) 2024-01-11T07:16:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fa7a2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92f40b35e0 0x7f92f40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:24.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92f97a0700 1 -- 172.21.15.7:0/3120624516 learned_addr learned my addr 172.21.15.7:0/3120624516 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fa7a2700 1 -- 172.21.15.7:0/3120624516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40add00 msgr2=0x7f92f4158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fa7a2700 1 --2- 172.21.15.7:0/3120624516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40add00 0x7f92f4158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fa7a2700 1 -- 172.21.15.7:0/3120624516 >> v1:172.21.15.113:6789/0 conn(0x7f92f40b0430 legacy=0x7f92f40cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fa7a2700 1 -- 172.21.15.7:0/3120624516 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40b08b0 con 0x7f92f40b35e0 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92f9fa1700 1 --2- 172.21.15.7:0/3120624516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40add00 0x7f92f4158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.394+0000 7f92fa7a2700 1 --2- 172.21.15.7:0/3120624516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40b35e0 0x7f92f40b3a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009e60 tx=0x7f92ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d498534f66cfcca server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92f8f9f700 1 -- 172.21.15.7:0/3120624516 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00e070 con 0x7f92f40b35e0 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92f8f9f700 1 -- 172.21.15.7:0/3120624516 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec00ba60 con 0x7f92f40b35e0 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92f8f9f700 1 -- 172.21.15.7:0/3120624516 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00be20 con 0x7f92f40b35e0 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92fc205700 1 -- 172.21.15.7:0/3120624516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40b35e0 msgr2=0x7f92f40b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:24.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92fc205700 1 --2- 172.21.15.7:0/3120624516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40b35e0 0x7f92f40b3a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009e60 tx=0x7f92ec009780 comp rx=0 tx=0).stop 2024-01-11T07:16:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92fc205700 1 -- 172.21.15.7:0/3120624516 shutdown_connections 2024-01-11T07:16:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92fc205700 1 --2- 172.21.15.7:0/3120624516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40b35e0 0x7f92f40b3a00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92fc205700 1 --2- 172.21.15.7:0/3120624516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40add00 0x7f92f4158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92fc205700 1 -- 172.21.15.7:0/3120624516 >> 172.21.15.7:0/3120624516 conn(0x7f92f4071110 msgr2=0x7f92f406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92fc205700 1 -- 172.21.15.7:0/3120624516 shutdown_connections 2024-01-11T07:16:24.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.395+0000 7f92fc205700 1 -- 172.21.15.7:0/3120624516 wait complete. 2024-01-11T07:16:24.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.397+0000 7f92fc205700 1 Processor -- start 2024-01-11T07:16:24.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.397+0000 7f92fc205700 1 -- start start 2024-01-11T07:16:24.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.397+0000 7f92fc205700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40add00 0x7f92f40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:24.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.397+0000 7f92fc205700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40b0430 0x7f92f4145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:24.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.397+0000 7f92fc205700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f40b35e0 0x7f92f4068ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:24.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.397+0000 7f92fc205700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40cd180 con 0x7f92f40b35e0 2024-01-11T07:16:24.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.397+0000 7f92fc205700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40cc570 con 0x7f92f40add00 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.397+0000 7f92fc205700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40cc850 con 0x7f92f40b0430 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f40b35e0 0x7f92f4068ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f40b35e0 0x7f92f4068ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53984/0 (socket says 172.21.15.7:53984) 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 -- 172.21.15.7:0/1754616649 learned_addr learned my addr 172.21.15.7:0/1754616649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92f97a0700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40b0430 0x7f92f4145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92f9fa1700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40add00 0x7f92f40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 -- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40b0430 msgr2=0x7f92f4145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40b0430 0x7f92f4145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 -- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40add00 msgr2=0x7f92f40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40add00 0x7f92f40cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 -- 172.21.15.7:0/1754616649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f0009420 con 0x7f92f40b35e0 2024-01-11T07:16:24.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fa7a2700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f40b35e0 0x7f92f4068ec0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f92f4066af0 tx=0x7f92ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92e9ffb700 1 -- 172.21.15.7:0/1754616649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec012d60 con 0x7f92f40b35e0 2024-01-11T07:16:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92e9ffb700 1 -- 172.21.15.7:0/1754616649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec01e430 con 0x7f92f40b35e0 2024-01-11T07:16:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec009a20 con 0x7f92f40b35e0 2024-01-11T07:16:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.398+0000 7f92e9ffb700 1 -- 172.21.15.7:0/1754616649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec01e7f0 con 0x7f92f40b35e0 2024-01-11T07:16:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.399+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f40cce10 con 0x7f92f40b35e0 2024-01-11T07:16:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.400+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f40cd460 con 0x7f92f40b35e0 2024-01-11T07:16:24.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.401+0000 7f92e9ffb700 1 -- 172.21.15.7:0/1754616649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f92ec01ebe0 con 0x7f92f40b35e0 2024-01-11T07:16:24.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.401+0000 7f92e9ffb700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92d8066f30 0x7f92d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:24.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.401+0000 7f92e9ffb700 1 -- 172.21.15.7:0/1754616649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f92ec0870b0 con 0x7f92f40b35e0 2024-01-11T07:16:24.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.401+0000 7f92e9ffb700 1 -- 172.21.15.7:0/1754616649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=158}) v3 -- 0x7f92d806ebb0 con 0x7f92f40b35e0 2024-01-11T07:16:24.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.401+0000 7f92f9fa1700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92d8066f30 0x7f92d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:24.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.402+0000 7f92f9fa1700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92d8066f30 0x7f92d80693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009f40 tx=0x7f92f00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:24.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.405+0000 7f92e9ffb700 1 -- 172.21.15.7:0/1754616649 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92ec054310 con 0x7f92f40b35e0 2024-01-11T07:16:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:24.659+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f92f404c3c0 con 0x7f92f40b35e0 2024-01-11T07:16:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.241+0000 7f92e9ffb700 1 -- 172.21.15.7:0/1754616649 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7f92ec02c0a0 con 0x7f92f40b35e0 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92d8066f30 msgr2=0x7f92d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92d8066f30 0x7f92d80693e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009f40 tx=0x7f92f00091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f40b35e0 msgr2=0x7f92f4068ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f40b35e0 0x7f92f4068ec0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f92f4066af0 tx=0x7f92ec018000 comp rx=0 tx=0).stop 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 shutdown_connections 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f92d8066f30 0x7f92d80693e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f40b35e0 0x7f92f4068ec0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92f40add00 0x7f92f40cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 --2- 172.21.15.7:0/1754616649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92f40b0430 0x7f92f4145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.244+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 >> 172.21.15.7:0/1754616649 conn(0x7f92f4071110 msgr2=0x7f92f40b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.245+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 shutdown_connections 2024-01-11T07:16:25.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.245+0000 7f92fc205700 1 -- 172.21.15.7:0/1754616649 wait complete. 2024-01-11T07:16:25.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:16:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.444+0000 7f80a71d1700 1 Processor -- start 2024-01-11T07:16:25.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.445+0000 7f80a71d1700 1 -- start start 2024-01-11T07:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.445+0000 7f80a71d1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80a0159fc0 0x7f80a0069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.445+0000 7f80a71d1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80a0063940 0x7f80a006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.445+0000 7f80a71d1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a00c8570 con 0x7f80a0159bf0 2024-01-11T07:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.445+0000 7f80a71d1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a006e920 con 0x7f80a0159fc0 2024-01-11T07:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.445+0000 7f80a71d1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a006ec00 con 0x7f80a0063940 2024-01-11T07:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80a0159fc0 0x7f80a0069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:25.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80a0159fc0 0x7f80a0069680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53996/0 (socket says 172.21.15.7:53996) 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809ffff700 1 -- 172.21.15.7:0/812722 learned_addr learned my addr 172.21.15.7:0/812722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f80a576e700 1 --2- 172.21.15.7:0/812722 >> v2:172.21.15.113:3300/0 conn(0x7f80a0063940 0x7f80a006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809ffff700 1 -- 172.21.15.7:0/812722 >> v2:172.21.15.113:3300/0 conn(0x7f80a0063940 msgr2=0x7f80a006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809ffff700 1 --2- 172.21.15.7:0/812722 >> v2:172.21.15.113:3300/0 conn(0x7f80a0063940 0x7f80a006dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809ffff700 1 -- 172.21.15.7:0/812722 >> v1:172.21.15.113:6789/0 conn(0x7f80a0159bf0 legacy=0x7f80a0063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809ffff700 1 -- 172.21.15.7:0/812722 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00c7f20 con 0x7f80a0159fc0 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809ffff700 1 --2- 172.21.15.7:0/812722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159fc0 0x7f80a0069680 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f80a0077970 tx=0x7f808c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96942bb5595b604 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809f7fe700 1 -- 172.21.15.7:0/812722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c00b860 con 0x7f80a0159fc0 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809f7fe700 1 -- 172.21.15.7:0/812722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c00b9c0 con 0x7f80a0159fc0 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.446+0000 7f809f7fe700 1 -- 172.21.15.7:0/812722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c00bd60 con 0x7f80a0159fc0 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.447+0000 7f80a71d1700 1 -- 172.21.15.7:0/812722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159fc0 msgr2=0x7f80a0069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.447+0000 7f80a71d1700 1 --2- 172.21.15.7:0/812722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159fc0 0x7f80a0069680 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f80a0077970 tx=0x7f808c009250 comp rx=0 tx=0).stop 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.447+0000 7f80a71d1700 1 -- 172.21.15.7:0/812722 shutdown_connections 2024-01-11T07:16:25.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.447+0000 7f80a71d1700 1 --2- 172.21.15.7:0/812722 >> v2:172.21.15.113:3300/0 conn(0x7f80a0063940 0x7f80a006dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.447+0000 7f80a71d1700 1 --2- 172.21.15.7:0/812722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159fc0 0x7f80a0069680 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.447+0000 7f80a71d1700 1 -- 172.21.15.7:0/812722 >> 172.21.15.7:0/812722 conn(0x7f80a00680a0 msgr2=0x7f80a00cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:25.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.447+0000 7f80a71d1700 1 -- 172.21.15.7:0/812722 shutdown_connections 2024-01-11T07:16:25.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.447+0000 7f80a71d1700 1 -- 172.21.15.7:0/812722 wait complete. 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.448+0000 7f80a71d1700 1 Processor -- start 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f80a71d1700 1 -- start start 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f80a71d1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a0063940 0x7f80a00c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f80a71d1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159bf0 0x7f80a00c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f80a71d1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a0159fc0 0x7f80a00b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f80a71d1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a015a700 con 0x7f80a0159bf0 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f80a71d1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a015cfa0 con 0x7f80a0159fc0 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f80a71d1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a00c67d0 con 0x7f80a0063940 2024-01-11T07:16:25.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f809ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159bf0 0x7f80a00c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f809ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159bf0 0x7f80a00c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54000/0 (socket says 172.21.15.7:54000) 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.449+0000 7f809ffff700 1 -- 172.21.15.7:0/672290526 learned_addr learned my addr 172.21.15.7:0/672290526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f80a4f6d700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a0063940 0x7f80a00c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f80a576e700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a0159fc0 0x7f80a00b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809ffff700 1 -- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a0063940 msgr2=0x7f80a00c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809ffff700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a0063940 0x7f80a00c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809ffff700 1 -- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a0159fc0 msgr2=0x7f80a00b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809ffff700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a0159fc0 0x7f80a00b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809ffff700 1 -- 172.21.15.7:0/672290526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0094f0 con 0x7f80a0159bf0 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f80a4f6d700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a0063940 0x7f80a00c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809ffff700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159bf0 0x7f80a00c7120 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f808c006fd0 tx=0x7f808c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c011e20 con 0x7f80a0159bf0 2024-01-11T07:16:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c01a630 con 0x7f80a0159bf0 2024-01-11T07:16:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a015c390 con 0x7f80a0159bf0 2024-01-11T07:16:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c01a9f0 con 0x7f80a0159bf0 2024-01-11T07:16:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.450+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a015c950 con 0x7f80a0159bf0 2024-01-11T07:16:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.451+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a004c3c0 con 0x7f80a0159bf0 2024-01-11T07:16:25.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.452+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f808c02d430 con 0x7f80a0159bf0 2024-01-11T07:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.452+0000 7f809cff9700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8084066f60 0x7f8084069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.452+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f808c0861c0 con 0x7f80a0159bf0 2024-01-11T07:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.453+0000 7f80a4f6d700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8084066f60 0x7f8084069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.453+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=158}) v3 -- 0x7f808406ebe0 con 0x7f80a0159bf0 2024-01-11T07:16:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.453+0000 7f80a4f6d700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8084066f60 0x7f8084069410 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8094007fd0 tx=0x7f8094009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:25.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.457+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f808c0533a0 con 0x7f80a0159bf0 2024-01-11T07:16:25.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:25.664+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80a015f5d0 con 0x7f80a0159bf0 2024-01-11T07:16:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.463+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(158..158 src has 1..158) v4 ==== 288+0+0 (secure 0 0 0) 0x7f808c00f2e0 con 0x7f80a0159bf0 2024-01-11T07:16:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.463+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f808406d010 con 0x7f80a0159bf0 2024-01-11T07:16:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.467+0000 7f809cff9700 1 -- 172.21.15.7:0/672290526 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) v1 ==== 181+0+0 (secure 0 0 0) 0x7f808c015020 con 0x7f80a0159bf0 2024-01-11T07:16:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.473+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8084066f60 msgr2=0x7f8084069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.473+0000 7f80a71d1700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8084066f60 0x7f8084069410 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8094007fd0 tx=0x7f8094009040 comp rx=0 tx=0).stop 2024-01-11T07:16:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.473+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159bf0 msgr2=0x7f80a00c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.473+0000 7f80a71d1700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159bf0 0x7f80a00c7120 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f808c006fd0 tx=0x7f808c017000 comp rx=0 tx=0).stop 2024-01-11T07:16:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.473+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 shutdown_connections 2024-01-11T07:16:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.474+0000 7f80a71d1700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8084066f60 0x7f8084069410 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.474+0000 7f80a71d1700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80a0159fc0 0x7f80a00b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.474+0000 7f80a71d1700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80a0063940 0x7f80a00c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.474+0000 7f80a71d1700 1 --2- 172.21.15.7:0/672290526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a0159bf0 0x7f80a00c7120 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.474+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 >> 172.21.15.7:0/672290526 conn(0x7f80a00680a0 msgr2=0x7f80a0159370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.474+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 shutdown_connections 2024-01-11T07:16:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.474+0000 7f80a71d1700 1 -- 172.21.15.7:0/672290526 wait complete. 2024-01-11T07:16:26.475 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:16:26.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:16:26.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.630+0000 7ff93375d700 1 Processor -- start 2024-01-11T07:16:26.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.630+0000 7ff93375d700 1 -- start start 2024-01-11T07:16:26.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.631+0000 7ff93375d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff92c0b5360 0x7ff92c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:26.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.631+0000 7ff93375d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff92c06e170 0x7ff92c163b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:26.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.631+0000 7ff93375d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92c15a3b0 con 0x7ff92c159fe0 2024-01-11T07:16:26.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.631+0000 7ff93375d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c06e610 con 0x7ff92c0b5360 2024-01-11T07:16:26.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.631+0000 7ff93375d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c0b3ec0 con 0x7ff92c06e170 2024-01-11T07:16:26.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff930cf8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff92c0b5360 0x7ff92c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:26.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff930cf8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff92c0b5360 0x7ff92c06dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54014/0 (socket says 172.21.15.7:54014) 2024-01-11T07:16:26.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff930cf8700 1 -- 172.21.15.7:0/3398028443 learned_addr learned my addr 172.21.15.7:0/3398028443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff931cfa700 1 --2- 172.21.15.7:0/3398028443 >> v2:172.21.15.113:3300/0 conn(0x7ff92c06e170 0x7ff92c163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff930cf8700 1 -- 172.21.15.7:0/3398028443 >> v2:172.21.15.113:3300/0 conn(0x7ff92c06e170 msgr2=0x7ff92c163b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff930cf8700 1 --2- 172.21.15.7:0/3398028443 >> v2:172.21.15.113:3300/0 conn(0x7ff92c06e170 0x7ff92c163b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff930cf8700 1 -- 172.21.15.7:0/3398028443 >> v1:172.21.15.113:6789/0 conn(0x7ff92c159fe0 legacy=0x7ff92c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff930cf8700 1 -- 172.21.15.7:0/3398028443 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0b3b70 con 0x7ff92c0b5360 2024-01-11T07:16:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff930cf8700 1 --2- 172.21.15.7:0/3398028443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c0b5360 0x7ff92c06dc30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0b0240 tx=0x7ff920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27ad01e50023686 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff91ffff700 1 -- 172.21.15.7:0/3398028443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920009e70 con 0x7ff92c0b5360 2024-01-11T07:16:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff91ffff700 1 -- 172.21.15.7:0/3398028443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92000b860 con 0x7ff92c0b5360 2024-01-11T07:16:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.632+0000 7ff91ffff700 1 -- 172.21.15.7:0/3398028443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000bc20 con 0x7ff92c0b5360 2024-01-11T07:16:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.633+0000 7ff93375d700 1 -- 172.21.15.7:0/3398028443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c0b5360 msgr2=0x7ff92c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.633+0000 7ff93375d700 1 --2- 172.21.15.7:0/3398028443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c0b5360 0x7ff92c06dc30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0b0240 tx=0x7ff920009250 comp rx=0 tx=0).stop 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.633+0000 7ff93375d700 1 -- 172.21.15.7:0/3398028443 shutdown_connections 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.633+0000 7ff93375d700 1 --2- 172.21.15.7:0/3398028443 >> v2:172.21.15.113:3300/0 conn(0x7ff92c06e170 0x7ff92c163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.633+0000 7ff93375d700 1 --2- 172.21.15.7:0/3398028443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c0b5360 0x7ff92c06dc30 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.633+0000 7ff93375d700 1 -- 172.21.15.7:0/3398028443 >> 172.21.15.7:0/3398028443 conn(0x7ff92c063cc0 msgr2=0x7ff92c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.633+0000 7ff93375d700 1 -- 172.21.15.7:0/3398028443 shutdown_connections 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.633+0000 7ff93375d700 1 -- 172.21.15.7:0/3398028443 wait complete. 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.634+0000 7ff93375d700 1 Processor -- start 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.634+0000 7ff93375d700 1 -- start start 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff93375d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff92c06e170 0x7ff92c15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff93375d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff92c0b5360 0x7ff92c14a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff93375d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c159fe0 0x7ff92c14a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff93375d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c14adf0 con 0x7ff92c159fe0 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff93375d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c1575e0 con 0x7ff92c06e170 2024-01-11T07:16:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff93375d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c1578c0 con 0x7ff92c0b5360 2024-01-11T07:16:26.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff931cfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c159fe0 0x7ff92c14a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff931cfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c159fe0 0x7ff92c14a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54018/0 (socket says 172.21.15.7:54018) 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.635+0000 7ff931cfa700 1 -- 172.21.15.7:0/4224680526 learned_addr learned my addr 172.21.15.7:0/4224680526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff930cf8700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff92c0b5360 0x7ff92c14a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff9314f9700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff92c06e170 0x7ff92c15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff931cfa700 1 -- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff92c0b5360 msgr2=0x7ff92c14a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff931cfa700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff92c0b5360 0x7ff92c14a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff931cfa700 1 -- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff92c06e170 msgr2=0x7ff92c15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff931cfa700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff92c06e170 0x7ff92c15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff931cfa700 1 -- 172.21.15.7:0/4224680526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9200094f0 con 0x7ff92c159fe0 2024-01-11T07:16:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff9314f9700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff92c06e170 0x7ff92c15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff930cf8700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff92c0b5360 0x7ff92c14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff931cfa700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c159fe0 0x7ff92c14a8b0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff924009e90 tx=0x7ff92400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400cc80 con 0x7ff92c159fe0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92400f040 con 0x7ff92c159fe0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c157ba0 con 0x7ff92c159fe0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924014480 con 0x7ff92c159fe0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.636+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c15e710 con 0x7ff92c159fe0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.637+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c089160 con 0x7ff92c159fe0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.638+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff924014830 con 0x7ff92c159fe0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.638+0000 7ff91d7fa700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff910066f30 0x7ff9100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.638+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5411+0+0 (secure 0 0 0) 0x7ff924084f20 con 0x7ff92c159fe0 2024-01-11T07:16:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.638+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7ff91006ddb0 con 0x7ff92c159fe0 2024-01-11T07:16:26.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.638+0000 7ff9314f9700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff910066f30 0x7ff9100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:26.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.639+0000 7ff9314f9700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff910066f30 0x7ff9100693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff9280017a0 tx=0x7ff928009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:26.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.643+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9240522b0 con 0x7ff92c159fe0 2024-01-11T07:16:26.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:26.847+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff92c04c3c0 con 0x7ff92c159fe0 2024-01-11T07:16:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.272+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(159..159 src has 1..159) v4 ==== 713+0+0 (secure 0 0 0) 0x7ff924056670 con 0x7ff92c159fe0 2024-01-11T07:16:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.273+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7ff91006cb40 con 0x7ff92c159fe0 2024-01-11T07:16:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.280+0000 7ff91d7fa700 1 -- 172.21.15.7:0/4224680526 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff924018030 con 0x7ff92c159fe0 2024-01-11T07:16:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.287+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff910066f30 msgr2=0x7ff9100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.287+0000 7ff93375d700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff910066f30 0x7ff9100693e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff9280017a0 tx=0x7ff928009040 comp rx=0 tx=0).stop 2024-01-11T07:16:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.287+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c159fe0 msgr2=0x7ff92c14a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.287+0000 7ff93375d700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c159fe0 0x7ff92c14a8b0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff924009e90 tx=0x7ff92400b040 comp rx=0 tx=0).stop 2024-01-11T07:16:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.288+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 shutdown_connections 2024-01-11T07:16:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.288+0000 7ff93375d700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff910066f30 0x7ff9100693e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.288+0000 7ff93375d700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c159fe0 0x7ff92c14a8b0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.288+0000 7ff93375d700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff92c06e170 0x7ff92c15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.288+0000 7ff93375d700 1 --2- 172.21.15.7:0/4224680526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff92c0b5360 0x7ff92c14a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.288+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 >> 172.21.15.7:0/4224680526 conn(0x7ff92c063cc0 msgr2=0x7ff92c0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.288+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 shutdown_connections 2024-01-11T07:16:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.288+0000 7ff93375d700 1 -- 172.21.15.7:0/4224680526 wait complete. 2024-01-11T07:16:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:16:27.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:16:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.441+0000 7f2052167700 1 Processor -- start 2024-01-11T07:16:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.441+0000 7f2052167700 1 -- start start 2024-01-11T07:16:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f2052167700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f204c063480 0x7f204c0ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f2052167700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c069c60 0x7f204c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f2052167700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c0ace80 con 0x7f204c063850 2024-01-11T07:16:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f2052167700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c15c910 con 0x7f204c063480 2024-01-11T07:16:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f2052167700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0ad160 con 0x7f204c069c60 2024-01-11T07:16:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f204b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f204c063480 0x7f204c0ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f204bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f204c063850 0x7f204c1577c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36862/0 (socket says 172.21.15.7:36862) 2024-01-11T07:16:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f204b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f204c063480 0x7f204c0ac7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48916/0 (socket says 172.21.15.7:48916) 2024-01-11T07:16:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f204bfff700 1 -- 172.21.15.7:0/1239860286 learned_addr learned my addr 172.21.15.7:0/1239860286 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.442+0000 7f204affd700 1 --2- 172.21.15.7:0/1239860286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c069c60 0x7f204c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204b7fe700 1 -- 172.21.15.7:0/1239860286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c069c60 msgr2=0x7f204c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204b7fe700 1 --2- 172.21.15.7:0/1239860286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c069c60 0x7f204c15c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204a7fc700 1 -- 172.21.15.7:0/1239860286 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168610536 0 0) 0x7f204c0ace80 con 0x7f204c063850 2024-01-11T07:16:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204b7fe700 1 -- 172.21.15.7:0/1239860286 >> v1:172.21.15.7:6789/0 conn(0x7f204c063850 legacy=0x7f204c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204b7fe700 1 -- 172.21.15.7:0/1239860286 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0ad440 con 0x7f204c063480 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204b7fe700 1 --2- 172.21.15.7:0/1239860286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c063480 0x7f204c0ac7c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f203c0098e0 tx=0x7f203c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713afcab6a5a8877 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204a7fc700 1 -- 172.21.15.7:0/1239860286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c00b640 con 0x7f204c063480 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204a7fc700 1 -- 172.21.15.7:0/1239860286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c00b7a0 con 0x7f204c063480 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.443+0000 7f204a7fc700 1 -- 172.21.15.7:0/1239860286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c00bb40 con 0x7f204c063480 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.444+0000 7f2052167700 1 -- 172.21.15.7:0/1239860286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c063480 msgr2=0x7f204c0ac7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.444+0000 7f2052167700 1 --2- 172.21.15.7:0/1239860286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c063480 0x7f204c0ac7c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f203c0098e0 tx=0x7f203c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.444+0000 7f2052167700 1 -- 172.21.15.7:0/1239860286 shutdown_connections 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.444+0000 7f2052167700 1 --2- 172.21.15.7:0/1239860286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c063480 0x7f204c0ac7c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.444+0000 7f2052167700 1 --2- 172.21.15.7:0/1239860286 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c069c60 0x7f204c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.444+0000 7f2052167700 1 -- 172.21.15.7:0/1239860286 >> 172.21.15.7:0/1239860286 conn(0x7f204c0cd8f0 msgr2=0x7f204c0cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.444+0000 7f2052167700 1 -- 172.21.15.7:0/1239860286 shutdown_connections 2024-01-11T07:16:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.444+0000 7f2052167700 1 -- 172.21.15.7:0/1239860286 wait complete. 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.445+0000 7f2052167700 1 Processor -- start 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.445+0000 7f2052167700 1 -- start start 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f2052167700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f204c063480 0x7f204c148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f2052167700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c063850 0x7f204c0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f2052167700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c069c60 0x7f204c14cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f2052167700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0cbef0 con 0x7f204c069c60 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f2052167700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0cc1d0 con 0x7f204c063480 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f2052167700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0b0a00 con 0x7f204c063850 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c069c60 0x7f204c14cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c069c60 0x7f204c14cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48932/0 (socket says 172.21.15.7:48932) 2024-01-11T07:16:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204bfff700 1 -- 172.21.15.7:0/218677053 learned_addr learned my addr 172.21.15.7:0/218677053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204b7fe700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f204c063480 0x7f204c148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204affd700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c063850 0x7f204c0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204bfff700 1 -- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c063850 msgr2=0x7f204c0cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204bfff700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c063850 0x7f204c0cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204bfff700 1 -- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f204c063480 msgr2=0x7f204c148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204bfff700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f204c063480 0x7f204c148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204bfff700 1 -- 172.21.15.7:0/218677053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c009420 con 0x7f204c069c60 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.446+0000 7f204affd700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c063850 0x7f204c0cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.447+0000 7f204bfff700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c069c60 0x7f204c14cae0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f204400cc90 tx=0x7f2044001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.447+0000 7f20337fe700 1 -- 172.21.15.7:0/218677053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204400a960 con 0x7f204c069c60 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.447+0000 7f20337fe700 1 -- 172.21.15.7:0/218677053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204400aac0 con 0x7f204c069c60 2024-01-11T07:16:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.447+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c0b0ce0 con 0x7f204c069c60 2024-01-11T07:16:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.447+0000 7f20337fe700 1 -- 172.21.15.7:0/218677053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204400ae80 con 0x7f204c069c60 2024-01-11T07:16:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.447+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c148aa0 con 0x7f204c069c60 2024-01-11T07:16:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.448+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c0b0860 con 0x7f204c069c60 2024-01-11T07:16:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.449+0000 7f20337fe700 1 -- 172.21.15.7:0/218677053 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2044007300 con 0x7f204c069c60 2024-01-11T07:16:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.449+0000 7f20337fe700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2038066f30 0x7f20380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.449+0000 7f20337fe700 1 -- 172.21.15.7:0/218677053 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2044084610 con 0x7f204c069c60 2024-01-11T07:16:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.450+0000 7f204b7fe700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2038066f30 0x7f20380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.450+0000 7f204b7fe700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2038066f30 0x7f20380693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f203c0093f0 tx=0x7f203c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.454+0000 7f20337fe700 1 -- 172.21.15.7:0/218677053 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f204c0b0860 con 0x7f204c069c60 2024-01-11T07:16:27.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.659+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f204c0b0860 con 0x7f204c069c60 2024-01-11T07:16:27.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.663+0000 7f20337fe700 1 -- 172.21.15.7:0/218677053 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f204c0b0860 con 0x7f204c069c60 2024-01-11T07:16:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2038066f30 msgr2=0x7f20380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2038066f30 0x7f20380693e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f203c0093f0 tx=0x7f203c017000 comp rx=0 tx=0).stop 2024-01-11T07:16:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c069c60 msgr2=0x7f204c14cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c069c60 0x7f204c14cae0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f204400cc90 tx=0x7f2044001a80 comp rx=0 tx=0).stop 2024-01-11T07:16:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 shutdown_connections 2024-01-11T07:16:27.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2038066f30 0x7f20380693e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f204c069c60 0x7f204c14cae0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f204c063480 0x7f204c148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 --2- 172.21.15.7:0/218677053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f204c063850 0x7f204c0cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 >> 172.21.15.7:0/218677053 conn(0x7f204c0cd8f0 msgr2=0x7f204c0ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 shutdown_connections 2024-01-11T07:16:27.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.665+0000 7f2052167700 1 -- 172.21.15.7:0/218677053 wait complete. 2024-01-11T07:16:27.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-01-11T07:16:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.810+0000 7fa520666700 1 Processor -- start 2024-01-11T07:16:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.810+0000 7fa520666700 1 -- start start 2024-01-11T07:16:27.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.811+0000 7fa520666700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa518063230 0x7fa518063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.811+0000 7fa520666700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa51815a280 0x7fa5180aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.811+0000 7fa520666700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51815d880 con 0x7fa5180ac5b0 2024-01-11T07:16:27.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.811+0000 7fa520666700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51806c5b0 con 0x7fa51815a280 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.811+0000 7fa520666700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa518063b40 con 0x7fa518063230 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51e402700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa518063230 0x7fa518063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51dc01700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa51815a280 0x7fa5180aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51ec03700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa5180ac5b0 0x7fa5180ac9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36878/0 (socket says 172.21.15.7:36878) 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51ec03700 1 -- 172.21.15.7:0/2343810310 learned_addr learned my addr 172.21.15.7:0/2343810310 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51dc01700 1 -- 172.21.15.7:0/2343810310 >> v2:172.21.15.113:3300/0 conn(0x7fa518063230 msgr2=0x7fa518063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51dc01700 1 --2- 172.21.15.7:0/2343810310 >> v2:172.21.15.113:3300/0 conn(0x7fa518063230 0x7fa518063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51dc01700 1 -- 172.21.15.7:0/2343810310 >> v1:172.21.15.7:6789/0 conn(0x7fa5180ac5b0 legacy=0x7fa5180ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51dc01700 1 -- 172.21.15.7:0/2343810310 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51815a7f0 con 0x7fa51815a280 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51dc01700 1 --2- 172.21.15.7:0/2343810310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51815a280 0x7fa5180aa9f0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa5180ad1f0 tx=0x7fa5080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53afc6b5a37fad67 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51d400700 1 -- 172.21.15.7:0/2343810310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508011630 con 0x7fa51815a280 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51d400700 1 -- 172.21.15.7:0/2343810310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa508011790 con 0x7fa51815a280 2024-01-11T07:16:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.812+0000 7fa51d400700 1 -- 172.21.15.7:0/2343810310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508011b30 con 0x7fa51815a280 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.813+0000 7fa520666700 1 -- 172.21.15.7:0/2343810310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51815a280 msgr2=0x7fa5180aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.813+0000 7fa520666700 1 --2- 172.21.15.7:0/2343810310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51815a280 0x7fa5180aa9f0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa5180ad1f0 tx=0x7fa5080091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.813+0000 7fa520666700 1 -- 172.21.15.7:0/2343810310 shutdown_connections 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.813+0000 7fa520666700 1 --2- 172.21.15.7:0/2343810310 >> v2:172.21.15.113:3300/0 conn(0x7fa518063230 0x7fa518063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.813+0000 7fa520666700 1 --2- 172.21.15.7:0/2343810310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa51815a280 0x7fa5180aa9f0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.813+0000 7fa520666700 1 -- 172.21.15.7:0/2343810310 >> 172.21.15.7:0/2343810310 conn(0x7fa5180680e0 msgr2=0x7fa5180cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.813+0000 7fa520666700 1 -- 172.21.15.7:0/2343810310 shutdown_connections 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.813+0000 7fa520666700 1 -- 172.21.15.7:0/2343810310 wait complete. 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.815+0000 7fa520666700 1 Processor -- start 2024-01-11T07:16:27.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.815+0000 7fa520666700 1 -- start start 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa520666700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa518063230 0x7fa5181531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa520666700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5180ac5b0 0x7fa5180c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa520666700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51815a280 0x7fa51816a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa520666700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5180c12a0 con 0x7fa518063230 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa520666700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51816ad90 con 0x7fa5180ac5b0 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa520666700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51816b070 con 0x7fa51815a280 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa518063230 0x7fa5181531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa518063230 0x7fa5181531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48946/0 (socket says 172.21.15.7:48946) 2024-01-11T07:16:27.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 -- 172.21.15.7:0/1875002759 learned_addr learned my addr 172.21.15.7:0/1875002759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51ec03700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51815a280 0x7fa51816a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51dc01700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5180ac5b0 0x7fa5180c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 -- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51815a280 msgr2=0x7fa51816a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51815a280 0x7fa51816a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 -- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5180ac5b0 msgr2=0x7fa5180c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5180ac5b0 0x7fa5180c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 -- 172.21.15.7:0/1875002759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa508009420 con 0x7fa518063230 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51dc01700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5180ac5b0 0x7fa5180c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa51e402700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa518063230 0x7fa5181531d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa51400e930 tx=0x7fa5140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa50e7fc700 1 -- 172.21.15.7:0/1875002759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400eed0 con 0x7fa518063230 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa50e7fc700 1 -- 172.21.15.7:0/1875002759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51400c850 con 0x7fa518063230 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa50e7fc700 1 -- 172.21.15.7:0/1875002759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400cbf0 con 0x7fa518063230 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.816+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5180c14f0 con 0x7fa518063230 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.817+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa518153710 con 0x7fa518063230 2024-01-11T07:16:27.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.817+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51816b230 con 0x7fa518063230 2024-01-11T07:16:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.818+0000 7fa50e7fc700 1 -- 172.21.15.7:0/1875002759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa514013470 con 0x7fa518063230 2024-01-11T07:16:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.818+0000 7fa50e7fc700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4fc066f30 0x7fa4fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.818+0000 7fa50e7fc700 1 -- 172.21.15.7:0/1875002759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa514084460 con 0x7fa518063230 2024-01-11T07:16:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.819+0000 7fa51dc01700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4fc066f30 0x7fa4fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:27.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.819+0000 7fa51dc01700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4fc066f30 0x7fa4fc0693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa5080093f0 tx=0x7fa508017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:27.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:27.823+0000 7fa50e7fc700 1 -- 172.21.15.7:0/1875002759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5140519d0 con 0x7fa518063230 2024-01-11T07:16:28.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.112+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fa51804c3c0 con 0x7fa518063230 2024-01-11T07:16:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.468+0000 7fa50e7fc700 1 -- 172.21.15.7:0/1875002759 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v451) v1 ==== 132+0+0 (secure 0 0 0) 0x7fa514054ff0 con 0x7fa518063230 2024-01-11T07:16:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4fc066f30 msgr2=0x7fa4fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4fc066f30 0x7fa4fc0693e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa5080093f0 tx=0x7fa508017000 comp rx=0 tx=0).stop 2024-01-11T07:16:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa518063230 msgr2=0x7fa5181531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa518063230 0x7fa5181531d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa51400e930 tx=0x7fa5140091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 shutdown_connections 2024-01-11T07:16:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa4fc066f30 0x7fa4fc0693e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa51815a280 0x7fa51816a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa518063230 0x7fa5181531d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 --2- 172.21.15.7:0/1875002759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5180ac5b0 0x7fa5180c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 >> 172.21.15.7:0/1875002759 conn(0x7fa5180680e0 msgr2=0x7fa51815b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 shutdown_connections 2024-01-11T07:16:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.471+0000 7fa520666700 1 -- 172.21.15.7:0/1875002759 wait complete. 2024-01-11T07:16:28.487 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-01-11T07:16:28.487 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2024-01-11T07:16:28.487 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2024-01-11T07:16:28.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-01-11T07:16:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.624+0000 7f81b1697700 1 Processor -- start 2024-01-11T07:16:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.624+0000 7f81b1697700 1 -- start start 2024-01-11T07:16:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.625+0000 7f81b1697700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac06e340 0x7f81ac0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.625+0000 7f81b1697700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0d18c0 con 0x7f81ac159ca0 2024-01-11T07:16:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.625+0000 7f81b1697700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0d0010 con 0x7f81ac15a070 2024-01-11T07:16:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.625+0000 7f81b1697700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0cf400 con 0x7f81ac06e340 2024-01-11T07:16:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.625+0000 7f81aaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f81ac159ca0 0x7f81ac06dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36880/0 (socket says 172.21.15.7:36880) 2024-01-11T07:16:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.625+0000 7f81aaffd700 1 -- 172.21.15.7:0/1626646238 learned_addr learned my addr 172.21.15.7:0/1626646238 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81aa7fc700 1 --2- 172.21.15.7:0/1626646238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac06e340 0x7f81ac0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81a9ffb700 1 -- 172.21.15.7:0/1626646238 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835741862 0 0) 0x7f81ac0d18c0 con 0x7f81ac159ca0 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81a9ffb700 1 -- 172.21.15.7:0/1626646238 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8198003570 con 0x7f81ac159ca0 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81a9ffb700 1 -- 172.21.15.7:0/1626646238 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578086225 0 0) 0x7f81ac0d0010 con 0x7f81ac15a070 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81a9ffb700 1 -- 172.21.15.7:0/1626646238 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ac0d18c0 con 0x7f81ac15a070 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81a9ffb700 1 -- 172.21.15.7:0/1626646238 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2292966561 0 0) 0x7f8198003570 con 0x7f81ac159ca0 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81aa7fc700 1 -- 172.21.15.7:0/1626646238 >> v1:172.21.15.113:6789/0 conn(0x7f81ac15a070 legacy=0x7f81ac0b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81aa7fc700 1 -- 172.21.15.7:0/1626646238 >> v1:172.21.15.7:6789/0 conn(0x7f81ac159ca0 legacy=0x7f81ac06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81aa7fc700 1 -- 172.21.15.7:0/1626646238 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac0cf6e0 con 0x7f81ac06e340 2024-01-11T07:16:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81aa7fc700 1 --2- 172.21.15.7:0/1626646238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac06e340 0x7f81ac0ce7f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8194009910 tx=0x7f81940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1dc7b23bcc2cea2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81a9ffb700 1 -- 172.21.15.7:0/1626646238 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c001470 con 0x7f81ac06e340 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81a9ffb700 1 -- 172.21.15.7:0/1626646238 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194011420 con 0x7f81ac06e340 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.626+0000 7f81a9ffb700 1 -- 172.21.15.7:0/1626646238 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940117c0 con 0x7f81ac06e340 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.627+0000 7f81b1697700 1 -- 172.21.15.7:0/1626646238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac06e340 msgr2=0x7f81ac0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.627+0000 7f81b1697700 1 --2- 172.21.15.7:0/1626646238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac06e340 0x7f81ac0ce7f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8194009910 tx=0x7f81940091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.627+0000 7f81b1697700 1 -- 172.21.15.7:0/1626646238 shutdown_connections 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.627+0000 7f81b1697700 1 --2- 172.21.15.7:0/1626646238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac06e340 0x7f81ac0ce7f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.627+0000 7f81b1697700 1 -- 172.21.15.7:0/1626646238 >> 172.21.15.7:0/1626646238 conn(0x7f81ac063cc0 msgr2=0x7f81ac0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.627+0000 7f81b1697700 1 -- 172.21.15.7:0/1626646238 shutdown_connections 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.627+0000 7f81b1697700 1 -- 172.21.15.7:0/1626646238 wait complete. 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.628+0000 7f81b1697700 1 Processor -- start 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81b1697700 1 -- start start 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81b1697700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ac06e340 0x7f81ac0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81b1697700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac159ca0 0x7f81ac15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81b1697700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ac15a070 0x7f81ac15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81b1697700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0b65f0 con 0x7f81ac06e340 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81b1697700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0b5cc0 con 0x7f81ac15a070 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81b1697700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0b5fa0 con 0x7f81ac159ca0 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ac06e340 0x7f81ac0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81aaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ac06e340 0x7f81ac0b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48956/0 (socket says 172.21.15.7:48956) 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81aaffd700 1 -- 172.21.15.7:0/2015326463 learned_addr learned my addr 172.21.15.7:0/2015326463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.629+0000 7f81ab7fe700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ac15a070 0x7f81ac15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81aa7fc700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac159ca0 0x7f81ac15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81aaffd700 1 -- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac159ca0 msgr2=0x7f81ac15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81aaffd700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac159ca0 0x7f81ac15c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81aaffd700 1 -- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ac15a070 msgr2=0x7f81ac15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81aaffd700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ac15a070 0x7f81ac15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81aaffd700 1 -- 172.21.15.7:0/2015326463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8194009420 con 0x7f81ac06e340 2024-01-11T07:16:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81aaffd700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ac06e340 0x7f81ac0b2410 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f819c00cb70 tx=0x7f819c001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81a2ffd700 1 -- 172.21.15.7:0/2015326463 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c007150 con 0x7f81ac06e340 2024-01-11T07:16:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81a2ffd700 1 -- 172.21.15.7:0/2015326463 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c0072d0 con 0x7f81ac06e340 2024-01-11T07:16:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac0b6280 con 0x7f81ac06e340 2024-01-11T07:16:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81a2ffd700 1 -- 172.21.15.7:0/2015326463 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c007690 con 0x7f81ac06e340 2024-01-11T07:16:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.630+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac0b5970 con 0x7f81ac06e340 2024-01-11T07:16:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.631+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac04c3c0 con 0x7f81ac06e340 2024-01-11T07:16:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.632+0000 7f81a2ffd700 1 -- 172.21.15.7:0/2015326463 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f819c004a40 con 0x7f81ac06e340 2024-01-11T07:16:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.632+0000 7f81a2ffd700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f818c066f30 0x7f818c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.632+0000 7f81a2ffd700 1 -- 172.21.15.7:0/2015326463 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f819c083920 con 0x7f81ac06e340 2024-01-11T07:16:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.632+0000 7f81aa7fc700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f818c066f30 0x7f818c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.633+0000 7f81aa7fc700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f818c066f30 0x7f818c0693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f81940030c0 tx=0x7f8194016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.637+0000 7f81a2ffd700 1 -- 172.21.15.7:0/2015326463 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f819c050f10 con 0x7f81ac06e340 2024-01-11T07:16:28.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:28.932+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f81ac0b68d0 con 0x7f81ac06e340 2024-01-11T07:16:29.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.469+0000 7f81a2ffd700 1 -- 172.21.15.7:0/2015326463 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v452) v1 ==== 134+0+0 (secure 0 0 0) 0x7f819c018020 con 0x7f81ac06e340 2024-01-11T07:16:29.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f818c066f30 msgr2=0x7f818c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:29.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f818c066f30 0x7f818c0693e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f81940030c0 tx=0x7f8194016000 comp rx=0 tx=0).stop 2024-01-11T07:16:29.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ac06e340 msgr2=0x7f81ac0b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:29.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ac06e340 0x7f81ac0b2410 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f819c00cb70 tx=0x7f819c001ab0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 shutdown_connections 2024-01-11T07:16:29.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f818c066f30 0x7f818c0693e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81ac15a070 0x7f81ac15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81ac06e340 0x7f81ac0b2410 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 --2- 172.21.15.7:0/2015326463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f81ac159ca0 0x7f81ac15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 >> 172.21.15.7:0/2015326463 conn(0x7f81ac063cc0 msgr2=0x7f81ac06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:29.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 shutdown_connections 2024-01-11T07:16:29.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.472+0000 7f81b1697700 1 -- 172.21.15.7:0/2015326463 wait complete. 2024-01-11T07:16:29.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:16:29.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.626+0000 7f3366310700 1 Processor -- start 2024-01-11T07:16:29.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.626+0000 7f3366310700 1 -- start start 2024-01-11T07:16:29.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f3366310700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3360155720 0x7f33601580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:29.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f3366310700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f336006b5e0 0x7f33600ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:29.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f3366310700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33600cb680 con 0x7f336006baa0 2024-01-11T07:16:29.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f3366310700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3360155af0 con 0x7f3360155720 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f3366310700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33600caa70 con 0x7f336006b5e0 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f335ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3360155720 0x7f33601580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f335ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3360155720 0x7f33601580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48964/0 (socket says 172.21.15.7:48964) 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f335ffff700 1 -- 172.21.15.7:0/203060407 learned_addr learned my addr 172.21.15.7:0/203060407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f33648ad700 1 --2- 172.21.15.7:0/203060407 >> v2:172.21.15.113:3300/0 conn(0x7f336006b5e0 0x7f33600ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f335ffff700 1 -- 172.21.15.7:0/203060407 >> v2:172.21.15.113:3300/0 conn(0x7f336006b5e0 msgr2=0x7f33600ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f335ffff700 1 --2- 172.21.15.7:0/203060407 >> v2:172.21.15.113:3300/0 conn(0x7f336006b5e0 0x7f33600ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f335ffff700 1 -- 172.21.15.7:0/203060407 >> v1:172.21.15.113:6789/0 conn(0x7f336006baa0 legacy=0x7f336006aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.627+0000 7f335ffff700 1 -- 172.21.15.7:0/203060407 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33600cad50 con 0x7f3360155720 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f335ffff700 1 --2- 172.21.15.7:0/203060407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3360155720 0x7f33601580e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f33600c90c0 tx=0x7f3350009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b495339a8cd140 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f335effd700 1 -- 172.21.15.7:0/203060407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335000b860 con 0x7f3360155720 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f335effd700 1 -- 172.21.15.7:0/203060407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335000b9c0 con 0x7f3360155720 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f335effd700 1 -- 172.21.15.7:0/203060407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335000bd60 con 0x7f3360155720 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f3366310700 1 -- 172.21.15.7:0/203060407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3360155720 msgr2=0x7f33601580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:29.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f3366310700 1 --2- 172.21.15.7:0/203060407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3360155720 0x7f33601580e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f33600c90c0 tx=0x7f3350009250 comp rx=0 tx=0).stop 2024-01-11T07:16:29.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f3366310700 1 -- 172.21.15.7:0/203060407 shutdown_connections 2024-01-11T07:16:29.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f3366310700 1 --2- 172.21.15.7:0/203060407 >> v2:172.21.15.113:3300/0 conn(0x7f336006b5e0 0x7f33600ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f3366310700 1 --2- 172.21.15.7:0/203060407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3360155720 0x7f33601580e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f3366310700 1 -- 172.21.15.7:0/203060407 >> 172.21.15.7:0/203060407 conn(0x7f336006ed90 msgr2=0x7f336006f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:29.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f3366310700 1 -- 172.21.15.7:0/203060407 shutdown_connections 2024-01-11T07:16:29.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.628+0000 7f3366310700 1 -- 172.21.15.7:0/203060407 wait complete. 2024-01-11T07:16:29.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.630+0000 7f3366310700 1 Processor -- start 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.630+0000 7f3366310700 1 -- start start 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f3366310700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336006b5e0 0x7f336014cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f3366310700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336006baa0 0x7f336014a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f3366310700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3360155720 0x7f33600c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f3366310700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336014abb0 con 0x7f336006baa0 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f3366310700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33600c85e0 con 0x7f3360155720 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f3366310700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336014bfa0 con 0x7f336006b5e0 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336006baa0 0x7f336014a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336006baa0 0x7f336014a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48974/0 (socket says 172.21.15.7:48974) 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 -- 172.21.15.7:0/861880799 learned_addr learned my addr 172.21.15.7:0/861880799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f33648ad700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3360155720 0x7f33600c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335ffff700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336006b5e0 0x7f336014cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:29.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 -- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336006b5e0 msgr2=0x7f336014cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336006b5e0 0x7f336014cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 -- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3360155720 msgr2=0x7f33600c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3360155720 0x7f33600c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 -- 172.21.15.7:0/861880799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33500094f0 con 0x7f336006baa0 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f335f7fe700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336006baa0 0x7f336014a670 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f334800c590 tx=0x7f334800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.631+0000 7f3357fff700 1 -- 172.21.15.7:0/861880799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348011590 con 0x7f336006baa0 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.632+0000 7f3357fff700 1 -- 172.21.15.7:0/861880799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33480116f0 con 0x7f336006baa0 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.632+0000 7f3357fff700 1 -- 172.21.15.7:0/861880799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348011ab0 con 0x7f336006baa0 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.632+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336014c280 con 0x7f336006baa0 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.632+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336014c840 con 0x7f336006baa0 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.633+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336004c3c0 con 0x7f336006baa0 2024-01-11T07:16:29.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.633+0000 7f3357fff700 1 -- 172.21.15.7:0/861880799 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f334801b3b0 con 0x7f336006baa0 2024-01-11T07:16:29.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.633+0000 7f3357fff700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3340066f30 0x7f33400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:29.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.634+0000 7f3357fff700 1 -- 172.21.15.7:0/861880799 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f33480837e0 con 0x7f336006baa0 2024-01-11T07:16:29.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.634+0000 7f335ffff700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3340066f30 0x7f33400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:29.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.634+0000 7f335ffff700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3340066f30 0x7f33400693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f33600c90c0 tx=0x7f3350017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:29.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.638+0000 7f3357fff700 1 -- 172.21.15.7:0/861880799 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3348050d50 con 0x7f336006baa0 2024-01-11T07:16:29.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.848+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f33600cb030 con 0x7f336006baa0 2024-01-11T07:16:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.851+0000 7f3357fff700 1 -- 172.21.15.7:0/861880799 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3348017020 con 0x7f336006baa0 2024-01-11T07:16:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3340066f30 msgr2=0x7f33400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3340066f30 0x7f33400693e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f33600c90c0 tx=0x7f3350017000 comp rx=0 tx=0).stop 2024-01-11T07:16:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336006baa0 msgr2=0x7f336014a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336006baa0 0x7f336014a670 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f334800c590 tx=0x7f334800c950 comp rx=0 tx=0).stop 2024-01-11T07:16:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 shutdown_connections 2024-01-11T07:16:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3340066f30 0x7f33400693e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3360155720 0x7f33600c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f336006b5e0 0x7f336014cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 --2- 172.21.15.7:0/861880799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336006baa0 0x7f336014a670 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 >> 172.21.15.7:0/861880799 conn(0x7f336006ed90 msgr2=0x7f33600b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 shutdown_connections 2024-01-11T07:16:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:29.854+0000 7f3366310700 1 -- 172.21.15.7:0/861880799 wait complete. 2024-01-11T07:16:29.867 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:16:29.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:16:29.920 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:16:29.974 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:16:29.974 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:16:29.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:16:30.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.113+0000 7f2e943b9700 1 Processor -- start 2024-01-11T07:16:30.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.113+0000 7f2e943b9700 1 -- start start 2024-01-11T07:16:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e943b9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c0c7210 0x7f2e8c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e943b9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e8c15dec0 0x7f2e8c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e943b9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c06dc00 con 0x7f2e8c0c6e40 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e943b9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c06e4a0 con 0x7f2e8c15dec0 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e943b9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c15e6e0 con 0x7f2e8c0c7210 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e92956700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e8c15dec0 0x7f2e8c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e92155700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2e8c0c6e40 0x7f2e8c06e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36894/0 (socket says 172.21.15.7:36894) 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e92155700 1 -- 172.21.15.7:0/1943377614 learned_addr learned my addr 172.21.15.7:0/1943377614 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e91954700 1 --2- 172.21.15.7:0/1943377614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c0c7210 0x7f2e8c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e92956700 1 -- 172.21.15.7:0/1943377614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c0c7210 msgr2=0x7f2e8c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e92956700 1 --2- 172.21.15.7:0/1943377614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c0c7210 0x7f2e8c15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e92956700 1 -- 172.21.15.7:0/1943377614 >> v1:172.21.15.7:6789/0 conn(0x7f2e8c0c6e40 legacy=0x7f2e8c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e91153700 1 -- 172.21.15.7:0/1943377614 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929605464 0 0) 0x7f2e8c06dc00 con 0x7f2e8c0c6e40 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.114+0000 7f2e92956700 1 -- 172.21.15.7:0/1943377614 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c06de50 con 0x7f2e8c15dec0 2024-01-11T07:16:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e92956700 1 --2- 172.21.15.7:0/1943377614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c15dec0 0x7f2e8c15b4a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009e60 tx=0x7f2e84009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f67482e6dcb0ace server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e91153700 1 -- 172.21.15.7:0/1943377614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8400e070 con 0x7f2e8c15dec0 2024-01-11T07:16:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e91153700 1 -- 172.21.15.7:0/1943377614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8400bb60 con 0x7f2e8c15dec0 2024-01-11T07:16:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e91153700 1 -- 172.21.15.7:0/1943377614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84012430 con 0x7f2e8c15dec0 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e943b9700 1 -- 172.21.15.7:0/1943377614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c15dec0 msgr2=0x7f2e8c15b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1943377614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c15dec0 0x7f2e8c15b4a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009e60 tx=0x7f2e84009780 comp rx=0 tx=0).stop 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e943b9700 1 -- 172.21.15.7:0/1943377614 shutdown_connections 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1943377614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c15dec0 0x7f2e8c15b4a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1943377614 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c0c7210 0x7f2e8c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.115+0000 7f2e943b9700 1 -- 172.21.15.7:0/1943377614 >> 172.21.15.7:0/1943377614 conn(0x7f2e8c067610 msgr2=0x7f2e8c067a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.116+0000 7f2e943b9700 1 -- 172.21.15.7:0/1943377614 shutdown_connections 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.116+0000 7f2e943b9700 1 -- 172.21.15.7:0/1943377614 wait complete. 2024-01-11T07:16:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.117+0000 7f2e943b9700 1 Processor -- start 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.117+0000 7f2e943b9700 1 -- start start 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.117+0000 7f2e943b9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8c0c6e40 0x7f2e8c148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.117+0000 7f2e943b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c0c7210 0x7f2e8c147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.117+0000 7f2e943b9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c15dec0 0x7f2e8c1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.117+0000 7f2e943b9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c147750 con 0x7f2e8c0c7210 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.117+0000 7f2e943b9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c1569f0 con 0x7f2e8c0c6e40 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.117+0000 7f2e943b9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c16d730 con 0x7f2e8c15dec0 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c0c7210 0x7f2e8c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c0c7210 0x7f2e8c147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48994/0 (socket says 172.21.15.7:48994) 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 -- 172.21.15.7:0/1373214825 learned_addr learned my addr 172.21.15.7:0/1373214825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e92956700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c15dec0 0x7f2e8c1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e92155700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8c0c6e40 0x7f2e8c148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 -- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c15dec0 msgr2=0x7f2e8c1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c15dec0 0x7f2e8c1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 -- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8c0c6e40 msgr2=0x7f2e8c148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8c0c6e40 0x7f2e8c148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 -- 172.21.15.7:0/1373214825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e84009a20 con 0x7f2e8c0c7210 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e92155700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8c0c6e40 0x7f2e8c148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e92956700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c15dec0 0x7f2e8c1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e91954700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c0c7210 0x7f2e8c147210 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00ec80 tx=0x7f2e7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e827fc700 1 -- 172.21.15.7:0/1373214825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00c970 con 0x7f2e8c0c7210 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e827fc700 1 -- 172.21.15.7:0/1373214825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c00cad0 con 0x7f2e8c0c7210 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c16da10 con 0x7f2e8c0c7210 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e827fc700 1 -- 172.21.15.7:0/1373214825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00ce70 con 0x7f2e8c0c7210 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.118+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c16dfd0 con 0x7f2e8c0c7210 2024-01-11T07:16:30.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.119+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c04c3c0 con 0x7f2e8c0c7210 2024-01-11T07:16:30.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.120+0000 7f2e827fc700 1 -- 172.21.15.7:0/1373214825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2e7c013680 con 0x7f2e8c0c7210 2024-01-11T07:16:30.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.120+0000 7f2e827fc700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e70066f30 0x7f2e700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.121+0000 7f2e827fc700 1 -- 172.21.15.7:0/1373214825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2e7c0845c0 con 0x7f2e8c0c7210 2024-01-11T07:16:30.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.121+0000 7f2e92155700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e70066f30 0x7f2e700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.121+0000 7f2e92155700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e70066f30 0x7f2e700693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2e88007fd0 tx=0x7f2e88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:30.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.125+0000 7f2e827fc700 1 -- 172.21.15.7:0/1373214825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e7c051a90 con 0x7f2e8c0c7210 2024-01-11T07:16:30.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.377+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e8c06e130 con 0x7f2e8c0c7210 2024-01-11T07:16:30.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.378+0000 7f2e827fc700 1 -- 172.21.15.7:0/1373214825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f2e7c017de0 con 0x7f2e8c0c7210 2024-01-11T07:16:30.378 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:30.378 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1458931008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1458931008},{"type":"v1","addr":"172.21.15.113:6836","nonce":1458931008}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3377988669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3377988669},{"type":"v1","addr":"172.21.15.113:6837","nonce":3377988669}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7416,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1420951490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1420951490},{"type":"v1","addr":"172.21.15.7:6833","nonce":1420951490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7671,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/590657684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":590657684},{"type":"v1","addr":"172.21.15.7:6835","nonce":590657684}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[]} 2024-01-11T07:16:30.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e70066f30 msgr2=0x7f2e700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e70066f30 0x7f2e700693e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2e88007fd0 tx=0x7f2e88009040 comp rx=0 tx=0).stop 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c0c7210 msgr2=0x7f2e8c147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c0c7210 0x7f2e8c147210 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00ec80 tx=0x7f2e7c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 shutdown_connections 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e70066f30 0x7f2e700693e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8c15dec0 0x7f2e8c1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e8c0c6e40 0x7f2e8c148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 --2- 172.21.15.7:0/1373214825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e8c0c7210 0x7f2e8c147210 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 >> 172.21.15.7:0/1373214825 conn(0x7f2e8c067610 msgr2=0x7f2e8c0b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 shutdown_connections 2024-01-11T07:16:30.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.380+0000 7f2e943b9700 1 -- 172.21.15.7:0/1373214825 wait complete. 2024-01-11T07:16:30.382 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-01-11T07:16:30.394 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:16:30.394 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:16:30.394 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:16:30.394 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:16:30.395 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:16:30.395 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:16:30.395 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:16:30.395 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:16:30.395 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:16:30.395 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:16:30.395 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:16:30.395 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:16:30.396 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:16:30.396 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:16:30.396 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:16:30.396 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:16:30.404 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:16:30.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:16:30.405 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:16:30.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:16:30.407 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:16:30.407 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:16:30.408 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:16:30.408 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:16:30.447 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:16:30.447 DEBUG:teuthology.parallel:result is None 2024-01-11T07:16:30.448 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:16:30.449 DEBUG:teuthology.parallel:result is None 2024-01-11T07:16:30.449 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:16:30.450 DEBUG:teuthology.parallel:result is None 2024-01-11T07:16:30.450 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:16:30.450 DEBUG:teuthology.parallel:result is None 2024-01-11T07:16:30.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:16:30.531 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:16:30.531 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:16:30.537 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:16:30.545 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:16:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.601+0000 7f46fb2aa700 1 Processor -- start 2024-01-11T07:16:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.601+0000 7f46fb2aa700 1 -- start start 2024-01-11T07:16:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.602+0000 7f46fb2aa700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46f41579d0 0x7f46f415c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.602+0000 7f46fb2aa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46f40ce3f0 0x7f46f40ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.602+0000 7f46fb2aa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f406d270 con 0x7f46f4157da0 2024-01-11T07:16:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.602+0000 7f46fb2aa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f40cda40 con 0x7f46f40ce3f0 2024-01-11T07:16:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.602+0000 7f46fb2aa700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f40cd5b0 con 0x7f46f41579d0 2024-01-11T07:16:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.602+0000 7f46f8845700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f46f4157da0 0x7f46f40cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36906/0 (socket says 172.21.15.7:36906) 2024-01-11T07:16:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46f8845700 1 -- 172.21.15.7:0/172248601 learned_addr learned my addr 172.21.15.7:0/172248601 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46f9847700 1 --2- 172.21.15.7:0/172248601 >> v2:172.21.15.7:3300/0 conn(0x7f46f40ce3f0 0x7f46f40ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46f9046700 1 --2- 172.21.15.7:0/172248601 >> v2:172.21.15.113:3300/0 conn(0x7f46f41579d0 0x7f46f415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46e7fff700 1 -- 172.21.15.7:0/172248601 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129315494 0 0) 0x7f46f406d270 con 0x7f46f4157da0 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46e7fff700 1 -- 172.21.15.7:0/172248601 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e0003570 con 0x7f46f4157da0 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46f9847700 1 -- 172.21.15.7:0/172248601 >> v2:172.21.15.113:3300/0 conn(0x7f46f41579d0 msgr2=0x7f46f415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46f9847700 1 --2- 172.21.15.7:0/172248601 >> v2:172.21.15.113:3300/0 conn(0x7f46f41579d0 0x7f46f415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46f9847700 1 -- 172.21.15.7:0/172248601 >> v1:172.21.15.7:6789/0 conn(0x7f46f4157da0 legacy=0x7f46f40cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46f9847700 1 -- 172.21.15.7:0/172248601 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40cc9a0 con 0x7f46f40ce3f0 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46f9847700 1 --2- 172.21.15.7:0/172248601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40ca500 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f46f406c560 tx=0x7f46ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d91d841add4b640 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46e7fff700 1 -- 172.21.15.7:0/172248601 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00e070 con 0x7f46f40ce3f0 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46e7fff700 1 -- 172.21.15.7:0/172248601 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec00bab0 con 0x7f46f40ce3f0 2024-01-11T07:16:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.603+0000 7f46e7fff700 1 -- 172.21.15.7:0/172248601 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00be70 con 0x7f46f40ce3f0 2024-01-11T07:16:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.604+0000 7f46fb2aa700 1 -- 172.21.15.7:0/172248601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 msgr2=0x7f46f40ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.604+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/172248601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40ca500 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f46f406c560 tx=0x7f46ec009780 comp rx=0 tx=0).stop 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.604+0000 7f46fb2aa700 1 -- 172.21.15.7:0/172248601 shutdown_connections 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.604+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/172248601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40ca500 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.604+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/172248601 >> v2:172.21.15.113:3300/0 conn(0x7f46f41579d0 0x7f46f415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.604+0000 7f46fb2aa700 1 -- 172.21.15.7:0/172248601 >> 172.21.15.7:0/172248601 conn(0x7f46f406ae00 msgr2=0x7f46f406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.604+0000 7f46fb2aa700 1 -- 172.21.15.7:0/172248601 shutdown_connections 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.604+0000 7f46fb2aa700 1 -- 172.21.15.7:0/172248601 wait complete. 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.605+0000 7f46fb2aa700 1 Processor -- start 2024-01-11T07:16:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.605+0000 7f46fb2aa700 1 -- start start 2024-01-11T07:16:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46fb2aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46fb2aa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46f41579d0 0x7f46f41544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46fb2aa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f4157da0 0x7f46f41549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46fb2aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4167630 con 0x7f46f40ce3f0 2024-01-11T07:16:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46fb2aa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4167840 con 0x7f46f4157da0 2024-01-11T07:16:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46fb2aa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4167b20 con 0x7f46f41579d0 2024-01-11T07:16:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46f9046700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46f9046700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49042/0 (socket says 172.21.15.7:49042) 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.606+0000 7f46f9046700 1 -- 172.21.15.7:0/4033451273 learned_addr learned my addr 172.21.15.7:0/4033451273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f9847700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f4157da0 0x7f46f41549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f8845700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46f41579d0 0x7f46f41544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f9046700 1 -- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46f41579d0 msgr2=0x7f46f41544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f9046700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46f41579d0 0x7f46f41544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f9046700 1 -- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f4157da0 msgr2=0x7f46f41549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f9046700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f4157da0 0x7f46f41549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f9046700 1 -- 172.21.15.7:0/4033451273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec009a20 con 0x7f46f40ce3f0 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f9847700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f4157da0 0x7f46f41549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46f9046700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40b1ee0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f46e800ec80 tx=0x7f46e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46e57fa700 1 -- 172.21.15.7:0/4033451273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e800c9d0 con 0x7f46f40ce3f0 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46e57fa700 1 -- 172.21.15.7:0/4033451273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e800cb30 con 0x7f46f40ce3f0 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f4167e00 con 0x7f46f40ce3f0 2024-01-11T07:16:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46e57fa700 1 -- 172.21.15.7:0/4033451273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8013470 con 0x7f46f40ce3f0 2024-01-11T07:16:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.607+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f41683c0 con 0x7f46f40ce3f0 2024-01-11T07:16:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.608+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f404c3c0 con 0x7f46f40ce3f0 2024-01-11T07:16:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.609+0000 7f46e57fa700 1 -- 172.21.15.7:0/4033451273 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f46e8013820 con 0x7f46f40ce3f0 2024-01-11T07:16:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.609+0000 7f46e57fa700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f46d8066f30 0x7f46d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.609+0000 7f46e57fa700 1 -- 172.21.15.7:0/4033451273 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f46e80845c0 con 0x7f46f40ce3f0 2024-01-11T07:16:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.609+0000 7f46f8845700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f46d8066f30 0x7f46d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.610+0000 7f46f8845700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f46d8066f30 0x7f46d80693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f46f00058d0 tx=0x7f46f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.613+0000 7f46e57fa700 1 -- 172.21.15.7:0/4033451273 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46e8051b30 con 0x7f46f40ce3f0 2024-01-11T07:16:30.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:30.830+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f46f40ccc80 con 0x7f46f40ce3f0 2024-01-11T07:16:31.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:16:31.727 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:16:31.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.864+0000 7f46e57fa700 1 -- 172.21.15.7:0/4033451273 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v164) v1 ==== 97+0+0 (secure 0 0 0) 0x7f46e8055150 con 0x7f46f40ce3f0 2024-01-11T07:16:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f46d8066f30 msgr2=0x7f46d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f46d8066f30 0x7f46d80693e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f46f00058d0 tx=0x7f46f0009040 comp rx=0 tx=0).stop 2024-01-11T07:16:31.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 msgr2=0x7f46f40b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40b1ee0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f46e800ec80 tx=0x7f46e80091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 shutdown_connections 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f46d8066f30 0x7f46d80693e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f4157da0 0x7f46f41549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46f40ce3f0 0x7f46f40b1ee0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 --2- 172.21.15.7:0/4033451273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46f41579d0 0x7f46f41544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 >> 172.21.15.7:0/4033451273 conn(0x7f46f406ae00 msgr2=0x7f46f41532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 shutdown_connections 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:31.867+0000 7f46fb2aa700 1 -- 172.21.15.7:0/4033451273 wait complete. 2024-01-11T07:16:31.868 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:16:31.883 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:16:32.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.019+0000 7f7401158700 1 Processor -- start 2024-01-11T07:16:32.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.019+0000 7f7401158700 1 -- start start 2024-01-11T07:16:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f7401158700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f73fc157da0 0x7f73fc0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f7401158700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc0ce220 0x7f73fc0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f7401158700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc06d270 con 0x7f73fc1579d0 2024-01-11T07:16:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f7401158700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc0cda40 con 0x7f73fc157da0 2024-01-11T07:16:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f7401158700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0cd5b0 con 0x7f73fc0ce220 2024-01-11T07:16:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f73fad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73fc1579d0 0x7f73fc15c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36912/0 (socket says 172.21.15.7:36912) 2024-01-11T07:16:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f73fad9d700 1 -- 172.21.15.7:0/2029203510 learned_addr learned my addr 172.21.15.7:0/2029203510 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f73fb59e700 1 --2- 172.21.15.7:0/2029203510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc0ce220 0x7f73fc0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f73fa59c700 1 --2- 172.21.15.7:0/2029203510 >> v2:172.21.15.113:3300/0 conn(0x7f73fc157da0 0x7f73fc0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f73f9d9b700 1 -- 172.21.15.7:0/2029203510 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749770898 0 0) 0x7f73fc06d270 con 0x7f73fc1579d0 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f73f9d9b700 1 -- 172.21.15.7:0/2029203510 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e4003570 con 0x7f73fc1579d0 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.020+0000 7f73f9d9b700 1 -- 172.21.15.7:0/2029203510 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3063766988 0 0) 0x7f73e4003570 con 0x7f73fc1579d0 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73fb59e700 1 -- 172.21.15.7:0/2029203510 >> v2:172.21.15.113:3300/0 conn(0x7f73fc157da0 msgr2=0x7f73fc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73fb59e700 1 --2- 172.21.15.7:0/2029203510 >> v2:172.21.15.113:3300/0 conn(0x7f73fc157da0 0x7f73fc0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73fb59e700 1 -- 172.21.15.7:0/2029203510 >> v1:172.21.15.7:6789/0 conn(0x7f73fc1579d0 legacy=0x7f73fc15c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73fb59e700 1 -- 172.21.15.7:0/2029203510 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc0ce660 con 0x7f73fc0ce220 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73fa59c700 1 --2- 172.21.15.7:0/2029203510 >> v2:172.21.15.113:3300/0 conn(0x7f73fc157da0 0x7f73fc0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73fb59e700 1 --2- 172.21.15.7:0/2029203510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc0ce220 0x7f73fc0ca500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0ce7f0 tx=0x7f73f4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e96b8f07834bf03c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73f9d9b700 1 -- 172.21.15.7:0/2029203510 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400e070 con 0x7f73fc0ce220 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73f9d9b700 1 -- 172.21.15.7:0/2029203510 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f400b7a0 con 0x7f73fc0ce220 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f73f9d9b700 1 -- 172.21.15.7:0/2029203510 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec001470 con 0x7f73fc0ce220 2024-01-11T07:16:32.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f7401158700 1 -- 172.21.15.7:0/2029203510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc0ce220 msgr2=0x7f73fc0ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.021+0000 7f7401158700 1 --2- 172.21.15.7:0/2029203510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc0ce220 0x7f73fc0ca500 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0ce7f0 tx=0x7f73f4009780 comp rx=0 tx=0).stop 2024-01-11T07:16:32.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.022+0000 7f7401158700 1 -- 172.21.15.7:0/2029203510 shutdown_connections 2024-01-11T07:16:32.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.022+0000 7f7401158700 1 --2- 172.21.15.7:0/2029203510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc0ce220 0x7f73fc0ca500 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.022+0000 7f7401158700 1 --2- 172.21.15.7:0/2029203510 >> v2:172.21.15.113:3300/0 conn(0x7f73fc157da0 0x7f73fc0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.022+0000 7f7401158700 1 -- 172.21.15.7:0/2029203510 >> 172.21.15.7:0/2029203510 conn(0x7f73fc06ae00 msgr2=0x7f73fc06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:32.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.022+0000 7f7401158700 1 -- 172.21.15.7:0/2029203510 shutdown_connections 2024-01-11T07:16:32.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.022+0000 7f7401158700 1 -- 172.21.15.7:0/2029203510 wait complete. 2024-01-11T07:16:32.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.023+0000 7f7401158700 1 Processor -- start 2024-01-11T07:16:32.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.024+0000 7f7401158700 1 -- start start 2024-01-11T07:16:32.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.024+0000 7f7401158700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73fc0ce220 0x7f73fc143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.024+0000 7f7401158700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc1579d0 0x7f73fc0c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.024+0000 7f7401158700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73fc157da0 0x7f73fc155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f7401158700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0c9060 con 0x7f73fc0ce220 2024-01-11T07:16:32.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f7401158700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0c8450 con 0x7f73fc157da0 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f7401158700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0c8730 con 0x7f73fc1579d0 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73fc0ce220 0x7f73fc143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73fc0ce220 0x7f73fc143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49052/0 (socket says 172.21.15.7:49052) 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 -- 172.21.15.7:0/3138830144 learned_addr learned my addr 172.21.15.7:0/3138830144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fa59c700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc1579d0 0x7f73fc0c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fb59e700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73fc157da0 0x7f73fc155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 -- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc1579d0 msgr2=0x7f73fc0c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc1579d0 0x7f73fc0c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 -- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73fc157da0 msgr2=0x7f73fc155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73fc157da0 0x7f73fc155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 -- 172.21.15.7:0/3138830144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0009420 con 0x7f73fc0ce220 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73fad9d700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73fc0ce220 0x7f73fc143560 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00cb90 tx=0x7f73ec001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73e2ffd700 1 -- 172.21.15.7:0/3138830144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec007150 con 0x7f73fc0ce220 2024-01-11T07:16:32.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73e2ffd700 1 -- 172.21.15.7:0/3138830144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec0072b0 con 0x7f73fc0ce220 2024-01-11T07:16:32.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f73e2ffd700 1 -- 172.21.15.7:0/3138830144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec007650 con 0x7f73fc0ce220 2024-01-11T07:16:32.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.025+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f4009a20 con 0x7f73fc0ce220 2024-01-11T07:16:32.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.026+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc0c8cf0 con 0x7f73fc0ce220 2024-01-11T07:16:32.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.027+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc04c3c0 con 0x7f73fc0ce220 2024-01-11T07:16:32.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.027+0000 7f73e2ffd700 1 -- 172.21.15.7:0/3138830144 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f73ec004a40 con 0x7f73fc0ce220 2024-01-11T07:16:32.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.028+0000 7f73e2ffd700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f73e8066f30 0x7f73e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.028+0000 7f73e2ffd700 1 -- 172.21.15.7:0/3138830144 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f73ec083970 con 0x7f73fc0ce220 2024-01-11T07:16:32.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.028+0000 7f73fa59c700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f73e8066f30 0x7f73e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.028+0000 7f73fa59c700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f73e8066f30 0x7f73e80693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f73fc16b060 tx=0x7f73f00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.032+0000 7f73e2ffd700 1 -- 172.21.15.7:0/3138830144 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73ec050fb0 con 0x7f73fc0ce220 2024-01-11T07:16:32.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.316+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f73fc0cc9a0 con 0x7f73fc0ce220 2024-01-11T07:16:32.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.316+0000 7f73e2ffd700 1 -- 172.21.15.7:0/3138830144 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f73ec018020 con 0x7f73fc0ce220 2024-01-11T07:16:32.316 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:32.319 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:16:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:16:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:16:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:16:32.319 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:16:32.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:16:32.321 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:16:32.322 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:16:32.324 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.326 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:16:32.328 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:16:32.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:16:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:16:32.332 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:16:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f73e8066f30 msgr2=0x7f73e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f73e8066f30 0x7f73e80693e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f73fc16b060 tx=0x7f73f00091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73fc0ce220 msgr2=0x7f73fc143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73fc0ce220 0x7f73fc143560 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00cb90 tx=0x7f73ec001a80 comp rx=0 tx=0).stop 2024-01-11T07:16:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 shutdown_connections 2024-01-11T07:16:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f73e8066f30 0x7f73e80693e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f73fc157da0 0x7f73fc155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73fc0ce220 0x7f73fc143560 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 --2- 172.21.15.7:0/3138830144 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f73fc1579d0 0x7f73fc0c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 >> 172.21.15.7:0/3138830144 conn(0x7f73fc06ae00 msgr2=0x7f73fc0ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:32.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 shutdown_connections 2024-01-11T07:16:32.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.319+0000 7f7401158700 1 -- 172.21.15.7:0/3138830144 wait complete. 2024-01-11T07:16:32.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:16:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.468+0000 7fd7f6479700 1 Processor -- start 2024-01-11T07:16:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.468+0000 7fd7f6479700 1 -- start start 2024-01-11T07:16:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7f6479700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7f00b0b40 0x7fd7f0162770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7f6479700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f0159300 con 0x7fd7f00cafb0 2024-01-11T07:16:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7f6479700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f00cf000 con 0x7fd7f0158280 2024-01-11T07:16:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7f6479700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f00cf260 con 0x7fd7f00b0b40 2024-01-11T07:16:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7f4a16700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7f00b0b40 0x7fd7f0162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7effff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd7f00cafb0 0x7fd7f00ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36918/0 (socket says 172.21.15.7:36918) 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7effff700 1 -- 172.21.15.7:0/566339973 learned_addr learned my addr 172.21.15.7:0/566339973 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7f4a16700 1 -- 172.21.15.7:0/566339973 >> v1:172.21.15.113:6789/0 conn(0x7fd7f0158280 legacy=0x7fd7f00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7f4a16700 1 -- 172.21.15.7:0/566339973 >> v1:172.21.15.7:6789/0 conn(0x7fd7f00cafb0 legacy=0x7fd7f00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7f4a16700 1 -- 172.21.15.7:0/566339973 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00cf540 con 0x7fd7f00b0b40 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.469+0000 7fd7eeffd700 1 -- 172.21.15.7:0/566339973 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275407279 0 0) 0x7fd7f0159300 con 0x7fd7f00cafb0 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7f4a16700 1 --2- 172.21.15.7:0/566339973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0162770 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009e60 tx=0x7fd7e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b1d1293170c067 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7eeffd700 1 -- 172.21.15.7:0/566339973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800e070 con 0x7fd7f00b0b40 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7eeffd700 1 -- 172.21.15.7:0/566339973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e800bbf0 con 0x7fd7f00b0b40 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7eeffd700 1 -- 172.21.15.7:0/566339973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8012430 con 0x7fd7f00b0b40 2024-01-11T07:16:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7f6479700 1 -- 172.21.15.7:0/566339973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 msgr2=0x7fd7f0162770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7f6479700 1 --2- 172.21.15.7:0/566339973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0162770 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009e60 tx=0x7fd7e8009780 comp rx=0 tx=0).stop 2024-01-11T07:16:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7f6479700 1 -- 172.21.15.7:0/566339973 shutdown_connections 2024-01-11T07:16:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7f6479700 1 --2- 172.21.15.7:0/566339973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0162770 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7f6479700 1 -- 172.21.15.7:0/566339973 >> 172.21.15.7:0/566339973 conn(0x7fd7f0071100 msgr2=0x7fd7f006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7f6479700 1 -- 172.21.15.7:0/566339973 shutdown_connections 2024-01-11T07:16:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.470+0000 7fd7f6479700 1 -- 172.21.15.7:0/566339973 wait complete. 2024-01-11T07:16:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.472+0000 7fd7f6479700 1 Processor -- start 2024-01-11T07:16:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.472+0000 7fd7f6479700 1 -- start start 2024-01-11T07:16:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7f6479700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0067fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7f6479700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f00cafb0 0x7fd7f014f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7f6479700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0158280 0x7fd7f01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7f6479700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00ae3d0 con 0x7fd7f00b0b40 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7f6479700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00ae5d0 con 0x7fd7f00cafb0 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7f6479700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00ae8b0 con 0x7fd7f0158280 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0067fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0067fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49066/0 (socket says 172.21.15.7:49066) 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 -- 172.21.15.7:0/2656350273 learned_addr learned my addr 172.21.15.7:0/2656350273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7f4a16700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0158280 0x7fd7f01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7ef7fe700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f00cafb0 0x7fd7f014f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 -- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0158280 msgr2=0x7fd7f01548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0158280 0x7fd7f01548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 -- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f00cafb0 msgr2=0x7fd7f014f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f00cafb0 0x7fd7f014f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 -- 172.21.15.7:0/2656350273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8009a20 con 0x7fd7f00b0b40 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7effff700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0067fd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c7e0 tx=0x7fd7e0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7d7fff700 1 -- 172.21.15.7:0/2656350273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0005b30 con 0x7fd7f00b0b40 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7d7fff700 1 -- 172.21.15.7:0/2656350273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e0005c90 con 0x7fd7f00b0b40 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.473+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f00aebf0 con 0x7fd7f00b0b40 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.474+0000 7fd7d7fff700 1 -- 172.21.15.7:0/2656350273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0007300 con 0x7fd7f00b0b40 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.474+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f0068750 con 0x7fd7f00b0b40 2024-01-11T07:16:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.475+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f004c3c0 con 0x7fd7f00b0b40 2024-01-11T07:16:32.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.475+0000 7fd7d7fff700 1 -- 172.21.15.7:0/2656350273 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd7e00076b0 con 0x7fd7f00b0b40 2024-01-11T07:16:32.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.476+0000 7fd7d7fff700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7dc066f30 0x7fd7dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.476+0000 7fd7ef7fe700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7dc066f30 0x7fd7dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.477+0000 7fd7ef7fe700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7dc066f30 0x7fd7dc0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd7f01535b0 tx=0x7fd7e4009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.477+0000 7fd7d7fff700 1 -- 172.21.15.7:0/2656350273 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd7e0054ea0 con 0x7fd7f00b0b40 2024-01-11T07:16:32.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.482+0000 7fd7d7fff700 1 -- 172.21.15.7:0/2656350273 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7e0050ae0 con 0x7fd7f00b0b40 2024-01-11T07:16:32.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.687+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f0068750 con 0x7fd7f00b0b40 2024-01-11T07:16:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.688+0000 7fd7d7fff700 1 -- 172.21.15.7:0/2656350273 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+12097 (secure 0 0 0) 0x7fd7e0017020 con 0x7fd7f00b0b40 2024-01-11T07:16:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:32.688 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":164,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:16:31.861308+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:16:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7dc066f30 msgr2=0x7fd7dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7dc066f30 0x7fd7dc0693e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd7f01535b0 tx=0x7fd7e4009060 comp rx=0 tx=0).stop 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 msgr2=0x7fd7f0067fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0067fd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c7e0 tx=0x7fd7e0007df0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 shutdown_connections 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd7dc066f30 0x7fd7dc0693e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd7f0158280 0x7fd7f01548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7f00b0b40 0x7fd7f0067fd0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 --2- 172.21.15.7:0/2656350273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f00cafb0 0x7fd7f014f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 >> 172.21.15.7:0/2656350273 conn(0x7fd7f0071100 msgr2=0x7fd7f0157070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.690+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 shutdown_connections 2024-01-11T07:16:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.691+0000 7fd7f6479700 1 -- 172.21.15.7:0/2656350273 wait complete. 2024-01-11T07:16:32.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:16:32.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:16:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.840+0000 7f6827d4f700 1 Processor -- start 2024-01-11T07:16:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.840+0000 7f6827d4f700 1 -- start start 2024-01-11T07:16:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f6827d4f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6820157c50 0x7f68200cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f6827d4f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68200cc4f0 0x7f68200cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f6827d4f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68200cce80 0x7f68200c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f6827d4f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68200b1790 con 0x7f68200cc4f0 2024-01-11T07:16:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f6827d4f700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68200b1d50 con 0x7f68200cce80 2024-01-11T07:16:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f6827d4f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68200cf840 con 0x7f6820157c50 2024-01-11T07:16:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f68252ea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68200cc4f0 0x7f68200cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f68252ea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68200cc4f0 0x7f68200cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49072/0 (socket says 172.21.15.7:49072) 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.841+0000 7f68252ea700 1 -- 172.21.15.7:0/1581746571 learned_addr learned my addr 172.21.15.7:0/1581746571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f68262ec700 1 --2- 172.21.15.7:0/1581746571 >> v2:172.21.15.113:3300/0 conn(0x7f68200cce80 0x7f68200c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f6825aeb700 1 --2- 172.21.15.7:0/1581746571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6820157c50 0x7f68200cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f68252ea700 1 -- 172.21.15.7:0/1581746571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6820157c50 msgr2=0x7f68200cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f68252ea700 1 --2- 172.21.15.7:0/1581746571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6820157c50 0x7f68200cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f68252ea700 1 -- 172.21.15.7:0/1581746571 >> v2:172.21.15.113:3300/0 conn(0x7f68200cce80 msgr2=0x7f68200c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f68252ea700 1 --2- 172.21.15.7:0/1581746571 >> v2:172.21.15.113:3300/0 conn(0x7f68200cce80 0x7f68200c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f68252ea700 1 -- 172.21.15.7:0/1581746571 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68200cd2e0 con 0x7f68200cc4f0 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f68262ec700 1 --2- 172.21.15.7:0/1581746571 >> v2:172.21.15.113:3300/0 conn(0x7f68200cce80 0x7f68200c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f6825aeb700 1 --2- 172.21.15.7:0/1581746571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6820157c50 0x7f68200cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f68252ea700 1 --2- 172.21.15.7:0/1581746571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cc4f0 0x7f68200cc940 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f682006d6e0 tx=0x7f6810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c58085f9de8374 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f6824ae9700 1 -- 172.21.15.7:0/1581746571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681000b860 con 0x7f68200cc4f0 2024-01-11T07:16:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f6824ae9700 1 -- 172.21.15.7:0/1581746571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681000b9c0 con 0x7f68200cc4f0 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.842+0000 7f6824ae9700 1 -- 172.21.15.7:0/1581746571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681000bd80 con 0x7f68200cc4f0 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 -- 172.21.15.7:0/1581746571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cc4f0 msgr2=0x7f68200cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 --2- 172.21.15.7:0/1581746571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cc4f0 0x7f68200cc940 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f682006d6e0 tx=0x7f6810009250 comp rx=0 tx=0).stop 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 -- 172.21.15.7:0/1581746571 shutdown_connections 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 --2- 172.21.15.7:0/1581746571 >> v2:172.21.15.113:3300/0 conn(0x7f68200cce80 0x7f68200c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 --2- 172.21.15.7:0/1581746571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6820157c50 0x7f68200cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 --2- 172.21.15.7:0/1581746571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cc4f0 0x7f68200cc940 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 -- 172.21.15.7:0/1581746571 >> 172.21.15.7:0/1581746571 conn(0x7f6820097a50 msgr2=0x7f68200ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 -- 172.21.15.7:0/1581746571 shutdown_connections 2024-01-11T07:16:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.843+0000 7f6827d4f700 1 -- 172.21.15.7:0/1581746571 wait complete. 2024-01-11T07:16:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.844+0000 7f6827d4f700 1 Processor -- start 2024-01-11T07:16:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.845+0000 7f6827d4f700 1 -- start start 2024-01-11T07:16:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6827d4f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68200cc4f0 0x7f6820154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6827d4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cce80 0x7f682015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6827d4f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6820157c50 0x7f682006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6827d4f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68200c8530 con 0x7f68200cce80 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6827d4f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682015a4f0 con 0x7f6820157c50 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6827d4f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6820155250 con 0x7f68200cc4f0 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cce80 0x7f682015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cce80 0x7f682015ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49078/0 (socket says 172.21.15.7:49078) 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 -- 172.21.15.7:0/826940861 learned_addr learned my addr 172.21.15.7:0/826940861 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6825aeb700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68200cc4f0 0x7f6820154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68262ec700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6820157c50 0x7f682006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 -- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68200cc4f0 msgr2=0x7f6820154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68200cc4f0 0x7f6820154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 -- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6820157c50 msgr2=0x7f682006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6820157c50 0x7f682006ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 -- 172.21.15.7:0/826940861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100094f0 con 0x7f68200cce80 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f68252ea700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cce80 0x7f682015ce40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6810006fd0 tx=0x7f6810017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6815ffb700 1 -- 172.21.15.7:0/826940861 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681001f070 con 0x7f68200cce80 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6815ffb700 1 -- 172.21.15.7:0/826940861 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681001a420 con 0x7f68200cce80 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68200c7c90 con 0x7f68200cce80 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.846+0000 7f6815ffb700 1 -- 172.21.15.7:0/826940861 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681001a7e0 con 0x7f68200cce80 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.847+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68200c8250 con 0x7f68200cce80 2024-01-11T07:16:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.848+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682015a2c0 con 0x7f68200cce80 2024-01-11T07:16:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.849+0000 7f6815ffb700 1 -- 172.21.15.7:0/826940861 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f681001ab50 con 0x7f68200cce80 2024-01-11T07:16:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.849+0000 7f6815ffb700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6804066f30 0x7f68040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.849+0000 7f6815ffb700 1 -- 172.21.15.7:0/826940861 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6810085990 con 0x7f68200cce80 2024-01-11T07:16:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.849+0000 7f6825aeb700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6804066f30 0x7f68040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:32.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.850+0000 7f6825aeb700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6804066f30 0x7f68040693e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6820155630 tx=0x7f681c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:32.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:32.853+0000 7f6815ffb700 1 -- 172.21.15.7:0/826940861 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68100530b0 con 0x7f68200cce80 2024-01-11T07:16:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.054+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f682015a2c0 con 0x7f68200cce80 2024-01-11T07:16:33.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.055+0000 7f6815ffb700 1 -- 172.21.15.7:0/826940861 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6810028020 con 0x7f68200cce80 2024-01-11T07:16:33.055 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:16:33.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6804066f30 msgr2=0x7f68040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6804066f30 0x7f68040693e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6820155630 tx=0x7f681c008040 comp rx=0 tx=0).stop 2024-01-11T07:16:33.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cce80 msgr2=0x7f682015ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cce80 0x7f682015ce40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6810006fd0 tx=0x7f6810017000 comp rx=0 tx=0).stop 2024-01-11T07:16:33.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 shutdown_connections 2024-01-11T07:16:33.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6804066f30 0x7f68040693e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6820157c50 0x7f682006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f68200cc4f0 0x7f6820154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 --2- 172.21.15.7:0/826940861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68200cce80 0x7f682015ce40 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 >> 172.21.15.7:0/826940861 conn(0x7f6820097a50 msgr2=0x7f68200ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:33.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 shutdown_connections 2024-01-11T07:16:33.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.057+0000 7f6827d4f700 1 -- 172.21.15.7:0/826940861 wait complete. 2024-01-11T07:16:33.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:16:33.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.202+0000 7ff6c1421700 1 Processor -- start 2024-01-11T07:16:33.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.202+0000 7ff6c1421700 1 -- start start 2024-01-11T07:16:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.203+0000 7ff6c1421700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.203+0000 7ff6c1421700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc159300 con 0x7ff6bc0b0b40 2024-01-11T07:16:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.203+0000 7ff6c1421700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc0cf260 con 0x7ff6bc158280 2024-01-11T07:16:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.203+0000 7ff6c1421700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc0cf540 con 0x7ff6bc0cafb0 2024-01-11T07:16:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6bb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff6bc0b0b40 0x7ff6bc161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36934/0 (socket says 172.21.15.7:36934) 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6bb7fe700 1 -- 172.21.15.7:0/1212249016 learned_addr learned my addr 172.21.15.7:0/1212249016 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6baffd700 1 --2- 172.21.15.7:0/1212249016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6b9ffb700 1 -- 172.21.15.7:0/1212249016 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173480189 0 0) 0x7ff6bc159300 con 0x7ff6bc0b0b40 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6b9ffb700 1 -- 172.21.15.7:0/1212249016 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a4003570 con 0x7ff6bc0b0b40 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6b9ffb700 1 -- 172.21.15.7:0/1212249016 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484641310 0 0) 0x7ff6bc0cf260 con 0x7ff6bc158280 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6b9ffb700 1 -- 172.21.15.7:0/1212249016 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6bc159300 con 0x7ff6bc158280 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6baffd700 1 -- 172.21.15.7:0/1212249016 >> v1:172.21.15.113:6789/0 conn(0x7ff6bc158280 legacy=0x7ff6bc0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6baffd700 1 -- 172.21.15.7:0/1212249016 >> v1:172.21.15.7:6789/0 conn(0x7ff6bc0b0b40 legacy=0x7ff6bc161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6baffd700 1 -- 172.21.15.7:0/1212249016 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0cf820 con 0x7ff6bc0cafb0 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.204+0000 7ff6baffd700 1 --2- 172.21.15.7:0/1212249016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc0ce8f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009910 tx=0x7ff6ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8badca6451bc2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6b9ffb700 1 -- 172.21.15.7:0/1212249016 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac011630 con 0x7ff6bc0cafb0 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6b9ffb700 1 -- 172.21.15.7:0/1212249016 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac011790 con 0x7ff6bc0cafb0 2024-01-11T07:16:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6b9ffb700 1 -- 172.21.15.7:0/1212249016 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac011b30 con 0x7ff6bc0cafb0 2024-01-11T07:16:33.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6c1421700 1 -- 172.21.15.7:0/1212249016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 msgr2=0x7ff6bc0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6c1421700 1 --2- 172.21.15.7:0/1212249016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc0ce8f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009910 tx=0x7ff6ac0091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6c1421700 1 -- 172.21.15.7:0/1212249016 shutdown_connections 2024-01-11T07:16:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6c1421700 1 --2- 172.21.15.7:0/1212249016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc0ce8f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6c1421700 1 -- 172.21.15.7:0/1212249016 >> 172.21.15.7:0/1212249016 conn(0x7ff6bc071100 msgr2=0x7ff6bc06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6c1421700 1 -- 172.21.15.7:0/1212249016 shutdown_connections 2024-01-11T07:16:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.205+0000 7ff6c1421700 1 -- 172.21.15.7:0/1212249016 wait complete. 2024-01-11T07:16:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.207+0000 7ff6c1421700 1 Processor -- start 2024-01-11T07:16:33.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.207+0000 7ff6c1421700 1 -- start start 2024-01-11T07:16:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.207+0000 7ff6c1421700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6bc0b0b40 0x7ff6bc0a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.207+0000 7ff6c1421700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc16b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.207+0000 7ff6c1421700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc158280 0x7ff6bc16b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6c1421700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc0b2ff0 con 0x7ff6bc158280 2024-01-11T07:16:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6c1421700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc0b2410 con 0x7ff6bc0b0b40 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6c1421700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc0b26c0 con 0x7ff6bc0cafb0 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc158280 0x7ff6bc16b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc158280 0x7ff6bc16b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49080/0 (socket says 172.21.15.7:49080) 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 -- 172.21.15.7:0/104101468 learned_addr learned my addr 172.21.15.7:0/104101468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6baffd700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6bc0b0b40 0x7ff6bc0a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6ba7fc700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc16b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 -- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 msgr2=0x7ff6bc16b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc16b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 -- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6bc0b0b40 msgr2=0x7ff6bc0a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6bc0b0b40 0x7ff6bc0a6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 -- 172.21.15.7:0/104101468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac009420 con 0x7ff6bc158280 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6bb7fe700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc158280 0x7ff6bc16b910 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4001460 tx=0x7ff6b4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6a2ffd700 1 -- 172.21.15.7:0/104101468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400a8b0 con 0x7ff6bc158280 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.208+0000 7ff6a2ffd700 1 -- 172.21.15.7:0/104101468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b400aa10 con 0x7ff6bc158280 2024-01-11T07:16:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.209+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6bc0b29a0 con 0x7ff6bc158280 2024-01-11T07:16:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.209+0000 7ff6a2ffd700 1 -- 172.21.15.7:0/104101468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400adb0 con 0x7ff6bc158280 2024-01-11T07:16:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.209+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6bc0a73c0 con 0x7ff6bc158280 2024-01-11T07:16:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.210+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6bc04c3c0 con 0x7ff6bc158280 2024-01-11T07:16:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.210+0000 7ff6a2ffd700 1 -- 172.21.15.7:0/104101468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff6b4007300 con 0x7ff6bc158280 2024-01-11T07:16:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.210+0000 7ff6a2ffd700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6a8066f30 0x7ff6a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.210+0000 7ff6a2ffd700 1 -- 172.21.15.7:0/104101468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff6b4084540 con 0x7ff6bc158280 2024-01-11T07:16:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.211+0000 7ff6baffd700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6a8066f30 0x7ff6a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.211+0000 7ff6baffd700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6a8066f30 0x7ff6a80693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc06f420 tx=0x7ff6ac017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:33.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.215+0000 7ff6a2ffd700 1 -- 172.21.15.7:0/104101468 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6b4051b80 con 0x7ff6bc158280 2024-01-11T07:16:33.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.423+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff6bc159550 con 0x7ff6bc158280 2024-01-11T07:16:33.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.423+0000 7ff6a2ffd700 1 -- 172.21.15.7:0/104101468 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v164) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff6b40551a0 con 0x7ff6bc158280 2024-01-11T07:16:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:33.423 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:16:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:16:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:16:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:16:33.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:16:33.424 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:16:33.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6a8066f30 msgr2=0x7ff6a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6a8066f30 0x7ff6a80693e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc06f420 tx=0x7ff6ac017000 comp rx=0 tx=0).stop 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc158280 msgr2=0x7ff6bc16b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc158280 0x7ff6bc16b910 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4001460 tx=0x7ff6b4001ac0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 shutdown_connections 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff6a8066f30 0x7ff6a80693e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc158280 0x7ff6bc16b910 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6bc0b0b40 0x7ff6bc0a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 --2- 172.21.15.7:0/104101468 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6bc0cafb0 0x7ff6bc16b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.425+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 >> 172.21.15.7:0/104101468 conn(0x7ff6bc071100 msgr2=0x7ff6bc155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.426+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 shutdown_connections 2024-01-11T07:16:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.426+0000 7ff6c1421700 1 -- 172.21.15.7:0/104101468 wait complete. 2024-01-11T07:16:33.439 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:16:33.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:16:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.575+0000 7f39e6b5e700 1 Processor -- start 2024-01-11T07:16:33.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.576+0000 7f39e6b5e700 1 -- start start 2024-01-11T07:16:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.576+0000 7f39e6b5e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.576+0000 7f39e6b5e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39e00c9e40 0x7f39e0063770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.576+0000 7f39e6b5e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e0159070 con 0x7f39e0157ed0 2024-01-11T07:16:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.576+0000 7f39e6b5e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e006ed70 con 0x7f39e00c9e40 2024-01-11T07:16:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.576+0000 7f39e6b5e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0158460 con 0x7f39e0157b00 2024-01-11T07:16:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39dffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f39e0157ed0 0x7f39e00c9730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36946/0 (socket says 172.21.15.7:36946) 2024-01-11T07:16:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39dffff700 1 -- 172.21.15.7:0/1811258847 learned_addr learned my addr 172.21.15.7:0/1811258847 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39e48fa700 1 --2- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39e50fb700 1 --2- 172.21.15.7:0/1811258847 >> v2:172.21.15.113:3300/0 conn(0x7f39e00c9e40 0x7f39e0063770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39df7fe700 1 -- 172.21.15.7:0/1811258847 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289792553 0 0) 0x7f39e0159070 con 0x7f39e0157ed0 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39df7fe700 1 -- 172.21.15.7:0/1811258847 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39d0003570 con 0x7f39e0157ed0 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39e50fb700 1 -- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 msgr2=0x7f39e0063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39df7fe700 1 -- 172.21.15.7:0/1811258847 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260502733 0 0) 0x7f39d0003570 con 0x7f39e0157ed0 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39e50fb700 1 --2- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39e50fb700 1 -- 172.21.15.7:0/1811258847 >> v1:172.21.15.7:6789/0 conn(0x7f39e0157ed0 legacy=0x7f39e00c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39e50fb700 1 -- 172.21.15.7:0/1811258847 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0158740 con 0x7f39e00c9e40 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39e48fa700 1 --2- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.577+0000 7f39e50fb700 1 --2- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e00c9e40 0x7f39e0063770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009e60 tx=0x7f39d8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b366b8a56c3a529 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39df7fe700 1 -- 172.21.15.7:0/1811258847 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d800e070 con 0x7f39e00c9e40 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39df7fe700 1 -- 172.21.15.7:0/1811258847 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d800b7a0 con 0x7f39e00c9e40 2024-01-11T07:16:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39df7fe700 1 -- 172.21.15.7:0/1811258847 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc001470 con 0x7f39e00c9e40 2024-01-11T07:16:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39e6b5e700 1 -- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e00c9e40 msgr2=0x7f39e0063770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e00c9e40 0x7f39e0063770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009e60 tx=0x7f39d8009780 comp rx=0 tx=0).stop 2024-01-11T07:16:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39e6b5e700 1 -- 172.21.15.7:0/1811258847 shutdown_connections 2024-01-11T07:16:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e00c9e40 0x7f39e0063770 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/1811258847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39e6b5e700 1 -- 172.21.15.7:0/1811258847 >> 172.21.15.7:0/1811258847 conn(0x7f39e00680a0 msgr2=0x7f39e00cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39e6b5e700 1 -- 172.21.15.7:0/1811258847 shutdown_connections 2024-01-11T07:16:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.578+0000 7f39e6b5e700 1 -- 172.21.15.7:0/1811258847 wait complete. 2024-01-11T07:16:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.579+0000 7f39e6b5e700 1 Processor -- start 2024-01-11T07:16:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.580+0000 7f39e6b5e700 1 -- start start 2024-01-11T07:16:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.580+0000 7f39e6b5e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e00c9e40 0x7f39e0155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e6b5e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e6b5e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e0157ed0 0x7f39e0152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e6b5e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0168060 con 0x7f39e00c9e40 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e6b5e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00c6df0 con 0x7f39e0157ed0 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e6b5e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00c70d0 con 0x7f39e0157b00 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e00c9e40 0x7f39e0155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e00c9e40 0x7f39e0155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49082/0 (socket says 172.21.15.7:49082) 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 -- 172.21.15.7:0/236193515 learned_addr learned my addr 172.21.15.7:0/236193515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39dffff700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e50fb700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e0157ed0 0x7f39e0152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 -- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 msgr2=0x7f39e0151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 -- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e0157ed0 msgr2=0x7f39e0152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e0157ed0 0x7f39e0152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 -- 172.21.15.7:0/236193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d4009420 con 0x7f39e00c9e40 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e50fb700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e0157ed0 0x7f39e0152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e48fa700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e00c9e40 0x7f39e0155460 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f39d400ec80 tx=0x7f39d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39dcff9700 1 -- 172.21.15.7:0/236193515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d400c8f0 con 0x7f39e00c9e40 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39dcff9700 1 -- 172.21.15.7:0/236193515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d400ca50 con 0x7f39e00c9e40 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d8009a20 con 0x7f39e00c9e40 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39dcff9700 1 -- 172.21.15.7:0/236193515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d400cdd0 con 0x7f39e00c9e40 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.581+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e01679d0 con 0x7f39e00c9e40 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.582+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e0168340 con 0x7f39e00c9e40 2024-01-11T07:16:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.583+0000 7f39dcff9700 1 -- 172.21.15.7:0/236193515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f39d4013680 con 0x7f39e00c9e40 2024-01-11T07:16:33.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.583+0000 7f39dcff9700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f39c4066f30 0x7f39c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:33.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.584+0000 7f39dcff9700 1 -- 172.21.15.7:0/236193515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f39d4084600 con 0x7f39e00c9e40 2024-01-11T07:16:33.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.584+0000 7f39dffff700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f39c4066f30 0x7f39c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:33.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.584+0000 7f39dffff700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f39c4066f30 0x7f39c40693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f39e015ccd0 tx=0x7f39cc001a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.588+0000 7f39dcff9700 1 -- 172.21.15.7:0/236193515 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39d4051b20 con 0x7f39e00c9e40 2024-01-11T07:16:33.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:33.790+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f39e004c3c0 con 0x7f39e00c9e40 2024-01-11T07:16:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.483+0000 7f39dcff9700 1 -- 172.21.15.7:0/236193515 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v165) v1 ==== 138+0+0 (secure 0 0 0) 0x7f39d4055140 con 0x7f39e00c9e40 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.487+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f39c4066f30 msgr2=0x7f39c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.487+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f39c4066f30 0x7f39c40693e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f39e015ccd0 tx=0x7f39cc001a00 comp rx=0 tx=0).stop 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.487+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e00c9e40 msgr2=0x7f39e0155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.487+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e00c9e40 0x7f39e0155460 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f39d400ec80 tx=0x7f39d40091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.488+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 shutdown_connections 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.488+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f39c4066f30 0x7f39c40693e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.488+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39e0157ed0 0x7f39e0152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.488+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e00c9e40 0x7f39e0155460 secure :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f39d400ec80 tx=0x7f39d40091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.488+0000 7f39e6b5e700 1 --2- 172.21.15.7:0/236193515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39e0157b00 0x7f39e0151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.488+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 >> 172.21.15.7:0/236193515 conn(0x7f39e00680a0 msgr2=0x7f39e0153280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.488+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 shutdown_connections 2024-01-11T07:16:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.488+0000 7f39e6b5e700 1 -- 172.21.15.7:0/236193515 wait complete. 2024-01-11T07:16:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:16:34.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:16:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.647+0000 7fc35cd99700 1 Processor -- start 2024-01-11T07:16:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.647+0000 7fc35cd99700 1 -- start start 2024-01-11T07:16:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc35cd99700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc358157d80 0x7fc358063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc35cd99700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35815c660 con 0x7fc3581579b0 2024-01-11T07:16:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc35cd99700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc358063b90 con 0x7fc358063770 2024-01-11T07:16:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc35cd99700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3580aeab0 con 0x7fc358157d80 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc355d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc358157d80 0x7fc358063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc355d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc358157d80 0x7fc358063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49086/0 (socket says 172.21.15.7:49086) 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc355d9b700 1 -- 172.21.15.7:0/3293899279 learned_addr learned my addr 172.21.15.7:0/3293899279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc35659c700 1 --1- 172.21.15.7:0/3293899279 >> v1:172.21.15.7:6789/0 conn(0x7fc3581579b0 0x7fc35815bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36962/0 (socket says 172.21.15.7:36962) 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc355d9b700 1 -- 172.21.15.7:0/3293899279 >> v1:172.21.15.113:6789/0 conn(0x7fc358063770 legacy=0x7fc358069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc355d9b700 1 -- 172.21.15.7:0/3293899279 >> v1:172.21.15.7:6789/0 conn(0x7fc3581579b0 legacy=0x7fc35815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc35559a700 1 -- 172.21.15.7:0/3293899279 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906519664 0 0) 0x7fc35815c660 con 0x7fc3581579b0 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc355d9b700 1 -- 172.21.15.7:0/3293899279 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580aed90 con 0x7fc358157d80 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.649+0000 7fc355d9b700 1 --2- 172.21.15.7:0/3293899279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc358157d80 0x7fc358063230 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0099b0 tx=0x7fc34c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b28e07fc4b3c91b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35559a700 1 -- 172.21.15.7:0/3293899279 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00b860 con 0x7fc358157d80 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35559a700 1 -- 172.21.15.7:0/3293899279 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c00b9c0 con 0x7fc358157d80 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35559a700 1 -- 172.21.15.7:0/3293899279 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00bd60 con 0x7fc358157d80 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35cd99700 1 -- 172.21.15.7:0/3293899279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc358157d80 msgr2=0x7fc358063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35cd99700 1 --2- 172.21.15.7:0/3293899279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc358157d80 0x7fc358063230 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0099b0 tx=0x7fc34c009250 comp rx=0 tx=0).stop 2024-01-11T07:16:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35cd99700 1 -- 172.21.15.7:0/3293899279 shutdown_connections 2024-01-11T07:16:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35cd99700 1 --2- 172.21.15.7:0/3293899279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc358157d80 0x7fc358063230 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35cd99700 1 -- 172.21.15.7:0/3293899279 >> 172.21.15.7:0/3293899279 conn(0x7fc3580680e0 msgr2=0x7fc3580cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35cd99700 1 -- 172.21.15.7:0/3293899279 shutdown_connections 2024-01-11T07:16:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.650+0000 7fc35cd99700 1 -- 172.21.15.7:0/3293899279 wait complete. 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.652+0000 7fc35cd99700 1 Processor -- start 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.652+0000 7fc35cd99700 1 -- start start 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc35cd99700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc358063770 0x7fc3580c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc35cd99700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3581579b0 0x7fc3581556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc35cd99700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc358157d80 0x7fc35816d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc35cd99700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580c9740 con 0x7fc3581579b0 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc35cd99700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580c86a0 con 0x7fc358063770 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc35cd99700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3580c8980 con 0x7fc358157d80 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc355d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3581579b0 0x7fc3581556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc355d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3581579b0 0x7fc3581556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49096/0 (socket says 172.21.15.7:49096) 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc355d9b700 1 -- 172.21.15.7:0/2022800896 learned_addr learned my addr 172.21.15.7:0/2022800896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc35659c700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc358063770 0x7fc3580c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.653+0000 7fc356d9d700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc358157d80 0x7fc35816d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc355d9b700 1 -- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc358157d80 msgr2=0x7fc35816d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc355d9b700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc358157d80 0x7fc35816d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc355d9b700 1 -- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc358063770 msgr2=0x7fc3580c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc355d9b700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc358063770 0x7fc3580c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc355d9b700 1 -- 172.21.15.7:0/2022800896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0094f0 con 0x7fc3581579b0 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc355d9b700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3581579b0 0x7fc3581556d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009fd0 tx=0x7fc34c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc3467fc700 1 -- 172.21.15.7:0/2022800896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c011cd0 con 0x7fc3581579b0 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc3467fc700 1 -- 172.21.15.7:0/2022800896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c020070 con 0x7fc3581579b0 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3580c8c60 con 0x7fc3581579b0 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc3467fc700 1 -- 172.21.15.7:0/2022800896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c01b6a0 con 0x7fc3581579b0 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.654+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3580c9220 con 0x7fc3581579b0 2024-01-11T07:16:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.655+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35804c3c0 con 0x7fc3581579b0 2024-01-11T07:16:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.656+0000 7fc3467fc700 1 -- 172.21.15.7:0/2022800896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc34c022030 con 0x7fc3581579b0 2024-01-11T07:16:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.656+0000 7fc3467fc700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc338066f30 0x7fc3380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.656+0000 7fc3467fc700 1 -- 172.21.15.7:0/2022800896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fc34c086110 con 0x7fc3581579b0 2024-01-11T07:16:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.656+0000 7fc35659c700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc338066f30 0x7fc3380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.657+0000 7fc35659c700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc338066f30 0x7fc3380693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc348007fd0 tx=0x7fc348007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.661+0000 7fc3467fc700 1 -- 172.21.15.7:0/2022800896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc34c053560 con 0x7fc3581579b0 2024-01-11T07:16:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:34.870+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc3580c9410 con 0x7fc3581579b0 2024-01-11T07:16:35.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.488+0000 7fc3467fc700 1 -- 172.21.15.7:0/2022800896 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v166) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc34c056b80 con 0x7fc3581579b0 2024-01-11T07:16:35.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc338066f30 msgr2=0x7fc3380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:35.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc338066f30 0x7fc3380693e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc348007fd0 tx=0x7fc348007d50 comp rx=0 tx=0).stop 2024-01-11T07:16:35.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3581579b0 msgr2=0x7fc3581556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:35.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3581579b0 0x7fc3581556d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009fd0 tx=0x7fc34c016000 comp rx=0 tx=0).stop 2024-01-11T07:16:35.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 shutdown_connections 2024-01-11T07:16:35.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc338066f30 0x7fc3380693e0 secure :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc348007fd0 tx=0x7fc348007d50 comp rx=0 tx=0).stop 2024-01-11T07:16:35.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc358157d80 0x7fc35816d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc358063770 0x7fc3580c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 --2- 172.21.15.7:0/2022800896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3581579b0 0x7fc3581556d0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.496+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 >> 172.21.15.7:0/2022800896 conn(0x7fc3580680e0 msgr2=0x7fc358156db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:35.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.497+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 shutdown_connections 2024-01-11T07:16:35.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.498+0000 7fc35cd99700 1 -- 172.21.15.7:0/2022800896 wait complete. 2024-01-11T07:16:35.502 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:16:35.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:16:35.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.656+0000 7f390ceff700 1 Processor -- start 2024-01-11T07:16:35.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.656+0000 7f390ceff700 1 -- start start 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390ceff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3908070690 0x7f390815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390ceff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390815bee0 con 0x7f390806f230 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390ceff700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390815c110 con 0x7f3908157d30 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390ceff700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390806d670 con 0x7f3908070690 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f3906d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f390806f230 0x7f39080c24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36970/0 (socket says 172.21.15.7:36970) 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f3906d9d700 1 -- 172.21.15.7:0/2851168020 learned_addr learned my addr 172.21.15.7:0/2851168020 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f3905d9b700 1 --2- 172.21.15.7:0/2851168020 >> v2:172.21.15.113:3300/0 conn(0x7f3908070690 0x7f390815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714634328 0 0) 0x7f390815bee0 con 0x7f390806f230 2024-01-11T07:16:35.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f0003570 con 0x7f390806f230 2024-01-11T07:16:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731571180 0 0) 0x7f390815c110 con 0x7f3908157d30 2024-01-11T07:16:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f390815bee0 con 0x7f3908157d30 2024-01-11T07:16:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2092634453 0 0) 0x7f38f0003570 con 0x7f390806f230 2024-01-11T07:16:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 >> v2:172.21.15.113:3300/0 conn(0x7f3908070690 msgr2=0x7f390815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390559a700 1 --2- 172.21.15.7:0/2851168020 >> v2:172.21.15.113:3300/0 conn(0x7f3908070690 0x7f390815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.657+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 >> v1:172.21.15.113:6789/0 conn(0x7f3908157d30 legacy=0x7f390806eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:35.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390815c3f0 con 0x7f390806f230 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f3900001770 con 0x7f390806f230 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 >> v1:172.21.15.7:6789/0 conn(0x7f390806f230 legacy=0x7f39080c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0003c40 0x7f38f0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f0004810 0x7f38f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38f0005210 0x7f38f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390806d670 con 0x7f38f0004810 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f000c130 con 0x7f38f0003c40 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f000c2c0 con 0x7f38f0005210 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3905d9b700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f0004810 0x7f38f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390659c700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0003c40 0x7f38f0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3906d9d700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38f0005210 0x7f38f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3905d9b700 1 -- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38f0005210 msgr2=0x7f38f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3905d9b700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f38f0005210 0x7f38f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3905d9b700 1 -- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0003c40 msgr2=0x7f38f0004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3905d9b700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0003c40 0x7f38f0004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3905d9b700 1 -- 172.21.15.7:0/2851168020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f0003570 con 0x7f38f0004810 2024-01-11T07:16:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3906d9d700 1 -- 172.21.15.7:0/2851168020 reap_dead start 2024-01-11T07:16:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390659c700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0003c40 0x7f38f0004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:35.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f3905d9b700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f0004810 0x7f38f0004c80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009c30 tx=0x7f38fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ed795a2fc12d84 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00e710 con 0x7f38f0004810 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc00e870 con 0x7f38f0004810 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.658+0000 7f390559a700 1 -- 172.21.15.7:0/2851168020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00ec30 con 0x7f38f0004810 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.659+0000 7f390ceff700 1 -- 172.21.15.7:0/2851168020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f0004810 msgr2=0x7f38f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.659+0000 7f390ceff700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f0004810 0x7f38f0004c80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009c30 tx=0x7f38fc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.659+0000 7f390ceff700 1 -- 172.21.15.7:0/2851168020 shutdown_connections 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.659+0000 7f390ceff700 1 --2- 172.21.15.7:0/2851168020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f0004810 0x7f38f0004c80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.659+0000 7f390ceff700 1 -- 172.21.15.7:0/2851168020 >> 172.21.15.7:0/2851168020 conn(0x7f39080cd070 msgr2=0x7f39080cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.659+0000 7f390ceff700 1 -- 172.21.15.7:0/2851168020 shutdown_connections 2024-01-11T07:16:35.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.659+0000 7f390ceff700 1 -- 172.21.15.7:0/2851168020 wait complete. 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.660+0000 7f390ceff700 1 Processor -- start 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.660+0000 7f390ceff700 1 -- start start 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f390ceff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0004810 0x7f39081549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f390ceff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39080ae240 0x7f39081558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f390ceff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39080ae6b0 0x7f3908155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f390ceff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3908154f10 con 0x7f39080ae6b0 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f390ceff700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39080aec70 con 0x7f38f0004810 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f390ceff700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39080aef50 con 0x7f39080ae240 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f3906d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39080ae6b0 0x7f3908155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f3906d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39080ae6b0 0x7f3908155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49112/0 (socket says 172.21.15.7:49112) 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f3906d9d700 1 -- 172.21.15.7:0/155942528 learned_addr learned my addr 172.21.15.7:0/155942528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f390659c700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0004810 0x7f39081549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:35.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.661+0000 7f3905d9b700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39080ae240 0x7f39081558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f3906d9d700 1 -- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39080ae240 msgr2=0x7f39081558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f3906d9d700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39080ae240 0x7f39081558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f3906d9d700 1 -- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0004810 msgr2=0x7f39081549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f3906d9d700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0004810 0x7f39081549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f3906d9d700 1 -- 172.21.15.7:0/155942528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39080af230 con 0x7f39080ae6b0 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f390659c700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0004810 0x7f39081549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f3905d9b700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39080ae240 0x7f39081558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f3906d9d700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39080ae6b0 0x7f3908155df0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f390809ab60 tx=0x7f3900002020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f38f67fc700 1 -- 172.21.15.7:0/155942528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900019030 con 0x7f39080ae6b0 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f38f67fc700 1 -- 172.21.15.7:0/155942528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390000fd00 con 0x7f39080ae6b0 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f38f67fc700 1 -- 172.21.15.7:0/155942528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900001d80 con 0x7f39080ae6b0 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39080af510 con 0x7f39080ae6b0 2024-01-11T07:16:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.662+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39080ac840 con 0x7f39080ae6b0 2024-01-11T07:16:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.663+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390804c3c0 con 0x7f39080ae6b0 2024-01-11T07:16:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.664+0000 7f38f67fc700 1 -- 172.21.15.7:0/155942528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f390001c070 con 0x7f39080ae6b0 2024-01-11T07:16:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.664+0000 7f38f67fc700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f38e8066f30 0x7f38e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.664+0000 7f38f67fc700 1 -- 172.21.15.7:0/155942528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f390001a070 con 0x7f39080ae6b0 2024-01-11T07:16:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.664+0000 7f390659c700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f38e8066f30 0x7f38e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.665+0000 7f390659c700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f38e8066f30 0x7f38e80693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f38f8000c00 tx=0x7f38f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.669+0000 7f38f67fc700 1 -- 172.21.15.7:0/155942528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39000555d0 con 0x7f39080ae6b0 2024-01-11T07:16:35.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:35.922+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f39080cce50 con 0x7f39080ae6b0 2024-01-11T07:16:36.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.508+0000 7f38f67fc700 1 -- 172.21.15.7:0/155942528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3900055160 con 0x7f39080ae6b0 2024-01-11T07:16:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f38e8066f30 msgr2=0x7f38e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f38e8066f30 0x7f38e80693e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f38f8000c00 tx=0x7f38f8009040 comp rx=0 tx=0).stop 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39080ae6b0 msgr2=0x7f3908155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39080ae6b0 0x7f3908155df0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f390809ab60 tx=0x7f3900002020 comp rx=0 tx=0).stop 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 shutdown_connections 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f38e8066f30 0x7f38e80693e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39080ae6b0 0x7f3908155df0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38f0004810 0x7f39081549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 --2- 172.21.15.7:0/155942528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39080ae240 0x7f39081558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.514+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 >> 172.21.15.7:0/155942528 conn(0x7f39080cd070 msgr2=0x7f39080abcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.515+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 shutdown_connections 2024-01-11T07:16:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.515+0000 7f390ceff700 1 -- 172.21.15.7:0/155942528 wait complete. 2024-01-11T07:16:36.517 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 20 and data pool 21 2024-01-11T07:16:36.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:16:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.670+0000 7f3cd2981700 1 Processor -- start 2024-01-11T07:16:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.670+0000 7f3cd2981700 1 -- start start 2024-01-11T07:16:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.670+0000 7f3cd2981700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ccc0cafb0 0x7f3ccc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.670+0000 7f3cd2981700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc158280 0x7f3ccc0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.670+0000 7f3cd2981700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc159300 con 0x7f3ccc0b0970 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.670+0000 7f3cd2981700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc0b0df0 con 0x7f3ccc0cafb0 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.670+0000 7f3cd2981700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0cf260 con 0x7f3ccc158280 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccb7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc158280 0x7f3ccc0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccbfff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ccc0cafb0 0x7f3ccc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccbfff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ccc0cafb0 0x7f3ccc0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:37516/0 (socket says 172.21.15.7:37516) 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3cd0f1e700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3ccc0b0970 0x7f3ccc159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:46222/0 (socket says 172.21.15.7:46222) 2024-01-11T07:16:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccbfff700 1 -- 172.21.15.7:0/2710319120 learned_addr learned my addr 172.21.15.7:0/2710319120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccbfff700 1 -- 172.21.15.7:0/2710319120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc158280 msgr2=0x7f3ccc0b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccbfff700 1 --2- 172.21.15.7:0/2710319120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc158280 0x7f3ccc0b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccbfff700 1 -- 172.21.15.7:0/2710319120 >> v1:172.21.15.113:6789/0 conn(0x7f3ccc0b0970 legacy=0x7f3ccc159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccbfff700 1 -- 172.21.15.7:0/2710319120 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0cf540 con 0x7f3ccc0cafb0 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccaffd700 1 -- 172.21.15.7:0/2710319120 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923552336 0 0) 0x7f3ccc159300 con 0x7f3ccc0b0970 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.671+0000 7f3ccb7fe700 1 --2- 172.21.15.7:0/2710319120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc158280 0x7f3ccc0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3ccbfff700 1 --2- 172.21.15.7:0/2710319120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc0cafb0 0x7f3ccc0ce8f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0099b0 tx=0x7f3cbc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fbf8c633236e813 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3ccaffd700 1 -- 172.21.15.7:0/2710319120 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc009ea0 con 0x7f3ccc0cafb0 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3ccaffd700 1 -- 172.21.15.7:0/2710319120 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc00b860 con 0x7f3ccc0cafb0 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3ccaffd700 1 -- 172.21.15.7:0/2710319120 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00bc00 con 0x7f3ccc0cafb0 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3cd2981700 1 -- 172.21.15.7:0/2710319120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc0cafb0 msgr2=0x7f3ccc0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3cd2981700 1 --2- 172.21.15.7:0/2710319120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc0cafb0 0x7f3ccc0ce8f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0099b0 tx=0x7f3cbc009250 comp rx=0 tx=0).stop 2024-01-11T07:16:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3cd2981700 1 -- 172.21.15.7:0/2710319120 shutdown_connections 2024-01-11T07:16:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3cd2981700 1 --2- 172.21.15.7:0/2710319120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc0cafb0 0x7f3ccc0ce8f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3cd2981700 1 --2- 172.21.15.7:0/2710319120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc158280 0x7f3ccc0b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3cd2981700 1 -- 172.21.15.7:0/2710319120 >> 172.21.15.7:0/2710319120 conn(0x7f3ccc071100 msgr2=0x7f3ccc06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3cd2981700 1 -- 172.21.15.7:0/2710319120 shutdown_connections 2024-01-11T07:16:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.672+0000 7f3cd2981700 1 -- 172.21.15.7:0/2710319120 wait complete. 2024-01-11T07:16:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.674+0000 7f3cd2981700 1 Processor -- start 2024-01-11T07:16:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.674+0000 7f3cd2981700 1 -- start start 2024-01-11T07:16:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3cd2981700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc0b0970 0x7f3ccc14f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3cd2981700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc0cafb0 0x7f3ccc0ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3cd2981700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc158280 0x7f3ccc16d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3cd2981700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0c8f90 con 0x7f3ccc0b0970 2024-01-11T07:16:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3cd2981700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc14fe00 con 0x7f3ccc158280 2024-01-11T07:16:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3cd2981700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc16e0e0 con 0x7f3ccc0cafb0 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc0b0970 0x7f3ccc14f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc0b0970 0x7f3ccc14f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49128/0 (socket says 172.21.15.7:49128) 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 -- 172.21.15.7:0/3499800638 learned_addr learned my addr 172.21.15.7:0/3499800638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3cd0f1e700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc158280 0x7f3ccc16d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccb7fe700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc0cafb0 0x7f3ccc0ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 -- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc0cafb0 msgr2=0x7f3ccc0ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc0cafb0 0x7f3ccc0ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 -- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc158280 msgr2=0x7f3ccc16d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc158280 0x7f3ccc16d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 -- 172.21.15.7:0/3499800638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0009420 con 0x7f3ccc0b0970 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3ccbfff700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc0b0970 0x7f3ccc14f8c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc06a1d0 tx=0x7f3cbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.675+0000 7f3cb3fff700 1 -- 172.21.15.7:0/3499800638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc000d10 con 0x7f3ccc0b0970 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.676+0000 7f3cb3fff700 1 -- 172.21.15.7:0/3499800638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc00bbe0 con 0x7f3ccc0b0970 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.676+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc0094f0 con 0x7f3ccc0b0970 2024-01-11T07:16:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.676+0000 7f3cb3fff700 1 -- 172.21.15.7:0/3499800638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011690 con 0x7f3ccc0b0970 2024-01-11T07:16:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.676+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc0c8c20 con 0x7f3ccc0b0970 2024-01-11T07:16:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.677+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc0697e0 con 0x7f3ccc0b0970 2024-01-11T07:16:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.677+0000 7f3cb3fff700 1 -- 172.21.15.7:0/3499800638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f3cbc011ac0 con 0x7f3ccc0b0970 2024-01-11T07:16:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.678+0000 7f3cb3fff700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3cb8066f30 0x7f3cb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.678+0000 7f3cb3fff700 1 -- 172.21.15.7:0/3499800638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3cbc085d30 con 0x7f3ccc0b0970 2024-01-11T07:16:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.678+0000 7f3ccb7fe700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3cb8066f30 0x7f3cb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.678+0000 7f3ccb7fe700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3cb8066f30 0x7f3cb80693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc158650 tx=0x7f3cc00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.682+0000 7f3cb3fff700 1 -- 172.21.15.7:0/3499800638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cbc052fa0 con 0x7f3ccc0b0970 2024-01-11T07:16:36.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.886+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ccc04c3c0 con 0x7f3ccc0b0970 2024-01-11T07:16:36.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.887+0000 7f3cb3fff700 1 -- 172.21.15.7:0/3499800638 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f3cbc0565c0 con 0x7f3ccc0b0970 2024-01-11T07:16:36.887 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:36.887 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":167,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:16:35.923471+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:16:33.823565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-11T07:16:34.902544+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3cb8066f30 msgr2=0x7f3cb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3cb8066f30 0x7f3cb80693e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc158650 tx=0x7f3cc00091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc0b0970 msgr2=0x7f3ccc14f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc0b0970 0x7f3ccc14f8c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc06a1d0 tx=0x7f3cbc017000 comp rx=0 tx=0).stop 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 shutdown_connections 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3cb8066f30 0x7f3cb80693e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ccc158280 0x7f3ccc16d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc0b0970 0x7f3ccc14f8c0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 --2- 172.21.15.7:0/3499800638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ccc0cafb0 0x7f3ccc0ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 >> 172.21.15.7:0/3499800638 conn(0x7f3ccc071100 msgr2=0x7f3ccc0ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:36.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 shutdown_connections 2024-01-11T07:16:36.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:36.890+0000 7f3cd2981700 1 -- 172.21.15.7:0/3499800638 wait complete. 2024-01-11T07:16:36.904 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:16:37.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.035+0000 7f36b8f68700 1 Processor -- start 2024-01-11T07:16:37.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.035+0000 7f36b8f68700 1 -- start start 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.036+0000 7f36b8f68700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36b4063230 0x7f36b4063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.036+0000 7f36b8f68700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b40ac5b0 0x7f36b40aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.036+0000 7f36b8f68700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b40ab100 con 0x7f36b415a280 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.036+0000 7f36b8f68700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b415d880 con 0x7f36b4063230 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.036+0000 7f36b8f68700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b406c5b0 con 0x7f36b40ac5b0 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.036+0000 7f36b259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36b4063230 0x7f36b4063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.036+0000 7f36b259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36b4063230 0x7f36b4063600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49130/0 (socket says 172.21.15.7:49130) 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.036+0000 7f36b259c700 1 -- 172.21.15.7:0/1313367011 learned_addr learned my addr 172.21.15.7:0/1313367011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b2d9d700 1 --2- 172.21.15.7:0/1313367011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b40ac5b0 0x7f36b40aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b259c700 1 -- 172.21.15.7:0/1313367011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b40ac5b0 msgr2=0x7f36b40aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b259c700 1 --2- 172.21.15.7:0/1313367011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b40ac5b0 0x7f36b40aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b259c700 1 -- 172.21.15.7:0/1313367011 >> v1:172.21.15.113:6789/0 conn(0x7f36b415a280 legacy=0x7f36b40aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b259c700 1 -- 172.21.15.7:0/1313367011 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4063b40 con 0x7f36b4063230 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b159a700 1 -- 172.21.15.7:0/1313367011 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108793003 0 0) 0x7f36b40ab100 con 0x7f36b415a280 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b259c700 1 --2- 172.21.15.7:0/1313367011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4063230 0x7f36b4063600 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f36a40099b0 tx=0x7f36a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1983b823c78c52d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b159a700 1 -- 172.21.15.7:0/1313367011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4009ea0 con 0x7f36b4063230 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b159a700 1 -- 172.21.15.7:0/1313367011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a400b860 con 0x7f36b4063230 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.037+0000 7f36b159a700 1 -- 172.21.15.7:0/1313367011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a400bc00 con 0x7f36b4063230 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.038+0000 7f36b8f68700 1 -- 172.21.15.7:0/1313367011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4063230 msgr2=0x7f36b4063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.038+0000 7f36b8f68700 1 --2- 172.21.15.7:0/1313367011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4063230 0x7f36b4063600 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f36a40099b0 tx=0x7f36a4009250 comp rx=0 tx=0).stop 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.038+0000 7f36b8f68700 1 -- 172.21.15.7:0/1313367011 shutdown_connections 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.038+0000 7f36b8f68700 1 --2- 172.21.15.7:0/1313367011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b40ac5b0 0x7f36b40aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.038+0000 7f36b8f68700 1 --2- 172.21.15.7:0/1313367011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4063230 0x7f36b4063600 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.038+0000 7f36b8f68700 1 -- 172.21.15.7:0/1313367011 >> 172.21.15.7:0/1313367011 conn(0x7f36b40680e0 msgr2=0x7f36b40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.038+0000 7f36b8f68700 1 -- 172.21.15.7:0/1313367011 shutdown_connections 2024-01-11T07:16:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.038+0000 7f36b8f68700 1 -- 172.21.15.7:0/1313367011 wait complete. 2024-01-11T07:16:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.039+0000 7f36b8f68700 1 Processor -- start 2024-01-11T07:16:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.039+0000 7f36b8f68700 1 -- start start 2024-01-11T07:16:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b8f68700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b4063230 0x7f36b4169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b8f68700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40ac5b0 0x7f36b4153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b8f68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b415a280 0x7f36b40c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b8f68700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4069e70 con 0x7f36b415a280 2024-01-11T07:16:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b8f68700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40c2320 con 0x7f36b40ac5b0 2024-01-11T07:16:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b8f68700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40c17e0 con 0x7f36b4063230 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b415a280 0x7f36b40c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b415a280 0x7f36b40c12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49138/0 (socket says 172.21.15.7:49138) 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b2d9d700 1 -- 172.21.15.7:0/4141439911 learned_addr learned my addr 172.21.15.7:0/4141439911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b259c700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b4063230 0x7f36b4169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.040+0000 7f36b1d9b700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40ac5b0 0x7f36b4153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36b2d9d700 1 -- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b4063230 msgr2=0x7f36b4169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36b2d9d700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b4063230 0x7f36b4169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36b2d9d700 1 -- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40ac5b0 msgr2=0x7f36b4153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36b2d9d700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40ac5b0 0x7f36b4153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36b2d9d700 1 -- 172.21.15.7:0/4141439911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40094f0 con 0x7f36b415a280 2024-01-11T07:16:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36b2d9d700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b415a280 0x7f36b40c12a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009df0 tx=0x7f36ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36a27fc700 1 -- 172.21.15.7:0/4141439911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00caa0 con 0x7f36b415a280 2024-01-11T07:16:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36a27fc700 1 -- 172.21.15.7:0/4141439911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac00f040 con 0x7f36b415a280 2024-01-11T07:16:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b40c1a80 con 0x7f36b415a280 2024-01-11T07:16:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36a27fc700 1 -- 172.21.15.7:0/4141439911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00ce70 con 0x7f36b415a280 2024-01-11T07:16:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.041+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b40c2040 con 0x7f36b415a280 2024-01-11T07:16:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.042+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b4069c40 con 0x7f36b415a280 2024-01-11T07:16:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.043+0000 7f36a27fc700 1 -- 172.21.15.7:0/4141439911 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f36ac014690 con 0x7f36b415a280 2024-01-11T07:16:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.043+0000 7f36a27fc700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3694066f30 0x7f36940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.043+0000 7f36a27fc700 1 -- 172.21.15.7:0/4141439911 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f36ac085050 con 0x7f36b415a280 2024-01-11T07:16:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.043+0000 7f36b259c700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3694066f30 0x7f36940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.044+0000 7f36b259c700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3694066f30 0x7f36940693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f36a40099b0 tx=0x7f36a4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.047+0000 7f36a27fc700 1 -- 172.21.15.7:0/4141439911 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36b4069c40 con 0x7f36b415a280 2024-01-11T07:16:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.253+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36b406a020 con 0x7f36b415a280 2024-01-11T07:16:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.254+0000 7f36a27fc700 1 -- 172.21.15.7:0/4141439911 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f36b406a020 con 0x7f36b415a280 2024-01-11T07:16:37.253 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:37.254 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":167,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:16:35.923471+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:16:33.823565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-11T07:16:34.902544+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3694066f30 msgr2=0x7f36940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3694066f30 0x7f36940693e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f36a40099b0 tx=0x7f36a4017000 comp rx=0 tx=0).stop 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b415a280 msgr2=0x7f36b40c12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b415a280 0x7f36b40c12a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009df0 tx=0x7f36ac00b040 comp rx=0 tx=0).stop 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 shutdown_connections 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f3694066f30 0x7f36940693e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b415a280 0x7f36b40c12a0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36b4063230 0x7f36b4169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 --2- 172.21.15.7:0/4141439911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36b40ac5b0 0x7f36b4153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 >> 172.21.15.7:0/4141439911 conn(0x7f36b40680e0 msgr2=0x7f36b415b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 shutdown_connections 2024-01-11T07:16:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.256+0000 7f36b8f68700 1 -- 172.21.15.7:0/4141439911 wait complete. 2024-01-11T07:16:37.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:16:37.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.405+0000 7f9d346b5700 1 Processor -- start 2024-01-11T07:16:37.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.405+0000 7f9d346b5700 1 -- start start 2024-01-11T07:16:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.406+0000 7f9d346b5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9d2c154920 0x7f9d2c06ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.406+0000 7f9d346b5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c15a790 con 0x7f9d2c06e200 2024-01-11T07:16:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.406+0000 7f9d346b5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c15a9e0 con 0x7f9d2c06de30 2024-01-11T07:16:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.406+0000 7f9d346b5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c15b7a0 con 0x7f9d2c154920 2024-01-11T07:16:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.406+0000 7f9d32c52700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9d2c06e200 0x7f9d2c0c7f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57896/0 (socket says 172.21.15.7:57896) 2024-01-11T07:16:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.406+0000 7f9d32c52700 1 -- 172.21.15.7:0/2141421322 learned_addr learned my addr 172.21.15.7:0/2141421322 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d31c50700 1 --2- 172.21.15.7:0/2141421322 >> v2:172.21.15.113:3300/0 conn(0x7f9d2c154920 0x7f9d2c06ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d3144f700 1 -- 172.21.15.7:0/2141421322 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211970557 0 0) 0x7f9d2c15a790 con 0x7f9d2c06e200 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d3144f700 1 -- 172.21.15.7:0/2141421322 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d18003570 con 0x7f9d2c06e200 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d3144f700 1 -- 172.21.15.7:0/2141421322 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813728403 0 0) 0x7f9d2c15a9e0 con 0x7f9d2c06de30 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d3144f700 1 -- 172.21.15.7:0/2141421322 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d2c15a790 con 0x7f9d2c06de30 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d31c50700 1 -- 172.21.15.7:0/2141421322 >> v1:172.21.15.113:6789/0 conn(0x7f9d2c06de30 legacy=0x7f9d2c154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d3144f700 1 -- 172.21.15.7:0/2141421322 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75236275 0 0) 0x7f9d18003570 con 0x7f9d2c06e200 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d31c50700 1 -- 172.21.15.7:0/2141421322 >> v1:172.21.15.7:6789/0 conn(0x7f9d2c06e200 legacy=0x7f9d2c0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d31c50700 1 -- 172.21.15.7:0/2141421322 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c15acc0 con 0x7f9d2c154920 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d31c50700 1 --2- 172.21.15.7:0/2141421322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c154920 0x7f9d2c06ad40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9d280098e0 tx=0x7f9d280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3016b67e85f430b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.407+0000 7f9d3144f700 1 -- 172.21.15.7:0/2141421322 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240017f0 con 0x7f9d2c154920 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d3144f700 1 -- 172.21.15.7:0/2141421322 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28011420 con 0x7f9d2c154920 2024-01-11T07:16:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d3144f700 1 -- 172.21.15.7:0/2141421322 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280117c0 con 0x7f9d2c154920 2024-01-11T07:16:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d346b5700 1 -- 172.21.15.7:0/2141421322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c154920 msgr2=0x7f9d2c06ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d346b5700 1 --2- 172.21.15.7:0/2141421322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c154920 0x7f9d2c06ad40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9d280098e0 tx=0x7f9d280091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d346b5700 1 -- 172.21.15.7:0/2141421322 shutdown_connections 2024-01-11T07:16:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d346b5700 1 --2- 172.21.15.7:0/2141421322 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c154920 0x7f9d2c06ad40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d346b5700 1 -- 172.21.15.7:0/2141421322 >> 172.21.15.7:0/2141421322 conn(0x7f9d2c0cd8f0 msgr2=0x7f9d2c0cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d346b5700 1 -- 172.21.15.7:0/2141421322 shutdown_connections 2024-01-11T07:16:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.408+0000 7f9d346b5700 1 -- 172.21.15.7:0/2141421322 wait complete. 2024-01-11T07:16:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.409+0000 7f9d346b5700 1 Processor -- start 2024-01-11T07:16:37.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d346b5700 1 -- start start 2024-01-11T07:16:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d346b5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d2c06de30 0x7f9d2c0aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d346b5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06e200 0x7f9d2c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d346b5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c154920 0x7f9d2c1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d346b5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b0980 con 0x7f9d2c06de30 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d346b5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b0bd0 con 0x7f9d2c06e200 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d346b5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b0eb0 con 0x7f9d2c154920 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d32451700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d2c06de30 0x7f9d2c0aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.410+0000 7f9d32451700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d2c06de30 0x7f9d2c0aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56050/0 (socket says 172.21.15.7:56050) 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32451700 1 -- 172.21.15.7:0/3598939379 learned_addr learned my addr 172.21.15.7:0/3598939379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d31c50700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06e200 0x7f9d2c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32c52700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c154920 0x7f9d2c1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32451700 1 -- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c154920 msgr2=0x7f9d2c1510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32451700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c154920 0x7f9d2c1510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32451700 1 -- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06e200 msgr2=0x7f9d2c156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32451700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06e200 0x7f9d2c156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32451700 1 -- 172.21.15.7:0/3598939379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d28009420 con 0x7f9d2c06de30 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32c52700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c154920 0x7f9d2c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d32451700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d2c06de30 0x7f9d2c0aed50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00e9d0 tx=0x7f9d1c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d227fc700 1 -- 172.21.15.7:0/3598939379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00c7e0 con 0x7f9d2c06de30 2024-01-11T07:16:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d227fc700 1 -- 172.21.15.7:0/3598939379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c00c980 con 0x7f9d2c06de30 2024-01-11T07:16:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c0b1190 con 0x7f9d2c06de30 2024-01-11T07:16:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d227fc700 1 -- 172.21.15.7:0/3598939379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00cd40 con 0x7f9d2c06de30 2024-01-11T07:16:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.411+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c15d7c0 con 0x7f9d2c06de30 2024-01-11T07:16:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.412+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c0b17e0 con 0x7f9d2c06de30 2024-01-11T07:16:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.413+0000 7f9d227fc700 1 -- 172.21.15.7:0/3598939379 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9d1c011680 con 0x7f9d2c06de30 2024-01-11T07:16:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.413+0000 7f9d227fc700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d10066f30 0x7f9d100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.413+0000 7f9d227fc700 1 -- 172.21.15.7:0/3598939379 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9d1c015070 con 0x7f9d2c06de30 2024-01-11T07:16:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.414+0000 7f9d31c50700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d10066f30 0x7f9d100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.414+0000 7f9d31c50700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d10066f30 0x7f9d100693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c154d80 tx=0x7f9d28016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.418+0000 7f9d227fc700 1 -- 172.21.15.7:0/3598939379 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d1c054070 con 0x7f9d2c06de30 2024-01-11T07:16:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:37.674+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9d2c152d00 con 0x7f9d2c06de30 2024-01-11T07:16:38.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.505+0000 7f9d227fc700 1 -- 172.21.15.7:0/3598939379 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9d1c057690 con 0x7f9d2c06de30 2024-01-11T07:16:38.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d10066f30 msgr2=0x7f9d100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:38.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d10066f30 0x7f9d100693e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c154d80 tx=0x7f9d28016000 comp rx=0 tx=0).stop 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d2c06de30 msgr2=0x7f9d2c0aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d2c06de30 0x7f9d2c0aed50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00e9d0 tx=0x7f9d1c007ea0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 shutdown_connections 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9d10066f30 0x7f9d100693e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2c154920 0x7f9d2c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d2c06de30 0x7f9d2c0aed50 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 --2- 172.21.15.7:0/3598939379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d2c06e200 0x7f9d2c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 >> 172.21.15.7:0/3598939379 conn(0x7f9d2c0cd8f0 msgr2=0x7f9d2c158340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 shutdown_connections 2024-01-11T07:16:38.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.508+0000 7f9d346b5700 1 -- 172.21.15.7:0/3598939379 wait complete. 2024-01-11T07:16:38.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:16:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.663+0000 7fc10a081700 1 Processor -- start 2024-01-11T07:16:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.663+0000 7fc10a081700 1 -- start start 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc10a081700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc104150de0 0x7fc1040ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc10a081700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc10406a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc10a081700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1040acc30 con 0x7fc10406bf20 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc10a081700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1040ab260 con 0x7fc104150de0 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc10a081700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040acf10 con 0x7fc1041511b0 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc102ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc10406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc102ffd700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc10406a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:60068/0 (socket says 172.21.15.7:60068) 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc102ffd700 1 -- 172.21.15.7:0/219238889 learned_addr learned my addr 172.21.15.7:0/219238889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc1037fe700 1 --2- 172.21.15.7:0/219238889 >> v2:172.21.15.113:3300/0 conn(0x7fc104150de0 0x7fc1040ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.664+0000 7fc1027fc700 1 -- 172.21.15.7:0/219238889 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922473535 0 0) 0x7fc1040acc30 con 0x7fc10406bf20 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc102ffd700 1 -- 172.21.15.7:0/219238889 >> v2:172.21.15.113:3300/0 conn(0x7fc104150de0 msgr2=0x7fc1040ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:38.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc102ffd700 1 --2- 172.21.15.7:0/219238889 >> v2:172.21.15.113:3300/0 conn(0x7fc104150de0 0x7fc1040ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc102ffd700 1 -- 172.21.15.7:0/219238889 >> v1:172.21.15.113:6789/0 conn(0x7fc10406bf20 legacy=0x7fc104063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc102ffd700 1 -- 172.21.15.7:0/219238889 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040ad1f0 con 0x7fc1041511b0 2024-01-11T07:16:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc102ffd700 1 --2- 172.21.15.7:0/219238889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc10406a480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80099b0 tx=0x7fc0f8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc66d5b19516a9e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc1027fc700 1 -- 172.21.15.7:0/219238889 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8009ea0 con 0x7fc1041511b0 2024-01-11T07:16:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc1027fc700 1 -- 172.21.15.7:0/219238889 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f800b860 con 0x7fc1041511b0 2024-01-11T07:16:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc1027fc700 1 -- 172.21.15.7:0/219238889 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800bc00 con 0x7fc1041511b0 2024-01-11T07:16:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.665+0000 7fc10a081700 1 -- 172.21.15.7:0/219238889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 msgr2=0x7fc10406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.666+0000 7fc10a081700 1 --2- 172.21.15.7:0/219238889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc10406a480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80099b0 tx=0x7fc0f8009250 comp rx=0 tx=0).stop 2024-01-11T07:16:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.666+0000 7fc10a081700 1 -- 172.21.15.7:0/219238889 shutdown_connections 2024-01-11T07:16:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.666+0000 7fc10a081700 1 --2- 172.21.15.7:0/219238889 >> v2:172.21.15.113:3300/0 conn(0x7fc104150de0 0x7fc1040ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.666+0000 7fc10a081700 1 --2- 172.21.15.7:0/219238889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc10406a480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.666+0000 7fc10a081700 1 -- 172.21.15.7:0/219238889 >> 172.21.15.7:0/219238889 conn(0x7fc1040cd910 msgr2=0x7fc1040cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.666+0000 7fc10a081700 1 -- 172.21.15.7:0/219238889 shutdown_connections 2024-01-11T07:16:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.666+0000 7fc10a081700 1 -- 172.21.15.7:0/219238889 wait complete. 2024-01-11T07:16:38.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.667+0000 7fc10a081700 1 Processor -- start 2024-01-11T07:16:38.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.667+0000 7fc10a081700 1 -- start start 2024-01-11T07:16:38.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc10a081700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10406bf20 0x7fc10414fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:38.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc10a081700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc104150de0 0x7fc1040aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:38.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc10a081700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc1040cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:38.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc10a081700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc104153f40 con 0x7fc10406bf20 2024-01-11T07:16:38.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc10a081700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc104150310 con 0x7fc104150de0 2024-01-11T07:16:38.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc10a081700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040af210 con 0x7fc1041511b0 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc1037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10406bf20 0x7fc10414fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc1037fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10406bf20 0x7fc10414fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56064/0 (socket says 172.21.15.7:56064) 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc1037fe700 1 -- 172.21.15.7:0/3923225408 learned_addr learned my addr 172.21.15.7:0/3923225408 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc102ffd700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc104150de0 0x7fc1040aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc103fff700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc1040cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc1037fe700 1 -- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 msgr2=0x7fc1040cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc1037fe700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc1040cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc1037fe700 1 -- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc104150de0 msgr2=0x7fc1040aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc1037fe700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc104150de0 0x7fc1040aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.668+0000 7fc1037fe700 1 -- 172.21.15.7:0/3923225408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f80094f0 con 0x7fc10406bf20 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.669+0000 7fc1037fe700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10406bf20 0x7fc10414fdd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400ecb0 tx=0x7fc0f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.669+0000 7fc0eb7fe700 1 -- 172.21.15.7:0/3923225408 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400c7f0 con 0x7fc10406bf20 2024-01-11T07:16:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.669+0000 7fc0eb7fe700 1 -- 172.21.15.7:0/3923225408 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f400c950 con 0x7fc10406bf20 2024-01-11T07:16:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.669+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1040cc070 con 0x7fc10406bf20 2024-01-11T07:16:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.669+0000 7fc0eb7fe700 1 -- 172.21.15.7:0/3923225408 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400ccf0 con 0x7fc10406bf20 2024-01-11T07:16:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.669+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1040cb2d0 con 0x7fc10406bf20 2024-01-11T07:16:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.670+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc104064a40 con 0x7fc10406bf20 2024-01-11T07:16:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.670+0000 7fc0eb7fe700 1 -- 172.21.15.7:0/3923225408 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc0f4013680 con 0x7fc10406bf20 2024-01-11T07:16:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.671+0000 7fc0eb7fe700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc0f0066f30 0x7fc0f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.671+0000 7fc0eb7fe700 1 -- 172.21.15.7:0/3923225408 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc0f4084780 con 0x7fc10406bf20 2024-01-11T07:16:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.671+0000 7fc102ffd700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc0f0066f30 0x7fc0f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.672+0000 7fc102ffd700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc0f0066f30 0x7fc0f00693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80099b0 tx=0x7fc0f8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:38.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.675+0000 7fc0eb7fe700 1 -- 172.21.15.7:0/3923225408 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0f4051950 con 0x7fc10406bf20 2024-01-11T07:16:38.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:38.933+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc10404c3c0 con 0x7fc10406bf20 2024-01-11T07:16:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.506+0000 7fc0eb7fe700 1 -- 172.21.15.7:0/3923225408 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc0f4054f70 con 0x7fc10406bf20 2024-01-11T07:16:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc0f0066f30 msgr2=0x7fc0f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc0f0066f30 0x7fc0f00693e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80099b0 tx=0x7fc0f8017000 comp rx=0 tx=0).stop 2024-01-11T07:16:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10406bf20 msgr2=0x7fc10414fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10406bf20 0x7fc10414fdd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400ecb0 tx=0x7fc0f40091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 shutdown_connections 2024-01-11T07:16:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc0f0066f30 0x7fc0f00693e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1041511b0 0x7fc1040cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc10406bf20 0x7fc10414fdd0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 --2- 172.21.15.7:0/3923225408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc104150de0 0x7fc1040aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 >> 172.21.15.7:0/3923225408 conn(0x7fc1040cd910 msgr2=0x7fc10406a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 shutdown_connections 2024-01-11T07:16:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.509+0000 7fc10a081700 1 -- 172.21.15.7:0/3923225408 wait complete. 2024-01-11T07:16:39.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:16:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.659+0000 7fd316855700 1 Processor -- start 2024-01-11T07:16:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.659+0000 7fd316855700 1 -- start start 2024-01-11T07:16:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.660+0000 7fd316855700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd310159d80 0x7fd31006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.660+0000 7fd316855700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31006e170 0x7fd31015da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.660+0000 7fd316855700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31006e5e0 con 0x7fd31015dfa0 2024-01-11T07:16:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.660+0000 7fd316855700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3100d06b0 con 0x7fd310159d80 2024-01-11T07:16:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.660+0000 7fd316855700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100d0220 con 0x7fd31006e170 2024-01-11T07:16:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.660+0000 7fd30f7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31006e170 0x7fd31015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.660+0000 7fd30ffff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd310159d80 0x7fd31006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30f7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31006e170 0x7fd31015da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:60096/0 (socket says 172.21.15.7:60096) 2024-01-11T07:16:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30ffff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd310159d80 0x7fd31006dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:49084/0 (socket says 172.21.15.7:49084) 2024-01-11T07:16:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30f7fe700 1 -- 172.21.15.7:0/1761573289 learned_addr learned my addr 172.21.15.7:0/1761573289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30ffff700 1 -- 172.21.15.7:0/1761573289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31006e170 msgr2=0x7fd31015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30ffff700 1 --2- 172.21.15.7:0/1761573289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31006e170 0x7fd31015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30ffff700 1 -- 172.21.15.7:0/1761573289 >> v1:172.21.15.113:6789/0 conn(0x7fd31015dfa0 legacy=0x7fd31015aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30ffff700 1 -- 172.21.15.7:0/1761573289 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100cf610 con 0x7fd310159d80 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30effd700 1 -- 172.21.15.7:0/1761573289 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706439561 0 0) 0x7fd31006e5e0 con 0x7fd31015dfa0 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30f7fe700 1 --2- 172.21.15.7:0/1761573289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31006e170 0x7fd31015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30ffff700 1 --2- 172.21.15.7:0/1761573289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd310159d80 0x7fd31006dc30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd3000098e0 tx=0x7fd3000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1282970aad0a5d2c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30effd700 1 -- 172.21.15.7:0/1761573289 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000b640 con 0x7fd310159d80 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30effd700 1 -- 172.21.15.7:0/1761573289 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30000b7a0 con 0x7fd310159d80 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.661+0000 7fd30effd700 1 -- 172.21.15.7:0/1761573289 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000bb40 con 0x7fd310159d80 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.662+0000 7fd316855700 1 -- 172.21.15.7:0/1761573289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd310159d80 msgr2=0x7fd31006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.662+0000 7fd316855700 1 --2- 172.21.15.7:0/1761573289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd310159d80 0x7fd31006dc30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd3000098e0 tx=0x7fd3000091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.662+0000 7fd316855700 1 -- 172.21.15.7:0/1761573289 shutdown_connections 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.662+0000 7fd316855700 1 --2- 172.21.15.7:0/1761573289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd310159d80 0x7fd31006dc30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.662+0000 7fd316855700 1 --2- 172.21.15.7:0/1761573289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd31006e170 0x7fd31015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.662+0000 7fd316855700 1 -- 172.21.15.7:0/1761573289 >> 172.21.15.7:0/1761573289 conn(0x7fd310071130 msgr2=0x7fd3100b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:39.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.662+0000 7fd316855700 1 -- 172.21.15.7:0/1761573289 shutdown_connections 2024-01-11T07:16:39.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.662+0000 7fd316855700 1 -- 172.21.15.7:0/1761573289 wait complete. 2024-01-11T07:16:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.663+0000 7fd316855700 1 Processor -- start 2024-01-11T07:16:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.663+0000 7fd316855700 1 -- start start 2024-01-11T07:16:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd316855700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31006e170 0x7fd31015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd316855700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd310159d80 0x7fd3100c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd316855700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31015dfa0 0x7fd31016a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd316855700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31015c260 con 0x7fd31006e170 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd316855700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31015b650 con 0x7fd31015dfa0 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd316855700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31015b930 con 0x7fd310159d80 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd30ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31006e170 0x7fd31015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd30ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31006e170 0x7fd31015c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56070/0 (socket says 172.21.15.7:56070) 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.664+0000 7fd30ffff700 1 -- 172.21.15.7:0/581816781 learned_addr learned my addr 172.21.15.7:0/581816781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd30f7fe700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd310159d80 0x7fd3100c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd314df2700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31015dfa0 0x7fd31016a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd30ffff700 1 -- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd310159d80 msgr2=0x7fd3100c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd30ffff700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd310159d80 0x7fd3100c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd30ffff700 1 -- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31015dfa0 msgr2=0x7fd31016a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd30ffff700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31015dfa0 0x7fd31016a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd30ffff700 1 -- 172.21.15.7:0/581816781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3040094f0 con 0x7fd31006e170 2024-01-11T07:16:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd30f7fe700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd310159d80 0x7fd3100c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd314df2700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31015dfa0 0x7fd31016a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd30ffff700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31006e170 0x7fd31015c6f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd3100d1c10 tx=0x7fd300017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd2f7fff700 1 -- 172.21.15.7:0/581816781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30001f070 con 0x7fd31006e170 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd2f7fff700 1 -- 172.21.15.7:0/581816781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300000bc0 con 0x7fd31006e170 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd300009420 con 0x7fd31006e170 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd2f7fff700 1 -- 172.21.15.7:0/581816781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300025420 con 0x7fd31006e170 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.665+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31015bef0 con 0x7fd31006e170 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.666+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd310089080 con 0x7fd31006e170 2024-01-11T07:16:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.667+0000 7fd2f7fff700 1 -- 172.21.15.7:0/581816781 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd300023070 con 0x7fd31006e170 2024-01-11T07:16:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.667+0000 7fd2f7fff700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2fc066f30 0x7fd2fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.667+0000 7fd2f7fff700 1 -- 172.21.15.7:0/581816781 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd300085ab0 con 0x7fd31006e170 2024-01-11T07:16:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.667+0000 7fd30f7fe700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2fc066f30 0x7fd2fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.668+0000 7fd30f7fe700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2fc066f30 0x7fd2fc0693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd3040094c0 tx=0x7fd304008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.672+0000 7fd2f7fff700 1 -- 172.21.15.7:0/581816781 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd300052d20 con 0x7fd31006e170 2024-01-11T07:16:39.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.930+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd31004c3c0 con 0x7fd31006e170 2024-01-11T07:16:39.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.930+0000 7fd2f7fff700 1 -- 172.21.15.7:0/581816781 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fd300056340 con 0x7fd31006e170 2024-01-11T07:16:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4097846800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4097846800},{"type":"v1","addr":"172.21.15.113:6836","nonce":4097846800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1898628721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1898628721},{"type":"v1","addr":"172.21.15.113:6837","nonce":1898628721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3371959241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3371959241},{"type":"v1","addr":"172.21.15.7:6835","nonce":3371959241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:16:35.923785+0000","modified":"2024-01-11T07:16:39.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/2285850291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2285850291},{"type":"v1","addr":"172.21.15.7:6833","nonce":2285850291}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-11T07:16:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2fc066f30 msgr2=0x7fd2fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2fc066f30 0x7fd2fc0693e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd3040094c0 tx=0x7fd304008040 comp rx=0 tx=0).stop 2024-01-11T07:16:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31006e170 msgr2=0x7fd31015c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31006e170 0x7fd31015c6f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd3100d1c10 tx=0x7fd300017000 comp rx=0 tx=0).stop 2024-01-11T07:16:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 shutdown_connections 2024-01-11T07:16:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd2fc066f30 0x7fd2fc0693e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd31015dfa0 0x7fd31016a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd31006e170 0x7fd31015c6f0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 --2- 172.21.15.7:0/581816781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd310159d80 0x7fd3100c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:39.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 >> 172.21.15.7:0/581816781 conn(0x7fd310071130 msgr2=0x7fd3100b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:39.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 shutdown_connections 2024-01-11T07:16:39.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:39.933+0000 7fd316855700 1 -- 172.21.15.7:0/581816781 wait complete. 2024-01-11T07:16:39.934 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2024-01-11T07:16:39.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:16:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.079+0000 7f51de4e5700 1 Processor -- start 2024-01-11T07:16:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.079+0000 7f51de4e5700 1 -- start start 2024-01-11T07:16:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.080+0000 7f51de4e5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f51d81579b0 0x7f51d815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.080+0000 7f51de4e5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d80aeab0 con 0x7f51d8157d80 2024-01-11T07:16:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.080+0000 7f51de4e5700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d80aed90 con 0x7f51d8063940 2024-01-11T07:16:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.080+0000 7f51de4e5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d80af070 con 0x7f51d81579b0 2024-01-11T07:16:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.080+0000 7f51d7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f51d81579b0 0x7f51d815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f51d8157d80 0x7f51d8063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57912/0 (socket says 172.21.15.7:57912) 2024-01-11T07:16:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d77fe700 1 -- 172.21.15.7:0/2442820104 learned_addr learned my addr 172.21.15.7:0/2442820104 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d7fff700 1 -- 172.21.15.7:0/2442820104 >> v1:172.21.15.113:6789/0 conn(0x7f51d8063940 legacy=0x7f51d8069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d7fff700 1 -- 172.21.15.7:0/2442820104 >> v1:172.21.15.7:6789/0 conn(0x7f51d8157d80 legacy=0x7f51d8063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d7fff700 1 -- 172.21.15.7:0/2442820104 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d80af350 con 0x7f51d81579b0 2024-01-11T07:16:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d6ffd700 1 -- 172.21.15.7:0/2442820104 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678433383 0 0) 0x7f51d80aeab0 con 0x7f51d8157d80 2024-01-11T07:16:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d7fff700 1 --2- 172.21.15.7:0/2442820104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d81579b0 0x7f51d815bf50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f51c80098e0 tx=0x7f51c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2152e8d40f1c6ec3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d6ffd700 1 -- 172.21.15.7:0/2442820104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8009e80 con 0x7f51d81579b0 2024-01-11T07:16:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d6ffd700 1 -- 172.21.15.7:0/2442820104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c800b860 con 0x7f51d81579b0 2024-01-11T07:16:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.081+0000 7f51d6ffd700 1 -- 172.21.15.7:0/2442820104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c800bbe0 con 0x7f51d81579b0 2024-01-11T07:16:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.082+0000 7f51de4e5700 1 -- 172.21.15.7:0/2442820104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d81579b0 msgr2=0x7f51d815bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.082+0000 7f51de4e5700 1 --2- 172.21.15.7:0/2442820104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d81579b0 0x7f51d815bf50 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f51c80098e0 tx=0x7f51c80091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.082+0000 7f51de4e5700 1 -- 172.21.15.7:0/2442820104 shutdown_connections 2024-01-11T07:16:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.082+0000 7f51de4e5700 1 --2- 172.21.15.7:0/2442820104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d81579b0 0x7f51d815bf50 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.082+0000 7f51de4e5700 1 -- 172.21.15.7:0/2442820104 >> 172.21.15.7:0/2442820104 conn(0x7f51d80680e0 msgr2=0x7f51d80cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.082+0000 7f51de4e5700 1 -- 172.21.15.7:0/2442820104 shutdown_connections 2024-01-11T07:16:40.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.082+0000 7f51de4e5700 1 -- 172.21.15.7:0/2442820104 wait complete. 2024-01-11T07:16:40.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.083+0000 7f51de4e5700 1 Processor -- start 2024-01-11T07:16:40.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.083+0000 7f51de4e5700 1 -- start start 2024-01-11T07:16:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51de4e5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51d8063940 0x7f51d80c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51de4e5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d81579b0 0x7f51d80abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51de4e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d8157d80 0x7f51d8156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51de4e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80ac450 con 0x7f51d8157d80 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51de4e5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81590b0 con 0x7f51d81579b0 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51de4e5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81572b0 con 0x7f51d8063940 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51dca82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d8157d80 0x7f51d8156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51dca82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d8157d80 0x7f51d8156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56086/0 (socket says 172.21.15.7:56086) 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51dca82700 1 -- 172.21.15.7:0/3468371759 learned_addr learned my addr 172.21.15.7:0/3468371759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51d77fe700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d81579b0 0x7f51d80abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.084+0000 7f51d7fff700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51d8063940 0x7f51d80c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51dca82700 1 -- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51d8063940 msgr2=0x7f51d80c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51dca82700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51d8063940 0x7f51d80c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51dca82700 1 -- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d81579b0 msgr2=0x7f51d80abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51dca82700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d81579b0 0x7f51d80abf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51dca82700 1 -- 172.21.15.7:0/3468371759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8009420 con 0x7f51d8157d80 2024-01-11T07:16:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51dca82700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d8157d80 0x7f51d8156d70 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c910 tx=0x7f51d00016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51bffff700 1 -- 172.21.15.7:0/3468371759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0012630 con 0x7f51d8157d80 2024-01-11T07:16:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51bffff700 1 -- 172.21.15.7:0/3468371759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0012790 con 0x7f51d8157d80 2024-01-11T07:16:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51bffff700 1 -- 172.21.15.7:0/3468371759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0012b50 con 0x7f51d8157d80 2024-01-11T07:16:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d81584a0 con 0x7f51d8157d80 2024-01-11T07:16:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.085+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d8158a60 con 0x7f51d8157d80 2024-01-11T07:16:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.086+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d804c3c0 con 0x7f51d8157d80 2024-01-11T07:16:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.087+0000 7f51bffff700 1 -- 172.21.15.7:0/3468371759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f51d001b450 con 0x7f51d8157d80 2024-01-11T07:16:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.087+0000 7f51bffff700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51c4066f30 0x7f51c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.087+0000 7f51bffff700 1 -- 172.21.15.7:0/3468371759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f51d00846a0 con 0x7f51d8157d80 2024-01-11T07:16:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.087+0000 7f51d7fff700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51c4066f30 0x7f51c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.088+0000 7f51d7fff700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51c4066f30 0x7f51c40693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51c80093f0 tx=0x7f51c8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.092+0000 7f51bffff700 1 -- 172.21.15.7:0/3468371759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51d0051910 con 0x7f51d8157d80 2024-01-11T07:16:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.296+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51d80af630 con 0x7f51d8157d80 2024-01-11T07:16:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.297+0000 7f51bffff700 1 -- 172.21.15.7:0/3468371759 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+15317 (secure 0 0 0) 0x7f51d0054f30 con 0x7f51d8157d80 2024-01-11T07:16:40.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:40.297 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:16:37.489150+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:16:33.823565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-11T07:16:34.902544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51c4066f30 msgr2=0x7f51c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51c4066f30 0x7f51c40693e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51c80093f0 tx=0x7f51c8017000 comp rx=0 tx=0).stop 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d8157d80 msgr2=0x7f51d8156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d8157d80 0x7f51d8156d70 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c910 tx=0x7f51d00016f0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 shutdown_connections 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f51c4066f30 0x7f51c40693e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51d8157d80 0x7f51d8156d70 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51d8063940 0x7f51d80c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 --2- 172.21.15.7:0/3468371759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d81579b0 0x7f51d80abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 >> 172.21.15.7:0/3468371759 conn(0x7f51d80680e0 msgr2=0x7f51d806d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 shutdown_connections 2024-01-11T07:16:40.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.299+0000 7f51de4e5700 1 -- 172.21.15.7:0/3468371759 wait complete. 2024-01-11T07:16:40.313 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:16:40.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:16:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.446+0000 7f9baedbf700 1 Processor -- start 2024-01-11T07:16:40.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.446+0000 7f9baedbf700 1 -- start start 2024-01-11T07:16:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.447+0000 7f9baedbf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba80cd6d0 0x7f9ba80cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.447+0000 7f9baedbf700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ba815d8a0 0x7f9ba815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.447+0000 7f9baedbf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba815e690 con 0x7f9ba815e250 2024-01-11T07:16:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.447+0000 7f9baedbf700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba815acf0 con 0x7f9ba815d8a0 2024-01-11T07:16:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.447+0000 7f9baedbf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba815afd0 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.447+0000 7f9bad35c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9ba815e250 0x7f9ba80cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57924/0 (socket says 172.21.15.7:57924) 2024-01-11T07:16:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.447+0000 7f9bad35c700 1 -- 172.21.15.7:0/3963099282 learned_addr learned my addr 172.21.15.7:0/3963099282 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba7fff700 1 --2- 172.21.15.7:0/3963099282 >> v2:172.21.15.113:3300/0 conn(0x7f9ba815d8a0 0x7f9ba815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9bacb5b700 1 --2- 172.21.15.7:0/3963099282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba80cd6d0 0x7f9ba80cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba77fe700 1 -- 172.21.15.7:0/3963099282 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078635682 0 0) 0x7f9ba815e690 con 0x7f9ba815e250 2024-01-11T07:16:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba77fe700 1 -- 172.21.15.7:0/3963099282 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b98003570 con 0x7f9ba815e250 2024-01-11T07:16:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba7fff700 1 -- 172.21.15.7:0/3963099282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba80cd6d0 msgr2=0x7f9ba80cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba7fff700 1 --2- 172.21.15.7:0/3963099282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba80cd6d0 0x7f9ba80cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba7fff700 1 -- 172.21.15.7:0/3963099282 >> v1:172.21.15.7:6789/0 conn(0x7f9ba815e250 legacy=0x7f9ba80cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba7fff700 1 -- 172.21.15.7:0/3963099282 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80ce980 con 0x7f9ba815d8a0 2024-01-11T07:16:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba7fff700 1 --2- 172.21.15.7:0/3963099282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815d8a0 0x7f9ba815dd10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9b940098e0 tx=0x7f9b940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66379f54029e806 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba77fe700 1 -- 172.21.15.7:0/3963099282 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94011630 con 0x7f9ba815d8a0 2024-01-11T07:16:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba77fe700 1 -- 172.21.15.7:0/3963099282 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b94011790 con 0x7f9ba815d8a0 2024-01-11T07:16:40.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.448+0000 7f9ba77fe700 1 -- 172.21.15.7:0/3963099282 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94011b30 con 0x7f9ba815d8a0 2024-01-11T07:16:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.449+0000 7f9baedbf700 1 -- 172.21.15.7:0/3963099282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815d8a0 msgr2=0x7f9ba815dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.449+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3963099282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815d8a0 0x7f9ba815dd10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9b940098e0 tx=0x7f9b940091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.449+0000 7f9baedbf700 1 -- 172.21.15.7:0/3963099282 shutdown_connections 2024-01-11T07:16:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.449+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3963099282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba80cd6d0 0x7f9ba80cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.449+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3963099282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815d8a0 0x7f9ba815dd10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.449+0000 7f9baedbf700 1 -- 172.21.15.7:0/3963099282 >> 172.21.15.7:0/3963099282 conn(0x7f9ba8071120 msgr2=0x7f9ba806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.449+0000 7f9baedbf700 1 -- 172.21.15.7:0/3963099282 shutdown_connections 2024-01-11T07:16:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.449+0000 7f9baedbf700 1 -- 172.21.15.7:0/3963099282 wait complete. 2024-01-11T07:16:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.450+0000 7f9baedbf700 1 Processor -- start 2024-01-11T07:16:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9baedbf700 1 -- start start 2024-01-11T07:16:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9baedbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ba80cd6d0 0x7f9ba80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9baedbf700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba815d8a0 0x7f9ba816a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9baedbf700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815e250 0x7f9ba815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9baedbf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80cabc0 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9baedbf700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba816a8a0 con 0x7f9ba815e250 2024-01-11T07:16:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9baedbf700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba816ab80 con 0x7f9ba815d8a0 2024-01-11T07:16:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9bacb5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ba80cd6d0 0x7f9ba80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9bacb5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ba80cd6d0 0x7f9ba80b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56102/0 (socket says 172.21.15.7:56102) 2024-01-11T07:16:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.451+0000 7f9bacb5b700 1 -- 172.21.15.7:0/3435327193 learned_addr learned my addr 172.21.15.7:0/3435327193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9ba7fff700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba815d8a0 0x7f9ba816a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9bad35c700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815e250 0x7f9ba815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9bacb5b700 1 -- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba815d8a0 msgr2=0x7f9ba816a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9bacb5b700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba815d8a0 0x7f9ba816a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9bacb5b700 1 -- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815e250 msgr2=0x7f9ba815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9bacb5b700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815e250 0x7f9ba815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9bacb5b700 1 -- 172.21.15.7:0/3435327193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94009420 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9bacb5b700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ba80cd6d0 0x7f9ba80b4d60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00ec80 tx=0x7f9b9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9ba4ff9700 1 -- 172.21.15.7:0/3435327193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c009690 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9ba4ff9700 1 -- 172.21.15.7:0/3435327193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c00c850 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba816ae60 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9ba4ff9700 1 -- 172.21.15.7:0/3435327193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00cc10 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.452+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba80ca7e0 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.453+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba80c93d0 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.454+0000 7f9ba4ff9700 1 -- 172.21.15.7:0/3435327193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9b9c013470 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.454+0000 7f9ba4ff9700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b8c066f30 0x7f9b8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.454+0000 7f9ba4ff9700 1 -- 172.21.15.7:0/3435327193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9b9c084690 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.454+0000 7f9ba7fff700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b8c066f30 0x7f9b8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.455+0000 7f9ba7fff700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b8c066f30 0x7f9b8c0693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9b940091e0 tx=0x7f9b94017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.459+0000 7f9ba4ff9700 1 -- 172.21.15.7:0/3435327193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b9c055390 con 0x7f9ba80cd6d0 2024-01-11T07:16:40.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.661+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ba80cad00 con 0x7f9b8c066f30 2024-01-11T07:16:40.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.669+0000 7f9ba4ff9700 1 -- 172.21.15.7:0/3435327193 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285879 (secure 0 0 0) 0x7f9b94011900 con 0x7f9b8c066f30 2024-01-11T07:16:40.673 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b8c066f30 msgr2=0x7f9b8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b8c066f30 0x7f9b8c0693e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9b940091e0 tx=0x7f9b94017000 comp rx=0 tx=0).stop 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ba80cd6d0 msgr2=0x7f9ba80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ba80cd6d0 0x7f9ba80b4d60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00ec80 tx=0x7f9b9c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 shutdown_connections 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9b8c066f30 0x7f9b8c0693e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ba815e250 0x7f9ba815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ba80cd6d0 0x7f9ba80b4d60 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 --2- 172.21.15.7:0/3435327193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ba815d8a0 0x7f9ba816a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 >> 172.21.15.7:0/3435327193 conn(0x7f9ba8071120 msgr2=0x7f9ba80cfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 shutdown_connections 2024-01-11T07:16:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.673+0000 7f9baedbf700 1 -- 172.21.15.7:0/3435327193 wait complete. 2024-01-11T07:16:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:16:40.687 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-01-11T07:16:39.979764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8459,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554040,"statfs":{"total":1600000000,"available":1591337932,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.494656"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505163+0000","last_change":"2024-01-11T07:16:36.505163+0000","last_active":"2024-01-11T07:16:36.505163+0000","last_peered":"2024-01-11T07:16:36.505163+0000","last_clean":"2024-01-11T07:16:36.505163+0000","last_became_active":"2024-01-11T07:16:36.504025+0000","last_became_peered":"2024-01-11T07:16:36.504025+0000","last_unstale":"2024-01-11T07:16:36.505163+0000","last_undegraded":"2024-01-11T07:16:36.505163+0000","last_fullsized":"2024-01-11T07:16:36.505163+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:37:32.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.502980+0000","last_change":"2024-01-11T07:16:36.502980+0000","last_active":"2024-01-11T07:16:36.502980+0000","last_peered":"2024-01-11T07:16:36.502980+0000","last_clean":"2024-01-11T07:16:36.502980+0000","last_became_active":"2024-01-11T07:16:36.502710+0000","last_became_peered":"2024-01-11T07:16:36.502710+0000","last_unstale":"2024-01-11T07:16:36.502980+0000","last_undegraded":"2024-01-11T07:16:36.502980+0000","last_fullsized":"2024-01-11T07:16:36.502980+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:29:26.090555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501525+0000","last_change":"2024-01-11T07:16:36.501429+0000","last_active":"2024-01-11T07:16:36.501525+0000","last_peered":"2024-01-11T07:16:36.501525+0000","last_clean":"2024-01-11T07:16:36.501525+0000","last_became_active":"2024-01-11T07:16:36.501018+0000","last_became_peered":"2024-01-11T07:16:36.501018+0000","last_unstale":"2024-01-11T07:16:36.501525+0000","last_undegraded":"2024-01-11T07:16:36.501525+0000","last_fullsized":"2024-01-11T07:16:36.501525+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:09:13.989257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501723+0000","last_change":"2024-01-11T07:16:36.501226+0000","last_active":"2024-01-11T07:16:36.501723+0000","last_peered":"2024-01-11T07:16:36.501723+0000","last_clean":"2024-01-11T07:16:36.501723+0000","last_became_active":"2024-01-11T07:16:36.500968+0000","last_became_peered":"2024-01-11T07:16:36.500968+0000","last_unstale":"2024-01-11T07:16:36.501723+0000","last_undegraded":"2024-01-11T07:16:36.501723+0000","last_fullsized":"2024-01-11T07:16:36.501723+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:13.517356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.500570+0000","last_change":"2024-01-11T07:16:36.500475+0000","last_active":"2024-01-11T07:16:36.500570+0000","last_peered":"2024-01-11T07:16:36.500570+0000","last_clean":"2024-01-11T07:16:36.500570+0000","last_became_active":"2024-01-11T07:16:36.500233+0000","last_became_peered":"2024-01-11T07:16:36.500233+0000","last_unstale":"2024-01-11T07:16:36.500570+0000","last_undegraded":"2024-01-11T07:16:36.500570+0000","last_fullsized":"2024-01-11T07:16:36.500570+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:10:04.224492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.498376+0000","last_change":"2024-01-11T07:16:36.498376+0000","last_active":"2024-01-11T07:16:36.498376+0000","last_peered":"2024-01-11T07:16:36.498376+0000","last_clean":"2024-01-11T07:16:36.498376+0000","last_became_active":"2024-01-11T07:16:36.497929+0000","last_became_peered":"2024-01-11T07:16:36.497929+0000","last_unstale":"2024-01-11T07:16:36.498376+0000","last_undegraded":"2024-01-11T07:16:36.498376+0000","last_fullsized":"2024-01-11T07:16:36.498376+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:33.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501987+0000","last_change":"2024-01-11T07:16:36.501846+0000","last_active":"2024-01-11T07:16:36.501987+0000","last_peered":"2024-01-11T07:16:36.501987+0000","last_clean":"2024-01-11T07:16:36.501987+0000","last_became_active":"2024-01-11T07:16:36.501526+0000","last_became_peered":"2024-01-11T07:16:36.501526+0000","last_unstale":"2024-01-11T07:16:36.501987+0000","last_undegraded":"2024-01-11T07:16:36.501987+0000","last_fullsized":"2024-01-11T07:16:36.501987+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:41:43.262266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501311+0000","last_change":"2024-01-11T07:16:36.501311+0000","last_active":"2024-01-11T07:16:36.501311+0000","last_peered":"2024-01-11T07:16:36.501311+0000","last_clean":"2024-01-11T07:16:36.501311+0000","last_became_active":"2024-01-11T07:16:36.500980+0000","last_became_peered":"2024-01-11T07:16:36.500980+0000","last_unstale":"2024-01-11T07:16:36.501311+0000","last_undegraded":"2024-01-11T07:16:36.501311+0000","last_fullsized":"2024-01-11T07:16:36.501311+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:06:55.151282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.498120+0000","last_change":"2024-01-11T07:16:36.498120+0000","last_active":"2024-01-11T07:16:36.498120+0000","last_peered":"2024-01-11T07:16:36.498120+0000","last_clean":"2024-01-11T07:16:36.498120+0000","last_became_active":"2024-01-11T07:16:36.497905+0000","last_became_peered":"2024-01-11T07:16:36.497905+0000","last_unstale":"2024-01-11T07:16:36.498120+0000","last_undegraded":"2024-01-11T07:16:36.498120+0000","last_fullsized":"2024-01-11T07:16:36.498120+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:01:39.153593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505060+0000","last_change":"2024-01-11T07:16:36.504912+0000","last_active":"2024-01-11T07:16:36.505060+0000","last_peered":"2024-01-11T07:16:36.505060+0000","last_clean":"2024-01-11T07:16:36.505060+0000","last_became_active":"2024-01-11T07:16:36.504688+0000","last_became_peered":"2024-01-11T07:16:36.504688+0000","last_unstale":"2024-01-11T07:16:36.505060+0000","last_undegraded":"2024-01-11T07:16:36.505060+0000","last_fullsized":"2024-01-11T07:16:36.505060+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:06.421656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.498908+0000","last_change":"2024-01-11T07:16:36.498908+0000","last_active":"2024-01-11T07:16:36.498908+0000","last_peered":"2024-01-11T07:16:36.498908+0000","last_clean":"2024-01-11T07:16:36.498908+0000","last_became_active":"2024-01-11T07:16:36.498638+0000","last_became_peered":"2024-01-11T07:16:36.498638+0000","last_unstale":"2024-01-11T07:16:36.498908+0000","last_undegraded":"2024-01-11T07:16:36.498908+0000","last_fullsized":"2024-01-11T07:16:36.498908+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:57.174064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.500063+0000","last_change":"2024-01-11T07:16:36.499999+0000","last_active":"2024-01-11T07:16:36.500063+0000","last_peered":"2024-01-11T07:16:36.500063+0000","last_clean":"2024-01-11T07:16:36.500063+0000","last_became_active":"2024-01-11T07:16:36.499745+0000","last_became_peered":"2024-01-11T07:16:36.499745+0000","last_unstale":"2024-01-11T07:16:36.500063+0000","last_undegraded":"2024-01-11T07:16:36.500063+0000","last_fullsized":"2024-01-11T07:16:36.500063+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:10:02.982128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501960+0000","last_change":"2024-01-11T07:16:36.501798+0000","last_active":"2024-01-11T07:16:36.501960+0000","last_peered":"2024-01-11T07:16:36.501960+0000","last_clean":"2024-01-11T07:16:36.501960+0000","last_became_active":"2024-01-11T07:16:36.501293+0000","last_became_peered":"2024-01-11T07:16:36.501293+0000","last_unstale":"2024-01-11T07:16:36.501960+0000","last_undegraded":"2024-01-11T07:16:36.501960+0000","last_fullsized":"2024-01-11T07:16:36.501960+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:46:42.393585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501163+0000","last_change":"2024-01-11T07:16:36.500293+0000","last_active":"2024-01-11T07:16:36.501163+0000","last_peered":"2024-01-11T07:16:36.501163+0000","last_clean":"2024-01-11T07:16:36.501163+0000","last_became_active":"2024-01-11T07:16:36.500053+0000","last_became_peered":"2024-01-11T07:16:36.500053+0000","last_unstale":"2024-01-11T07:16:36.501163+0000","last_undegraded":"2024-01-11T07:16:36.501163+0000","last_fullsized":"2024-01-11T07:16:36.501163+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:34:19.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.508506+0000","last_change":"2024-01-11T07:16:36.508440+0000","last_active":"2024-01-11T07:16:36.508506+0000","last_peered":"2024-01-11T07:16:36.508506+0000","last_clean":"2024-01-11T07:16:36.508506+0000","last_became_active":"2024-01-11T07:16:36.508178+0000","last_became_peered":"2024-01-11T07:16:36.508178+0000","last_unstale":"2024-01-11T07:16:36.508506+0000","last_undegraded":"2024-01-11T07:16:36.508506+0000","last_fullsized":"2024-01-11T07:16:36.508506+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:43.153489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.507952+0000","last_change":"2024-01-11T07:16:36.507952+0000","last_active":"2024-01-11T07:16:36.507952+0000","last_peered":"2024-01-11T07:16:36.507952+0000","last_clean":"2024-01-11T07:16:36.507952+0000","last_became_active":"2024-01-11T07:16:36.507656+0000","last_became_peered":"2024-01-11T07:16:36.507656+0000","last_unstale":"2024-01-11T07:16:36.507952+0000","last_undegraded":"2024-01-11T07:16:36.507952+0000","last_fullsized":"2024-01-11T07:16:36.507952+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:38:15.114347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505030+0000","last_change":"2024-01-11T07:16:36.504237+0000","last_active":"2024-01-11T07:16:36.505030+0000","last_peered":"2024-01-11T07:16:36.505030+0000","last_clean":"2024-01-11T07:16:36.505030+0000","last_became_active":"2024-01-11T07:16:36.503969+0000","last_became_peered":"2024-01-11T07:16:36.503969+0000","last_unstale":"2024-01-11T07:16:36.505030+0000","last_undegraded":"2024-01-11T07:16:36.505030+0000","last_fullsized":"2024-01-11T07:16:36.505030+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:12:56.271162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501491+0000","last_change":"2024-01-11T07:16:36.501277+0000","last_active":"2024-01-11T07:16:36.501491+0000","last_peered":"2024-01-11T07:16:36.501491+0000","last_clean":"2024-01-11T07:16:36.501491+0000","last_became_active":"2024-01-11T07:16:36.500690+0000","last_became_peered":"2024-01-11T07:16:36.500690+0000","last_unstale":"2024-01-11T07:16:36.501491+0000","last_undegraded":"2024-01-11T07:16:36.501491+0000","last_fullsized":"2024-01-11T07:16:36.501491+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:41:07.175635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512245+0000","last_change":"2024-01-11T07:16:36.512245+0000","last_active":"2024-01-11T07:16:36.512245+0000","last_peered":"2024-01-11T07:16:36.512245+0000","last_clean":"2024-01-11T07:16:36.512245+0000","last_became_active":"2024-01-11T07:16:36.511982+0000","last_became_peered":"2024-01-11T07:16:36.511982+0000","last_unstale":"2024-01-11T07:16:36.512245+0000","last_undegraded":"2024-01-11T07:16:36.512245+0000","last_fullsized":"2024-01-11T07:16:36.512245+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:37:20.568288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.516474+0000","last_change":"2024-01-11T07:16:36.516474+0000","last_active":"2024-01-11T07:16:36.516474+0000","last_peered":"2024-01-11T07:16:36.516474+0000","last_clean":"2024-01-11T07:16:36.516474+0000","last_became_active":"2024-01-11T07:16:36.516240+0000","last_became_peered":"2024-01-11T07:16:36.516240+0000","last_unstale":"2024-01-11T07:16:36.516474+0000","last_undegraded":"2024-01-11T07:16:36.516474+0000","last_fullsized":"2024-01-11T07:16:36.516474+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:08:29.474805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505045+0000","last_change":"2024-01-11T07:16:36.504765+0000","last_active":"2024-01-11T07:16:36.505045+0000","last_peered":"2024-01-11T07:16:36.505045+0000","last_clean":"2024-01-11T07:16:36.505045+0000","last_became_active":"2024-01-11T07:16:36.504499+0000","last_became_peered":"2024-01-11T07:16:36.504499+0000","last_unstale":"2024-01-11T07:16:36.505045+0000","last_undegraded":"2024-01-11T07:16:36.505045+0000","last_fullsized":"2024-01-11T07:16:36.505045+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:50:43.199972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501137+0000","last_change":"2024-01-11T07:16:36.499636+0000","last_active":"2024-01-11T07:16:36.501137+0000","last_peered":"2024-01-11T07:16:36.501137+0000","last_clean":"2024-01-11T07:16:36.501137+0000","last_became_active":"2024-01-11T07:16:36.499305+0000","last_became_peered":"2024-01-11T07:16:36.499305+0000","last_unstale":"2024-01-11T07:16:36.501137+0000","last_undegraded":"2024-01-11T07:16:36.501137+0000","last_fullsized":"2024-01-11T07:16:36.501137+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:06:55.151282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.502696+0000","last_change":"2024-01-11T07:16:36.502218+0000","last_active":"2024-01-11T07:16:36.502696+0000","last_peered":"2024-01-11T07:16:36.502696+0000","last_clean":"2024-01-11T07:16:36.502696+0000","last_became_active":"2024-01-11T07:16:36.501987+0000","last_became_peered":"2024-01-11T07:16:36.501987+0000","last_unstale":"2024-01-11T07:16:36.502696+0000","last_undegraded":"2024-01-11T07:16:36.502696+0000","last_fullsized":"2024-01-11T07:16:36.502696+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:33.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.504995+0000","last_change":"2024-01-11T07:16:36.502174+0000","last_active":"2024-01-11T07:16:36.504995+0000","last_peered":"2024-01-11T07:16:36.504995+0000","last_clean":"2024-01-11T07:16:36.504995+0000","last_became_active":"2024-01-11T07:16:36.501800+0000","last_became_peered":"2024-01-11T07:16:36.501800+0000","last_unstale":"2024-01-11T07:16:36.504995+0000","last_undegraded":"2024-01-11T07:16:36.504995+0000","last_fullsized":"2024-01-11T07:16:36.504995+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:53.191130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.508210+0000","last_change":"2024-01-11T07:16:36.508210+0000","last_active":"2024-01-11T07:16:36.508210+0000","last_peered":"2024-01-11T07:16:36.508210+0000","last_clean":"2024-01-11T07:16:36.508210+0000","last_became_active":"2024-01-11T07:16:36.507384+0000","last_became_peered":"2024-01-11T07:16:36.507384+0000","last_unstale":"2024-01-11T07:16:36.508210+0000","last_undegraded":"2024-01-11T07:16:36.508210+0000","last_fullsized":"2024-01-11T07:16:36.508210+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:07:15.105894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.514293+0000","last_change":"2024-01-11T07:16:36.514293+0000","last_active":"2024-01-11T07:16:36.514293+0000","last_peered":"2024-01-11T07:16:36.514293+0000","last_clean":"2024-01-11T07:16:36.514293+0000","last_became_active":"2024-01-11T07:16:36.513319+0000","last_became_peered":"2024-01-11T07:16:36.513319+0000","last_unstale":"2024-01-11T07:16:36.514293+0000","last_undegraded":"2024-01-11T07:16:36.514293+0000","last_fullsized":"2024-01-11T07:16:36.514293+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:13.517356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.509569+0000","last_change":"2024-01-11T07:16:36.509569+0000","last_active":"2024-01-11T07:16:36.509569+0000","last_peered":"2024-01-11T07:16:36.509569+0000","last_clean":"2024-01-11T07:16:36.509569+0000","last_became_active":"2024-01-11T07:16:36.509327+0000","last_became_peered":"2024-01-11T07:16:36.509327+0000","last_unstale":"2024-01-11T07:16:36.509569+0000","last_undegraded":"2024-01-11T07:16:36.509569+0000","last_fullsized":"2024-01-11T07:16:36.509569+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:34.462281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.494296+0000","last_change":"2024-01-11T07:16:36.494220+0000","last_active":"2024-01-11T07:16:36.494296+0000","last_peered":"2024-01-11T07:16:36.494296+0000","last_clean":"2024-01-11T07:16:36.494296+0000","last_became_active":"2024-01-11T07:16:36.493971+0000","last_became_peered":"2024-01-11T07:16:36.493971+0000","last_unstale":"2024-01-11T07:16:36.494296+0000","last_undegraded":"2024-01-11T07:16:36.494296+0000","last_fullsized":"2024-01-11T07:16:36.494296+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:41.270820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505324+0000","last_change":"2024-01-11T07:16:36.505324+0000","last_active":"2024-01-11T07:16:36.505324+0000","last_peered":"2024-01-11T07:16:36.505324+0000","last_clean":"2024-01-11T07:16:36.505324+0000","last_became_active":"2024-01-11T07:16:36.504846+0000","last_became_peered":"2024-01-11T07:16:36.504846+0000","last_unstale":"2024-01-11T07:16:36.505324+0000","last_undegraded":"2024-01-11T07:16:36.505324+0000","last_fullsized":"2024-01-11T07:16:36.505324+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:05:43.217209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.504066+0000","last_change":"2024-01-11T07:16:36.503990+0000","last_active":"2024-01-11T07:16:36.504066+0000","last_peered":"2024-01-11T07:16:36.504066+0000","last_clean":"2024-01-11T07:16:36.504066+0000","last_became_active":"2024-01-11T07:16:36.503709+0000","last_became_peered":"2024-01-11T07:16:36.503709+0000","last_unstale":"2024-01-11T07:16:36.504066+0000","last_undegraded":"2024-01-11T07:16:36.504066+0000","last_fullsized":"2024-01-11T07:16:36.504066+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:37:32.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.497676+0000","last_change":"2024-01-11T07:16:36.497676+0000","last_active":"2024-01-11T07:16:36.497676+0000","last_peered":"2024-01-11T07:16:36.497676+0000","last_clean":"2024-01-11T07:16:36.497676+0000","last_became_active":"2024-01-11T07:16:36.497429+0000","last_became_peered":"2024-01-11T07:16:36.497429+0000","last_unstale":"2024-01-11T07:16:36.497676+0000","last_undegraded":"2024-01-11T07:16:36.497676+0000","last_fullsized":"2024-01-11T07:16:36.497676+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:50:43.199972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.508686+0000","last_change":"2024-01-11T07:16:36.508686+0000","last_active":"2024-01-11T07:16:36.508686+0000","last_peered":"2024-01-11T07:16:36.508686+0000","last_clean":"2024-01-11T07:16:36.508686+0000","last_became_active":"2024-01-11T07:16:36.506780+0000","last_became_peered":"2024-01-11T07:16:36.506780+0000","last_unstale":"2024-01-11T07:16:36.508686+0000","last_undegraded":"2024-01-11T07:16:36.508686+0000","last_fullsized":"2024-01-11T07:16:36.508686+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:26:46.565949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499390+0000","last_change":"2024-01-11T07:16:36.498465+0000","last_active":"2024-01-11T07:16:36.499390+0000","last_peered":"2024-01-11T07:16:36.499390+0000","last_clean":"2024-01-11T07:16:36.499390+0000","last_became_active":"2024-01-11T07:16:36.498096+0000","last_became_peered":"2024-01-11T07:16:36.498096+0000","last_unstale":"2024-01-11T07:16:36.499390+0000","last_undegraded":"2024-01-11T07:16:36.499390+0000","last_fullsized":"2024-01-11T07:16:36.499390+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:32:41.616964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"167'4","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.513460+0000","last_change":"2024-01-11T07:16:35.505257+0000","last_active":"2024-01-11T07:16:36.513460+0000","last_peered":"2024-01-11T07:16:36.513460+0000","last_clean":"2024-01-11T07:16:36.513460+0000","last_became_active":"2024-01-11T07:16:35.505038+0000","last_became_peered":"2024-01-11T07:16:35.505038+0000","last_unstale":"2024-01-11T07:16:36.513460+0000","last_undegraded":"2024-01-11T07:16:36.513460+0000","last_fullsized":"2024-01-11T07:16:36.513460+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:38:02.490699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499104+0000","last_change":"2024-01-11T07:16:36.499104+0000","last_active":"2024-01-11T07:16:36.499104+0000","last_peered":"2024-01-11T07:16:36.499104+0000","last_clean":"2024-01-11T07:16:36.499104+0000","last_became_active":"2024-01-11T07:16:36.498795+0000","last_became_peered":"2024-01-11T07:16:36.498795+0000","last_unstale":"2024-01-11T07:16:36.499104+0000","last_undegraded":"2024-01-11T07:16:36.499104+0000","last_fullsized":"2024-01-11T07:16:36.499104+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:41.641936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"167'1","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.494340+0000","last_change":"2024-01-11T07:16:35.504601+0000","last_active":"2024-01-11T07:16:36.494340+0000","last_peered":"2024-01-11T07:16:36.494340+0000","last_clean":"2024-01-11T07:16:36.494340+0000","last_became_active":"2024-01-11T07:16:35.504009+0000","last_became_peered":"2024-01-11T07:16:35.504009+0000","last_unstale":"2024-01-11T07:16:36.494340+0000","last_undegraded":"2024-01-11T07:16:36.494340+0000","last_fullsized":"2024-01-11T07:16:36.494340+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:59:27.797942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505316+0000","last_change":"2024-01-11T07:16:36.505316+0000","last_active":"2024-01-11T07:16:36.505316+0000","last_peered":"2024-01-11T07:16:36.505316+0000","last_clean":"2024-01-11T07:16:36.505316+0000","last_became_active":"2024-01-11T07:16:36.504914+0000","last_became_peered":"2024-01-11T07:16:36.504914+0000","last_unstale":"2024-01-11T07:16:36.505316+0000","last_undegraded":"2024-01-11T07:16:36.505316+0000","last_fullsized":"2024-01-11T07:16:36.505316+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:37:41.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512866+0000","last_change":"2024-01-11T07:16:35.497036+0000","last_active":"2024-01-11T07:16:36.512866+0000","last_peered":"2024-01-11T07:16:36.512866+0000","last_clean":"2024-01-11T07:16:36.512866+0000","last_became_active":"2024-01-11T07:16:35.496767+0000","last_became_peered":"2024-01-11T07:16:35.496767+0000","last_unstale":"2024-01-11T07:16:36.512866+0000","last_undegraded":"2024-01-11T07:16:36.512866+0000","last_fullsized":"2024-01-11T07:16:36.512866+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:42:39.165288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505014+0000","last_change":"2024-01-11T07:16:36.503352+0000","last_active":"2024-01-11T07:16:36.505014+0000","last_peered":"2024-01-11T07:16:36.505014+0000","last_clean":"2024-01-11T07:16:36.505014+0000","last_became_active":"2024-01-11T07:16:36.503160+0000","last_became_peered":"2024-01-11T07:16:36.503160+0000","last_unstale":"2024-01-11T07:16:36.505014+0000","last_undegraded":"2024-01-11T07:16:36.505014+0000","last_fullsized":"2024-01-11T07:16:36.505014+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:49:28.386663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.514335+0000","last_change":"2024-01-11T07:16:35.505271+0000","last_active":"2024-01-11T07:16:36.514335+0000","last_peered":"2024-01-11T07:16:36.514335+0000","last_clean":"2024-01-11T07:16:36.514335+0000","last_became_active":"2024-01-11T07:16:35.505000+0000","last_became_peered":"2024-01-11T07:16:35.505000+0000","last_unstale":"2024-01-11T07:16:36.514335+0000","last_undegraded":"2024-01-11T07:16:36.514335+0000","last_fullsized":"2024-01-11T07:16:36.514335+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:12.601760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.503509+0000","last_change":"2024-01-11T07:16:36.503509+0000","last_active":"2024-01-11T07:16:36.503509+0000","last_peered":"2024-01-11T07:16:36.503509+0000","last_clean":"2024-01-11T07:16:36.503509+0000","last_became_active":"2024-01-11T07:16:36.502796+0000","last_became_peered":"2024-01-11T07:16:36.502796+0000","last_unstale":"2024-01-11T07:16:36.503509+0000","last_undegraded":"2024-01-11T07:16:36.503509+0000","last_fullsized":"2024-01-11T07:16:36.503509+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:53:18.161636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"167'1","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.506132+0000","last_change":"2024-01-11T07:16:35.504632+0000","last_active":"2024-01-11T07:16:36.506132+0000","last_peered":"2024-01-11T07:16:36.506132+0000","last_clean":"2024-01-11T07:16:36.506132+0000","last_became_active":"2024-01-11T07:16:35.504383+0000","last_became_peered":"2024-01-11T07:16:35.504383+0000","last_unstale":"2024-01-11T07:16:36.506132+0000","last_undegraded":"2024-01-11T07:16:36.506132+0000","last_fullsized":"2024-01-11T07:16:36.506132+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:08:44.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499700+0000","last_change":"2024-01-11T07:16:36.498985+0000","last_active":"2024-01-11T07:16:36.499700+0000","last_peered":"2024-01-11T07:16:36.499700+0000","last_clean":"2024-01-11T07:16:36.499700+0000","last_became_active":"2024-01-11T07:16:36.498734+0000","last_became_peered":"2024-01-11T07:16:36.498734+0000","last_unstale":"2024-01-11T07:16:36.499700+0000","last_undegraded":"2024-01-11T07:16:36.499700+0000","last_fullsized":"2024-01-11T07:16:36.499700+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:41:17.542100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512639+0000","last_change":"2024-01-11T07:16:35.503345+0000","last_active":"2024-01-11T07:16:36.512639+0000","last_peered":"2024-01-11T07:16:36.512639+0000","last_clean":"2024-01-11T07:16:36.512639+0000","last_became_active":"2024-01-11T07:16:35.502905+0000","last_became_peered":"2024-01-11T07:16:35.502905+0000","last_unstale":"2024-01-11T07:16:36.512639+0000","last_undegraded":"2024-01-11T07:16:36.512639+0000","last_fullsized":"2024-01-11T07:16:36.512639+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:26:45.560925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499993+0000","last_change":"2024-01-11T07:16:36.499993+0000","last_active":"2024-01-11T07:16:36.499993+0000","last_peered":"2024-01-11T07:16:36.499993+0000","last_clean":"2024-01-11T07:16:36.499993+0000","last_became_active":"2024-01-11T07:16:36.499383+0000","last_became_peered":"2024-01-11T07:16:36.499383+0000","last_unstale":"2024-01-11T07:16:36.499993+0000","last_undegraded":"2024-01-11T07:16:36.499993+0000","last_fullsized":"2024-01-11T07:16:36.499993+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:32:11.874290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.502206+0000","last_change":"2024-01-11T07:16:35.502093+0000","last_active":"2024-01-11T07:16:35.502206+0000","last_peered":"2024-01-11T07:16:35.502206+0000","last_clean":"2024-01-11T07:16:35.502206+0000","last_became_active":"2024-01-11T07:16:35.501592+0000","last_became_peered":"2024-01-11T07:16:35.501592+0000","last_unstale":"2024-01-11T07:16:35.502206+0000","last_undegraded":"2024-01-11T07:16:35.502206+0000","last_fullsized":"2024-01-11T07:16:35.502206+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:04.421915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.500550+0000","last_change":"2024-01-11T07:16:36.500266+0000","last_active":"2024-01-11T07:16:36.500550+0000","last_peered":"2024-01-11T07:16:36.500550+0000","last_clean":"2024-01-11T07:16:36.500550+0000","last_became_active":"2024-01-11T07:16:36.499905+0000","last_became_peered":"2024-01-11T07:16:36.499905+0000","last_unstale":"2024-01-11T07:16:36.500550+0000","last_undegraded":"2024-01-11T07:16:36.500550+0000","last_fullsized":"2024-01-11T07:16:36.500550+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:51:11.550574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.502923+0000","last_change":"2024-01-11T07:16:35.503460+0000","last_active":"2024-01-11T07:16:36.502923+0000","last_peered":"2024-01-11T07:16:36.502923+0000","last_clean":"2024-01-11T07:16:36.502923+0000","last_became_active":"2024-01-11T07:16:35.503215+0000","last_became_peered":"2024-01-11T07:16:35.503215+0000","last_unstale":"2024-01-11T07:16:36.502923+0000","last_undegraded":"2024-01-11T07:16:36.502923+0000","last_fullsized":"2024-01-11T07:16:36.502923+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:33:05.083702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.507734+0000","last_change":"2024-01-11T07:16:36.507670+0000","last_active":"2024-01-11T07:16:36.507734+0000","last_peered":"2024-01-11T07:16:36.507734+0000","last_clean":"2024-01-11T07:16:36.507734+0000","last_became_active":"2024-01-11T07:16:36.507320+0000","last_became_peered":"2024-01-11T07:16:36.507320+0000","last_unstale":"2024-01-11T07:16:36.507734+0000","last_undegraded":"2024-01-11T07:16:36.507734+0000","last_fullsized":"2024-01-11T07:16:36.507734+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:40:10.794059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.501070+0000","last_change":"2024-01-11T07:16:35.500997+0000","last_active":"2024-01-11T07:16:35.501070+0000","last_peered":"2024-01-11T07:16:35.501070+0000","last_clean":"2024-01-11T07:16:35.501070+0000","last_became_active":"2024-01-11T07:16:35.500757+0000","last_became_peered":"2024-01-11T07:16:35.500757+0000","last_unstale":"2024-01-11T07:16:35.501070+0000","last_undegraded":"2024-01-11T07:16:35.501070+0000","last_fullsized":"2024-01-11T07:16:35.501070+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:54:14.924711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499421+0000","last_change":"2024-01-11T07:16:36.499036+0000","last_active":"2024-01-11T07:16:36.499421+0000","last_peered":"2024-01-11T07:16:36.499421+0000","last_clean":"2024-01-11T07:16:36.499421+0000","last_became_active":"2024-01-11T07:16:36.498798+0000","last_became_peered":"2024-01-11T07:16:36.498798+0000","last_unstale":"2024-01-11T07:16:36.499421+0000","last_undegraded":"2024-01-11T07:16:36.499421+0000","last_fullsized":"2024-01-11T07:16:36.499421+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:45:50.182569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.495048+0000","last_change":"2024-01-11T07:16:35.506320+0000","last_active":"2024-01-11T07:16:36.495048+0000","last_peered":"2024-01-11T07:16:36.495048+0000","last_clean":"2024-01-11T07:16:36.495048+0000","last_became_active":"2024-01-11T07:16:35.505622+0000","last_became_peered":"2024-01-11T07:16:35.505622+0000","last_unstale":"2024-01-11T07:16:36.495048+0000","last_undegraded":"2024-01-11T07:16:36.495048+0000","last_fullsized":"2024-01-11T07:16:36.495048+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:40.615120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.509523+0000","last_change":"2024-01-11T07:16:36.509523+0000","last_active":"2024-01-11T07:16:36.509523+0000","last_peered":"2024-01-11T07:16:36.509523+0000","last_clean":"2024-01-11T07:16:36.509523+0000","last_became_active":"2024-01-11T07:16:36.509010+0000","last_became_peered":"2024-01-11T07:16:36.509010+0000","last_unstale":"2024-01-11T07:16:36.509523+0000","last_undegraded":"2024-01-11T07:16:36.509523+0000","last_fullsized":"2024-01-11T07:16:36.509523+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:31:39.883912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"167'6","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.519790+0000","last_change":"2024-01-11T07:16:35.499343+0000","last_active":"2024-01-11T07:16:36.519790+0000","last_peered":"2024-01-11T07:16:36.519790+0000","last_clean":"2024-01-11T07:16:36.519790+0000","last_became_active":"2024-01-11T07:16:35.499095+0000","last_became_peered":"2024-01-11T07:16:35.499095+0000","last_unstale":"2024-01-11T07:16:36.519790+0000","last_undegraded":"2024-01-11T07:16:36.519790+0000","last_fullsized":"2024-01-11T07:16:36.519790+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:52.186106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505996+0000","last_change":"2024-01-11T07:16:36.505917+0000","last_active":"2024-01-11T07:16:36.505996+0000","last_peered":"2024-01-11T07:16:36.505996+0000","last_clean":"2024-01-11T07:16:36.505996+0000","last_became_active":"2024-01-11T07:16:36.505670+0000","last_became_peered":"2024-01-11T07:16:36.505670+0000","last_unstale":"2024-01-11T07:16:36.505996+0000","last_undegraded":"2024-01-11T07:16:36.505996+0000","last_fullsized":"2024-01-11T07:16:36.505996+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:50:38.815660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512172+0000","last_change":"2024-01-11T07:16:35.494583+0000","last_active":"2024-01-11T07:16:36.512172+0000","last_peered":"2024-01-11T07:16:36.512172+0000","last_clean":"2024-01-11T07:16:36.512172+0000","last_became_active":"2024-01-11T07:16:35.494264+0000","last_became_peered":"2024-01-11T07:16:35.494264+0000","last_unstale":"2024-01-11T07:16:36.512172+0000","last_undegraded":"2024-01-11T07:16:36.512172+0000","last_fullsized":"2024-01-11T07:16:36.512172+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:44:21.430212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.500151+0000","last_change":"2024-01-11T07:16:36.500151+0000","last_active":"2024-01-11T07:16:36.500151+0000","last_peered":"2024-01-11T07:16:36.500151+0000","last_clean":"2024-01-11T07:16:36.500151+0000","last_became_active":"2024-01-11T07:16:36.499736+0000","last_became_peered":"2024-01-11T07:16:36.499736+0000","last_unstale":"2024-01-11T07:16:36.500151+0000","last_undegraded":"2024-01-11T07:16:36.500151+0000","last_fullsized":"2024-01-11T07:16:36.500151+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:06.421656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.493355+0000","last_change":"2024-01-11T07:16:35.499968+0000","last_active":"2024-01-11T07:16:36.493355+0000","last_peered":"2024-01-11T07:16:36.493355+0000","last_clean":"2024-01-11T07:16:36.493355+0000","last_became_active":"2024-01-11T07:16:35.499437+0000","last_became_peered":"2024-01-11T07:16:35.499437+0000","last_unstale":"2024-01-11T07:16:36.493355+0000","last_undegraded":"2024-01-11T07:16:36.493355+0000","last_fullsized":"2024-01-11T07:16:36.493355+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:54:40.965598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501943+0000","last_change":"2024-01-11T07:16:36.501657+0000","last_active":"2024-01-11T07:16:36.501943+0000","last_peered":"2024-01-11T07:16:36.501943+0000","last_clean":"2024-01-11T07:16:36.501943+0000","last_became_active":"2024-01-11T07:16:36.500994+0000","last_became_peered":"2024-01-11T07:16:36.500994+0000","last_unstale":"2024-01-11T07:16:36.501943+0000","last_undegraded":"2024-01-11T07:16:36.501943+0000","last_fullsized":"2024-01-11T07:16:36.501943+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:48.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.503963+0000","last_change":"2024-01-11T07:16:35.502494+0000","last_active":"2024-01-11T07:16:35.503963+0000","last_peered":"2024-01-11T07:16:35.503963+0000","last_clean":"2024-01-11T07:16:35.503963+0000","last_became_active":"2024-01-11T07:16:35.501518+0000","last_became_peered":"2024-01-11T07:16:35.501518+0000","last_unstale":"2024-01-11T07:16:35.503963+0000","last_undegraded":"2024-01-11T07:16:35.503963+0000","last_fullsized":"2024-01-11T07:16:35.503963+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:11:17.891925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.507669+0000","last_change":"2024-01-11T07:16:36.507597+0000","last_active":"2024-01-11T07:16:36.507669+0000","last_peered":"2024-01-11T07:16:36.507669+0000","last_clean":"2024-01-11T07:16:36.507669+0000","last_became_active":"2024-01-11T07:16:36.507459+0000","last_became_peered":"2024-01-11T07:16:36.507459+0000","last_unstale":"2024-01-11T07:16:36.507669+0000","last_undegraded":"2024-01-11T07:16:36.507669+0000","last_fullsized":"2024-01-11T07:16:36.507669+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:50:54.198682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"167'8","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.519250+0000","last_change":"2024-01-11T07:16:35.500368+0000","last_active":"2024-01-11T07:16:36.519250+0000","last_peered":"2024-01-11T07:16:36.519250+0000","last_clean":"2024-01-11T07:16:36.519250+0000","last_became_active":"2024-01-11T07:16:35.500172+0000","last_became_peered":"2024-01-11T07:16:35.500172+0000","last_unstale":"2024-01-11T07:16:36.519250+0000","last_undegraded":"2024-01-11T07:16:36.519250+0000","last_fullsized":"2024-01-11T07:16:36.519250+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:12:05.320369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.507399+0000","last_change":"2024-01-11T07:16:36.507310+0000","last_active":"2024-01-11T07:16:36.507399+0000","last_peered":"2024-01-11T07:16:36.507399+0000","last_clean":"2024-01-11T07:16:36.507399+0000","last_became_active":"2024-01-11T07:16:36.506971+0000","last_became_peered":"2024-01-11T07:16:36.506971+0000","last_unstale":"2024-01-11T07:16:36.507399+0000","last_undegraded":"2024-01-11T07:16:36.507399+0000","last_fullsized":"2024-01-11T07:16:36.507399+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:47:10.377333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.511669+0000","last_change":"2024-01-11T07:16:35.499633+0000","last_active":"2024-01-11T07:16:36.511669+0000","last_peered":"2024-01-11T07:16:36.511669+0000","last_clean":"2024-01-11T07:16:36.511669+0000","last_became_active":"2024-01-11T07:16:35.499414+0000","last_became_peered":"2024-01-11T07:16:35.499414+0000","last_unstale":"2024-01-11T07:16:36.511669+0000","last_undegraded":"2024-01-11T07:16:36.511669+0000","last_fullsized":"2024-01-11T07:16:36.511669+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:33.457256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.516447+0000","last_change":"2024-01-11T07:16:36.516447+0000","last_active":"2024-01-11T07:16:36.516447+0000","last_peered":"2024-01-11T07:16:36.516447+0000","last_clean":"2024-01-11T07:16:36.516447+0000","last_became_active":"2024-01-11T07:16:36.508058+0000","last_became_peered":"2024-01-11T07:16:36.508058+0000","last_unstale":"2024-01-11T07:16:36.516447+0000","last_undegraded":"2024-01-11T07:16:36.516447+0000","last_fullsized":"2024-01-11T07:16:36.516447+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:17:17.116289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"167'2","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.511915+0000","last_change":"2024-01-11T07:16:35.497508+0000","last_active":"2024-01-11T07:16:36.511915+0000","last_peered":"2024-01-11T07:16:36.511915+0000","last_clean":"2024-01-11T07:16:36.511915+0000","last_became_active":"2024-01-11T07:16:35.497320+0000","last_became_peered":"2024-01-11T07:16:35.497320+0000","last_unstale":"2024-01-11T07:16:36.511915+0000","last_undegraded":"2024-01-11T07:16:36.511915+0000","last_fullsized":"2024-01-11T07:16:36.511915+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:43:59.719379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501588+0000","last_change":"2024-01-11T07:16:36.501509+0000","last_active":"2024-01-11T07:16:36.501588+0000","last_peered":"2024-01-11T07:16:36.501588+0000","last_clean":"2024-01-11T07:16:36.501588+0000","last_became_active":"2024-01-11T07:16:36.501127+0000","last_became_peered":"2024-01-11T07:16:36.501127+0000","last_unstale":"2024-01-11T07:16:36.501588+0000","last_undegraded":"2024-01-11T07:16:36.501588+0000","last_fullsized":"2024-01-11T07:16:36.501588+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:42:40.170313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"167'1","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501096+0000","last_change":"2024-01-11T07:16:35.503269+0000","last_active":"2024-01-11T07:16:36.501096+0000","last_peered":"2024-01-11T07:16:36.501096+0000","last_clean":"2024-01-11T07:16:36.501096+0000","last_became_active":"2024-01-11T07:16:35.503032+0000","last_became_peered":"2024-01-11T07:16:35.503032+0000","last_unstale":"2024-01-11T07:16:36.501096+0000","last_undegraded":"2024-01-11T07:16:36.501096+0000","last_fullsized":"2024-01-11T07:16:36.501096+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:24:24.061998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.498194+0000","last_change":"2024-01-11T07:16:36.498194+0000","last_active":"2024-01-11T07:16:36.498194+0000","last_peered":"2024-01-11T07:16:36.498194+0000","last_clean":"2024-01-11T07:16:36.498194+0000","last_became_active":"2024-01-11T07:16:36.497529+0000","last_became_peered":"2024-01-11T07:16:36.497529+0000","last_unstale":"2024-01-11T07:16:36.498194+0000","last_undegraded":"2024-01-11T07:16:36.498194+0000","last_fullsized":"2024-01-11T07:16:36.498194+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:41:17.542100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.499788+0000","last_change":"2024-01-11T07:16:35.499694+0000","last_active":"2024-01-11T07:16:35.499788+0000","last_peered":"2024-01-11T07:16:35.499788+0000","last_clean":"2024-01-11T07:16:35.499788+0000","last_became_active":"2024-01-11T07:16:35.499437+0000","last_became_peered":"2024-01-11T07:16:35.499437+0000","last_unstale":"2024-01-11T07:16:35.499788+0000","last_undegraded":"2024-01-11T07:16:35.499788+0000","last_fullsized":"2024-01-11T07:16:35.499788+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:40.615120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.503415+0000","last_change":"2024-01-11T07:16:36.503028+0000","last_active":"2024-01-11T07:16:36.503415+0000","last_peered":"2024-01-11T07:16:36.503415+0000","last_clean":"2024-01-11T07:16:36.503415+0000","last_became_active":"2024-01-11T07:16:36.502646+0000","last_became_peered":"2024-01-11T07:16:36.502646+0000","last_unstale":"2024-01-11T07:16:36.503415+0000","last_undegraded":"2024-01-11T07:16:36.503415+0000","last_fullsized":"2024-01-11T07:16:36.503415+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:37:41.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.511391+0000","last_change":"2024-01-11T07:16:35.502571+0000","last_active":"2024-01-11T07:16:36.511391+0000","last_peered":"2024-01-11T07:16:36.511391+0000","last_clean":"2024-01-11T07:16:36.511391+0000","last_became_active":"2024-01-11T07:16:35.502110+0000","last_became_peered":"2024-01-11T07:16:35.502110+0000","last_unstale":"2024-01-11T07:16:36.511391+0000","last_undegraded":"2024-01-11T07:16:36.511391+0000","last_fullsized":"2024-01-11T07:16:36.511391+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:47:09.372309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.504708+0000","last_change":"2024-01-11T07:16:36.504387+0000","last_active":"2024-01-11T07:16:36.504708+0000","last_peered":"2024-01-11T07:16:36.504708+0000","last_clean":"2024-01-11T07:16:36.504708+0000","last_became_active":"2024-01-11T07:16:36.503393+0000","last_became_peered":"2024-01-11T07:16:36.503393+0000","last_unstale":"2024-01-11T07:16:36.504708+0000","last_undegraded":"2024-01-11T07:16:36.504708+0000","last_fullsized":"2024-01-11T07:16:36.504708+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:15:08.671542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.497289+0000","last_change":"2024-01-11T07:16:35.505776+0000","last_active":"2024-01-11T07:16:36.497289+0000","last_peered":"2024-01-11T07:16:36.497289+0000","last_clean":"2024-01-11T07:16:36.497289+0000","last_became_active":"2024-01-11T07:16:35.505552+0000","last_became_peered":"2024-01-11T07:16:35.505552+0000","last_unstale":"2024-01-11T07:16:36.497289+0000","last_undegraded":"2024-01-11T07:16:36.497289+0000","last_fullsized":"2024-01-11T07:16:36.497289+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:43:59.719379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.504047+0000","last_change":"2024-01-11T07:16:36.503978+0000","last_active":"2024-01-11T07:16:36.504047+0000","last_peered":"2024-01-11T07:16:36.504047+0000","last_clean":"2024-01-11T07:16:36.504047+0000","last_became_active":"2024-01-11T07:16:36.503740+0000","last_became_peered":"2024-01-11T07:16:36.503740+0000","last_unstale":"2024-01-11T07:16:36.504047+0000","last_undegraded":"2024-01-11T07:16:36.504047+0000","last_fullsized":"2024-01-11T07:16:36.504047+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:57.174064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499271+0000","last_change":"2024-01-11T07:16:35.504408+0000","last_active":"2024-01-11T07:16:36.499271+0000","last_peered":"2024-01-11T07:16:36.499271+0000","last_clean":"2024-01-11T07:16:36.499271+0000","last_became_active":"2024-01-11T07:16:35.504195+0000","last_became_peered":"2024-01-11T07:16:35.504195+0000","last_unstale":"2024-01-11T07:16:36.499271+0000","last_undegraded":"2024-01-11T07:16:36.499271+0000","last_fullsized":"2024-01-11T07:16:36.499271+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:25:32.485892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505002+0000","last_change":"2024-01-11T07:16:36.504932+0000","last_active":"2024-01-11T07:16:36.505002+0000","last_peered":"2024-01-11T07:16:36.505002+0000","last_clean":"2024-01-11T07:16:36.505002+0000","last_became_active":"2024-01-11T07:16:36.504660+0000","last_became_peered":"2024-01-11T07:16:36.504660+0000","last_unstale":"2024-01-11T07:16:36.505002+0000","last_undegraded":"2024-01-11T07:16:36.505002+0000","last_fullsized":"2024-01-11T07:16:36.505002+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:21:30.422304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.493467+0000","last_change":"2024-01-11T07:16:35.498372+0000","last_active":"2024-01-11T07:16:36.493467+0000","last_peered":"2024-01-11T07:16:36.493467+0000","last_clean":"2024-01-11T07:16:36.493467+0000","last_became_active":"2024-01-11T07:16:35.498117+0000","last_became_peered":"2024-01-11T07:16:35.498117+0000","last_unstale":"2024-01-11T07:16:36.493467+0000","last_undegraded":"2024-01-11T07:16:36.493467+0000","last_fullsized":"2024-01-11T07:16:36.493467+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:10:03.219468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501920+0000","last_change":"2024-01-11T07:16:36.501513+0000","last_active":"2024-01-11T07:16:36.501920+0000","last_peered":"2024-01-11T07:16:36.501920+0000","last_clean":"2024-01-11T07:16:36.501920+0000","last_became_active":"2024-01-11T07:16:36.500644+0000","last_became_peered":"2024-01-11T07:16:36.500644+0000","last_unstale":"2024-01-11T07:16:36.501920+0000","last_undegraded":"2024-01-11T07:16:36.501920+0000","last_fullsized":"2024-01-11T07:16:36.501920+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:34:40.895765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512434+0000","last_change":"2024-01-11T07:16:35.506446+0000","last_active":"2024-01-11T07:16:36.512434+0000","last_peered":"2024-01-11T07:16:36.512434+0000","last_clean":"2024-01-11T07:16:36.512434+0000","last_became_active":"2024-01-11T07:16:35.505916+0000","last_became_peered":"2024-01-11T07:16:35.505916+0000","last_unstale":"2024-01-11T07:16:36.512434+0000","last_undegraded":"2024-01-11T07:16:36.512434+0000","last_fullsized":"2024-01-11T07:16:36.512434+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:54:14.924711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501509+0000","last_change":"2024-01-11T07:16:36.501369+0000","last_active":"2024-01-11T07:16:36.501509+0000","last_peered":"2024-01-11T07:16:36.501509+0000","last_clean":"2024-01-11T07:16:36.501509+0000","last_became_active":"2024-01-11T07:16:36.500805+0000","last_became_peered":"2024-01-11T07:16:36.500805+0000","last_unstale":"2024-01-11T07:16:36.501509+0000","last_undegraded":"2024-01-11T07:16:36.501509+0000","last_fullsized":"2024-01-11T07:16:36.501509+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:13.606784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.504017+0000","last_change":"2024-01-11T07:16:35.503371+0000","last_active":"2024-01-11T07:16:35.504017+0000","last_peered":"2024-01-11T07:16:35.504017+0000","last_clean":"2024-01-11T07:16:35.504017+0000","last_became_active":"2024-01-11T07:16:35.503155+0000","last_became_peered":"2024-01-11T07:16:35.503155+0000","last_unstale":"2024-01-11T07:16:35.504017+0000","last_undegraded":"2024-01-11T07:16:35.504017+0000","last_fullsized":"2024-01-11T07:16:35.504017+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:51:10.545550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.509302+0000","last_change":"2024-01-11T07:16:36.509302+0000","last_active":"2024-01-11T07:16:36.509302+0000","last_peered":"2024-01-11T07:16:36.509302+0000","last_clean":"2024-01-11T07:16:36.509302+0000","last_became_active":"2024-01-11T07:16:36.508505+0000","last_became_peered":"2024-01-11T07:16:36.508505+0000","last_unstale":"2024-01-11T07:16:36.509302+0000","last_undegraded":"2024-01-11T07:16:36.509302+0000","last_fullsized":"2024-01-11T07:16:36.509302+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:45:50.182569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.503941+0000","last_change":"2024-01-11T07:16:35.501284+0000","last_active":"2024-01-11T07:16:35.503941+0000","last_peered":"2024-01-11T07:16:35.503941+0000","last_clean":"2024-01-11T07:16:35.503941+0000","last_became_active":"2024-01-11T07:16:35.500679+0000","last_became_peered":"2024-01-11T07:16:35.500679+0000","last_unstale":"2024-01-11T07:16:35.503941+0000","last_undegraded":"2024-01-11T07:16:35.503941+0000","last_fullsized":"2024-01-11T07:16:35.503941+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:34:39.890740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505149+0000","last_change":"2024-01-11T07:16:36.505149+0000","last_active":"2024-01-11T07:16:36.505149+0000","last_peered":"2024-01-11T07:16:36.505149+0000","last_clean":"2024-01-11T07:16:36.505149+0000","last_became_active":"2024-01-11T07:16:36.501191+0000","last_became_peered":"2024-01-11T07:16:36.501191+0000","last_unstale":"2024-01-11T07:16:36.505149+0000","last_undegraded":"2024-01-11T07:16:36.505149+0000","last_fullsized":"2024-01-11T07:16:36.505149+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:54.300795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.500631+0000","last_change":"2024-01-11T07:16:35.500532+0000","last_active":"2024-01-11T07:16:35.500631+0000","last_peered":"2024-01-11T07:16:35.500631+0000","last_clean":"2024-01-11T07:16:35.500631+0000","last_became_active":"2024-01-11T07:16:35.500285+0000","last_became_peered":"2024-01-11T07:16:35.500285+0000","last_unstale":"2024-01-11T07:16:35.500631+0000","last_undegraded":"2024-01-11T07:16:35.500631+0000","last_fullsized":"2024-01-11T07:16:35.500631+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:47.836502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.508859+0000","last_change":"2024-01-11T07:16:36.508859+0000","last_active":"2024-01-11T07:16:36.508859+0000","last_peered":"2024-01-11T07:16:36.508859+0000","last_clean":"2024-01-11T07:16:36.508859+0000","last_became_active":"2024-01-11T07:16:36.507203+0000","last_became_peered":"2024-01-11T07:16:36.507203+0000","last_unstale":"2024-01-11T07:16:36.508859+0000","last_undegraded":"2024-01-11T07:16:36.508859+0000","last_fullsized":"2024-01-11T07:16:36.508859+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:45:28.431412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.490170+0000","last_change":"2024-01-11T07:16:35.495523+0000","last_active":"2024-01-11T07:16:36.490170+0000","last_peered":"2024-01-11T07:16:36.490170+0000","last_clean":"2024-01-11T07:16:36.490170+0000","last_became_active":"2024-01-11T07:16:35.495287+0000","last_became_peered":"2024-01-11T07:16:35.495287+0000","last_unstale":"2024-01-11T07:16:36.490170+0000","last_undegraded":"2024-01-11T07:16:36.490170+0000","last_fullsized":"2024-01-11T07:16:36.490170+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:01:38.148568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.495500+0000","last_change":"2024-01-11T07:16:36.495431+0000","last_active":"2024-01-11T07:16:36.495500+0000","last_peered":"2024-01-11T07:16:36.495500+0000","last_clean":"2024-01-11T07:16:36.495500+0000","last_became_active":"2024-01-11T07:16:36.495172+0000","last_became_peered":"2024-01-11T07:16:36.495172+0000","last_unstale":"2024-01-11T07:16:36.495500+0000","last_undegraded":"2024-01-11T07:16:36.495500+0000","last_fullsized":"2024-01-11T07:16:36.495500+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:11:18.896950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.495241+0000","last_change":"2024-01-11T07:16:35.506542+0000","last_active":"2024-01-11T07:16:36.495241+0000","last_peered":"2024-01-11T07:16:36.495241+0000","last_clean":"2024-01-11T07:16:36.495241+0000","last_became_active":"2024-01-11T07:16:35.506155+0000","last_became_peered":"2024-01-11T07:16:35.506155+0000","last_unstale":"2024-01-11T07:16:36.495241+0000","last_undegraded":"2024-01-11T07:16:36.495241+0000","last_fullsized":"2024-01-11T07:16:36.495241+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:59:27.797942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.509081+0000","last_change":"2024-01-11T07:16:36.509032+0000","last_active":"2024-01-11T07:16:36.509081+0000","last_peered":"2024-01-11T07:16:36.509081+0000","last_clean":"2024-01-11T07:16:36.509081+0000","last_became_active":"2024-01-11T07:16:36.508729+0000","last_became_peered":"2024-01-11T07:16:36.508729+0000","last_unstale":"2024-01-11T07:16:36.509081+0000","last_undegraded":"2024-01-11T07:16:36.509081+0000","last_fullsized":"2024-01-11T07:16:36.509081+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:46:03.532281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.492637+0000","last_change":"2024-01-11T07:16:35.499876+0000","last_active":"2024-01-11T07:16:36.492637+0000","last_peered":"2024-01-11T07:16:36.492637+0000","last_clean":"2024-01-11T07:16:36.492637+0000","last_became_active":"2024-01-11T07:16:35.499426+0000","last_became_peered":"2024-01-11T07:16:35.499426+0000","last_unstale":"2024-01-11T07:16:36.492637+0000","last_undegraded":"2024-01-11T07:16:36.492637+0000","last_fullsized":"2024-01-11T07:16:36.492637+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:40:09.789035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.511086+0000","last_change":"2024-01-11T07:16:36.511006+0000","last_active":"2024-01-11T07:16:36.511086+0000","last_peered":"2024-01-11T07:16:36.511086+0000","last_clean":"2024-01-11T07:16:36.511086+0000","last_became_active":"2024-01-11T07:16:36.510780+0000","last_became_peered":"2024-01-11T07:16:36.510780+0000","last_unstale":"2024-01-11T07:16:36.511086+0000","last_undegraded":"2024-01-11T07:16:36.511086+0000","last_fullsized":"2024-01-11T07:16:36.511086+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:26:51.311106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.504539+0000","last_change":"2024-01-11T07:16:35.504444+0000","last_active":"2024-01-11T07:16:35.504539+0000","last_peered":"2024-01-11T07:16:35.504539+0000","last_clean":"2024-01-11T07:16:35.504539+0000","last_became_active":"2024-01-11T07:16:35.503689+0000","last_became_peered":"2024-01-11T07:16:35.503689+0000","last_unstale":"2024-01-11T07:16:35.504539+0000","last_undegraded":"2024-01-11T07:16:35.504539+0000","last_fullsized":"2024-01-11T07:16:35.504539+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:38:02.490699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.502527+0000","last_change":"2024-01-11T07:16:36.502450+0000","last_active":"2024-01-11T07:16:36.502527+0000","last_peered":"2024-01-11T07:16:36.502527+0000","last_clean":"2024-01-11T07:16:36.502527+0000","last_became_active":"2024-01-11T07:16:36.502209+0000","last_became_peered":"2024-01-11T07:16:36.502209+0000","last_unstale":"2024-01-11T07:16:36.502527+0000","last_undegraded":"2024-01-11T07:16:36.502527+0000","last_fullsized":"2024-01-11T07:16:36.502527+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:34:24.284543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"167'3","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.514086+0000","last_change":"2024-01-11T07:16:35.499172+0000","last_active":"2024-01-11T07:16:36.514086+0000","last_peered":"2024-01-11T07:16:36.514086+0000","last_clean":"2024-01-11T07:16:36.514086+0000","last_became_active":"2024-01-11T07:16:35.498876+0000","last_became_peered":"2024-01-11T07:16:35.498876+0000","last_unstale":"2024-01-11T07:16:36.514086+0000","last_undegraded":"2024-01-11T07:16:36.514086+0000","last_fullsized":"2024-01-11T07:16:36.514086+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:59.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.497389+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:16:36.497389+0000","last_peered":"2024-01-11T07:16:36.497389+0000","last_clean":"2024-01-11T07:16:36.497389+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:16:36.497389+0000","last_undegraded":"2024-01-11T07:16:36.497389+0000","last_fullsized":"2024-01-11T07:16:36.497389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1492,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193820,"statfs":{"total":200000000,"available":198472060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.45500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72699999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56999999999999995,"15min":0.54600000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.54100000000000004},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":0.78000000000000003,"15min":0.80500000000000005},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.54000000000000004},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.77500000000000002,"15min":0.81399999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.70599999999999996,"5min":0.79000000000000004,"15min":0.82399999999999995},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.40000000000000002,"15min":0.35099999999999998},"max":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.82999999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77200000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46999999999999997,"15min":0.48099999999999998},"min":{"1min":0.308,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":0.77900000000000003,"15min":0.85999999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.495,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.78200000000000003,"15min":0.85899999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.625},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.37,"15min":0.34100000000000003},"max":{"1min":0.69999999999999996,"5min":0.85499999999999998,"15min":0.91600000000000004},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.41399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.752,"5min":0.85399999999999998,"15min":0.92500000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.82899999999999996,"15min":0.90300000000000002},"last":0.57699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":915,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194397,"statfs":{"total":200000000,"available":199062906,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.56100000000000005},"min":{"1min":0.41199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.81399999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69299999999999995,"5min":0.76200000000000001,"15min":1.157},"last":0.62}]},{"osd":1,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.437,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.752,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.35599999999999998,"15min":0.33800000000000002},"max":{"1min":0.77100000000000002,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.39300000000000002,"15min":0.29999999999999999},"max":{"1min":0.73299999999999998,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.29599999999999999},"max":{"1min":0.79600000000000004,"5min":2.621,"15min":2.621},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":0.85799999999999998,"15min":0.88500000000000001},"last":0.63200000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48099999999999998,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65600000000000003,"5min":0.77500000000000002,"15min":0.82699999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.64800000000000002,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.51200000000000001},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.72099999999999997,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.80500000000000005,"15min":0.89100000000000001},"last":0.501}]},{"osd":7,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.505,"15min":0.52500000000000002},"min":{"1min":0.373,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68600000000000005,"5min":0.749,"15min":0.76700000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.48199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":909,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194403,"statfs":{"total":200000000,"available":199068786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.46700000000000003,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.247,"15min":0.216},"max":{"1min":0.47399999999999998,"5min":8.5150000000000006,"15min":20.41},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47899999999999998,"15min":0.46100000000000002},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.56599999999999995,"5min":3.7029999999999998,"15min":3.7029999999999998},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55100000000000005,"5min":7.8099999999999996,"15min":15.611000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59399999999999997,"5min":6.1390000000000002,"15min":20.199999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53100000000000003,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.30199999999999999,"15min":0.29999999999999999},"max":{"1min":0.60299999999999998,"5min":5.6449999999999996,"15min":20.369},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.54100000000000004,"15min":0.47099999999999997},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":0.61199999999999999,"5min":8.5790000000000006,"15min":8.5790000000000006},"last":0.379}]},{"osd":4,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.317},"max":{"1min":0.78100000000000003,"5min":7.6749999999999998,"15min":9.9570000000000007},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.44,"15min":0.41899999999999998},"max":{"1min":0.85699999999999998,"5min":5.7830000000000004,"15min":12.449999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.63300000000000001},"min":{"1min":0.51100000000000001,"5min":0.437,"15min":0.376},"max":{"1min":0.84499999999999997,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72199999999999998,"15min":0.69899999999999995},"min":{"1min":0.56999999999999995,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":7.7480000000000002,"15min":9.891},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.58699999999999997,"5min":0.45800000000000002,"15min":0.41899999999999998},"max":{"1min":0.83999999999999997,"5min":1.389,"15min":1.5569999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.82599999999999996,"5min":7.7770000000000001,"15min":9.9900000000000002},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72299999999999998,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":5.8449999999999998,"15min":20.324000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.46999999999999997,"15min":0.46800000000000003},"max":{"1min":0.92300000000000004,"5min":2.548,"15min":2.548},"last":0.70399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193819,"statfs":{"total":200000000,"available":198471114,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.44400000000000001},"min":{"1min":0.27500000000000002,"5min":0.22700000000000001,"15min":0.224},"max":{"1min":0.63900000000000001,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64700000000000002,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.61399999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.23999999999999999},"max":{"1min":0.61399999999999999,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46400000000000002,"15min":0.504},"min":{"1min":0.372,"5min":0.29099999999999998,"15min":0.27300000000000002},"max":{"1min":0.63900000000000001,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47299999999999998,"15min":0.63100000000000001},"min":{"1min":0.27700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55400000000000005,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.54300000000000004},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":0.83499999999999996,"15min":2.855},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63700000000000001,"15min":0.61699999999999999},"min":{"1min":0.51300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90400000000000003,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":0.81399999999999995},"min":{"1min":0.45700000000000002,"5min":0.41499999999999998,"15min":0.40999999999999998},"max":{"1min":0.98399999999999999,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66100000000000003,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.96199999999999997,"5min":1.032,"15min":2.819},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67800000000000005,"15min":0.65500000000000003},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93500000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.85999999999999999},"min":{"1min":0.44500000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0349999999999999,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72199999999999998,"15min":0.76200000000000001},"min":{"1min":0.52900000000000003,"5min":0.48099999999999998,"15min":0.41799999999999998},"max":{"1min":1.0209999999999999,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70399999999999996,"15min":0.67300000000000004},"min":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.276,"15min":1.276},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.73699999999999999,"15min":0.86599999999999999},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.44900000000000001},"max":{"1min":1.0780000000000001,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":0.621}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":910,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194402,"statfs":{"total":200000000,"available":199067734,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":1.472},"last":0.316},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.28899999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.86099999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41999999999999998,"15min":0.435},"min":{"1min":0.30299999999999999,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":0.628,"15min":1.2430000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.39600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.246},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.752,"15min":0.76800000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.29999999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":1.139},"last":0.438}]},{"osd":4,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61799999999999999,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":0.94399999999999995,"15min":1.0660000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.40100000000000002},"max":{"1min":0.89100000000000001,"5min":1.089,"15min":1.089},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.53800000000000003,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.94699999999999995,"5min":1.099,"15min":1.099},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67800000000000005,"15min":0.69699999999999995},"min":{"1min":0.52800000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91500000000000004,"5min":1.0229999999999999,"15min":1.3100000000000001},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.51200000000000001,"5min":0.47099999999999997,"15min":0.45800000000000002},"max":{"1min":0.89300000000000002,"5min":1.006,"15min":1.052},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66700000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":0.89800000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94799999999999995,"5min":1.042,"15min":1.042},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":908,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194403,"statfs":{"total":200000000,"available":199069250,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38800000000000001,"15min":0.64300000000000002},"min":{"1min":0.27900000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":11.481},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.46300000000000002,"15min":0.47599999999999998},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.31},"max":{"1min":0.59599999999999997,"5min":0.65400000000000003,"15min":1.6200000000000001},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39900000000000002,"15min":0.76600000000000001},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.23899999999999999},"max":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":17.516999999999999},"last":0.432},{"interface":"front","average":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.56999999999999995,"5min":0.63800000000000001,"15min":0.67400000000000004},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.42399999999999999},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53700000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.46300000000000002,"15min":0.82599999999999996},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.628,"5min":0.69499999999999995,"15min":17.553000000000001},"last":0.37}]},{"osd":4,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.55200000000000005,"5min":0.40899999999999997,"15min":0.40300000000000002},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":0.94599999999999995},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67000000000000004,"15min":1.052},"min":{"1min":0.54800000000000004,"5min":0.38500000000000001,"15min":0.36199999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":17.474},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.99399999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":17.393000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65100000000000002,"15min":0.88200000000000001},"min":{"1min":0.501,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.94499999999999995,"5min":1.002,"15min":11.394},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.89500000000000002},"min":{"1min":0.54600000000000004,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.91900000000000004,"5min":0.95799999999999996,"15min":11.436999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.67100000000000004},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.67800000000000005},"min":{"1min":0.55800000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96999999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68100000000000005,"15min":1.0349999999999999},"min":{"1min":0.55100000000000005,"5min":0.45700000000000002,"15min":0.42999999999999999},"max":{"1min":0.95499999999999996,"5min":1.1839999999999999,"15min":17.434000000000001},"last":0.57299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":914,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194397,"statfs":{"total":200000000,"available":199063052,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.89100000000000001},"min":{"1min":0.40899999999999997,"5min":0.30299999999999999,"15min":0.248},"max":{"1min":0.80500000000000005,"5min":0.81399999999999995,"15min":8.484},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.93200000000000005},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.82799999999999996,"15min":9.2599999999999998},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.76100000000000001},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.30199999999999999},"max":{"1min":0.73699999999999999,"5min":0.95299999999999996,"15min":8.5340000000000007},"last":0.504},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.85699999999999998},"min":{"1min":0.51800000000000002,"5min":0.34999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.78300000000000003,"15min":9.2279999999999998},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.84899999999999998},"min":{"1min":0.45000000000000001,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":9.5640000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.76900000000000002},"min":{"1min":0.45200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.90600000000000003,"15min":6.8639999999999999},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.873},"min":{"1min":0.46100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68200000000000005,"5min":0.90800000000000003,"15min":8.9779999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.93700000000000006},"min":{"1min":0.439,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":0.89100000000000001,"15min":8.7309999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45700000000000002,"15min":0.76700000000000002},"min":{"1min":0.315,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.65400000000000003,"5min":0.77700000000000002,"15min":11.388999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47699999999999998,"15min":0.72399999999999998},"min":{"1min":0.35399999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.68600000000000005,"5min":0.76800000000000002,"15min":8.6509999999999998},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.77000000000000002},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.30099999999999999},"max":{"1min":0.70499999999999996,"5min":0.80100000000000005,"15min":9.1449999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.59799999999999998},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":0.81200000000000006,"15min":1.119},"last":0.81899999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.76300000000000001},"min":{"1min":0.375,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.83299999999999996,"15min":8.6050000000000004},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.80000000000000004,"15min":9.4640000000000004},"last":0.89200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":915,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194397,"statfs":{"total":200000000,"available":199063030,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.52400000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74399999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.48999999999999999,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.72499999999999998,"5min":0.77400000000000002,"15min":0.81899999999999995},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.55800000000000005,"5min":0.36099999999999999,"15min":0.24399999999999999},"max":{"1min":0.76200000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.626},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.51300000000000001,"5min":0.39600000000000002,"15min":0.32100000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.84099999999999997},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.53700000000000003,"5min":0.36499999999999999,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.56100000000000005,"5min":0.42199999999999999,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.93999999999999995},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.56599999999999995,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78500000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.622,"15min":0.62},"min":{"1min":0.52600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.746}]},{"osd":4,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.375,"5min":0.29399999999999998,"15min":0.28299999999999997},"max":{"1min":0.63200000000000001,"5min":0.73899999999999999,"15min":0.76000000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.40799999999999997,"5min":0.34599999999999997,"15min":0.32500000000000001},"max":{"1min":0.65000000000000002,"5min":0.73999999999999999,"15min":0.76100000000000001},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.61499999999999999,"5min":0.75900000000000001,"15min":0.78600000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54500000000000004,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.55300000000000005},"min":{"1min":0.42199999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":0.77700000000000002,"15min":1.0109999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.34399999999999997,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.88200000000000001},"last":0.53900000000000003}]}]}],"pool_statfs":[]}} 2024-01-11T07:16:40.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:16:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.861+0000 7fbdc0320700 1 Processor -- start 2024-01-11T07:16:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.861+0000 7fbdc0320700 1 -- start start 2024-01-11T07:16:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.862+0000 7fbdc0320700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdb81560d0 0x7fbdb8069e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.862+0000 7fbdc0320700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdb80c7520 0x7fbdb80cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.862+0000 7fbdc0320700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb80699a0 con 0x7fbdb80cadf0 2024-01-11T07:16:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.862+0000 7fbdc0320700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb80cc2f0 con 0x7fbdb80c7520 2024-01-11T07:16:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.862+0000 7fbdc0320700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb8068d90 con 0x7fbdb81560d0 2024-01-11T07:16:40.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe8bd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdb80c7520 0x7fbdb80cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe8bd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdb80c7520 0x7fbdb80cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56104/0 (socket says 172.21.15.7:56104) 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe8bd700 1 -- 172.21.15.7:0/1639279900 learned_addr learned my addr 172.21.15.7:0/1639279900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe0bc700 1 --2- 172.21.15.7:0/1639279900 >> v2:172.21.15.113:3300/0 conn(0x7fbdb81560d0 0x7fbdb8069e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe8bd700 1 -- 172.21.15.7:0/1639279900 >> v2:172.21.15.113:3300/0 conn(0x7fbdb81560d0 msgr2=0x7fbdb8069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe8bd700 1 --2- 172.21.15.7:0/1639279900 >> v2:172.21.15.113:3300/0 conn(0x7fbdb81560d0 0x7fbdb8069e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe8bd700 1 -- 172.21.15.7:0/1639279900 >> v1:172.21.15.113:6789/0 conn(0x7fbdb80cadf0 legacy=0x7fbdb80c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe8bd700 1 -- 172.21.15.7:0/1639279900 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8069070 con 0x7fbdb80c7520 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbe8bd700 1 --2- 172.21.15.7:0/1639279900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb80c7520 0x7fbdb80cb290 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80716b0 tx=0x7fbdb000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d0c7fa50b2ca3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbd0ba700 1 -- 172.21.15.7:0/1639279900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0010070 con 0x7fbdb80c7520 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbd0ba700 1 -- 172.21.15.7:0/1639279900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb000dc80 con 0x7fbdb80c7520 2024-01-11T07:16:40.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.863+0000 7fbdbd0ba700 1 -- 172.21.15.7:0/1639279900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0016400 con 0x7fbdb80c7520 2024-01-11T07:16:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.864+0000 7fbdc0320700 1 -- 172.21.15.7:0/1639279900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb80c7520 msgr2=0x7fbdb80cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.864+0000 7fbdc0320700 1 --2- 172.21.15.7:0/1639279900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb80c7520 0x7fbdb80cb290 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80716b0 tx=0x7fbdb000ba50 comp rx=0 tx=0).stop 2024-01-11T07:16:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.864+0000 7fbdc0320700 1 -- 172.21.15.7:0/1639279900 shutdown_connections 2024-01-11T07:16:40.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.864+0000 7fbdc0320700 1 --2- 172.21.15.7:0/1639279900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb80c7520 0x7fbdb80cb290 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.864+0000 7fbdc0320700 1 --2- 172.21.15.7:0/1639279900 >> v2:172.21.15.113:3300/0 conn(0x7fbdb81560d0 0x7fbdb8069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.864+0000 7fbdc0320700 1 -- 172.21.15.7:0/1639279900 >> 172.21.15.7:0/1639279900 conn(0x7fbdb8063230 msgr2=0x7fbdb8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:40.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.864+0000 7fbdc0320700 1 -- 172.21.15.7:0/1639279900 shutdown_connections 2024-01-11T07:16:40.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.864+0000 7fbdc0320700 1 -- 172.21.15.7:0/1639279900 wait complete. 2024-01-11T07:16:40.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.865+0000 7fbdc0320700 1 Processor -- start 2024-01-11T07:16:40.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.865+0000 7fbdc0320700 1 -- start start 2024-01-11T07:16:40.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdc0320700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb80c7520 0x7fbdb81513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdc0320700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb80cadf0 0x7fbdb815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdc0320700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb81560d0 0x7fbdb80aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdc0320700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb81518f0 con 0x7fbdb81560d0 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdc0320700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb815b730 con 0x7fbdb80c7520 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdc0320700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb815ba10 con 0x7fbdb80cadf0 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdbe8bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb81560d0 0x7fbdb80aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdbe8bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb81560d0 0x7fbdb80aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56108/0 (socket says 172.21.15.7:56108) 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.866+0000 7fbdbe8bd700 1 -- 172.21.15.7:0/2676044699 learned_addr learned my addr 172.21.15.7:0/2676044699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbe0bc700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb80c7520 0x7fbdb81513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbd8bb700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb80cadf0 0x7fbdb815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbe8bd700 1 -- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb80cadf0 msgr2=0x7fbdb815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbe8bd700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb80cadf0 0x7fbdb815ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbe8bd700 1 -- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb80c7520 msgr2=0x7fbdb81513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbe8bd700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb80c7520 0x7fbdb81513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:40.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbe8bd700 1 -- 172.21.15.7:0/2676044699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb000bcf0 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbe0bc700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb80c7520 0x7fbdb81513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbd8bb700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb80cadf0 0x7fbdb815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdbe8bd700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb81560d0 0x7fbdb80aa8c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0016000 tx=0x7fbdb001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdae7fc700 1 -- 172.21.15.7:0/2676044699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0016400 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdae7fc700 1 -- 172.21.15.7:0/2676044699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb00032f0 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb815bcf0 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdae7fc700 1 -- 172.21.15.7:0/2676044699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb001f870 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.867+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb815c2b0 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.868+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb804c3c0 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.868+0000 7fbdae7fc700 1 -- 172.21.15.7:0/2676044699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbdb0025030 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.869+0000 7fbdae7fc700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd9c066f30 0x7fbd9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.869+0000 7fbdae7fc700 1 -- 172.21.15.7:0/2676044699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbdb0023070 con 0x7fbdb81560d0 2024-01-11T07:16:40.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.869+0000 7fbdbe0bc700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd9c066f30 0x7fbd9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:40.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.870+0000 7fbdbe0bc700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd9c066f30 0x7fbd9c0693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009990 tx=0x7fbdb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:40.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:40.873+0000 7fbdae7fc700 1 -- 172.21.15.7:0/2676044699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdb0053760 con 0x7fbdb81560d0 2024-01-11T07:16:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.073+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbdb815c4a0 con 0x7fbd9c066f30 2024-01-11T07:16:41.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.082+0000 7fbdae7fc700 1 -- 172.21.15.7:0/2676044699 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285879 (secure 0 0 0) 0x7fbdb815c4a0 con 0x7fbd9c066f30 2024-01-11T07:16:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd9c066f30 msgr2=0x7fbd9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd9c066f30 0x7fbd9c0693e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4009990 tx=0x7fbdb4008040 comp rx=0 tx=0).stop 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb81560d0 msgr2=0x7fbdb80aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb81560d0 0x7fbdb80aa8c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0016000 tx=0x7fbdb001c000 comp rx=0 tx=0).stop 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 shutdown_connections 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbd9c066f30 0x7fbd9c0693e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb81560d0 0x7fbdb80aa8c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbdb80c7520 0x7fbdb81513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 --2- 172.21.15.7:0/2676044699 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbdb80cadf0 0x7fbdb815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 >> 172.21.15.7:0/2676044699 conn(0x7fbdb8063230 msgr2=0x7fbdb8069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 shutdown_connections 2024-01-11T07:16:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.085+0000 7fbdc0320700 1 -- 172.21.15.7:0/2676044699 wait complete. 2024-01-11T07:16:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:16:41.099 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-01-11T07:16:39.979764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8459,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554040,"statfs":{"total":1600000000,"available":1591337932,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.494656"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505163+0000","last_change":"2024-01-11T07:16:36.505163+0000","last_active":"2024-01-11T07:16:36.505163+0000","last_peered":"2024-01-11T07:16:36.505163+0000","last_clean":"2024-01-11T07:16:36.505163+0000","last_became_active":"2024-01-11T07:16:36.504025+0000","last_became_peered":"2024-01-11T07:16:36.504025+0000","last_unstale":"2024-01-11T07:16:36.505163+0000","last_undegraded":"2024-01-11T07:16:36.505163+0000","last_fullsized":"2024-01-11T07:16:36.505163+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:37:32.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.502980+0000","last_change":"2024-01-11T07:16:36.502980+0000","last_active":"2024-01-11T07:16:36.502980+0000","last_peered":"2024-01-11T07:16:36.502980+0000","last_clean":"2024-01-11T07:16:36.502980+0000","last_became_active":"2024-01-11T07:16:36.502710+0000","last_became_peered":"2024-01-11T07:16:36.502710+0000","last_unstale":"2024-01-11T07:16:36.502980+0000","last_undegraded":"2024-01-11T07:16:36.502980+0000","last_fullsized":"2024-01-11T07:16:36.502980+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:29:26.090555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501525+0000","last_change":"2024-01-11T07:16:36.501429+0000","last_active":"2024-01-11T07:16:36.501525+0000","last_peered":"2024-01-11T07:16:36.501525+0000","last_clean":"2024-01-11T07:16:36.501525+0000","last_became_active":"2024-01-11T07:16:36.501018+0000","last_became_peered":"2024-01-11T07:16:36.501018+0000","last_unstale":"2024-01-11T07:16:36.501525+0000","last_undegraded":"2024-01-11T07:16:36.501525+0000","last_fullsized":"2024-01-11T07:16:36.501525+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:09:13.989257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501723+0000","last_change":"2024-01-11T07:16:36.501226+0000","last_active":"2024-01-11T07:16:36.501723+0000","last_peered":"2024-01-11T07:16:36.501723+0000","last_clean":"2024-01-11T07:16:36.501723+0000","last_became_active":"2024-01-11T07:16:36.500968+0000","last_became_peered":"2024-01-11T07:16:36.500968+0000","last_unstale":"2024-01-11T07:16:36.501723+0000","last_undegraded":"2024-01-11T07:16:36.501723+0000","last_fullsized":"2024-01-11T07:16:36.501723+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:13.517356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.500570+0000","last_change":"2024-01-11T07:16:36.500475+0000","last_active":"2024-01-11T07:16:36.500570+0000","last_peered":"2024-01-11T07:16:36.500570+0000","last_clean":"2024-01-11T07:16:36.500570+0000","last_became_active":"2024-01-11T07:16:36.500233+0000","last_became_peered":"2024-01-11T07:16:36.500233+0000","last_unstale":"2024-01-11T07:16:36.500570+0000","last_undegraded":"2024-01-11T07:16:36.500570+0000","last_fullsized":"2024-01-11T07:16:36.500570+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:10:04.224492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.498376+0000","last_change":"2024-01-11T07:16:36.498376+0000","last_active":"2024-01-11T07:16:36.498376+0000","last_peered":"2024-01-11T07:16:36.498376+0000","last_clean":"2024-01-11T07:16:36.498376+0000","last_became_active":"2024-01-11T07:16:36.497929+0000","last_became_peered":"2024-01-11T07:16:36.497929+0000","last_unstale":"2024-01-11T07:16:36.498376+0000","last_undegraded":"2024-01-11T07:16:36.498376+0000","last_fullsized":"2024-01-11T07:16:36.498376+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:33.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501987+0000","last_change":"2024-01-11T07:16:36.501846+0000","last_active":"2024-01-11T07:16:36.501987+0000","last_peered":"2024-01-11T07:16:36.501987+0000","last_clean":"2024-01-11T07:16:36.501987+0000","last_became_active":"2024-01-11T07:16:36.501526+0000","last_became_peered":"2024-01-11T07:16:36.501526+0000","last_unstale":"2024-01-11T07:16:36.501987+0000","last_undegraded":"2024-01-11T07:16:36.501987+0000","last_fullsized":"2024-01-11T07:16:36.501987+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:41:43.262266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501311+0000","last_change":"2024-01-11T07:16:36.501311+0000","last_active":"2024-01-11T07:16:36.501311+0000","last_peered":"2024-01-11T07:16:36.501311+0000","last_clean":"2024-01-11T07:16:36.501311+0000","last_became_active":"2024-01-11T07:16:36.500980+0000","last_became_peered":"2024-01-11T07:16:36.500980+0000","last_unstale":"2024-01-11T07:16:36.501311+0000","last_undegraded":"2024-01-11T07:16:36.501311+0000","last_fullsized":"2024-01-11T07:16:36.501311+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:06:55.151282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.498120+0000","last_change":"2024-01-11T07:16:36.498120+0000","last_active":"2024-01-11T07:16:36.498120+0000","last_peered":"2024-01-11T07:16:36.498120+0000","last_clean":"2024-01-11T07:16:36.498120+0000","last_became_active":"2024-01-11T07:16:36.497905+0000","last_became_peered":"2024-01-11T07:16:36.497905+0000","last_unstale":"2024-01-11T07:16:36.498120+0000","last_undegraded":"2024-01-11T07:16:36.498120+0000","last_fullsized":"2024-01-11T07:16:36.498120+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:01:39.153593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505060+0000","last_change":"2024-01-11T07:16:36.504912+0000","last_active":"2024-01-11T07:16:36.505060+0000","last_peered":"2024-01-11T07:16:36.505060+0000","last_clean":"2024-01-11T07:16:36.505060+0000","last_became_active":"2024-01-11T07:16:36.504688+0000","last_became_peered":"2024-01-11T07:16:36.504688+0000","last_unstale":"2024-01-11T07:16:36.505060+0000","last_undegraded":"2024-01-11T07:16:36.505060+0000","last_fullsized":"2024-01-11T07:16:36.505060+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:06.421656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.498908+0000","last_change":"2024-01-11T07:16:36.498908+0000","last_active":"2024-01-11T07:16:36.498908+0000","last_peered":"2024-01-11T07:16:36.498908+0000","last_clean":"2024-01-11T07:16:36.498908+0000","last_became_active":"2024-01-11T07:16:36.498638+0000","last_became_peered":"2024-01-11T07:16:36.498638+0000","last_unstale":"2024-01-11T07:16:36.498908+0000","last_undegraded":"2024-01-11T07:16:36.498908+0000","last_fullsized":"2024-01-11T07:16:36.498908+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:57.174064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.500063+0000","last_change":"2024-01-11T07:16:36.499999+0000","last_active":"2024-01-11T07:16:36.500063+0000","last_peered":"2024-01-11T07:16:36.500063+0000","last_clean":"2024-01-11T07:16:36.500063+0000","last_became_active":"2024-01-11T07:16:36.499745+0000","last_became_peered":"2024-01-11T07:16:36.499745+0000","last_unstale":"2024-01-11T07:16:36.500063+0000","last_undegraded":"2024-01-11T07:16:36.500063+0000","last_fullsized":"2024-01-11T07:16:36.500063+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:10:02.982128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501960+0000","last_change":"2024-01-11T07:16:36.501798+0000","last_active":"2024-01-11T07:16:36.501960+0000","last_peered":"2024-01-11T07:16:36.501960+0000","last_clean":"2024-01-11T07:16:36.501960+0000","last_became_active":"2024-01-11T07:16:36.501293+0000","last_became_peered":"2024-01-11T07:16:36.501293+0000","last_unstale":"2024-01-11T07:16:36.501960+0000","last_undegraded":"2024-01-11T07:16:36.501960+0000","last_fullsized":"2024-01-11T07:16:36.501960+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:46:42.393585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501163+0000","last_change":"2024-01-11T07:16:36.500293+0000","last_active":"2024-01-11T07:16:36.501163+0000","last_peered":"2024-01-11T07:16:36.501163+0000","last_clean":"2024-01-11T07:16:36.501163+0000","last_became_active":"2024-01-11T07:16:36.500053+0000","last_became_peered":"2024-01-11T07:16:36.500053+0000","last_unstale":"2024-01-11T07:16:36.501163+0000","last_undegraded":"2024-01-11T07:16:36.501163+0000","last_fullsized":"2024-01-11T07:16:36.501163+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:34:19.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.508506+0000","last_change":"2024-01-11T07:16:36.508440+0000","last_active":"2024-01-11T07:16:36.508506+0000","last_peered":"2024-01-11T07:16:36.508506+0000","last_clean":"2024-01-11T07:16:36.508506+0000","last_became_active":"2024-01-11T07:16:36.508178+0000","last_became_peered":"2024-01-11T07:16:36.508178+0000","last_unstale":"2024-01-11T07:16:36.508506+0000","last_undegraded":"2024-01-11T07:16:36.508506+0000","last_fullsized":"2024-01-11T07:16:36.508506+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:00:43.153489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.507952+0000","last_change":"2024-01-11T07:16:36.507952+0000","last_active":"2024-01-11T07:16:36.507952+0000","last_peered":"2024-01-11T07:16:36.507952+0000","last_clean":"2024-01-11T07:16:36.507952+0000","last_became_active":"2024-01-11T07:16:36.507656+0000","last_became_peered":"2024-01-11T07:16:36.507656+0000","last_unstale":"2024-01-11T07:16:36.507952+0000","last_undegraded":"2024-01-11T07:16:36.507952+0000","last_fullsized":"2024-01-11T07:16:36.507952+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:38:15.114347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505030+0000","last_change":"2024-01-11T07:16:36.504237+0000","last_active":"2024-01-11T07:16:36.505030+0000","last_peered":"2024-01-11T07:16:36.505030+0000","last_clean":"2024-01-11T07:16:36.505030+0000","last_became_active":"2024-01-11T07:16:36.503969+0000","last_became_peered":"2024-01-11T07:16:36.503969+0000","last_unstale":"2024-01-11T07:16:36.505030+0000","last_undegraded":"2024-01-11T07:16:36.505030+0000","last_fullsized":"2024-01-11T07:16:36.505030+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:12:56.271162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501491+0000","last_change":"2024-01-11T07:16:36.501277+0000","last_active":"2024-01-11T07:16:36.501491+0000","last_peered":"2024-01-11T07:16:36.501491+0000","last_clean":"2024-01-11T07:16:36.501491+0000","last_became_active":"2024-01-11T07:16:36.500690+0000","last_became_peered":"2024-01-11T07:16:36.500690+0000","last_unstale":"2024-01-11T07:16:36.501491+0000","last_undegraded":"2024-01-11T07:16:36.501491+0000","last_fullsized":"2024-01-11T07:16:36.501491+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:41:07.175635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512245+0000","last_change":"2024-01-11T07:16:36.512245+0000","last_active":"2024-01-11T07:16:36.512245+0000","last_peered":"2024-01-11T07:16:36.512245+0000","last_clean":"2024-01-11T07:16:36.512245+0000","last_became_active":"2024-01-11T07:16:36.511982+0000","last_became_peered":"2024-01-11T07:16:36.511982+0000","last_unstale":"2024-01-11T07:16:36.512245+0000","last_undegraded":"2024-01-11T07:16:36.512245+0000","last_fullsized":"2024-01-11T07:16:36.512245+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:37:20.568288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.516474+0000","last_change":"2024-01-11T07:16:36.516474+0000","last_active":"2024-01-11T07:16:36.516474+0000","last_peered":"2024-01-11T07:16:36.516474+0000","last_clean":"2024-01-11T07:16:36.516474+0000","last_became_active":"2024-01-11T07:16:36.516240+0000","last_became_peered":"2024-01-11T07:16:36.516240+0000","last_unstale":"2024-01-11T07:16:36.516474+0000","last_undegraded":"2024-01-11T07:16:36.516474+0000","last_fullsized":"2024-01-11T07:16:36.516474+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:08:29.474805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505045+0000","last_change":"2024-01-11T07:16:36.504765+0000","last_active":"2024-01-11T07:16:36.505045+0000","last_peered":"2024-01-11T07:16:36.505045+0000","last_clean":"2024-01-11T07:16:36.505045+0000","last_became_active":"2024-01-11T07:16:36.504499+0000","last_became_peered":"2024-01-11T07:16:36.504499+0000","last_unstale":"2024-01-11T07:16:36.505045+0000","last_undegraded":"2024-01-11T07:16:36.505045+0000","last_fullsized":"2024-01-11T07:16:36.505045+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:50:43.199972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501137+0000","last_change":"2024-01-11T07:16:36.499636+0000","last_active":"2024-01-11T07:16:36.501137+0000","last_peered":"2024-01-11T07:16:36.501137+0000","last_clean":"2024-01-11T07:16:36.501137+0000","last_became_active":"2024-01-11T07:16:36.499305+0000","last_became_peered":"2024-01-11T07:16:36.499305+0000","last_unstale":"2024-01-11T07:16:36.501137+0000","last_undegraded":"2024-01-11T07:16:36.501137+0000","last_fullsized":"2024-01-11T07:16:36.501137+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:06:55.151282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.502696+0000","last_change":"2024-01-11T07:16:36.502218+0000","last_active":"2024-01-11T07:16:36.502696+0000","last_peered":"2024-01-11T07:16:36.502696+0000","last_clean":"2024-01-11T07:16:36.502696+0000","last_became_active":"2024-01-11T07:16:36.501987+0000","last_became_peered":"2024-01-11T07:16:36.501987+0000","last_unstale":"2024-01-11T07:16:36.502696+0000","last_undegraded":"2024-01-11T07:16:36.502696+0000","last_fullsized":"2024-01-11T07:16:36.502696+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:33.924327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.504995+0000","last_change":"2024-01-11T07:16:36.502174+0000","last_active":"2024-01-11T07:16:36.504995+0000","last_peered":"2024-01-11T07:16:36.504995+0000","last_clean":"2024-01-11T07:16:36.504995+0000","last_became_active":"2024-01-11T07:16:36.501800+0000","last_became_peered":"2024-01-11T07:16:36.501800+0000","last_unstale":"2024-01-11T07:16:36.504995+0000","last_undegraded":"2024-01-11T07:16:36.504995+0000","last_fullsized":"2024-01-11T07:16:36.504995+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:53.191130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.508210+0000","last_change":"2024-01-11T07:16:36.508210+0000","last_active":"2024-01-11T07:16:36.508210+0000","last_peered":"2024-01-11T07:16:36.508210+0000","last_clean":"2024-01-11T07:16:36.508210+0000","last_became_active":"2024-01-11T07:16:36.507384+0000","last_became_peered":"2024-01-11T07:16:36.507384+0000","last_unstale":"2024-01-11T07:16:36.508210+0000","last_undegraded":"2024-01-11T07:16:36.508210+0000","last_fullsized":"2024-01-11T07:16:36.508210+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:07:15.105894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.514293+0000","last_change":"2024-01-11T07:16:36.514293+0000","last_active":"2024-01-11T07:16:36.514293+0000","last_peered":"2024-01-11T07:16:36.514293+0000","last_clean":"2024-01-11T07:16:36.514293+0000","last_became_active":"2024-01-11T07:16:36.513319+0000","last_became_peered":"2024-01-11T07:16:36.513319+0000","last_unstale":"2024-01-11T07:16:36.514293+0000","last_undegraded":"2024-01-11T07:16:36.514293+0000","last_fullsized":"2024-01-11T07:16:36.514293+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:06:13.517356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.509569+0000","last_change":"2024-01-11T07:16:36.509569+0000","last_active":"2024-01-11T07:16:36.509569+0000","last_peered":"2024-01-11T07:16:36.509569+0000","last_clean":"2024-01-11T07:16:36.509569+0000","last_became_active":"2024-01-11T07:16:36.509327+0000","last_became_peered":"2024-01-11T07:16:36.509327+0000","last_unstale":"2024-01-11T07:16:36.509569+0000","last_undegraded":"2024-01-11T07:16:36.509569+0000","last_fullsized":"2024-01-11T07:16:36.509569+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:34.462281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.494296+0000","last_change":"2024-01-11T07:16:36.494220+0000","last_active":"2024-01-11T07:16:36.494296+0000","last_peered":"2024-01-11T07:16:36.494296+0000","last_clean":"2024-01-11T07:16:36.494296+0000","last_became_active":"2024-01-11T07:16:36.493971+0000","last_became_peered":"2024-01-11T07:16:36.493971+0000","last_unstale":"2024-01-11T07:16:36.494296+0000","last_undegraded":"2024-01-11T07:16:36.494296+0000","last_fullsized":"2024-01-11T07:16:36.494296+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:41.270820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505324+0000","last_change":"2024-01-11T07:16:36.505324+0000","last_active":"2024-01-11T07:16:36.505324+0000","last_peered":"2024-01-11T07:16:36.505324+0000","last_clean":"2024-01-11T07:16:36.505324+0000","last_became_active":"2024-01-11T07:16:36.504846+0000","last_became_peered":"2024-01-11T07:16:36.504846+0000","last_unstale":"2024-01-11T07:16:36.505324+0000","last_undegraded":"2024-01-11T07:16:36.505324+0000","last_fullsized":"2024-01-11T07:16:36.505324+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:05:43.217209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.504066+0000","last_change":"2024-01-11T07:16:36.503990+0000","last_active":"2024-01-11T07:16:36.504066+0000","last_peered":"2024-01-11T07:16:36.504066+0000","last_clean":"2024-01-11T07:16:36.504066+0000","last_became_active":"2024-01-11T07:16:36.503709+0000","last_became_peered":"2024-01-11T07:16:36.503709+0000","last_unstale":"2024-01-11T07:16:36.504066+0000","last_undegraded":"2024-01-11T07:16:36.504066+0000","last_fullsized":"2024-01-11T07:16:36.504066+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:37:32.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.497676+0000","last_change":"2024-01-11T07:16:36.497676+0000","last_active":"2024-01-11T07:16:36.497676+0000","last_peered":"2024-01-11T07:16:36.497676+0000","last_clean":"2024-01-11T07:16:36.497676+0000","last_became_active":"2024-01-11T07:16:36.497429+0000","last_became_peered":"2024-01-11T07:16:36.497429+0000","last_unstale":"2024-01-11T07:16:36.497676+0000","last_undegraded":"2024-01-11T07:16:36.497676+0000","last_fullsized":"2024-01-11T07:16:36.497676+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:50:43.199972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.508686+0000","last_change":"2024-01-11T07:16:36.508686+0000","last_active":"2024-01-11T07:16:36.508686+0000","last_peered":"2024-01-11T07:16:36.508686+0000","last_clean":"2024-01-11T07:16:36.508686+0000","last_became_active":"2024-01-11T07:16:36.506780+0000","last_became_peered":"2024-01-11T07:16:36.506780+0000","last_unstale":"2024-01-11T07:16:36.508686+0000","last_undegraded":"2024-01-11T07:16:36.508686+0000","last_fullsized":"2024-01-11T07:16:36.508686+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:26:46.565949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499390+0000","last_change":"2024-01-11T07:16:36.498465+0000","last_active":"2024-01-11T07:16:36.499390+0000","last_peered":"2024-01-11T07:16:36.499390+0000","last_clean":"2024-01-11T07:16:36.499390+0000","last_became_active":"2024-01-11T07:16:36.498096+0000","last_became_peered":"2024-01-11T07:16:36.498096+0000","last_unstale":"2024-01-11T07:16:36.499390+0000","last_undegraded":"2024-01-11T07:16:36.499390+0000","last_fullsized":"2024-01-11T07:16:36.499390+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:32:41.616964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"167'4","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.513460+0000","last_change":"2024-01-11T07:16:35.505257+0000","last_active":"2024-01-11T07:16:36.513460+0000","last_peered":"2024-01-11T07:16:36.513460+0000","last_clean":"2024-01-11T07:16:36.513460+0000","last_became_active":"2024-01-11T07:16:35.505038+0000","last_became_peered":"2024-01-11T07:16:35.505038+0000","last_unstale":"2024-01-11T07:16:36.513460+0000","last_undegraded":"2024-01-11T07:16:36.513460+0000","last_fullsized":"2024-01-11T07:16:36.513460+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:38:02.490699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499104+0000","last_change":"2024-01-11T07:16:36.499104+0000","last_active":"2024-01-11T07:16:36.499104+0000","last_peered":"2024-01-11T07:16:36.499104+0000","last_clean":"2024-01-11T07:16:36.499104+0000","last_became_active":"2024-01-11T07:16:36.498795+0000","last_became_peered":"2024-01-11T07:16:36.498795+0000","last_unstale":"2024-01-11T07:16:36.499104+0000","last_undegraded":"2024-01-11T07:16:36.499104+0000","last_fullsized":"2024-01-11T07:16:36.499104+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:40:41.641936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"167'1","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.494340+0000","last_change":"2024-01-11T07:16:35.504601+0000","last_active":"2024-01-11T07:16:36.494340+0000","last_peered":"2024-01-11T07:16:36.494340+0000","last_clean":"2024-01-11T07:16:36.494340+0000","last_became_active":"2024-01-11T07:16:35.504009+0000","last_became_peered":"2024-01-11T07:16:35.504009+0000","last_unstale":"2024-01-11T07:16:36.494340+0000","last_undegraded":"2024-01-11T07:16:36.494340+0000","last_fullsized":"2024-01-11T07:16:36.494340+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:59:27.797942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505316+0000","last_change":"2024-01-11T07:16:36.505316+0000","last_active":"2024-01-11T07:16:36.505316+0000","last_peered":"2024-01-11T07:16:36.505316+0000","last_clean":"2024-01-11T07:16:36.505316+0000","last_became_active":"2024-01-11T07:16:36.504914+0000","last_became_peered":"2024-01-11T07:16:36.504914+0000","last_unstale":"2024-01-11T07:16:36.505316+0000","last_undegraded":"2024-01-11T07:16:36.505316+0000","last_fullsized":"2024-01-11T07:16:36.505316+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:37:41.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512866+0000","last_change":"2024-01-11T07:16:35.497036+0000","last_active":"2024-01-11T07:16:36.512866+0000","last_peered":"2024-01-11T07:16:36.512866+0000","last_clean":"2024-01-11T07:16:36.512866+0000","last_became_active":"2024-01-11T07:16:35.496767+0000","last_became_peered":"2024-01-11T07:16:35.496767+0000","last_unstale":"2024-01-11T07:16:36.512866+0000","last_undegraded":"2024-01-11T07:16:36.512866+0000","last_fullsized":"2024-01-11T07:16:36.512866+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:42:39.165288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505014+0000","last_change":"2024-01-11T07:16:36.503352+0000","last_active":"2024-01-11T07:16:36.505014+0000","last_peered":"2024-01-11T07:16:36.505014+0000","last_clean":"2024-01-11T07:16:36.505014+0000","last_became_active":"2024-01-11T07:16:36.503160+0000","last_became_peered":"2024-01-11T07:16:36.503160+0000","last_unstale":"2024-01-11T07:16:36.505014+0000","last_undegraded":"2024-01-11T07:16:36.505014+0000","last_fullsized":"2024-01-11T07:16:36.505014+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:49:28.386663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.514335+0000","last_change":"2024-01-11T07:16:35.505271+0000","last_active":"2024-01-11T07:16:36.514335+0000","last_peered":"2024-01-11T07:16:36.514335+0000","last_clean":"2024-01-11T07:16:36.514335+0000","last_became_active":"2024-01-11T07:16:35.505000+0000","last_became_peered":"2024-01-11T07:16:35.505000+0000","last_unstale":"2024-01-11T07:16:36.514335+0000","last_undegraded":"2024-01-11T07:16:36.514335+0000","last_fullsized":"2024-01-11T07:16:36.514335+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:12.601760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.503509+0000","last_change":"2024-01-11T07:16:36.503509+0000","last_active":"2024-01-11T07:16:36.503509+0000","last_peered":"2024-01-11T07:16:36.503509+0000","last_clean":"2024-01-11T07:16:36.503509+0000","last_became_active":"2024-01-11T07:16:36.502796+0000","last_became_peered":"2024-01-11T07:16:36.502796+0000","last_unstale":"2024-01-11T07:16:36.503509+0000","last_undegraded":"2024-01-11T07:16:36.503509+0000","last_fullsized":"2024-01-11T07:16:36.503509+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:53:18.161636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"167'1","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.506132+0000","last_change":"2024-01-11T07:16:35.504632+0000","last_active":"2024-01-11T07:16:36.506132+0000","last_peered":"2024-01-11T07:16:36.506132+0000","last_clean":"2024-01-11T07:16:36.506132+0000","last_became_active":"2024-01-11T07:16:35.504383+0000","last_became_peered":"2024-01-11T07:16:35.504383+0000","last_unstale":"2024-01-11T07:16:36.506132+0000","last_undegraded":"2024-01-11T07:16:36.506132+0000","last_fullsized":"2024-01-11T07:16:36.506132+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:08:44.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499700+0000","last_change":"2024-01-11T07:16:36.498985+0000","last_active":"2024-01-11T07:16:36.499700+0000","last_peered":"2024-01-11T07:16:36.499700+0000","last_clean":"2024-01-11T07:16:36.499700+0000","last_became_active":"2024-01-11T07:16:36.498734+0000","last_became_peered":"2024-01-11T07:16:36.498734+0000","last_unstale":"2024-01-11T07:16:36.499700+0000","last_undegraded":"2024-01-11T07:16:36.499700+0000","last_fullsized":"2024-01-11T07:16:36.499700+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:41:17.542100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512639+0000","last_change":"2024-01-11T07:16:35.503345+0000","last_active":"2024-01-11T07:16:36.512639+0000","last_peered":"2024-01-11T07:16:36.512639+0000","last_clean":"2024-01-11T07:16:36.512639+0000","last_became_active":"2024-01-11T07:16:35.502905+0000","last_became_peered":"2024-01-11T07:16:35.502905+0000","last_unstale":"2024-01-11T07:16:36.512639+0000","last_undegraded":"2024-01-11T07:16:36.512639+0000","last_fullsized":"2024-01-11T07:16:36.512639+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:26:45.560925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499993+0000","last_change":"2024-01-11T07:16:36.499993+0000","last_active":"2024-01-11T07:16:36.499993+0000","last_peered":"2024-01-11T07:16:36.499993+0000","last_clean":"2024-01-11T07:16:36.499993+0000","last_became_active":"2024-01-11T07:16:36.499383+0000","last_became_peered":"2024-01-11T07:16:36.499383+0000","last_unstale":"2024-01-11T07:16:36.499993+0000","last_undegraded":"2024-01-11T07:16:36.499993+0000","last_fullsized":"2024-01-11T07:16:36.499993+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:32:11.874290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.502206+0000","last_change":"2024-01-11T07:16:35.502093+0000","last_active":"2024-01-11T07:16:35.502206+0000","last_peered":"2024-01-11T07:16:35.502206+0000","last_clean":"2024-01-11T07:16:35.502206+0000","last_became_active":"2024-01-11T07:16:35.501592+0000","last_became_peered":"2024-01-11T07:16:35.501592+0000","last_unstale":"2024-01-11T07:16:35.502206+0000","last_undegraded":"2024-01-11T07:16:35.502206+0000","last_fullsized":"2024-01-11T07:16:35.502206+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:04.421915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.500550+0000","last_change":"2024-01-11T07:16:36.500266+0000","last_active":"2024-01-11T07:16:36.500550+0000","last_peered":"2024-01-11T07:16:36.500550+0000","last_clean":"2024-01-11T07:16:36.500550+0000","last_became_active":"2024-01-11T07:16:36.499905+0000","last_became_peered":"2024-01-11T07:16:36.499905+0000","last_unstale":"2024-01-11T07:16:36.500550+0000","last_undegraded":"2024-01-11T07:16:36.500550+0000","last_fullsized":"2024-01-11T07:16:36.500550+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:51:11.550574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.502923+0000","last_change":"2024-01-11T07:16:35.503460+0000","last_active":"2024-01-11T07:16:36.502923+0000","last_peered":"2024-01-11T07:16:36.502923+0000","last_clean":"2024-01-11T07:16:36.502923+0000","last_became_active":"2024-01-11T07:16:35.503215+0000","last_became_peered":"2024-01-11T07:16:35.503215+0000","last_unstale":"2024-01-11T07:16:36.502923+0000","last_undegraded":"2024-01-11T07:16:36.502923+0000","last_fullsized":"2024-01-11T07:16:36.502923+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:33:05.083702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.507734+0000","last_change":"2024-01-11T07:16:36.507670+0000","last_active":"2024-01-11T07:16:36.507734+0000","last_peered":"2024-01-11T07:16:36.507734+0000","last_clean":"2024-01-11T07:16:36.507734+0000","last_became_active":"2024-01-11T07:16:36.507320+0000","last_became_peered":"2024-01-11T07:16:36.507320+0000","last_unstale":"2024-01-11T07:16:36.507734+0000","last_undegraded":"2024-01-11T07:16:36.507734+0000","last_fullsized":"2024-01-11T07:16:36.507734+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:40:10.794059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.501070+0000","last_change":"2024-01-11T07:16:35.500997+0000","last_active":"2024-01-11T07:16:35.501070+0000","last_peered":"2024-01-11T07:16:35.501070+0000","last_clean":"2024-01-11T07:16:35.501070+0000","last_became_active":"2024-01-11T07:16:35.500757+0000","last_became_peered":"2024-01-11T07:16:35.500757+0000","last_unstale":"2024-01-11T07:16:35.501070+0000","last_undegraded":"2024-01-11T07:16:35.501070+0000","last_fullsized":"2024-01-11T07:16:35.501070+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:54:14.924711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499421+0000","last_change":"2024-01-11T07:16:36.499036+0000","last_active":"2024-01-11T07:16:36.499421+0000","last_peered":"2024-01-11T07:16:36.499421+0000","last_clean":"2024-01-11T07:16:36.499421+0000","last_became_active":"2024-01-11T07:16:36.498798+0000","last_became_peered":"2024-01-11T07:16:36.498798+0000","last_unstale":"2024-01-11T07:16:36.499421+0000","last_undegraded":"2024-01-11T07:16:36.499421+0000","last_fullsized":"2024-01-11T07:16:36.499421+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:45:50.182569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.495048+0000","last_change":"2024-01-11T07:16:35.506320+0000","last_active":"2024-01-11T07:16:36.495048+0000","last_peered":"2024-01-11T07:16:36.495048+0000","last_clean":"2024-01-11T07:16:36.495048+0000","last_became_active":"2024-01-11T07:16:35.505622+0000","last_became_peered":"2024-01-11T07:16:35.505622+0000","last_unstale":"2024-01-11T07:16:36.495048+0000","last_undegraded":"2024-01-11T07:16:36.495048+0000","last_fullsized":"2024-01-11T07:16:36.495048+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:40.615120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.509523+0000","last_change":"2024-01-11T07:16:36.509523+0000","last_active":"2024-01-11T07:16:36.509523+0000","last_peered":"2024-01-11T07:16:36.509523+0000","last_clean":"2024-01-11T07:16:36.509523+0000","last_became_active":"2024-01-11T07:16:36.509010+0000","last_became_peered":"2024-01-11T07:16:36.509010+0000","last_unstale":"2024-01-11T07:16:36.509523+0000","last_undegraded":"2024-01-11T07:16:36.509523+0000","last_fullsized":"2024-01-11T07:16:36.509523+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:31:39.883912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"167'6","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.519790+0000","last_change":"2024-01-11T07:16:35.499343+0000","last_active":"2024-01-11T07:16:36.519790+0000","last_peered":"2024-01-11T07:16:36.519790+0000","last_clean":"2024-01-11T07:16:36.519790+0000","last_became_active":"2024-01-11T07:16:35.499095+0000","last_became_peered":"2024-01-11T07:16:35.499095+0000","last_unstale":"2024-01-11T07:16:36.519790+0000","last_undegraded":"2024-01-11T07:16:36.519790+0000","last_fullsized":"2024-01-11T07:16:36.519790+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:07:52.186106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505996+0000","last_change":"2024-01-11T07:16:36.505917+0000","last_active":"2024-01-11T07:16:36.505996+0000","last_peered":"2024-01-11T07:16:36.505996+0000","last_clean":"2024-01-11T07:16:36.505996+0000","last_became_active":"2024-01-11T07:16:36.505670+0000","last_became_peered":"2024-01-11T07:16:36.505670+0000","last_unstale":"2024-01-11T07:16:36.505996+0000","last_undegraded":"2024-01-11T07:16:36.505996+0000","last_fullsized":"2024-01-11T07:16:36.505996+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:50:38.815660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512172+0000","last_change":"2024-01-11T07:16:35.494583+0000","last_active":"2024-01-11T07:16:36.512172+0000","last_peered":"2024-01-11T07:16:36.512172+0000","last_clean":"2024-01-11T07:16:36.512172+0000","last_became_active":"2024-01-11T07:16:35.494264+0000","last_became_peered":"2024-01-11T07:16:35.494264+0000","last_unstale":"2024-01-11T07:16:36.512172+0000","last_undegraded":"2024-01-11T07:16:36.512172+0000","last_fullsized":"2024-01-11T07:16:36.512172+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:44:21.430212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.500151+0000","last_change":"2024-01-11T07:16:36.500151+0000","last_active":"2024-01-11T07:16:36.500151+0000","last_peered":"2024-01-11T07:16:36.500151+0000","last_clean":"2024-01-11T07:16:36.500151+0000","last_became_active":"2024-01-11T07:16:36.499736+0000","last_became_peered":"2024-01-11T07:16:36.499736+0000","last_unstale":"2024-01-11T07:16:36.500151+0000","last_undegraded":"2024-01-11T07:16:36.500151+0000","last_fullsized":"2024-01-11T07:16:36.500151+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:39:06.421656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.493355+0000","last_change":"2024-01-11T07:16:35.499968+0000","last_active":"2024-01-11T07:16:36.493355+0000","last_peered":"2024-01-11T07:16:36.493355+0000","last_clean":"2024-01-11T07:16:36.493355+0000","last_became_active":"2024-01-11T07:16:35.499437+0000","last_became_peered":"2024-01-11T07:16:35.499437+0000","last_unstale":"2024-01-11T07:16:36.493355+0000","last_undegraded":"2024-01-11T07:16:36.493355+0000","last_fullsized":"2024-01-11T07:16:36.493355+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:54:40.965598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501943+0000","last_change":"2024-01-11T07:16:36.501657+0000","last_active":"2024-01-11T07:16:36.501943+0000","last_peered":"2024-01-11T07:16:36.501943+0000","last_clean":"2024-01-11T07:16:36.501943+0000","last_became_active":"2024-01-11T07:16:36.500994+0000","last_became_peered":"2024-01-11T07:16:36.500994+0000","last_unstale":"2024-01-11T07:16:36.501943+0000","last_undegraded":"2024-01-11T07:16:36.501943+0000","last_fullsized":"2024-01-11T07:16:36.501943+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:48.841526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.503963+0000","last_change":"2024-01-11T07:16:35.502494+0000","last_active":"2024-01-11T07:16:35.503963+0000","last_peered":"2024-01-11T07:16:35.503963+0000","last_clean":"2024-01-11T07:16:35.503963+0000","last_became_active":"2024-01-11T07:16:35.501518+0000","last_became_peered":"2024-01-11T07:16:35.501518+0000","last_unstale":"2024-01-11T07:16:35.503963+0000","last_undegraded":"2024-01-11T07:16:35.503963+0000","last_fullsized":"2024-01-11T07:16:35.503963+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:11:17.891925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.507669+0000","last_change":"2024-01-11T07:16:36.507597+0000","last_active":"2024-01-11T07:16:36.507669+0000","last_peered":"2024-01-11T07:16:36.507669+0000","last_clean":"2024-01-11T07:16:36.507669+0000","last_became_active":"2024-01-11T07:16:36.507459+0000","last_became_peered":"2024-01-11T07:16:36.507459+0000","last_unstale":"2024-01-11T07:16:36.507669+0000","last_undegraded":"2024-01-11T07:16:36.507669+0000","last_fullsized":"2024-01-11T07:16:36.507669+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:50:54.198682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"167'8","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.519250+0000","last_change":"2024-01-11T07:16:35.500368+0000","last_active":"2024-01-11T07:16:36.519250+0000","last_peered":"2024-01-11T07:16:36.519250+0000","last_clean":"2024-01-11T07:16:36.519250+0000","last_became_active":"2024-01-11T07:16:35.500172+0000","last_became_peered":"2024-01-11T07:16:35.500172+0000","last_unstale":"2024-01-11T07:16:36.519250+0000","last_undegraded":"2024-01-11T07:16:36.519250+0000","last_fullsized":"2024-01-11T07:16:36.519250+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:12:05.320369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.507399+0000","last_change":"2024-01-11T07:16:36.507310+0000","last_active":"2024-01-11T07:16:36.507399+0000","last_peered":"2024-01-11T07:16:36.507399+0000","last_clean":"2024-01-11T07:16:36.507399+0000","last_became_active":"2024-01-11T07:16:36.506971+0000","last_became_peered":"2024-01-11T07:16:36.506971+0000","last_unstale":"2024-01-11T07:16:36.507399+0000","last_undegraded":"2024-01-11T07:16:36.507399+0000","last_fullsized":"2024-01-11T07:16:36.507399+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:47:10.377333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.511669+0000","last_change":"2024-01-11T07:16:35.499633+0000","last_active":"2024-01-11T07:16:36.511669+0000","last_peered":"2024-01-11T07:16:36.511669+0000","last_clean":"2024-01-11T07:16:36.511669+0000","last_became_active":"2024-01-11T07:16:35.499414+0000","last_became_peered":"2024-01-11T07:16:35.499414+0000","last_unstale":"2024-01-11T07:16:36.511669+0000","last_undegraded":"2024-01-11T07:16:36.511669+0000","last_fullsized":"2024-01-11T07:16:36.511669+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:23:33.457256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.516447+0000","last_change":"2024-01-11T07:16:36.516447+0000","last_active":"2024-01-11T07:16:36.516447+0000","last_peered":"2024-01-11T07:16:36.516447+0000","last_clean":"2024-01-11T07:16:36.516447+0000","last_became_active":"2024-01-11T07:16:36.508058+0000","last_became_peered":"2024-01-11T07:16:36.508058+0000","last_unstale":"2024-01-11T07:16:36.516447+0000","last_undegraded":"2024-01-11T07:16:36.516447+0000","last_fullsized":"2024-01-11T07:16:36.516447+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:17:17.116289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"167'2","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.511915+0000","last_change":"2024-01-11T07:16:35.497508+0000","last_active":"2024-01-11T07:16:36.511915+0000","last_peered":"2024-01-11T07:16:36.511915+0000","last_clean":"2024-01-11T07:16:36.511915+0000","last_became_active":"2024-01-11T07:16:35.497320+0000","last_became_peered":"2024-01-11T07:16:35.497320+0000","last_unstale":"2024-01-11T07:16:36.511915+0000","last_undegraded":"2024-01-11T07:16:36.511915+0000","last_fullsized":"2024-01-11T07:16:36.511915+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:43:59.719379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501588+0000","last_change":"2024-01-11T07:16:36.501509+0000","last_active":"2024-01-11T07:16:36.501588+0000","last_peered":"2024-01-11T07:16:36.501588+0000","last_clean":"2024-01-11T07:16:36.501588+0000","last_became_active":"2024-01-11T07:16:36.501127+0000","last_became_peered":"2024-01-11T07:16:36.501127+0000","last_unstale":"2024-01-11T07:16:36.501588+0000","last_undegraded":"2024-01-11T07:16:36.501588+0000","last_fullsized":"2024-01-11T07:16:36.501588+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:42:40.170313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"167'1","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501096+0000","last_change":"2024-01-11T07:16:35.503269+0000","last_active":"2024-01-11T07:16:36.501096+0000","last_peered":"2024-01-11T07:16:36.501096+0000","last_clean":"2024-01-11T07:16:36.501096+0000","last_became_active":"2024-01-11T07:16:35.503032+0000","last_became_peered":"2024-01-11T07:16:35.503032+0000","last_unstale":"2024-01-11T07:16:36.501096+0000","last_undegraded":"2024-01-11T07:16:36.501096+0000","last_fullsized":"2024-01-11T07:16:36.501096+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:24:24.061998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.498194+0000","last_change":"2024-01-11T07:16:36.498194+0000","last_active":"2024-01-11T07:16:36.498194+0000","last_peered":"2024-01-11T07:16:36.498194+0000","last_clean":"2024-01-11T07:16:36.498194+0000","last_became_active":"2024-01-11T07:16:36.497529+0000","last_became_peered":"2024-01-11T07:16:36.497529+0000","last_unstale":"2024-01-11T07:16:36.498194+0000","last_undegraded":"2024-01-11T07:16:36.498194+0000","last_fullsized":"2024-01-11T07:16:36.498194+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:41:17.542100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.499788+0000","last_change":"2024-01-11T07:16:35.499694+0000","last_active":"2024-01-11T07:16:35.499788+0000","last_peered":"2024-01-11T07:16:35.499788+0000","last_clean":"2024-01-11T07:16:35.499788+0000","last_became_active":"2024-01-11T07:16:35.499437+0000","last_became_peered":"2024-01-11T07:16:35.499437+0000","last_unstale":"2024-01-11T07:16:35.499788+0000","last_undegraded":"2024-01-11T07:16:35.499788+0000","last_fullsized":"2024-01-11T07:16:35.499788+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:03:40.615120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.503415+0000","last_change":"2024-01-11T07:16:36.503028+0000","last_active":"2024-01-11T07:16:36.503415+0000","last_peered":"2024-01-11T07:16:36.503415+0000","last_clean":"2024-01-11T07:16:36.503415+0000","last_became_active":"2024-01-11T07:16:36.502646+0000","last_became_peered":"2024-01-11T07:16:36.502646+0000","last_unstale":"2024-01-11T07:16:36.503415+0000","last_undegraded":"2024-01-11T07:16:36.503415+0000","last_fullsized":"2024-01-11T07:16:36.503415+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:37:41.769728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.511391+0000","last_change":"2024-01-11T07:16:35.502571+0000","last_active":"2024-01-11T07:16:36.511391+0000","last_peered":"2024-01-11T07:16:36.511391+0000","last_clean":"2024-01-11T07:16:36.511391+0000","last_became_active":"2024-01-11T07:16:35.502110+0000","last_became_peered":"2024-01-11T07:16:35.502110+0000","last_unstale":"2024-01-11T07:16:36.511391+0000","last_undegraded":"2024-01-11T07:16:36.511391+0000","last_fullsized":"2024-01-11T07:16:36.511391+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:47:09.372309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.504708+0000","last_change":"2024-01-11T07:16:36.504387+0000","last_active":"2024-01-11T07:16:36.504708+0000","last_peered":"2024-01-11T07:16:36.504708+0000","last_clean":"2024-01-11T07:16:36.504708+0000","last_became_active":"2024-01-11T07:16:36.503393+0000","last_became_peered":"2024-01-11T07:16:36.503393+0000","last_unstale":"2024-01-11T07:16:36.504708+0000","last_undegraded":"2024-01-11T07:16:36.504708+0000","last_fullsized":"2024-01-11T07:16:36.504708+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:15:08.671542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.497289+0000","last_change":"2024-01-11T07:16:35.505776+0000","last_active":"2024-01-11T07:16:36.497289+0000","last_peered":"2024-01-11T07:16:36.497289+0000","last_clean":"2024-01-11T07:16:36.497289+0000","last_became_active":"2024-01-11T07:16:35.505552+0000","last_became_peered":"2024-01-11T07:16:35.505552+0000","last_unstale":"2024-01-11T07:16:36.497289+0000","last_undegraded":"2024-01-11T07:16:36.497289+0000","last_fullsized":"2024-01-11T07:16:36.497289+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:43:59.719379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.504047+0000","last_change":"2024-01-11T07:16:36.503978+0000","last_active":"2024-01-11T07:16:36.504047+0000","last_peered":"2024-01-11T07:16:36.504047+0000","last_clean":"2024-01-11T07:16:36.504047+0000","last_became_active":"2024-01-11T07:16:36.503740+0000","last_became_peered":"2024-01-11T07:16:36.503740+0000","last_unstale":"2024-01-11T07:16:36.504047+0000","last_undegraded":"2024-01-11T07:16:36.504047+0000","last_fullsized":"2024-01-11T07:16:36.504047+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:02:57.174064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.499271+0000","last_change":"2024-01-11T07:16:35.504408+0000","last_active":"2024-01-11T07:16:36.499271+0000","last_peered":"2024-01-11T07:16:36.499271+0000","last_clean":"2024-01-11T07:16:36.499271+0000","last_became_active":"2024-01-11T07:16:35.504195+0000","last_became_peered":"2024-01-11T07:16:35.504195+0000","last_unstale":"2024-01-11T07:16:36.499271+0000","last_undegraded":"2024-01-11T07:16:36.499271+0000","last_fullsized":"2024-01-11T07:16:36.499271+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:25:32.485892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505002+0000","last_change":"2024-01-11T07:16:36.504932+0000","last_active":"2024-01-11T07:16:36.505002+0000","last_peered":"2024-01-11T07:16:36.505002+0000","last_clean":"2024-01-11T07:16:36.505002+0000","last_became_active":"2024-01-11T07:16:36.504660+0000","last_became_peered":"2024-01-11T07:16:36.504660+0000","last_unstale":"2024-01-11T07:16:36.505002+0000","last_undegraded":"2024-01-11T07:16:36.505002+0000","last_fullsized":"2024-01-11T07:16:36.505002+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:21:30.422304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.493467+0000","last_change":"2024-01-11T07:16:35.498372+0000","last_active":"2024-01-11T07:16:36.493467+0000","last_peered":"2024-01-11T07:16:36.493467+0000","last_clean":"2024-01-11T07:16:36.493467+0000","last_became_active":"2024-01-11T07:16:35.498117+0000","last_became_peered":"2024-01-11T07:16:35.498117+0000","last_unstale":"2024-01-11T07:16:36.493467+0000","last_undegraded":"2024-01-11T07:16:36.493467+0000","last_fullsized":"2024-01-11T07:16:36.493467+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:10:03.219468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501920+0000","last_change":"2024-01-11T07:16:36.501513+0000","last_active":"2024-01-11T07:16:36.501920+0000","last_peered":"2024-01-11T07:16:36.501920+0000","last_clean":"2024-01-11T07:16:36.501920+0000","last_became_active":"2024-01-11T07:16:36.500644+0000","last_became_peered":"2024-01-11T07:16:36.500644+0000","last_unstale":"2024-01-11T07:16:36.501920+0000","last_undegraded":"2024-01-11T07:16:36.501920+0000","last_fullsized":"2024-01-11T07:16:36.501920+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:34:40.895765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"167'2","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.512434+0000","last_change":"2024-01-11T07:16:35.506446+0000","last_active":"2024-01-11T07:16:36.512434+0000","last_peered":"2024-01-11T07:16:36.512434+0000","last_clean":"2024-01-11T07:16:36.512434+0000","last_became_active":"2024-01-11T07:16:35.505916+0000","last_became_peered":"2024-01-11T07:16:35.505916+0000","last_unstale":"2024-01-11T07:16:36.512434+0000","last_undegraded":"2024-01-11T07:16:36.512434+0000","last_fullsized":"2024-01-11T07:16:36.512434+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:54:14.924711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.501509+0000","last_change":"2024-01-11T07:16:36.501369+0000","last_active":"2024-01-11T07:16:36.501509+0000","last_peered":"2024-01-11T07:16:36.501509+0000","last_clean":"2024-01-11T07:16:36.501509+0000","last_became_active":"2024-01-11T07:16:36.500805+0000","last_became_peered":"2024-01-11T07:16:36.500805+0000","last_unstale":"2024-01-11T07:16:36.501509+0000","last_undegraded":"2024-01-11T07:16:36.501509+0000","last_fullsized":"2024-01-11T07:16:36.501509+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:42:13.606784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.504017+0000","last_change":"2024-01-11T07:16:35.503371+0000","last_active":"2024-01-11T07:16:35.504017+0000","last_peered":"2024-01-11T07:16:35.504017+0000","last_clean":"2024-01-11T07:16:35.504017+0000","last_became_active":"2024-01-11T07:16:35.503155+0000","last_became_peered":"2024-01-11T07:16:35.503155+0000","last_unstale":"2024-01-11T07:16:35.504017+0000","last_undegraded":"2024-01-11T07:16:35.504017+0000","last_fullsized":"2024-01-11T07:16:35.504017+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:51:10.545550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.509302+0000","last_change":"2024-01-11T07:16:36.509302+0000","last_active":"2024-01-11T07:16:36.509302+0000","last_peered":"2024-01-11T07:16:36.509302+0000","last_clean":"2024-01-11T07:16:36.509302+0000","last_became_active":"2024-01-11T07:16:36.508505+0000","last_became_peered":"2024-01-11T07:16:36.508505+0000","last_unstale":"2024-01-11T07:16:36.509302+0000","last_undegraded":"2024-01-11T07:16:36.509302+0000","last_fullsized":"2024-01-11T07:16:36.509302+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:45:50.182569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.503941+0000","last_change":"2024-01-11T07:16:35.501284+0000","last_active":"2024-01-11T07:16:35.503941+0000","last_peered":"2024-01-11T07:16:35.503941+0000","last_clean":"2024-01-11T07:16:35.503941+0000","last_became_active":"2024-01-11T07:16:35.500679+0000","last_became_peered":"2024-01-11T07:16:35.500679+0000","last_unstale":"2024-01-11T07:16:35.503941+0000","last_undegraded":"2024-01-11T07:16:35.503941+0000","last_fullsized":"2024-01-11T07:16:35.503941+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:34:39.890740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.505149+0000","last_change":"2024-01-11T07:16:36.505149+0000","last_active":"2024-01-11T07:16:36.505149+0000","last_peered":"2024-01-11T07:16:36.505149+0000","last_clean":"2024-01-11T07:16:36.505149+0000","last_became_active":"2024-01-11T07:16:36.501191+0000","last_became_peered":"2024-01-11T07:16:36.501191+0000","last_unstale":"2024-01-11T07:16:36.505149+0000","last_undegraded":"2024-01-11T07:16:36.505149+0000","last_fullsized":"2024-01-11T07:16:36.505149+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:15:54.300795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.500631+0000","last_change":"2024-01-11T07:16:35.500532+0000","last_active":"2024-01-11T07:16:35.500631+0000","last_peered":"2024-01-11T07:16:35.500631+0000","last_clean":"2024-01-11T07:16:35.500631+0000","last_became_active":"2024-01-11T07:16:35.500285+0000","last_became_peered":"2024-01-11T07:16:35.500285+0000","last_unstale":"2024-01-11T07:16:35.500631+0000","last_undegraded":"2024-01-11T07:16:35.500631+0000","last_fullsized":"2024-01-11T07:16:35.500631+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:57:47.836502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":8,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.508859+0000","last_change":"2024-01-11T07:16:36.508859+0000","last_active":"2024-01-11T07:16:36.508859+0000","last_peered":"2024-01-11T07:16:36.508859+0000","last_clean":"2024-01-11T07:16:36.508859+0000","last_became_active":"2024-01-11T07:16:36.507203+0000","last_became_peered":"2024-01-11T07:16:36.507203+0000","last_unstale":"2024-01-11T07:16:36.508859+0000","last_undegraded":"2024-01-11T07:16:36.508859+0000","last_fullsized":"2024-01-11T07:16:36.508859+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:45:28.431412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.490170+0000","last_change":"2024-01-11T07:16:35.495523+0000","last_active":"2024-01-11T07:16:36.490170+0000","last_peered":"2024-01-11T07:16:36.490170+0000","last_clean":"2024-01-11T07:16:36.490170+0000","last_became_active":"2024-01-11T07:16:35.495287+0000","last_became_peered":"2024-01-11T07:16:35.495287+0000","last_unstale":"2024-01-11T07:16:36.490170+0000","last_undegraded":"2024-01-11T07:16:36.490170+0000","last_fullsized":"2024-01-11T07:16:36.490170+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:01:38.148568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.495500+0000","last_change":"2024-01-11T07:16:36.495431+0000","last_active":"2024-01-11T07:16:36.495500+0000","last_peered":"2024-01-11T07:16:36.495500+0000","last_clean":"2024-01-11T07:16:36.495500+0000","last_became_active":"2024-01-11T07:16:36.495172+0000","last_became_peered":"2024-01-11T07:16:36.495172+0000","last_unstale":"2024-01-11T07:16:36.495500+0000","last_undegraded":"2024-01-11T07:16:36.495500+0000","last_fullsized":"2024-01-11T07:16:36.495500+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:11:18.896950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.495241+0000","last_change":"2024-01-11T07:16:35.506542+0000","last_active":"2024-01-11T07:16:36.495241+0000","last_peered":"2024-01-11T07:16:36.495241+0000","last_clean":"2024-01-11T07:16:36.495241+0000","last_became_active":"2024-01-11T07:16:35.506155+0000","last_became_peered":"2024-01-11T07:16:35.506155+0000","last_unstale":"2024-01-11T07:16:36.495241+0000","last_undegraded":"2024-01-11T07:16:36.495241+0000","last_fullsized":"2024-01-11T07:16:36.495241+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:59:27.797942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.509081+0000","last_change":"2024-01-11T07:16:36.509032+0000","last_active":"2024-01-11T07:16:36.509081+0000","last_peered":"2024-01-11T07:16:36.509081+0000","last_clean":"2024-01-11T07:16:36.509081+0000","last_became_active":"2024-01-11T07:16:36.508729+0000","last_became_peered":"2024-01-11T07:16:36.508729+0000","last_unstale":"2024-01-11T07:16:36.509081+0000","last_undegraded":"2024-01-11T07:16:36.509081+0000","last_fullsized":"2024-01-11T07:16:36.509081+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:46:03.532281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.492637+0000","last_change":"2024-01-11T07:16:35.499876+0000","last_active":"2024-01-11T07:16:36.492637+0000","last_peered":"2024-01-11T07:16:36.492637+0000","last_clean":"2024-01-11T07:16:36.492637+0000","last_became_active":"2024-01-11T07:16:35.499426+0000","last_became_peered":"2024-01-11T07:16:35.499426+0000","last_unstale":"2024-01-11T07:16:36.492637+0000","last_undegraded":"2024-01-11T07:16:36.492637+0000","last_fullsized":"2024-01-11T07:16:36.492637+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:40:09.789035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.511086+0000","last_change":"2024-01-11T07:16:36.511006+0000","last_active":"2024-01-11T07:16:36.511086+0000","last_peered":"2024-01-11T07:16:36.511086+0000","last_clean":"2024-01-11T07:16:36.511086+0000","last_became_active":"2024-01-11T07:16:36.510780+0000","last_became_peered":"2024-01-11T07:16:36.510780+0000","last_unstale":"2024-01-11T07:16:36.511086+0000","last_undegraded":"2024-01-11T07:16:36.511086+0000","last_fullsized":"2024-01-11T07:16:36.511086+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:26:51.311106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-01-11T07:16:35.504539+0000","last_change":"2024-01-11T07:16:35.504444+0000","last_active":"2024-01-11T07:16:35.504539+0000","last_peered":"2024-01-11T07:16:35.504539+0000","last_clean":"2024-01-11T07:16:35.504539+0000","last_became_active":"2024-01-11T07:16:35.503689+0000","last_became_peered":"2024-01-11T07:16:35.503689+0000","last_unstale":"2024-01-11T07:16:35.504539+0000","last_undegraded":"2024-01-11T07:16:35.504539+0000","last_fullsized":"2024-01-11T07:16:35.504539+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:38:02.490699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.502527+0000","last_change":"2024-01-11T07:16:36.502450+0000","last_active":"2024-01-11T07:16:36.502527+0000","last_peered":"2024-01-11T07:16:36.502527+0000","last_clean":"2024-01-11T07:16:36.502527+0000","last_became_active":"2024-01-11T07:16:36.502209+0000","last_became_peered":"2024-01-11T07:16:36.502209+0000","last_unstale":"2024-01-11T07:16:36.502527+0000","last_undegraded":"2024-01-11T07:16:36.502527+0000","last_fullsized":"2024-01-11T07:16:36.502527+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:35.482363+0000","last_clean_scrub_stamp":"2024-01-11T07:16:35.482363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:34:24.284543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"167'3","reported_seq":13,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.514086+0000","last_change":"2024-01-11T07:16:35.499172+0000","last_active":"2024-01-11T07:16:36.514086+0000","last_peered":"2024-01-11T07:16:36.514086+0000","last_clean":"2024-01-11T07:16:36.514086+0000","last_became_active":"2024-01-11T07:16:35.498876+0000","last_became_peered":"2024-01-11T07:16:35.498876+0000","last_unstale":"2024-01-11T07:16:36.514086+0000","last_undegraded":"2024-01-11T07:16:36.514086+0000","last_fullsized":"2024-01-11T07:16:36.514086+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:16:34.477339+0000","last_clean_scrub_stamp":"2024-01-11T07:16:34.477339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:43:59.095218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-01-11T07:16:36.497389+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:16:36.497389+0000","last_peered":"2024-01-11T07:16:36.497389+0000","last_clean":"2024-01-11T07:16:36.497389+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:16:36.497389+0000","last_undegraded":"2024-01-11T07:16:36.497389+0000","last_fullsized":"2024-01-11T07:16:36.497389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1492,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193820,"statfs":{"total":200000000,"available":198472060,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.45500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72699999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56999999999999995,"15min":0.54600000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.54100000000000004},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.42299999999999999,"5min":0.41499999999999998,"15min":0.36499999999999999},"max":{"1min":0.71099999999999997,"5min":0.78000000000000003,"15min":0.80500000000000005},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.54000000000000004},"min":{"1min":0.433,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.75700000000000001,"5min":0.77500000000000002,"15min":0.81399999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.56499999999999995},"min":{"1min":0.439,"5min":0.41199999999999998,"15min":0.39800000000000002},"max":{"1min":0.70599999999999996,"5min":0.79000000000000004,"15min":0.82399999999999995},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.40000000000000002,"15min":0.35099999999999998},"max":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.82999999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77200000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46999999999999997,"15min":0.48099999999999998},"min":{"1min":0.308,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":0.77900000000000003,"15min":0.85999999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.495,"15min":0.48799999999999999},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.78200000000000003,"15min":0.85899999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.625},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.37,"15min":0.34100000000000003},"max":{"1min":0.69999999999999996,"5min":0.85499999999999998,"15min":0.91600000000000004},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.41399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.752,"5min":0.85399999999999998,"15min":0.92500000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.82899999999999996,"15min":0.90300000000000002},"last":0.57699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":915,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194397,"statfs":{"total":200000000,"available":199062906,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.56100000000000005},"min":{"1min":0.41199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.81399999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.45800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.69299999999999995,"5min":0.76200000000000001,"15min":1.157},"last":0.62}]},{"osd":1,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.57199999999999995},"min":{"1min":0.437,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.752,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.56200000000000006},"min":{"1min":0.42299999999999999,"5min":0.35599999999999998,"15min":0.33800000000000002},"max":{"1min":0.77100000000000002,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.39300000000000002,"15min":0.29999999999999999},"max":{"1min":0.73299999999999998,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.56200000000000006},"min":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.35499999999999998,"15min":0.29599999999999999},"max":{"1min":0.79600000000000004,"5min":2.621,"15min":2.621},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.373},"max":{"1min":0.80400000000000005,"5min":0.85799999999999998,"15min":0.88500000000000001},"last":0.63200000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48099999999999998,"15min":0.52700000000000002},"min":{"1min":0.34000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65600000000000003,"5min":0.77500000000000002,"15min":0.82699999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.64800000000000002,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.504,"15min":0.51200000000000001},"min":{"1min":0.34999999999999998,"5min":0.27100000000000002,"15min":0.253},"max":{"1min":0.72099999999999997,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.71099999999999997,"5min":0.80500000000000005,"15min":0.89100000000000001},"last":0.501}]},{"osd":7,"last update":"Thu Jan 11 07:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.505,"15min":0.52500000000000002},"min":{"1min":0.373,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68600000000000005,"5min":0.749,"15min":0.76700000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.56799999999999995},"min":{"1min":0.40100000000000002,"5min":0.30199999999999999,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.48199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":909,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194403,"statfs":{"total":200000000,"available":199068786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.46700000000000003,"15min":0.498},"min":{"1min":0.29299999999999998,"5min":0.247,"15min":0.216},"max":{"1min":0.47399999999999998,"5min":8.5150000000000006,"15min":20.41},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47899999999999998,"15min":0.46100000000000002},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.28100000000000003},"max":{"1min":0.56599999999999995,"5min":3.7029999999999998,"15min":3.7029999999999998},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.50800000000000001,"15min":0.497},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55100000000000005,"5min":7.8099999999999996,"15min":15.611000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.36799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59399999999999997,"5min":6.1390000000000002,"15min":20.199999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53100000000000003,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.30199999999999999,"15min":0.29999999999999999},"max":{"1min":0.60299999999999998,"5min":5.6449999999999996,"15min":20.369},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.54100000000000004,"15min":0.47099999999999997},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.28299999999999997},"max":{"1min":0.61199999999999999,"5min":8.5790000000000006,"15min":8.5790000000000006},"last":0.379}]},{"osd":4,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.317},"max":{"1min":0.78100000000000003,"5min":7.6749999999999998,"15min":9.9570000000000007},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.44,"15min":0.41899999999999998},"max":{"1min":0.85699999999999998,"5min":5.7830000000000004,"15min":12.449999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.63300000000000001},"min":{"1min":0.51100000000000001,"5min":0.437,"15min":0.376},"max":{"1min":0.84499999999999997,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72199999999999998,"15min":0.69899999999999995},"min":{"1min":0.56999999999999995,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":7.7480000000000002,"15min":9.891},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.58699999999999997,"5min":0.45800000000000002,"15min":0.41899999999999998},"max":{"1min":0.83999999999999997,"5min":1.389,"15min":1.5569999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.74099999999999999,"15min":0.73199999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.82599999999999996,"5min":7.7770000000000001,"15min":9.9900000000000002},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72299999999999998,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.84799999999999998,"5min":5.8449999999999998,"15min":20.324000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.46999999999999997,"15min":0.46800000000000003},"max":{"1min":0.92300000000000004,"5min":2.548,"15min":2.548},"last":0.70399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193819,"statfs":{"total":200000000,"available":198471114,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40000000000000002,"15min":0.44400000000000001},"min":{"1min":0.27500000000000002,"5min":0.22700000000000001,"15min":0.224},"max":{"1min":0.63900000000000001,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.26800000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64700000000000002,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44900000000000001,"15min":0.61399999999999999},"min":{"1min":0.313,"5min":0.28000000000000003,"15min":0.23999999999999999},"max":{"1min":0.61399999999999999,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46400000000000002,"15min":0.504},"min":{"1min":0.372,"5min":0.29099999999999998,"15min":0.27300000000000002},"max":{"1min":0.63900000000000001,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.47299999999999998,"15min":0.63100000000000001},"min":{"1min":0.27700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55400000000000005,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.54300000000000004},"min":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68899999999999995,"5min":0.83499999999999996,"15min":2.855},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63700000000000001,"15min":0.61699999999999999},"min":{"1min":0.51300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90400000000000003,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68300000000000005,"15min":0.81399999999999995},"min":{"1min":0.45700000000000002,"5min":0.41499999999999998,"15min":0.40999999999999998},"max":{"1min":0.98399999999999999,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66100000000000003,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.436,"15min":0.39400000000000002},"max":{"1min":0.96199999999999997,"5min":1.032,"15min":2.819},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67800000000000005,"15min":0.65500000000000003},"min":{"1min":0.47299999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93500000000000005,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.85999999999999999},"min":{"1min":0.44500000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0349999999999999,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72199999999999998,"15min":0.76200000000000001},"min":{"1min":0.52900000000000003,"5min":0.48099999999999998,"15min":0.41799999999999998},"max":{"1min":1.0209999999999999,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70399999999999996,"15min":0.67300000000000004},"min":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.276,"15min":1.276},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.73699999999999999,"15min":0.86599999999999999},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.44900000000000001},"max":{"1min":1.0780000000000001,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":0.621}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":910,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194402,"statfs":{"total":200000000,"available":199067734,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":1.472},"last":0.316},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.28899999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.60699999999999998,"5min":0.65500000000000003,"15min":0.86099999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41999999999999998,"15min":0.435},"min":{"1min":0.30299999999999999,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":0.628,"15min":1.2430000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.39600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.246},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48599999999999999},"min":{"1min":0.34699999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.752,"15min":0.76800000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.29999999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":1.139},"last":0.438}]},{"osd":4,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61799999999999999,"15min":0.65200000000000002},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.86899999999999999,"5min":0.94399999999999995,"15min":1.0660000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.40100000000000002},"max":{"1min":0.89100000000000001,"5min":1.089,"15min":1.089},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.53800000000000003,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.94699999999999995,"5min":1.099,"15min":1.099},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67800000000000005,"15min":0.69699999999999995},"min":{"1min":0.52800000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91500000000000004,"5min":1.0229999999999999,"15min":1.3100000000000001},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86299999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.51200000000000001,"5min":0.47099999999999997,"15min":0.45800000000000002},"max":{"1min":0.89300000000000002,"5min":1.006,"15min":1.052},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66700000000000004,"15min":0.67600000000000005},"min":{"1min":0.52100000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":0.89800000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94799999999999995,"5min":1.042,"15min":1.042},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":908,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194403,"statfs":{"total":200000000,"available":199069250,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38800000000000001,"15min":0.64300000000000002},"min":{"1min":0.27900000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":11.481},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.46300000000000002,"15min":0.47599999999999998},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.31},"max":{"1min":0.59599999999999997,"5min":0.65400000000000003,"15min":1.6200000000000001},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39900000000000002,"15min":0.76600000000000001},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.23899999999999999},"max":{"1min":0.64500000000000002,"5min":0.69999999999999996,"15min":17.516999999999999},"last":0.432},{"interface":"front","average":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.56999999999999995,"5min":0.63800000000000001,"15min":0.67400000000000004},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.42399999999999999},"min":{"1min":0.309,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53700000000000003,"5min":0.746,"15min":0.76500000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.46300000000000002,"15min":0.82599999999999996},"min":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.628,"5min":0.69499999999999995,"15min":17.553000000000001},"last":0.37}]},{"osd":4,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.55200000000000005,"5min":0.40899999999999997,"15min":0.40300000000000002},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":0.94599999999999995},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67000000000000004,"15min":1.052},"min":{"1min":0.54800000000000004,"5min":0.38500000000000001,"15min":0.36199999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":17.474},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.99399999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":17.393000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65100000000000002,"15min":0.88200000000000001},"min":{"1min":0.501,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.94499999999999995,"5min":1.002,"15min":11.394},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65800000000000003,"15min":0.89500000000000002},"min":{"1min":0.54600000000000004,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.91900000000000004,"5min":0.95799999999999996,"15min":11.436999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.67100000000000004},"min":{"1min":0.52700000000000002,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:15:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.67800000000000005},"min":{"1min":0.55800000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96999999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68100000000000005,"15min":1.0349999999999999},"min":{"1min":0.55100000000000005,"5min":0.45700000000000002,"15min":0.42999999999999999},"max":{"1min":0.95499999999999996,"5min":1.1839999999999999,"15min":17.434000000000001},"last":0.57299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":914,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194397,"statfs":{"total":200000000,"available":199063052,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58799999999999997,"15min":0.89100000000000001},"min":{"1min":0.40899999999999997,"5min":0.30299999999999999,"15min":0.248},"max":{"1min":0.80500000000000005,"5min":0.81399999999999995,"15min":8.484},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.93200000000000005},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.82799999999999996,"15min":9.2599999999999998},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.76100000000000001},"min":{"1min":0.45000000000000001,"5min":0.36599999999999999,"15min":0.30199999999999999},"max":{"1min":0.73699999999999999,"5min":0.95299999999999996,"15min":8.5340000000000007},"last":0.504},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.85699999999999998},"min":{"1min":0.51800000000000002,"5min":0.34999999999999998,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.78300000000000003,"15min":9.2279999999999998},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59899999999999998,"15min":0.84899999999999998},"min":{"1min":0.45000000000000001,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":9.5640000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.76900000000000002},"min":{"1min":0.45200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.90600000000000003,"15min":6.8639999999999999},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.873},"min":{"1min":0.46100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68200000000000005,"5min":0.90800000000000003,"15min":8.9779999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.93700000000000006},"min":{"1min":0.439,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":0.89100000000000001,"15min":8.7309999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45700000000000002,"15min":0.76700000000000002},"min":{"1min":0.315,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.65400000000000003,"5min":0.77700000000000002,"15min":11.388999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.47699999999999998,"15min":0.72399999999999998},"min":{"1min":0.35399999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.68600000000000005,"5min":0.76800000000000002,"15min":8.6509999999999998},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.77000000000000002},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.30099999999999999},"max":{"1min":0.70499999999999996,"5min":0.80100000000000005,"15min":9.1449999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58499999999999996,"15min":0.59799999999999998},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":0.81200000000000006,"15min":1.119},"last":0.81899999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:16:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.76300000000000001},"min":{"1min":0.375,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.83299999999999996,"15min":8.6050000000000004},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.84999999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.80000000000000004,"15min":9.4640000000000004},"last":0.89200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":915,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194397,"statfs":{"total":200000000,"available":199063030,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.52400000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74399999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.48999999999999999,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.72499999999999998,"5min":0.77400000000000002,"15min":0.81899999999999995},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.55800000000000005,"5min":0.36099999999999999,"15min":0.24399999999999999},"max":{"1min":0.76200000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.626},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.51300000000000001,"5min":0.39600000000000002,"15min":0.32100000000000001},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.84099999999999997},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.53700000000000003,"5min":0.36499999999999999,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.56100000000000005,"5min":0.42199999999999999,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.93999999999999995},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.56599999999999995,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78500000000000003,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.622,"15min":0.62},"min":{"1min":0.52600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.753,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.746}]},{"osd":4,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.375,"5min":0.29399999999999998,"15min":0.28299999999999997},"max":{"1min":0.63200000000000001,"5min":0.73899999999999999,"15min":0.76000000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.40799999999999997,"5min":0.34599999999999997,"15min":0.32500000000000001},"max":{"1min":0.65000000000000002,"5min":0.73999999999999999,"15min":0.76100000000000001},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.39900000000000002,"5min":0.33700000000000002,"15min":0.313},"max":{"1min":0.61499999999999999,"5min":0.75900000000000001,"15min":0.78600000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54500000000000004,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.55300000000000005},"min":{"1min":0.42199999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":0.77700000000000002,"15min":1.0109999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57999999999999996,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.34399999999999997,"15min":0.34000000000000002},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.88200000000000001},"last":0.53900000000000003}]}]}],"pool_statfs":[]}} 2024-01-11T07:16:41.103 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:16:41.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:16:41.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.282+0000 7f1522ec6700 1 Processor -- start 2024-01-11T07:16:41.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.282+0000 7f1522ec6700 1 -- start start 2024-01-11T07:16:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.283+0000 7f1522ec6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f151c15e9c0 0x7f151c0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.283+0000 7f1522ec6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f151c071130 0x7f151c0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.283+0000 7f1522ec6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151c15e530 con 0x7f151c159f50 2024-01-11T07:16:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.283+0000 7f1522ec6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0cc970 con 0x7f151c15e9c0 2024-01-11T07:16:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.283+0000 7f1522ec6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0cc690 con 0x7f151c071130 2024-01-11T07:16:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f151c15e9c0 0x7f151c0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f151c15e9c0 0x7f151c0cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56114/0 (socket says 172.21.15.7:56114) 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151bfff700 1 -- 172.21.15.7:0/4239214335 learned_addr learned my addr 172.21.15.7:0/4239214335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f1521463700 1 --2- 172.21.15.7:0/4239214335 >> v2:172.21.15.113:3300/0 conn(0x7f151c071130 0x7f151c0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151bfff700 1 -- 172.21.15.7:0/4239214335 >> v2:172.21.15.113:3300/0 conn(0x7f151c071130 msgr2=0x7f151c0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151bfff700 1 --2- 172.21.15.7:0/4239214335 >> v2:172.21.15.113:3300/0 conn(0x7f151c071130 0x7f151c0cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151bfff700 1 -- 172.21.15.7:0/4239214335 >> v1:172.21.15.113:6789/0 conn(0x7f151c159f50 legacy=0x7f151c0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151bfff700 1 -- 172.21.15.7:0/4239214335 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c15dee0 con 0x7f151c15e9c0 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151bfff700 1 --2- 172.21.15.7:0/4239214335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c15e9c0 0x7f151c0cd630 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f151c064c10 tx=0x7f1508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b098fdfb565a153 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151b7fe700 1 -- 172.21.15.7:0/4239214335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508009e70 con 0x7f151c15e9c0 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151b7fe700 1 -- 172.21.15.7:0/4239214335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150800b860 con 0x7f151c15e9c0 2024-01-11T07:16:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.284+0000 7f151b7fe700 1 -- 172.21.15.7:0/4239214335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800bc00 con 0x7f151c15e9c0 2024-01-11T07:16:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.285+0000 7f1522ec6700 1 -- 172.21.15.7:0/4239214335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c15e9c0 msgr2=0x7f151c0cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.285+0000 7f1522ec6700 1 --2- 172.21.15.7:0/4239214335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c15e9c0 0x7f151c0cd630 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f151c064c10 tx=0x7f1508009250 comp rx=0 tx=0).stop 2024-01-11T07:16:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.285+0000 7f1522ec6700 1 -- 172.21.15.7:0/4239214335 shutdown_connections 2024-01-11T07:16:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.285+0000 7f1522ec6700 1 --2- 172.21.15.7:0/4239214335 >> v2:172.21.15.113:3300/0 conn(0x7f151c071130 0x7f151c0cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.285+0000 7f1522ec6700 1 --2- 172.21.15.7:0/4239214335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c15e9c0 0x7f151c0cd630 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.285+0000 7f1522ec6700 1 -- 172.21.15.7:0/4239214335 >> 172.21.15.7:0/4239214335 conn(0x7f151c06e6b0 msgr2=0x7f151c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.285+0000 7f1522ec6700 1 -- 172.21.15.7:0/4239214335 shutdown_connections 2024-01-11T07:16:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.285+0000 7f1522ec6700 1 -- 172.21.15.7:0/4239214335 wait complete. 2024-01-11T07:16:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.286+0000 7f1522ec6700 1 Processor -- start 2024-01-11T07:16:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.286+0000 7f1522ec6700 1 -- start start 2024-01-11T07:16:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f1522ec6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f151c071130 0x7f151c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f1522ec6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c159f50 0x7f151c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f1522ec6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f151c15e9c0 0x7f151c1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f1522ec6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c152c20 con 0x7f151c159f50 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f1522ec6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c16b880 con 0x7f151c15e9c0 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f1522ec6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c1530e0 con 0x7f151c071130 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f151bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c159f50 0x7f151c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f151bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c159f50 0x7f151c1526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56116/0 (socket says 172.21.15.7:56116) 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.287+0000 7f151bfff700 1 -- 172.21.15.7:0/348410477 learned_addr learned my addr 172.21.15.7:0/348410477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f1520c62700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f151c071130 0x7f151c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f1521463700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f151c15e9c0 0x7f151c1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f151bfff700 1 -- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f151c071130 msgr2=0x7f151c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f151bfff700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f151c071130 0x7f151c0b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f151bfff700 1 -- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f151c15e9c0 msgr2=0x7f151c1591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f151bfff700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f151c15e9c0 0x7f151c1591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f151bfff700 1 -- 172.21.15.7:0/348410477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15080094f0 con 0x7f151c159f50 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f1521463700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f151c15e9c0 0x7f151c1591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f151bfff700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c159f50 0x7f151c1526e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f151c067f20 tx=0x7f1508017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f1518ff9700 1 -- 172.21.15.7:0/348410477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800bbe0 con 0x7f151c159f50 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f1518ff9700 1 -- 172.21.15.7:0/348410477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1508027450 con 0x7f151c159f50 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c16ac70 con 0x7f151c159f50 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f1518ff9700 1 -- 172.21.15.7:0/348410477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15080277f0 con 0x7f151c159f50 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.288+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c16b230 con 0x7f151c159f50 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.289+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c0b39d0 con 0x7f151c159f50 2024-01-11T07:16:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.290+0000 7f1518ff9700 1 -- 172.21.15.7:0/348410477 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1508027ba0 con 0x7f151c159f50 2024-01-11T07:16:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.290+0000 7f1518ff9700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1500066f30 0x7f15000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.290+0000 7f1518ff9700 1 -- 172.21.15.7:0/348410477 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15080863b0 con 0x7f151c159f50 2024-01-11T07:16:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.290+0000 7f1520c62700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1500066f30 0x7f15000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.291+0000 7f1520c62700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1500066f30 0x7f15000693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f151c0c8490 tx=0x7f1510009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:41.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.295+0000 7f1518ff9700 1 -- 172.21.15.7:0/348410477 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f151c0b39d0 con 0x7f151c159f50 2024-01-11T07:16:41.479 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-01-11T07:16:41.479+0000 7ff55954b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:16:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.591+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f151c0b39d0 con 0x7f151c159f50 2024-01-11T07:16:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.595+0000 7f1518ff9700 1 -- 172.21.15.7:0/348410477 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f151c0b39d0 con 0x7f151c159f50 2024-01-11T07:16:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1500066f30 msgr2=0x7f15000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1500066f30 0x7f15000693e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f151c0c8490 tx=0x7f1510009040 comp rx=0 tx=0).stop 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c159f50 msgr2=0x7f151c1526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c159f50 0x7f151c1526e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f151c067f20 tx=0x7f1508017000 comp rx=0 tx=0).stop 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 shutdown_connections 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1500066f30 0x7f15000693e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f151c15e9c0 0x7f151c1591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f151c071130 0x7f151c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 --2- 172.21.15.7:0/348410477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f151c159f50 0x7f151c1526e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.597+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 >> 172.21.15.7:0/348410477 conn(0x7f151c06e6b0 msgr2=0x7f151c153980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.598+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 shutdown_connections 2024-01-11T07:16:41.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.598+0000 7f1522ec6700 1 -- 172.21.15.7:0/348410477 wait complete. 2024-01-11T07:16:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:16:41.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:16:41.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.743+0000 7f1f23110700 1 Processor -- start 2024-01-11T07:16:41.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.743+0000 7f1f23110700 1 -- start start 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f23110700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f1c0cafb0 0x7f1f1c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f23110700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c158280 0x7f1f1c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f23110700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1c159300 con 0x7f1f1c0b0970 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f23110700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1c0b0df0 con 0x7f1f1c0cafb0 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f23110700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0cf260 con 0x7f1f1c158280 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f216ad700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f1c0b0970 0x7f1f1c159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57928/0 (socket says 172.21.15.7:57928) 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f216ad700 1 -- 172.21.15.7:0/2424199651 learned_addr learned my addr 172.21.15.7:0/2424199651 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f1bfff700 1 --2- 172.21.15.7:0/2424199651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c158280 0x7f1f1c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f20eac700 1 --2- 172.21.15.7:0/2424199651 >> v2:172.21.15.113:3300/0 conn(0x7f1f1c0cafb0 0x7f1f1c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f1b7fe700 1 -- 172.21.15.7:0/2424199651 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130305573 0 0) 0x7f1f1c159300 con 0x7f1f1c0b0970 2024-01-11T07:16:41.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.744+0000 7f1f1b7fe700 1 -- 172.21.15.7:0/2424199651 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f0c003570 con 0x7f1f1c0b0970 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f1bfff700 1 -- 172.21.15.7:0/2424199651 >> v2:172.21.15.113:3300/0 conn(0x7f1f1c0cafb0 msgr2=0x7f1f1c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f1bfff700 1 --2- 172.21.15.7:0/2424199651 >> v2:172.21.15.113:3300/0 conn(0x7f1f1c0cafb0 0x7f1f1c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f1bfff700 1 -- 172.21.15.7:0/2424199651 >> v1:172.21.15.7:6789/0 conn(0x7f1f1c0b0970 legacy=0x7f1f1c159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f1bfff700 1 -- 172.21.15.7:0/2424199651 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c0cf540 con 0x7f1f1c158280 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f20eac700 1 --2- 172.21.15.7:0/2424199651 >> v2:172.21.15.113:3300/0 conn(0x7f1f1c0cafb0 0x7f1f1c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f1bfff700 1 --2- 172.21.15.7:0/2424199651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c158280 0x7f1f1c0b0430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f080098e0 tx=0x7f1f080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d76ef95a11d1c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f1b7fe700 1 -- 172.21.15.7:0/2424199651 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08011630 con 0x7f1f1c158280 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f1b7fe700 1 -- 172.21.15.7:0/2424199651 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f08011790 con 0x7f1f1c158280 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.745+0000 7f1f1b7fe700 1 -- 172.21.15.7:0/2424199651 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08011b30 con 0x7f1f1c158280 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.746+0000 7f1f23110700 1 -- 172.21.15.7:0/2424199651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c158280 msgr2=0x7f1f1c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.746+0000 7f1f23110700 1 --2- 172.21.15.7:0/2424199651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c158280 0x7f1f1c0b0430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f080098e0 tx=0x7f1f080091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.746+0000 7f1f23110700 1 -- 172.21.15.7:0/2424199651 shutdown_connections 2024-01-11T07:16:41.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.746+0000 7f1f23110700 1 --2- 172.21.15.7:0/2424199651 >> v2:172.21.15.113:3300/0 conn(0x7f1f1c0cafb0 0x7f1f1c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.746+0000 7f1f23110700 1 --2- 172.21.15.7:0/2424199651 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c158280 0x7f1f1c0b0430 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.746+0000 7f1f23110700 1 -- 172.21.15.7:0/2424199651 >> 172.21.15.7:0/2424199651 conn(0x7f1f1c071100 msgr2=0x7f1f1c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.746+0000 7f1f23110700 1 -- 172.21.15.7:0/2424199651 shutdown_connections 2024-01-11T07:16:41.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.746+0000 7f1f23110700 1 -- 172.21.15.7:0/2424199651 wait complete. 2024-01-11T07:16:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.747+0000 7f1f23110700 1 Processor -- start 2024-01-11T07:16:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.747+0000 7f1f23110700 1 -- start start 2024-01-11T07:16:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f23110700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1c0b0970 0x7f1f1c168a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f23110700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c0cafb0 0x7f1f1c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f23110700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c158280 0x7f1f1c0c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f23110700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c168fb0 con 0x7f1f1c158280 2024-01-11T07:16:41.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f23110700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c15b840 con 0x7f1f1c0b0970 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f23110700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c15bb20 con 0x7f1f1c0cafb0 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f216ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c158280 0x7f1f1c0c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f216ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c158280 0x7f1f1c0c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56128/0 (socket says 172.21.15.7:56128) 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f216ad700 1 -- 172.21.15.7:0/2163173620 learned_addr learned my addr 172.21.15.7:0/2163173620 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f20eac700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1c0b0970 0x7f1f1c168a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f1bfff700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c0cafb0 0x7f1f1c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f216ad700 1 -- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c0cafb0 msgr2=0x7f1f1c0af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f216ad700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c0cafb0 0x7f1f1c0af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f216ad700 1 -- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1c0b0970 msgr2=0x7f1f1c168a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.748+0000 7f1f216ad700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1c0b0970 0x7f1f1c168a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.749+0000 7f1f216ad700 1 -- 172.21.15.7:0/2163173620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10009420 con 0x7f1f1c158280 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.749+0000 7f1f216ad700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c158280 0x7f1f1c0c4e60 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001460 tx=0x7f1f14001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.749+0000 7f1f18ff9700 1 -- 172.21.15.7:0/2163173620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400a890 con 0x7f1f1c158280 2024-01-11T07:16:41.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.749+0000 7f1f18ff9700 1 -- 172.21.15.7:0/2163173620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1400a9f0 con 0x7f1f1c158280 2024-01-11T07:16:41.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.749+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f08009420 con 0x7f1f1c158280 2024-01-11T07:16:41.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.749+0000 7f1f18ff9700 1 -- 172.21.15.7:0/2163173620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400ad70 con 0x7f1f1c158280 2024-01-11T07:16:41.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.749+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c15c0e0 con 0x7f1f1c158280 2024-01-11T07:16:41.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.750+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c15c730 con 0x7f1f1c158280 2024-01-11T07:16:41.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.750+0000 7f1f18ff9700 1 -- 172.21.15.7:0/2163173620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f14007300 con 0x7f1f1c158280 2024-01-11T07:16:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.751+0000 7f1f18ff9700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.751+0000 7f1f18ff9700 1 -- 172.21.15.7:0/2163173620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f14084910 con 0x7f1f1c158280 2024-01-11T07:16:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.751+0000 7f1f20eac700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.752+0000 7f1f20eac700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009f40 tx=0x7f1f100091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:41.755+0000 7f1f18ff9700 1 -- 172.21.15.7:0/2163173620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f14051b60 con 0x7f1f1c158280 2024-01-11T07:16:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.006+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f1c04c3c0 con 0x7f1f1c158280 2024-01-11T07:16:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.007+0000 7f1f18ff9700 1 -- 172.21.15.7:0/2163173620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f1f14055180 con 0x7f1f1c158280 2024-01-11T07:16:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:42.007 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4097846800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4097846800},{"type":"v1","addr":"172.21.15.113:6836","nonce":4097846800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1898628721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1898628721},{"type":"v1","addr":"172.21.15.113:6837","nonce":1898628721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3371959241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3371959241},{"type":"v1","addr":"172.21.15.7:6835","nonce":3371959241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:16:35.923785+0000","modified":"2024-01-11T07:16:39.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/2285850291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2285850291},{"type":"v1","addr":"172.21.15.7:6833","nonce":2285850291}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 msgr2=0x7f1f000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009f40 tx=0x7f1f100091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c158280 msgr2=0x7f1f1c0c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c158280 0x7f1f1c0c4e60 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001460 tx=0x7f1f14001ae0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 shutdown_connections 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f00066f30 0x7f1f000693e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c158280 0x7f1f1c0c4e60 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f1c0b0970 0x7f1f1c168a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 --2- 172.21.15.7:0/2163173620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f1c0cafb0 0x7f1f1c0af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 >> 172.21.15.7:0/2163173620 conn(0x7f1f1c071100 msgr2=0x7f1f1c159970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 shutdown_connections 2024-01-11T07:16:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.009+0000 7f1f23110700 1 -- 172.21.15.7:0/2163173620 wait complete. 2024-01-11T07:16:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2024-01-11T07:16:42.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:16:35.923785+0000', 'modified': '2024-01-11T07:16:39.504707+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7746}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7746': {'gid': 7746, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6833/2285850291', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6832', 'nonce': 2285850291}, {'type': 'v1', 'addr': '172.21.15.7:6833', 'nonce': 2285850291}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:16:42.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:16:42.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-11T07:16:42.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.570+0000 7fa23f164700 1 Processor -- start 2024-01-11T07:16:42.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.570+0000 7fa23f164700 1 -- start start 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.571+0000 7fa23f164700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa238157d80 0x7fa238063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.571+0000 7fa23f164700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23815c660 con 0x7fa2381579b0 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.571+0000 7fa23f164700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa238063bb0 con 0x7fa238063770 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.571+0000 7fa23f164700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380aeab0 con 0x7fa238157d80 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.571+0000 7fa23cf00700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa2381579b0 0x7fa23815bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57942/0 (socket says 172.21.15.7:57942) 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.571+0000 7fa23cf00700 1 -- 172.21.15.7:0/2335175778 learned_addr learned my addr 172.21.15.7:0/2335175778 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa237fff700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa238157d80 0x7fa238063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704391330 0 0) 0x7fa23815c660 con 0x7fa2381579b0 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa220003570 con 0x7fa2381579b0 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1693542042 0 0) 0x7fa220003570 con 0x7fa2381579b0 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa238157d80 msgr2=0x7fa238063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa238157d80 0x7fa238063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 >> v1:172.21.15.113:6789/0 conn(0x7fa238063770 legacy=0x7fa238069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380aed90 con 0x7fa2381579b0 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa228001470 con 0x7fa2381579b0 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 >> v1:172.21.15.7:6789/0 conn(0x7fa2381579b0 legacy=0x7fa23815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220003a90 0x7fa220003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa238157d80 0x7fa220004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa220004d50 0x7fa2200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380aeab0 con 0x7fa220004d50 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22000bb70 con 0x7fa220003a90 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22000bd30 con 0x7fa238157d80 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa23cf00700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220003a90 0x7fa220003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa237fff700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa238157d80 0x7fa220004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa23d701700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa220004d50 0x7fa2200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa23d701700 1 -- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa238157d80 msgr2=0x7fa220004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa23d701700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa238157d80 0x7fa220004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa23d701700 1 -- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220003a90 msgr2=0x7fa220003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa23d701700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220003a90 0x7fa220003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.572+0000 7fa23d701700 1 -- 172.21.15.7:0/2335175778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220003570 con 0x7fa220004d50 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa23d701700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa220004d50 0x7fa2200051c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa230001f70 tx=0x7fa230001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d509c48514664875 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230004070 con 0x7fa220004d50 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23000c8b0 con 0x7fa220004d50 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa2377fe700 1 -- 172.21.15.7:0/2335175778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230001930 con 0x7fa220004d50 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa23f164700 1 -- 172.21.15.7:0/2335175778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa220004d50 msgr2=0x7fa2200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa23f164700 1 --2- 172.21.15.7:0/2335175778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa220004d50 0x7fa2200051c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa230001f70 tx=0x7fa230001640 comp rx=0 tx=0).stop 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa23d701700 1 -- 172.21.15.7:0/2335175778 reap_dead start 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa23f164700 1 -- 172.21.15.7:0/2335175778 shutdown_connections 2024-01-11T07:16:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa23f164700 1 -- 172.21.15.7:0/2335175778 >> 172.21.15.7:0/2335175778 conn(0x7fa2380680e0 msgr2=0x7fa2380cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:42.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.573+0000 7fa23f164700 1 -- 172.21.15.7:0/2335175778 shutdown_connections 2024-01-11T07:16:42.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.574+0000 7fa23f164700 1 -- 172.21.15.7:0/2335175778 wait complete. 2024-01-11T07:16:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.575+0000 7fa23f164700 1 Processor -- start 2024-01-11T07:16:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.575+0000 7fa23f164700 1 -- start start 2024-01-11T07:16:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23f164700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220004d50 0x7fa238158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23f164700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2380b0730 0x7fa238149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23f164700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2380c29b0 0x7fa2380c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23f164700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380c3390 con 0x7fa2380c29b0 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23f164700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa238159b40 con 0x7fa220004d50 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23f164700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa238158a30 con 0x7fa2380b0730 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2380c29b0 0x7fa2380c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2380c29b0 0x7fa2380c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56158/0 (socket says 172.21.15.7:56158) 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 -- 172.21.15.7:0/1210546864 learned_addr learned my addr 172.21.15.7:0/1210546864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23cf00700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220004d50 0x7fa238158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa237fff700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2380b0730 0x7fa238149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 -- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2380b0730 msgr2=0x7fa238149b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2380b0730 0x7fa238149b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 -- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220004d50 msgr2=0x7fa238158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220004d50 0x7fa238158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 -- 172.21.15.7:0/1210546864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa238158d10 con 0x7fa2380c29b0 2024-01-11T07:16:42.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa23d701700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2380c29b0 0x7fa2380c2e50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa22000bb40 tx=0x7fa230015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.576+0000 7fa234ff9700 1 -- 172.21.15.7:0/1210546864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230001930 con 0x7fa2380c29b0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.577+0000 7fa234ff9700 1 -- 172.21.15.7:0/1210546864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23000ce80 con 0x7fa2380c29b0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.577+0000 7fa23f164700 1 -- 172.21.15.7:0/1210546864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa238158ff0 con 0x7fa2380c29b0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.577+0000 7fa234ff9700 1 -- 172.21.15.7:0/1210546864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230004070 con 0x7fa2380c29b0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.577+0000 7fa23f164700 1 -- 172.21.15.7:0/1210546864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2380c48e0 con 0x7fa2380c29b0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.578+0000 7fa234ff9700 1 -- 172.21.15.7:0/1210546864 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa230019070 con 0x7fa2380c29b0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.578+0000 7fa234ff9700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa224066f30 0x7fa2240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.579+0000 7fa234ff9700 1 -- 172.21.15.7:0/1210546864 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa230085bc0 con 0x7fa2380c29b0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.579+0000 7fa23cf00700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa224066f30 0x7fa2240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.579+0000 7fa23cf00700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa224066f30 0x7fa2240693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa2280015a0 tx=0x7fa228001e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.581+0000 7fa248464b80 1 Processor -- start 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.581+0000 7fa248464b80 1 -- start start 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.581+0000 7fa248464b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b034370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.581+0000 7fa248464b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b02fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.581+0000 7fa248464b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b050360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.581+0000 7fa248464b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9af623e0 con 0x561b9b030170 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.581+0000 7fa248464b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9af5fbe0 con 0x561b9b033fa0 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.581+0000 7fa248464b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b9af5b9d0 con 0x561b9b0348b0 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b050360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b050360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56166/0 (socket says 172.21.15.7:56166) 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 -- 172.21.15.7:0/249509848 learned_addr learned my addr 172.21.15.7:0/249509848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa237fff700 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b02fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23cf00700 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b034370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 -- 172.21.15.7:0/249509848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 msgr2=0x561b9b02fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b02fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 -- 172.21.15.7:0/249509848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 msgr2=0x561b9b034370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b034370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 -- 172.21.15.7:0/249509848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b9adef360 con 0x561b9b030170 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa23d701700 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b050360 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa230084010 tx=0x7fa230083fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0dfad0cf64e6cc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa21dffb700 1 -- 172.21.15.7:0/249509848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230082070 con 0x561b9b030170 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa21dffb700 1 -- 172.21.15.7:0/249509848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa230027360 con 0x561b9b030170 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.582+0000 7fa21dffb700 1 -- 172.21.15.7:0/249509848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230057290 con 0x561b9b030170 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 -- 172.21.15.7:0/249509848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 msgr2=0x561b9b050360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b050360 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa230084010 tx=0x7fa230083fa0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 -- 172.21.15.7:0/249509848 shutdown_connections 2024-01-11T07:16:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b050360 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b034370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 --2- 172.21.15.7:0/249509848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b02fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 -- 172.21.15.7:0/249509848 >> 172.21.15.7:0/249509848 conn(0x561b9b00a440 msgr2=0x561b9b055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 -- 172.21.15.7:0/249509848 shutdown_connections 2024-01-11T07:16:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.583+0000 7fa248464b80 1 -- 172.21.15.7:0/249509848 wait complete. 2024-01-11T07:16:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.584+0000 7fa248464b80 1 -- start start 2024-01-11T07:16:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.584+0000 7fa248464b80 1 Processor -- start 2024-01-11T07:16:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa248464b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b049900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa248464b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b04b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa248464b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b04b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa248464b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9af081b0 con 0x561b9b030170 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa248464b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9af07e50 con 0x561b9b033fa0 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa248464b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b9ae54e40 con 0x561b9b0348b0 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa248464b80 10 client.0 _ll_register_callbacks cb 0x561b9ae577b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b049900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b049900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56182/0 (socket says 172.21.15.7:56182) 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 -- 172.21.15.7:0/492819223 learned_addr learned my addr 172.21.15.7:0/492819223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23d701700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b04b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa237fff700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b04b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 -- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 msgr2=0x561b9b04b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b04b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 -- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 msgr2=0x561b9b04b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b04b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 -- 172.21.15.7:0/492819223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2380c48e0 con 0x561b9b030170 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23d701700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b04b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa237fff700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b04b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.585+0000 7fa23cf00700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b049900 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa228015b20 tx=0x7fa228017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779ee3ace27119f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa216ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa216ffd700 1 -- 172.21.15.7:0/492819223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22801a040 con 0x561b9b030170 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa216ffd700 1 -- 172.21.15.7:0/492819223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa228027830 con 0x561b9b030170 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 1 -- 172.21.15.7:0/492819223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b9afdceb0 con 0x561b9b030170 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa216ffd700 1 -- 172.21.15.7:0/492819223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa228007380 con 0x561b9b030170 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 10 client.7878 fetch_fsmap learned FSMap version 81 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 1 -- 172.21.15.7:0/492819223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x561b9ad9bb10 con 0x561b9b030170 2024-01-11T07:16:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa216ffd700 1 -- 172.21.15.7:0/492819223 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561b9afdceb0 con 0x561b9b030170 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa216ffd700 1 -- 172.21.15.7:0/492819223 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 81) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa228027c00 con 0x561b9b030170 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 10 client.7878 fetch_fsmap finished waiting for FSMap version 81 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 10 client.7878 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 20 client.7878 populate_metadata read hostname 'smithi007' 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] conn(0x561b9b036ee0 0x561b9b06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 4 client.7878 mds_command: new command op to 7746 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa248464b80 1 -- 172.21.15.7:0/492819223 --> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b9af32880 con 0x561b9b036ee0 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.586+0000 7fa237fff700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] conn(0x561b9b036ee0 0x561b9b06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.587+0000 7fa237fff700 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] conn(0x561b9b036ee0 0x561b9b06ea20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee9bae62a377faf server_cookie=f645c89ce5d82322 in_seq=0 out_seq=0 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.587+0000 7fa216ffd700 10 client.7878 ms_handle_connect on v2:172.21.15.7:6832/2285850291 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.587+0000 7fa216ffd700 1 -- 172.21.15.7:0/492819223 <== mds.0 v2:172.21.15.7:6832/2285850291 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561b9af32880 con 0x561b9b036ee0 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.587+0000 7fa216ffd700 10 client.7878 handle_command_reply: tid=0 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.587+0000 7fa248464b80 1 client.7878 shutdown 2024-01-11T07:16:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 -- 172.21.15.7:0/492819223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 msgr2=0x561b9b049900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b049900 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa228015b20 tx=0x7fa228017000 comp rx=0 tx=0).stop 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 -- 172.21.15.7:0/492819223 shutdown_connections 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] conn(0x561b9b036ee0 0x561b9b06ea20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b0348b0 0x561b9b04b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b049900 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa216ffd700 0 client.7878 ms_handle_reset on v2:172.21.15.7:6832/2285850291 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 --2- 172.21.15.7:0/492819223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b033fa0 0x561b9b04b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 -- 172.21.15.7:0/492819223 >> 172.21.15.7:0/492819223 conn(0x561b9b00a440 msgr2=0x561b9b00a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 -- 172.21.15.7:0/492819223 shutdown_connections 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 1 -- 172.21.15.7:0/492819223 wait complete. 2024-01-11T07:16:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.588+0000 7fa248464b80 20 client.7878 trim_cache size 0 max 16384 2024-01-11T07:16:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.606+0000 7fa248464b80 1 Processor -- start 2024-01-11T07:16:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.606+0000 7fa248464b80 1 -- start start 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa248464b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b0027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa248464b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b033fa0 0x561b9b04b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa248464b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa248464b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9b00ca10 con 0x561b9b030170 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa248464b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9b038a40 con 0x561b9b0348b0 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa248464b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b9b02f110 con 0x561b9b033fa0 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa23cf00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b0027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa23cf00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b0027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56184/0 (socket says 172.21.15.7:56184) 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa23cf00700 1 -- 172.21.15.7:0/2674023011 learned_addr learned my addr 172.21.15.7:0/2674023011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa237fff700 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b033fa0 0x561b9b04b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa23d701700 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa23cf00700 1 -- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b033fa0 msgr2=0x561b9b04b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa23cf00700 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b033fa0 0x561b9b04b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa23cf00700 1 -- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 msgr2=0x561b9b008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.607+0000 7fa23cf00700 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.608+0000 7fa23cf00700 1 -- 172.21.15.7:0/2674023011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b9af2a290 con 0x561b9b030170 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.608+0000 7fa23d701700 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b008630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.608+0000 7fa23cf00700 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b0027b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa228001ad0 tx=0x7fa228018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d9602fe11127a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.608+0000 7fa21e7fc700 1 -- 172.21.15.7:0/2674023011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228007380 con 0x561b9b030170 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.608+0000 7fa21e7fc700 1 -- 172.21.15.7:0/2674023011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228014730 con 0x561b9b030170 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.608+0000 7fa21e7fc700 1 -- 172.21.15.7:0/2674023011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228014970 con 0x561b9b030170 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 -- 172.21.15.7:0/2674023011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 msgr2=0x561b9b0027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b0027b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa228001ad0 tx=0x7fa228018000 comp rx=0 tx=0).stop 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 -- 172.21.15.7:0/2674023011 shutdown_connections 2024-01-11T07:16:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b008630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b030170 0x561b9b0027b0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 --2- 172.21.15.7:0/2674023011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b033fa0 0x561b9b04b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 -- 172.21.15.7:0/2674023011 >> 172.21.15.7:0/2674023011 conn(0x561b9b00a440 msgr2=0x561b9b0372b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 -- 172.21.15.7:0/2674023011 shutdown_connections 2024-01-11T07:16:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 -- 172.21.15.7:0/2674023011 wait complete. 2024-01-11T07:16:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.609+0000 7fa248464b80 1 -- start start 2024-01-11T07:16:42.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.610+0000 7fa248464b80 1 Processor -- start 2024-01-11T07:16:42.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.610+0000 7fa248464b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b030170 0x561b9b0567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.610+0000 7fa248464b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b033fa0 0x561b9b0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.610+0000 7fa248464b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b00a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.610+0000 7fa248464b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9b048f60 con 0x561b9b033fa0 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.610+0000 7fa248464b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b9b049e50 con 0x561b9b0348b0 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.610+0000 7fa248464b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b9b04a3d0 con 0x561b9b030170 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.610+0000 7fa248464b80 10 client.0 _ll_register_callbacks cb 0x7fa2280078b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b033fa0 0x561b9b0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b033fa0 0x561b9b0b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56192/0 (socket says 172.21.15.7:56192) 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 -- 172.21.15.7:0/3782076457 learned_addr learned my addr 172.21.15.7:0/3782076457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa23cf00700 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b030170 0x561b9b0567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa23d701700 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b00a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 -- 172.21.15.7:0/3782076457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b030170 msgr2=0x561b9b0567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b030170 0x561b9b0567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 -- 172.21.15.7:0/3782076457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 msgr2=0x561b9b00a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b00a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 -- 172.21.15.7:0/3782076457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b9ad9bb10 con 0x561b9b033fa0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa237fff700 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b033fa0 0x561b9b0b7140 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa2380b0d30 tx=0x7fa22c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f753c787dd561525 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa216ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa216ffd700 1 -- 172.21.15.7:0/3782076457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0224b0 con 0x561b9b033fa0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa216ffd700 1 -- 172.21.15.7:0/3782076457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa22c022610 con 0x561b9b033fa0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa248464b80 1 -- 172.21.15.7:0/3782076457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b9b000490 con 0x561b9b033fa0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.611+0000 7fa216ffd700 1 -- 172.21.15.7:0/3782076457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa22c0229e0 con 0x561b9b033fa0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa248464b80 10 client.7884 fetch_fsmap learned FSMap version 81 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa248464b80 1 -- 172.21.15.7:0/3782076457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x561b9adf3c80 con 0x561b9b033fa0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa216ffd700 1 -- 172.21.15.7:0/3782076457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561b9b000490 con 0x561b9b033fa0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa216ffd700 1 -- 172.21.15.7:0/3782076457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 81) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa22c022b40 con 0x561b9b033fa0 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa248464b80 10 client.7884 fetch_fsmap finished waiting for FSMap version 81 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa248464b80 10 client.7884 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa248464b80 20 client.7884 populate_metadata read hostname 'smithi007' 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa248464b80 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] conn(0x561b9b037370 0x561b9b037740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa248464b80 4 client.7884 mds_command: new command op to 7746 tid=0[{"prefix": "status"}] 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa248464b80 1 -- 172.21.15.7:0/3782076457 --> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561b9aefe910 con 0x561b9b037370 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa23cf00700 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] conn(0x561b9b037370 0x561b9b037740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa216ffd700 10 client.7884 ms_handle_connect on v2:172.21.15.7:6832/2285850291 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.612+0000 7fa23cf00700 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] conn(0x561b9b037370 0x561b9b037740 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9738d614cbfd07bb server_cookie=fa337a862a981129 in_seq=0 out_seq=0 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa216ffd700 1 -- 172.21.15.7:0/3782076457 <== mds.0 v2:172.21.15.7:6832/2285850291 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561b9aefe910 con 0x561b9b037370 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa216ffd700 10 client.7884 handle_command_reply: tid=0 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 client.7884 shutdown 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 -- 172.21.15.7:0/3782076457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b033fa0 msgr2=0x561b9b0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b033fa0 0x561b9b0b7140 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa2380b0d30 tx=0x7fa22c008040 comp rx=0 tx=0).stop 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 -- 172.21.15.7:0/3782076457 shutdown_connections 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x561b9b0348b0 0x561b9b00a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.7:6832/2285850291,v1:172.21.15.7:6833/2285850291] conn(0x561b9b037370 0x561b9b037740 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x561b9b030170 0x561b9b0567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 --2- 172.21.15.7:0/3782076457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b9b033fa0 0x561b9b0b7140 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 -- 172.21.15.7:0/3782076457 >> 172.21.15.7:0/3782076457 conn(0x561b9b00a440 msgr2=0x561b9b04b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa216ffd700 0 client.7884 ms_handle_reset on v2:172.21.15.7:6832/2285850291 2024-01-11T07:16:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 -- 172.21.15.7:0/3782076457 shutdown_connections 2024-01-11T07:16:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 1 -- 172.21.15.7:0/3782076457 wait complete. 2024-01-11T07:16:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa248464b80 20 client.7884 trim_cache size 0 max 16384 2024-01-11T07:16:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:16:42.619 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:16:42.619 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7746, 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1157142960000002, 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 81, 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 168, 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 168, 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 12.073274541 2024-01-11T07:16:42.620 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:16:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa23f164700 1 -- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa224066f30 msgr2=0x7fa2240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa23f164700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa224066f30 0x7fa2240693e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa2280015a0 tx=0x7fa228001e50 comp rx=0 tx=0).stop 2024-01-11T07:16:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.613+0000 7fa23f164700 1 -- 172.21.15.7:0/1210546864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2380c29b0 msgr2=0x7fa2380c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2380c29b0 0x7fa2380c2e50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa22000bb40 tx=0x7fa230015000 comp rx=0 tx=0).stop 2024-01-11T07:16:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 -- 172.21.15.7:0/1210546864 shutdown_connections 2024-01-11T07:16:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa224066f30 0x7fa2240693e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2380c29b0 0x7fa2380c2e50 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa220004d50 0x7fa238158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 --2- 172.21.15.7:0/1210546864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa2380b0730 0x7fa238149b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 -- 172.21.15.7:0/1210546864 >> 172.21.15.7:0/1210546864 conn(0x7fa2380680e0 msgr2=0x7fa238156f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 -- 172.21.15.7:0/1210546864 shutdown_connections 2024-01-11T07:16:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.614+0000 7fa23f164700 1 -- 172.21.15.7:0/1210546864 wait complete. 2024-01-11T07:16:42.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:16:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.759+0000 7fe501f0e700 1 Processor -- start 2024-01-11T07:16:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.759+0000 7fe501f0e700 1 -- start start 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe501f0e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc159c90 0x7fe4fc0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe501f0e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4fc15a060 0x7fe4fc0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe501f0e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0ce9b0 con 0x7fe4fc069320 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe501f0e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cf530 con 0x7fe4fc15a060 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe501f0e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cf9c0 con 0x7fe4fc159c90 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe4faffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4fc15a060 0x7fe4fc0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe4faffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4fc15a060 0x7fe4fc0b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56206/0 (socket says 172.21.15.7:56206) 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe4faffd700 1 -- 172.21.15.7:0/2875924841 learned_addr learned my addr 172.21.15.7:0/2875924841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe4fb7fe700 1 --2- 172.21.15.7:0/2875924841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc159c90 0x7fe4fc0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe4faffd700 1 -- 172.21.15.7:0/2875924841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc159c90 msgr2=0x7fe4fc0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe4faffd700 1 --2- 172.21.15.7:0/2875924841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc159c90 0x7fe4fc0b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe4faffd700 1 -- 172.21.15.7:0/2875924841 >> v1:172.21.15.113:6789/0 conn(0x7fe4fc069320 legacy=0x7fe4fc067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.760+0000 7fe4faffd700 1 -- 172.21.15.7:0/2875924841 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc0cec00 con 0x7fe4fc15a060 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe4fb7fe700 1 --2- 172.21.15.7:0/2875924841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc159c90 0x7fe4fc0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe4faffd700 1 --2- 172.21.15.7:0/2875924841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc0b0430 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00099b0 tx=0x7fe4f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c33efce1abb104f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe4fa7fc700 1 -- 172.21.15.7:0/2875924841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000b860 con 0x7fe4fc15a060 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe4fa7fc700 1 -- 172.21.15.7:0/2875924841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f000b9c0 con 0x7fe4fc15a060 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe4fa7fc700 1 -- 172.21.15.7:0/2875924841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000bd60 con 0x7fe4fc15a060 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe501f0e700 1 -- 172.21.15.7:0/2875924841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 msgr2=0x7fe4fc0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe501f0e700 1 --2- 172.21.15.7:0/2875924841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc0b0430 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00099b0 tx=0x7fe4f0009250 comp rx=0 tx=0).stop 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe501f0e700 1 -- 172.21.15.7:0/2875924841 shutdown_connections 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe501f0e700 1 --2- 172.21.15.7:0/2875924841 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc159c90 0x7fe4fc0b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe501f0e700 1 --2- 172.21.15.7:0/2875924841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc0b0430 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.761+0000 7fe501f0e700 1 -- 172.21.15.7:0/2875924841 >> 172.21.15.7:0/2875924841 conn(0x7fe4fc071100 msgr2=0x7fe4fc06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.762+0000 7fe501f0e700 1 -- 172.21.15.7:0/2875924841 shutdown_connections 2024-01-11T07:16:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.762+0000 7fe501f0e700 1 -- 172.21.15.7:0/2875924841 wait complete. 2024-01-11T07:16:42.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.763+0000 7fe501f0e700 1 Processor -- start 2024-01-11T07:16:42.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.763+0000 7fe501f0e700 1 -- start start 2024-01-11T07:16:42.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe501f0e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc069320 0x7fe4fc0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe501f0e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4fc159c90 0x7fe4fc147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe501f0e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc1475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe501f0e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0aeca0 con 0x7fe4fc15a060 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe501f0e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0aef80 con 0x7fe4fc159c90 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe501f0e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0cde90 con 0x7fe4fc069320 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc1475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc1475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56214/0 (socket says 172.21.15.7:56214) 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 -- 172.21.15.7:0/150502991 learned_addr learned my addr 172.21.15.7:0/150502991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fb7fe700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc069320 0x7fe4fc0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4faffd700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4fc159c90 0x7fe4fc147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 -- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc069320 msgr2=0x7fe4fc0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc069320 0x7fe4fc0cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 -- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4fc159c90 msgr2=0x7fe4fc147090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4fc159c90 0x7fe4fc147090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 -- 172.21.15.7:0/150502991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00094f0 con 0x7fe4fc15a060 2024-01-11T07:16:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4fbfff700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc1475d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c810 tx=0x7fe4f4001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.764+0000 7fe4e37fe700 1 -- 172.21.15.7:0/150502991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4012630 con 0x7fe4fc15a060 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.765+0000 7fe4e37fe700 1 -- 172.21.15.7:0/150502991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f40127b0 con 0x7fe4fc15a060 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.765+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc0ce170 con 0x7fe4fc15a060 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.765+0000 7fe4e37fe700 1 -- 172.21.15.7:0/150502991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4012b70 con 0x7fe4fc15a060 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.765+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc0cb7f0 con 0x7fe4fc15a060 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.766+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc0af0f0 con 0x7fe4fc15a060 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.766+0000 7fe4e37fe700 1 -- 172.21.15.7:0/150502991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe4f4003030 con 0x7fe4fc15a060 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.766+0000 7fe4e37fe700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4e8066f60 0x7fe4e8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.766+0000 7fe4e37fe700 1 -- 172.21.15.7:0/150502991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe4f4084370 con 0x7fe4fc15a060 2024-01-11T07:16:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.767+0000 7fe4fb7fe700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4e8066f60 0x7fe4e8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:42.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.767+0000 7fe4fb7fe700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4e8066f60 0x7fe4e8069410 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec006f00 tx=0x7fe4ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:42.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:42.771+0000 7fe4e37fe700 1 -- 172.21.15.7:0/150502991 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4f40515e0 con 0x7fe4fc15a060 2024-01-11T07:16:43.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.027+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe4fc04c3c0 con 0x7fe4fc15a060 2024-01-11T07:16:43.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.027+0000 7fe4e37fe700 1 -- 172.21.15.7:0/150502991 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe4f4054c00 con 0x7fe4fc15a060 2024-01-11T07:16:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4e8066f60 msgr2=0x7fe4e8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4e8066f60 0x7fe4e8069410 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec006f00 tx=0x7fe4ec008040 comp rx=0 tx=0).stop 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 msgr2=0x7fe4fc1475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc1475d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c810 tx=0x7fe4f4001650 comp rx=0 tx=0).stop 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 shutdown_connections 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4e8066f60 0x7fe4e8069410 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4fc15a060 0x7fe4fc1475d0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4fc069320 0x7fe4fc0cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 --2- 172.21.15.7:0/150502991 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4fc159c90 0x7fe4fc147090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 >> 172.21.15.7:0/150502991 conn(0x7fe4fc071100 msgr2=0x7fe4fc1554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:43.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 shutdown_connections 2024-01-11T07:16:43.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.030+0000 7fe501f0e700 1 -- 172.21.15.7:0/150502991 wait complete. 2024-01-11T07:16:43.043 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:16:43.043 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:16:43.043 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:16:43.043 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:16:43.043 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:16:43.043 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:16:43.043 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:16:43.043 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:16:43.044 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:16:43.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:43.044 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:43.064 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41815sec preferred_lft 41815sec 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:16:43.065 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:16:43.066 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:43.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:43.067 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:16:43.134 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:16:43.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:43.136 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:16:43.201 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:16:43.201 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:16:43.201 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:16:43.201 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:16:43.201 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:16:43.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:43.201 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:16:43.202 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:16:43.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:16:43.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:16:43.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:16:43.202 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:16:43.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:43.324 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:16:43.344 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:16:43.344 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:16:43.344 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:16:43.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:43.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:16:43.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:16:43.419 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:16:43.482 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:16:43.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:16:43.482 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:16:43.543 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:16:43.543 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:16:43.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:16:43.637 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:16:43.637 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:16:43.637 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:16:43.647 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:16:43.647 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:16:43.647 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:16:43.647 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:16:43.647 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:16:43.665 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:16:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:16:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:16:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:16:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:16:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:16:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:16:43.667 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:16:43.668 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:16:43.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:43.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:16:43.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:16:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.883+0000 7f9c3d71d700 1 Processor -- start 2024-01-11T07:16:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.883+0000 7f9c3d71d700 1 -- start start 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c3d71d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c38159d80 0x7f9c3806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c3d71d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c3806e170 0x7f9c3815da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c3d71d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3815dfa0 0x7f9c3815e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c3d71d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3806e610 con 0x7f9c3806e170 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c3d71d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c380d06b0 con 0x7f9c38159d80 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c3d71d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380d0220 con 0x7f9c3815dfa0 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c367fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c3806e170 0x7f9c3815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c367fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c3806e170 0x7f9c3815da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56224/0 (socket says 172.21.15.7:56224) 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c367fc700 1 -- 172.21.15.7:0/154560845 learned_addr learned my addr 172.21.15.7:0/154560845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c36ffd700 1 --2- 172.21.15.7:0/154560845 >> v2:172.21.15.113:3300/0 conn(0x7f9c38159d80 0x7f9c3806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c377fe700 1 --2- 172.21.15.7:0/154560845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3815dfa0 0x7f9c3815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:43.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c367fc700 1 -- 172.21.15.7:0/154560845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3815dfa0 msgr2=0x7f9c3815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c367fc700 1 --2- 172.21.15.7:0/154560845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3815dfa0 0x7f9c3815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c367fc700 1 -- 172.21.15.7:0/154560845 >> v2:172.21.15.113:3300/0 conn(0x7f9c38159d80 msgr2=0x7f9c3806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c367fc700 1 --2- 172.21.15.7:0/154560845 >> v2:172.21.15.113:3300/0 conn(0x7f9c38159d80 0x7f9c3806dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c367fc700 1 -- 172.21.15.7:0/154560845 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c380cf610 con 0x7f9c3806e170 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c377fe700 1 --2- 172.21.15.7:0/154560845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3815dfa0 0x7f9c3815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.884+0000 7f9c36ffd700 1 --2- 172.21.15.7:0/154560845 >> v2:172.21.15.113:3300/0 conn(0x7f9c38159d80 0x7f9c3806dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c367fc700 1 --2- 172.21.15.7:0/154560845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3806e170 0x7f9c3815da60 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9c38063ce0 tx=0x7f9c2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da67dfed49a0b3ac server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c35ffb700 1 -- 172.21.15.7:0/154560845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00b860 con 0x7f9c3806e170 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c35ffb700 1 -- 172.21.15.7:0/154560845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c00b9c0 con 0x7f9c3806e170 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c35ffb700 1 -- 172.21.15.7:0/154560845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00bd60 con 0x7f9c3806e170 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c3d71d700 1 -- 172.21.15.7:0/154560845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3806e170 msgr2=0x7f9c3815da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/154560845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3806e170 0x7f9c3815da60 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9c38063ce0 tx=0x7f9c2c009250 comp rx=0 tx=0).stop 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c3d71d700 1 -- 172.21.15.7:0/154560845 shutdown_connections 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/154560845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3815dfa0 0x7f9c3815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/154560845 >> v2:172.21.15.113:3300/0 conn(0x7f9c38159d80 0x7f9c3806dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/154560845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3806e170 0x7f9c3815da60 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c3d71d700 1 -- 172.21.15.7:0/154560845 >> 172.21.15.7:0/154560845 conn(0x7f9c38071130 msgr2=0x7f9c380b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.885+0000 7f9c3d71d700 1 -- 172.21.15.7:0/154560845 shutdown_connections 2024-01-11T07:16:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.886+0000 7f9c3d71d700 1 -- 172.21.15.7:0/154560845 wait complete. 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.887+0000 7f9c3d71d700 1 Processor -- start 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.887+0000 7f9c3d71d700 1 -- start start 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c3d71d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3806e170 0x7f9c38155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c3d71d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c38159d80 0x7f9c3816a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c3d71d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3815dfa0 0x7f9c380b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c3d71d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c38155850 con 0x7f9c3815dfa0 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c3d71d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380c6890 con 0x7f9c38159d80 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c3d71d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3816ae90 con 0x7f9c3806e170 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3815dfa0 0x7f9c380b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3815dfa0 0x7f9c380b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56238/0 (socket says 172.21.15.7:56238) 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 -- 172.21.15.7:0/3904438011 learned_addr learned my addr 172.21.15.7:0/3904438011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c36ffd700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3806e170 0x7f9c38155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c367fc700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c38159d80 0x7f9c3816a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 -- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3806e170 msgr2=0x7f9c38155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3806e170 0x7f9c38155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 -- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c38159d80 msgr2=0x7f9c3816a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c38159d80 0x7f9c3816a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 -- 172.21.15.7:0/3904438011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0094f0 con 0x7f9c3815dfa0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.888+0000 7f9c377fe700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3815dfa0 0x7f9c380b1680 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000cef0 tx=0x7f9c30007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.889+0000 7f9c1effd700 1 -- 172.21.15.7:0/3904438011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000ad20 con 0x7f9c3815dfa0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.889+0000 7f9c1effd700 1 -- 172.21.15.7:0/3904438011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c30016420 con 0x7f9c3815dfa0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.889+0000 7f9c1effd700 1 -- 172.21.15.7:0/3904438011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000aea0 con 0x7f9c3815dfa0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.889+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c380c59a0 con 0x7f9c3815dfa0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.889+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c380c5f60 con 0x7f9c3815dfa0 2024-01-11T07:16:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.890+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c380cbc20 con 0x7f9c3815dfa0 2024-01-11T07:16:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.890+0000 7f9c1effd700 1 -- 172.21.15.7:0/3904438011 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9c30016a70 con 0x7f9c3815dfa0 2024-01-11T07:16:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.891+0000 7f9c1effd700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c24066f30 0x7f9c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.891+0000 7f9c1effd700 1 -- 172.21.15.7:0/3904438011 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9c30012070 con 0x7f9c3815dfa0 2024-01-11T07:16:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.891+0000 7f9c36ffd700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c24066f30 0x7f9c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.891+0000 7f9c36ffd700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c24066f30 0x7f9c240693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c380b24b0 tx=0x7f9c28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:43.895+0000 7f9c1effd700 1 -- 172.21.15.7:0/3904438011 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c3004e5a0 con 0x7f9c3815dfa0 2024-01-11T07:16:44.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.155+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c38089080 con 0x7f9c3815dfa0 2024-01-11T07:16:44.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.155+0000 7f9c1effd700 1 -- 172.21.15.7:0/3904438011 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9c380cbc20 con 0x7f9c3815dfa0 2024-01-11T07:16:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.157+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c24066f30 msgr2=0x7f9c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.157+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c24066f30 0x7f9c240693e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c380b24b0 tx=0x7f9c28008040 comp rx=0 tx=0).stop 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3815dfa0 msgr2=0x7f9c380b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3815dfa0 0x7f9c380b1680 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000cef0 tx=0x7f9c30007860 comp rx=0 tx=0).stop 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 shutdown_connections 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9c24066f30 0x7f9c240693e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c3815dfa0 0x7f9c380b1680 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c3806e170 0x7f9c38155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 --2- 172.21.15.7:0/3904438011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c38159d80 0x7f9c3816a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 >> 172.21.15.7:0/3904438011 conn(0x7f9c38071130 msgr2=0x7f9c380b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:44.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 shutdown_connections 2024-01-11T07:16:44.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.158+0000 7f9c3d71d700 1 -- 172.21.15.7:0/3904438011 wait complete. 2024-01-11T07:16:44.171 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:16:44.171 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:16:44.171 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:16:44.171 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:16:44.172 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:16:44.172 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:16:44.172 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:16:44.172 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:16:44.172 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:16:44.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:44.172 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:16:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:16:44.194 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:16:44.194 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:16:44.194 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:44.194 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:16:44.194 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:16:44.194 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 41856sec preferred_lft 41856sec 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:16:44.195 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:16:44.196 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:16:44.197 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:16:44.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:44.197 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:16:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:16:44.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:44.268 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:16:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:16:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:16:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:16:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:16:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:16:44.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:44.330 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:16:44.330 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:16:44.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:16:44.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:16:44.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:16:44.330 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:16:44.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:44.441 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:16:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:16:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:16:44.458 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:16:44.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:44.459 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:16:44.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:16:44.526 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:16:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:16:44.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:16:44.584 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:16:44.641 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:16:44.642 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:16:44.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:16:44.723 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:16:44.723 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:16:44.723 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:16:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:16:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:16:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:16:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:16:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:16:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:16:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:16:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:16:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:16:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:16:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:16:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:16:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:16:44.752 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:16:44.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:16:44.752 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:16:44.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:16:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.961+0000 7ff818ac3700 1 Processor -- start 2024-01-11T07:16:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.961+0000 7ff818ac3700 1 -- start start 2024-01-11T07:16:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff818ac3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff814159c90 0x7ff8140b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff818ac3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff814069320 0x7ff8140b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff818ac3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8140ce9b0 con 0x7ff81415a060 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff818ac3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8140cf530 con 0x7ff814069320 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff818ac3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140cf9c0 con 0x7ff814159c90 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff811d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff81415a060 0x7ff8140b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57956/0 (socket says 172.21.15.7:57956) 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff811d9b700 1 -- 172.21.15.7:0/4286276636 learned_addr learned my addr 172.21.15.7:0/4286276636 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff812d9d700 1 --2- 172.21.15.7:0/4286276636 >> v2:172.21.15.113:3300/0 conn(0x7ff814069320 0x7ff8140b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff81259c700 1 --2- 172.21.15.7:0/4286276636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff814159c90 0x7ff8140b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff81159a700 1 -- 172.21.15.7:0/4286276636 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878063457 0 0) 0x7ff8140ce9b0 con 0x7ff81415a060 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff81159a700 1 -- 172.21.15.7:0/4286276636 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7fc003570 con 0x7ff81415a060 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff81259c700 1 -- 172.21.15.7:0/4286276636 >> v2:172.21.15.113:3300/0 conn(0x7ff814069320 msgr2=0x7ff8140b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff81259c700 1 --2- 172.21.15.7:0/4286276636 >> v2:172.21.15.113:3300/0 conn(0x7ff814069320 0x7ff8140b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff81259c700 1 -- 172.21.15.7:0/4286276636 >> v1:172.21.15.7:6789/0 conn(0x7ff81415a060 legacy=0x7ff8140b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff81259c700 1 -- 172.21.15.7:0/4286276636 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8140cec00 con 0x7ff814159c90 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.962+0000 7ff812d9d700 1 --2- 172.21.15.7:0/4286276636 >> v2:172.21.15.113:3300/0 conn(0x7ff814069320 0x7ff8140b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff81259c700 1 --2- 172.21.15.7:0/4286276636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff814159c90 0x7ff8140b1820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098e0 tx=0x7ff8040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767614fdaae1cd2e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff81159a700 1 -- 172.21.15.7:0/4286276636 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804011630 con 0x7ff814159c90 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff81159a700 1 -- 172.21.15.7:0/4286276636 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff804011790 con 0x7ff814159c90 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff81159a700 1 -- 172.21.15.7:0/4286276636 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804011b30 con 0x7ff814159c90 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff818ac3700 1 -- 172.21.15.7:0/4286276636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff814159c90 msgr2=0x7ff8140b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff818ac3700 1 --2- 172.21.15.7:0/4286276636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff814159c90 0x7ff8140b1820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098e0 tx=0x7ff8040091a0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff818ac3700 1 -- 172.21.15.7:0/4286276636 shutdown_connections 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff818ac3700 1 --2- 172.21.15.7:0/4286276636 >> v2:172.21.15.113:3300/0 conn(0x7ff814069320 0x7ff8140b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff818ac3700 1 --2- 172.21.15.7:0/4286276636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff814159c90 0x7ff8140b1820 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.963+0000 7ff818ac3700 1 -- 172.21.15.7:0/4286276636 >> 172.21.15.7:0/4286276636 conn(0x7ff814071100 msgr2=0x7ff81406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.964+0000 7ff818ac3700 1 -- 172.21.15.7:0/4286276636 shutdown_connections 2024-01-11T07:16:44.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.964+0000 7ff818ac3700 1 -- 172.21.15.7:0/4286276636 wait complete. 2024-01-11T07:16:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.965+0000 7ff818ac3700 1 Processor -- start 2024-01-11T07:16:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.965+0000 7ff818ac3700 1 -- start start 2024-01-11T07:16:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.965+0000 7ff818ac3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff814069320 0x7ff814146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.965+0000 7ff818ac3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff814159c90 0x7ff8140ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.965+0000 7ff818ac3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff81415a060 0x7ff8140cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:44.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.965+0000 7ff818ac3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140af9e0 con 0x7ff814069320 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.965+0000 7ff818ac3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140cca20 con 0x7ff814159c90 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.965+0000 7ff818ac3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140ccd00 con 0x7ff81415a060 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff814069320 0x7ff814146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff814069320 0x7ff814146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56242/0 (socket says 172.21.15.7:56242) 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 -- 172.21.15.7:0/3161392287 learned_addr learned my addr 172.21.15.7:0/3161392287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff811d9b700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff814159c90 0x7ff8140ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff812d9d700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff81415a060 0x7ff8140cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 -- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff81415a060 msgr2=0x7ff8140cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff81415a060 0x7ff8140cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 -- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff814159c90 msgr2=0x7ff8140ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff814159c90 0x7ff8140ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 -- 172.21.15.7:0/3161392287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c009a20 con 0x7ff814069320 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff81259c700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff814069320 0x7ff814146360 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff8140cfb00 tx=0x7ff804017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff8027fc700 1 -- 172.21.15.7:0/3161392287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804011b10 con 0x7ff814069320 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff8027fc700 1 -- 172.21.15.7:0/3161392287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8040233f0 con 0x7ff814069320 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff804009420 con 0x7ff814069320 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff8027fc700 1 -- 172.21.15.7:0/3161392287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804023770 con 0x7ff814069320 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.966+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8140cc6d0 con 0x7ff814069320 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.967+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81404c3c0 con 0x7ff814069320 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.968+0000 7ff8027fc700 1 -- 172.21.15.7:0/3161392287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff80402c3b0 con 0x7ff814069320 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.968+0000 7ff8027fc700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7f4066f30 0x7ff7f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.968+0000 7ff8027fc700 1 -- 172.21.15.7:0/3161392287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff8040859d0 con 0x7ff814069320 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.968+0000 7ff811d9b700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7f4066f30 0x7ff7f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:44.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.969+0000 7ff811d9b700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7f4066f30 0x7ff7f40693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff81414a170 tx=0x7ff808009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:44.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:44.973+0000 7ff8027fc700 1 -- 172.21.15.7:0/3161392287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff804052cc0 con 0x7ff814069320 2024-01-11T07:16:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.223+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8140cd220 con 0x7ff814069320 2024-01-11T07:16:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.224+0000 7ff8027fc700 1 -- 172.21.15.7:0/3161392287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ff8140cd220 con 0x7ff814069320 2024-01-11T07:16:45.224 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:16:45.224 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4097846800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4097846800},{"type":"v1","addr":"172.21.15.113:6836","nonce":4097846800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1898628721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1898628721},{"type":"v1","addr":"172.21.15.113:6837","nonce":1898628721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3371959241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3371959241},{"type":"v1","addr":"172.21.15.7:6835","nonce":3371959241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:16:35.923785+0000","modified":"2024-01-11T07:16:39.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/2285850291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2285850291},{"type":"v1","addr":"172.21.15.7:6833","nonce":2285850291}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-11T07:16:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7f4066f30 msgr2=0x7ff7f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7f4066f30 0x7ff7f40693e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff81414a170 tx=0x7ff808009040 comp rx=0 tx=0).stop 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff814069320 msgr2=0x7ff814146360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff814069320 0x7ff814146360 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff8140cfb00 tx=0x7ff804017000 comp rx=0 tx=0).stop 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 shutdown_connections 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7f4066f30 0x7ff7f40693e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff81415a060 0x7ff8140cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff814069320 0x7ff814146360 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 --2- 172.21.15.7:0/3161392287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff814159c90 0x7ff8140ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 >> 172.21.15.7:0/3161392287 conn(0x7ff814071100 msgr2=0x7ff8140b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.226+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 shutdown_connections 2024-01-11T07:16:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.227+0000 7ff818ac3700 1 -- 172.21.15.7:0/3161392287 wait complete. 2024-01-11T07:16:45.228 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2024-01-11T07:16:45.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-01-11T07:16:45.393 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":7746,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8847596380000002,"mdsmap_epoch":81,"osdmap_epoch":168,"osdmap_epoch_barrier":168,"uptime":14.842322543} 2024-01-11T07:16:45.393 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 7746, "mdsmap_epoch": 81, "osdmap_epoch": 168, "osdmap_epoch_barrier": 168, "rank_uptime": 8.884759638, "state": "up:active", "uptime": 14.842322543, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:16:45.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-11T07:16:45.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.530+0000 7f02b106e700 1 Processor -- start 2024-01-11T07:16:45.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.530+0000 7f02b106e700 1 -- start start 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.530+0000 7f02b106e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac157990 0x7f02ac06d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02b106e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ac157d60 0x7f02ac0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02b106e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac0cd260 con 0x7f02ac15c3b0 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02b106e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac0cc690 con 0x7f02ac157d60 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02b106e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0cc930 con 0x7f02ac157990 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aa59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ac157d60 0x7f02ac0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aa59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ac157d60 0x7f02ac0ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56254/0 (socket says 172.21.15.7:56254) 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aa59c700 1 -- 172.21.15.7:0/3523345808 learned_addr learned my addr 172.21.15.7:0/3523345808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aad9d700 1 --2- 172.21.15.7:0/3523345808 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac157990 0x7f02ac06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:45.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aa59c700 1 -- 172.21.15.7:0/3523345808 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac157990 msgr2=0x7f02ac06d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aa59c700 1 --2- 172.21.15.7:0/3523345808 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac157990 0x7f02ac06d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aa59c700 1 -- 172.21.15.7:0/3523345808 >> v1:172.21.15.113:6789/0 conn(0x7f02ac15c3b0 legacy=0x7f02ac0cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aa59c700 1 -- 172.21.15.7:0/3523345808 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac0ccc10 con 0x7f02ac157d60 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02aa59c700 1 --2- 172.21.15.7:0/3523345808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac0ca4c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f02a00099b0 tx=0x7f02a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b852bed99f978347 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02a9d9b700 1 -- 172.21.15.7:0/3523345808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0009ea0 con 0x7f02ac157d60 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02a9d9b700 1 -- 172.21.15.7:0/3523345808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a000b860 con 0x7f02ac157d60 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.531+0000 7f02a9d9b700 1 -- 172.21.15.7:0/3523345808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a000bc00 con 0x7f02ac157d60 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.532+0000 7f02b106e700 1 -- 172.21.15.7:0/3523345808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 msgr2=0x7f02ac0ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.532+0000 7f02b106e700 1 --2- 172.21.15.7:0/3523345808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac0ca4c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f02a00099b0 tx=0x7f02a0009250 comp rx=0 tx=0).stop 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.532+0000 7f02b106e700 1 -- 172.21.15.7:0/3523345808 shutdown_connections 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.532+0000 7f02b106e700 1 --2- 172.21.15.7:0/3523345808 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac157990 0x7f02ac06d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.532+0000 7f02b106e700 1 --2- 172.21.15.7:0/3523345808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac0ca4c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.532+0000 7f02b106e700 1 -- 172.21.15.7:0/3523345808 >> 172.21.15.7:0/3523345808 conn(0x7f02ac063230 msgr2=0x7f02ac063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:45.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.532+0000 7f02b106e700 1 -- 172.21.15.7:0/3523345808 shutdown_connections 2024-01-11T07:16:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.532+0000 7f02b106e700 1 -- 172.21.15.7:0/3523345808 wait complete. 2024-01-11T07:16:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.534+0000 7f02b106e700 1 Processor -- start 2024-01-11T07:16:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.534+0000 7f02b106e700 1 -- start start 2024-01-11T07:16:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02b106e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac157990 0x7f02ac0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02b106e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac1386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02b106e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac15c3b0 0x7f02ac14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02b106e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac167dd0 con 0x7f02ac157d60 2024-01-11T07:16:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02b106e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac138f70 con 0x7f02ac157990 2024-01-11T07:16:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02b106e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac139250 con 0x7f02ac15c3b0 2024-01-11T07:16:45.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac1386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac1386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56258/0 (socket says 172.21.15.7:56258) 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aa59c700 1 -- 172.21.15.7:0/3787609525 learned_addr learned my addr 172.21.15.7:0/3787609525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02ab59e700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac15c3b0 0x7f02ac14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aad9d700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac157990 0x7f02ac0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aa59c700 1 -- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac15c3b0 msgr2=0x7f02ac14f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aa59c700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac15c3b0 0x7f02ac14f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aa59c700 1 -- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac157990 msgr2=0x7f02ac0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aa59c700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac157990 0x7f02ac0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aa59c700 1 -- 172.21.15.7:0/3787609525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a00094f0 con 0x7f02ac157d60 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.535+0000 7f02aad9d700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac157990 0x7f02ac0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.536+0000 7f02aa59c700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac1386f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0c58c0 tx=0x7f02a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.536+0000 7f0292ffd700 1 -- 172.21.15.7:0/3787609525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a000bbe0 con 0x7f02ac157d60 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.536+0000 7f0292ffd700 1 -- 172.21.15.7:0/3787609525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0011420 con 0x7f02ac157d60 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.536+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac139530 con 0x7f02ac157d60 2024-01-11T07:16:45.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.536+0000 7f0292ffd700 1 -- 172.21.15.7:0/3787609525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a00117e0 con 0x7f02ac157d60 2024-01-11T07:16:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.536+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac0b1b40 con 0x7f02ac157d60 2024-01-11T07:16:45.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.537+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac0b31e0 con 0x7f02ac157d60 2024-01-11T07:16:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.537+0000 7f0292ffd700 1 -- 172.21.15.7:0/3787609525 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f02a0011b90 con 0x7f02ac157d60 2024-01-11T07:16:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.538+0000 7f0292ffd700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0298066f30 0x7f02980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:16:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.538+0000 7f0292ffd700 1 -- 172.21.15.7:0/3787609525 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f02a0085d30 con 0x7f02ac157d60 2024-01-11T07:16:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.538+0000 7f02aad9d700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0298066f30 0x7f02980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:16:45.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.539+0000 7f02aad9d700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0298066f30 0x7f02980693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0c9730 tx=0x7f029c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:16:45.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.543+0000 7f0292ffd700 1 -- 172.21.15.7:0/3787609525 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02a0052f00 con 0x7f02ac157d60 2024-01-11T07:16:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:45.746+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f02ac04c3c0 con 0x7f02ac157d60 2024-01-11T07:16:46.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.610+0000 7f0292ffd700 1 -- 172.21.15.7:0/3787609525 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v169) v1 ==== 181+0+0 (secure 0 0 0) 0x7f02a0056520 con 0x7f02ac157d60 2024-01-11T07:16:46.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0298066f30 msgr2=0x7f02980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0298066f30 0x7f02980693e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0c9730 tx=0x7f029c008040 comp rx=0 tx=0).stop 2024-01-11T07:16:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 msgr2=0x7f02ac1386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:16:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac1386f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0c58c0 tx=0x7f02a0017000 comp rx=0 tx=0).stop 2024-01-11T07:16:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 shutdown_connections 2024-01-11T07:16:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0298066f30 0x7f02980693e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ac15c3b0 0x7f02ac14f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ac157990 0x7f02ac0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 --2- 172.21.15.7:0/3787609525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac157d60 0x7f02ac1386f0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:16:46.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.612+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 >> 172.21.15.7:0/3787609525 conn(0x7f02ac063230 msgr2=0x7f02ac151050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:16:46.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.613+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 shutdown_connections 2024-01-11T07:16:46.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:16:46.613+0000 7f02b106e700 1 -- 172.21.15.7:0/3787609525 wait complete. 2024-01-11T07:16:46.615 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-11T07:16:46.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-11T07:16:46.785 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-11T07:16:46.785 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-11T07:16:46.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-01-11T07:16:46.927 INFO:teuthology.orchestra.run.smithi007.stdout:{"mon_tick_interval":"5"} 2024-01-11T07:16:46.927 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-01-11T07:16:46.927 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-01-11T07:16:46.927 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-01-11T07:16:46.927 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> import datetime 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> # Write some buffered data through before going full, all should be well 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> print("writing some data through which we expect to succeed") 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> bytes = 0 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-01-11T07:16:46.928 DEBUG:teuthology.orchestra.run.smithi007:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> # buffered IO, the error should always only appear from fsync and not 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> # from write 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> full = False 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> for n in range(0, int(32 * 1.1)): 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> print("wrote bytes via buffered write, moving on to fsync") 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> raise 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-11T07:16:46.929 DEBUG:teuthology.orchestra.run.smithi007:> full = True 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f) 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> print("fsync'"'"'ed successfully") 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> full = True 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> else: 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> if n > 32 * 0.9: 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> # Be cautious in the last region where we expect to hit 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-01-11T07:16:46.930 DEBUG:teuthology.orchestra.run.smithi007:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(15.0) 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> if not full: 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> # close() should not raise an error because we already caught it in 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> # since then because all IOs got cancelled on the full flag. 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> print("calling close") 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> os.close(f) 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> print("close() did not raise error") 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-01-11T07:16:46.931 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-01-11T07:17:01.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:17:01.814 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:17:31.907 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:17:31.911 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:17:50.783 INFO:teuthology.orchestra.run.smithi007.stdout:writing some data through which we expect to succeed 2024-01-11T07:17:50.783 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-01-11T07:17:50.783 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.783 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.783 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 1.00 MB 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 2.00 MB 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 3.00 MB 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 4.00 MB 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 5.00 MB 2024-01-11T07:17:50.784 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 6.00 MB 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 7.00 MB 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 8.00 MB 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 9.00 MB 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.785 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 10.00 MB 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 11.00 MB 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 12.00 MB 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 13.00 MB 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 14.00 MB 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.786 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 15.00 MB 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 16.00 MB 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 17.00 MB 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 18.00 MB 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 19.00 MB 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.787 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 20.00 MB 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 21.00 MB 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 22.00 MB 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 23.00 MB 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.788 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 24.00 MB 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 25.00 MB 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 26.00 MB 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 27.00 MB 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 28.00 MB 2024-01-11T07:17:50.789 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 29.00 MB 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 30.00 MB 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 31.00 MB 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.790 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 32.00 MB 2024-01-11T07:17:50.791 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:17:50.791 INFO:teuthology.orchestra.run.smithi007.stdout:wrote bytes via buffered write, moving on to fsync 2024-01-11T07:17:50.791 INFO:teuthology.orchestra.run.smithi007.stdout:fsync'ed successfully 2024-01-11T07:17:50.791 INFO:teuthology.orchestra.run.smithi007.stdout:Not full yet after 33.00 MB 2024-01-11T07:17:50.791 INFO:teuthology.orchestra.run.smithi007.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-01-11T07:17:50.791 INFO:teuthology.orchestra.run.smithi007.stdout:Reached fullness after 33.00 MB 2024-01-11T07:17:50.791 INFO:teuthology.orchestra.run.smithi007.stdout:calling close 2024-01-11T07:17:50.791 INFO:teuthology.orchestra.run.smithi007.stdout:close() did not raise error 2024-01-11T07:17:50.888 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:17:50.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:17:50.936 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:17:50.987 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:17:50.988 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:17:50.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:17:50.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:17:51.045 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:17:51.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:17:51.045 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:17:51.066 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:17:51.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:17:51.067 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:17:51.112 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:17:51.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:17:51.113 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:17:51.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:17:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.271+0000 7f02eb476700 1 Processor -- start 2024-01-11T07:17:51.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.271+0000 7f02eb476700 1 -- start start 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.272+0000 7f02eb476700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02e4159c90 0x7f02e40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.272+0000 7f02eb476700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f02e415a060 0x7f02e40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.272+0000 7f02eb476700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e40ce9b0 con 0x7f02e4069320 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.272+0000 7f02eb476700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e40cf530 con 0x7f02e4159c90 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.272+0000 7f02eb476700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e40cf9c0 con 0x7f02e415a060 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.272+0000 7f02e9212700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02e4159c90 0x7f02e40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02e9212700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02e4159c90 0x7f02e40b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50280/0 (socket says 172.21.15.7:50280) 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02e9212700 1 -- 172.21.15.7:0/4050388677 learned_addr learned my addr 172.21.15.7:0/4050388677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02e8a11700 1 --2- 172.21.15.7:0/4050388677 >> v2:172.21.15.113:3300/0 conn(0x7f02e415a060 0x7f02e40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02e9212700 1 -- 172.21.15.7:0/4050388677 >> v2:172.21.15.113:3300/0 conn(0x7f02e415a060 msgr2=0x7f02e40b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:51.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02e9212700 1 --2- 172.21.15.7:0/4050388677 >> v2:172.21.15.113:3300/0 conn(0x7f02e415a060 0x7f02e40b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02e9212700 1 -- 172.21.15.7:0/4050388677 >> v1:172.21.15.113:6789/0 conn(0x7f02e4069320 legacy=0x7f02e4067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02e9212700 1 -- 172.21.15.7:0/4050388677 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e40cec00 con 0x7f02e4159c90 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02e9212700 1 --2- 172.21.15.7:0/4050388677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4159c90 0x7f02e40b1820 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f02e40699e0 tx=0x7f02d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf101f6ffac2378c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02dbfff700 1 -- 172.21.15.7:0/4050388677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d400b820 con 0x7f02e4159c90 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02dbfff700 1 -- 172.21.15.7:0/4050388677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d400b980 con 0x7f02e4159c90 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.273+0000 7f02dbfff700 1 -- 172.21.15.7:0/4050388677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d400bd20 con 0x7f02e4159c90 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.274+0000 7f02eb476700 1 -- 172.21.15.7:0/4050388677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4159c90 msgr2=0x7f02e40b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.274+0000 7f02eb476700 1 --2- 172.21.15.7:0/4050388677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4159c90 0x7f02e40b1820 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f02e40699e0 tx=0x7f02d4009250 comp rx=0 tx=0).stop 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.274+0000 7f02eb476700 1 -- 172.21.15.7:0/4050388677 shutdown_connections 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.274+0000 7f02eb476700 1 --2- 172.21.15.7:0/4050388677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4159c90 0x7f02e40b1820 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.274+0000 7f02eb476700 1 --2- 172.21.15.7:0/4050388677 >> v2:172.21.15.113:3300/0 conn(0x7f02e415a060 0x7f02e40b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.274+0000 7f02eb476700 1 -- 172.21.15.7:0/4050388677 >> 172.21.15.7:0/4050388677 conn(0x7f02e4071100 msgr2=0x7f02e406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:51.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.274+0000 7f02eb476700 1 -- 172.21.15.7:0/4050388677 shutdown_connections 2024-01-11T07:17:51.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.274+0000 7f02eb476700 1 -- 172.21.15.7:0/4050388677 wait complete. 2024-01-11T07:17:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.275+0000 7f02eb476700 1 Processor -- start 2024-01-11T07:17:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.275+0000 7f02eb476700 1 -- start start 2024-01-11T07:17:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02eb476700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4069320 0x7f02e40ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02eb476700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02e4159c90 0x7f02e40cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02eb476700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02e415a060 0x7f02e4155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02eb476700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40cacb0 con 0x7f02e4069320 2024-01-11T07:17:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02eb476700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40cd5a0 con 0x7f02e4159c90 2024-01-11T07:17:51.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02eb476700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40ca6f0 con 0x7f02e415a060 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02e9212700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4069320 0x7f02e40ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02e9212700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4069320 0x7f02e40ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50290/0 (socket says 172.21.15.7:50290) 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.276+0000 7f02e9212700 1 -- 172.21.15.7:0/3624590247 learned_addr learned my addr 172.21.15.7:0/3624590247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e8a11700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02e4159c90 0x7f02e40cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e9a13700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02e415a060 0x7f02e4155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e9212700 1 -- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02e415a060 msgr2=0x7f02e4155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e9212700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02e415a060 0x7f02e4155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e9212700 1 -- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02e4159c90 msgr2=0x7f02e40cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e9212700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02e4159c90 0x7f02e40cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e9212700 1 -- 172.21.15.7:0/3624590247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40094f0 con 0x7f02e4069320 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e8a11700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02e4159c90 0x7f02e40cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02e9212700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4069320 0x7f02e40ca060 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f02d40030c0 tx=0x7f02d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02d97fa700 1 -- 172.21.15.7:0/3624590247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4011e60 con 0x7f02e4069320 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02d97fa700 1 -- 172.21.15.7:0/3624590247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d4020070 con 0x7f02e4069320 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e40ca9d0 con 0x7f02e4069320 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02d97fa700 1 -- 172.21.15.7:0/3624590247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d401b6a0 con 0x7f02e4069320 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.277+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e40ae150 con 0x7f02e4069320 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.278+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e40adcd0 con 0x7f02e4069320 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.279+0000 7f02d97fa700 1 -- 172.21.15.7:0/3624590247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f02d4022030 con 0x7f02e4069320 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.279+0000 7f02d97fa700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02c8066f30 0x7f02c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.279+0000 7f02d97fa700 1 -- 172.21.15.7:0/3624590247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f02d4086500 con 0x7f02e4069320 2024-01-11T07:17:51.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.279+0000 7f02d97fa700 1 -- 172.21.15.7:0/3624590247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f02c806eab0 con 0x7f02e4069320 2024-01-11T07:17:51.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.279+0000 7f02e8a11700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02c8066f30 0x7f02c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.280+0000 7f02e8a11700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02c8066f30 0x7f02c80693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f02e0006fd0 tx=0x7f02e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:51.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.284+0000 7f02d97fa700 1 -- 172.21.15.7:0/3624590247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02e40adcd0 con 0x7f02e4069320 2024-01-11T07:17:51.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.535+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02e40adcd0 con 0x7f02e4069320 2024-01-11T07:17:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.535+0000 7f02d97fa700 1 -- 172.21.15.7:0/3624590247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f02e40adcd0 con 0x7f02e4069320 2024-01-11T07:17:51.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:17:51.536 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4097846800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4097846800},{"type":"v1","addr":"172.21.15.113:6836","nonce":4097846800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1898628721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1898628721},{"type":"v1","addr":"172.21.15.113:6837","nonce":1898628721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3371959241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3371959241},{"type":"v1","addr":"172.21.15.7:6835","nonce":3371959241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:16:35.923785+0000","modified":"2024-01-11T07:16:39.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/2285850291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2285850291},{"type":"v1","addr":"172.21.15.7:6833","nonce":2285850291}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-11T07:17:51.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02c8066f30 msgr2=0x7f02c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02c8066f30 0x7f02c80693e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f02e0006fd0 tx=0x7f02e0008040 comp rx=0 tx=0).stop 2024-01-11T07:17:51.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4069320 msgr2=0x7f02e40ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4069320 0x7f02e40ca060 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f02d40030c0 tx=0x7f02d4016000 comp rx=0 tx=0).stop 2024-01-11T07:17:51.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 shutdown_connections 2024-01-11T07:17:51.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02c8066f30 0x7f02c80693e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02e415a060 0x7f02e4155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e4069320 0x7f02e40ca060 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 --2- 172.21.15.7:0/3624590247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02e4159c90 0x7f02e40cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 >> 172.21.15.7:0/3624590247 conn(0x7f02e4071100 msgr2=0x7f02e40b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:51.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 shutdown_connections 2024-01-11T07:17:51.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.539+0000 7f02eb476700 1 -- 172.21.15.7:0/3624590247 wait complete. 2024-01-11T07:17:51.540 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2024-01-11T07:17:51.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:17:51.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.689+0000 7fe58d408700 1 Processor -- start 2024-01-11T07:17:51.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.689+0000 7fe58d408700 1 -- start start 2024-01-11T07:17:51.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.689+0000 7fe58d408700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe588159c90 0x7fe5880b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe58d408700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe58815a060 0x7fe5880b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe58d408700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880ce9b0 con 0x7fe588069320 2024-01-11T07:17:51.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe58d408700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5880cf530 con 0x7fe58815a060 2024-01-11T07:17:51.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe58d408700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5880cf9c0 con 0x7fe588159c90 2024-01-11T07:17:51.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe5867fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe58815a060 0x7fe5880b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe5867fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe58815a060 0x7fe5880b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50306/0 (socket says 172.21.15.7:50306) 2024-01-11T07:17:51.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe5867fc700 1 -- 172.21.15.7:0/4116148459 learned_addr learned my addr 172.21.15.7:0/4116148459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:51.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe586ffd700 1 --2- 172.21.15.7:0/4116148459 >> v2:172.21.15.113:3300/0 conn(0x7fe588159c90 0x7fe5880b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe5867fc700 1 -- 172.21.15.7:0/4116148459 >> v2:172.21.15.113:3300/0 conn(0x7fe588159c90 msgr2=0x7fe5880b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:51.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe5867fc700 1 --2- 172.21.15.7:0/4116148459 >> v2:172.21.15.113:3300/0 conn(0x7fe588159c90 0x7fe5880b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe5867fc700 1 -- 172.21.15.7:0/4116148459 >> v1:172.21.15.113:6789/0 conn(0x7fe588069320 legacy=0x7fe588067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe5867fc700 1 -- 172.21.15.7:0/4116148459 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5880cec00 con 0x7fe58815a060 2024-01-11T07:17:51.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.690+0000 7fe5867fc700 1 --2- 172.21.15.7:0/4116148459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5880b0430 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe58805f2e0 tx=0x7fe57c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5a62a3729da047 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:51.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe585ffb700 1 -- 172.21.15.7:0/4116148459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c009e70 con 0x7fe58815a060 2024-01-11T07:17:51.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe585ffb700 1 -- 172.21.15.7:0/4116148459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00b860 con 0x7fe58815a060 2024-01-11T07:17:51.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe585ffb700 1 -- 172.21.15.7:0/4116148459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00bc00 con 0x7fe58815a060 2024-01-11T07:17:51.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe58d408700 1 -- 172.21.15.7:0/4116148459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 msgr2=0x7fe5880b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe58d408700 1 --2- 172.21.15.7:0/4116148459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5880b0430 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe58805f2e0 tx=0x7fe57c009250 comp rx=0 tx=0).stop 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe58d408700 1 -- 172.21.15.7:0/4116148459 shutdown_connections 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe58d408700 1 --2- 172.21.15.7:0/4116148459 >> v2:172.21.15.113:3300/0 conn(0x7fe588159c90 0x7fe5880b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe58d408700 1 --2- 172.21.15.7:0/4116148459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5880b0430 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe58d408700 1 -- 172.21.15.7:0/4116148459 >> 172.21.15.7:0/4116148459 conn(0x7fe588071100 msgr2=0x7fe58806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe58d408700 1 -- 172.21.15.7:0/4116148459 shutdown_connections 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.691+0000 7fe58d408700 1 -- 172.21.15.7:0/4116148459 wait complete. 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.693+0000 7fe58d408700 1 Processor -- start 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.693+0000 7fe58d408700 1 -- start start 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.693+0000 7fe58d408700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe588069320 0x7fe588067860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.693+0000 7fe58d408700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe588159c90 0x7fe588154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.693+0000 7fe58d408700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5881549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.693+0000 7fe58d408700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588154f10 con 0x7fe58815a060 2024-01-11T07:17:51.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.693+0000 7fe58d408700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880c7cb0 con 0x7fe588159c90 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.693+0000 7fe58d408700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880c7f50 con 0x7fe588069320 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5881549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5881549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50312/0 (socket says 172.21.15.7:50312) 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 -- 172.21.15.7:0/2740290638 learned_addr learned my addr 172.21.15.7:0/2740290638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5867fc700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe588159c90 0x7fe588154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe586ffd700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe588069320 0x7fe588067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 -- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe588069320 msgr2=0x7fe588067860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe588069320 0x7fe588067860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 -- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe588159c90 msgr2=0x7fe588154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe588159c90 0x7fe588154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 -- 172.21.15.7:0/2740290638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0094f0 con 0x7fe58815a060 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe5877fe700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5881549d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c8e0 tx=0x7fe580001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe56effd700 1 -- 172.21.15.7:0/2740290638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580012630 con 0x7fe58815a060 2024-01-11T07:17:51.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe56effd700 1 -- 172.21.15.7:0/2740290638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe580012790 con 0x7fe58815a060 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5880c8230 con 0x7fe58815a060 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe56effd700 1 -- 172.21.15.7:0/2740290638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580012b50 con 0x7fe58815a060 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.694+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe588067da0 con 0x7fe58815a060 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.695+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5880c8880 con 0x7fe58815a060 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.696+0000 7fe56effd700 1 -- 172.21.15.7:0/2740290638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe58001a3a0 con 0x7fe58815a060 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.696+0000 7fe56effd700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe574066f30 0x7fe5740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.696+0000 7fe56effd700 1 -- 172.21.15.7:0/2740290638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fe580084760 con 0x7fe58815a060 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.697+0000 7fe56effd700 1 -- 172.21.15.7:0/2740290638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fe57406eab0 con 0x7fe58815a060 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.697+0000 7fe586ffd700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe574066f30 0x7fe5740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:51.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.697+0000 7fe586ffd700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe574066f30 0x7fe5740693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe578009db0 tx=0x7fe5780091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:51.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.701+0000 7fe56effd700 1 -- 172.21.15.7:0/2740290638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe580051910 con 0x7fe58815a060 2024-01-11T07:17:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.908+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5880c8a30 con 0x7fe58815a060 2024-01-11T07:17:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.909+0000 7fe56effd700 1 -- 172.21.15.7:0/2740290638 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+15392 (secure 0 0 0) 0x7fe5880c8a30 con 0x7fe58815a060 2024-01-11T07:17:51.909 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:17:51.909 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":172,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:17:27.283818+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:16:33.823565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-11T07:16:34.902544+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.5","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:17:51.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe574066f30 msgr2=0x7fe5740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe574066f30 0x7fe5740693e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe578009db0 tx=0x7fe5780091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 msgr2=0x7fe5881549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5881549d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c8e0 tx=0x7fe580001710 comp rx=0 tx=0).stop 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 shutdown_connections 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe574066f30 0x7fe5740693e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe58815a060 0x7fe5881549d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe588069320 0x7fe588067860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 --2- 172.21.15.7:0/2740290638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe588159c90 0x7fe588154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 >> 172.21.15.7:0/2740290638 conn(0x7fe588071100 msgr2=0x7fe588158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 shutdown_connections 2024-01-11T07:17:51.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:51.911+0000 7fe58d408700 1 -- 172.21.15.7:0/2740290638 wait complete. 2024-01-11T07:17:51.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:16:33.823565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:16:34.902544+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:17:51.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:17:52.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.060+0000 7fbc1333e700 1 Processor -- start 2024-01-11T07:17:52.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.061+0000 7fbc1333e700 1 -- start start 2024-01-11T07:17:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.061+0000 7fbc1333e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c158c80 0x7fbc0c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.061+0000 7fbc1333e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0c159100 con 0x7fbc0c0add00 2024-01-11T07:17:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.061+0000 7fbc1333e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0c0cee70 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.061+0000 7fbc1333e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c0cfa10 con 0x7fbc0c158c80 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbc110da700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbc0c0add00 0x7fbc0c0cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51838/0 (socket says 172.21.15.7:51838) 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbc110da700 1 -- 172.21.15.7:0/1034603164 learned_addr learned my addr 172.21.15.7:0/1034603164 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbc108d9700 1 --2- 172.21.15.7:0/1034603164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c158c80 0x7fbc0c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbbfffff700 1 -- 172.21.15.7:0/1034603164 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639871045 0 0) 0x7fbc0c159100 con 0x7fbc0c0add00 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbbfffff700 1 -- 172.21.15.7:0/1034603164 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbf8003570 con 0x7fbc0c0add00 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbbfffff700 1 -- 172.21.15.7:0/1034603164 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227289965 0 0) 0x7fbc0c0cee70 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbbfffff700 1 -- 172.21.15.7:0/1034603164 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc0c159100 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbbfffff700 1 -- 172.21.15.7:0/1034603164 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1973063419 0 0) 0x7fbbf8003570 con 0x7fbc0c0add00 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbc108d9700 1 -- 172.21.15.7:0/1034603164 >> v1:172.21.15.113:6789/0 conn(0x7fbc0c0b35e0 legacy=0x7fbc0c0b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbc108d9700 1 -- 172.21.15.7:0/1034603164 >> v1:172.21.15.7:6789/0 conn(0x7fbc0c0add00 legacy=0x7fbc0c0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbc108d9700 1 -- 172.21.15.7:0/1034603164 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0ce840 con 0x7fbc0c158c80 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.062+0000 7fbc108d9700 1 --2- 172.21.15.7:0/1034603164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c158c80 0x7fbc0c0b0430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009910 tx=0x7fbc080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d418ac4df334a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbbfffff700 1 -- 172.21.15.7:0/1034603164 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00001470 con 0x7fbc0c158c80 2024-01-11T07:17:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbbfffff700 1 -- 172.21.15.7:0/1034603164 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc08011420 con 0x7fbc0c158c80 2024-01-11T07:17:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbbfffff700 1 -- 172.21.15.7:0/1034603164 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc080117a0 con 0x7fbc0c158c80 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbc1333e700 1 -- 172.21.15.7:0/1034603164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c158c80 msgr2=0x7fbc0c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbc1333e700 1 --2- 172.21.15.7:0/1034603164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c158c80 0x7fbc0c0b0430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009910 tx=0x7fbc080091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbc1333e700 1 -- 172.21.15.7:0/1034603164 shutdown_connections 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbc1333e700 1 --2- 172.21.15.7:0/1034603164 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c158c80 0x7fbc0c0b0430 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbc1333e700 1 -- 172.21.15.7:0/1034603164 >> 172.21.15.7:0/1034603164 conn(0x7fbc0c071110 msgr2=0x7fbc0c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbc1333e700 1 -- 172.21.15.7:0/1034603164 shutdown_connections 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.063+0000 7fbc1333e700 1 -- 172.21.15.7:0/1034603164 wait complete. 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.065+0000 7fbc1333e700 1 Processor -- start 2024-01-11T07:17:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.065+0000 7fbc1333e700 1 -- start start 2024-01-11T07:17:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc1333e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c0add00 0x7fbc0c0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc1333e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c0b35e0 0x7fbc0c067eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc1333e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc0c158c80 0x7fbc0c155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc1333e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c15d210 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc1333e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c15c600 con 0x7fbc0c158c80 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc1333e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c15c8e0 con 0x7fbc0c0add00 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c0b35e0 0x7fbc0c067eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c0b35e0 0x7fbc0c067eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50318/0 (socket says 172.21.15.7:50318) 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 -- 172.21.15.7:0/596512768 learned_addr learned my addr 172.21.15.7:0/596512768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc110da700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c0add00 0x7fbc0c0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc118db700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc0c158c80 0x7fbc0c155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 -- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c0add00 msgr2=0x7fbc0c0acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c0add00 0x7fbc0c0acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 -- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc0c158c80 msgr2=0x7fbc0c155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc0c158c80 0x7fbc0c155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 -- 172.21.15.7:0/596512768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009420 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc110da700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c0add00 0x7fbc0c0acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc118db700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc0c158c80 0x7fbc0c155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbc108d9700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c0b35e0 0x7fbc0c067eb0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009860 tx=0x7fbc08016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.066+0000 7fbbfd7fa700 1 -- 172.21.15.7:0/596512768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0801cc50 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.067+0000 7fbbfd7fa700 1 -- 172.21.15.7:0/596512768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0801cdb0 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.067+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0c15cbc0 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.067+0000 7fbbfd7fa700 1 -- 172.21.15.7:0/596512768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0801a6a0 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.067+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0c15c240 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.068+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0c04c3c0 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.068+0000 7fbbfd7fa700 1 -- 172.21.15.7:0/596512768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbc0801aad0 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.068+0000 7fbbfd7fa700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbf0066f30 0x7fbbf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.069+0000 7fbbfd7fa700 1 -- 172.21.15.7:0/596512768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fbc08085f00 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.069+0000 7fbc110da700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbf0066f30 0x7fbbf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.069+0000 7fbbfd7fa700 1 -- 172.21.15.7:0/596512768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fbbf006eab0 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.069+0000 7fbc110da700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbf0066f30 0x7fbbf00693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbc00001ed0 tx=0x7fbc00000bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.073+0000 7fbbfd7fa700 1 -- 172.21.15.7:0/596512768 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc08053150 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.276+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc0c0cf7f0 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.276+0000 7fbbfd7fa700 1 -- 172.21.15.7:0/596512768 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc08056770 con 0x7fbc0c0b35e0 2024-01-11T07:17:52.276 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:17:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.278+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbf0066f30 msgr2=0x7fbbf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.278+0000 7fbc1333e700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbf0066f30 0x7fbbf00693e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbc00001ed0 tx=0x7fbc00000bc0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.278+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c0b35e0 msgr2=0x7fbc0c067eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.278+0000 7fbc1333e700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c0b35e0 0x7fbc0c067eb0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009860 tx=0x7fbc08016000 comp rx=0 tx=0).stop 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.279+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 shutdown_connections 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.279+0000 7fbc1333e700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbbf0066f30 0x7fbbf00693e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.279+0000 7fbc1333e700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc0c158c80 0x7fbc0c155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.279+0000 7fbc1333e700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc0c0add00 0x7fbc0c0acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.279+0000 7fbc1333e700 1 --2- 172.21.15.7:0/596512768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c0b35e0 0x7fbc0c067eb0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.279+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 >> 172.21.15.7:0/596512768 conn(0x7fbc0c071110 msgr2=0x7fbc0c0af480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.279+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 shutdown_connections 2024-01-11T07:17:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.279+0000 7fbc1333e700 1 -- 172.21.15.7:0/596512768 wait complete. 2024-01-11T07:17:52.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:17:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.430+0000 7f63dd622700 1 Processor -- start 2024-01-11T07:17:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.430+0000 7f63dd622700 1 -- start start 2024-01-11T07:17:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.431+0000 7f63dd622700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d80cbb60 0x7f63d815a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.431+0000 7f63dd622700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63d815e5a0 0x7f63d815a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.431+0000 7f63dd622700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d815e110 con 0x7f63d80cb790 2024-01-11T07:17:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.431+0000 7f63dd622700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d815b330 con 0x7f63d815e5a0 2024-01-11T07:17:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.431+0000 7f63dd622700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815ad90 con 0x7f63d80cbb60 2024-01-11T07:17:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.431+0000 7f63d77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63d815e5a0 0x7f63d815a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63d815e5a0 0x7f63d815a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50326/0 (socket says 172.21.15.7:50326) 2024-01-11T07:17:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d77fe700 1 -- 172.21.15.7:0/4055210930 learned_addr learned my addr 172.21.15.7:0/4055210930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d67fc700 1 --2- 172.21.15.7:0/4055210930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d80cbb60 0x7f63d815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d77fe700 1 -- 172.21.15.7:0/4055210930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d80cbb60 msgr2=0x7f63d815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d77fe700 1 --2- 172.21.15.7:0/4055210930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d80cbb60 0x7f63d815a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d77fe700 1 -- 172.21.15.7:0/4055210930 >> v1:172.21.15.113:6789/0 conn(0x7f63d80cb790 legacy=0x7f63d80cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d77fe700 1 -- 172.21.15.7:0/4055210930 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d815b050 con 0x7f63d815e5a0 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d77fe700 1 --2- 172.21.15.7:0/4055210930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d815e5a0 0x7f63d815a850 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009f20 tx=0x7f63d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bbc42e2bae085d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d5ffb700 1 -- 172.21.15.7:0/4055210930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000e070 con 0x7f63d815e5a0 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d5ffb700 1 -- 172.21.15.7:0/4055210930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d000ba80 con 0x7f63d815e5a0 2024-01-11T07:17:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.432+0000 7f63d5ffb700 1 -- 172.21.15.7:0/4055210930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d00092e0 con 0x7f63d815e5a0 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.433+0000 7f63dd622700 1 -- 172.21.15.7:0/4055210930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d815e5a0 msgr2=0x7f63d815a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.433+0000 7f63dd622700 1 --2- 172.21.15.7:0/4055210930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d815e5a0 0x7f63d815a850 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009f20 tx=0x7f63d0009840 comp rx=0 tx=0).stop 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.433+0000 7f63dd622700 1 -- 172.21.15.7:0/4055210930 shutdown_connections 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.433+0000 7f63dd622700 1 --2- 172.21.15.7:0/4055210930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d815e5a0 0x7f63d815a850 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.433+0000 7f63dd622700 1 --2- 172.21.15.7:0/4055210930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d80cbb60 0x7f63d815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.433+0000 7f63dd622700 1 -- 172.21.15.7:0/4055210930 >> 172.21.15.7:0/4055210930 conn(0x7f63d806ae00 msgr2=0x7f63d806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.433+0000 7f63dd622700 1 -- 172.21.15.7:0/4055210930 shutdown_connections 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.433+0000 7f63dd622700 1 -- 172.21.15.7:0/4055210930 wait complete. 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.434+0000 7f63dd622700 1 Processor -- start 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.434+0000 7f63dd622700 1 -- start start 2024-01-11T07:17:52.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63dd622700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d80cb790 0x7f63d80b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63dd622700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d80cbb60 0x7f63d8157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63dd622700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d815e5a0 0x7f63d80c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63dd622700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80c9e60 con 0x7f63d80cbb60 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63dd622700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8157b70 con 0x7f63d80cb790 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63dd622700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80c9250 con 0x7f63d815e5a0 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63d67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d80cbb60 0x7f63d8157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63d67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d80cbb60 0x7f63d8157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50334/0 (socket says 172.21.15.7:50334) 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63d67fc700 1 -- 172.21.15.7:0/615528435 learned_addr learned my addr 172.21.15.7:0/615528435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63d6ffd700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d80cb790 0x7f63d80b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.435+0000 7f63d77fe700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d815e5a0 0x7f63d80c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63d67fc700 1 -- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d815e5a0 msgr2=0x7f63d80c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63d67fc700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d815e5a0 0x7f63d80c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63d67fc700 1 -- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d80cb790 msgr2=0x7f63d80b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63d67fc700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d80cb790 0x7f63d80b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63d67fc700 1 -- 172.21.15.7:0/615528435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0009ae0 con 0x7f63d80cbb60 2024-01-11T07:17:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63d77fe700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d815e5a0 0x7f63d80c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:17:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63d67fc700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d80cbb60 0x7f63d8157630 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f63c800d890 tx=0x7f63c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63ceffd700 1 -- 172.21.15.7:0/615528435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8016d20 con 0x7f63d80cbb60 2024-01-11T07:17:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63ceffd700 1 -- 172.21.15.7:0/615528435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c801a070 con 0x7f63d80cbb60 2024-01-11T07:17:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d80c9530 con 0x7f63d80cbb60 2024-01-11T07:17:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63ceffd700 1 -- 172.21.15.7:0/615528435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80216b0 con 0x7f63d80cbb60 2024-01-11T07:17:52.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.436+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d80c9af0 con 0x7f63d80cbb60 2024-01-11T07:17:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.437+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d80ca010 con 0x7f63d80cbb60 2024-01-11T07:17:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.437+0000 7f63ceffd700 1 -- 172.21.15.7:0/615528435 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f63c8021a60 con 0x7f63d80cbb60 2024-01-11T07:17:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.438+0000 7f63ceffd700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.438+0000 7f63ceffd700 1 -- 172.21.15.7:0/615528435 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f63c801c070 con 0x7f63d80cbb60 2024-01-11T07:17:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.438+0000 7f63ceffd700 1 -- 172.21.15.7:0/615528435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f63b806eab0 con 0x7f63d80cbb60 2024-01-11T07:17:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.438+0000 7f63d6ffd700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.439+0000 7f63d6ffd700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f63b8066f30 0x7f63b80693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f63c0003a80 tx=0x7f63c0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.443+0000 7f63ceffd700 1 -- 172.21.15.7:0/615528435 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63d80ca010 con 0x7f63d80cbb60 2024-01-11T07:17:52.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.647+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63d804c3c0 con 0x7f63d80cbb60 2024-01-11T07:17:52.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.647+0000 7f63ceffd700 1 -- 172.21.15.7:0/615528435 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63c80260a0 con 0x7f63d80cbb60 2024-01-11T07:17:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:17:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f63b8066f30 msgr2=0x7f63b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f63b8066f30 0x7f63b80693e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f63c0003a80 tx=0x7f63c0007ec0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d80cbb60 msgr2=0x7f63d8157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d80cbb60 0x7f63d8157630 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f63c800d890 tx=0x7f63c8008040 comp rx=0 tx=0).stop 2024-01-11T07:17:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 shutdown_connections 2024-01-11T07:17:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f63b8066f30 0x7f63b80693e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63d815e5a0 0x7f63d80c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63d80cb790 0x7f63d80b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 --2- 172.21.15.7:0/615528435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d80cbb60 0x7f63d8157630 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 >> 172.21.15.7:0/615528435 conn(0x7f63d806ae00 msgr2=0x7f63d80c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 shutdown_connections 2024-01-11T07:17:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.649+0000 7f63dd622700 1 -- 172.21.15.7:0/615528435 wait complete. 2024-01-11T07:17:52.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:17:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.797+0000 7fb319cb8700 1 Processor -- start 2024-01-11T07:17:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.798+0000 7fb319cb8700 1 -- start start 2024-01-11T07:17:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.798+0000 7fb319cb8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 0x7fb3140ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.798+0000 7fb319cb8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31415c060 con 0x7fb3140ce580 2024-01-11T07:17:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.798+0000 7fb319cb8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31415b750 con 0x7fb314157d10 2024-01-11T07:17:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.798+0000 7fb319cb8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140cb0c0 con 0x7fb314157940 2024-01-11T07:17:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb313fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb3140ce580 0x7fb3140cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51840/0 (socket says 172.21.15.7:51840) 2024-01-11T07:17:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb313fff700 1 -- 172.21.15.7:0/3318696743 learned_addr learned my addr 172.21.15.7:0/3318696743 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3137fe700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 0x7fb3140ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191107966 0 0) 0x7fb31415c060 con 0x7fb3140ce580 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2fc003570 con 0x7fb3140ce580 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233121235 0 0) 0x7fb31415b750 con 0x7fb314157d10 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb31415c060 con 0x7fb314157d10 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3357757920 0 0) 0x7fb2fc003570 con 0x7fb3140ce580 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 msgr2=0x7fb3140ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 0x7fb3140ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 >> v1:172.21.15.113:6789/0 conn(0x7fb314157d10 legacy=0x7fb3140cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31415ba10 con 0x7fb3140ce580 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fb30c0017f0 con 0x7fb3140ce580 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3137fe700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 0x7fb3140ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 >> v1:172.21.15.7:6789/0 conn(0x7fb3140ce580 legacy=0x7fb3140cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc003c20 0x7fb2fc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 0x7fb2fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc004d50 0x7fb2fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140cb0c0 con 0x7fb2fc004d50 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc00bb70 con 0x7fb2fc003c20 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.799+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc00bd30 con 0x7fb314157940 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb313fff700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc004d50 0x7fb2fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb3137fe700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc003c20 0x7fb2fc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb312ffd700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 0x7fb2fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb313fff700 1 -- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 msgr2=0x7fb2fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb313fff700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb314157940 0x7fb2fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb313fff700 1 -- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc003c20 msgr2=0x7fb2fc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb313fff700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc003c20 0x7fb2fc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb313fff700 1 -- 172.21.15.7:0/3318696743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304009420 con 0x7fb2fc004d50 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb3137fe700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2fc003c20 0x7fb2fc004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb313fff700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc004d50 0x7fb2fc0051c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb31406bef0 tx=0x7fb30c0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c40ccff33cda53 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0071a0 con 0x7fb2fc004d50 2024-01-11T07:17:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c007300 con 0x7fb2fc004d50 2024-01-11T07:17:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.800+0000 7fb3127fc700 1 -- 172.21.15.7:0/3318696743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0076c0 con 0x7fb2fc004d50 2024-01-11T07:17:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.801+0000 7fb319cb8700 1 -- 172.21.15.7:0/3318696743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc004d50 msgr2=0x7fb2fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.801+0000 7fb319cb8700 1 --2- 172.21.15.7:0/3318696743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2fc004d50 0x7fb2fc0051c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb31406bef0 tx=0x7fb30c0020c0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.801+0000 7fb313fff700 1 -- 172.21.15.7:0/3318696743 reap_dead start 2024-01-11T07:17:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.801+0000 7fb319cb8700 1 -- 172.21.15.7:0/3318696743 shutdown_connections 2024-01-11T07:17:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.801+0000 7fb319cb8700 1 -- 172.21.15.7:0/3318696743 >> 172.21.15.7:0/3318696743 conn(0x7fb31406ae00 msgr2=0x7fb31406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.801+0000 7fb319cb8700 1 -- 172.21.15.7:0/3318696743 shutdown_connections 2024-01-11T07:17:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.801+0000 7fb319cb8700 1 -- 172.21.15.7:0/3318696743 wait complete. 2024-01-11T07:17:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.802+0000 7fb319cb8700 1 Processor -- start 2024-01-11T07:17:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.802+0000 7fb319cb8700 1 -- start start 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb319cb8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31414a540 0x7fb3140b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb319cb8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb31414a910 0x7fb3140b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb319cb8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3140cb3d0 0x7fb3140cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb319cb8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140cbd60 con 0x7fb31414a540 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb319cb8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31415c790 con 0x7fb31414a910 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb319cb8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31415ca10 con 0x7fb3140cb3d0 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb3137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31414a540 0x7fb3140b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb3137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31414a540 0x7fb3140b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50344/0 (socket says 172.21.15.7:50344) 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb3137fe700 1 -- 172.21.15.7:0/442530152 learned_addr learned my addr 172.21.15.7:0/442530152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb312ffd700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb31414a910 0x7fb3140b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb313fff700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3140cb3d0 0x7fb3140cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.803+0000 7fb3137fe700 1 -- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3140cb3d0 msgr2=0x7fb3140cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb3137fe700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3140cb3d0 0x7fb3140cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb3137fe700 1 -- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb31414a910 msgr2=0x7fb3140b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb3137fe700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb31414a910 0x7fb3140b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb3137fe700 1 -- 172.21.15.7:0/442530152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31415ccf0 con 0x7fb31414a540 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb312ffd700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb31414a910 0x7fb3140b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb3137fe700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31414a540 0x7fb3140b3b30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb304011ee0 tx=0x7fb3040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb2fb7fe700 1 -- 172.21.15.7:0/442530152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400fe70 con 0x7fb31414a540 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb2fb7fe700 1 -- 172.21.15.7:0/442530152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30401d480 con 0x7fb31414a540 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31415cfd0 con 0x7fb31414a540 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb2fb7fe700 1 -- 172.21.15.7:0/442530152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30401d840 con 0x7fb31414a540 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.804+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31415d590 con 0x7fb31414a540 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.805+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31404c3c0 con 0x7fb31414a540 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.805+0000 7fb2fb7fe700 1 -- 172.21.15.7:0/442530152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb30401dc70 con 0x7fb31414a540 2024-01-11T07:17:52.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.806+0000 7fb2fb7fe700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb300066f30 0x7fb3000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.806+0000 7fb2fb7fe700 1 -- 172.21.15.7:0/442530152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fb304018070 con 0x7fb31414a540 2024-01-11T07:17:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.806+0000 7fb2fb7fe700 1 -- 172.21.15.7:0/442530152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fb30006eab0 con 0x7fb31414a540 2024-01-11T07:17:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.806+0000 7fb312ffd700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb300066f30 0x7fb3000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.807+0000 7fb312ffd700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb300066f30 0x7fb3000693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb308007bc0 tx=0x7fb308009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:52.811+0000 7fb2fb7fe700 1 -- 172.21.15.7:0/442530152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3040528b0 con 0x7fb31414a540 2024-01-11T07:17:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.012+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb31415d900 con 0x7fb31414a540 2024-01-11T07:17:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.012+0000 7fb2fb7fe700 1 -- 172.21.15.7:0/442530152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb304022020 con 0x7fb31414a540 2024-01-11T07:17:53.012 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.014+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb300066f30 msgr2=0x7fb3000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.014+0000 7fb319cb8700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb300066f30 0x7fb3000693e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb308007bc0 tx=0x7fb308009040 comp rx=0 tx=0).stop 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.014+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31414a540 msgr2=0x7fb3140b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.014+0000 7fb319cb8700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31414a540 0x7fb3140b3b30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb304011ee0 tx=0x7fb3040091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.015+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 shutdown_connections 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.015+0000 7fb319cb8700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb300066f30 0x7fb3000693e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.015+0000 7fb319cb8700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb3140cb3d0 0x7fb3140cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.015+0000 7fb319cb8700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31414a540 0x7fb3140b3b30 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.015+0000 7fb319cb8700 1 --2- 172.21.15.7:0/442530152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb31414a910 0x7fb3140b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.015+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 >> 172.21.15.7:0/442530152 conn(0x7fb31406ae00 msgr2=0x7fb314069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.015+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 shutdown_connections 2024-01-11T07:17:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.015+0000 7fb319cb8700 1 -- 172.21.15.7:0/442530152 wait complete. 2024-01-11T07:17:53.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:17:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.159+0000 7f2a436f5700 1 Processor -- start 2024-01-11T07:17:53.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.160+0000 7f2a436f5700 1 -- start start 2024-01-11T07:17:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.160+0000 7f2a436f5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2a3c0aff20 0x7f2a3c0aab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.160+0000 7f2a436f5700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a3c063680 0x7f2a3c06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.160+0000 7f2a436f5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c0ab040 con 0x7f2a3c063230 2024-01-11T07:17:53.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a436f5700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0b02f0 con 0x7f2a3c0aff20 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a436f5700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c154860 con 0x7f2a3c063680 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a41491700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2a3c0aff20 0x7f2a3c0aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a40c90700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2a3c063230 0x7f2a3c06dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51854/0 (socket says 172.21.15.7:51854) 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a41491700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2a3c0aff20 0x7f2a3c0aab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50348/0 (socket says 172.21.15.7:50348) 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a40c90700 1 -- 172.21.15.7:0/2076246287 learned_addr learned my addr 172.21.15.7:0/2076246287 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a41c92700 1 --2- 172.21.15.7:0/2076246287 >> v2:172.21.15.113:3300/0 conn(0x7f2a3c063680 0x7f2a3c06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a41491700 1 -- 172.21.15.7:0/2076246287 >> v2:172.21.15.113:3300/0 conn(0x7f2a3c063680 msgr2=0x7f2a3c06e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a41491700 1 --2- 172.21.15.7:0/2076246287 >> v2:172.21.15.113:3300/0 conn(0x7f2a3c063680 0x7f2a3c06e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a41491700 1 -- 172.21.15.7:0/2076246287 >> v1:172.21.15.7:6789/0 conn(0x7f2a3c063230 legacy=0x7f2a3c06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a41491700 1 -- 172.21.15.7:0/2076246287 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c15d910 con 0x7f2a3c0aff20 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a33fff700 1 -- 172.21.15.7:0/2076246287 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030112424 0 0) 0x7f2a3c0ab040 con 0x7f2a3c063230 2024-01-11T07:17:53.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a41491700 1 --2- 172.21.15.7:0/2076246287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c0aff20 0x7f2a3c0aab00 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0ce8b0 tx=0x7f2a380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92f33e8af390e17 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a33fff700 1 -- 172.21.15.7:0/2076246287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800b6b0 con 0x7f2a3c0aff20 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.161+0000 7f2a33fff700 1 -- 172.21.15.7:0/2076246287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3800b810 con 0x7f2a3c0aff20 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a33fff700 1 -- 172.21.15.7:0/2076246287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800bbb0 con 0x7f2a3c0aff20 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a436f5700 1 -- 172.21.15.7:0/2076246287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c0aff20 msgr2=0x7f2a3c0aab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2076246287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c0aff20 0x7f2a3c0aab00 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0ce8b0 tx=0x7f2a380091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a436f5700 1 -- 172.21.15.7:0/2076246287 shutdown_connections 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2076246287 >> v2:172.21.15.113:3300/0 conn(0x7f2a3c063680 0x7f2a3c06e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2076246287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c0aff20 0x7f2a3c0aab00 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a436f5700 1 -- 172.21.15.7:0/2076246287 >> 172.21.15.7:0/2076246287 conn(0x7f2a3c068090 msgr2=0x7f2a3c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a436f5700 1 -- 172.21.15.7:0/2076246287 shutdown_connections 2024-01-11T07:17:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.162+0000 7f2a436f5700 1 -- 172.21.15.7:0/2076246287 wait complete. 2024-01-11T07:17:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.163+0000 7f2a436f5700 1 Processor -- start 2024-01-11T07:17:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.163+0000 7f2a436f5700 1 -- start start 2024-01-11T07:17:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.164+0000 7f2a436f5700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c063230 0x7f2a3c1710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.164+0000 7f2a436f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c063680 0x7f2a3c16ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.164+0000 7f2a436f5700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c0aff20 0x7f2a3c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.164+0000 7f2a436f5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c171630 con 0x7f2a3c063680 2024-01-11T07:17:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.164+0000 7f2a436f5700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c16f1d0 con 0x7f2a3c0aff20 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.164+0000 7f2a436f5700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0c5c20 con 0x7f2a3c063230 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c063680 0x7f2a3c16ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c063680 0x7f2a3c16ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50360/0 (socket says 172.21.15.7:50360) 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 -- 172.21.15.7:0/2911212960 learned_addr learned my addr 172.21.15.7:0/2911212960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a41491700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c063230 0x7f2a3c1710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a41c92700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c0aff20 0x7f2a3c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 -- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c063230 msgr2=0x7f2a3c1710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c063230 0x7f2a3c1710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 -- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c0aff20 msgr2=0x7f2a3c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c0aff20 0x7f2a3c0ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 -- 172.21.15.7:0/2911212960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a38009420 con 0x7f2a3c063680 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a40c90700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c063680 0x7f2a3c16ec90 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00c7e0 tx=0x7f2a2c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a317fa700 1 -- 172.21.15.7:0/2911212960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c005b30 con 0x7f2a3c063680 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a317fa700 1 -- 172.21.15.7:0/2911212960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c005c90 con 0x7f2a3c063680 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c0c5f00 con 0x7f2a3c063680 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a317fa700 1 -- 172.21.15.7:0/2911212960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c007300 con 0x7f2a3c063680 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.165+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c0c64c0 con 0x7f2a3c063680 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.166+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c04c3c0 con 0x7f2a3c063680 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.167+0000 7f2a317fa700 1 -- 172.21.15.7:0/2911212960 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2a2c01b3b0 con 0x7f2a3c063680 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.167+0000 7f2a317fa700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2a20066f30 0x7f2a200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.167+0000 7f2a317fa700 1 -- 172.21.15.7:0/2911212960 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7f2a2c083a70 con 0x7f2a3c063680 2024-01-11T07:17:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.167+0000 7f2a317fa700 1 -- 172.21.15.7:0/2911212960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f2a2006eab0 con 0x7f2a3c063680 2024-01-11T07:17:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.167+0000 7f2a41491700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2a20066f30 0x7f2a200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.168+0000 7f2a41491700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2a20066f30 0x7f2a200693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2a380093f0 tx=0x7f2a38017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.172+0000 7f2a317fa700 1 -- 172.21.15.7:0/2911212960 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a2c017020 con 0x7f2a3c063680 2024-01-11T07:17:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.420+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a3c15dbf0 con 0x7f2a3c063680 2024-01-11T07:17:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.420+0000 7f2a317fa700 1 -- 172.21.15.7:0/2911212960 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f2a2c050cc0 con 0x7f2a3c063680 2024-01-11T07:17:53.420 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:17:53.420 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4097846800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4097846800},{"type":"v1","addr":"172.21.15.113:6836","nonce":4097846800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1898628721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1898628721},{"type":"v1","addr":"172.21.15.113:6837","nonce":1898628721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3371959241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3371959241},{"type":"v1","addr":"172.21.15.7:6835","nonce":3371959241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:16:35.923785+0000","modified":"2024-01-11T07:16:39.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/2285850291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2285850291},{"type":"v1","addr":"172.21.15.7:6833","nonce":2285850291}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2a20066f30 msgr2=0x7f2a200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2a20066f30 0x7f2a200693e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2a380093f0 tx=0x7f2a38017000 comp rx=0 tx=0).stop 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c063680 msgr2=0x7f2a3c16ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c063680 0x7f2a3c16ec90 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00c7e0 tx=0x7f2a2c007df0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 shutdown_connections 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2a20066f30 0x7f2a200693e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c0aff20 0x7f2a3c0ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c063230 0x7f2a3c1710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 --2- 172.21.15.7:0/2911212960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a3c063680 0x7f2a3c16ec90 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 >> 172.21.15.7:0/2911212960 conn(0x7f2a3c068090 msgr2=0x7f2a3c0c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 shutdown_connections 2024-01-11T07:17:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.423+0000 7f2a436f5700 1 -- 172.21.15.7:0/2911212960 wait complete. 2024-01-11T07:17:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2024-01-11T07:17:53.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:17:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.568+0000 7fbfd028e700 1 Processor -- start 2024-01-11T07:17:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.569+0000 7fbfd028e700 1 -- start start 2024-01-11T07:17:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfd028e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbfc80cafb0 0x7fbfc80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfd028e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc8159300 con 0x7fbfc8158280 2024-01-11T07:17:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfd028e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc80cf260 con 0x7fbfc80b0b40 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfd028e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc80cf540 con 0x7fbfc80cafb0 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfcd829700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbfc8158280 0x7fbfc80b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51864/0 (socket says 172.21.15.7:51864) 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfcd829700 1 -- 172.21.15.7:0/3662924213 learned_addr learned my addr 172.21.15.7:0/3662924213 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfce02a700 1 --2- 172.21.15.7:0/3662924213 >> v2:172.21.15.113:3300/0 conn(0x7fbfc80cafb0 0x7fbfc80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfcd028700 1 -- 172.21.15.7:0/3662924213 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147690815 0 0) 0x7fbfc8159300 con 0x7fbfc8158280 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfcd028700 1 -- 172.21.15.7:0/3662924213 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb4003570 con 0x7fbfc8158280 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfce02a700 1 -- 172.21.15.7:0/3662924213 >> v1:172.21.15.113:6789/0 conn(0x7fbfc80b0b40 legacy=0x7fbfc8161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfcd028700 1 -- 172.21.15.7:0/3662924213 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650242420 0 0) 0x7fbfc80cf260 con 0x7fbfc80b0b40 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfce02a700 1 -- 172.21.15.7:0/3662924213 >> v1:172.21.15.7:6789/0 conn(0x7fbfc8158280 legacy=0x7fbfc80b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.570+0000 7fbfce02a700 1 -- 172.21.15.7:0/3662924213 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80cf820 con 0x7fbfc80cafb0 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfce02a700 1 --2- 172.21.15.7:0/3662924213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 0x7fbfc80ce8f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40098e0 tx=0x7fbfc40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a136d18ba721319 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfcd028700 1 -- 172.21.15.7:0/3662924213 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400b640 con 0x7fbfc80cafb0 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfcd028700 1 -- 172.21.15.7:0/3662924213 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc400b7a0 con 0x7fbfc80cafb0 2024-01-11T07:17:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfcd028700 1 -- 172.21.15.7:0/3662924213 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc400bb40 con 0x7fbfc80cafb0 2024-01-11T07:17:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfd028e700 1 -- 172.21.15.7:0/3662924213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 msgr2=0x7fbfc80ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfd028e700 1 --2- 172.21.15.7:0/3662924213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 0x7fbfc80ce8f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40098e0 tx=0x7fbfc40091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfd028e700 1 -- 172.21.15.7:0/3662924213 shutdown_connections 2024-01-11T07:17:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfd028e700 1 --2- 172.21.15.7:0/3662924213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 0x7fbfc80ce8f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfd028e700 1 -- 172.21.15.7:0/3662924213 >> 172.21.15.7:0/3662924213 conn(0x7fbfc8071100 msgr2=0x7fbfc806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfd028e700 1 -- 172.21.15.7:0/3662924213 shutdown_connections 2024-01-11T07:17:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.571+0000 7fbfd028e700 1 -- 172.21.15.7:0/3662924213 wait complete. 2024-01-11T07:17:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.573+0000 7fbfd028e700 1 Processor -- start 2024-01-11T07:17:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.573+0000 7fbfd028e700 1 -- start start 2024-01-11T07:17:53.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.573+0000 7fbfd028e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfc80b0b40 0x7fbfc80c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.573+0000 7fbfd028e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 0x7fbfc81554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.573+0000 7fbfd028e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc8158280 0x7fbfc80689a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.573+0000 7fbfd028e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8068510 con 0x7fbfc8158280 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.573+0000 7fbfd028e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8067640 con 0x7fbfc80cafb0 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.573+0000 7fbfd028e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8067900 con 0x7fbfc80b0b40 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc8158280 0x7fbfc80689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc8158280 0x7fbfc80689a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50368/0 (socket says 172.21.15.7:50368) 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 -- 172.21.15.7:0/2902136845 learned_addr learned my addr 172.21.15.7:0/2902136845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce02a700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfc80b0b40 0x7fbfc80c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfcd829700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 0x7fbfc81554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 -- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfc80b0b40 msgr2=0x7fbfc80c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfc80b0b40 0x7fbfc80c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 -- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 msgr2=0x7fbfc81554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 0x7fbfc81554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 -- 172.21.15.7:0/2902136845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4009420 con 0x7fbfc8158280 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfce82b700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc8158280 0x7fbfc80689a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0001560 tx=0x7fbfc0001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/2902136845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0004160 con 0x7fbfc8158280 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/2902136845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc000a7e0 con 0x7fbfc8158280 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc8067be0 con 0x7fbfc8158280 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/2902136845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000aba0 con 0x7fbfc8158280 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.574+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80681a0 con 0x7fbfc8158280 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.575+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc80686c0 con 0x7fbfc8158280 2024-01-11T07:17:53.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.576+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/2902136845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbfc0012420 con 0x7fbfc8158280 2024-01-11T07:17:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.576+0000 7fbfbe7fc700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbfac066f30 0x7fbfac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.577+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/2902136845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fbfc00842e0 con 0x7fbfc8158280 2024-01-11T07:17:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.577+0000 7fbfce02a700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbfac066f30 0x7fbfac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.577+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/2902136845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fbfac06eab0 con 0x7fbfc8158280 2024-01-11T07:17:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.577+0000 7fbfce02a700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbfac066f30 0x7fbfac0693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40093f0 tx=0x7fbfc4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.581+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/2902136845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfc80686c0 con 0x7fbfc8158280 2024-01-11T07:17:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.786+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfc80c8ba0 con 0x7fbfc8158280 2024-01-11T07:17:53.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.787+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/2902136845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+15392 (secure 0 0 0) 0x7fbfc0051530 con 0x7fbfc8158280 2024-01-11T07:17:53.787 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:17:53.787 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":172,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:17:27.283818+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:16:33.823565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-11T07:16:34.902544+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.5","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:17:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbfac066f30 msgr2=0x7fbfac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbfac066f30 0x7fbfac0693e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40093f0 tx=0x7fbfc4017000 comp rx=0 tx=0).stop 2024-01-11T07:17:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc8158280 msgr2=0x7fbfc80689a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc8158280 0x7fbfc80689a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0001560 tx=0x7fbfc0001770 comp rx=0 tx=0).stop 2024-01-11T07:17:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 shutdown_connections 2024-01-11T07:17:53.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbfac066f30 0x7fbfac0693e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc8158280 0x7fbfc80689a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbfc80b0b40 0x7fbfc80c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 --2- 172.21.15.7:0/2902136845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfc80cafb0 0x7fbfc81554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 >> 172.21.15.7:0/2902136845 conn(0x7fbfc8071100 msgr2=0x7fbfc8157590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:53.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 shutdown_connections 2024-01-11T07:17:53.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.789+0000 7fbfd028e700 1 -- 172.21.15.7:0/2902136845 wait complete. 2024-01-11T07:17:53.803 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:17:53.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:17:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.934+0000 7facc5729700 1 Processor -- start 2024-01-11T07:17:53.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.934+0000 7facc5729700 1 -- start start 2024-01-11T07:17:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.935+0000 7facc5729700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facc00cd6d0 0x7facc00cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.935+0000 7facc5729700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc015acf0 con 0x7facc015d8a0 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.935+0000 7facc5729700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc015afd0 con 0x7facc015e450 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.935+0000 7facc5729700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc00ce980 con 0x7facc00cd6d0 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.935+0000 7facbeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facc00cd6d0 0x7facc00cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.935+0000 7facbe7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7facc015d8a0 0x7facc015dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51872/0 (socket says 172.21.15.7:51872) 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.935+0000 7facbe7fc700 1 -- 172.21.15.7:0/3161877898 learned_addr learned my addr 172.21.15.7:0/3161877898 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.936+0000 7facbeffd700 1 -- 172.21.15.7:0/3161877898 >> v1:172.21.15.113:6789/0 conn(0x7facc015e450 legacy=0x7facc0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.936+0000 7facbeffd700 1 -- 172.21.15.7:0/3161877898 >> v1:172.21.15.7:6789/0 conn(0x7facc015d8a0 legacy=0x7facc015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.936+0000 7facbdffb700 1 -- 172.21.15.7:0/3161877898 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917851364 0 0) 0x7facc015acf0 con 0x7facc015d8a0 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.936+0000 7facbeffd700 1 -- 172.21.15.7:0/3161877898 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00cec60 con 0x7facc00cd6d0 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.936+0000 7facbeffd700 1 --2- 172.21.15.7:0/3161877898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cdaa0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7facb00098e0 tx=0x7facb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e532048358c81419 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.936+0000 7facbdffb700 1 -- 172.21.15.7:0/3161877898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0009e80 con 0x7facc00cd6d0 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.936+0000 7facbdffb700 1 -- 172.21.15.7:0/3161877898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb000b820 con 0x7facc00cd6d0 2024-01-11T07:17:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.936+0000 7facbdffb700 1 -- 172.21.15.7:0/3161877898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb000bba0 con 0x7facc00cd6d0 2024-01-11T07:17:53.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.937+0000 7facc5729700 1 -- 172.21.15.7:0/3161877898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 msgr2=0x7facc00cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.937+0000 7facc5729700 1 --2- 172.21.15.7:0/3161877898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cdaa0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7facb00098e0 tx=0x7facb00091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.937+0000 7facc5729700 1 -- 172.21.15.7:0/3161877898 shutdown_connections 2024-01-11T07:17:53.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.937+0000 7facc5729700 1 --2- 172.21.15.7:0/3161877898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cdaa0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.937+0000 7facc5729700 1 -- 172.21.15.7:0/3161877898 >> 172.21.15.7:0/3161877898 conn(0x7facc0071120 msgr2=0x7facc006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:53.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.937+0000 7facc5729700 1 -- 172.21.15.7:0/3161877898 shutdown_connections 2024-01-11T07:17:53.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.937+0000 7facc5729700 1 -- 172.21.15.7:0/3161877898 wait complete. 2024-01-11T07:17:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.938+0000 7facc5729700 1 Processor -- start 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.938+0000 7facc5729700 1 -- start start 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facc5729700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facc5729700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc015d8a0 0x7facc00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facc5729700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facc015e450 0x7facc00b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facc5729700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00cbc30 con 0x7facc00cd6d0 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facc5729700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00cb950 con 0x7facc015d8a0 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facc5729700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00b53a0 con 0x7facc015e450 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50386/0 (socket says 172.21.15.7:50386) 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbeffd700 1 -- 172.21.15.7:0/330186063 learned_addr learned my addr 172.21.15.7:0/330186063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbe7fc700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc015d8a0 0x7facc00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbeffd700 1 -- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facc015e450 msgr2=0x7facc00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbf7fe700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facc015e450 0x7facc00b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbeffd700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facc015e450 0x7facc00b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbeffd700 1 -- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc015d8a0 msgr2=0x7facc00b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbeffd700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc015d8a0 0x7facc00b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbeffd700 1 -- 172.21.15.7:0/330186063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb0009420 con 0x7facc00cd6d0 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.939+0000 7facbf7fe700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facc015e450 0x7facc00b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.940+0000 7facbeffd700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cb0d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7facb00030c0 tx=0x7facb0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.940+0000 7facb6ffd700 1 -- 172.21.15.7:0/330186063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb000bae0 con 0x7facc00cd6d0 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.940+0000 7facb6ffd700 1 -- 172.21.15.7:0/330186063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb0020070 con 0x7facc00cd6d0 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.940+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc00b5680 con 0x7facc00cd6d0 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.940+0000 7facb6ffd700 1 -- 172.21.15.7:0/330186063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb001b6a0 con 0x7facc00cd6d0 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.940+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc00cb610 con 0x7facc00cd6d0 2024-01-11T07:17:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.941+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc00b5cd0 con 0x7facc00cd6d0 2024-01-11T07:17:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.941+0000 7facb6ffd700 1 -- 172.21.15.7:0/330186063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7facb0022030 con 0x7facc00cd6d0 2024-01-11T07:17:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.942+0000 7facb6ffd700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faca0066f30 0x7faca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.942+0000 7facb6ffd700 1 -- 172.21.15.7:0/330186063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7facb00865b0 con 0x7facc00cd6d0 2024-01-11T07:17:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.942+0000 7facbe7fc700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faca0066f30 0x7faca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.942+0000 7facb6ffd700 1 -- 172.21.15.7:0/330186063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7faca006eab0 con 0x7facc00cd6d0 2024-01-11T07:17:53.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.943+0000 7facbe7fc700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faca0066f30 0x7faca00693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7facc016d800 tx=0x7faca8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:53.946+0000 7facb6ffd700 1 -- 172.21.15.7:0/330186063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facb0053760 con 0x7facc00cd6d0 2024-01-11T07:17:54.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.196+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facc004c3c0 con 0x7facc00cd6d0 2024-01-11T07:17:54.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.197+0000 7facb6ffd700 1 -- 172.21.15.7:0/330186063 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4601 (secure 0 0 0) 0x7facb0056d80 con 0x7facc00cd6d0 2024-01-11T07:17:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:17:54.197 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4097846800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4097846800},{"type":"v1","addr":"172.21.15.113:6836","nonce":4097846800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1898628721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1898628721},{"type":"v1","addr":"172.21.15.113:6837","nonce":1898628721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3371959241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3371959241},{"type":"v1","addr":"172.21.15.7:6835","nonce":3371959241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:16:35.923785+0000","modified":"2024-01-11T07:16:39.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/2285850291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2285850291},{"type":"v1","addr":"172.21.15.7:6833","nonce":2285850291}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-11T07:17:54.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faca0066f30 msgr2=0x7faca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faca0066f30 0x7faca00693e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7facc016d800 tx=0x7faca8009040 comp rx=0 tx=0).stop 2024-01-11T07:17:54.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 msgr2=0x7facc00cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cb0d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7facb00030c0 tx=0x7facb0016000 comp rx=0 tx=0).stop 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 shutdown_connections 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faca0066f30 0x7faca00693e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7facc015e450 0x7facc00b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc00cd6d0 0x7facc00cb0d0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 --2- 172.21.15.7:0/330186063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facc015d8a0 0x7facc00b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 >> 172.21.15.7:0/330186063 conn(0x7facc0071120 msgr2=0x7facc00c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 shutdown_connections 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.199+0000 7facc5729700 1 -- 172.21.15.7:0/330186063 wait complete. 2024-01-11T07:17:54.200 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2024-01-11T07:17:54.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:17:54.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.345+0000 7ff56d7b3700 1 Processor -- start 2024-01-11T07:17:54.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.345+0000 7ff56d7b3700 1 -- start start 2024-01-11T07:17:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff56d7b3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5680cadf0 0x7ff5680cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff56d7b3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5680699a0 con 0x7ff5681560d0 2024-01-11T07:17:54.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff56d7b3700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56806a540 con 0x7ff5680c6e10 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff56d7b3700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5680cc2f0 con 0x7ff5680cadf0 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff566ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff5681560d0 0x7ff568069e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51888/0 (socket says 172.21.15.7:51888) 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff566ffd700 1 -- 172.21.15.7:0/1728964132 learned_addr learned my addr 172.21.15.7:0/1728964132 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff5667fc700 1 --2- 172.21.15.7:0/1728964132 >> v2:172.21.15.113:3300/0 conn(0x7ff5680cadf0 0x7ff5680cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff565ffb700 1 -- 172.21.15.7:0/1728964132 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171674446 0 0) 0x7ff5680699a0 con 0x7ff5681560d0 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff565ffb700 1 -- 172.21.15.7:0/1728964132 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff550003570 con 0x7ff5681560d0 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff5667fc700 1 -- 172.21.15.7:0/1728964132 >> v1:172.21.15.113:6789/0 conn(0x7ff5680c6e10 legacy=0x7ff56806cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff5667fc700 1 -- 172.21.15.7:0/1728964132 >> v1:172.21.15.7:6789/0 conn(0x7ff5681560d0 legacy=0x7ff568069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff5667fc700 1 -- 172.21.15.7:0/1728964132 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff568068d90 con 0x7ff5680cadf0 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.346+0000 7ff5667fc700 1 --2- 172.21.15.7:0/1728964132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5680cadf0 0x7ff5680cb290 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0098e0 tx=0x7ff55c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a97fb17dd6bf41d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:54.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff565ffb700 1 -- 172.21.15.7:0/1728964132 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00b640 con 0x7ff5680cadf0 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff565ffb700 1 -- 172.21.15.7:0/1728964132 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c00b7a0 con 0x7ff5680cadf0 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff565ffb700 1 -- 172.21.15.7:0/1728964132 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00bb40 con 0x7ff5680cadf0 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff56d7b3700 1 -- 172.21.15.7:0/1728964132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5680cadf0 msgr2=0x7ff5680cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff56d7b3700 1 --2- 172.21.15.7:0/1728964132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5680cadf0 0x7ff5680cb290 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0098e0 tx=0x7ff55c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff56d7b3700 1 -- 172.21.15.7:0/1728964132 shutdown_connections 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff56d7b3700 1 --2- 172.21.15.7:0/1728964132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5680cadf0 0x7ff5680cb290 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff56d7b3700 1 -- 172.21.15.7:0/1728964132 >> 172.21.15.7:0/1728964132 conn(0x7ff568063230 msgr2=0x7ff568063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff56d7b3700 1 -- 172.21.15.7:0/1728964132 shutdown_connections 2024-01-11T07:17:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.347+0000 7ff56d7b3700 1 -- 172.21.15.7:0/1728964132 wait complete. 2024-01-11T07:17:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.349+0000 7ff56d7b3700 1 Processor -- start 2024-01-11T07:17:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.349+0000 7ff56d7b3700 1 -- start start 2024-01-11T07:17:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff56d7b3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5680c6e10 0x7ff568169590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff56d7b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5680cadf0 0x7ff5680ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff56d7b3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5681560d0 0x7ff5680ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff56d7b3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680ada70 con 0x7ff5680cadf0 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff56d7b3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56814dc60 con 0x7ff5681560d0 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff56d7b3700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff568169340 con 0x7ff5680c6e10 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5680cadf0 0x7ff5680ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5680cadf0 0x7ff5680ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50400/0 (socket says 172.21.15.7:50400) 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 -- 172.21.15.7:0/3772484516 learned_addr learned my addr 172.21.15.7:0/3772484516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5677fe700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5681560d0 0x7ff5680ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff566ffd700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5680c6e10 0x7ff568169590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 -- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5680c6e10 msgr2=0x7ff568169590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5680c6e10 0x7ff568169590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 -- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5681560d0 msgr2=0x7ff5680ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5681560d0 0x7ff5680ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 -- 172.21.15.7:0/3772484516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c009420 con 0x7ff5680cadf0 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff5667fc700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5680cadf0 0x7ff5680ade00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff56806ac50 tx=0x7ff55c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.350+0000 7ff54effd700 1 -- 172.21.15.7:0/3772484516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00bb20 con 0x7ff5680cadf0 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.351+0000 7ff54effd700 1 -- 172.21.15.7:0/3772484516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c011420 con 0x7ff5680cadf0 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.351+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5680c3f70 con 0x7ff5680cadf0 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.351+0000 7ff54effd700 1 -- 172.21.15.7:0/3772484516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0117a0 con 0x7ff5680cadf0 2024-01-11T07:17:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.351+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5680c3920 con 0x7ff5680cadf0 2024-01-11T07:17:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.351+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5680c4100 con 0x7ff5680cadf0 2024-01-11T07:17:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.352+0000 7ff54effd700 1 -- 172.21.15.7:0/3772484516 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff55c011b50 con 0x7ff5680cadf0 2024-01-11T07:17:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.352+0000 7ff54effd700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff554066f30 0x7ff5540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.353+0000 7ff54effd700 1 -- 172.21.15.7:0/3772484516 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7ff55c085c80 con 0x7ff5680cadf0 2024-01-11T07:17:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.353+0000 7ff566ffd700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff554066f30 0x7ff5540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.353+0000 7ff54effd700 1 -- 172.21.15.7:0/3772484516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7ff55406eab0 con 0x7ff5680cadf0 2024-01-11T07:17:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.353+0000 7ff566ffd700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff554066f30 0x7ff5540693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff5580058d0 tx=0x7ff558009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.357+0000 7ff54effd700 1 -- 172.21.15.7:0/3772484516 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff55c052ed0 con 0x7ff5680cadf0 2024-01-11T07:17:54.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.609+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff56804c3c0 con 0x7ff5680cadf0 2024-01-11T07:17:54.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.610+0000 7ff54effd700 1 -- 172.21.15.7:0/3772484516 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ff55c029030 con 0x7ff5680cadf0 2024-01-11T07:17:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:17:54.610 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4097846800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4097846800},{"type":"v1","addr":"172.21.15.113:6836","nonce":4097846800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1898628721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1898628721},{"type":"v1","addr":"172.21.15.113:6837","nonce":1898628721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3371959241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3371959241},{"type":"v1","addr":"172.21.15.7:6835","nonce":3371959241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:16:35.923785+0000","modified":"2024-01-11T07:16:39.504707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/2285850291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2285850291},{"type":"v1","addr":"172.21.15.7:6833","nonce":2285850291}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff554066f30 msgr2=0x7ff5540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff554066f30 0x7ff5540693e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff5580058d0 tx=0x7ff558009040 comp rx=0 tx=0).stop 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5680cadf0 msgr2=0x7ff5680ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5680cadf0 0x7ff5680ade00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff56806ac50 tx=0x7ff55c017000 comp rx=0 tx=0).stop 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 shutdown_connections 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff554066f30 0x7ff5540693e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5681560d0 0x7ff5680ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5680c6e10 0x7ff568169590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 --2- 172.21.15.7:0/3772484516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5680cadf0 0x7ff5680ade00 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 >> 172.21.15.7:0/3772484516 conn(0x7ff568063230 msgr2=0x7ff5680c1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:54.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 shutdown_connections 2024-01-11T07:17:54.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.612+0000 7ff56d7b3700 1 -- 172.21.15.7:0/3772484516 wait complete. 2024-01-11T07:17:54.613 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2024-01-11T07:17:54.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:17:54.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.757+0000 7fa3fc2d8700 1 Processor -- start 2024-01-11T07:17:54.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.757+0000 7fa3fc2d8700 1 -- start start 2024-01-11T07:17:54.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.758+0000 7fa3fc2d8700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3f406dd00 0x7fa3f40c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.758+0000 7fa3fc2d8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f415b770 con 0x7fa3f406e0d0 2024-01-11T07:17:54.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.758+0000 7fa3fc2d8700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f415a620 con 0x7fa3f415a1c0 2024-01-11T07:17:54.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.758+0000 7fa3fc2d8700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f415abf0 con 0x7fa3f406dd00 2024-01-11T07:17:54.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.758+0000 7fa3f9873700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa3f406e0d0 0x7fa3f4159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51898/0 (socket says 172.21.15.7:51898) 2024-01-11T07:17:54.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.758+0000 7fa3f9873700 1 -- 172.21.15.7:0/3248554238 learned_addr learned my addr 172.21.15.7:0/3248554238 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.758+0000 7fa3fa074700 1 --2- 172.21.15.7:0/3248554238 >> v2:172.21.15.113:3300/0 conn(0x7fa3f406dd00 0x7fa3f40c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.758+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361236207 0 0) 0x7fa3f415b770 con 0x7fa3f406e0d0 2024-01-11T07:17:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e0003570 con 0x7fa3f406e0d0 2024-01-11T07:17:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2797623364 0 0) 0x7fa3e0003570 con 0x7fa3f406e0d0 2024-01-11T07:17:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 >> v2:172.21.15.113:3300/0 conn(0x7fa3f406dd00 msgr2=0x7fa3f40c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 --2- 172.21.15.7:0/3248554238 >> v2:172.21.15.113:3300/0 conn(0x7fa3f406dd00 0x7fa3f40c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 >> v1:172.21.15.113:6789/0 conn(0x7fa3f415a1c0 legacy=0x7fa3f4069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f415aed0 con 0x7fa3f406e0d0 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fa3f0001470 con 0x7fa3f406e0d0 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 >> v1:172.21.15.7:6789/0 conn(0x7fa3f406e0d0 legacy=0x7fa3f4159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e0003a60 0x7fa3e0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3e0004810 0x7fa3e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3e00051c0 0x7fa3e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f415abf0 con 0x7fa3e00051c0 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e000c0e0 con 0x7fa3e0004810 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e000c2a0 con 0x7fa3e0003a60 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3fa875700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3e00051c0 0x7fa3e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3f9873700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3e0004810 0x7fa3e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3fa074700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e0003a60 0x7fa3e0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3fa875700 1 -- 172.21.15.7:0/3248554238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e0003a60 msgr2=0x7fa3e0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3fa875700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e0003a60 0x7fa3e0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3fa875700 1 -- 172.21.15.7:0/3248554238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3e0004810 msgr2=0x7fa3e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:54.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3fa875700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3e0004810 0x7fa3e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.759+0000 7fa3fa875700 1 -- 172.21.15.7:0/3248554238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e0003570 con 0x7fa3e00051c0 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3fa875700 1 -- 172.21.15.7:0/3248554238 reap_dead start 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3fa875700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3e00051c0 0x7fa3e000ba60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0020a0 tx=0x7fa3ec0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dba477d0a6e5356 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec0019e0 con 0x7fa3e00051c0 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec00c8b0 con 0x7fa3e00051c0 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3f9072700 1 -- 172.21.15.7:0/3248554238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00cc70 con 0x7fa3e00051c0 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/3248554238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3e00051c0 msgr2=0x7fa3e000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3fc2d8700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3e00051c0 0x7fa3e000ba60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0020a0 tx=0x7fa3ec0016d0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/3248554238 shutdown_connections 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.760+0000 7fa3fc2d8700 1 --2- 172.21.15.7:0/3248554238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3e00051c0 0x7fa3e000ba60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.761+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/3248554238 >> 172.21.15.7:0/3248554238 conn(0x7fa3f40680e0 msgr2=0x7fa3f40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.761+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/3248554238 shutdown_connections 2024-01-11T07:17:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.761+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/3248554238 wait complete. 2024-01-11T07:17:54.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.762+0000 7fa3fc2d8700 1 Processor -- start 2024-01-11T07:17:54.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.762+0000 7fa3fc2d8700 1 -- start start 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.762+0000 7fa3fc2d8700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e00051c0 0x7fa3f4158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.762+0000 7fa3fc2d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40c3910 0x7fa3f40c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3fc2d8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f40abea0 0x7fa3f40ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3fc2d8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40ac860 con 0x7fa3f40c3910 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3fc2d8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40c2290 con 0x7fa3f40abea0 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3fc2d8700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40c1680 con 0x7fa3e00051c0 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40c3910 0x7fa3f40c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40c3910 0x7fa3f40c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50418/0 (socket says 172.21.15.7:50418) 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 -- 172.21.15.7:0/2921376868 learned_addr learned my addr 172.21.15.7:0/2921376868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3fa875700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f40abea0 0x7fa3f40ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3fa074700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e00051c0 0x7fa3f4158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 -- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e00051c0 msgr2=0x7fa3f4158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e00051c0 0x7fa3f4158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 -- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f40abea0 msgr2=0x7fa3f40ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f40abea0 0x7fa3f40ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 -- 172.21.15.7:0/2921376868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40c1960 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3f9873700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40c3910 0x7fa3f40c3d80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00073f0 tx=0x7fa3f0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3e67fc700 1 -- 172.21.15.7:0/2921376868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0011590 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3e67fc700 1 -- 172.21.15.7:0/2921376868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0011710 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3e67fc700 1 -- 172.21.15.7:0/2921376868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0011ad0 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f40c1c40 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.763+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f40c1330 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.765+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f40c2420 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.765+0000 7fa3e67fc700 1 -- 172.21.15.7:0/2921376868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fa3f001d460 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.765+0000 7fa3e67fc700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3d8066f30 0x7fa3d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.765+0000 7fa3e67fc700 1 -- 172.21.15.7:0/2921376868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fa3f00845c0 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.765+0000 7fa3e67fc700 1 -- 172.21.15.7:0/2921376868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fa3d806eab0 con 0x7fa3f40c3910 2024-01-11T07:17:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.765+0000 7fa3fa074700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3d8066f30 0x7fa3d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.766+0000 7fa3fa074700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3d8066f30 0x7fa3d80693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8006f00 tx=0x7fa3e80091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.770+0000 7fa3e67fc700 1 -- 172.21.15.7:0/2921376868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3f0051810 con 0x7fa3f40c3910 2024-01-11T07:17:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.974+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3f404c3c0 con 0x7fa3f40c3910 2024-01-11T07:17:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.975+0000 7fa3e67fc700 1 -- 172.21.15.7:0/2921376868 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+15392 (secure 0 0 0) 0x7fa3f0054e30 con 0x7fa3f40c3910 2024-01-11T07:17:54.974 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:17:54.975 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":172,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:17:27.283818+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:16:33.823565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-01-11T07:16:34.902544+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.5","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3d8066f30 msgr2=0x7fa3d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3d8066f30 0x7fa3d80693e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8006f00 tx=0x7fa3e80091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40c3910 msgr2=0x7fa3f40c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40c3910 0x7fa3f40c3d80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00073f0 tx=0x7fa3f0001e70 comp rx=0 tx=0).stop 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 shutdown_connections 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fa3d8066f30 0x7fa3d80693e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3f40abea0 0x7fa3f40ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3e00051c0 0x7fa3f4158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 --2- 172.21.15.7:0/2921376868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40c3910 0x7fa3f40c3d80 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 >> 172.21.15.7:0/2921376868 conn(0x7fa3f40680e0 msgr2=0x7fa3f4069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 shutdown_connections 2024-01-11T07:17:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:54.977+0000 7fa3fc2d8700 1 -- 172.21.15.7:0/2921376868 wait complete. 2024-01-11T07:17:54.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:17:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.123+0000 7fcda483f700 1 Processor -- start 2024-01-11T07:17:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.123+0000 7fcda483f700 1 -- start start 2024-01-11T07:17:55.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcda483f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcda0063230 0x7fcda0063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcda483f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcda00ac5b0 0x7fcda00ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcda483f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda00ab100 con 0x7fcda015a280 2024-01-11T07:17:55.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcda483f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda015d880 con 0x7fcda00ac5b0 2024-01-11T07:17:55.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcda483f700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda006c5b0 con 0x7fcda0063230 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcda00ac5b0 0x7fcda00ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcda00ac5b0 0x7fcda00ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50430/0 (socket says 172.21.15.7:50430) 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9ed9d700 1 -- 172.21.15.7:0/927799376 learned_addr learned my addr 172.21.15.7:0/927799376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9e59c700 1 --2- 172.21.15.7:0/927799376 >> v2:172.21.15.113:3300/0 conn(0x7fcda0063230 0x7fcda0063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9ed9d700 1 -- 172.21.15.7:0/927799376 >> v2:172.21.15.113:3300/0 conn(0x7fcda0063230 msgr2=0x7fcda0063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9ed9d700 1 --2- 172.21.15.7:0/927799376 >> v2:172.21.15.113:3300/0 conn(0x7fcda0063230 0x7fcda0063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9ed9d700 1 -- 172.21.15.7:0/927799376 >> v1:172.21.15.113:6789/0 conn(0x7fcda015a280 legacy=0x7fcda00aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9ed9d700 1 -- 172.21.15.7:0/927799376 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0063b40 con 0x7fcda00ac5b0 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.124+0000 7fcd9e59c700 1 --2- 172.21.15.7:0/927799376 >> v2:172.21.15.113:3300/0 conn(0x7fcda0063230 0x7fcda0063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcd9ed9d700 1 --2- 172.21.15.7:0/927799376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda00ac9f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcda00ad1f0 tx=0x7fcd98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d38628aecba4edd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcd9d59a700 1 -- 172.21.15.7:0/927799376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9800e070 con 0x7fcda00ac5b0 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcd9d59a700 1 -- 172.21.15.7:0/927799376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9800ba50 con 0x7fcda00ac5b0 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcd9d59a700 1 -- 172.21.15.7:0/927799376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd980092e0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcda483f700 1 -- 172.21.15.7:0/927799376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 msgr2=0x7fcda00ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcda483f700 1 --2- 172.21.15.7:0/927799376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda00ac9f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcda00ad1f0 tx=0x7fcd98009840 comp rx=0 tx=0).stop 2024-01-11T07:17:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcda483f700 1 -- 172.21.15.7:0/927799376 shutdown_connections 2024-01-11T07:17:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcda483f700 1 --2- 172.21.15.7:0/927799376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda00ac9f0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcda483f700 1 --2- 172.21.15.7:0/927799376 >> v2:172.21.15.113:3300/0 conn(0x7fcda0063230 0x7fcda0063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcda483f700 1 -- 172.21.15.7:0/927799376 >> 172.21.15.7:0/927799376 conn(0x7fcda00680e0 msgr2=0x7fcda00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcda483f700 1 -- 172.21.15.7:0/927799376 shutdown_connections 2024-01-11T07:17:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.125+0000 7fcda483f700 1 -- 172.21.15.7:0/927799376 wait complete. 2024-01-11T07:17:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.127+0000 7fcda483f700 1 Processor -- start 2024-01-11T07:17:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.127+0000 7fcda483f700 1 -- start start 2024-01-11T07:17:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.127+0000 7fcda483f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcda0063230 0x7fcda00cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.127+0000 7fcda483f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda016a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcda483f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda015a280 0x7fcda0167350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcda483f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00ca970 con 0x7fcda00ac5b0 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcda483f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda016aea0 con 0x7fcda015a280 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcda483f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda0167890 con 0x7fcda0063230 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda016a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda016a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50442/0 (socket says 172.21.15.7:50442) 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 -- 172.21.15.7:0/1362326861 learned_addr learned my addr 172.21.15.7:0/1362326861 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9e59c700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcda0063230 0x7fcda00cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9ed9d700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda015a280 0x7fcda0167350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 -- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcda0063230 msgr2=0x7fcda00cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcda0063230 0x7fcda00cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 -- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda015a280 msgr2=0x7fcda0167350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda015a280 0x7fcda0167350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 -- 172.21.15.7:0/1362326861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd98009ae0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd9dd9b700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda016a960 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c590 tx=0x7fcd9400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94011590 con 0x7fcda00ac5b0 2024-01-11T07:17:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd94011710 con 0x7fcda00ac5b0 2024-01-11T07:17:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94011ad0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda00c40c0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.128+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda00c9fd0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.129+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda00caab0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.130+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcd9401b3b0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.130+0000 7fcd8e7fc700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd80066f30 0x7fcd800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.130+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5734+0+0 (secure 0 0 0) 0x7fcd940839c0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.131+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fcd8006eab0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.131+0000 7fcd9e59c700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd80066f30 0x7fcd800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.131+0000 7fcd9e59c700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd80066f30 0x7fcd800693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcda00ad380 tx=0x7fcd90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:55.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.135+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd94017070 con 0x7fcda00ac5b0 2024-01-11T07:17:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.386+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcda004c3c0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.386 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:17:55.386+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:17:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.388+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(173..173 src has 1..173) v4 ==== 366+0+0 (secure 0 0 0) 0x7fcd94054fd0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.388+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=174}) v3 -- 0x7fcd8006f400 con 0x7fcda00ac5b0 2024-01-11T07:17:55.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.391+0000 7fcd8e7fc700 1 -- 172.21.15.7:0/1362326861 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcda00caab0 con 0x7fcda00ac5b0 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd80066f30 msgr2=0x7fcd800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd80066f30 0x7fcd800693e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcda00ad380 tx=0x7fcd90008040 comp rx=0 tx=0).stop 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 msgr2=0x7fcda016a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda016a960 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c590 tx=0x7fcd9400c8a0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 shutdown_connections 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd80066f30 0x7fcd800693e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcda015a280 0x7fcda0167350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcda0063230 0x7fcda00cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 --2- 172.21.15.7:0/1362326861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00ac5b0 0x7fcda016a960 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 >> 172.21.15.7:0/1362326861 conn(0x7fcda00680e0 msgr2=0x7fcda015b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 shutdown_connections 2024-01-11T07:17:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.394+0000 7fcda483f700 1 -- 172.21.15.7:0/1362326861 wait complete. 2024-01-11T07:17:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:17:55.395 INFO:tasks.ceph.mds.c.smithi007.stderr: -8708> 2024-01-11T07:16:41.479+0000 7ff55954b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:17:55.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:17:55.427 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:17:55.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.543+0000 7f29db406700 1 Processor -- start 2024-01-11T07:17:55.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.543+0000 7f29db406700 1 -- start start 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.544+0000 7f29db406700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d415a050 0x7f29d406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.544+0000 7f29db406700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29d40b00c0 0x7f29d406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.544+0000 7f29db406700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d40cf2b0 con 0x7f29d40b0cd0 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.544+0000 7f29db406700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d40b06a0 con 0x7f29d40b00c0 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.544+0000 7f29db406700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40d0750 con 0x7f29d415a050 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.544+0000 7f29d99a3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29d40b00c0 0x7f29d406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.544+0000 7f29d91a2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d415a050 0x7f29d406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d99a3700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29d40b00c0 0x7f29d406e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:45860/0 (socket says 172.21.15.7:45860) 2024-01-11T07:17:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d89a1700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f29d40b0cd0 0x7f29d40af9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:45178/0 (socket says 172.21.15.7:45178) 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d99a3700 1 -- 172.21.15.7:0/2881993702 learned_addr learned my addr 172.21.15.7:0/2881993702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d91a2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d415a050 0x7f29d406dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:55592/0 (socket says 172.21.15.7:55592) 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d99a3700 1 -- 172.21.15.7:0/2881993702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d415a050 msgr2=0x7f29d406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d99a3700 1 --2- 172.21.15.7:0/2881993702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d415a050 0x7f29d406dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d99a3700 1 -- 172.21.15.7:0/2881993702 >> v1:172.21.15.113:6789/0 conn(0x7f29d40b0cd0 legacy=0x7f29d40af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d99a3700 1 -- 172.21.15.7:0/2881993702 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40ce980 con 0x7f29d40b00c0 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29cbfff700 1 -- 172.21.15.7:0/2881993702 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449439939 0 0) 0x7f29d40cf2b0 con 0x7f29d40b0cd0 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29d99a3700 1 --2- 172.21.15.7:0/2881993702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 0x7f29d406e170 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009f20 tx=0x7f29cc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aee31c23c50f4efd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29cbfff700 1 -- 172.21.15.7:0/2881993702 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00e070 con 0x7f29d40b00c0 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29cbfff700 1 -- 172.21.15.7:0/2881993702 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc00b9d0 con 0x7f29d40b00c0 2024-01-11T07:17:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.545+0000 7f29cbfff700 1 -- 172.21.15.7:0/2881993702 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0092e0 con 0x7f29d40b00c0 2024-01-11T07:17:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.546+0000 7f29db406700 1 -- 172.21.15.7:0/2881993702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 msgr2=0x7f29d406e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.546+0000 7f29db406700 1 --2- 172.21.15.7:0/2881993702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 0x7f29d406e170 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009f20 tx=0x7f29cc009840 comp rx=0 tx=0).stop 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.546+0000 7f29db406700 1 -- 172.21.15.7:0/2881993702 shutdown_connections 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.546+0000 7f29db406700 1 --2- 172.21.15.7:0/2881993702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 0x7f29d406e170 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.546+0000 7f29db406700 1 --2- 172.21.15.7:0/2881993702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d415a050 0x7f29d406dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.546+0000 7f29db406700 1 -- 172.21.15.7:0/2881993702 >> 172.21.15.7:0/2881993702 conn(0x7f29d4071160 msgr2=0x7f29d40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.546+0000 7f29db406700 1 -- 172.21.15.7:0/2881993702 shutdown_connections 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.546+0000 7f29db406700 1 -- 172.21.15.7:0/2881993702 wait complete. 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.547+0000 7f29db406700 1 Processor -- start 2024-01-11T07:17:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.547+0000 7f29db406700 1 -- start start 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.548+0000 7f29db406700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 0x7f29d41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.548+0000 7f29db406700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d40b0cd0 0x7f29d4169f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.548+0000 7f29db406700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d415a050 0x7f29d416a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.548+0000 7f29db406700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40aff30 con 0x7f29d415a050 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.548+0000 7f29db406700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4157fb0 con 0x7f29d40b00c0 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.548+0000 7f29db406700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4157cb0 con 0x7f29d40b0cd0 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d415a050 0x7f29d416a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d415a050 0x7f29d416a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50468/0 (socket says 172.21.15.7:50468) 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 -- 172.21.15.7:0/494210389 learned_addr learned my addr 172.21.15.7:0/494210389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:55.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d89a1700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d40b0cd0 0x7f29d4169f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d91a2700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 0x7f29d41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 -- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d40b0cd0 msgr2=0x7f29d4169f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d40b0cd0 0x7f29d4169f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 -- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 msgr2=0x7f29d41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 0x7f29d41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 -- 172.21.15.7:0/494210389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc009ae0 con 0x7f29d415a050 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29d99a3700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d415a050 0x7f29d416a4c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f29d40715b0 tx=0x7f29cc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29c97fa700 1 -- 172.21.15.7:0/494210389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc008b10 con 0x7f29d415a050 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29c97fa700 1 -- 172.21.15.7:0/494210389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc008c90 con 0x7f29d415a050 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d4157190 con 0x7f29d415a050 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29c97fa700 1 -- 172.21.15.7:0/494210389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0266b0 con 0x7f29d415a050 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.549+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d4155960 con 0x7f29d415a050 2024-01-11T07:17:55.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.550+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d40ccd20 con 0x7f29d415a050 2024-01-11T07:17:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.551+0000 7f29c97fa700 1 -- 172.21.15.7:0/494210389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f29cc003040 con 0x7f29d415a050 2024-01-11T07:17:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.551+0000 7f29c97fa700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f29b8066f30 0x7f29b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:55.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.551+0000 7f29c97fa700 1 -- 172.21.15.7:0/494210389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5820+0+0 (secure 0 0 0) 0x7f29cc086530 con 0x7f29d415a050 2024-01-11T07:17:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.551+0000 7f29d91a2700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f29b8066f30 0x7f29b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.552+0000 7f29c97fa700 1 -- 172.21.15.7:0/494210389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=174}) v3 -- 0x7f29b806eb50 con 0x7f29d415a050 2024-01-11T07:17:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.552+0000 7f29d91a2700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f29b8066f30 0x7f29b80693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009d10 tx=0x7f29d00091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.556+0000 7f29c97fa700 1 -- 172.21.15.7:0/494210389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29cc053690 con 0x7f29d415a050 2024-01-11T07:17:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:55.810+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f29d406fa10 con 0x7f29d415a050 2024-01-11T07:17:56.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.395+0000 7f29c97fa700 1 -- 172.21.15.7:0/494210389 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 104+0+0 (secure 0 0 0) 0x7f29cc02b020 con 0x7f29d415a050 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.398+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f29b8066f30 msgr2=0x7f29b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.398+0000 7f29db406700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f29b8066f30 0x7f29b80693e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009d10 tx=0x7f29d00091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.398+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d415a050 msgr2=0x7f29d416a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.398+0000 7f29db406700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d415a050 0x7f29d416a4c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f29d40715b0 tx=0x7f29cc018000 comp rx=0 tx=0).stop 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.399+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 shutdown_connections 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.399+0000 7f29db406700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f29b8066f30 0x7f29b80693e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.399+0000 7f29db406700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d415a050 0x7f29d416a4c0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.399+0000 7f29db406700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29d40b00c0 0x7f29d41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.399+0000 7f29db406700 1 --2- 172.21.15.7:0/494210389 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f29d40b0cd0 0x7f29d4169f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.399+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 >> 172.21.15.7:0/494210389 conn(0x7f29d4071160 msgr2=0x7f29d406de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.399+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 shutdown_connections 2024-01-11T07:17:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.399+0000 7f29db406700 1 -- 172.21.15.7:0/494210389 wait complete. 2024-01-11T07:17:56.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:17:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.605+0000 7f72e9e9a700 1 Processor -- start 2024-01-11T07:17:56.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.605+0000 7f72e9e9a700 1 -- start start 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e9e9a700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72e4157d60 0x7f72e40ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e9e9a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e406d850 con 0x7f72e415c3b0 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e9e9a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e40cd260 con 0x7f72e4157990 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e9e9a700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e40cc690 con 0x7f72e4157d60 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72e415c3b0 0x7f72e40cdcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51916/0 (socket says 172.21.15.7:51916) 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e3fff700 1 -- 172.21.15.7:0/840772380 learned_addr learned my addr 172.21.15.7:0/840772380 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e2ffd700 1 --2- 172.21.15.7:0/840772380 >> v2:172.21.15.113:3300/0 conn(0x7f72e4157d60 0x7f72e40ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e27fc700 1 -- 172.21.15.7:0/840772380 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104901373 0 0) 0x7f72e406d850 con 0x7f72e415c3b0 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.606+0000 7f72e27fc700 1 -- 172.21.15.7:0/840772380 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d0003570 con 0x7f72e415c3b0 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e27fc700 1 -- 172.21.15.7:0/840772380 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131185189 0 0) 0x7f72e40cd260 con 0x7f72e4157990 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e27fc700 1 -- 172.21.15.7:0/840772380 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e406d850 con 0x7f72e4157990 2024-01-11T07:17:56.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e2ffd700 1 -- 172.21.15.7:0/840772380 >> v1:172.21.15.113:6789/0 conn(0x7f72e4157990 legacy=0x7f72e406d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e2ffd700 1 -- 172.21.15.7:0/840772380 >> v1:172.21.15.7:6789/0 conn(0x7f72e415c3b0 legacy=0x7f72e40cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e2ffd700 1 -- 172.21.15.7:0/840772380 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e40cc930 con 0x7f72e4157d60 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e2ffd700 1 --2- 172.21.15.7:0/840772380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157d60 0x7f72e40ca4c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0098e0 tx=0x7f72cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d442618e531f88e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e27fc700 1 -- 172.21.15.7:0/840772380 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc011630 con 0x7f72e4157d60 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e27fc700 1 -- 172.21.15.7:0/840772380 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc011790 con 0x7f72e4157d60 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.607+0000 7f72e27fc700 1 -- 172.21.15.7:0/840772380 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc011b10 con 0x7f72e4157d60 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.608+0000 7f72e9e9a700 1 -- 172.21.15.7:0/840772380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157d60 msgr2=0x7f72e40ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.608+0000 7f72e9e9a700 1 --2- 172.21.15.7:0/840772380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157d60 0x7f72e40ca4c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0098e0 tx=0x7f72cc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.608+0000 7f72e9e9a700 1 -- 172.21.15.7:0/840772380 shutdown_connections 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.608+0000 7f72e9e9a700 1 --2- 172.21.15.7:0/840772380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157d60 0x7f72e40ca4c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.608+0000 7f72e9e9a700 1 -- 172.21.15.7:0/840772380 >> 172.21.15.7:0/840772380 conn(0x7f72e4063230 msgr2=0x7f72e4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.608+0000 7f72e9e9a700 1 -- 172.21.15.7:0/840772380 shutdown_connections 2024-01-11T07:17:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.608+0000 7f72e9e9a700 1 -- 172.21.15.7:0/840772380 wait complete. 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.609+0000 7f72e9e9a700 1 Processor -- start 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.609+0000 7f72e9e9a700 1 -- start start 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e9e9a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157990 0x7f72e4150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e9e9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e4157d60 0x7f72e40b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e9e9a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e415c3b0 0x7f72e40b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e9e9a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4155ba0 con 0x7f72e4157d60 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e9e9a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e40b1a80 con 0x7f72e4157990 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e9e9a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4150640 con 0x7f72e415c3b0 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e4157d60 0x7f72e40b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e4157d60 0x7f72e40b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50478/0 (socket says 172.21.15.7:50478) 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e2ffd700 1 -- 172.21.15.7:0/415636869 learned_addr learned my addr 172.21.15.7:0/415636869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:56.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e37fe700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157990 0x7f72e4150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e3fff700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e415c3b0 0x7f72e40b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e2ffd700 1 -- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e415c3b0 msgr2=0x7f72e40b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e2ffd700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e415c3b0 0x7f72e40b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e2ffd700 1 -- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157990 msgr2=0x7f72e4150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.610+0000 7f72e2ffd700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157990 0x7f72e4150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.611+0000 7f72e2ffd700 1 -- 172.21.15.7:0/415636869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc009420 con 0x7f72e4157d60 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.611+0000 7f72e37fe700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157990 0x7f72e4150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.611+0000 7f72e2ffd700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e4157d60 0x7f72e40b3d70 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009ed0 tx=0x7f72cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.611+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc011b10 con 0x7f72e4157d60 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.611+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc01a430 con 0x7f72e4157d60 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.611+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e4155d90 con 0x7f72e4157d60 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.611+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc01a7f0 con 0x7f72e4157d60 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.611+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e4156350 con 0x7f72e4157d60 2024-01-11T07:17:56.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.612+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e404c3c0 con 0x7f72e4157d60 2024-01-11T07:17:56.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.612+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f72cc01aba0 con 0x7f72e4157d60 2024-01-11T07:17:56.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.613+0000 7f72db7fe700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72c4066f30 0x7f72c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:56.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.613+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5820+0+0 (secure 0 0 0) 0x7f72cc086310 con 0x7f72e4157d60 2024-01-11T07:17:56.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.613+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=174}) v3 -- 0x7f72c406eb50 con 0x7f72e4157d60 2024-01-11T07:17:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.613+0000 7f72e37fe700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72c4066f30 0x7f72c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:56.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.614+0000 7f72e37fe700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72c4066f30 0x7f72c40693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f72d4001120 tx=0x7f72d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:56.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.618+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72cc053470 con 0x7f72e4157d60 2024-01-11T07:17:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:56.823+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72e40ccc10 con 0x7f72e4157d60 2024-01-11T07:17:57.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.290+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(174..174 src has 1..174) v4 ==== 679+0+0 (secure 0 0 0) 0x7f72cc057830 con 0x7f72e4157d60 2024-01-11T07:17:57.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.290+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=175}) v3 -- 0x7f72c406cf80 con 0x7f72e4157d60 2024-01-11T07:17:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.293+0000 7f72db7fe700 1 -- 172.21.15.7:0/415636869 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v174) v1 ==== 181+0+0 (secure 0 0 0) 0x7f72cc015020 con 0x7f72e4157d60 2024-01-11T07:17:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.298+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72c4066f30 msgr2=0x7f72c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72c4066f30 0x7f72c40693e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f72d4001120 tx=0x7f72d4009040 comp rx=0 tx=0).stop 2024-01-11T07:17:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e4157d60 msgr2=0x7f72e40b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e4157d60 0x7f72e40b3d70 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009ed0 tx=0x7f72cc017000 comp rx=0 tx=0).stop 2024-01-11T07:17:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 shutdown_connections 2024-01-11T07:17:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72c4066f30 0x7f72c40693e0 secure :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f72d4001120 tx=0x7f72d4009040 comp rx=0 tx=0).stop 2024-01-11T07:17:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72e415c3b0 0x7f72e40b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:57.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e4157990 0x7f72e4150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:57.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 --2- 172.21.15.7:0/415636869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e4157d60 0x7f72e40b3d70 secure :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009ed0 tx=0x7f72cc017000 comp rx=0 tx=0).stop 2024-01-11T07:17:57.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.299+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 >> 172.21.15.7:0/415636869 conn(0x7f72e4063230 msgr2=0x7f72e40c44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.302+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 shutdown_connections 2024-01-11T07:17:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.302+0000 7f72e9e9a700 1 -- 172.21.15.7:0/415636869 wait complete. 2024-01-11T07:17:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:17:57.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:17:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.457+0000 7fb8eb8eb700 1 Processor -- start 2024-01-11T07:17:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.458+0000 7fb8eb8eb700 1 -- start start 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.458+0000 7fb8eb8eb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8e415e520 0x7fb8e4067d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.458+0000 7fb8eb8eb700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb8e40ce3b0 0x7fb8e41597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.458+0000 7fb8eb8eb700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e415b2b0 con 0x7fb8e40cdfe0 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.458+0000 7fb8eb8eb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e40691f0 con 0x7fb8e415e520 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.458+0000 7fb8eb8eb700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e415ad00 con 0x7fb8e40ce3b0 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e8e86700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8e415e520 0x7fb8e4067d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e8e86700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8e415e520 0x7fb8e4067d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55564/0 (socket says 172.21.15.7:55564) 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e8e86700 1 -- 172.21.15.7:0/1162620577 learned_addr learned my addr 172.21.15.7:0/1162620577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e9e88700 1 --2- 172.21.15.7:0/1162620577 >> v2:172.21.15.113:3300/0 conn(0x7fb8e40ce3b0 0x7fb8e41597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:57.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e8e86700 1 -- 172.21.15.7:0/1162620577 >> v2:172.21.15.113:3300/0 conn(0x7fb8e40ce3b0 msgr2=0x7fb8e41597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e8e86700 1 --2- 172.21.15.7:0/1162620577 >> v2:172.21.15.113:3300/0 conn(0x7fb8e40ce3b0 0x7fb8e41597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e8e86700 1 -- 172.21.15.7:0/1162620577 >> v1:172.21.15.113:6789/0 conn(0x7fb8e40cdfe0 legacy=0x7fb8e4067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e8e86700 1 -- 172.21.15.7:0/1162620577 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e415afd0 con 0x7fb8e415e520 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e9e88700 1 --2- 172.21.15.7:0/1162620577 >> v2:172.21.15.113:3300/0 conn(0x7fb8e40ce3b0 0x7fb8e41597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8e8e86700 1 --2- 172.21.15.7:0/1162620577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4067d20 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb8e406d970 tx=0x7fb8d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5fd40451350b64 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8dbfff700 1 -- 172.21.15.7:0/1162620577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400b820 con 0x7fb8e415e520 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8dbfff700 1 -- 172.21.15.7:0/1162620577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d400b980 con 0x7fb8e415e520 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.459+0000 7fb8dbfff700 1 -- 172.21.15.7:0/1162620577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400bd20 con 0x7fb8e415e520 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.460+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/1162620577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 msgr2=0x7fb8e4067d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.460+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/1162620577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4067d20 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb8e406d970 tx=0x7fb8d4009250 comp rx=0 tx=0).stop 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.460+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/1162620577 shutdown_connections 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.460+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/1162620577 >> v2:172.21.15.113:3300/0 conn(0x7fb8e40ce3b0 0x7fb8e41597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.460+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/1162620577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4067d20 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.460+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/1162620577 >> 172.21.15.7:0/1162620577 conn(0x7fb8e406edc0 msgr2=0x7fb8e406f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:57.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.460+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/1162620577 shutdown_connections 2024-01-11T07:17:57.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.460+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/1162620577 wait complete. 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.461+0000 7fb8eb8eb700 1 Processor -- start 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.461+0000 7fb8eb8eb700 1 -- start start 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8eb8eb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e40cdfe0 0x7fb8e4142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8eb8eb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e40ce3b0 0x7fb8e40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8eb8eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8eb8eb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e415b460 con 0x7fb8e415e520 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8eb8eb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40b4050 con 0x7fb8e40ce3b0 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8eb8eb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e40b4330 con 0x7fb8e40cdfe0 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8e9e88700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8e9e88700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55574/0 (socket says 172.21.15.7:55574) 2024-01-11T07:17:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.462+0000 7fb8e9e88700 1 -- 172.21.15.7:0/2402190716 learned_addr learned my addr 172.21.15.7:0/2402190716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8e8e86700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e40ce3b0 0x7fb8e40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8e9687700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e40cdfe0 0x7fb8e4142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8e9e88700 1 -- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e40cdfe0 msgr2=0x7fb8e4142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8e9e88700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e40cdfe0 0x7fb8e4142330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8e9e88700 1 -- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e40ce3b0 msgr2=0x7fb8e40b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8e9e88700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e40ce3b0 0x7fb8e40b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8e9e88700 1 -- 172.21.15.7:0/2402190716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40094f0 con 0x7fb8e415e520 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8e9e88700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4153a80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00a380 tx=0x7fb8dc007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8d97fa700 1 -- 172.21.15.7:0/2402190716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00ae10 con 0x7fb8e415e520 2024-01-11T07:17:57.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e40b4610 con 0x7fb8e415e520 2024-01-11T07:17:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e4142870 con 0x7fb8e415e520 2024-01-11T07:17:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8d97fa700 1 -- 172.21.15.7:0/2402190716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc007a40 con 0x7fb8e415e520 2024-01-11T07:17:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.463+0000 7fb8d97fa700 1 -- 172.21.15.7:0/2402190716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00d860 con 0x7fb8e415e520 2024-01-11T07:17:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.464+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c40052a0 con 0x7fb8e415e520 2024-01-11T07:17:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.465+0000 7fb8d97fa700 1 -- 172.21.15.7:0/2402190716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb8dc00d9c0 con 0x7fb8e415e520 2024-01-11T07:17:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.465+0000 7fb8d97fa700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c8066f30 0x7fb8c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.465+0000 7fb8d97fa700 1 -- 172.21.15.7:0/2402190716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fb8dc084a30 con 0x7fb8e415e520 2024-01-11T07:17:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.465+0000 7fb8e9687700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c8066f30 0x7fb8c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.466+0000 7fb8e9687700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c8066f30 0x7fb8c80693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0003870 tx=0x7fb8e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.470+0000 7fb8d97fa700 1 -- 172.21.15.7:0/2402190716 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8dc051e90 con 0x7fb8e415e520 2024-01-11T07:17:57.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:57.669+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8c4005550 con 0x7fb8e415e520 2024-01-11T07:17:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.387+0000 7fb8d97fa700 1 -- 172.21.15.7:0/2402190716 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v175) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb8dc01a020 con 0x7fb8e415e520 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c8066f30 msgr2=0x7fb8c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c8066f30 0x7fb8c80693e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0003870 tx=0x7fb8e0009040 comp rx=0 tx=0).stop 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 msgr2=0x7fb8e4153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4153a80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00a380 tx=0x7fb8dc007900 comp rx=0 tx=0).stop 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 shutdown_connections 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb8c8066f30 0x7fb8c80693e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e415e520 0x7fb8e4153a80 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb8e40cdfe0 0x7fb8e4142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 --2- 172.21.15.7:0/2402190716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8e40ce3b0 0x7fb8e40b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 >> 172.21.15.7:0/2402190716 conn(0x7fb8e406edc0 msgr2=0x7fb8e40c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 shutdown_connections 2024-01-11T07:17:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.393+0000 7fb8eb8eb700 1 -- 172.21.15.7:0/2402190716 wait complete. 2024-01-11T07:17:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:17:58.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:17:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.546+0000 7f2e90048700 1 Processor -- start 2024-01-11T07:17:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.546+0000 7f2e90048700 1 -- start start 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e90048700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e88069450 0x7f2e880cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e90048700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e880adee0 0x7f2e880b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e90048700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e880b1820 con 0x7f2e880adb10 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e90048700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e880cee50 con 0x7f2e880adee0 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e90048700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e880b1a70 con 0x7f2e88069450 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e8e5e5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e880adee0 0x7f2e880b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e8e5e5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e880adee0 0x7f2e880b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55580/0 (socket says 172.21.15.7:55580) 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e8dde4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2e880adb10 0x7f2e8815ad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41958/0 (socket says 172.21.15.7:41958) 2024-01-11T07:17:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.547+0000 7f2e8e5e5700 1 -- 172.21.15.7:0/1391639018 learned_addr learned my addr 172.21.15.7:0/1391639018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8d5e3700 1 --2- 172.21.15.7:0/1391639018 >> v2:172.21.15.113:3300/0 conn(0x7f2e88069450 0x7f2e880cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8e5e5700 1 -- 172.21.15.7:0/1391639018 >> v2:172.21.15.113:3300/0 conn(0x7f2e88069450 msgr2=0x7f2e880cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8e5e5700 1 --2- 172.21.15.7:0/1391639018 >> v2:172.21.15.113:3300/0 conn(0x7f2e88069450 0x7f2e880cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8e5e5700 1 -- 172.21.15.7:0/1391639018 >> v1:172.21.15.7:6789/0 conn(0x7f2e880adb10 legacy=0x7f2e8815ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8e5e5700 1 -- 172.21.15.7:0/1391639018 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e880b1d50 con 0x7f2e880adee0 2024-01-11T07:17:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8cde2700 1 -- 172.21.15.7:0/1391639018 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294877792 0 0) 0x7f2e880b1820 con 0x7f2e880adb10 2024-01-11T07:17:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8e5e5700 1 --2- 172.21.15.7:0/1391639018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e880b0430 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2e880b15a0 tx=0x7f2e80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f22ba6f05adfa3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8cde2700 1 -- 172.21.15.7:0/1391639018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000e070 con 0x7f2e880adee0 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8cde2700 1 -- 172.21.15.7:0/1391639018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8000ba50 con 0x7f2e880adee0 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.548+0000 7f2e8cde2700 1 -- 172.21.15.7:0/1391639018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e800092e0 con 0x7f2e880adee0 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.549+0000 7f2e90048700 1 -- 172.21.15.7:0/1391639018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 msgr2=0x7f2e880b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.549+0000 7f2e90048700 1 --2- 172.21.15.7:0/1391639018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e880b0430 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2e880b15a0 tx=0x7f2e80009840 comp rx=0 tx=0).stop 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.549+0000 7f2e90048700 1 -- 172.21.15.7:0/1391639018 shutdown_connections 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.549+0000 7f2e90048700 1 --2- 172.21.15.7:0/1391639018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e880b0430 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.549+0000 7f2e90048700 1 --2- 172.21.15.7:0/1391639018 >> v2:172.21.15.113:3300/0 conn(0x7f2e88069450 0x7f2e880cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.549+0000 7f2e90048700 1 -- 172.21.15.7:0/1391639018 >> 172.21.15.7:0/1391639018 conn(0x7f2e88071110 msgr2=0x7f2e8806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.549+0000 7f2e90048700 1 -- 172.21.15.7:0/1391639018 shutdown_connections 2024-01-11T07:17:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.549+0000 7f2e90048700 1 -- 172.21.15.7:0/1391639018 wait complete. 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.550+0000 7f2e90048700 1 Processor -- start 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.550+0000 7f2e90048700 1 -- start start 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e90048700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e88069450 0x7f2e8816cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e90048700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880adb10 0x7f2e880cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e90048700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e88067ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e90048700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e88157ae0 con 0x7f2e880adee0 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e90048700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880cc200 con 0x7f2e88069450 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e90048700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8816d6d0 con 0x7f2e880adb10 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e88067ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e88067ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55588/0 (socket says 172.21.15.7:55588) 2024-01-11T07:17:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 -- 172.21.15.7:0/4287655404 learned_addr learned my addr 172.21.15.7:0/4287655404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8d5e3700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880adb10 0x7f2e880cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8dde4700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e88069450 0x7f2e8816cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 -- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880adb10 msgr2=0x7f2e880cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880adb10 0x7f2e880cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 -- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e88069450 msgr2=0x7f2e8816cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e88069450 0x7f2e8816cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 -- 172.21.15.7:0/4287655404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e80009ae0 con 0x7f2e880adee0 2024-01-11T07:17:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8d5e3700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880adb10 0x7f2e880cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.551+0000 7f2e8e5e5700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e88067ce0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2e88066af0 tx=0x7f2e80018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.552+0000 7f2e79ffb700 1 -- 172.21.15.7:0/4287655404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000be60 con 0x7f2e880adee0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.552+0000 7f2e79ffb700 1 -- 172.21.15.7:0/4287655404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e80008d90 con 0x7f2e880adee0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.552+0000 7f2e79ffb700 1 -- 172.21.15.7:0/4287655404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e800266b0 con 0x7f2e880adee0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.552+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8816d9b0 con 0x7f2e880adee0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.552+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e880c7d90 con 0x7f2e880adee0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.553+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8816ded0 con 0x7f2e880adee0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.554+0000 7f2e79ffb700 1 -- 172.21.15.7:0/4287655404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2e80003040 con 0x7f2e880adee0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.554+0000 7f2e79ffb700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.554+0000 7f2e79ffb700 1 -- 172.21.15.7:0/4287655404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2e80085c50 con 0x7f2e880adee0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.554+0000 7f2e8dde4700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.555+0000 7f2e8dde4700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c007fd0 tx=0x7f2e7c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.558+0000 7f2e79ffb700 1 -- 172.21.15.7:0/4287655404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e80053270 con 0x7f2e880adee0 2024-01-11T07:17:58.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.761+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2e8804c3c0 con 0x7f2e880adee0 2024-01-11T07:17:58.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.765+0000 7f2e79ffb700 1 -- 172.21.15.7:0/4287655404 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2e80056890 con 0x7f2e880adee0 2024-01-11T07:17:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 msgr2=0x7f2e6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c007fd0 tx=0x7f2e7c007d50 comp rx=0 tx=0).stop 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 msgr2=0x7f2e88067ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e88067ce0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2e88066af0 tx=0x7f2e80018000 comp rx=0 tx=0).stop 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 shutdown_connections 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e6c066f30 0x7f2e6c0693e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e880adee0 0x7f2e88067ce0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e88069450 0x7f2e8816cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 --2- 172.21.15.7:0/4287655404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e880adb10 0x7f2e880cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 >> 172.21.15.7:0/4287655404 conn(0x7f2e88071110 msgr2=0x7f2e880b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 shutdown_connections 2024-01-11T07:17:58.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.767+0000 7f2e90048700 1 -- 172.21.15.7:0/4287655404 wait complete. 2024-01-11T07:17:58.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-01-11T07:17:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.919+0000 7ffb66fd4700 1 Processor -- start 2024-01-11T07:17:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.920+0000 7ffb66fd4700 1 -- start start 2024-01-11T07:17:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.920+0000 7ffb66fd4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 0x7ffb600b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.920+0000 7ffb66fd4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb60069320 0x7ffb600b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.920+0000 7ffb66fd4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb600ce9b0 con 0x7ffb6015a060 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.920+0000 7ffb66fd4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb600cf530 con 0x7ffb60069320 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.920+0000 7ffb66fd4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600cf9c0 con 0x7ffb60159c90 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb65571700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb60069320 0x7ffb600b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb5ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffb6015a060 0x7ffb600b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41974/0 (socket says 172.21.15.7:41974) 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb5ffff700 1 -- 172.21.15.7:0/952938087 learned_addr learned my addr 172.21.15.7:0/952938087 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb64d70700 1 --2- 172.21.15.7:0/952938087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 0x7ffb600b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb65571700 1 -- 172.21.15.7:0/952938087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 msgr2=0x7ffb600b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb65571700 1 --2- 172.21.15.7:0/952938087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 0x7ffb600b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb65571700 1 -- 172.21.15.7:0/952938087 >> v1:172.21.15.7:6789/0 conn(0x7ffb6015a060 legacy=0x7ffb600b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb65571700 1 -- 172.21.15.7:0/952938087 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600cec00 con 0x7ffb60069320 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb5f7fe700 1 -- 172.21.15.7:0/952938087 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596776352 0 0) 0x7ffb600ce9b0 con 0x7ffb6015a060 2024-01-11T07:17:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb65571700 1 --2- 172.21.15.7:0/952938087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb60069320 0x7ffb600b3520 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009e60 tx=0x7ffb58009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e3b7073c7c762b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb5f7fe700 1 -- 172.21.15.7:0/952938087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800e070 con 0x7ffb60069320 2024-01-11T07:17:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb5f7fe700 1 -- 172.21.15.7:0/952938087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5800bb60 con 0x7ffb60069320 2024-01-11T07:17:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.921+0000 7ffb5f7fe700 1 -- 172.21.15.7:0/952938087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58012430 con 0x7ffb60069320 2024-01-11T07:17:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.922+0000 7ffb66fd4700 1 -- 172.21.15.7:0/952938087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb60069320 msgr2=0x7ffb600b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.922+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/952938087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb60069320 0x7ffb600b3520 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009e60 tx=0x7ffb58009780 comp rx=0 tx=0).stop 2024-01-11T07:17:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.922+0000 7ffb66fd4700 1 -- 172.21.15.7:0/952938087 shutdown_connections 2024-01-11T07:17:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.922+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/952938087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb60069320 0x7ffb600b3520 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.922+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/952938087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 0x7ffb600b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.922+0000 7ffb66fd4700 1 -- 172.21.15.7:0/952938087 >> 172.21.15.7:0/952938087 conn(0x7ffb60071100 msgr2=0x7ffb6006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.922+0000 7ffb66fd4700 1 -- 172.21.15.7:0/952938087 shutdown_connections 2024-01-11T07:17:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.922+0000 7ffb66fd4700 1 -- 172.21.15.7:0/952938087 wait complete. 2024-01-11T07:17:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.923+0000 7ffb66fd4700 1 Processor -- start 2024-01-11T07:17:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.923+0000 7ffb66fd4700 1 -- start start 2024-01-11T07:17:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb66fd4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb60069320 0x7ffb60144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb66fd4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 0x7ffb601445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb66fd4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb6015a060 0x7ffb6016d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb66fd4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600c8c30 con 0x7ffb6015a060 2024-01-11T07:17:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb66fd4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600c8110 con 0x7ffb60069320 2024-01-11T07:17:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb66fd4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600c8670 con 0x7ffb60159c90 2024-01-11T07:17:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb65571700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb6015a060 0x7ffb6016d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb65571700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb6015a060 0x7ffb6016d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55604/0 (socket says 172.21.15.7:55604) 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb65571700 1 -- 172.21.15.7:0/1565176282 learned_addr learned my addr 172.21.15.7:0/1565176282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.924+0000 7ffb64d70700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb60069320 0x7ffb60144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb5ffff700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 0x7ffb601445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb65571700 1 -- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 msgr2=0x7ffb601445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb65571700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 0x7ffb601445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb65571700 1 -- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb60069320 msgr2=0x7ffb60144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb65571700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb60069320 0x7ffb60144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb65571700 1 -- 172.21.15.7:0/1565176282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009a20 con 0x7ffb6015a060 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb65571700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb6015a060 0x7ffb6016d470 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ffb6006a1d0 tx=0x7ffb58019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb5cff9700 1 -- 172.21.15.7:0/1565176282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58012e20 con 0x7ffb6015a060 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb5cff9700 1 -- 172.21.15.7:0/1565176282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb58027470 con 0x7ffb6015a060 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb600c8950 con 0x7ffb6015a060 2024-01-11T07:17:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb5cff9700 1 -- 172.21.15.7:0/1565176282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58027810 con 0x7ffb6015a060 2024-01-11T07:17:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.925+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb60154dd0 con 0x7ffb6015a060 2024-01-11T07:17:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.926+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb600aec80 con 0x7ffb6015a060 2024-01-11T07:17:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.927+0000 7ffb5cff9700 1 -- 172.21.15.7:0/1565176282 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ffb58024070 con 0x7ffb6015a060 2024-01-11T07:17:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.927+0000 7ffb5cff9700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffb44066f30 0x7ffb440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.927+0000 7ffb5cff9700 1 -- 172.21.15.7:0/1565176282 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffb58086850 con 0x7ffb6015a060 2024-01-11T07:17:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.927+0000 7ffb64d70700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffb44066f30 0x7ffb440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.928+0000 7ffb64d70700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffb44066f30 0x7ffb440693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009db0 tx=0x7ffb540091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:58.931+0000 7ffb5cff9700 1 -- 172.21.15.7:0/1565176282 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb58053dc0 con 0x7ffb6015a060 2024-01-11T07:17:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.220+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7ffb6004c3c0 con 0x7ffb6015a060 2024-01-11T07:17:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.382+0000 7ffb5cff9700 1 -- 172.21.15.7:0/1565176282 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v512) v1 ==== 131+0+0 (secure 0 0 0) 0x7ffb5802c0c0 con 0x7ffb6015a060 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffb44066f30 msgr2=0x7ffb440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffb44066f30 0x7ffb440693e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009db0 tx=0x7ffb540091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb6015a060 msgr2=0x7ffb6016d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb6015a060 0x7ffb6016d470 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ffb6006a1d0 tx=0x7ffb58019000 comp rx=0 tx=0).stop 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 shutdown_connections 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ffb44066f30 0x7ffb440693e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb6015a060 0x7ffb6016d470 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb60069320 0x7ffb60144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 --2- 172.21.15.7:0/1565176282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffb60159c90 0x7ffb601445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 >> 172.21.15.7:0/1565176282 conn(0x7ffb60071100 msgr2=0x7ffb601582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 shutdown_connections 2024-01-11T07:17:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.385+0000 7ffb66fd4700 1 -- 172.21.15.7:0/1565176282 wait complete. 2024-01-11T07:17:59.400 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-01-11T07:17:59.400 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2024-01-11T07:17:59.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-01-11T07:17:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.542+0000 7f7296a73700 1 Processor -- start 2024-01-11T07:17:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.542+0000 7f7296a73700 1 -- start start 2024-01-11T07:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.543+0000 7f7296a73700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7290157d80 0x7f7290063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.543+0000 7f7296a73700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729015c660 con 0x7f7290063770 2024-01-11T07:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.543+0000 7f7296a73700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7290063b90 con 0x7f72901579b0 2024-01-11T07:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.543+0000 7f7296a73700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72900aeab0 con 0x7f7290157d80 2024-01-11T07:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.543+0000 7f7295010700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7290063770 0x7f7290069c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41980/0 (socket says 172.21.15.7:41980) 2024-01-11T07:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f7295010700 1 -- 172.21.15.7:0/3311448485 learned_addr learned my addr 172.21.15.7:0/3311448485 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728ffff700 1 --2- 172.21.15.7:0/3311448485 >> v2:172.21.15.113:3300/0 conn(0x7f7290157d80 0x7f7290063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728f7fe700 1 -- 172.21.15.7:0/3311448485 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092267721 0 0) 0x7f729015c660 con 0x7f7290063770 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728f7fe700 1 -- 172.21.15.7:0/3311448485 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7280003570 con 0x7f7290063770 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728f7fe700 1 -- 172.21.15.7:0/3311448485 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106096538 0 0) 0x7f7290063b90 con 0x7f72901579b0 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728f7fe700 1 -- 172.21.15.7:0/3311448485 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f729015c660 con 0x7f72901579b0 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728ffff700 1 -- 172.21.15.7:0/3311448485 >> v1:172.21.15.113:6789/0 conn(0x7f72901579b0 legacy=0x7f729015bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728ffff700 1 -- 172.21.15.7:0/3311448485 >> v1:172.21.15.7:6789/0 conn(0x7f7290063770 legacy=0x7f7290069c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728ffff700 1 -- 172.21.15.7:0/3311448485 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900aed90 con 0x7f7290157d80 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728ffff700 1 --2- 172.21.15.7:0/3311448485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290157d80 0x7f7290063230 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f727c0098e0 tx=0x7f727c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5a06eb8f90ced8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728f7fe700 1 -- 172.21.15.7:0/3311448485 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c011630 con 0x7f7290157d80 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.544+0000 7f728f7fe700 1 -- 172.21.15.7:0/3311448485 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c011790 con 0x7f7290157d80 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.545+0000 7f728f7fe700 1 -- 172.21.15.7:0/3311448485 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c011b30 con 0x7f7290157d80 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.545+0000 7f7296a73700 1 -- 172.21.15.7:0/3311448485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290157d80 msgr2=0x7f7290063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.545+0000 7f7296a73700 1 --2- 172.21.15.7:0/3311448485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290157d80 0x7f7290063230 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f727c0098e0 tx=0x7f727c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.545+0000 7f7296a73700 1 -- 172.21.15.7:0/3311448485 shutdown_connections 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.545+0000 7f7296a73700 1 --2- 172.21.15.7:0/3311448485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290157d80 0x7f7290063230 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.545+0000 7f7296a73700 1 -- 172.21.15.7:0/3311448485 >> 172.21.15.7:0/3311448485 conn(0x7f72900680e0 msgr2=0x7f72900cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:17:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.545+0000 7f7296a73700 1 -- 172.21.15.7:0/3311448485 shutdown_connections 2024-01-11T07:17:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.545+0000 7f7296a73700 1 -- 172.21.15.7:0/3311448485 wait complete. 2024-01-11T07:17:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.547+0000 7f7296a73700 1 Processor -- start 2024-01-11T07:17:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.547+0000 7f7296a73700 1 -- start start 2024-01-11T07:17:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7296a73700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290063770 0x7f7290155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7296a73700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72901579b0 0x7f7290158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7296a73700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7290157d80 0x7f72901586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7296a73700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b3060 con 0x7f7290157d80 2024-01-11T07:17:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7296a73700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290156200 con 0x7f7290063770 2024-01-11T07:17:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7296a73700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900c2370 con 0x7f72901579b0 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7295010700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7290157d80 0x7f72901586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7295010700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7290157d80 0x7f72901586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55618/0 (socket says 172.21.15.7:55618) 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f7295010700 1 -- 172.21.15.7:0/3999055102 learned_addr learned my addr 172.21.15.7:0/3999055102 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f728ffff700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72901579b0 0x7f7290158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.548+0000 7f729480f700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290063770 0x7f7290155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f7295010700 1 -- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72901579b0 msgr2=0x7f7290158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f7295010700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72901579b0 0x7f7290158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f7295010700 1 -- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290063770 msgr2=0x7f7290155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f7295010700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290063770 0x7f7290155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:17:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f7295010700 1 -- 172.21.15.7:0/3999055102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c009420 con 0x7f7290157d80 2024-01-11T07:17:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f7295010700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7290157d80 0x7f72901586b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f728800cc60 tx=0x7f7288001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f728cff9700 1 -- 172.21.15.7:0/3999055102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800a910 con 0x7f7290157d80 2024-01-11T07:17:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f728cff9700 1 -- 172.21.15.7:0/3999055102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728800aa90 con 0x7f7290157d80 2024-01-11T07:17:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7290155ca0 con 0x7f7290157d80 2024-01-11T07:17:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f728cff9700 1 -- 172.21.15.7:0/3999055102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728800ae50 con 0x7f7290157d80 2024-01-11T07:17:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.549+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72900c1d20 con 0x7f7290157d80 2024-01-11T07:17:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.550+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729004c3c0 con 0x7f7290157d80 2024-01-11T07:17:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.550+0000 7f728cff9700 1 -- 172.21.15.7:0/3999055102 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7288007300 con 0x7f7290157d80 2024-01-11T07:17:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.551+0000 7f728cff9700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7274066f30 0x7f72740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:17:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.551+0000 7f728cff9700 1 -- 172.21.15.7:0/3999055102 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f72880845a0 con 0x7f7290157d80 2024-01-11T07:17:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.551+0000 7f729480f700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7274066f30 0x7f72740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:17:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.551+0000 7f729480f700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7274066f30 0x7f72740693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7284007fd0 tx=0x7f7284007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:17:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.555+0000 7f728cff9700 1 -- 172.21.15.7:0/3999055102 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7288055620 con 0x7f7290157d80 2024-01-11T07:17:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:17:59.843+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f72900af0a0 con 0x7f7290157d80 2024-01-11T07:18:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.387+0000 7f728cff9700 1 -- 172.21.15.7:0/3999055102 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v513) v1 ==== 138+0+0 (secure 0 0 0) 0x7f72880551b0 con 0x7f7290157d80 2024-01-11T07:18:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7274066f30 msgr2=0x7f72740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7274066f30 0x7f72740693e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7284007fd0 tx=0x7f7284007d10 comp rx=0 tx=0).stop 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7290157d80 msgr2=0x7f72901586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7290157d80 0x7f72901586b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f728800cc60 tx=0x7f7288001ae0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 shutdown_connections 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7274066f30 0x7f72740693e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7290157d80 0x7f72901586b0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7290063770 0x7f7290155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 --2- 172.21.15.7:0/3999055102 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72901579b0 0x7f7290158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.390+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 >> 172.21.15.7:0/3999055102 conn(0x7f72900680e0 msgr2=0x7f72900cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.391+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 shutdown_connections 2024-01-11T07:18:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.391+0000 7f7296a73700 1 -- 172.21.15.7:0/3999055102 wait complete. 2024-01-11T07:18:00.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-11T07:18:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.544+0000 7f6fc96aa700 1 Processor -- start 2024-01-11T07:18:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.544+0000 7f6fc96aa700 1 -- start start 2024-01-11T07:18:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.545+0000 7f6fc96aa700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6fc415a140 0x7f6fc40c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.545+0000 7f6fc96aa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc40c9df0 0x7f6fc4067cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.545+0000 7f6fc96aa700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc4067840 con 0x7f6fc4159d70 2024-01-11T07:18:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.545+0000 7f6fc96aa700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc40cb0c0 con 0x7f6fc415a140 2024-01-11T07:18:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.545+0000 7f6fc96aa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40ca820 con 0x7f6fc40c9df0 2024-01-11T07:18:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc27fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6fc415a140 0x7f6fc40c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc37fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc40c9df0 0x7f6fc4067cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc27fc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6fc415a140 0x7f6fc40c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:41144/0 (socket says 172.21.15.7:41144) 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc27fc700 1 -- 172.21.15.7:0/570525079 learned_addr learned my addr 172.21.15.7:0/570525079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc37fe700 1 --2- 172.21.15.7:0/570525079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc40c9df0 0x7f6fc4067cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:41652/0 (socket says 172.21.15.7:41652) 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc2ffd700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6fc4159d70 0x7f6fc40cf0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:41576/0 (socket says 172.21.15.7:41576) 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc27fc700 1 -- 172.21.15.7:0/570525079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc40c9df0 msgr2=0x7f6fc4067cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc27fc700 1 --2- 172.21.15.7:0/570525079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc40c9df0 0x7f6fc4067cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc27fc700 1 -- 172.21.15.7:0/570525079 >> v1:172.21.15.113:6789/0 conn(0x7f6fc4159d70 legacy=0x7f6fc40cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc27fc700 1 -- 172.21.15.7:0/570525079 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40cab00 con 0x7f6fc415a140 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc1ffb700 1 -- 172.21.15.7:0/570525079 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082546151 0 0) 0x7f6fc4067840 con 0x7f6fc4159d70 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc27fc700 1 --2- 172.21.15.7:0/570525079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 0x7f6fc40c98b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80099b0 tx=0x7f6fb8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=647e0b3a8c2fddbc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc1ffb700 1 -- 172.21.15.7:0/570525079 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8009ea0 con 0x7f6fc415a140 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc1ffb700 1 -- 172.21.15.7:0/570525079 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb800b860 con 0x7f6fc415a140 2024-01-11T07:18:00.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.546+0000 7f6fc1ffb700 1 -- 172.21.15.7:0/570525079 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800bc00 con 0x7f6fc415a140 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.547+0000 7f6fc96aa700 1 -- 172.21.15.7:0/570525079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 msgr2=0x7f6fc40c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.547+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/570525079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 0x7f6fc40c98b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80099b0 tx=0x7f6fb8009250 comp rx=0 tx=0).stop 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.547+0000 7f6fc96aa700 1 -- 172.21.15.7:0/570525079 shutdown_connections 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.547+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/570525079 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc40c9df0 0x7f6fc4067cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.547+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/570525079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 0x7f6fc40c98b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.547+0000 7f6fc96aa700 1 -- 172.21.15.7:0/570525079 >> 172.21.15.7:0/570525079 conn(0x7f6fc4097a50 msgr2=0x7f6fc40ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.547+0000 7f6fc96aa700 1 -- 172.21.15.7:0/570525079 shutdown_connections 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.547+0000 7f6fc96aa700 1 -- 172.21.15.7:0/570525079 wait complete. 2024-01-11T07:18:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.548+0000 7f6fc96aa700 1 Processor -- start 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc96aa700 1 -- start start 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc96aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40c9df0 0x7f6fc40b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc96aa700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc4159d70 0x7f6fc4099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc96aa700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 0x7f6fc4157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc96aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40b4060 con 0x7f6fc40c9df0 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc96aa700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc416ad40 con 0x7f6fc415a140 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc96aa700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc4157850 con 0x7f6fc4159d70 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40c9df0 0x7f6fc40b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40c9df0 0x7f6fc40b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55626/0 (socket says 172.21.15.7:55626) 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.549+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/3445658672 learned_addr learned my addr 172.21.15.7:0/3445658672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc37fe700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 0x7f6fc4157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc27fc700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc4159d70 0x7f6fc4099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc4159d70 msgr2=0x7f6fc4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc2ffd700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc4159d70 0x7f6fc4099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 msgr2=0x7f6fc4157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc2ffd700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 0x7f6fc4157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc2ffd700 1 -- 172.21.15.7:0/3445658672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80094f0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc2ffd700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40c9df0 0x7f6fc40b4290 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400c810 tx=0x7f6fb4007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6faaffd700 1 -- 172.21.15.7:0/3445658672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4005ac0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6faaffd700 1 -- 172.21.15.7:0/3445658672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb4005c20 con 0x7f6fc40c9df0 2024-01-11T07:18:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6faaffd700 1 -- 172.21.15.7:0/3445658672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40070d0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc40b3aa0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.550+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc416a9d0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.551+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc404c3c0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.552+0000 7f6faaffd700 1 -- 172.21.15.7:0/3445658672 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6fb40074c0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.552+0000 7f6faaffd700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fb0066f30 0x7f6fb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.552+0000 7f6faaffd700 1 -- 172.21.15.7:0/3445658672 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6fb4083650 con 0x7f6fc40c9df0 2024-01-11T07:18:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.552+0000 7f6fc27fc700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fb0066f30 0x7f6fb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.553+0000 7f6fc27fc700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fb0066f30 0x7f6fb00693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80099b0 tx=0x7f6fb8017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.557+0000 7f6faaffd700 1 -- 172.21.15.7:0/3445658672 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fb4050bc0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.761+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6fc40cade0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.765+0000 7f6faaffd700 1 -- 172.21.15.7:0/3445658672 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6fb40170c0 con 0x7f6fc40c9df0 2024-01-11T07:18:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fb0066f30 msgr2=0x7f6fb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fb0066f30 0x7f6fb00693e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6fb80099b0 tx=0x7f6fb8017000 comp rx=0 tx=0).stop 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40c9df0 msgr2=0x7f6fc40b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40c9df0 0x7f6fc40b4290 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400c810 tx=0x7f6fb4007de0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 shutdown_connections 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6fb0066f30 0x7f6fb00693e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6fc415a140 0x7f6fc4157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc40c9df0 0x7f6fc40b4290 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 --2- 172.21.15.7:0/3445658672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6fc4159d70 0x7f6fc4099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 >> 172.21.15.7:0/3445658672 conn(0x7f6fc4097a50 msgr2=0x7f6fc4068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 shutdown_connections 2024-01-11T07:18:00.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:00.767+0000 7f6fc96aa700 1 -- 172.21.15.7:0/3445658672 wait complete. 2024-01-11T07:18:00.781 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:18:00.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:18:00.834 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:18:00.878 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:18:00.879 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-11T07:18:00.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.017+0000 7f5a37ab4700 1 Processor -- start 2024-01-11T07:18:01.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.017+0000 7f5a37ab4700 1 -- start start 2024-01-11T07:18:01.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a37ab4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a30158390 0x7f5a30158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a37ab4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a30063230 0x7f5a300636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a37ab4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3015f6d0 con 0x7f5a3015f230 2024-01-11T07:18:01.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a37ab4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a30063bf0 con 0x7f5a30063230 2024-01-11T07:18:01.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a37ab4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30161e30 con 0x7f5a30158390 2024-01-11T07:18:01.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a3504f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5a3015f230 0x7f5a300b0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41996/0 (socket says 172.21.15.7:41996) 2024-01-11T07:18:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a3504f700 1 -- 172.21.15.7:0/3984243093 learned_addr learned my addr 172.21.15.7:0/3984243093 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a36051700 1 --2- 172.21.15.7:0/3984243093 >> v2:172.21.15.113:3300/0 conn(0x7f5a30063230 0x7f5a300636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a35850700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a30158390 0x7f5a30158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175891901 0 0) 0x7f5a3015f6d0 con 0x7f5a3015f230 2024-01-11T07:18:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.018+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a1c003570 con 0x7f5a3015f230 2024-01-11T07:18:01.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4070889047 0 0) 0x7f5a1c003570 con 0x7f5a3015f230 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a30158390 msgr2=0x7f5a30158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a30158390 0x7f5a30158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 >> v2:172.21.15.113:3300/0 conn(0x7f5a30063230 msgr2=0x7f5a300636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 --2- 172.21.15.7:0/3984243093 >> v2:172.21.15.113:3300/0 conn(0x7f5a30063230 0x7f5a300636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30162110 con 0x7f5a3015f230 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f5a20001470 con 0x7f5a3015f230 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a36051700 1 --2- 172.21.15.7:0/3984243093 >> v2:172.21.15.113:3300/0 conn(0x7f5a30063230 0x7f5a300636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 >> v1:172.21.15.7:6789/0 conn(0x7f5a3015f230 legacy=0x7f5a300b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a30158390 0x7f5a1c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a1c003fe0 0x7f5a1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1c004d50 0x7f5a1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30063bf0 con 0x7f5a1c003fe0 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30161e30 con 0x7f5a1c004d50 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c00ba90 con 0x7f5a30158390 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3504f700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a1c003fe0 0x7f5a1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a35850700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a30158390 0x7f5a1c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a36051700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1c004d50 0x7f5a1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3504f700 1 -- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a30158390 msgr2=0x7f5a1c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3504f700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a30158390 0x7f5a1c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3504f700 1 -- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1c004d50 msgr2=0x7f5a1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3504f700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1c004d50 0x7f5a1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3504f700 1 -- 172.21.15.7:0/3984243093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c003570 con 0x7f5a1c003fe0 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.019+0000 7f5a3504f700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a1c003fe0 0x7f5a1c004810 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c3c0 tx=0x7f5a20001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cf38e2b603fc50 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:01.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000cc30 con 0x7f5a1c003fe0 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2000cdd0 con 0x7f5a1c003fe0 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a3484e700 1 -- 172.21.15.7:0/3984243093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20005b70 con 0x7f5a1c003fe0 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a37ab4700 1 -- 172.21.15.7:0/3984243093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a1c003fe0 msgr2=0x7f5a1c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a37ab4700 1 --2- 172.21.15.7:0/3984243093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a1c003fe0 0x7f5a1c004810 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c3c0 tx=0x7f5a20001e90 comp rx=0 tx=0).stop 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a36051700 1 -- 172.21.15.7:0/3984243093 reap_dead start 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a37ab4700 1 -- 172.21.15.7:0/3984243093 shutdown_connections 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a37ab4700 1 -- 172.21.15.7:0/3984243093 >> 172.21.15.7:0/3984243093 conn(0x7f5a300cd910 msgr2=0x7f5a300cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a37ab4700 1 -- 172.21.15.7:0/3984243093 shutdown_connections 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.020+0000 7f5a37ab4700 1 -- 172.21.15.7:0/3984243093 wait complete. 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.021+0000 7f5a37ab4700 1 Processor -- start 2024-01-11T07:18:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a37ab4700 1 -- start start 2024-01-11T07:18:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a37ab4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a1c003fe0 0x7f5a3016f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a37ab4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a30148b70 0x7f5a300c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a37ab4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a300c3e40 0x7f5a3015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a37ab4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300c42b0 con 0x7f5a30148b70 2024-01-11T07:18:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a37ab4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30159f00 con 0x7f5a300c3e40 2024-01-11T07:18:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a37ab4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015d2c0 con 0x7f5a1c003fe0 2024-01-11T07:18:01.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a3504f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a30148b70 0x7f5a300c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a3504f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a30148b70 0x7f5a300c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55646/0 (socket says 172.21.15.7:55646) 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a3504f700 1 -- 172.21.15.7:0/1813803024 learned_addr learned my addr 172.21.15.7:0/1813803024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a36051700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a300c3e40 0x7f5a3015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.022+0000 7f5a35850700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a1c003fe0 0x7f5a3016f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a3504f700 1 -- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a1c003fe0 msgr2=0x7f5a3016f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a3504f700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a1c003fe0 0x7f5a3016f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a3504f700 1 -- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a300c3e40 msgr2=0x7f5a3015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a3504f700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a300c3e40 0x7f5a3015c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a3504f700 1 -- 172.21.15.7:0/1813803024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a301598b0 con 0x7f5a30148b70 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a3504f700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a30148b70 0x7f5a300c3900 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5a20007d10 tx=0x7f5a20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a25ffb700 1 -- 172.21.15.7:0/1813803024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000cc30 con 0x7f5a30148b70 2024-01-11T07:18:01.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a25ffb700 1 -- 172.21.15.7:0/1813803024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a20000fe0 con 0x7f5a30148b70 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a30159b90 con 0x7f5a30148b70 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a25ffb700 1 -- 172.21.15.7:0/1813803024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20007530 con 0x7f5a30148b70 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.023+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3015ca90 con 0x7f5a30148b70 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.024+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3015a070 con 0x7f5a30148b70 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.025+0000 7f5a25ffb700 1 -- 172.21.15.7:0/1813803024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5a20019070 con 0x7f5a30148b70 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.025+0000 7f5a25ffb700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5a14066f30 0x7f5a140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.025+0000 7f5a25ffb700 1 -- 172.21.15.7:0/1813803024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5a20084fc0 con 0x7f5a30148b70 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.026+0000 7f5a35850700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5a14066f30 0x7f5a140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:01.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.026+0000 7f5a35850700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5a14066f30 0x7f5a140693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c006f00 tx=0x7f5a2c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:01.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.030+0000 7f5a25ffb700 1 -- 172.21.15.7:0/1813803024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a200b65a0 con 0x7f5a30148b70 2024-01-11T07:18:01.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.280+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a3004c3c0 con 0x7f5a30148b70 2024-01-11T07:18:01.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.281+0000 7f5a25ffb700 1 -- 172.21.15.7:0/1813803024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f5a20017de0 con 0x7f5a30148b70 2024-01-11T07:18:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:01.281 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6584,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4097846800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4097846800},{"type":"v1","addr":"172.21.15.113:6836","nonce":4097846800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":6587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1898628721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1898628721},{"type":"v1","addr":"172.21.15.113:6837","nonce":1898628721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7474,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3371959241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3371959241},{"type":"v1","addr":"172.21.15.7:6835","nonce":3371959241}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":77},{"gid":7989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1143726540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1143726540},{"type":"v1","addr":"172.21.15.7:6833","nonce":1143726540}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[]} 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5a14066f30 msgr2=0x7f5a140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5a14066f30 0x7f5a140693e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c006f00 tx=0x7f5a2c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a30148b70 msgr2=0x7f5a300c3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a30148b70 0x7f5a300c3900 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5a20007d10 tx=0x7f5a20015000 comp rx=0 tx=0).stop 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 shutdown_connections 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5a14066f30 0x7f5a140693e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a300c3e40 0x7f5a3015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a1c003fe0 0x7f5a3016f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 --2- 172.21.15.7:0/1813803024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a30148b70 0x7f5a300c3900 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 >> 172.21.15.7:0/1813803024 conn(0x7f5a300cd910 msgr2=0x7f5a3015ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 shutdown_connections 2024-01-11T07:18:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:01.283+0000 7f5a37ab4700 1 -- 172.21.15.7:0/1813803024 wait complete. 2024-01-11T07:18:01.284 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 83 2024-01-11T07:18:01.296 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-11T07:18:01.297 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-11T07:18:01.297 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:18:01.297 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:18:01.297 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-11T07:18:01.297 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-11T07:18:01.297 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:18:01.297 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:18:01.298 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-11T07:18:01.298 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-11T07:18:01.298 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:18:01.298 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:18:01.298 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-11T07:18:01.298 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-11T07:18:01.298 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:18:01.299 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:18:01.309 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:18:01.310 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-11T07:18:01.310 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:18:01.310 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-11T07:18:01.312 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:18:01.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-11T07:18:01.352 INFO:tasks.ceph.mds.b:Started 2024-01-11T07:18:01.353 DEBUG:teuthology.parallel:result is None 2024-01-11T07:18:01.353 INFO:tasks.ceph.mds.d:Started 2024-01-11T07:18:01.353 DEBUG:teuthology.parallel:result is None 2024-01-11T07:18:01.355 INFO:tasks.ceph.mds.c:Started 2024-01-11T07:18:01.355 DEBUG:teuthology.parallel:result is None 2024-01-11T07:18:01.438 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-01-11T07:18:01.438 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-01-11T07:18:01.442 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-01-11T07:18:01.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:18:01.979 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:18:07.312 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:18:07.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-11T07:18:07.315 INFO:tasks.ceph.mds.a:Started 2024-01-11T07:18:07.315 DEBUG:teuthology.parallel:result is None 2024-01-11T07:18:07.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-11T07:18:07.405 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:18:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.460+0000 7fbde3734700 1 Processor -- start 2024-01-11T07:18:07.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.460+0000 7fbde3734700 1 -- start start 2024-01-11T07:18:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde3734700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbddc06dd00 0x7fbddc0c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde3734700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 0x7fbddc159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde3734700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc15b770 con 0x7fbddc159ff0 2024-01-11T07:18:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde3734700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc15abf0 con 0x7fbddc06dd00 2024-01-11T07:18:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde3734700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc15aed0 con 0x7fbddc06e0d0 2024-01-11T07:18:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde14d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbddc06dd00 0x7fbddc0c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde14d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbddc06dd00 0x7fbddc0c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55404/0 (socket says 172.21.15.7:55404) 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde14d0700 1 -- 172.21.15.7:0/2810730131 learned_addr learned my addr 172.21.15.7:0/2810730131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde0ccf700 1 --2- 172.21.15.7:0/2810730131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 0x7fbddc159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde14d0700 1 -- 172.21.15.7:0/2810730131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 msgr2=0x7fbddc159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde14d0700 1 --2- 172.21.15.7:0/2810730131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 0x7fbddc159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde14d0700 1 -- 172.21.15.7:0/2810730131 >> v1:172.21.15.113:6789/0 conn(0x7fbddc159ff0 legacy=0x7fbddc0c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde14d0700 1 -- 172.21.15.7:0/2810730131 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc15b1b0 con 0x7fbddc06dd00 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.461+0000 7fbde14d0700 1 --2- 172.21.15.7:0/2810730131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c98d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80099b0 tx=0x7fbdd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cd78713a0751d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbdd3fff700 1 -- 172.21.15.7:0/2810730131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800b860 con 0x7fbddc06dd00 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbdd3fff700 1 -- 172.21.15.7:0/2810730131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd800b9c0 con 0x7fbddc06dd00 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbdd3fff700 1 -- 172.21.15.7:0/2810730131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800bd60 con 0x7fbddc06dd00 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbde3734700 1 -- 172.21.15.7:0/2810730131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 msgr2=0x7fbddc0c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbde3734700 1 --2- 172.21.15.7:0/2810730131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c98d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80099b0 tx=0x7fbdd8009250 comp rx=0 tx=0).stop 2024-01-11T07:18:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbde3734700 1 -- 172.21.15.7:0/2810730131 shutdown_connections 2024-01-11T07:18:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbde3734700 1 --2- 172.21.15.7:0/2810730131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c98d0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbde3734700 1 --2- 172.21.15.7:0/2810730131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 0x7fbddc159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbde3734700 1 -- 172.21.15.7:0/2810730131 >> 172.21.15.7:0/2810730131 conn(0x7fbddc0680e0 msgr2=0x7fbddc0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbde3734700 1 -- 172.21.15.7:0/2810730131 shutdown_connections 2024-01-11T07:18:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.462+0000 7fbde3734700 1 -- 172.21.15.7:0/2810730131 wait complete. 2024-01-11T07:18:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.464+0000 7fbde3734700 1 Processor -- start 2024-01-11T07:18:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.464+0000 7fbde3734700 1 -- start start 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.465+0000 7fbde3734700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.465+0000 7fbde3734700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 0x7fbddc16b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.465+0000 7fbde3734700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbddc159ff0 0x7fbddc158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.465+0000 7fbde3734700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0c41b0 con 0x7fbddc06dd00 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.465+0000 7fbde3734700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0c2e20 con 0x7fbddc159ff0 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.465+0000 7fbde3734700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc159340 con 0x7fbddc06e0d0 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55406/0 (socket says 172.21.15.7:55406) 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 -- 172.21.15.7:0/4135805271 learned_addr learned my addr 172.21.15.7:0/4135805271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde0ccf700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 0x7fbddc16b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde1cd1700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbddc159ff0 0x7fbddc158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 -- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 msgr2=0x7fbddc16b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:07.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 0x7fbddc16b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 -- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbddc159ff0 msgr2=0x7fbddc158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbddc159ff0 0x7fbddc158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 -- 172.21.15.7:0/4135805271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80094f0 con 0x7fbddc06dd00 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde14d0700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c7710 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbddc06d820 tx=0x7fbdd8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbdd17fa700 1 -- 172.21.15.7:0/4135805271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8011e40 con 0x7fbddc06dd00 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbdd17fa700 1 -- 172.21.15.7:0/4135805271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd8024440 con 0x7fbddc06dd00 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc0c3130 con 0x7fbddc06dd00 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbdd17fa700 1 -- 172.21.15.7:0/4135805271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8024800 con 0x7fbddc06dd00 2024-01-11T07:18:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.466+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc0c36d0 con 0x7fbddc06dd00 2024-01-11T07:18:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.467+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc04c3c0 con 0x7fbddc06dd00 2024-01-11T07:18:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.468+0000 7fbdd17fa700 1 -- 172.21.15.7:0/4135805271 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbdd802d440 con 0x7fbddc06dd00 2024-01-11T07:18:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.468+0000 7fbdd17fa700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.468+0000 7fbdd17fa700 1 -- 172.21.15.7:0/4135805271 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fbdd8085b10 con 0x7fbddc06dd00 2024-01-11T07:18:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.469+0000 7fbde0ccf700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.469+0000 7fbde0ccf700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdc0066f30 0x7fbdc00693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0c12c0 tx=0x7fbdcc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.473+0000 7fbdd17fa700 1 -- 172.21.15.7:0/4135805271 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdd8053080 con 0x7fbddc06dd00 2024-01-11T07:18:07.660 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-01-11T07:18:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:07.694+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbddc15b490 con 0x7fbddc06dd00 2024-01-11T07:18:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.407+0000 7fbdd17fa700 1 -- 172.21.15.7:0/4135805271 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v180) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbdd80566a0 con 0x7fbddc06dd00 2024-01-11T07:18:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdc0066f30 msgr2=0x7fbdc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdc0066f30 0x7fbdc00693e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0c12c0 tx=0x7fbdcc008040 comp rx=0 tx=0).stop 2024-01-11T07:18:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 msgr2=0x7fbddc0c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c7710 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbddc06d820 tx=0x7fbdd8017000 comp rx=0 tx=0).stop 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 shutdown_connections 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbdc0066f30 0x7fbdc00693e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbddc159ff0 0x7fbddc158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbddc06dd00 0x7fbddc0c7710 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 --2- 172.21.15.7:0/4135805271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbddc06e0d0 0x7fbddc16b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 >> 172.21.15.7:0/4135805271 conn(0x7fbddc0680e0 msgr2=0x7fbddc0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 shutdown_connections 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.410+0000 7fbde3734700 1 -- 172.21.15.7:0/4135805271 wait complete. 2024-01-11T07:18:09.411 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-01-11T07:18:09.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-11T07:18:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.561+0000 7f4f97be1700 1 Processor -- start 2024-01-11T07:18:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.562+0000 7f4f97be1700 1 -- start start 2024-01-11T07:18:09.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f97be1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f900c18a0 0x7f4f9006f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f97be1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4f90068830 0x7f4f90068c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f97be1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9006b490 con 0x7f4f900ad9b0 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f97be1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f90161f30 con 0x7f4f900c18a0 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f97be1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9015e860 con 0x7f4f90068830 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9517c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f900c18a0 0x7f4f9006f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9517c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f900c18a0 0x7f4f9006f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55408/0 (socket says 172.21.15.7:55408) 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9517c700 1 -- 172.21.15.7:0/1175987181 learned_addr learned my addr 172.21.15.7:0/1175987181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9617e700 1 --2- 172.21.15.7:0/1175987181 >> v2:172.21.15.113:3300/0 conn(0x7f4f90068830 0x7f4f90068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9517c700 1 -- 172.21.15.7:0/1175987181 >> v2:172.21.15.113:3300/0 conn(0x7f4f90068830 msgr2=0x7f4f90068c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9517c700 1 --2- 172.21.15.7:0/1175987181 >> v2:172.21.15.113:3300/0 conn(0x7f4f90068830 0x7f4f90068c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9517c700 1 -- 172.21.15.7:0/1175987181 >> v1:172.21.15.113:6789/0 conn(0x7f4f900ad9b0 legacy=0x7f4f901626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9517c700 1 -- 172.21.15.7:0/1175987181 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f90069170 con 0x7f4f900c18a0 2024-01-11T07:18:09.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9617e700 1 --2- 172.21.15.7:0/1175987181 >> v2:172.21.15.113:3300/0 conn(0x7f4f90068830 0x7f4f90068c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.563+0000 7f4f9517c700 1 --2- 172.21.15.7:0/1175987181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f900c18a0 0x7f4f9006f6c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4f9006aff0 tx=0x7f4f80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a53de1646fa82f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f9497b700 1 -- 172.21.15.7:0/1175987181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000b860 con 0x7f4f900c18a0 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f9497b700 1 -- 172.21.15.7:0/1175987181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8000b9c0 con 0x7f4f900c18a0 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f9497b700 1 -- 172.21.15.7:0/1175987181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000bd60 con 0x7f4f900c18a0 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f97be1700 1 -- 172.21.15.7:0/1175987181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f900c18a0 msgr2=0x7f4f9006f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f97be1700 1 --2- 172.21.15.7:0/1175987181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f900c18a0 0x7f4f9006f6c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4f9006aff0 tx=0x7f4f80009250 comp rx=0 tx=0).stop 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f97be1700 1 -- 172.21.15.7:0/1175987181 shutdown_connections 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f97be1700 1 --2- 172.21.15.7:0/1175987181 >> v2:172.21.15.113:3300/0 conn(0x7f4f90068830 0x7f4f90068c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f97be1700 1 --2- 172.21.15.7:0/1175987181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f900c18a0 0x7f4f9006f6c0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f97be1700 1 -- 172.21.15.7:0/1175987181 >> 172.21.15.7:0/1175987181 conn(0x7f4f90070690 msgr2=0x7f4f90070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f97be1700 1 -- 172.21.15.7:0/1175987181 shutdown_connections 2024-01-11T07:18:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.564+0000 7f4f97be1700 1 -- 172.21.15.7:0/1175987181 wait complete. 2024-01-11T07:18:09.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.565+0000 7f4f97be1700 1 Processor -- start 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f97be1700 1 -- start start 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f97be1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f90068830 0x7f4f9015c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f97be1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900ad9b0 0x7f4f9006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f97be1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f900c18a0 0x7f4f9006f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f97be1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900c29e0 con 0x7f4f90068830 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f97be1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901614c0 con 0x7f4f900c18a0 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f97be1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901617a0 con 0x7f4f900ad9b0 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f9597d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f90068830 0x7f4f9015c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f9597d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f90068830 0x7f4f9015c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55418/0 (socket says 172.21.15.7:55418) 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.566+0000 7f4f9597d700 1 -- 172.21.15.7:0/3285473662 learned_addr learned my addr 172.21.15.7:0/3285473662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f9517c700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900ad9b0 0x7f4f9006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f9617e700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f900c18a0 0x7f4f9006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f9597d700 1 -- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900ad9b0 msgr2=0x7f4f9006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f9597d700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900ad9b0 0x7f4f9006ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f9597d700 1 -- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f900c18a0 msgr2=0x7f4f9006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f9597d700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f900c18a0 0x7f4f9006f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f9597d700 1 -- 172.21.15.7:0/3285473662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800094f0 con 0x7f4f90068830 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f9597d700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f90068830 0x7f4f9015c8e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00c590 tx=0x7f4f8c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f85ffb700 1 -- 172.21.15.7:0/3285473662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011590 con 0x7f4f90068830 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f85ffb700 1 -- 172.21.15.7:0/3285473662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c0116f0 con 0x7f4f90068830 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f90161a80 con 0x7f4f90068830 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f85ffb700 1 -- 172.21.15.7:0/3285473662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011ab0 con 0x7f4f90068830 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.567+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9015ce20 con 0x7f4f90068830 2024-01-11T07:18:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.568+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9004c3c0 con 0x7f4f90068830 2024-01-11T07:18:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.570+0000 7f4f85ffb700 1 -- 172.21.15.7:0/3285473662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4f8c01b3b0 con 0x7f4f90068830 2024-01-11T07:18:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.570+0000 7f4f85ffb700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f74066f30 0x7f4f740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.570+0000 7f4f85ffb700 1 -- 172.21.15.7:0/3285473662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4f8c083bb0 con 0x7f4f90068830 2024-01-11T07:18:09.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.570+0000 7f4f9517c700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f74066f30 0x7f4f740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.571+0000 7f4f9517c700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f74066f30 0x7f4f740693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4f9006aff0 tx=0x7f4f80017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:09.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.574+0000 7f4f85ffb700 1 -- 172.21.15.7:0/3285473662 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f8c0511f0 con 0x7f4f90068830 2024-01-11T07:18:09.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.860+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4f90162db0 con 0x7f4f90068830 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.861+0000 7f4f85ffb700 1 -- 172.21.15.7:0/3285473662 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4f8c017020 con 0x7f4f90068830 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lfrMdBRAA/zhECYBYtcQ0b90Q3qwu3A==", 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luqd3EhAAvu1xAxjXX8WeJl01UMdrcA==", 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:18:09.862 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lIxhiDRAAbBSTv4jox7ZP943SThq3wQ==", 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-01-11T07:18:09.863 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lctiKGRAAJuThC2tp63SrC7hZyY+vjQ==", 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lx5BEARAAhWENuCG89dcuXENnaSUgwg==", 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:18:09.864 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59ltZpOCRAAwu/tTOfYP0T1xiYmSGgxNw==", 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-01-11T07:18:09.865 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59lsqBNERAAC1l8cckHPsgXG3c30xlreA==", 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB7k59l8/pHGRAA1b9CIZom4wuOQlFK3l0U9A==", 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lwuhIKBAAsdGQ1msyC6mycyOEZGCJ7A==", 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59ll9p6LxAA1lJqj9VnU8rBWmzA84qQoA==", 2024-01-11T07:18:09.867 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB+k59lBA/sOBAAHXMnfb49R26AB4GjEcRx1w==", 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB/k59lEa9BBxAAjbrdbUDy56CxoPDtq4jonA==", 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lFZdBHhAAkFqOqF2A2YGGdnDSjvIYaw==", 2024-01-11T07:18:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luDNRJRAASADDnqn7TrCQghevAX5dhw==", 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:18:09.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lowQEIhAAFUWw7kO7XHFcsp5BKb3+1Q==", 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.871 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59lPersKhAAV3gTVdEj9Mezg7rVaNc3tA==", 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lCpqeGBAAQ/PJVPUZ9AhkvXzIiyV+Iw==", 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:18:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB5k59luh8XARAAMQo0nG9qHNY4OC5Mj1O1Dw==", 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:09.873 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQB4k59lKpO/OBAAs7yjDH8dL+9wQYWkiewEng==", 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:09.874 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f74066f30 msgr2=0x7f4f740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f74066f30 0x7f4f740693e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4f9006aff0 tx=0x7f4f80017000 comp rx=0 tx=0).stop 2024-01-11T07:18:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f90068830 msgr2=0x7f4f9015c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f90068830 0x7f4f9015c8e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00c590 tx=0x7f4f8c00c950 comp rx=0 tx=0).stop 2024-01-11T07:18:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 shutdown_connections 2024-01-11T07:18:09.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4f74066f30 0x7f4f740693e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f900c18a0 0x7f4f9006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f90068830 0x7f4f9015c8e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 --2- 172.21.15.7:0/3285473662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4f900ad9b0 0x7f4f9006ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 >> 172.21.15.7:0/3285473662 conn(0x7f4f90070690 msgr2=0x7f4f9015dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 shutdown_connections 2024-01-11T07:18:09.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:09.863+0000 7f4f97be1700 1 -- 172.21.15.7:0/3285473662 wait complete. 2024-01-11T07:18:09.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.009+0000 7f34c95f0700 1 Processor -- start 2024-01-11T07:18:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.009+0000 7f34c95f0700 1 -- start start 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.010+0000 7f34c95f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.010+0000 7f34c95f0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34c415a060 0x7f34c40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.010+0000 7f34c95f0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c40ce9b0 con 0x7f34c4069320 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.010+0000 7f34c95f0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c40cf530 con 0x7f34c415a060 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.010+0000 7f34c95f0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c40cf9c0 con 0x7f34c4159c90 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.010+0000 7f34c27fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34c415a060 0x7f34c40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.010+0000 7f34c27fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34c415a060 0x7f34c40b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55422/0 (socket says 172.21.15.7:55422) 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.010+0000 7f34c27fc700 1 -- 172.21.15.7:0/3060604323 learned_addr learned my addr 172.21.15.7:0/3060604323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c2ffd700 1 --2- 172.21.15.7:0/3060604323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c27fc700 1 -- 172.21.15.7:0/3060604323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 msgr2=0x7f34c40b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c27fc700 1 --2- 172.21.15.7:0/3060604323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c27fc700 1 -- 172.21.15.7:0/3060604323 >> v1:172.21.15.113:6789/0 conn(0x7f34c4069320 legacy=0x7f34c4067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c27fc700 1 -- 172.21.15.7:0/3060604323 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c40cec00 con 0x7f34c415a060 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c2ffd700 1 --2- 172.21.15.7:0/3060604323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c27fc700 1 --2- 172.21.15.7:0/3060604323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c415a060 0x7f34c40b0430 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0099b0 tx=0x7f34ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b068c1e599833efb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c1ffb700 1 -- 172.21.15.7:0/3060604323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00b820 con 0x7f34c415a060 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c1ffb700 1 -- 172.21.15.7:0/3060604323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac00b980 con 0x7f34c415a060 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.011+0000 7f34c1ffb700 1 -- 172.21.15.7:0/3060604323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00bd20 con 0x7f34c415a060 2024-01-11T07:18:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.012+0000 7f34c95f0700 1 -- 172.21.15.7:0/3060604323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c415a060 msgr2=0x7f34c40b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.012+0000 7f34c95f0700 1 --2- 172.21.15.7:0/3060604323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c415a060 0x7f34c40b0430 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0099b0 tx=0x7f34ac009250 comp rx=0 tx=0).stop 2024-01-11T07:18:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.012+0000 7f34c95f0700 1 -- 172.21.15.7:0/3060604323 shutdown_connections 2024-01-11T07:18:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.012+0000 7f34c95f0700 1 --2- 172.21.15.7:0/3060604323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.012+0000 7f34c95f0700 1 --2- 172.21.15.7:0/3060604323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c415a060 0x7f34c40b0430 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.012+0000 7f34c95f0700 1 -- 172.21.15.7:0/3060604323 >> 172.21.15.7:0/3060604323 conn(0x7f34c4071100 msgr2=0x7f34c406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.012+0000 7f34c95f0700 1 -- 172.21.15.7:0/3060604323 shutdown_connections 2024-01-11T07:18:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.012+0000 7f34c95f0700 1 -- 172.21.15.7:0/3060604323 wait complete. 2024-01-11T07:18:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.013+0000 7f34c95f0700 1 Processor -- start 2024-01-11T07:18:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.013+0000 7f34c95f0700 1 -- start start 2024-01-11T07:18:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c95f0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4069320 0x7f34c40b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c95f0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c95f0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34c415a060 0x7f34c40686d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c95f0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c416a350 con 0x7f34c4069320 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c95f0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c40af4b0 con 0x7f34c415a060 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c95f0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c40ae930 con 0x7f34c4159c90 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4069320 0x7f34c40b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4069320 0x7f34c40b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55438/0 (socket says 172.21.15.7:55438) 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c2ffd700 1 -- 172.21.15.7:0/1078555179 learned_addr learned my addr 172.21.15.7:0/1078555179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c37fe700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34c415a060 0x7f34c40686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c27fc700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c2ffd700 1 -- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 msgr2=0x7f34c40adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c2ffd700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c2ffd700 1 -- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34c415a060 msgr2=0x7f34c40686d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c2ffd700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34c415a060 0x7f34c40686d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c2ffd700 1 -- 172.21.15.7:0/1078555179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac0094f0 con 0x7f34c4069320 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.014+0000 7f34c37fe700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34c415a060 0x7f34c40686d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.015+0000 7f34c2ffd700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4069320 0x7f34c40b02e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f34b400c830 tx=0x7f34b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.015+0000 7f34baffd700 1 -- 172.21.15.7:0/1078555179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4011600 con 0x7f34c4069320 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.015+0000 7f34baffd700 1 -- 172.21.15.7:0/1078555179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b4011760 con 0x7f34c4069320 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.015+0000 7f34baffd700 1 -- 172.21.15.7:0/1078555179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4011b20 con 0x7f34c4069320 2024-01-11T07:18:10.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.015+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c40aec10 con 0x7f34c4069320 2024-01-11T07:18:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.015+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c40af1d0 con 0x7f34c4069320 2024-01-11T07:18:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.016+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c416a5c0 con 0x7f34c4069320 2024-01-11T07:18:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.016+0000 7f34baffd700 1 -- 172.21.15.7:0/1078555179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f34b400a3e0 con 0x7f34c4069320 2024-01-11T07:18:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.017+0000 7f34baffd700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a4066f30 0x7f34a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.017+0000 7f34baffd700 1 -- 172.21.15.7:0/1078555179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f34b40846a0 con 0x7f34c4069320 2024-01-11T07:18:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.017+0000 7f34c27fc700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a4066f30 0x7f34a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.018+0000 7f34c27fc700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a4066f30 0x7f34a40693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0099b0 tx=0x7f34ac016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.021+0000 7f34baffd700 1 -- 172.21.15.7:0/1078555179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34b4051c60 con 0x7f34c4069320 2024-01-11T07:18:10.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.226+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34c406e5c0 con 0x7f34c4069320 2024-01-11T07:18:10.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.227+0000 7f34baffd700 1 -- 172.21.15.7:0/1078555179 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+12097 (secure 0 0 0) 0x7f34b4055280 con 0x7f34c4069320 2024-01-11T07:18:10.227 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:10.227 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":180,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:09.401070+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a4066f30 msgr2=0x7f34a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a4066f30 0x7f34a40693e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0099b0 tx=0x7f34ac016000 comp rx=0 tx=0).stop 2024-01-11T07:18:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4069320 msgr2=0x7f34c40b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4069320 0x7f34c40b02e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f34b400c830 tx=0x7f34b4008040 comp rx=0 tx=0).stop 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 shutdown_connections 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f34a4066f30 0x7f34a40693e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34c415a060 0x7f34c40686d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4069320 0x7f34c40b02e0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 --2- 172.21.15.7:0/1078555179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f34c4159c90 0x7f34c40adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 >> 172.21.15.7:0/1078555179 conn(0x7f34c4071100 msgr2=0x7f34c4155110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 shutdown_connections 2024-01-11T07:18:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.229+0000 7f34c95f0700 1 -- 172.21.15.7:0/1078555179 wait complete. 2024-01-11T07:18:10.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-01-11T07:18:10.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:18:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.378+0000 7f0e109a2700 1 Processor -- start 2024-01-11T07:18:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.378+0000 7f0e109a2700 1 -- start start 2024-01-11T07:18:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e109a2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e0c0cd1f0 0x7f0e0c0689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e109a2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0c0c4600 con 0x7f0e0c068f00 2024-01-11T07:18:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e109a2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0c0c48a0 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e109a2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0c0cf7e0 con 0x7f0e0c0cd1f0 2024-01-11T07:18:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e0affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0e0c068f00 0x7f0e0c0c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60172/0 (socket says 172.21.15.7:60172) 2024-01-11T07:18:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e0affd700 1 -- 172.21.15.7:0/3556453336 learned_addr learned my addr 172.21.15.7:0/3556453336 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e0b7fe700 1 --2- 172.21.15.7:0/3556453336 >> v2:172.21.15.113:3300/0 conn(0x7f0e0c0cd1f0 0x7f0e0c0689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e0a7fc700 1 -- 172.21.15.7:0/3556453336 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210057711 0 0) 0x7f0e0c0c4600 con 0x7f0e0c068f00 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e0a7fc700 1 -- 172.21.15.7:0/3556453336 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df8003570 con 0x7f0e0c068f00 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e0b7fe700 1 -- 172.21.15.7:0/3556453336 >> v1:172.21.15.113:6789/0 conn(0x7f0e0c0c95a0 legacy=0x7f0e0c1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e0b7fe700 1 -- 172.21.15.7:0/3556453336 >> v1:172.21.15.7:6789/0 conn(0x7f0e0c068f00 legacy=0x7f0e0c0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.379+0000 7f0e0b7fe700 1 -- 172.21.15.7:0/3556453336 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c0c4b80 con 0x7f0e0c0cd1f0 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e0b7fe700 1 --2- 172.21.15.7:0/3556453336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 0x7f0e0c0689c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0df40098d0 tx=0x7f0df4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f623e1c5857701 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e0a7fc700 1 -- 172.21.15.7:0/3556453336 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4005c10 con 0x7f0e0c0cd1f0 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e0a7fc700 1 -- 172.21.15.7:0/3556453336 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df4005d70 con 0x7f0e0c0cd1f0 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e0a7fc700 1 -- 172.21.15.7:0/3556453336 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4006280 con 0x7f0e0c0cd1f0 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e109a2700 1 -- 172.21.15.7:0/3556453336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 msgr2=0x7f0e0c0689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e109a2700 1 --2- 172.21.15.7:0/3556453336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 0x7f0e0c0689c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0df40098d0 tx=0x7f0df4009120 comp rx=0 tx=0).stop 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e109a2700 1 -- 172.21.15.7:0/3556453336 shutdown_connections 2024-01-11T07:18:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e109a2700 1 --2- 172.21.15.7:0/3556453336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 0x7f0e0c0689c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.380+0000 7f0e109a2700 1 -- 172.21.15.7:0/3556453336 >> 172.21.15.7:0/3556453336 conn(0x7f0e0c097a50 msgr2=0x7f0e0c0ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:10.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.381+0000 7f0e109a2700 1 -- 172.21.15.7:0/3556453336 shutdown_connections 2024-01-11T07:18:10.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.381+0000 7f0e109a2700 1 -- 172.21.15.7:0/3556453336 wait complete. 2024-01-11T07:18:10.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.382+0000 7f0e109a2700 1 Processor -- start 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.382+0000 7f0e109a2700 1 -- start start 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e109a2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c068f00 0x7f0e0c09aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e109a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c95a0 0x7f0e0c148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e109a2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 0x7f0e0c0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e109a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c148c80 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e109a2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c09c250 con 0x7f0e0c0cd1f0 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e109a2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c09b640 con 0x7f0e0c068f00 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c95a0 0x7f0e0c148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c95a0 0x7f0e0c148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55444/0 (socket says 172.21.15.7:55444) 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 -- 172.21.15.7:0/3860014835 learned_addr learned my addr 172.21.15.7:0/3860014835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0bfff700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 0x7f0e0c0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0b7fe700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c068f00 0x7f0e0c09aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 -- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c068f00 msgr2=0x7f0e0c09aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c068f00 0x7f0e0c09aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 -- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 msgr2=0x7f0e0c0c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 0x7f0e0c0c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 -- 172.21.15.7:0/3860014835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40093c0 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e0affd700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c95a0 0x7f0e0c148740 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00c7c0 tx=0x7f0dfc007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.383+0000 7f0e037fe700 1 -- 172.21.15.7:0/3860014835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc005c00 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.384+0000 7f0e037fe700 1 -- 172.21.15.7:0/3860014835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc005d60 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.384+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c09b920 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.384+0000 7f0e037fe700 1 -- 172.21.15.7:0/3860014835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc007320 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.384+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c09bee0 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.385+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c0cb800 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.385+0000 7f0e037fe700 1 -- 172.21.15.7:0/3860014835 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0dfc007690 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.385+0000 7f0e037fe700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0dec066f30 0x7f0dec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.385+0000 7f0e037fe700 1 -- 172.21.15.7:0/3860014835 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0dfc083670 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.386+0000 7f0e0b7fe700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0dec066f30 0x7f0dec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.386+0000 7f0e0b7fe700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0dec066f30 0x7f0dec0693e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009390 tx=0x7f0df4016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.390+0000 7f0e037fe700 1 -- 172.21.15.7:0/3860014835 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0dfc050c30 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.595+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e0c09c3c0 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.596+0000 7f0e037fe700 1 -- 172.21.15.7:0/3860014835 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0dfc017020 con 0x7f0e0c0c95a0 2024-01-11T07:18:10.596 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:18:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0dec066f30 msgr2=0x7f0dec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0dec066f30 0x7f0dec0693e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009390 tx=0x7f0df4016000 comp rx=0 tx=0).stop 2024-01-11T07:18:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c95a0 msgr2=0x7f0e0c148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c95a0 0x7f0e0c148740 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00c7c0 tx=0x7f0dfc007dd0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 shutdown_connections 2024-01-11T07:18:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0dec066f30 0x7f0dec0693e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0c0cd1f0 0x7f0e0c0c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e0c068f00 0x7f0e0c09aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 --2- 172.21.15.7:0/3860014835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c95a0 0x7f0e0c148740 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 >> 172.21.15.7:0/3860014835 conn(0x7f0e0c097a50 msgr2=0x7f0e0c156800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 shutdown_connections 2024-01-11T07:18:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.598+0000 7f0e109a2700 1 -- 172.21.15.7:0/3860014835 wait complete. 2024-01-11T07:18:10.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-11T07:18:10.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.743+0000 7f2974f83700 1 Processor -- start 2024-01-11T07:18:10.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.743+0000 7f2974f83700 1 -- start start 2024-01-11T07:18:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f2974f83700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f2974f83700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f297015e9c0 0x7f29700cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f2974f83700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297015e530 con 0x7f2970071130 2024-01-11T07:18:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f2974f83700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29700cc970 con 0x7f297015e9c0 2024-01-11T07:18:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f2974f83700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29700cc690 con 0x7f2970159f50 2024-01-11T07:18:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f296ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2970071130 0x7f297015b910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60184/0 (socket says 172.21.15.7:60184) 2024-01-11T07:18:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f296ed9d700 1 -- 172.21.15.7:0/339492036 learned_addr learned my addr 172.21.15.7:0/339492036 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f296e59c700 1 --2- 172.21.15.7:0/339492036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.744+0000 7f2967fff700 1 --2- 172.21.15.7:0/339492036 >> v2:172.21.15.113:3300/0 conn(0x7f297015e9c0 0x7f29700cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296dd9b700 1 -- 172.21.15.7:0/339492036 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652529101 0 0) 0x7f297015e530 con 0x7f2970071130 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296dd9b700 1 -- 172.21.15.7:0/339492036 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f295c003570 con 0x7f2970071130 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296e59c700 1 -- 172.21.15.7:0/339492036 >> v2:172.21.15.113:3300/0 conn(0x7f297015e9c0 msgr2=0x7f29700cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296e59c700 1 --2- 172.21.15.7:0/339492036 >> v2:172.21.15.113:3300/0 conn(0x7f297015e9c0 0x7f29700cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296e59c700 1 -- 172.21.15.7:0/339492036 >> v1:172.21.15.7:6789/0 conn(0x7f2970071130 legacy=0x7f297015b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296e59c700 1 -- 172.21.15.7:0/339492036 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297015dee0 con 0x7f2970159f50 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296e59c700 1 --2- 172.21.15.7:0/339492036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cbe40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f29600098e0 tx=0x7f29600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcffdf3b1134c8c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296dd9b700 1 -- 172.21.15.7:0/339492036 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960011630 con 0x7f2970159f50 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296dd9b700 1 -- 172.21.15.7:0/339492036 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2960011790 con 0x7f2970159f50 2024-01-11T07:18:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.745+0000 7f296dd9b700 1 -- 172.21.15.7:0/339492036 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960011b30 con 0x7f2970159f50 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.746+0000 7f2974f83700 1 -- 172.21.15.7:0/339492036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 msgr2=0x7f29700cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.746+0000 7f2974f83700 1 --2- 172.21.15.7:0/339492036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cbe40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f29600098e0 tx=0x7f29600091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.746+0000 7f2974f83700 1 -- 172.21.15.7:0/339492036 shutdown_connections 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.746+0000 7f2974f83700 1 --2- 172.21.15.7:0/339492036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cbe40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.746+0000 7f2974f83700 1 --2- 172.21.15.7:0/339492036 >> v2:172.21.15.113:3300/0 conn(0x7f297015e9c0 0x7f29700cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.746+0000 7f2974f83700 1 -- 172.21.15.7:0/339492036 >> 172.21.15.7:0/339492036 conn(0x7f297006e6b0 msgr2=0x7f2970070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.746+0000 7f2974f83700 1 -- 172.21.15.7:0/339492036 shutdown_connections 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.746+0000 7f2974f83700 1 -- 172.21.15.7:0/339492036 wait complete. 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.747+0000 7f2974f83700 1 Processor -- start 2024-01-11T07:18:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.747+0000 7f2974f83700 1 -- start start 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f2974f83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2970071130 0x7f29700d0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f2974f83700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f2974f83700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297015e9c0 0x7f2970152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f2974f83700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2970154680 con 0x7f2970071130 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f2974f83700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700cea00 con 0x7f297015e9c0 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f2974f83700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29700cdb70 con 0x7f2970159f50 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2970071130 0x7f29700d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2970071130 0x7f29700d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55460/0 (socket says 172.21.15.7:55460) 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296e59c700 1 -- 172.21.15.7:0/3100923274 learned_addr learned my addr 172.21.15.7:0/3100923274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f2967fff700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296ed9d700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297015e9c0 0x7f2970152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296e59c700 1 -- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 msgr2=0x7f29700cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296e59c700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296e59c700 1 -- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297015e9c0 msgr2=0x7f2970152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296e59c700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297015e9c0 0x7f2970152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f296e59c700 1 -- 172.21.15.7:0/3100923274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2960009420 con 0x7f2970071130 2024-01-11T07:18:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.748+0000 7f2967fff700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.749+0000 7f296e59c700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2970071130 0x7f29700d0320 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2960009ed0 tx=0x7f2960017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.749+0000 7f29667fc700 1 -- 172.21.15.7:0/3100923274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296001f070 con 0x7f2970071130 2024-01-11T07:18:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.749+0000 7f29667fc700 1 -- 172.21.15.7:0/3100923274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2960000c90 con 0x7f2970071130 2024-01-11T07:18:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.749+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29700cddf0 con 0x7f2970071130 2024-01-11T07:18:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.749+0000 7f29667fc700 1 -- 172.21.15.7:0/3100923274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2960011b30 con 0x7f2970071130 2024-01-11T07:18:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.749+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29700ce3b0 con 0x7f2970071130 2024-01-11T07:18:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.750+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2970089350 con 0x7f2970071130 2024-01-11T07:18:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.750+0000 7f29667fc700 1 -- 172.21.15.7:0/3100923274 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2960023070 con 0x7f2970071130 2024-01-11T07:18:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.751+0000 7f29667fc700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2950066f30 0x7f29500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.751+0000 7f29667fc700 1 -- 172.21.15.7:0/3100923274 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f29600856c0 con 0x7f2970071130 2024-01-11T07:18:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.751+0000 7f2967fff700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2950066f30 0x7f29500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:10.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.752+0000 7f2967fff700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2950066f30 0x7f29500693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f29700b24b0 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:10.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.755+0000 7f29667fc700 1 -- 172.21.15.7:0/3100923274 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2960052be0 con 0x7f2970071130 2024-01-11T07:18:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.961+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f29700890f0 con 0x7f2970071130 2024-01-11T07:18:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.961+0000 7f29667fc700 1 -- 172.21.15.7:0/3100923274 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v180) v1 ==== 84+0+65 (secure 0 0 0) 0x7f296002a090 con 0x7f2970071130 2024-01-11T07:18:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:10.961 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-01-11T07:18:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-01-11T07:18:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-01-11T07:18:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:10.962 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2950066f30 msgr2=0x7f29500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2950066f30 0x7f29500693e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f29700b24b0 tx=0x7f2958008040 comp rx=0 tx=0).stop 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2970071130 msgr2=0x7f29700d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2970071130 0x7f29700d0320 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2960009ed0 tx=0x7f2960017000 comp rx=0 tx=0).stop 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 shutdown_connections 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2950066f30 0x7f29500693e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f297015e9c0 0x7f2970152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2970071130 0x7f29700d0320 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 --2- 172.21.15.7:0/3100923274 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2970159f50 0x7f29700cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.963+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 >> 172.21.15.7:0/3100923274 conn(0x7f297006e6b0 msgr2=0x7f29701538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.964+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 shutdown_connections 2024-01-11T07:18:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:10.964+0000 7f2974f83700 1 -- 172.21.15.7:0/3100923274 wait complete. 2024-01-11T07:18:10.977 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-11T07:18:10.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-11T07:18:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.113+0000 7fcc0ffb4700 1 Processor -- start 2024-01-11T07:18:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.113+0000 7fcc0ffb4700 1 -- start start 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0ffb4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcc081567f0 0x7fcc080c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0ffb4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc080cbc70 0x7fcc0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0ffb4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0806df00 con 0x7fcc08156bc0 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0ffb4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc080c8bb0 con 0x7fcc081567f0 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0ffb4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0806e150 con 0x7fcc080cbc70 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0d54f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcc08156bc0 0x7fcc0806a5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60200/0 (socket says 172.21.15.7:60200) 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0d54f700 1 -- 172.21.15.7:0/2236659062 learned_addr learned my addr 172.21.15.7:0/2236659062 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0e551700 1 --2- 172.21.15.7:0/2236659062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc080cbc70 0x7fcc0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0dd50700 1 --2- 172.21.15.7:0/2236659062 >> v2:172.21.15.113:3300/0 conn(0x7fcc081567f0 0x7fcc080c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0cd4e700 1 -- 172.21.15.7:0/2236659062 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322963611 0 0) 0x7fcc0806df00 con 0x7fcc08156bc0 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.114+0000 7fcc0cd4e700 1 -- 172.21.15.7:0/2236659062 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf4003570 con 0x7fcc08156bc0 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0cd4e700 1 -- 172.21.15.7:0/2236659062 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2426977009 0 0) 0x7fcbf4003570 con 0x7fcc08156bc0 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0e551700 1 -- 172.21.15.7:0/2236659062 >> v2:172.21.15.113:3300/0 conn(0x7fcc081567f0 msgr2=0x7fcc080c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0e551700 1 --2- 172.21.15.7:0/2236659062 >> v2:172.21.15.113:3300/0 conn(0x7fcc081567f0 0x7fcc080c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0e551700 1 -- 172.21.15.7:0/2236659062 >> v1:172.21.15.7:6789/0 conn(0x7fcc08156bc0 legacy=0x7fcc0806a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0e551700 1 -- 172.21.15.7:0/2236659062 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0806e430 con 0x7fcc080cbc70 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0e551700 1 --2- 172.21.15.7:0/2236659062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc080cbc70 0x7fcc0806d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009e60 tx=0x7fcc00009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d59f906069d8fa server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0cd4e700 1 -- 172.21.15.7:0/2236659062 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000e070 con 0x7fcc080cbc70 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0cd4e700 1 -- 172.21.15.7:0/2236659062 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000b7a0 con 0x7fcc080cbc70 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.115+0000 7fcc0cd4e700 1 -- 172.21.15.7:0/2236659062 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8001470 con 0x7fcc080cbc70 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.116+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/2236659062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc080cbc70 msgr2=0x7fcc0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.116+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/2236659062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc080cbc70 0x7fcc0806d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009e60 tx=0x7fcc00009780 comp rx=0 tx=0).stop 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.116+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/2236659062 shutdown_connections 2024-01-11T07:18:11.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.116+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/2236659062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc080cbc70 0x7fcc0806d9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.116+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/2236659062 >> v2:172.21.15.113:3300/0 conn(0x7fcc081567f0 0x7fcc080c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.116+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/2236659062 >> 172.21.15.7:0/2236659062 conn(0x7fcc080680b0 msgr2=0x7fcc080cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.116+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/2236659062 shutdown_connections 2024-01-11T07:18:11.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.116+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/2236659062 wait complete. 2024-01-11T07:18:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.117+0000 7fcc0ffb4700 1 Processor -- start 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.117+0000 7fcc0ffb4700 1 -- start start 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0ffb4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc080cbc70 0x7fcc08154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0ffb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc081567f0 0x7fcc080b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0ffb4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc08156bc0 0x7fcc080b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0ffb4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081587a0 con 0x7fcc081567f0 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0ffb4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08154e40 con 0x7fcc080cbc70 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0ffb4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08157b20 con 0x7fcc08156bc0 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0d54f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc081567f0 0x7fcc080b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0d54f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc081567f0 0x7fcc080b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55476/0 (socket says 172.21.15.7:55476) 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0d54f700 1 -- 172.21.15.7:0/1442551000 learned_addr learned my addr 172.21.15.7:0/1442551000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0e551700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc08156bc0 0x7fcc080b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0dd50700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc080cbc70 0x7fcc08154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0d54f700 1 -- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc08156bc0 msgr2=0x7fcc080b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0d54f700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc08156bc0 0x7fcc080b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0d54f700 1 -- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc080cbc70 msgr2=0x7fcc08154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0d54f700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc080cbc70 0x7fcc08154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.118+0000 7fcc0d54f700 1 -- 172.21.15.7:0/1442551000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc00009a20 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.119+0000 7fcc0d54f700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc081567f0 0x7fcc080b1e40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c960 tx=0x7fcbf8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.119+0000 7fcbfdffb700 1 -- 172.21.15.7:0/1442551000 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8005b30 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.119+0000 7fcbfdffb700 1 -- 172.21.15.7:0/1442551000 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf8005c90 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.119+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc08157e00 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.119+0000 7fcbfdffb700 1 -- 172.21.15.7:0/1442551000 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8007320 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.119+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc081583c0 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.120+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0804c3c0 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.120+0000 7fcbfdffb700 1 -- 172.21.15.7:0/1442551000 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcbf801b400 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.121+0000 7fcbfdffb700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbec066f30 0x7fcbec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.121+0000 7fcbfdffb700 1 -- 172.21.15.7:0/1442551000 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcbf8083ea0 con 0x7fcc081567f0 2024-01-11T07:18:11.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.121+0000 7fcc0dd50700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbec066f30 0x7fcbec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.121+0000 7fcc0dd50700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbec066f30 0x7fcbec0693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcc080ce030 tx=0x7fcc040091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.125+0000 7fcbfdffb700 1 -- 172.21.15.7:0/1442551000 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbf8051460 con 0x7fcc081567f0 2024-01-11T07:18:11.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.328+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcc0806e710 con 0x7fcc081567f0 2024-01-11T07:18:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.417+0000 7fcbfdffb700 1 -- 172.21.15.7:0/1442551000 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v181) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcbf8054a80 con 0x7fcc081567f0 2024-01-11T07:18:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbec066f30 msgr2=0x7fcbec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbec066f30 0x7fcbec0693e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcc080ce030 tx=0x7fcc040091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc081567f0 msgr2=0x7fcc080b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc081567f0 0x7fcc080b1e40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c960 tx=0x7fcbf8001a00 comp rx=0 tx=0).stop 2024-01-11T07:18:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 shutdown_connections 2024-01-11T07:18:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcbec066f30 0x7fcbec0693e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcc08156bc0 0x7fcc080b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc080cbc70 0x7fcc08154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 --2- 172.21.15.7:0/1442551000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc081567f0 0x7fcc080b1e40 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 >> 172.21.15.7:0/1442551000 conn(0x7fcc080680b0 msgr2=0x7fcc08155960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 shutdown_connections 2024-01-11T07:18:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.421+0000 7fcc0ffb4700 1 -- 172.21.15.7:0/1442551000 wait complete. 2024-01-11T07:18:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-01-11T07:18:11.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-11T07:18:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.575+0000 7f6b341c0700 1 Processor -- start 2024-01-11T07:18:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.575+0000 7f6b341c0700 1 -- start start 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.576+0000 7f6b341c0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b2c063230 0x7f6b2c063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.576+0000 7f6b341c0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.576+0000 7f6b341c0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0ab100 con 0x7f6b2c15a280 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.576+0000 7f6b341c0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c15d880 con 0x7f6b2c063230 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.576+0000 7f6b341c0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c06c5b0 con 0x7f6b2c0ac5b0 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.576+0000 7f6b31f5c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b2c063230 0x7f6b2c063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.576+0000 7f6b31f5c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b2c063230 0x7f6b2c063600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55478/0 (socket says 172.21.15.7:55478) 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.576+0000 7f6b31f5c700 1 -- 172.21.15.7:0/1089026652 learned_addr learned my addr 172.21.15.7:0/1089026652 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b3275d700 1 --2- 172.21.15.7:0/1089026652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b31f5c700 1 -- 172.21.15.7:0/1089026652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c0ac5b0 msgr2=0x7f6b2c0aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b31f5c700 1 --2- 172.21.15.7:0/1089026652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b31f5c700 1 -- 172.21.15.7:0/1089026652 >> v1:172.21.15.113:6789/0 conn(0x7f6b2c15a280 legacy=0x7f6b2c0aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b31f5c700 1 -- 172.21.15.7:0/1089026652 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c063b40 con 0x7f6b2c063230 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b3275d700 1 --2- 172.21.15.7:0/1089026652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b31f5c700 1 --2- 172.21.15.7:0/1089026652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c063600 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6b280099b0 tx=0x7f6b28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec282502cecb43a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b30f5a700 1 -- 172.21.15.7:0/1089026652 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800b820 con 0x7f6b2c063230 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b30f5a700 1 -- 172.21.15.7:0/1089026652 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2800b980 con 0x7f6b2c063230 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.577+0000 7f6b30f5a700 1 -- 172.21.15.7:0/1089026652 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800bd20 con 0x7f6b2c063230 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.578+0000 7f6b341c0700 1 -- 172.21.15.7:0/1089026652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 msgr2=0x7f6b2c063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.578+0000 7f6b341c0700 1 --2- 172.21.15.7:0/1089026652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c063600 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6b280099b0 tx=0x7f6b28009250 comp rx=0 tx=0).stop 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.578+0000 7f6b341c0700 1 -- 172.21.15.7:0/1089026652 shutdown_connections 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.578+0000 7f6b341c0700 1 --2- 172.21.15.7:0/1089026652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.578+0000 7f6b341c0700 1 --2- 172.21.15.7:0/1089026652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c063600 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.578+0000 7f6b341c0700 1 -- 172.21.15.7:0/1089026652 >> 172.21.15.7:0/1089026652 conn(0x7f6b2c0680e0 msgr2=0x7f6b2c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.578+0000 7f6b341c0700 1 -- 172.21.15.7:0/1089026652 shutdown_connections 2024-01-11T07:18:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.578+0000 7f6b341c0700 1 -- 172.21.15.7:0/1089026652 wait complete. 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.579+0000 7f6b341c0700 1 Processor -- start 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.579+0000 7f6b341c0700 1 -- start start 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b341c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b341c0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b341c0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c15a280 0x7f6b2c0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b341c0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c3c10 con 0x7f6b2c063230 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b341c0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c1542f0 con 0x7f6b2c0ac5b0 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b341c0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c3000 con 0x7f6b2c15a280 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55488/0 (socket says 172.21.15.7:55488) 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 -- 172.21.15.7:0/4287713879 learned_addr learned my addr 172.21.15.7:0/4287713879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b3275d700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c15a280 0x7f6b2c0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b3175b700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 -- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c15a280 msgr2=0x7f6b2c0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c15a280 0x7f6b2c0b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 -- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2c0ac5b0 msgr2=0x7f6b2c0adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 -- 172.21.15.7:0/4287713879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280094f0 con 0x7f6b2c063230 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.580+0000 7f6b31f5c700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c153db0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009290 tx=0x7f6b28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.581+0000 7f6b21ffb700 1 -- 172.21.15.7:0/4287713879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28011d70 con 0x7f6b2c063230 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.581+0000 7f6b21ffb700 1 -- 172.21.15.7:0/4287713879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28020070 con 0x7f6b2c063230 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.581+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c0c32e0 con 0x7f6b2c063230 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.581+0000 7f6b21ffb700 1 -- 172.21.15.7:0/4287713879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2801b6a0 con 0x7f6b2c063230 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.581+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c0c38a0 con 0x7f6b2c063230 2024-01-11T07:18:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.582+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c04c3c0 con 0x7f6b2c063230 2024-01-11T07:18:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.583+0000 7f6b21ffb700 1 -- 172.21.15.7:0/4287713879 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6b28022030 con 0x7f6b2c063230 2024-01-11T07:18:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.583+0000 7f6b21ffb700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b10066f30 0x7f6b100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.583+0000 7f6b21ffb700 1 -- 172.21.15.7:0/4287713879 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f6b28086640 con 0x7f6b2c063230 2024-01-11T07:18:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.583+0000 7f6b3175b700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b10066f30 0x7f6b100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.584+0000 7f6b3175b700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b10066f30 0x7f6b100693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c16daf0 tx=0x7f6b1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.587+0000 7f6b21ffb700 1 -- 172.21.15.7:0/4287713879 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b28053b10 con 0x7f6b2c063230 2024-01-11T07:18:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:11.791+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6b2c04c3c0 con 0x7f6b2c063230 2024-01-11T07:18:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.423+0000 7f6b21ffb700 1 -- 172.21.15.7:0/4287713879 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v182) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6b28057130 con 0x7f6b2c063230 2024-01-11T07:18:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.432+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b10066f30 msgr2=0x7f6b100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.432+0000 7f6b341c0700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b10066f30 0x7f6b100693e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c16daf0 tx=0x7f6b1c009040 comp rx=0 tx=0).stop 2024-01-11T07:18:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.432+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 msgr2=0x7f6b2c153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.432+0000 7f6b341c0700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c153db0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009290 tx=0x7f6b28016000 comp rx=0 tx=0).stop 2024-01-11T07:18:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.434+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 shutdown_connections 2024-01-11T07:18:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.434+0000 7f6b341c0700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6b10066f30 0x7f6b100693e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.434+0000 7f6b341c0700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6b2c15a280 0x7f6b2c0b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.434+0000 7f6b341c0700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c063230 0x7f6b2c153db0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.434+0000 7f6b341c0700 1 --2- 172.21.15.7:0/4287713879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2c0ac5b0 0x7f6b2c0adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.434+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 >> 172.21.15.7:0/4287713879 conn(0x7f6b2c0680e0 msgr2=0x7f6b2c15b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.435+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 shutdown_connections 2024-01-11T07:18:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.435+0000 7f6b341c0700 1 -- 172.21.15.7:0/4287713879 wait complete. 2024-01-11T07:18:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-01-11T07:18:12.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-11T07:18:12.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.591+0000 7f47e30f9700 1 Processor -- start 2024-01-11T07:18:12.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.591+0000 7f47e30f9700 1 -- start start 2024-01-11T07:18:12.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47e30f9700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f47dc0ac310 0x7f47dc0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47e30f9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47dc15e540 0x7f47dc15d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47e30f9700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc15e2c0 con 0x7f47dc15de60 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47e30f9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc06a020 con 0x7f47dc15e540 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47e30f9700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc06aba0 con 0x7f47dc0ac310 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47dbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47dc15e540 0x7f47dc15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47dbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47dc15e540 0x7f47dc15d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55492/0 (socket says 172.21.15.7:55492) 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47dbfff700 1 -- 172.21.15.7:0/2896849735 learned_addr learned my addr 172.21.15.7:0/2896849735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.592+0000 7f47e0e95700 1 --2- 172.21.15.7:0/2896849735 >> v2:172.21.15.113:3300/0 conn(0x7f47dc0ac310 0x7f47dc0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.593+0000 7f47dbfff700 1 -- 172.21.15.7:0/2896849735 >> v2:172.21.15.113:3300/0 conn(0x7f47dc0ac310 msgr2=0x7f47dc0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.593+0000 7f47dbfff700 1 --2- 172.21.15.7:0/2896849735 >> v2:172.21.15.113:3300/0 conn(0x7f47dc0ac310 0x7f47dc0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.593+0000 7f47dbfff700 1 -- 172.21.15.7:0/2896849735 >> v1:172.21.15.113:6789/0 conn(0x7f47dc15de60 legacy=0x7f47dc15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.593+0000 7f47dbfff700 1 -- 172.21.15.7:0/2896849735 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc06a270 con 0x7f47dc15e540 2024-01-11T07:18:12.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.593+0000 7f47dbfff700 1 --2- 172.21.15.7:0/2896849735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15e540 0x7f47dc15d920 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0714e0 tx=0x7f47c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4c72ab6cc798d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.593+0000 7f47db7fe700 1 -- 172.21.15.7:0/2896849735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400b860 con 0x7f47dc15e540 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.593+0000 7f47db7fe700 1 -- 172.21.15.7:0/2896849735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c400b9c0 con 0x7f47dc15e540 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.593+0000 7f47db7fe700 1 -- 172.21.15.7:0/2896849735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400bd60 con 0x7f47dc15e540 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.594+0000 7f47e30f9700 1 -- 172.21.15.7:0/2896849735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15e540 msgr2=0x7f47dc15d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.594+0000 7f47e30f9700 1 --2- 172.21.15.7:0/2896849735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15e540 0x7f47dc15d920 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0714e0 tx=0x7f47c4009250 comp rx=0 tx=0).stop 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.594+0000 7f47e30f9700 1 -- 172.21.15.7:0/2896849735 shutdown_connections 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.594+0000 7f47e30f9700 1 --2- 172.21.15.7:0/2896849735 >> v2:172.21.15.113:3300/0 conn(0x7f47dc0ac310 0x7f47dc0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.594+0000 7f47e30f9700 1 --2- 172.21.15.7:0/2896849735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15e540 0x7f47dc15d920 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.594+0000 7f47e30f9700 1 -- 172.21.15.7:0/2896849735 >> 172.21.15.7:0/2896849735 conn(0x7f47dc0cd090 msgr2=0x7f47dc0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.594+0000 7f47e30f9700 1 -- 172.21.15.7:0/2896849735 shutdown_connections 2024-01-11T07:18:12.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.594+0000 7f47e30f9700 1 -- 172.21.15.7:0/2896849735 wait complete. 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.595+0000 7f47e30f9700 1 Processor -- start 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.595+0000 7f47e30f9700 1 -- start start 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47e30f9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47dc0ac310 0x7f47dc13fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47e30f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15de60 0x7f47dc157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47e30f9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47dc15e540 0x7f47dc15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47e30f9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0b06d0 con 0x7f47dc15de60 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47e30f9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0b09b0 con 0x7f47dc0ac310 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47e30f9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0b0c90 con 0x7f47dc15e540 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15de60 0x7f47dc157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15de60 0x7f47dc157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55502/0 (socket says 172.21.15.7:55502) 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47dbfff700 1 -- 172.21.15.7:0/1966714457 learned_addr learned my addr 172.21.15.7:0/1966714457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:12.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47e0e95700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47dc0ac310 0x7f47dc13fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47e1696700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47dc15e540 0x7f47dc15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47dbfff700 1 -- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47dc15e540 msgr2=0x7f47dc15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47dbfff700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47dc15e540 0x7f47dc15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47dbfff700 1 -- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47dc0ac310 msgr2=0x7f47dc13fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47dbfff700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47dc0ac310 0x7f47dc13fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.596+0000 7f47dbfff700 1 -- 172.21.15.7:0/1966714457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40094f0 con 0x7f47dc15de60 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.597+0000 7f47dbfff700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15de60 0x7f47dc157d50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f47dc06bf20 tx=0x7f47c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.597+0000 7f47d8ff9700 1 -- 172.21.15.7:0/1966714457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4024440 con 0x7f47dc15de60 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.597+0000 7f47d8ff9700 1 -- 172.21.15.7:0/1966714457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c40245a0 con 0x7f47dc15de60 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.597+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc0b0f70 con 0x7f47dc15de60 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.597+0000 7f47d8ff9700 1 -- 172.21.15.7:0/1966714457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4024960 con 0x7f47dc15de60 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.597+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc140240 con 0x7f47dc15de60 2024-01-11T07:18:12.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.598+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc04c3c0 con 0x7f47dc15de60 2024-01-11T07:18:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.598+0000 7f47d8ff9700 1 -- 172.21.15.7:0/1966714457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f47c401a420 con 0x7f47dc15de60 2024-01-11T07:18:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.599+0000 7f47d8ff9700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47bc066f30 0x7f47bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.599+0000 7f47d8ff9700 1 -- 172.21.15.7:0/1966714457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f47c4086360 con 0x7f47dc15de60 2024-01-11T07:18:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.599+0000 7f47e0e95700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47bc066f30 0x7f47bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:12.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.600+0000 7f47e0e95700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47bc066f30 0x7f47bc0693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006fd0 tx=0x7f47cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:12.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.604+0000 7f47d8ff9700 1 -- 172.21.15.7:0/1966714457 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47c4053630 con 0x7f47dc15de60 2024-01-11T07:18:12.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:12.852+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f47dc0b2ec0 con 0x7f47dc15de60 2024-01-11T07:18:13.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.429+0000 7f47d8ff9700 1 -- 172.21.15.7:0/1966714457 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v86) v1 ==== 174+0+0 (secure 0 0 0) 0x7f47c4056c50 con 0x7f47dc15de60 2024-01-11T07:18:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47bc066f30 msgr2=0x7f47bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47bc066f30 0x7f47bc0693e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006fd0 tx=0x7f47cc008040 comp rx=0 tx=0).stop 2024-01-11T07:18:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15de60 msgr2=0x7f47dc157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15de60 0x7f47dc157d50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f47dc06bf20 tx=0x7f47c4017000 comp rx=0 tx=0).stop 2024-01-11T07:18:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 shutdown_connections 2024-01-11T07:18:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f47bc066f30 0x7f47bc0693e0 secure :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006fd0 tx=0x7f47cc008040 comp rx=0 tx=0).stop 2024-01-11T07:18:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47dc15e540 0x7f47dc15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47dc0ac310 0x7f47dc13fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 --2- 172.21.15.7:0/1966714457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47dc15de60 0x7f47dc157d50 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.440+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 >> 172.21.15.7:0/1966714457 conn(0x7f47dc0cd090 msgr2=0x7f47dc070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.441+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 shutdown_connections 2024-01-11T07:18:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.443+0000 7f47e30f9700 1 -- 172.21.15.7:0/1966714457 wait complete. 2024-01-11T07:18:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 22 and data pool 23 2024-01-11T07:18:13.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.597+0000 7f219ff68700 1 Processor -- start 2024-01-11T07:18:13.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.598+0000 7f219ff68700 1 -- start start 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.598+0000 7f219ff68700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2198159a50 0x7f2198159e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.598+0000 7f219ff68700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f219815e4a0 0x7f21980ac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.598+0000 7f219ff68700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21980ac990 0x7f219806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.598+0000 7f219ff68700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219815e910 con 0x7f21980ac990 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.598+0000 7f219ff68700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219815e010 con 0x7f219815e4a0 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.598+0000 7f219ff68700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21980acf80 con 0x7f2198159a50 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21980ac990 0x7f219806bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21980ac990 0x7f219806bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55506/0 (socket says 172.21.15.7:55506) 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 -- 172.21.15.7:0/371616964 learned_addr learned my addr 172.21.15.7:0/371616964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219d503700 1 --2- 172.21.15.7:0/371616964 >> v2:172.21.15.113:3300/0 conn(0x7f219815e4a0 0x7f21980ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219dd04700 1 --2- 172.21.15.7:0/371616964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2198159a50 0x7f2198159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 -- 172.21.15.7:0/371616964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2198159a50 msgr2=0x7f2198159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 --2- 172.21.15.7:0/371616964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2198159a50 0x7f2198159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 -- 172.21.15.7:0/371616964 >> v2:172.21.15.113:3300/0 conn(0x7f219815e4a0 msgr2=0x7f21980ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 --2- 172.21.15.7:0/371616964 >> v2:172.21.15.113:3300/0 conn(0x7f219815e4a0 0x7f21980ac450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 -- 172.21.15.7:0/371616964 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219815d9c0 con 0x7f21980ac990 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219e505700 1 --2- 172.21.15.7:0/371616964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21980ac990 0x7f219806bc50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2198068ff0 tx=0x7f2190009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3e8d676c729e1b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219cd02700 1 -- 172.21.15.7:0/371616964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219000e070 con 0x7f21980ac990 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219cd02700 1 -- 172.21.15.7:0/371616964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219000b970 con 0x7f21980ac990 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.599+0000 7f219cd02700 1 -- 172.21.15.7:0/371616964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21900092e0 con 0x7f21980ac990 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 -- 172.21.15.7:0/371616964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21980ac990 msgr2=0x7f219806bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 --2- 172.21.15.7:0/371616964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21980ac990 0x7f219806bc50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2198068ff0 tx=0x7f2190009840 comp rx=0 tx=0).stop 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 -- 172.21.15.7:0/371616964 shutdown_connections 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 --2- 172.21.15.7:0/371616964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21980ac990 0x7f219806bc50 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 --2- 172.21.15.7:0/371616964 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2198159a50 0x7f2198159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 --2- 172.21.15.7:0/371616964 >> v2:172.21.15.113:3300/0 conn(0x7f219815e4a0 0x7f21980ac450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 -- 172.21.15.7:0/371616964 >> 172.21.15.7:0/371616964 conn(0x7f21980cd910 msgr2=0x7f21980cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 -- 172.21.15.7:0/371616964 shutdown_connections 2024-01-11T07:18:13.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.600+0000 7f219ff68700 1 -- 172.21.15.7:0/371616964 wait complete. 2024-01-11T07:18:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.601+0000 7f219ff68700 1 Processor -- start 2024-01-11T07:18:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.601+0000 7f219ff68700 1 -- start start 2024-01-11T07:18:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219ff68700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21980ac990 0x7f21981443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219ff68700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2198159a50 0x7f2198159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219ff68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f219815e4a0 0x7f219815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219ff68700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21981448e0 con 0x7f219815e4a0 2024-01-11T07:18:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219ff68700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219815a650 con 0x7f2198159a50 2024-01-11T07:18:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219ff68700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219815a930 con 0x7f21980ac990 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219e505700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f219815e4a0 0x7f219815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219e505700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f219815e4a0 0x7f219815c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55520/0 (socket says 172.21.15.7:55520) 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219e505700 1 -- 172.21.15.7:0/191499965 learned_addr learned my addr 172.21.15.7:0/191499965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219dd04700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21980ac990 0x7f21981443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.602+0000 7f219d503700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2198159a50 0x7f2198159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f219e505700 1 -- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21980ac990 msgr2=0x7f21981443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f219e505700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21980ac990 0x7f21981443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f219e505700 1 -- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2198159a50 msgr2=0x7f2198159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f219e505700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2198159a50 0x7f2198159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f219e505700 1 -- 172.21.15.7:0/191499965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2190009ae0 con 0x7f219815e4a0 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f219e505700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f219815e4a0 0x7f219815c6d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2190009880 tx=0x7f2190018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f2189ffb700 1 -- 172.21.15.7:0/191499965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190008b10 con 0x7f219815e4a0 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f2189ffb700 1 -- 172.21.15.7:0/191499965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2190008c70 con 0x7f219815e4a0 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219815ac10 con 0x7f219815e4a0 2024-01-11T07:18:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f2189ffb700 1 -- 172.21.15.7:0/191499965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190026460 con 0x7f219815e4a0 2024-01-11T07:18:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.603+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219815c1b0 con 0x7f219815e4a0 2024-01-11T07:18:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.604+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219804c3c0 con 0x7f219815e4a0 2024-01-11T07:18:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.605+0000 7f2189ffb700 1 -- 172.21.15.7:0/191499965 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2190003040 con 0x7f219815e4a0 2024-01-11T07:18:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.605+0000 7f2189ffb700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f217c066f30 0x7f217c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.606+0000 7f2189ffb700 1 -- 172.21.15.7:0/191499965 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f21900863b0 con 0x7f219815e4a0 2024-01-11T07:18:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.606+0000 7f219dd04700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f217c066f30 0x7f217c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.606+0000 7f219dd04700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f217c066f30 0x7f217c0693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f218c009db0 tx=0x7f218c0091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:13.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.610+0000 7f2189ffb700 1 -- 172.21.15.7:0/191499965 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2190053620 con 0x7f219815e4a0 2024-01-11T07:18:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.819+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f219815c3a0 con 0x7f219815e4a0 2024-01-11T07:18:13.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.819+0000 7f2189ffb700 1 -- 172.21.15.7:0/191499965 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15330 (secure 0 0 0) 0x7f2190056c40 con 0x7f219815e4a0 2024-01-11T07:18:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:13.820 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":183,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:12.853417+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:18:11.360758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-11T07:18:11.823991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:13.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f217c066f30 msgr2=0x7f217c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f217c066f30 0x7f217c0693e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f218c009db0 tx=0x7f218c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f219815e4a0 msgr2=0x7f219815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f219815e4a0 0x7f219815c6d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2190009880 tx=0x7f2190018000 comp rx=0 tx=0).stop 2024-01-11T07:18:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 shutdown_connections 2024-01-11T07:18:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f217c066f30 0x7f217c0693e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f219815e4a0 0x7f219815c6d0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21980ac990 0x7f21981443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 --2- 172.21.15.7:0/191499965 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2198159a50 0x7f2198159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 >> 172.21.15.7:0/191499965 conn(0x7f21980cd910 msgr2=0x7f21980b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 shutdown_connections 2024-01-11T07:18:13.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.822+0000 7f219ff68700 1 -- 172.21.15.7:0/191499965 wait complete. 2024-01-11T07:18:13.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.971+0000 7fe69ec13700 1 Processor -- start 2024-01-11T07:18:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.971+0000 7fe69ec13700 1 -- start start 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.972+0000 7fe69ec13700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe698159ff0 0x7fe6980b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.972+0000 7fe69ec13700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806f4d0 0x7fe6980b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.972+0000 7fe69ec13700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe698067de0 con 0x7fe698159c20 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.972+0000 7fe69ec13700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe698069570 con 0x7fe698159ff0 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.972+0000 7fe69ec13700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980b20e0 con 0x7fe69806f4d0 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.972+0000 7fe697fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe698159ff0 0x7fe6980b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.972+0000 7fe69c9af700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe698159c20 0x7fe698068270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60202/0 (socket says 172.21.15.7:60202) 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.972+0000 7fe69c9af700 1 -- 172.21.15.7:0/2569733528 learned_addr learned my addr 172.21.15.7:0/2569733528 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe69d1b0700 1 --2- 172.21.15.7:0/2569733528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806f4d0 0x7fe6980b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe697fff700 1 -- 172.21.15.7:0/2569733528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806f4d0 msgr2=0x7fe6980b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe697fff700 1 --2- 172.21.15.7:0/2569733528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806f4d0 0x7fe6980b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe697fff700 1 -- 172.21.15.7:0/2569733528 >> v1:172.21.15.7:6789/0 conn(0x7fe698159c20 legacy=0x7fe698068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe697fff700 1 -- 172.21.15.7:0/2569733528 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69806f950 con 0x7fe698159ff0 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe6977fe700 1 -- 172.21.15.7:0/2569733528 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433565858 0 0) 0x7fe698067de0 con 0x7fe698159c20 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe69d1b0700 1 --2- 172.21.15.7:0/2569733528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806f4d0 0x7fe6980b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe697fff700 1 --2- 172.21.15.7:0/2569733528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159ff0 0x7fe6980b1820 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe6840098e0 tx=0x7fe6840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137768423ade8058 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe6977fe700 1 -- 172.21.15.7:0/2569733528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684009e80 con 0x7fe698159ff0 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe6977fe700 1 -- 172.21.15.7:0/2569733528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400b860 con 0x7fe698159ff0 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.973+0000 7fe6977fe700 1 -- 172.21.15.7:0/2569733528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400bbe0 con 0x7fe698159ff0 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.974+0000 7fe69ec13700 1 -- 172.21.15.7:0/2569733528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159ff0 msgr2=0x7fe6980b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.974+0000 7fe69ec13700 1 --2- 172.21.15.7:0/2569733528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159ff0 0x7fe6980b1820 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe6840098e0 tx=0x7fe6840091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.974+0000 7fe69ec13700 1 -- 172.21.15.7:0/2569733528 shutdown_connections 2024-01-11T07:18:13.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.974+0000 7fe69ec13700 1 --2- 172.21.15.7:0/2569733528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe69806f4d0 0x7fe6980b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.974+0000 7fe69ec13700 1 --2- 172.21.15.7:0/2569733528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159ff0 0x7fe6980b1820 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.974+0000 7fe69ec13700 1 -- 172.21.15.7:0/2569733528 >> 172.21.15.7:0/2569733528 conn(0x7fe698071160 msgr2=0x7fe69806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:13.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.974+0000 7fe69ec13700 1 -- 172.21.15.7:0/2569733528 shutdown_connections 2024-01-11T07:18:13.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.974+0000 7fe69ec13700 1 -- 172.21.15.7:0/2569733528 wait complete. 2024-01-11T07:18:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.975+0000 7fe69ec13700 1 Processor -- start 2024-01-11T07:18:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.975+0000 7fe69ec13700 1 -- start start 2024-01-11T07:18:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe69ec13700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69806f4d0 0x7fe69814cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe69ec13700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159c20 0x7fe69814d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe69ec13700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159ff0 0x7fe69816a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe69ec13700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69814c3e0 con 0x7fe698159c20 2024-01-11T07:18:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe69ec13700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69814b360 con 0x7fe69806f4d0 2024-01-11T07:18:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe69ec13700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69814b640 con 0x7fe698159ff0 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe697fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159c20 0x7fe69814d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe697fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159c20 0x7fe69814d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55540/0 (socket says 172.21.15.7:55540) 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe697fff700 1 -- 172.21.15.7:0/146374413 learned_addr learned my addr 172.21.15.7:0/146374413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe69c9af700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69806f4d0 0x7fe69814cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.976+0000 7fe69d1b0700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159ff0 0x7fe69816a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe697fff700 1 -- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159ff0 msgr2=0x7fe69816a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe697fff700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159ff0 0x7fe69816a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe697fff700 1 -- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69806f4d0 msgr2=0x7fe69814cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe697fff700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69806f4d0 0x7fe69814cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe697fff700 1 -- 172.21.15.7:0/146374413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684009420 con 0x7fe698159c20 2024-01-11T07:18:13.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe697fff700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159c20 0x7fe69814d460 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe69806a1d0 tx=0x7fe684017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe694ff9700 1 -- 172.21.15.7:0/146374413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400bbe0 con 0x7fe698159c20 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe694ff9700 1 -- 172.21.15.7:0/146374413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684011420 con 0x7fe698159c20 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe694ff9700 1 -- 172.21.15.7:0/146374413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6840117a0 con 0x7fe698159c20 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69814b920 con 0x7fe698159c20 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.977+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69814bee0 con 0x7fe698159c20 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.978+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69804c3c0 con 0x7fe698159c20 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.978+0000 7fe694ff9700 1 -- 172.21.15.7:0/146374413 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe684011bd0 con 0x7fe698159c20 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.979+0000 7fe694ff9700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe67c066f30 0x7fe67c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.979+0000 7fe694ff9700 1 -- 172.21.15.7:0/146374413 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe684085a20 con 0x7fe698159c20 2024-01-11T07:18:13.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.979+0000 7fe69c9af700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe67c066f30 0x7fe67c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:13.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.980+0000 7fe69c9af700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe67c066f30 0x7fe67c0693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe6981500f0 tx=0x7fe68c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:13.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:13.984+0000 7fe694ff9700 1 -- 172.21.15.7:0/146374413 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe684052dc0 con 0x7fe698159c20 2024-01-11T07:18:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.188+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6980b1e00 con 0x7fe698159c20 2024-01-11T07:18:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.189+0000 7fe694ff9700 1 -- 172.21.15.7:0/146374413 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15330 (secure 0 0 0) 0x7fe6840563e0 con 0x7fe698159c20 2024-01-11T07:18:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:14.189 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":183,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:12.853417+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:18:11.360758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-11T07:18:11.823991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe67c066f30 msgr2=0x7fe67c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe67c066f30 0x7fe67c0693e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe6981500f0 tx=0x7fe68c009040 comp rx=0 tx=0).stop 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159c20 msgr2=0x7fe69814d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159c20 0x7fe69814d460 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe69806a1d0 tx=0x7fe684017000 comp rx=0 tx=0).stop 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 shutdown_connections 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe67c066f30 0x7fe67c0693e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe698159ff0 0x7fe69816a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe69806f4d0 0x7fe69814cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 --2- 172.21.15.7:0/146374413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe698159c20 0x7fe69814d460 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 >> 172.21.15.7:0/146374413 conn(0x7fe698071160 msgr2=0x7fe6981564c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 shutdown_connections 2024-01-11T07:18:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.192+0000 7fe69ec13700 1 -- 172.21.15.7:0/146374413 wait complete. 2024-01-11T07:18:14.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-11T07:18:14.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.341+0000 7fc4548ba700 1 Processor -- start 2024-01-11T07:18:14.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.341+0000 7fc4548ba700 1 -- start start 2024-01-11T07:18:14.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc4548ba700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc450063940 0x7fc45015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:14.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc4548ba700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45006adf0 con 0x7fc4500b32a0 2024-01-11T07:18:14.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc4548ba700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45006b0d0 con 0x7fc4500b2ed0 2024-01-11T07:18:14.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc4548ba700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4500ce390 con 0x7fc450063940 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc44dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc450063940 0x7fc45015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc44ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc4500b32a0 0x7fc45006dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60212/0 (socket says 172.21.15.7:60212) 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc44ed9d700 1 -- 172.21.15.7:0/1640269501 learned_addr learned my addr 172.21.15.7:0/1640269501 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc44dd9b700 1 -- 172.21.15.7:0/1640269501 >> v1:172.21.15.113:6789/0 conn(0x7fc4500b2ed0 legacy=0x7fc450063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc44dd9b700 1 -- 172.21.15.7:0/1640269501 >> v1:172.21.15.7:6789/0 conn(0x7fc4500b32a0 legacy=0x7fc45006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc44dd9b700 1 -- 172.21.15.7:0/1640269501 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4500ce670 con 0x7fc450063940 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc44d59a700 1 -- 172.21.15.7:0/1640269501 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535857570 0 0) 0x7fc45006adf0 con 0x7fc4500b32a0 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.342+0000 7fc44dd9b700 1 --2- 172.21.15.7:0/1640269501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015e590 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc4440098e0 tx=0x7fc4440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a3fadc2ac6b7af server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc44d59a700 1 -- 172.21.15.7:0/1640269501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444009e80 con 0x7fc450063940 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc44d59a700 1 -- 172.21.15.7:0/1640269501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44400b860 con 0x7fc450063940 2024-01-11T07:18:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc44d59a700 1 -- 172.21.15.7:0/1640269501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400bbe0 con 0x7fc450063940 2024-01-11T07:18:14.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc4548ba700 1 -- 172.21.15.7:0/1640269501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 msgr2=0x7fc45015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:14.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc4548ba700 1 --2- 172.21.15.7:0/1640269501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015e590 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc4440098e0 tx=0x7fc4440091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:14.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc4548ba700 1 -- 172.21.15.7:0/1640269501 shutdown_connections 2024-01-11T07:18:14.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc4548ba700 1 --2- 172.21.15.7:0/1640269501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015e590 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:14.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc4548ba700 1 -- 172.21.15.7:0/1640269501 >> 172.21.15.7:0/1640269501 conn(0x7fc4500680e0 msgr2=0x7fc4500cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:14.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc4548ba700 1 -- 172.21.15.7:0/1640269501 shutdown_connections 2024-01-11T07:18:14.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.343+0000 7fc4548ba700 1 -- 172.21.15.7:0/1640269501 wait complete. 2024-01-11T07:18:14.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.344+0000 7fc4548ba700 1 Processor -- start 2024-01-11T07:18:14.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.344+0000 7fc4548ba700 1 -- start start 2024-01-11T07:18:14.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc4548ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:14.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc4548ba700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4500b2ed0 0x7fc4500af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:14.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc4548ba700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4500b32a0 0x7fc45016a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:14.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc4548ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4500c3390 con 0x7fc450063940 2024-01-11T07:18:14.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc4548ba700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4500c2780 con 0x7fc4500b2ed0 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc4548ba700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4500c2a60 con 0x7fc4500b32a0 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc44e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc44e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55566/0 (socket says 172.21.15.7:55566) 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.345+0000 7fc44e59c700 1 -- 172.21.15.7:0/2601283163 learned_addr learned my addr 172.21.15.7:0/2601283163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44dd9b700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4500b2ed0 0x7fc4500af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44ed9d700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4500b32a0 0x7fc45016a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44e59c700 1 -- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4500b32a0 msgr2=0x7fc45016a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44e59c700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4500b32a0 0x7fc45016a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44e59c700 1 -- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4500b2ed0 msgr2=0x7fc4500af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44e59c700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4500b2ed0 0x7fc4500af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44e59c700 1 -- 172.21.15.7:0/2601283163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444009420 con 0x7fc450063940 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44dd9b700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4500b2ed0 0x7fc4500af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44ed9d700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4500b32a0 0x7fc45016a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc44e59c700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015d050 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc44000c590 tx=0x7fc44000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc43e7fc700 1 -- 172.21.15.7:0/2601283163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440011590 con 0x7fc450063940 2024-01-11T07:18:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc43e7fc700 1 -- 172.21.15.7:0/2601283163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4400116f0 con 0x7fc450063940 2024-01-11T07:18:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4500c2d40 con 0x7fc450063940 2024-01-11T07:18:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc43e7fc700 1 -- 172.21.15.7:0/2601283163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440011ab0 con 0x7fc450063940 2024-01-11T07:18:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.346+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4500af600 con 0x7fc450063940 2024-01-11T07:18:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.347+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4500c3670 con 0x7fc450063940 2024-01-11T07:18:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.348+0000 7fc43e7fc700 1 -- 172.21.15.7:0/2601283163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc44001b3b0 con 0x7fc450063940 2024-01-11T07:18:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.348+0000 7fc43e7fc700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc430066f30 0x7fc4300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.348+0000 7fc43e7fc700 1 -- 172.21.15.7:0/2601283163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc440083ae0 con 0x7fc450063940 2024-01-11T07:18:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.348+0000 7fc44dd9b700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc430066f30 0x7fc4300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.349+0000 7fc44dd9b700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc430066f30 0x7fc4300693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c1210 tx=0x7fc444017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.353+0000 7fc43e7fc700 1 -- 172.21.15.7:0/2601283163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc440050d50 con 0x7fc450063940 2024-01-11T07:18:14.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-01-11T07:18:14.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:14.608+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc450064a40 con 0x7fc450063940 2024-01-11T07:18:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.451+0000 7fc43e7fc700 1 -- 172.21.15.7:0/2601283163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc440017020 con 0x7fc450063940 2024-01-11T07:18:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc430066f30 msgr2=0x7fc4300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc430066f30 0x7fc4300693e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c1210 tx=0x7fc444017000 comp rx=0 tx=0).stop 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 msgr2=0x7fc45015d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015d050 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc44000c590 tx=0x7fc44000c950 comp rx=0 tx=0).stop 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 shutdown_connections 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc430066f30 0x7fc4300693e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4500b32a0 0x7fc45016a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc450063940 0x7fc45015d050 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 --2- 172.21.15.7:0/2601283163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4500b2ed0 0x7fc4500af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 >> 172.21.15.7:0/2601283163 conn(0x7fc4500680e0 msgr2=0x7fc450063230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.453+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 shutdown_connections 2024-01-11T07:18:15.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.454+0000 7fc4548ba700 1 -- 172.21.15.7:0/2601283163 wait complete. 2024-01-11T07:18:15.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-11T07:18:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.607+0000 7fe57c25c700 1 Processor -- start 2024-01-11T07:18:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.607+0000 7fe57c25c700 1 -- start start 2024-01-11T07:18:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe57c25c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 0x7fe5740cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe57c25c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5740b0390 0x7fe57406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe57c25c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe57415a2a0 0x7fe57406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe57c25c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5740d0130 con 0x7fe5740b0390 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe57c25c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5740cc5d0 con 0x7fe57415a2a0 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe57c25c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740cfae0 con 0x7fe574159ed0 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe5797f7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5740b0390 0x7fe57406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe5797f7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5740b0390 0x7fe57406dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55570/0 (socket says 172.21.15.7:55570) 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.608+0000 7fe5797f7700 1 -- 172.21.15.7:0/1876464689 learned_addr learned my addr 172.21.15.7:0/1876464689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe57a7f9700 1 --2- 172.21.15.7:0/1876464689 >> v2:172.21.15.113:3300/0 conn(0x7fe57415a2a0 0x7fe57406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe579ff8700 1 --2- 172.21.15.7:0/1876464689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 0x7fe5740cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe5797f7700 1 -- 172.21.15.7:0/1876464689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 msgr2=0x7fe5740cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe5797f7700 1 --2- 172.21.15.7:0/1876464689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 0x7fe5740cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe5797f7700 1 -- 172.21.15.7:0/1876464689 >> v2:172.21.15.113:3300/0 conn(0x7fe57415a2a0 msgr2=0x7fe57406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe5797f7700 1 --2- 172.21.15.7:0/1876464689 >> v2:172.21.15.113:3300/0 conn(0x7fe57415a2a0 0x7fe57406e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe5797f7700 1 -- 172.21.15.7:0/1876464689 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5740cfdc0 con 0x7fe5740b0390 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe5797f7700 1 --2- 172.21.15.7:0/1876464689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe57406dc30 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe5740cb1f0 tx=0x7fe568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b87a72a3fc32ace server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe578ff6700 1 -- 172.21.15.7:0/1876464689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56800b860 con 0x7fe5740b0390 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe578ff6700 1 -- 172.21.15.7:0/1876464689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56800b9c0 con 0x7fe5740b0390 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.609+0000 7fe578ff6700 1 -- 172.21.15.7:0/1876464689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56800bd60 con 0x7fe5740b0390 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 -- 172.21.15.7:0/1876464689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 msgr2=0x7fe57406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 --2- 172.21.15.7:0/1876464689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe57406dc30 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe5740cb1f0 tx=0x7fe568009250 comp rx=0 tx=0).stop 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 -- 172.21.15.7:0/1876464689 shutdown_connections 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 --2- 172.21.15.7:0/1876464689 >> v2:172.21.15.113:3300/0 conn(0x7fe57415a2a0 0x7fe57406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 --2- 172.21.15.7:0/1876464689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 0x7fe5740cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 --2- 172.21.15.7:0/1876464689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe57406dc30 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 -- 172.21.15.7:0/1876464689 >> 172.21.15.7:0/1876464689 conn(0x7fe574063d00 msgr2=0x7fe5740d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 -- 172.21.15.7:0/1876464689 shutdown_connections 2024-01-11T07:18:15.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.610+0000 7fe57c25c700 1 -- 172.21.15.7:0/1876464689 wait complete. 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.611+0000 7fe57c25c700 1 Processor -- start 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.611+0000 7fe57c25c700 1 -- start start 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe57c25c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe574169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe57c25c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 0x7fe574155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe57c25c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57415a2a0 0x7fe5740c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe57c25c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740ca2d0 con 0x7fe5740b0390 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe57c25c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740ca760 con 0x7fe57415a2a0 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe57c25c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c96c0 con 0x7fe574159ed0 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe579ff8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe574169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe579ff8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe574169f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55576/0 (socket says 172.21.15.7:55576) 2024-01-11T07:18:15.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe579ff8700 1 -- 172.21.15.7:0/3548999888 learned_addr learned my addr 172.21.15.7:0/3548999888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe5797f7700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 0x7fe574155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe57a7f9700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57415a2a0 0x7fe5740c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe579ff8700 1 -- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 msgr2=0x7fe574155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe579ff8700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 0x7fe574155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe579ff8700 1 -- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57415a2a0 msgr2=0x7fe5740c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe579ff8700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57415a2a0 0x7fe5740c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe579ff8700 1 -- 172.21.15.7:0/3548999888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680094f0 con 0x7fe5740b0390 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.612+0000 7fe57a7f9700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57415a2a0 0x7fe5740c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.613+0000 7fe579ff8700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe574169f30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe57000d890 tx=0x7fe570008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.613+0000 7fe565ffb700 1 -- 172.21.15.7:0/3548999888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700125f0 con 0x7fe5740b0390 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.613+0000 7fe565ffb700 1 -- 172.21.15.7:0/3548999888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570012750 con 0x7fe5740b0390 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.613+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5740c99a0 con 0x7fe5740b0390 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.613+0000 7fe565ffb700 1 -- 172.21.15.7:0/3548999888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570012ad0 con 0x7fe5740b0390 2024-01-11T07:18:15.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.613+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5740c9f60 con 0x7fe5740b0390 2024-01-11T07:18:15.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.614+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57406f660 con 0x7fe5740b0390 2024-01-11T07:18:15.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.615+0000 7fe565ffb700 1 -- 172.21.15.7:0/3548999888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe57001e460 con 0x7fe5740b0390 2024-01-11T07:18:15.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.615+0000 7fe565ffb700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe558066f30 0x7fe5580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:15.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.615+0000 7fe565ffb700 1 -- 172.21.15.7:0/3548999888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe570084b50 con 0x7fe5740b0390 2024-01-11T07:18:15.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.615+0000 7fe5797f7700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe558066f30 0x7fe5580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:15.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.616+0000 7fe5797f7700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe558066f30 0x7fe5580693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe5740cb1f0 tx=0x7fe568017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:15.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.620+0000 7fe565ffb700 1 -- 172.21.15.7:0/3548999888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe570051dc0 con 0x7fe5740b0390 2024-01-11T07:18:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:15.872+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe57404c3c0 con 0x7fe5740b0390 2024-01-11T07:18:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.456+0000 7fe565ffb700 1 -- 172.21.15.7:0/3548999888 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe5700553e0 con 0x7fe5740b0390 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe558066f30 msgr2=0x7fe5580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe558066f30 0x7fe5580693e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe5740cb1f0 tx=0x7fe568017000 comp rx=0 tx=0).stop 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 msgr2=0x7fe574169f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe574169f30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe57000d890 tx=0x7fe570008040 comp rx=0 tx=0).stop 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 shutdown_connections 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe558066f30 0x7fe5580693e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57415a2a0 0x7fe5740c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5740b0390 0x7fe574169f30 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 --2- 172.21.15.7:0/3548999888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe574159ed0 0x7fe574155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 >> 172.21.15.7:0/3548999888 conn(0x7fe574063d00 msgr2=0x7fe5741561f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 shutdown_connections 2024-01-11T07:18:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.459+0000 7fe57c25c700 1 -- 172.21.15.7:0/3548999888 wait complete. 2024-01-11T07:18:16.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.610+0000 7f0b262a2700 1 Processor -- start 2024-01-11T07:18:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.610+0000 7f0b262a2700 1 -- start start 2024-01-11T07:18:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.611+0000 7f0b262a2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b2015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:16.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.611+0000 7f0b262a2700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b20157d80 0x7f0b20063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:16.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.611+0000 7f0b262a2700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b20063bb0 con 0x7f0b20063770 2024-01-11T07:18:16.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.611+0000 7f0b262a2700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200aeab0 con 0x7f0b20157d80 2024-01-11T07:18:16.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.611+0000 7f0b262a2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200aed90 con 0x7f0b201579b0 2024-01-11T07:18:16.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b1f7fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b20157d80 0x7f0b20063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b1ffff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b2015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b2483f700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0b20063770 0x7f0b2006dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:41930/0 (socket says 172.21.15.7:41930) 2024-01-11T07:18:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b2483f700 1 -- 172.21.15.7:0/2020499566 learned_addr learned my addr 172.21.15.7:0/2020499566 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b1f7fe700 1 -- 172.21.15.7:0/2020499566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 msgr2=0x7f0b2015bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b1f7fe700 1 --2- 172.21.15.7:0/2020499566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b2015bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b1f7fe700 1 -- 172.21.15.7:0/2020499566 >> v1:172.21.15.113:6789/0 conn(0x7f0b20063770 legacy=0x7f0b2006dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:16.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b1f7fe700 1 -- 172.21.15.7:0/2020499566 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b200af070 con 0x7f0b20157d80 2024-01-11T07:18:16.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b1ffff700 1 --2- 172.21.15.7:0/2020499566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b2015bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:16.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.612+0000 7f0b1effd700 1 -- 172.21.15.7:0/2020499566 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738224138 0 0) 0x7f0b20063bb0 con 0x7f0b20063770 2024-01-11T07:18:16.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b1f7fe700 1 --2- 172.21.15.7:0/2020499566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 0x7f0b20063230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0b140098e0 tx=0x7f0b140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71004c6b5132d5bb server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:16.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b1effd700 1 -- 172.21.15.7:0/2020499566 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400b640 con 0x7f0b20157d80 2024-01-11T07:18:16.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b1effd700 1 -- 172.21.15.7:0/2020499566 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1400b7a0 con 0x7f0b20157d80 2024-01-11T07:18:16.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b1effd700 1 -- 172.21.15.7:0/2020499566 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1400bb40 con 0x7f0b20157d80 2024-01-11T07:18:16.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b262a2700 1 -- 172.21.15.7:0/2020499566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 msgr2=0x7f0b20063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:16.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b262a2700 1 --2- 172.21.15.7:0/2020499566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 0x7f0b20063230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0b140098e0 tx=0x7f0b140091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b262a2700 1 -- 172.21.15.7:0/2020499566 shutdown_connections 2024-01-11T07:18:16.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b262a2700 1 --2- 172.21.15.7:0/2020499566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b2015bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b262a2700 1 --2- 172.21.15.7:0/2020499566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 0x7f0b20063230 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b262a2700 1 -- 172.21.15.7:0/2020499566 >> 172.21.15.7:0/2020499566 conn(0x7f0b200680e0 msgr2=0x7f0b200cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:16.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b262a2700 1 -- 172.21.15.7:0/2020499566 shutdown_connections 2024-01-11T07:18:16.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.613+0000 7f0b262a2700 1 -- 172.21.15.7:0/2020499566 wait complete. 2024-01-11T07:18:16.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.615+0000 7f0b262a2700 1 Processor -- start 2024-01-11T07:18:16.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.615+0000 7f0b262a2700 1 -- start start 2024-01-11T07:18:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b262a2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b20063770 0x7f0b200ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b262a2700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b200b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b262a2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 0x7f0b200b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b262a2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200abda0 con 0x7f0b20063770 2024-01-11T07:18:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b262a2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200ac080 con 0x7f0b20157d80 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b262a2700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200ac360 con 0x7f0b201579b0 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b1ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b20063770 0x7f0b200ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b1ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b20063770 0x7f0b200ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55582/0 (socket says 172.21.15.7:55582) 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b2483f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 0x7f0b200b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b1ffff700 1 -- 172.21.15.7:0/4249437031 learned_addr learned my addr 172.21.15.7:0/4249437031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b1f7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b200b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b2483f700 1 -- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 msgr2=0x7f0b200b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b2483f700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b200b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b2483f700 1 -- 172.21.15.7:0/4249437031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b20063770 msgr2=0x7f0b200ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b2483f700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b20063770 0x7f0b200ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b2483f700 1 -- 172.21.15.7:0/4249437031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10009420 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b1ffff700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b20063770 0x7f0b200ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.616+0000 7f0b2483f700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 0x7f0b200b1840 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800cbe0 tx=0x7f0b180019a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.617+0000 7f0b07fff700 1 -- 172.21.15.7:0/4249437031 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800a8b0 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.617+0000 7f0b07fff700 1 -- 172.21.15.7:0/4249437031 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1800aa10 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.617+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b14009420 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.617+0000 7f0b07fff700 1 -- 172.21.15.7:0/4249437031 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800add0 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.617+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b200ac920 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.618+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b200acc90 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.618+0000 7f0b07fff700 1 -- 172.21.15.7:0/4249437031 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0b18007300 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.619+0000 7f0b07fff700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b0c066f30 0x7f0b0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.619+0000 7f0b07fff700 1 -- 172.21.15.7:0/4249437031 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0b18084780 con 0x7f0b20157d80 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.619+0000 7f0b1ffff700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b0c066f30 0x7f0b0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.619+0000 7f0b1ffff700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b0c066f30 0x7f0b0c0693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0b200c4890 tx=0x7f0b100091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.624+0000 7f0b07fff700 1 -- 172.21.15.7:0/4249437031 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b180519d0 con 0x7f0b20157d80 2024-01-11T07:18:16.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.873+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b200ac1c0 con 0x7f0b20157d80 2024-01-11T07:18:16.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.874+0000 7f0b07fff700 1 -- 172.21.15.7:0/4249437031 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f0b200ac1c0 con 0x7f0b20157d80 2024-01-11T07:18:16.875 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:16.875 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:16.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.876+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b0c066f30 msgr2=0x7f0b0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:16.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.876+0000 7f0b262a2700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b0c066f30 0x7f0b0c0693e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0b200c4890 tx=0x7f0b100091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.876+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 msgr2=0x7f0b200b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:16.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 0x7f0b200b1840 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800cbe0 tx=0x7f0b180019a0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 shutdown_connections 2024-01-11T07:18:16.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0b0c066f30 0x7f0b0c0693e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b20157d80 0x7f0b200b1840 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b20063770 0x7f0b200ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 --2- 172.21.15.7:0/4249437031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0b201579b0 0x7f0b200b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:16.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 >> 172.21.15.7:0/4249437031 conn(0x7f0b200680e0 msgr2=0x7f0b200ca180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:16.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 shutdown_connections 2024-01-11T07:18:16.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:16.877+0000 7f0b262a2700 1 -- 172.21.15.7:0/4249437031 wait complete. 2024-01-11T07:18:16.879 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:16.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.021+0000 7fe4fa322700 1 Processor -- start 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.021+0000 7fe4fa322700 1 -- start start 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.022+0000 7fe4fa322700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4f415dd90 0x7fe4f40aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.022+0000 7fe4fa322700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f406ac30 con 0x7fe4f415d920 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.022+0000 7fe4fa322700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f40cb370 con 0x7fe4f40ac3a0 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.022+0000 7fe4fa322700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f40ac770 con 0x7fe4f415dd90 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.022+0000 7fe4f88bf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4f415dd90 0x7fe4f40aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.022+0000 7fe4f37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe4f415d920 0x7fe4f40aa8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60224/0 (socket says 172.21.15.7:60224) 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.022+0000 7fe4f37fe700 1 -- 172.21.15.7:0/1647877154 learned_addr learned my addr 172.21.15.7:0/1647877154 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.023+0000 7fe4f88bf700 1 -- 172.21.15.7:0/1647877154 >> v1:172.21.15.113:6789/0 conn(0x7fe4f40ac3a0 legacy=0x7fe4f40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.023+0000 7fe4f88bf700 1 -- 172.21.15.7:0/1647877154 >> v1:172.21.15.7:6789/0 conn(0x7fe4f415d920 legacy=0x7fe4f40aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.023+0000 7fe4f2ffd700 1 -- 172.21.15.7:0/1647877154 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057724337 0 0) 0x7fe4f406ac30 con 0x7fe4f415d920 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.023+0000 7fe4f88bf700 1 -- 172.21.15.7:0/1647877154 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f406a300 con 0x7fe4f415dd90 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.023+0000 7fe4f88bf700 1 --2- 172.21.15.7:0/1647877154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f40aafd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009e60 tx=0x7fe4ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55025db2c8daf3f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.023+0000 7fe4f2ffd700 1 -- 172.21.15.7:0/1647877154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00e070 con 0x7fe4f415dd90 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.023+0000 7fe4f2ffd700 1 -- 172.21.15.7:0/1647877154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec00bc30 con 0x7fe4f415dd90 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.023+0000 7fe4f2ffd700 1 -- 172.21.15.7:0/1647877154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec012430 con 0x7fe4f415dd90 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.024+0000 7fe4fa322700 1 -- 172.21.15.7:0/1647877154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 msgr2=0x7fe4f40aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.024+0000 7fe4fa322700 1 --2- 172.21.15.7:0/1647877154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f40aafd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009e60 tx=0x7fe4ec009780 comp rx=0 tx=0).stop 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.024+0000 7fe4fa322700 1 -- 172.21.15.7:0/1647877154 shutdown_connections 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.024+0000 7fe4fa322700 1 --2- 172.21.15.7:0/1647877154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f40aafd0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.024+0000 7fe4fa322700 1 -- 172.21.15.7:0/1647877154 >> 172.21.15.7:0/1647877154 conn(0x7fe4f40cd090 msgr2=0x7fe4f40cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.024+0000 7fe4fa322700 1 -- 172.21.15.7:0/1647877154 shutdown_connections 2024-01-11T07:18:17.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.024+0000 7fe4fa322700 1 -- 172.21.15.7:0/1647877154 wait complete. 2024-01-11T07:18:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.025+0000 7fe4fa322700 1 Processor -- start 2024-01-11T07:18:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.026+0000 7fe4fa322700 1 -- start start 2024-01-11T07:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4fa322700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4f40ac3a0 0x7fe4f4157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4fa322700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4f415d920 0x7fe4f40c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4fa322700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f406e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4fa322700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40c9890 con 0x7fe4f415dd90 2024-01-11T07:18:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4fa322700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40c8c80 con 0x7fe4f415d920 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4fa322700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40c8f60 con 0x7fe4f40ac3a0 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f406e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f406e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55594/0 (socket says 172.21.15.7:55594) 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 -- 172.21.15.7:0/2856048390 learned_addr learned my addr 172.21.15.7:0/2856048390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f37fe700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4f415d920 0x7fe4f40c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f3fff700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4f40ac3a0 0x7fe4f4157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 -- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4f40ac3a0 msgr2=0x7fe4f4157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4f40ac3a0 0x7fe4f4157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 -- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4f415d920 msgr2=0x7fe4f40c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4f415d920 0x7fe4f40c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 -- 172.21.15.7:0/2856048390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec009a20 con 0x7fe4f415dd90 2024-01-11T07:18:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.027+0000 7fe4f88bf700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f406e030 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec013000 tx=0x7fe4ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.028+0000 7fe4ebfff700 1 -- 172.21.15.7:0/2856048390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec01b420 con 0x7fe4f415dd90 2024-01-11T07:18:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.028+0000 7fe4ebfff700 1 -- 172.21.15.7:0/2856048390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec01b580 con 0x7fe4f415dd90 2024-01-11T07:18:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.028+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f40c9240 con 0x7fe4f415dd90 2024-01-11T07:18:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.028+0000 7fe4ebfff700 1 -- 172.21.15.7:0/2856048390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec01b900 con 0x7fe4f415dd90 2024-01-11T07:18:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.028+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f406e570 con 0x7fe4f415dd90 2024-01-11T07:18:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.029+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f404c3c0 con 0x7fe4f415dd90 2024-01-11T07:18:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.029+0000 7fe4ebfff700 1 -- 172.21.15.7:0/2856048390 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe4ec01bc70 con 0x7fe4f415dd90 2024-01-11T07:18:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.030+0000 7fe4ebfff700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4d4066f30 0x7fe4d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.030+0000 7fe4ebfff700 1 -- 172.21.15.7:0/2856048390 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe4ec0862d0 con 0x7fe4f415dd90 2024-01-11T07:18:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.030+0000 7fe4f3fff700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4d4066f30 0x7fe4d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.030+0000 7fe4f3fff700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4d4066f30 0x7fe4d40693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4003a80 tx=0x7fe4e4007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.034+0000 7fe4ebfff700 1 -- 172.21.15.7:0/2856048390 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4ec0535f0 con 0x7fe4f415dd90 2024-01-11T07:18:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.240+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4f40c99d0 con 0x7fe4f415dd90 2024-01-11T07:18:17.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.241+0000 7fe4ebfff700 1 -- 172.21.15.7:0/2856048390 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+15317 (secure 0 0 0) 0x7fe4f40c99d0 con 0x7fe4f415dd90 2024-01-11T07:18:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:17.242 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":184,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:14.422654+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:18:11.360758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-11T07:18:11.823991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:17.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4d4066f30 msgr2=0x7fe4d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4d4066f30 0x7fe4d40693e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4003a80 tx=0x7fe4e4007ec0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 msgr2=0x7fe4f406e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f406e030 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec013000 tx=0x7fe4ec019000 comp rx=0 tx=0).stop 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 shutdown_connections 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe4d4066f30 0x7fe4d40693e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4f415dd90 0x7fe4f406e030 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe4f40ac3a0 0x7fe4f4157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 --2- 172.21.15.7:0/2856048390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4f415d920 0x7fe4f40c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 >> 172.21.15.7:0/2856048390 conn(0x7fe4f40cd090 msgr2=0x7fe4f413c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 shutdown_connections 2024-01-11T07:18:17.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.243+0000 7fe4fa322700 1 -- 172.21.15.7:0/2856048390 wait complete. 2024-01-11T07:18:17.258 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-11T07:18:17.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:18:17.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.389+0000 7f1134312700 1 Processor -- start 2024-01-11T07:18:17.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.389+0000 7f1134312700 1 -- start start 2024-01-11T07:18:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f1134312700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f112c06dd00 0x7f112c0c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f1134312700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06e0d0 0x7f112c159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f1134312700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f112c159ff0 0x7f112c0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f1134312700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112c15b770 con 0x7f112c159ff0 2024-01-11T07:18:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f1134312700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112c15abf0 con 0x7f112c06dd00 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f1134312700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c15aed0 con 0x7f112c06e0d0 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f11328af700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f112c159ff0 0x7f112c0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f11328af700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f112c159ff0 0x7f112c0c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57190/0 (socket says 172.21.15.7:57190) 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.390+0000 7f11328af700 1 -- 172.21.15.7:0/3429326768 learned_addr learned my addr 172.21.15.7:0/3429326768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11318ad700 1 --2- 172.21.15.7:0/3429326768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06e0d0 0x7f112c159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11320ae700 1 --2- 172.21.15.7:0/3429326768 >> v2:172.21.15.113:3300/0 conn(0x7f112c06dd00 0x7f112c0c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11328af700 1 -- 172.21.15.7:0/3429326768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06e0d0 msgr2=0x7f112c159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11328af700 1 --2- 172.21.15.7:0/3429326768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06e0d0 0x7f112c159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11328af700 1 -- 172.21.15.7:0/3429326768 >> v2:172.21.15.113:3300/0 conn(0x7f112c06dd00 msgr2=0x7f112c0c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11328af700 1 --2- 172.21.15.7:0/3429326768 >> v2:172.21.15.113:3300/0 conn(0x7f112c06dd00 0x7f112c0c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11328af700 1 -- 172.21.15.7:0/3429326768 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c15b1b0 con 0x7f112c159ff0 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11328af700 1 --2- 172.21.15.7:0/3429326768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c7e70 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f112c06d7f0 tx=0x7f1124009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c7a4caf30c3dff server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11310ac700 1 -- 172.21.15.7:0/3429326768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400e070 con 0x7f112c159ff0 2024-01-11T07:18:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11310ac700 1 -- 172.21.15.7:0/3429326768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112400ba50 con 0x7f112c159ff0 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.391+0000 7f11310ac700 1 -- 172.21.15.7:0/3429326768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11240092e0 con 0x7f112c159ff0 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 -- 172.21.15.7:0/3429326768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 msgr2=0x7f112c0c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 --2- 172.21.15.7:0/3429326768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c7e70 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f112c06d7f0 tx=0x7f1124009840 comp rx=0 tx=0).stop 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 -- 172.21.15.7:0/3429326768 shutdown_connections 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 --2- 172.21.15.7:0/3429326768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c7e70 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 --2- 172.21.15.7:0/3429326768 >> v2:172.21.15.113:3300/0 conn(0x7f112c06dd00 0x7f112c0c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 --2- 172.21.15.7:0/3429326768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06e0d0 0x7f112c159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 -- 172.21.15.7:0/3429326768 >> 172.21.15.7:0/3429326768 conn(0x7f112c0680e0 msgr2=0x7f112c0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 -- 172.21.15.7:0/3429326768 shutdown_connections 2024-01-11T07:18:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.392+0000 7f1134312700 1 -- 172.21.15.7:0/3429326768 wait complete. 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.393+0000 7f1134312700 1 Processor -- start 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.393+0000 7f1134312700 1 -- start start 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f1134312700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06dd00 0x7f112c0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f1134312700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c06e0d0 0x7f112c152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f1134312700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f1134312700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0af260 con 0x7f112c159ff0 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f1134312700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112c153370 con 0x7f112c06e0d0 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f1134312700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c0ae5e0 con 0x7f112c06dd00 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57198/0 (socket says 172.21.15.7:57198) 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 -- 172.21.15.7:0/2533764422 learned_addr learned my addr 172.21.15.7:0/2533764422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11320ae700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06dd00 0x7f112c0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11318ad700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c06e0d0 0x7f112c152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 -- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06dd00 msgr2=0x7f112c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06dd00 0x7f112c0b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 -- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c06e0d0 msgr2=0x7f112c152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c06e0d0 0x7f112c152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 -- 172.21.15.7:0/2533764422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124009ae0 con 0x7f112c159ff0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.394+0000 7f11328af700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c91c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f112c06d820 tx=0x7f1124018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.395+0000 7f11227fc700 1 -- 172.21.15.7:0/2533764422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11240032f0 con 0x7f112c159ff0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.395+0000 7f11227fc700 1 -- 172.21.15.7:0/2533764422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1124008d20 con 0x7f112c159ff0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.395+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112c0ae8c0 con 0x7f112c159ff0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.395+0000 7f11227fc700 1 -- 172.21.15.7:0/2533764422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124025690 con 0x7f112c159ff0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.395+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112c0aee80 con 0x7f112c159ff0 2024-01-11T07:18:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.396+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c0af3a0 con 0x7f112c159ff0 2024-01-11T07:18:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.396+0000 7f11227fc700 1 -- 172.21.15.7:0/2533764422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1124003040 con 0x7f112c159ff0 2024-01-11T07:18:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.397+0000 7f11227fc700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1110066f30 0x7f11100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.397+0000 7f11227fc700 1 -- 172.21.15.7:0/2533764422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1124086150 con 0x7f112c159ff0 2024-01-11T07:18:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.397+0000 7f11320ae700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1110066f30 0x7f11100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.397+0000 7f11320ae700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1110066f30 0x7f11100693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1128006fd0 tx=0x7f1128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.401+0000 7f11227fc700 1 -- 172.21.15.7:0/2533764422 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f112c0af3a0 con 0x7f112c159ff0 2024-01-11T07:18:17.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.601+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f112c0af3a0 con 0x7f1110066f30 2024-01-11T07:18:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.610+0000 7f11227fc700 1 -- 172.21.15.7:0/2533764422 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285651 (secure 0 0 0) 0x7f112c0af3a0 con 0x7f1110066f30 2024-01-11T07:18:17.615 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1110066f30 msgr2=0x7f11100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1110066f30 0x7f11100693e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1128006fd0 tx=0x7f1128008040 comp rx=0 tx=0).stop 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 msgr2=0x7f112c0c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c91c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f112c06d820 tx=0x7f1124018000 comp rx=0 tx=0).stop 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 shutdown_connections 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1110066f30 0x7f11100693e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112c159ff0 0x7f112c0c91c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f112c06dd00 0x7f112c0b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 --2- 172.21.15.7:0/2533764422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f112c06e0d0 0x7f112c152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 >> 172.21.15.7:0/2533764422 conn(0x7f112c0680e0 msgr2=0x7f112c0cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 shutdown_connections 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.613+0000 7f1134312700 1 -- 172.21.15.7:0/2533764422 wait complete. 2024-01-11T07:18:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:18:17.627 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-01-11T07:18:16.002684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9119,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553380,"statfs":{"total":1600000000,"available":1590661129,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.585813"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.432054+0000","last_change":"2024-01-11T07:18:13.432054+0000","last_active":"2024-01-11T07:18:13.432054+0000","last_peered":"2024-01-11T07:18:13.432054+0000","last_clean":"2024-01-11T07:18:13.432054+0000","last_became_active":"2024-01-11T07:18:13.431728+0000","last_became_peered":"2024-01-11T07:18:13.431728+0000","last_unstale":"2024-01-11T07:18:13.432054+0000","last_undegraded":"2024-01-11T07:18:13.432054+0000","last_fullsized":"2024-01-11T07:18:13.432054+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:47:25.278556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437487+0000","last_change":"2024-01-11T07:18:13.437487+0000","last_active":"2024-01-11T07:18:13.437487+0000","last_peered":"2024-01-11T07:18:13.437487+0000","last_clean":"2024-01-11T07:18:13.437487+0000","last_became_active":"2024-01-11T07:18:13.437233+0000","last_became_peered":"2024-01-11T07:18:13.437233+0000","last_unstale":"2024-01-11T07:18:13.437487+0000","last_undegraded":"2024-01-11T07:18:13.437487+0000","last_fullsized":"2024-01-11T07:18:13.437487+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:54.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434151+0000","last_change":"2024-01-11T07:18:13.434151+0000","last_active":"2024-01-11T07:18:13.434151+0000","last_peered":"2024-01-11T07:18:13.434151+0000","last_clean":"2024-01-11T07:18:13.434151+0000","last_became_active":"2024-01-11T07:18:13.433600+0000","last_became_peered":"2024-01-11T07:18:13.433600+0000","last_unstale":"2024-01-11T07:18:13.434151+0000","last_undegraded":"2024-01-11T07:18:13.434151+0000","last_fullsized":"2024-01-11T07:18:13.434151+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:42:10.817251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.440416+0000","last_change":"2024-01-11T07:18:13.440416+0000","last_active":"2024-01-11T07:18:13.440416+0000","last_peered":"2024-01-11T07:18:13.440416+0000","last_clean":"2024-01-11T07:18:13.440416+0000","last_became_active":"2024-01-11T07:18:13.440152+0000","last_became_peered":"2024-01-11T07:18:13.440152+0000","last_unstale":"2024-01-11T07:18:13.440416+0000","last_undegraded":"2024-01-11T07:18:13.440416+0000","last_fullsized":"2024-01-11T07:18:13.440416+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:25:16.299601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438035+0000","last_change":"2024-01-11T07:18:13.438035+0000","last_active":"2024-01-11T07:18:13.438035+0000","last_peered":"2024-01-11T07:18:13.438035+0000","last_clean":"2024-01-11T07:18:13.438035+0000","last_became_active":"2024-01-11T07:18:13.437749+0000","last_became_peered":"2024-01-11T07:18:13.437749+0000","last_unstale":"2024-01-11T07:18:13.438035+0000","last_undegraded":"2024-01-11T07:18:13.438035+0000","last_fullsized":"2024-01-11T07:18:13.438035+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:56:51.191605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435587+0000","last_change":"2024-01-11T07:18:13.435587+0000","last_active":"2024-01-11T07:18:13.435587+0000","last_peered":"2024-01-11T07:18:13.435587+0000","last_clean":"2024-01-11T07:18:13.435587+0000","last_became_active":"2024-01-11T07:18:13.435043+0000","last_became_peered":"2024-01-11T07:18:13.435043+0000","last_unstale":"2024-01-11T07:18:13.435587+0000","last_undegraded":"2024-01-11T07:18:13.435587+0000","last_fullsized":"2024-01-11T07:18:13.435587+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:13:47.195902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.428359+0000","last_change":"2024-01-11T07:18:13.428359+0000","last_active":"2024-01-11T07:18:13.428359+0000","last_peered":"2024-01-11T07:18:13.428359+0000","last_clean":"2024-01-11T07:18:13.428359+0000","last_became_active":"2024-01-11T07:18:13.428065+0000","last_became_peered":"2024-01-11T07:18:13.428065+0000","last_unstale":"2024-01-11T07:18:13.428359+0000","last_undegraded":"2024-01-11T07:18:13.428359+0000","last_fullsized":"2024-01-11T07:18:13.428359+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:17:45.043633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439057+0000","last_change":"2024-01-11T07:18:13.439057+0000","last_active":"2024-01-11T07:18:13.439057+0000","last_peered":"2024-01-11T07:18:13.439057+0000","last_clean":"2024-01-11T07:18:13.439057+0000","last_became_active":"2024-01-11T07:18:13.438817+0000","last_became_peered":"2024-01-11T07:18:13.438817+0000","last_unstale":"2024-01-11T07:18:13.439057+0000","last_undegraded":"2024-01-11T07:18:13.439057+0000","last_fullsized":"2024-01-11T07:18:13.439057+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:39.399759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437336+0000","last_change":"2024-01-11T07:18:13.437336+0000","last_active":"2024-01-11T07:18:13.437336+0000","last_peered":"2024-01-11T07:18:13.437336+0000","last_clean":"2024-01-11T07:18:13.437336+0000","last_became_active":"2024-01-11T07:18:13.437069+0000","last_became_peered":"2024-01-11T07:18:13.437069+0000","last_unstale":"2024-01-11T07:18:13.437336+0000","last_undegraded":"2024-01-11T07:18:13.437336+0000","last_fullsized":"2024-01-11T07:18:13.437336+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:18.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439732+0000","last_change":"2024-01-11T07:18:13.439732+0000","last_active":"2024-01-11T07:18:13.439732+0000","last_peered":"2024-01-11T07:18:13.439732+0000","last_clean":"2024-01-11T07:18:13.439732+0000","last_became_active":"2024-01-11T07:18:13.438253+0000","last_became_peered":"2024-01-11T07:18:13.438253+0000","last_unstale":"2024-01-11T07:18:13.439732+0000","last_undegraded":"2024-01-11T07:18:13.439732+0000","last_fullsized":"2024-01-11T07:18:13.439732+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:39:01.983933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438646+0000","last_change":"2024-01-11T07:18:13.438646+0000","last_active":"2024-01-11T07:18:13.438646+0000","last_peered":"2024-01-11T07:18:13.438646+0000","last_clean":"2024-01-11T07:18:13.438646+0000","last_became_active":"2024-01-11T07:18:13.438393+0000","last_became_peered":"2024-01-11T07:18:13.438393+0000","last_unstale":"2024-01-11T07:18:13.438646+0000","last_undegraded":"2024-01-11T07:18:13.438646+0000","last_fullsized":"2024-01-11T07:18:13.438646+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:08:20.052085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.440130+0000","last_change":"2024-01-11T07:18:13.440130+0000","last_active":"2024-01-11T07:18:13.440130+0000","last_peered":"2024-01-11T07:18:13.440130+0000","last_clean":"2024-01-11T07:18:13.440130+0000","last_became_active":"2024-01-11T07:18:13.439889+0000","last_became_peered":"2024-01-11T07:18:13.439889+0000","last_unstale":"2024-01-11T07:18:13.440130+0000","last_undegraded":"2024-01-11T07:18:13.440130+0000","last_fullsized":"2024-01-11T07:18:13.440130+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:04:10.626736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438547+0000","last_change":"2024-01-11T07:18:13.438547+0000","last_active":"2024-01-11T07:18:13.438547+0000","last_peered":"2024-01-11T07:18:13.438547+0000","last_clean":"2024-01-11T07:18:13.438547+0000","last_became_active":"2024-01-11T07:18:13.438303+0000","last_became_peered":"2024-01-11T07:18:13.438303+0000","last_unstale":"2024-01-11T07:18:13.438547+0000","last_undegraded":"2024-01-11T07:18:13.438547+0000","last_fullsized":"2024-01-11T07:18:13.438547+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:39.399759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.433328+0000","last_change":"2024-01-11T07:18:13.433328+0000","last_active":"2024-01-11T07:18:13.433328+0000","last_peered":"2024-01-11T07:18:13.433328+0000","last_clean":"2024-01-11T07:18:13.433328+0000","last_became_active":"2024-01-11T07:18:13.433077+0000","last_became_peered":"2024-01-11T07:18:13.433077+0000","last_unstale":"2024-01-11T07:18:13.433328+0000","last_undegraded":"2024-01-11T07:18:13.433328+0000","last_fullsized":"2024-01-11T07:18:13.433328+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:00:04.012143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438858+0000","last_change":"2024-01-11T07:18:13.438858+0000","last_active":"2024-01-11T07:18:13.438858+0000","last_peered":"2024-01-11T07:18:13.438858+0000","last_clean":"2024-01-11T07:18:13.438858+0000","last_became_active":"2024-01-11T07:18:13.437797+0000","last_became_peered":"2024-01-11T07:18:13.437797+0000","last_unstale":"2024-01-11T07:18:13.438858+0000","last_undegraded":"2024-01-11T07:18:13.438858+0000","last_fullsized":"2024-01-11T07:18:13.438858+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:23.954189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.433777+0000","last_change":"2024-01-11T07:18:13.433777+0000","last_active":"2024-01-11T07:18:13.433777+0000","last_peered":"2024-01-11T07:18:13.433777+0000","last_clean":"2024-01-11T07:18:13.433777+0000","last_became_active":"2024-01-11T07:18:13.433460+0000","last_became_peered":"2024-01-11T07:18:13.433460+0000","last_unstale":"2024-01-11T07:18:13.433777+0000","last_undegraded":"2024-01-11T07:18:13.433777+0000","last_fullsized":"2024-01-11T07:18:13.433777+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:59:56.414964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434689+0000","last_change":"2024-01-11T07:18:13.434689+0000","last_active":"2024-01-11T07:18:13.434689+0000","last_peered":"2024-01-11T07:18:13.434689+0000","last_clean":"2024-01-11T07:18:13.434689+0000","last_became_active":"2024-01-11T07:18:13.434451+0000","last_became_peered":"2024-01-11T07:18:13.434451+0000","last_unstale":"2024-01-11T07:18:13.434689+0000","last_undegraded":"2024-01-11T07:18:13.434689+0000","last_fullsized":"2024-01-11T07:18:13.434689+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:19:00.349243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435446+0000","last_change":"2024-01-11T07:18:13.435446+0000","last_active":"2024-01-11T07:18:13.435446+0000","last_peered":"2024-01-11T07:18:13.435446+0000","last_clean":"2024-01-11T07:18:13.435446+0000","last_became_active":"2024-01-11T07:18:13.434995+0000","last_became_peered":"2024-01-11T07:18:13.434995+0000","last_unstale":"2024-01-11T07:18:13.435446+0000","last_undegraded":"2024-01-11T07:18:13.435446+0000","last_fullsized":"2024-01-11T07:18:13.435446+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:05:57.809342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434157+0000","last_change":"2024-01-11T07:18:13.434157+0000","last_active":"2024-01-11T07:18:13.434157+0000","last_peered":"2024-01-11T07:18:13.434157+0000","last_clean":"2024-01-11T07:18:13.434157+0000","last_became_active":"2024-01-11T07:18:13.433994+0000","last_became_peered":"2024-01-11T07:18:13.433994+0000","last_unstale":"2024-01-11T07:18:13.434157+0000","last_undegraded":"2024-01-11T07:18:13.434157+0000","last_fullsized":"2024-01-11T07:18:13.434157+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:15:19.349095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.432612+0000","last_change":"2024-01-11T07:18:13.432612+0000","last_active":"2024-01-11T07:18:13.432612+0000","last_peered":"2024-01-11T07:18:13.432612+0000","last_clean":"2024-01-11T07:18:13.432612+0000","last_became_active":"2024-01-11T07:18:13.432013+0000","last_became_peered":"2024-01-11T07:18:13.432013+0000","last_unstale":"2024-01-11T07:18:13.432612+0000","last_undegraded":"2024-01-11T07:18:13.432612+0000","last_fullsized":"2024-01-11T07:18:13.432612+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:39:01.983933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435187+0000","last_change":"2024-01-11T07:18:13.435187+0000","last_active":"2024-01-11T07:18:13.435187+0000","last_peered":"2024-01-11T07:18:13.435187+0000","last_clean":"2024-01-11T07:18:13.435187+0000","last_became_active":"2024-01-11T07:18:13.434722+0000","last_became_peered":"2024-01-11T07:18:13.434722+0000","last_unstale":"2024-01-11T07:18:13.435187+0000","last_undegraded":"2024-01-11T07:18:13.435187+0000","last_fullsized":"2024-01-11T07:18:13.435187+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:59.464290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.443992+0000","last_change":"2024-01-11T07:18:13.443992+0000","last_active":"2024-01-11T07:18:13.443992+0000","last_peered":"2024-01-11T07:18:13.443992+0000","last_clean":"2024-01-11T07:18:13.443992+0000","last_became_active":"2024-01-11T07:18:13.443723+0000","last_became_peered":"2024-01-11T07:18:13.443723+0000","last_unstale":"2024-01-11T07:18:13.443992+0000","last_undegraded":"2024-01-11T07:18:13.443992+0000","last_fullsized":"2024-01-11T07:18:13.443992+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:51.546656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439724+0000","last_change":"2024-01-11T07:18:13.439724+0000","last_active":"2024-01-11T07:18:13.439724+0000","last_peered":"2024-01-11T07:18:13.439724+0000","last_clean":"2024-01-11T07:18:13.439724+0000","last_became_active":"2024-01-11T07:18:13.439455+0000","last_became_peered":"2024-01-11T07:18:13.439455+0000","last_unstale":"2024-01-11T07:18:13.439724+0000","last_undegraded":"2024-01-11T07:18:13.439724+0000","last_fullsized":"2024-01-11T07:18:13.439724+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:24.570897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439455+0000","last_change":"2024-01-11T07:18:13.439455+0000","last_active":"2024-01-11T07:18:13.439455+0000","last_peered":"2024-01-11T07:18:13.439455+0000","last_clean":"2024-01-11T07:18:13.439455+0000","last_became_active":"2024-01-11T07:18:13.439215+0000","last_became_peered":"2024-01-11T07:18:13.439215+0000","last_unstale":"2024-01-11T07:18:13.439455+0000","last_undegraded":"2024-01-11T07:18:13.439455+0000","last_fullsized":"2024-01-11T07:18:13.439455+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:38:51.503114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.432761+0000","last_change":"2024-01-11T07:18:13.432761+0000","last_active":"2024-01-11T07:18:13.432761+0000","last_peered":"2024-01-11T07:18:13.432761+0000","last_clean":"2024-01-11T07:18:13.432761+0000","last_became_active":"2024-01-11T07:18:13.432373+0000","last_became_peered":"2024-01-11T07:18:13.432373+0000","last_unstale":"2024-01-11T07:18:13.432761+0000","last_undegraded":"2024-01-11T07:18:13.432761+0000","last_fullsized":"2024-01-11T07:18:13.432761+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:04:10.626736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.443002+0000","last_change":"2024-01-11T07:18:13.443002+0000","last_active":"2024-01-11T07:18:13.443002+0000","last_peered":"2024-01-11T07:18:13.443002+0000","last_clean":"2024-01-11T07:18:13.443002+0000","last_became_active":"2024-01-11T07:18:13.437719+0000","last_became_peered":"2024-01-11T07:18:13.437719+0000","last_unstale":"2024-01-11T07:18:13.443002+0000","last_undegraded":"2024-01-11T07:18:13.443002+0000","last_fullsized":"2024-01-11T07:18:13.443002+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:00:08.705803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435741+0000","last_change":"2024-01-11T07:18:13.435741+0000","last_active":"2024-01-11T07:18:13.435741+0000","last_peered":"2024-01-11T07:18:13.435741+0000","last_clean":"2024-01-11T07:18:13.435741+0000","last_became_active":"2024-01-11T07:18:13.435450+0000","last_became_peered":"2024-01-11T07:18:13.435450+0000","last_unstale":"2024-01-11T07:18:13.435741+0000","last_undegraded":"2024-01-11T07:18:13.435741+0000","last_fullsized":"2024-01-11T07:18:13.435741+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:18.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438557+0000","last_change":"2024-01-11T07:18:13.438557+0000","last_active":"2024-01-11T07:18:13.438557+0000","last_peered":"2024-01-11T07:18:13.438557+0000","last_clean":"2024-01-11T07:18:13.438557+0000","last_became_active":"2024-01-11T07:18:13.438310+0000","last_became_peered":"2024-01-11T07:18:13.438310+0000","last_unstale":"2024-01-11T07:18:13.438557+0000","last_undegraded":"2024-01-11T07:18:13.438557+0000","last_fullsized":"2024-01-11T07:18:13.438557+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:15:23.934950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435796+0000","last_change":"2024-01-11T07:18:13.435796+0000","last_active":"2024-01-11T07:18:13.435796+0000","last_peered":"2024-01-11T07:18:13.435796+0000","last_clean":"2024-01-11T07:18:13.435796+0000","last_became_active":"2024-01-11T07:18:13.435522+0000","last_became_peered":"2024-01-11T07:18:13.435522+0000","last_unstale":"2024-01-11T07:18:13.435796+0000","last_undegraded":"2024-01-11T07:18:13.435796+0000","last_fullsized":"2024-01-11T07:18:13.435796+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:51.546656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.430873+0000","last_change":"2024-01-11T07:18:13.430873+0000","last_active":"2024-01-11T07:18:13.430873+0000","last_peered":"2024-01-11T07:18:13.430873+0000","last_clean":"2024-01-11T07:18:13.430873+0000","last_became_active":"2024-01-11T07:18:13.430304+0000","last_became_peered":"2024-01-11T07:18:13.430304+0000","last_unstale":"2024-01-11T07:18:13.430873+0000","last_undegraded":"2024-01-11T07:18:13.430873+0000","last_fullsized":"2024-01-11T07:18:13.430873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:27:35.430314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435559+0000","last_change":"2024-01-11T07:18:13.435559+0000","last_active":"2024-01-11T07:18:13.435559+0000","last_peered":"2024-01-11T07:18:13.435559+0000","last_clean":"2024-01-11T07:18:13.435559+0000","last_became_active":"2024-01-11T07:18:13.434507+0000","last_became_peered":"2024-01-11T07:18:13.434507+0000","last_unstale":"2024-01-11T07:18:13.435559+0000","last_undegraded":"2024-01-11T07:18:13.435559+0000","last_fullsized":"2024-01-11T07:18:13.435559+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:31:18.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437086+0000","last_change":"2024-01-11T07:18:13.437086+0000","last_active":"2024-01-11T07:18:13.437086+0000","last_peered":"2024-01-11T07:18:13.437086+0000","last_clean":"2024-01-11T07:18:13.437086+0000","last_became_active":"2024-01-11T07:18:13.436593+0000","last_became_peered":"2024-01-11T07:18:13.436593+0000","last_unstale":"2024-01-11T07:18:13.437086+0000","last_undegraded":"2024-01-11T07:18:13.437086+0000","last_fullsized":"2024-01-11T07:18:13.437086+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:57.208882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446691+0000","last_change":"2024-01-11T07:18:13.446691+0000","last_active":"2024-01-11T07:18:13.446691+0000","last_peered":"2024-01-11T07:18:13.446691+0000","last_clean":"2024-01-11T07:18:13.446691+0000","last_became_active":"2024-01-11T07:18:13.446257+0000","last_became_peered":"2024-01-11T07:18:13.446257+0000","last_unstale":"2024-01-11T07:18:13.446691+0000","last_undegraded":"2024-01-11T07:18:13.446691+0000","last_fullsized":"2024-01-11T07:18:13.446691+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:22:22.989834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"183'4","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.445178+0000","last_change":"2024-01-11T07:18:12.434944+0000","last_active":"2024-01-11T07:18:13.445178+0000","last_peered":"2024-01-11T07:18:13.445178+0000","last_clean":"2024-01-11T07:18:13.445178+0000","last_became_active":"2024-01-11T07:18:12.434704+0000","last_became_peered":"2024-01-11T07:18:12.434704+0000","last_unstale":"2024-01-11T07:18:13.445178+0000","last_undegraded":"2024-01-11T07:18:13.445178+0000","last_fullsized":"2024-01-11T07:18:13.445178+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:36:04.821856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435574+0000","last_change":"2024-01-11T07:18:13.435574+0000","last_active":"2024-01-11T07:18:13.435574+0000","last_peered":"2024-01-11T07:18:13.435574+0000","last_clean":"2024-01-11T07:18:13.435574+0000","last_became_active":"2024-01-11T07:18:13.434286+0000","last_became_peered":"2024-01-11T07:18:13.434286+0000","last_unstale":"2024-01-11T07:18:13.435574+0000","last_undegraded":"2024-01-11T07:18:13.435574+0000","last_fullsized":"2024-01-11T07:18:13.435574+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:38:33.528944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.427765+0000","last_change":"2024-01-11T07:18:12.434673+0000","last_active":"2024-01-11T07:18:13.427765+0000","last_peered":"2024-01-11T07:18:13.427765+0000","last_clean":"2024-01-11T07:18:13.427765+0000","last_became_active":"2024-01-11T07:18:12.434409+0000","last_became_peered":"2024-01-11T07:18:12.434409+0000","last_unstale":"2024-01-11T07:18:13.427765+0000","last_undegraded":"2024-01-11T07:18:13.427765+0000","last_fullsized":"2024-01-11T07:18:13.427765+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:39:00.978523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435758+0000","last_change":"2024-01-11T07:18:13.435758+0000","last_active":"2024-01-11T07:18:13.435758+0000","last_peered":"2024-01-11T07:18:13.435758+0000","last_clean":"2024-01-11T07:18:13.435758+0000","last_became_active":"2024-01-11T07:18:13.435414+0000","last_became_peered":"2024-01-11T07:18:13.435414+0000","last_unstale":"2024-01-11T07:18:13.435758+0000","last_undegraded":"2024-01-11T07:18:13.435758+0000","last_fullsized":"2024-01-11T07:18:13.435758+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:42:38.595631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.444209+0000","last_change":"2024-01-11T07:18:12.434576+0000","last_active":"2024-01-11T07:18:13.444209+0000","last_peered":"2024-01-11T07:18:13.444209+0000","last_clean":"2024-01-11T07:18:13.444209+0000","last_became_active":"2024-01-11T07:18:12.434186+0000","last_became_peered":"2024-01-11T07:18:12.434186+0000","last_unstale":"2024-01-11T07:18:13.444209+0000","last_undegraded":"2024-01-11T07:18:13.444209+0000","last_fullsized":"2024-01-11T07:18:13.444209+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:03:37.188892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434536+0000","last_change":"2024-01-11T07:18:13.434536+0000","last_active":"2024-01-11T07:18:13.434536+0000","last_peered":"2024-01-11T07:18:13.434536+0000","last_clean":"2024-01-11T07:18:13.434536+0000","last_became_active":"2024-01-11T07:18:13.434310+0000","last_became_peered":"2024-01-11T07:18:13.434310+0000","last_unstale":"2024-01-11T07:18:13.434536+0000","last_undegraded":"2024-01-11T07:18:13.434536+0000","last_fullsized":"2024-01-11T07:18:13.434536+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:54.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.444201+0000","last_change":"2024-01-11T07:18:12.434054+0000","last_active":"2024-01-11T07:18:13.444201+0000","last_peered":"2024-01-11T07:18:13.444201+0000","last_clean":"2024-01-11T07:18:13.444201+0000","last_became_active":"2024-01-11T07:18:12.433739+0000","last_became_peered":"2024-01-11T07:18:12.433739+0000","last_unstale":"2024-01-11T07:18:13.444201+0000","last_undegraded":"2024-01-11T07:18:13.444201+0000","last_fullsized":"2024-01-11T07:18:13.444201+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:27:34.424904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.433155+0000","last_change":"2024-01-11T07:18:13.433155+0000","last_active":"2024-01-11T07:18:13.433155+0000","last_peered":"2024-01-11T07:18:13.433155+0000","last_clean":"2024-01-11T07:18:13.433155+0000","last_became_active":"2024-01-11T07:18:13.432709+0000","last_became_peered":"2024-01-11T07:18:13.432709+0000","last_unstale":"2024-01-11T07:18:13.433155+0000","last_undegraded":"2024-01-11T07:18:13.433155+0000","last_fullsized":"2024-01-11T07:18:13.433155+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:53:35.336716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.431425+0000","last_change":"2024-01-11T07:18:12.432647+0000","last_active":"2024-01-11T07:18:13.431425+0000","last_peered":"2024-01-11T07:18:13.431425+0000","last_clean":"2024-01-11T07:18:13.431425+0000","last_became_active":"2024-01-11T07:18:12.429452+0000","last_became_peered":"2024-01-11T07:18:12.429452+0000","last_unstale":"2024-01-11T07:18:13.431425+0000","last_undegraded":"2024-01-11T07:18:13.431425+0000","last_fullsized":"2024-01-11T07:18:13.431425+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:58.458880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.447261+0000","last_change":"2024-01-11T07:18:13.447261+0000","last_active":"2024-01-11T07:18:13.447261+0000","last_peered":"2024-01-11T07:18:13.447261+0000","last_clean":"2024-01-11T07:18:13.447261+0000","last_became_active":"2024-01-11T07:18:13.446825+0000","last_became_peered":"2024-01-11T07:18:13.446825+0000","last_unstale":"2024-01-11T07:18:13.447261+0000","last_undegraded":"2024-01-11T07:18:13.447261+0000","last_fullsized":"2024-01-11T07:18:13.447261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:45:00.490536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.430195+0000","last_change":"2024-01-11T07:18:12.436217+0000","last_active":"2024-01-11T07:18:13.430195+0000","last_peered":"2024-01-11T07:18:13.430195+0000","last_clean":"2024-01-11T07:18:13.430195+0000","last_became_active":"2024-01-11T07:18:12.435827+0000","last_became_peered":"2024-01-11T07:18:12.435827+0000","last_unstale":"2024-01-11T07:18:13.430195+0000","last_undegraded":"2024-01-11T07:18:13.430195+0000","last_fullsized":"2024-01-11T07:18:13.430195+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:09:47.377371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434648+0000","last_change":"2024-01-11T07:18:13.434648+0000","last_active":"2024-01-11T07:18:13.434648+0000","last_peered":"2024-01-11T07:18:13.434648+0000","last_clean":"2024-01-11T07:18:13.434648+0000","last_became_active":"2024-01-11T07:18:13.434108+0000","last_became_peered":"2024-01-11T07:18:13.434108+0000","last_unstale":"2024-01-11T07:18:13.434648+0000","last_undegraded":"2024-01-11T07:18:13.434648+0000","last_fullsized":"2024-01-11T07:18:13.434648+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:00:45.671429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.434290+0000","last_change":"2024-01-11T07:18:12.434191+0000","last_active":"2024-01-11T07:18:12.434290+0000","last_peered":"2024-01-11T07:18:12.434290+0000","last_clean":"2024-01-11T07:18:12.434290+0000","last_became_active":"2024-01-11T07:18:12.433522+0000","last_became_peered":"2024-01-11T07:18:12.433522+0000","last_unstale":"2024-01-11T07:18:12.434290+0000","last_undegraded":"2024-01-11T07:18:12.434290+0000","last_fullsized":"2024-01-11T07:18:12.434290+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:05:48.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446975+0000","last_change":"2024-01-11T07:18:13.446975+0000","last_active":"2024-01-11T07:18:13.446975+0000","last_peered":"2024-01-11T07:18:13.446975+0000","last_clean":"2024-01-11T07:18:13.446975+0000","last_became_active":"2024-01-11T07:18:13.446777+0000","last_became_peered":"2024-01-11T07:18:13.446777+0000","last_unstale":"2024-01-11T07:18:13.446975+0000","last_undegraded":"2024-01-11T07:18:13.446975+0000","last_fullsized":"2024-01-11T07:18:13.446975+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:10:57.278546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.430244+0000","last_change":"2024-01-11T07:18:12.430160+0000","last_active":"2024-01-11T07:18:12.430244+0000","last_peered":"2024-01-11T07:18:12.430244+0000","last_clean":"2024-01-11T07:18:12.430244+0000","last_became_active":"2024-01-11T07:18:12.429880+0000","last_became_peered":"2024-01-11T07:18:12.429880+0000","last_unstale":"2024-01-11T07:18:12.430244+0000","last_undegraded":"2024-01-11T07:18:12.430244+0000","last_fullsized":"2024-01-11T07:18:12.430244+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:45:56.778317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438048+0000","last_change":"2024-01-11T07:18:13.438048+0000","last_active":"2024-01-11T07:18:13.438048+0000","last_peered":"2024-01-11T07:18:13.438048+0000","last_clean":"2024-01-11T07:18:13.438048+0000","last_became_active":"2024-01-11T07:18:13.437771+0000","last_became_peered":"2024-01-11T07:18:13.437771+0000","last_unstale":"2024-01-11T07:18:13.438048+0000","last_undegraded":"2024-01-11T07:18:13.438048+0000","last_fullsized":"2024-01-11T07:18:13.438048+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:11:50.494109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.429748+0000","last_change":"2024-01-11T07:18:12.429686+0000","last_active":"2024-01-11T07:18:12.429748+0000","last_peered":"2024-01-11T07:18:12.429748+0000","last_clean":"2024-01-11T07:18:12.429748+0000","last_became_active":"2024-01-11T07:18:12.429317+0000","last_became_peered":"2024-01-11T07:18:12.429317+0000","last_unstale":"2024-01-11T07:18:12.429748+0000","last_undegraded":"2024-01-11T07:18:12.429748+0000","last_fullsized":"2024-01-11T07:18:12.429748+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:50:45.944040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.440563+0000","last_change":"2024-01-11T07:18:13.440563+0000","last_active":"2024-01-11T07:18:13.440563+0000","last_peered":"2024-01-11T07:18:13.440563+0000","last_clean":"2024-01-11T07:18:13.440563+0000","last_became_active":"2024-01-11T07:18:13.440073+0000","last_became_peered":"2024-01-11T07:18:13.440073+0000","last_unstale":"2024-01-11T07:18:13.440563+0000","last_undegraded":"2024-01-11T07:18:13.440563+0000","last_fullsized":"2024-01-11T07:18:13.440563+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:47:25.278556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"183'3","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.445394+0000","last_change":"2024-01-11T07:18:12.433268+0000","last_active":"2024-01-11T07:18:13.445394+0000","last_peered":"2024-01-11T07:18:13.445394+0000","last_clean":"2024-01-11T07:18:13.445394+0000","last_became_active":"2024-01-11T07:18:12.433027+0000","last_became_peered":"2024-01-11T07:18:12.433027+0000","last_unstale":"2024-01-11T07:18:13.445394+0000","last_undegraded":"2024-01-11T07:18:13.445394+0000","last_fullsized":"2024-01-11T07:18:13.445394+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:03:20.180539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435423+0000","last_change":"2024-01-11T07:18:13.435423+0000","last_active":"2024-01-11T07:18:13.435423+0000","last_peered":"2024-01-11T07:18:13.435423+0000","last_clean":"2024-01-11T07:18:13.435423+0000","last_became_active":"2024-01-11T07:18:13.433970+0000","last_became_peered":"2024-01-11T07:18:13.433970+0000","last_unstale":"2024-01-11T07:18:13.435423+0000","last_undegraded":"2024-01-11T07:18:13.435423+0000","last_fullsized":"2024-01-11T07:18:13.435423+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:26:16.527337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"183'6","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446795+0000","last_change":"2024-01-11T07:18:12.431882+0000","last_active":"2024-01-11T07:18:13.446795+0000","last_peered":"2024-01-11T07:18:13.446795+0000","last_clean":"2024-01-11T07:18:13.446795+0000","last_became_active":"2024-01-11T07:18:12.431583+0000","last_became_peered":"2024-01-11T07:18:12.431583+0000","last_unstale":"2024-01-11T07:18:13.446795+0000","last_undegraded":"2024-01-11T07:18:13.446795+0000","last_fullsized":"2024-01-11T07:18:13.446795+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:38.394350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.441281+0000","last_change":"2024-01-11T07:18:13.441281+0000","last_active":"2024-01-11T07:18:13.441281+0000","last_peered":"2024-01-11T07:18:13.441281+0000","last_clean":"2024-01-11T07:18:13.441281+0000","last_became_active":"2024-01-11T07:18:13.439782+0000","last_became_peered":"2024-01-11T07:18:13.439782+0000","last_unstale":"2024-01-11T07:18:13.441281+0000","last_undegraded":"2024-01-11T07:18:13.441281+0000","last_fullsized":"2024-01-11T07:18:13.441281+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:42:12.592049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446153+0000","last_change":"2024-01-11T07:18:12.438486+0000","last_active":"2024-01-11T07:18:13.446153+0000","last_peered":"2024-01-11T07:18:13.446153+0000","last_clean":"2024-01-11T07:18:13.446153+0000","last_became_active":"2024-01-11T07:18:12.438244+0000","last_became_peered":"2024-01-11T07:18:12.438244+0000","last_unstale":"2024-01-11T07:18:13.446153+0000","last_undegraded":"2024-01-11T07:18:13.446153+0000","last_fullsized":"2024-01-11T07:18:13.446153+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:17.380079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437065+0000","last_change":"2024-01-11T07:18:13.437065+0000","last_active":"2024-01-11T07:18:13.437065+0000","last_peered":"2024-01-11T07:18:13.437065+0000","last_clean":"2024-01-11T07:18:13.437065+0000","last_became_active":"2024-01-11T07:18:13.436829+0000","last_became_peered":"2024-01-11T07:18:13.436829+0000","last_unstale":"2024-01-11T07:18:13.437065+0000","last_undegraded":"2024-01-11T07:18:13.437065+0000","last_fullsized":"2024-01-11T07:18:13.437065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:33.525439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.435958+0000","last_change":"2024-01-11T07:18:12.435851+0000","last_active":"2024-01-11T07:18:12.435958+0000","last_peered":"2024-01-11T07:18:12.435958+0000","last_clean":"2024-01-11T07:18:12.435958+0000","last_became_active":"2024-01-11T07:18:12.435484+0000","last_became_peered":"2024-01-11T07:18:12.435484+0000","last_unstale":"2024-01-11T07:18:12.435958+0000","last_undegraded":"2024-01-11T07:18:12.435958+0000","last_fullsized":"2024-01-11T07:18:12.435958+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:05:49.052324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.433803+0000","last_change":"2024-01-11T07:18:13.433803+0000","last_active":"2024-01-11T07:18:13.433803+0000","last_peered":"2024-01-11T07:18:13.433803+0000","last_clean":"2024-01-11T07:18:13.433803+0000","last_became_active":"2024-01-11T07:18:13.433527+0000","last_became_peered":"2024-01-11T07:18:13.433527+0000","last_unstale":"2024-01-11T07:18:13.433803+0000","last_undegraded":"2024-01-11T07:18:13.433803+0000","last_fullsized":"2024-01-11T07:18:13.433803+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:23:32.552145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.437869+0000","last_change":"2024-01-11T07:18:12.437767+0000","last_active":"2024-01-11T07:18:12.437869+0000","last_peered":"2024-01-11T07:18:12.437869+0000","last_clean":"2024-01-11T07:18:12.437869+0000","last_became_active":"2024-01-11T07:18:12.437024+0000","last_became_peered":"2024-01-11T07:18:12.437024+0000","last_unstale":"2024-01-11T07:18:12.437869+0000","last_undegraded":"2024-01-11T07:18:12.437869+0000","last_fullsized":"2024-01-11T07:18:12.437869+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:42:57.744348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.443960+0000","last_change":"2024-01-11T07:18:13.443960+0000","last_active":"2024-01-11T07:18:13.443960+0000","last_peered":"2024-01-11T07:18:13.443960+0000","last_clean":"2024-01-11T07:18:13.443960+0000","last_became_active":"2024-01-11T07:18:13.443685+0000","last_became_peered":"2024-01-11T07:18:13.443685+0000","last_unstale":"2024-01-11T07:18:13.443960+0000","last_undegraded":"2024-01-11T07:18:13.443960+0000","last_fullsized":"2024-01-11T07:18:13.443960+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:40:59.297452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"183'8","reported_seq":17,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446472+0000","last_change":"2024-01-11T07:18:12.433398+0000","last_active":"2024-01-11T07:18:13.446472+0000","last_peered":"2024-01-11T07:18:13.446472+0000","last_clean":"2024-01-11T07:18:13.446472+0000","last_became_active":"2024-01-11T07:18:12.433080+0000","last_became_peered":"2024-01-11T07:18:12.433080+0000","last_unstale":"2024-01-11T07:18:13.446472+0000","last_undegraded":"2024-01-11T07:18:13.446472+0000","last_fullsized":"2024-01-11T07:18:13.446472+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:38:50.497704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434792+0000","last_change":"2024-01-11T07:18:13.434792+0000","last_active":"2024-01-11T07:18:13.434792+0000","last_peered":"2024-01-11T07:18:13.434792+0000","last_clean":"2024-01-11T07:18:13.434792+0000","last_became_active":"2024-01-11T07:18:13.434420+0000","last_became_peered":"2024-01-11T07:18:13.434420+0000","last_unstale":"2024-01-11T07:18:13.434792+0000","last_undegraded":"2024-01-11T07:18:13.434792+0000","last_fullsized":"2024-01-11T07:18:13.434792+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:18:00.125897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.430847+0000","last_change":"2024-01-11T07:18:12.432911+0000","last_active":"2024-01-11T07:18:13.430847+0000","last_peered":"2024-01-11T07:18:13.430847+0000","last_clean":"2024-01-11T07:18:13.430847+0000","last_became_active":"2024-01-11T07:18:12.432730+0000","last_became_peered":"2024-01-11T07:18:12.432730+0000","last_unstale":"2024-01-11T07:18:13.430847+0000","last_undegraded":"2024-01-11T07:18:13.430847+0000","last_fullsized":"2024-01-11T07:18:13.430847+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:18:59.343833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.436900+0000","last_change":"2024-01-11T07:18:13.436900+0000","last_active":"2024-01-11T07:18:13.436900+0000","last_peered":"2024-01-11T07:18:13.436900+0000","last_clean":"2024-01-11T07:18:13.436900+0000","last_became_active":"2024-01-11T07:18:13.435690+0000","last_became_peered":"2024-01-11T07:18:13.435690+0000","last_unstale":"2024-01-11T07:18:13.436900+0000","last_undegraded":"2024-01-11T07:18:13.436900+0000","last_fullsized":"2024-01-11T07:18:13.436900+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:16:28.860520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.444183+0000","last_change":"2024-01-11T07:18:12.437228+0000","last_active":"2024-01-11T07:18:13.444183+0000","last_peered":"2024-01-11T07:18:13.444183+0000","last_clean":"2024-01-11T07:18:13.444183+0000","last_became_active":"2024-01-11T07:18:12.436991+0000","last_became_peered":"2024-01-11T07:18:12.436991+0000","last_unstale":"2024-01-11T07:18:13.444183+0000","last_undegraded":"2024-01-11T07:18:13.444183+0000","last_fullsized":"2024-01-11T07:18:13.444183+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:13:46.190492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.447100+0000","last_change":"2024-01-11T07:18:13.447100+0000","last_active":"2024-01-11T07:18:13.447100+0000","last_peered":"2024-01-11T07:18:13.447100+0000","last_clean":"2024-01-11T07:18:13.447100+0000","last_became_active":"2024-01-11T07:18:13.446480+0000","last_became_peered":"2024-01-11T07:18:13.446480+0000","last_unstale":"2024-01-11T07:18:13.447100+0000","last_undegraded":"2024-01-11T07:18:13.447100+0000","last_fullsized":"2024-01-11T07:18:13.447100+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:18:53.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.428874+0000","last_change":"2024-01-11T07:18:12.432131+0000","last_active":"2024-01-11T07:18:13.428874+0000","last_peered":"2024-01-11T07:18:13.428874+0000","last_clean":"2024-01-11T07:18:13.428874+0000","last_became_active":"2024-01-11T07:18:12.431903+0000","last_became_peered":"2024-01-11T07:18:12.431903+0000","last_unstale":"2024-01-11T07:18:13.428874+0000","last_undegraded":"2024-01-11T07:18:13.428874+0000","last_fullsized":"2024-01-11T07:18:13.428874+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:05:19.542803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.436011+0000","last_change":"2024-01-11T07:18:13.436011+0000","last_active":"2024-01-11T07:18:13.436011+0000","last_peered":"2024-01-11T07:18:13.436011+0000","last_clean":"2024-01-11T07:18:13.436011+0000","last_became_active":"2024-01-11T07:18:13.435608+0000","last_became_peered":"2024-01-11T07:18:13.435608+0000","last_unstale":"2024-01-11T07:18:13.436011+0000","last_undegraded":"2024-01-11T07:18:13.436011+0000","last_fullsized":"2024-01-11T07:18:13.436011+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:32:17.344291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.436379+0000","last_change":"2024-01-11T07:18:12.436309+0000","last_active":"2024-01-11T07:18:12.436379+0000","last_peered":"2024-01-11T07:18:12.436379+0000","last_clean":"2024-01-11T07:18:12.436379+0000","last_became_active":"2024-01-11T07:18:12.436046+0000","last_became_peered":"2024-01-11T07:18:12.436046+0000","last_unstale":"2024-01-11T07:18:12.436379+0000","last_undegraded":"2024-01-11T07:18:12.436379+0000","last_fullsized":"2024-01-11T07:18:12.436379+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:39.044311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438191+0000","last_change":"2024-01-11T07:18:13.438191+0000","last_active":"2024-01-11T07:18:13.438191+0000","last_peered":"2024-01-11T07:18:13.438191+0000","last_clean":"2024-01-11T07:18:13.438191+0000","last_became_active":"2024-01-11T07:18:13.437914+0000","last_became_peered":"2024-01-11T07:18:13.437914+0000","last_unstale":"2024-01-11T07:18:13.438191+0000","last_undegraded":"2024-01-11T07:18:13.438191+0000","last_fullsized":"2024-01-11T07:18:13.438191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:38:51.503114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.445458+0000","last_change":"2024-01-11T07:18:12.436073+0000","last_active":"2024-01-11T07:18:13.445458+0000","last_peered":"2024-01-11T07:18:13.445458+0000","last_clean":"2024-01-11T07:18:13.445458+0000","last_became_active":"2024-01-11T07:18:12.435530+0000","last_became_peered":"2024-01-11T07:18:12.435530+0000","last_unstale":"2024-01-11T07:18:13.445458+0000","last_undegraded":"2024-01-11T07:18:13.445458+0000","last_fullsized":"2024-01-11T07:18:13.445458+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:50.541247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.432992+0000","last_change":"2024-01-11T07:18:13.432992+0000","last_active":"2024-01-11T07:18:13.432992+0000","last_peered":"2024-01-11T07:18:13.432992+0000","last_clean":"2024-01-11T07:18:13.432992+0000","last_became_active":"2024-01-11T07:18:13.432356+0000","last_became_peered":"2024-01-11T07:18:13.432356+0000","last_unstale":"2024-01-11T07:18:13.432992+0000","last_undegraded":"2024-01-11T07:18:13.432992+0000","last_fullsized":"2024-01-11T07:18:13.432992+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:26:44.488764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.430076+0000","last_change":"2024-01-11T07:18:12.430024+0000","last_active":"2024-01-11T07:18:12.430076+0000","last_peered":"2024-01-11T07:18:12.430076+0000","last_clean":"2024-01-11T07:18:12.430076+0000","last_became_active":"2024-01-11T07:18:12.429807+0000","last_became_peered":"2024-01-11T07:18:12.429807+0000","last_unstale":"2024-01-11T07:18:12.430076+0000","last_undegraded":"2024-01-11T07:18:12.430076+0000","last_fullsized":"2024-01-11T07:18:12.430076+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:18:14.435862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439579+0000","last_change":"2024-01-11T07:18:13.439579+0000","last_active":"2024-01-11T07:18:13.439579+0000","last_peered":"2024-01-11T07:18:13.439579+0000","last_clean":"2024-01-11T07:18:13.439579+0000","last_became_active":"2024-01-11T07:18:13.439335+0000","last_became_peered":"2024-01-11T07:18:13.439335+0000","last_unstale":"2024-01-11T07:18:13.439579+0000","last_undegraded":"2024-01-11T07:18:13.439579+0000","last_fullsized":"2024-01-11T07:18:13.439579+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:05:57.809342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.434524+0000","last_change":"2024-01-11T07:18:12.434422+0000","last_active":"2024-01-11T07:18:12.434524+0000","last_peered":"2024-01-11T07:18:12.434524+0000","last_clean":"2024-01-11T07:18:12.434524+0000","last_became_active":"2024-01-11T07:18:12.433849+0000","last_became_peered":"2024-01-11T07:18:12.433849+0000","last_unstale":"2024-01-11T07:18:12.434524+0000","last_undegraded":"2024-01-11T07:18:12.434524+0000","last_fullsized":"2024-01-11T07:18:12.434524+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:28:03.367767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.443306+0000","last_change":"2024-01-11T07:18:13.443306+0000","last_active":"2024-01-11T07:18:13.443306+0000","last_peered":"2024-01-11T07:18:13.443306+0000","last_clean":"2024-01-11T07:18:13.443306+0000","last_became_active":"2024-01-11T07:18:13.443066+0000","last_became_peered":"2024-01-11T07:18:13.443066+0000","last_unstale":"2024-01-11T07:18:13.443306+0000","last_undegraded":"2024-01-11T07:18:13.443306+0000","last_fullsized":"2024-01-11T07:18:13.443306+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:33:42.414027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.432669+0000","last_change":"2024-01-11T07:18:12.432596+0000","last_active":"2024-01-11T07:18:12.432669+0000","last_peered":"2024-01-11T07:18:12.432669+0000","last_clean":"2024-01-11T07:18:12.432669+0000","last_became_active":"2024-01-11T07:18:12.432369+0000","last_became_peered":"2024-01-11T07:18:12.432369+0000","last_unstale":"2024-01-11T07:18:12.432669+0000","last_undegraded":"2024-01-11T07:18:12.432669+0000","last_fullsized":"2024-01-11T07:18:12.432669+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:45:56.778317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.444102+0000","last_change":"2024-01-11T07:18:13.444102+0000","last_active":"2024-01-11T07:18:13.444102+0000","last_peered":"2024-01-11T07:18:13.444102+0000","last_clean":"2024-01-11T07:18:13.444102+0000","last_became_active":"2024-01-11T07:18:13.443624+0000","last_became_peered":"2024-01-11T07:18:13.443624+0000","last_unstale":"2024-01-11T07:18:13.444102+0000","last_undegraded":"2024-01-11T07:18:13.444102+0000","last_fullsized":"2024-01-11T07:18:13.444102+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:00:08.705803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.445077+0000","last_change":"2024-01-11T07:18:12.436901+0000","last_active":"2024-01-11T07:18:13.445077+0000","last_peered":"2024-01-11T07:18:13.445077+0000","last_clean":"2024-01-11T07:18:13.445077+0000","last_became_active":"2024-01-11T07:18:12.436392+0000","last_became_peered":"2024-01-11T07:18:12.436392+0000","last_unstale":"2024-01-11T07:18:13.445077+0000","last_undegraded":"2024-01-11T07:18:13.445077+0000","last_fullsized":"2024-01-11T07:18:13.445077+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:39:59.914390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.442311+0000","last_change":"2024-01-11T07:18:13.442311+0000","last_active":"2024-01-11T07:18:13.442311+0000","last_peered":"2024-01-11T07:18:13.442311+0000","last_clean":"2024-01-11T07:18:13.442311+0000","last_became_active":"2024-01-11T07:18:13.441964+0000","last_became_peered":"2024-01-11T07:18:13.441964+0000","last_unstale":"2024-01-11T07:18:13.442311+0000","last_undegraded":"2024-01-11T07:18:13.442311+0000","last_fullsized":"2024-01-11T07:18:13.442311+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:32:58.893111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.434303+0000","last_change":"2024-01-11T07:18:12.434238+0000","last_active":"2024-01-11T07:18:12.434303+0000","last_peered":"2024-01-11T07:18:12.434303+0000","last_clean":"2024-01-11T07:18:12.434303+0000","last_became_active":"2024-01-11T07:18:12.434042+0000","last_became_peered":"2024-01-11T07:18:12.434042+0000","last_unstale":"2024-01-11T07:18:12.434303+0000","last_undegraded":"2024-01-11T07:18:12.434303+0000","last_fullsized":"2024-01-11T07:18:12.434303+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:42:37.590222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437355+0000","last_change":"2024-01-11T07:18:13.437355+0000","last_active":"2024-01-11T07:18:13.437355+0000","last_peered":"2024-01-11T07:18:13.437355+0000","last_clean":"2024-01-11T07:18:13.437355+0000","last_became_active":"2024-01-11T07:18:13.437093+0000","last_became_peered":"2024-01-11T07:18:13.437093+0000","last_unstale":"2024-01-11T07:18:13.437355+0000","last_undegraded":"2024-01-11T07:18:13.437355+0000","last_fullsized":"2024-01-11T07:18:13.437355+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:03:31.690384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.437614+0000","last_change":"2024-01-11T07:18:12.437526+0000","last_active":"2024-01-11T07:18:12.437614+0000","last_peered":"2024-01-11T07:18:12.437614+0000","last_clean":"2024-01-11T07:18:12.437614+0000","last_became_active":"2024-01-11T07:18:12.437266+0000","last_became_peered":"2024-01-11T07:18:12.437266+0000","last_unstale":"2024-01-11T07:18:12.437614+0000","last_undegraded":"2024-01-11T07:18:12.437614+0000","last_fullsized":"2024-01-11T07:18:12.437614+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:10:56.273136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437055+0000","last_change":"2024-01-11T07:18:13.437055+0000","last_active":"2024-01-11T07:18:13.437055+0000","last_peered":"2024-01-11T07:18:13.437055+0000","last_clean":"2024-01-11T07:18:13.437055+0000","last_became_active":"2024-01-11T07:18:13.436546+0000","last_became_peered":"2024-01-11T07:18:13.436546+0000","last_unstale":"2024-01-11T07:18:13.437055+0000","last_undegraded":"2024-01-11T07:18:13.437055+0000","last_fullsized":"2024-01-11T07:18:13.437055+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:05:20.548212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.439388+0000","last_change":"2024-01-11T07:18:12.437511+0000","last_active":"2024-01-11T07:18:12.439388+0000","last_peered":"2024-01-11T07:18:12.439388+0000","last_clean":"2024-01-11T07:18:12.439388+0000","last_became_active":"2024-01-11T07:18:12.436266+0000","last_became_peered":"2024-01-11T07:18:12.436266+0000","last_unstale":"2024-01-11T07:18:12.439388+0000","last_undegraded":"2024-01-11T07:18:12.439388+0000","last_fullsized":"2024-01-11T07:18:12.439388+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:28:14.594448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.436926+0000","last_change":"2024-01-11T07:18:13.436926+0000","last_active":"2024-01-11T07:18:13.436926+0000","last_peered":"2024-01-11T07:18:13.436926+0000","last_clean":"2024-01-11T07:18:13.436926+0000","last_became_active":"2024-01-11T07:18:13.436203+0000","last_became_peered":"2024-01-11T07:18:13.436203+0000","last_unstale":"2024-01-11T07:18:13.436926+0000","last_undegraded":"2024-01-11T07:18:13.436926+0000","last_fullsized":"2024-01-11T07:18:13.436926+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:13:29.642574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.436807+0000","last_change":"2024-01-11T07:18:12.436738+0000","last_active":"2024-01-11T07:18:12.436807+0000","last_peered":"2024-01-11T07:18:12.436807+0000","last_clean":"2024-01-11T07:18:12.436807+0000","last_became_active":"2024-01-11T07:18:12.436496+0000","last_became_peered":"2024-01-11T07:18:12.436496+0000","last_unstale":"2024-01-11T07:18:12.436807+0000","last_undegraded":"2024-01-11T07:18:12.436807+0000","last_fullsized":"2024-01-11T07:18:12.436807+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:50.541247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435710+0000","last_change":"2024-01-11T07:18:13.435710+0000","last_active":"2024-01-11T07:18:13.435710+0000","last_peered":"2024-01-11T07:18:13.435710+0000","last_clean":"2024-01-11T07:18:13.435710+0000","last_became_active":"2024-01-11T07:18:13.434837+0000","last_became_peered":"2024-01-11T07:18:13.434837+0000","last_unstale":"2024-01-11T07:18:13.435710+0000","last_undegraded":"2024-01-11T07:18:13.435710+0000","last_fullsized":"2024-01-11T07:18:13.435710+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:24:42.362653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.437850+0000","last_change":"2024-01-11T07:18:12.437585+0000","last_active":"2024-01-11T07:18:12.437850+0000","last_peered":"2024-01-11T07:18:12.437850+0000","last_clean":"2024-01-11T07:18:12.437850+0000","last_became_active":"2024-01-11T07:18:12.436473+0000","last_became_peered":"2024-01-11T07:18:12.436473+0000","last_unstale":"2024-01-11T07:18:12.437850+0000","last_undegraded":"2024-01-11T07:18:12.437850+0000","last_fullsized":"2024-01-11T07:18:12.437850+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:42:09.811842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446543+0000","last_change":"2024-01-11T07:18:13.446543+0000","last_active":"2024-01-11T07:18:13.446543+0000","last_peered":"2024-01-11T07:18:13.446543+0000","last_clean":"2024-01-11T07:18:13.446543+0000","last_became_active":"2024-01-11T07:18:13.445913+0000","last_became_peered":"2024-01-11T07:18:13.445913+0000","last_unstale":"2024-01-11T07:18:13.446543+0000","last_undegraded":"2024-01-11T07:18:13.446543+0000","last_fullsized":"2024-01-11T07:18:13.446543+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:14:22.129741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.437221+0000","last_change":"2024-01-11T07:18:12.437168+0000","last_active":"2024-01-11T07:18:12.437221+0000","last_peered":"2024-01-11T07:18:12.437221+0000","last_clean":"2024-01-11T07:18:12.437221+0000","last_became_active":"2024-01-11T07:18:12.436979+0000","last_became_peered":"2024-01-11T07:18:12.436979+0000","last_unstale":"2024-01-11T07:18:12.437221+0000","last_undegraded":"2024-01-11T07:18:12.437221+0000","last_fullsized":"2024-01-11T07:18:12.437221+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:03:30.684974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434298+0000","last_change":"2024-01-11T07:18:13.434298+0000","last_active":"2024-01-11T07:18:13.434298+0000","last_peered":"2024-01-11T07:18:13.434298+0000","last_clean":"2024-01-11T07:18:13.434298+0000","last_became_active":"2024-01-11T07:18:13.433918+0000","last_became_peered":"2024-01-11T07:18:13.433918+0000","last_unstale":"2024-01-11T07:18:13.434298+0000","last_undegraded":"2024-01-11T07:18:13.434298+0000","last_fullsized":"2024-01-11T07:18:13.434298+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:56:51.191605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.428221+0000","last_change":"2024-01-11T07:18:12.428149+0000","last_active":"2024-01-11T07:18:12.428221+0000","last_peered":"2024-01-11T07:18:12.428221+0000","last_clean":"2024-01-11T07:18:12.428221+0000","last_became_active":"2024-01-11T07:18:12.427881+0000","last_became_peered":"2024-01-11T07:18:12.427881+0000","last_unstale":"2024-01-11T07:18:12.428221+0000","last_undegraded":"2024-01-11T07:18:12.428221+0000","last_fullsized":"2024-01-11T07:18:12.428221+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:53.959251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439231+0000","last_change":"2024-01-11T07:18:13.439231+0000","last_active":"2024-01-11T07:18:13.439231+0000","last_peered":"2024-01-11T07:18:13.439231+0000","last_clean":"2024-01-11T07:18:13.439231+0000","last_became_active":"2024-01-11T07:18:13.438912+0000","last_became_peered":"2024-01-11T07:18:13.438912+0000","last_unstale":"2024-01-11T07:18:13.439231+0000","last_undegraded":"2024-01-11T07:18:13.439231+0000","last_fullsized":"2024-01-11T07:18:13.439231+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:33.525439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.438152+0000","last_change":"2024-01-11T07:18:12.438083+0000","last_active":"2024-01-11T07:18:12.438152+0000","last_peered":"2024-01-11T07:18:12.438152+0000","last_clean":"2024-01-11T07:18:12.438152+0000","last_became_active":"2024-01-11T07:18:12.437846+0000","last_became_peered":"2024-01-11T07:18:12.437846+0000","last_unstale":"2024-01-11T07:18:12.438152+0000","last_undegraded":"2024-01-11T07:18:12.438152+0000","last_fullsized":"2024-01-11T07:18:12.438152+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:04:09.621327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":308,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.427740+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:18:12.427740+0000","last_peered":"2024-01-11T07:18:12.427740+0000","last_clean":"2024-01-11T07:18:12.427740+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:18:12.427740+0000","last_undegraded":"2024-01-11T07:18:12.427740+0000","last_fullsized":"2024-01-11T07:18:12.427740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1569,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193742,"statfs":{"total":200000000,"available":198392404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.504}]},{"osd":1,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.433,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.56599999999999995},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.80500000000000005},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.81399999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75800000000000001,"5min":0.78700000000000003,"15min":0.82399999999999995},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.82999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42799999999999999,"15min":0.40600000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.83799999999999997},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45600000000000002,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.77700000000000002,"15min":0.85999999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.33800000000000002,"15min":0.308},"max":{"1min":0.59599999999999997,"5min":0.755,"15min":0.85899999999999999},"last":0.379}]},{"osd":5,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.66300000000000003,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.66600000000000004,"5min":0.84599999999999997,"15min":0.91600000000000004},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.82199999999999995,"15min":0.92500000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.90300000000000002},"last":0.437}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1000,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194312,"statfs":{"total":200000000,"available":198975987,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.81399999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.44600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77100000000000002,"5min":0.85199999999999998,"15min":1.157},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.45800000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65000000000000002,"5min":0.79500000000000004,"15min":2.5720000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":0.67300000000000004,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.84599999999999997,"15min":2.621},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88500000000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48099999999999998,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.25800000000000001},"max":{"1min":0.66300000000000003,"5min":0.76500000000000001,"15min":0.82699999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.73999999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.253},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":2.0209999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.313},"max":{"1min":0.69299999999999995,"5min":0.77900000000000003,"15min":0.89100000000000001},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28699999999999998},"max":{"1min":0.64100000000000001,"5min":0.68600000000000005,"15min":0.76700000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53200000000000003,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":0.70899999999999996,"15min":2.0750000000000002},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":993,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194319,"statfs":{"total":200000000,"available":198982772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.247,"15min":0.216},"max":{"1min":0.53100000000000003,"5min":0.753,"15min":20.41},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.46100000000000002},"min":{"1min":0.371,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":0.60699999999999998,"5min":0.72099999999999997,"15min":3.7029999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44,"15min":0.497},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60199999999999998,"5min":3.597,"15min":15.611000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":3.0710000000000002,"15min":20.199999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":0.67900000000000005,"5min":3.8460000000000001,"15min":20.369},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.28299999999999997},"max":{"1min":0.499,"5min":4.7569999999999997,"15min":8.5790000000000006},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":1.3300000000000001,"15min":9.9570000000000007},"last":0.443},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":1.2030000000000001,"15min":12.449999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":1.3540000000000001,"15min":2.6030000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.51300000000000001,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.90900000000000003,"5min":1.177,"15min":9.891},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":1.389,"15min":1.5569999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69199999999999995,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.94699999999999995,"5min":2.9049999999999998,"15min":9.9900000000000002},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.1599999999999999,"15min":20.324000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.46800000000000003},"max":{"1min":0.96999999999999997,"5min":1.268,"15min":2.548},"last":0.42399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1576,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193735,"statfs":{"total":200000000,"available":198385641,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39600000000000002,"15min":0.42999999999999999},"min":{"1min":0.27100000000000002,"5min":0.23300000000000001,"15min":0.224},"max":{"1min":0.57899999999999996,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.29299999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57399999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.48799999999999999},"min":{"1min":0.36499999999999999,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.61699999999999999,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.57799999999999996},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58899999999999997,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64000000000000001,"5min":0.83499999999999996,"15min":2.855},"last":0.37}]},{"osd":4,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.495,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90900000000000003,"15min":1.3959999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.40999999999999998},"max":{"1min":0.92500000000000004,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":1.032,"15min":2.819},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.96599999999999997,"15min":1.2490000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95899999999999996,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.501,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.68000000000000005},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.276},"last":0.746},{"interface":"front","average":{"1min":0.748,"5min":0.747,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.47499999999999998,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":993,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194319,"statfs":{"total":200000000,"available":198982772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.504,"5min":0.72999999999999998,"15min":1.472},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.311,"5min":0.27900000000000003,"15min":0.23599999999999999},"max":{"1min":0.57799999999999996,"5min":0.65500000000000003,"15min":0.86099999999999999},"last":0.29699999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.58899999999999997,"5min":0.628,"15min":1.2430000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48799999999999999},"min":{"1min":0.41399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63300000000000001,"5min":0.752,"15min":0.76800000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.63100000000000001,"5min":0.76500000000000001,"15min":1.139},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.0660000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.089},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48199999999999998,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":1.054,"15min":1.099},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":1.0389999999999999,"15min":1.3100000000000001},"last":0.623}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0109999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.052},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0580000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.55400000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.54600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":999,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194312,"statfs":{"total":200000000,"available":198976009,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.57399999999999995},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":0.58199999999999996,"5min":0.59699999999999998,"15min":11.481},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.47299999999999998},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":1.6200000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40699999999999997,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.23899999999999999},"max":{"1min":0.59899999999999998,"5min":0.70499999999999996,"15min":17.516999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67400000000000004},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":0.56000000000000005,"5min":0.746,"15min":0.76500000000000001},"last":0.5},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46400000000000002,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":17.553000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.94599999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.95099999999999996},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.36199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":17.474},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64600000000000002,"15min":0.89600000000000002},"min":{"1min":0.49199999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":17.393000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.81899999999999995},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.94499999999999995,"15min":11.394},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.82899999999999996},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.91900000000000004,"15min":11.436999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.878,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.96999999999999997,"15min":1.0449999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.93799999999999994},"min":{"1min":0.53700000000000003,"5min":0.45800000000000002,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":0.95499999999999996,"15min":17.434000000000001},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":993,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194319,"statfs":{"total":200000000,"available":198982772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.81299999999999994},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.73599999999999999,"5min":0.94799999999999995,"15min":8.484},"last":0.246},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.84599999999999997},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":0.84699999999999998,"15min":9.2599999999999998},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":1.1599999999999999,"15min":8.5340000000000007},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.78100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.81000000000000005,"15min":9.2279999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59999999999999998,"15min":0.78200000000000003},"min":{"1min":0.376,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.71999999999999997,"5min":1.1120000000000001,"15min":9.5640000000000001},"last":0.376},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.88600000000000001,"15min":6.8639999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.78500000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":8.9779999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.84699999999999998},"min":{"1min":0.50800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":1.218,"15min":8.7309999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.67700000000000005},"min":{"1min":0.318,"5min":0.313,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":11.388999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46899999999999997,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":8.6509999999999998},"last":0.501}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.69399999999999995},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":9.1449999999999996},"last":0.44},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.81899999999999995,"15min":1.119},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52200000000000002,"15min":0.69499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33500000000000002},"max":{"1min":0.70599999999999996,"5min":0.85399999999999998,"15min":8.6050000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":9.4640000000000004},"last":0.56399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":993,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194319,"statfs":{"total":200000000,"available":198982772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.81899999999999995},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.57699999999999996},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.24399999999999999},"max":{"1min":0.73999999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.93999999999999995},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34999999999999998},"max":{"1min":0.70399999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.74199999999999999,"15min":0.76000000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.496,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.73799999999999999,"15min":0.78600000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.56100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.0109999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.88200000000000001},"last":0.70799999999999996}]}]}],"pool_statfs":[]}} 2024-01-11T07:18:17.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:18:17.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.801+0000 7f608eb99700 1 Processor -- start 2024-01-11T07:18:17.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.801+0000 7f608eb99700 1 -- start start 2024-01-11T07:18:17.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608eb99700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806dd00 0x7f60880c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608eb99700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f608815a1c0 0x7f6088069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608eb99700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608815a640 con 0x7f608806e0d0 2024-01-11T07:18:17.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608eb99700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608815b770 con 0x7f608815a1c0 2024-01-11T07:18:17.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608eb99700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608815abf0 con 0x7f608806dd00 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608d136700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f608815a1c0 0x7f6088069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608c935700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806dd00 0x7f60880c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f6087fff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f608806e0d0 0x7f6088159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:46698/0 (socket says 172.21.15.7:46698) 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608c935700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806dd00 0x7f60880c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:33030/0 (socket says 172.21.15.7:33030) 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f608d136700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f608815a1c0 0x7f6088069fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:41518/0 (socket says 172.21.15.7:41518) 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.802+0000 7f6087fff700 1 -- 172.21.15.7:0/3355979117 learned_addr learned my addr 172.21.15.7:0/3355979117 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f60877fe700 1 -- 172.21.15.7:0/3355979117 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655874601 0 0) 0x7f6070001110 con 0x7f608806e0d0 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f60877fe700 1 -- 172.21.15.7:0/3355979117 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6074003570 con 0x7f608806e0d0 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f608c935700 1 -- 172.21.15.7:0/3355979117 >> v2:172.21.15.113:3300/0 conn(0x7f608815a1c0 msgr2=0x7f6088069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f608c935700 1 --2- 172.21.15.7:0/3355979117 >> v2:172.21.15.113:3300/0 conn(0x7f608815a1c0 0x7f6088069fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f608c935700 1 -- 172.21.15.7:0/3355979117 >> v1:172.21.15.113:6789/0 conn(0x7f608806e0d0 legacy=0x7f6088159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f608c935700 1 -- 172.21.15.7:0/3355979117 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608815aed0 con 0x7f608806dd00 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f608d136700 1 --2- 172.21.15.7:0/3355979117 >> v2:172.21.15.113:3300/0 conn(0x7f608815a1c0 0x7f6088069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f608c935700 1 --2- 172.21.15.7:0/3355979117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806dd00 0x7f60880c98d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f60780098e0 tx=0x7f60780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1287ada4c93c383 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f60877fe700 1 -- 172.21.15.7:0/3355979117 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078011630 con 0x7f608806dd00 2024-01-11T07:18:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f60877fe700 1 -- 172.21.15.7:0/3355979117 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6078011790 con 0x7f608806dd00 2024-01-11T07:18:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.803+0000 7f60877fe700 1 -- 172.21.15.7:0/3355979117 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078011b30 con 0x7f608806dd00 2024-01-11T07:18:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.804+0000 7f608eb99700 1 -- 172.21.15.7:0/3355979117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806dd00 msgr2=0x7f60880c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.804+0000 7f608eb99700 1 --2- 172.21.15.7:0/3355979117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806dd00 0x7f60880c98d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f60780098e0 tx=0x7f60780091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.804+0000 7f608eb99700 1 -- 172.21.15.7:0/3355979117 shutdown_connections 2024-01-11T07:18:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.804+0000 7f608eb99700 1 --2- 172.21.15.7:0/3355979117 >> v2:172.21.15.113:3300/0 conn(0x7f608815a1c0 0x7f6088069fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.804+0000 7f608eb99700 1 --2- 172.21.15.7:0/3355979117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806dd00 0x7f60880c98d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.804+0000 7f608eb99700 1 -- 172.21.15.7:0/3355979117 >> 172.21.15.7:0/3355979117 conn(0x7f60880680e0 msgr2=0x7f60880cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.804+0000 7f608eb99700 1 -- 172.21.15.7:0/3355979117 shutdown_connections 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.804+0000 7f608eb99700 1 -- 172.21.15.7:0/3355979117 wait complete. 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.805+0000 7f608eb99700 1 Processor -- start 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.805+0000 7f608eb99700 1 -- start start 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608eb99700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f608806dd00 0x7f60880ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608eb99700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806e0d0 0x7f608816a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608eb99700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f608815a1c0 0x7f608816aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608eb99700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60880b0750 con 0x7f608815a1c0 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608eb99700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60880ab850 con 0x7f608806dd00 2024-01-11T07:18:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608eb99700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60880abb30 con 0x7f608806e0d0 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608d136700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f608815a1c0 0x7f608816aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608d136700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f608815a1c0 0x7f608816aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57204/0 (socket says 172.21.15.7:57204) 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.806+0000 7f608d136700 1 -- 172.21.15.7:0/1535199834 learned_addr learned my addr 172.21.15.7:0/1535199834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608c935700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f608806dd00 0x7f60880ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f6087fff700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806e0d0 0x7f608816a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608d136700 1 -- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806e0d0 msgr2=0x7f608816a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608d136700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806e0d0 0x7f608816a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608d136700 1 -- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f608806dd00 msgr2=0x7f60880ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608d136700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f608806dd00 0x7f60880ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608d136700 1 -- 172.21.15.7:0/1535199834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080009a20 con 0x7f608815a1c0 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608d136700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f608815a1c0 0x7f608816aa80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f6080009f20 tx=0x7f608000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f6084ff9700 1 -- 172.21.15.7:0/1535199834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608000cb70 con 0x7f608815a1c0 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f6084ff9700 1 -- 172.21.15.7:0/1535199834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608000f040 con 0x7f608815a1c0 2024-01-11T07:18:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f6084ff9700 1 -- 172.21.15.7:0/1535199834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080014480 con 0x7f608815a1c0 2024-01-11T07:18:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6078009420 con 0x7f608815a1c0 2024-01-11T07:18:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.807+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60880ac0f0 con 0x7f608815a1c0 2024-01-11T07:18:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.808+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60880b08c0 con 0x7f608815a1c0 2024-01-11T07:18:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.809+0000 7f6084ff9700 1 -- 172.21.15.7:0/1535199834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6080003020 con 0x7f608815a1c0 2024-01-11T07:18:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.809+0000 7f6084ff9700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6068066f30 0x7f60680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.809+0000 7f6084ff9700 1 -- 172.21.15.7:0/1535199834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6080084e10 con 0x7f608815a1c0 2024-01-11T07:18:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.809+0000 7f608c935700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6068066f30 0x7f60680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.810+0000 7f608c935700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6068066f30 0x7f60680693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f60780091e0 tx=0x7f6078017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:17.814+0000 7f6084ff9700 1 -- 172.21.15.7:0/1535199834 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6080052080 con 0x7f608815a1c0 2024-01-11T07:18:18.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.013+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f608804c3c0 con 0x7f6068066f30 2024-01-11T07:18:18.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.022+0000 7f6084ff9700 1 -- 172.21.15.7:0/1535199834 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285651 (secure 0 0 0) 0x7f608804c3c0 con 0x7f6068066f30 2024-01-11T07:18:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6068066f30 msgr2=0x7f60680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6068066f30 0x7f60680693e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f60780091e0 tx=0x7f6078017000 comp rx=0 tx=0).stop 2024-01-11T07:18:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f608815a1c0 msgr2=0x7f608816aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f608815a1c0 0x7f608816aa80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f6080009f20 tx=0x7f608000b040 comp rx=0 tx=0).stop 2024-01-11T07:18:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 shutdown_connections 2024-01-11T07:18:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6068066f30 0x7f60680693e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f608815a1c0 0x7f608816aa80 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f608806dd00 0x7f60880ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 --2- 172.21.15.7:0/1535199834 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f608806e0d0 0x7f608816a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 >> 172.21.15.7:0/1535199834 conn(0x7f60880680e0 msgr2=0x7f6088069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 shutdown_connections 2024-01-11T07:18:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.025+0000 7f608eb99700 1 -- 172.21.15.7:0/1535199834 wait complete. 2024-01-11T07:18:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:18:18.040 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-01-11T07:18:18.003049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9119,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553380,"statfs":{"total":1600000000,"available":1590661129,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001250"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.432054+0000","last_change":"2024-01-11T07:18:13.432054+0000","last_active":"2024-01-11T07:18:13.432054+0000","last_peered":"2024-01-11T07:18:13.432054+0000","last_clean":"2024-01-11T07:18:13.432054+0000","last_became_active":"2024-01-11T07:18:13.431728+0000","last_became_peered":"2024-01-11T07:18:13.431728+0000","last_unstale":"2024-01-11T07:18:13.432054+0000","last_undegraded":"2024-01-11T07:18:13.432054+0000","last_fullsized":"2024-01-11T07:18:13.432054+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:47:25.278556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437487+0000","last_change":"2024-01-11T07:18:13.437487+0000","last_active":"2024-01-11T07:18:13.437487+0000","last_peered":"2024-01-11T07:18:13.437487+0000","last_clean":"2024-01-11T07:18:13.437487+0000","last_became_active":"2024-01-11T07:18:13.437233+0000","last_became_peered":"2024-01-11T07:18:13.437233+0000","last_unstale":"2024-01-11T07:18:13.437487+0000","last_undegraded":"2024-01-11T07:18:13.437487+0000","last_fullsized":"2024-01-11T07:18:13.437487+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:54.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434151+0000","last_change":"2024-01-11T07:18:13.434151+0000","last_active":"2024-01-11T07:18:13.434151+0000","last_peered":"2024-01-11T07:18:13.434151+0000","last_clean":"2024-01-11T07:18:13.434151+0000","last_became_active":"2024-01-11T07:18:13.433600+0000","last_became_peered":"2024-01-11T07:18:13.433600+0000","last_unstale":"2024-01-11T07:18:13.434151+0000","last_undegraded":"2024-01-11T07:18:13.434151+0000","last_fullsized":"2024-01-11T07:18:13.434151+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:42:10.817251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.440416+0000","last_change":"2024-01-11T07:18:13.440416+0000","last_active":"2024-01-11T07:18:13.440416+0000","last_peered":"2024-01-11T07:18:13.440416+0000","last_clean":"2024-01-11T07:18:13.440416+0000","last_became_active":"2024-01-11T07:18:13.440152+0000","last_became_peered":"2024-01-11T07:18:13.440152+0000","last_unstale":"2024-01-11T07:18:13.440416+0000","last_undegraded":"2024-01-11T07:18:13.440416+0000","last_fullsized":"2024-01-11T07:18:13.440416+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:25:16.299601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438035+0000","last_change":"2024-01-11T07:18:13.438035+0000","last_active":"2024-01-11T07:18:13.438035+0000","last_peered":"2024-01-11T07:18:13.438035+0000","last_clean":"2024-01-11T07:18:13.438035+0000","last_became_active":"2024-01-11T07:18:13.437749+0000","last_became_peered":"2024-01-11T07:18:13.437749+0000","last_unstale":"2024-01-11T07:18:13.438035+0000","last_undegraded":"2024-01-11T07:18:13.438035+0000","last_fullsized":"2024-01-11T07:18:13.438035+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:56:51.191605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435587+0000","last_change":"2024-01-11T07:18:13.435587+0000","last_active":"2024-01-11T07:18:13.435587+0000","last_peered":"2024-01-11T07:18:13.435587+0000","last_clean":"2024-01-11T07:18:13.435587+0000","last_became_active":"2024-01-11T07:18:13.435043+0000","last_became_peered":"2024-01-11T07:18:13.435043+0000","last_unstale":"2024-01-11T07:18:13.435587+0000","last_undegraded":"2024-01-11T07:18:13.435587+0000","last_fullsized":"2024-01-11T07:18:13.435587+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:13:47.195902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.428359+0000","last_change":"2024-01-11T07:18:13.428359+0000","last_active":"2024-01-11T07:18:13.428359+0000","last_peered":"2024-01-11T07:18:13.428359+0000","last_clean":"2024-01-11T07:18:13.428359+0000","last_became_active":"2024-01-11T07:18:13.428065+0000","last_became_peered":"2024-01-11T07:18:13.428065+0000","last_unstale":"2024-01-11T07:18:13.428359+0000","last_undegraded":"2024-01-11T07:18:13.428359+0000","last_fullsized":"2024-01-11T07:18:13.428359+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:17:45.043633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439057+0000","last_change":"2024-01-11T07:18:13.439057+0000","last_active":"2024-01-11T07:18:13.439057+0000","last_peered":"2024-01-11T07:18:13.439057+0000","last_clean":"2024-01-11T07:18:13.439057+0000","last_became_active":"2024-01-11T07:18:13.438817+0000","last_became_peered":"2024-01-11T07:18:13.438817+0000","last_unstale":"2024-01-11T07:18:13.439057+0000","last_undegraded":"2024-01-11T07:18:13.439057+0000","last_fullsized":"2024-01-11T07:18:13.439057+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:39.399759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437336+0000","last_change":"2024-01-11T07:18:13.437336+0000","last_active":"2024-01-11T07:18:13.437336+0000","last_peered":"2024-01-11T07:18:13.437336+0000","last_clean":"2024-01-11T07:18:13.437336+0000","last_became_active":"2024-01-11T07:18:13.437069+0000","last_became_peered":"2024-01-11T07:18:13.437069+0000","last_unstale":"2024-01-11T07:18:13.437336+0000","last_undegraded":"2024-01-11T07:18:13.437336+0000","last_fullsized":"2024-01-11T07:18:13.437336+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:18.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439732+0000","last_change":"2024-01-11T07:18:13.439732+0000","last_active":"2024-01-11T07:18:13.439732+0000","last_peered":"2024-01-11T07:18:13.439732+0000","last_clean":"2024-01-11T07:18:13.439732+0000","last_became_active":"2024-01-11T07:18:13.438253+0000","last_became_peered":"2024-01-11T07:18:13.438253+0000","last_unstale":"2024-01-11T07:18:13.439732+0000","last_undegraded":"2024-01-11T07:18:13.439732+0000","last_fullsized":"2024-01-11T07:18:13.439732+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:39:01.983933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438646+0000","last_change":"2024-01-11T07:18:13.438646+0000","last_active":"2024-01-11T07:18:13.438646+0000","last_peered":"2024-01-11T07:18:13.438646+0000","last_clean":"2024-01-11T07:18:13.438646+0000","last_became_active":"2024-01-11T07:18:13.438393+0000","last_became_peered":"2024-01-11T07:18:13.438393+0000","last_unstale":"2024-01-11T07:18:13.438646+0000","last_undegraded":"2024-01-11T07:18:13.438646+0000","last_fullsized":"2024-01-11T07:18:13.438646+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:08:20.052085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.440130+0000","last_change":"2024-01-11T07:18:13.440130+0000","last_active":"2024-01-11T07:18:13.440130+0000","last_peered":"2024-01-11T07:18:13.440130+0000","last_clean":"2024-01-11T07:18:13.440130+0000","last_became_active":"2024-01-11T07:18:13.439889+0000","last_became_peered":"2024-01-11T07:18:13.439889+0000","last_unstale":"2024-01-11T07:18:13.440130+0000","last_undegraded":"2024-01-11T07:18:13.440130+0000","last_fullsized":"2024-01-11T07:18:13.440130+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:04:10.626736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438547+0000","last_change":"2024-01-11T07:18:13.438547+0000","last_active":"2024-01-11T07:18:13.438547+0000","last_peered":"2024-01-11T07:18:13.438547+0000","last_clean":"2024-01-11T07:18:13.438547+0000","last_became_active":"2024-01-11T07:18:13.438303+0000","last_became_peered":"2024-01-11T07:18:13.438303+0000","last_unstale":"2024-01-11T07:18:13.438547+0000","last_undegraded":"2024-01-11T07:18:13.438547+0000","last_fullsized":"2024-01-11T07:18:13.438547+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:39.399759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.433328+0000","last_change":"2024-01-11T07:18:13.433328+0000","last_active":"2024-01-11T07:18:13.433328+0000","last_peered":"2024-01-11T07:18:13.433328+0000","last_clean":"2024-01-11T07:18:13.433328+0000","last_became_active":"2024-01-11T07:18:13.433077+0000","last_became_peered":"2024-01-11T07:18:13.433077+0000","last_unstale":"2024-01-11T07:18:13.433328+0000","last_undegraded":"2024-01-11T07:18:13.433328+0000","last_fullsized":"2024-01-11T07:18:13.433328+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:00:04.012143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438858+0000","last_change":"2024-01-11T07:18:13.438858+0000","last_active":"2024-01-11T07:18:13.438858+0000","last_peered":"2024-01-11T07:18:13.438858+0000","last_clean":"2024-01-11T07:18:13.438858+0000","last_became_active":"2024-01-11T07:18:13.437797+0000","last_became_peered":"2024-01-11T07:18:13.437797+0000","last_unstale":"2024-01-11T07:18:13.438858+0000","last_undegraded":"2024-01-11T07:18:13.438858+0000","last_fullsized":"2024-01-11T07:18:13.438858+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:44:23.954189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.433777+0000","last_change":"2024-01-11T07:18:13.433777+0000","last_active":"2024-01-11T07:18:13.433777+0000","last_peered":"2024-01-11T07:18:13.433777+0000","last_clean":"2024-01-11T07:18:13.433777+0000","last_became_active":"2024-01-11T07:18:13.433460+0000","last_became_peered":"2024-01-11T07:18:13.433460+0000","last_unstale":"2024-01-11T07:18:13.433777+0000","last_undegraded":"2024-01-11T07:18:13.433777+0000","last_fullsized":"2024-01-11T07:18:13.433777+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:59:56.414964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434689+0000","last_change":"2024-01-11T07:18:13.434689+0000","last_active":"2024-01-11T07:18:13.434689+0000","last_peered":"2024-01-11T07:18:13.434689+0000","last_clean":"2024-01-11T07:18:13.434689+0000","last_became_active":"2024-01-11T07:18:13.434451+0000","last_became_peered":"2024-01-11T07:18:13.434451+0000","last_unstale":"2024-01-11T07:18:13.434689+0000","last_undegraded":"2024-01-11T07:18:13.434689+0000","last_fullsized":"2024-01-11T07:18:13.434689+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:19:00.349243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435446+0000","last_change":"2024-01-11T07:18:13.435446+0000","last_active":"2024-01-11T07:18:13.435446+0000","last_peered":"2024-01-11T07:18:13.435446+0000","last_clean":"2024-01-11T07:18:13.435446+0000","last_became_active":"2024-01-11T07:18:13.434995+0000","last_became_peered":"2024-01-11T07:18:13.434995+0000","last_unstale":"2024-01-11T07:18:13.435446+0000","last_undegraded":"2024-01-11T07:18:13.435446+0000","last_fullsized":"2024-01-11T07:18:13.435446+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:05:57.809342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434157+0000","last_change":"2024-01-11T07:18:13.434157+0000","last_active":"2024-01-11T07:18:13.434157+0000","last_peered":"2024-01-11T07:18:13.434157+0000","last_clean":"2024-01-11T07:18:13.434157+0000","last_became_active":"2024-01-11T07:18:13.433994+0000","last_became_peered":"2024-01-11T07:18:13.433994+0000","last_unstale":"2024-01-11T07:18:13.434157+0000","last_undegraded":"2024-01-11T07:18:13.434157+0000","last_fullsized":"2024-01-11T07:18:13.434157+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:15:19.349095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.432612+0000","last_change":"2024-01-11T07:18:13.432612+0000","last_active":"2024-01-11T07:18:13.432612+0000","last_peered":"2024-01-11T07:18:13.432612+0000","last_clean":"2024-01-11T07:18:13.432612+0000","last_became_active":"2024-01-11T07:18:13.432013+0000","last_became_peered":"2024-01-11T07:18:13.432013+0000","last_unstale":"2024-01-11T07:18:13.432612+0000","last_undegraded":"2024-01-11T07:18:13.432612+0000","last_fullsized":"2024-01-11T07:18:13.432612+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:39:01.983933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435187+0000","last_change":"2024-01-11T07:18:13.435187+0000","last_active":"2024-01-11T07:18:13.435187+0000","last_peered":"2024-01-11T07:18:13.435187+0000","last_clean":"2024-01-11T07:18:13.435187+0000","last_became_active":"2024-01-11T07:18:13.434722+0000","last_became_peered":"2024-01-11T07:18:13.434722+0000","last_unstale":"2024-01-11T07:18:13.435187+0000","last_undegraded":"2024-01-11T07:18:13.435187+0000","last_fullsized":"2024-01-11T07:18:13.435187+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:59.464290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.443992+0000","last_change":"2024-01-11T07:18:13.443992+0000","last_active":"2024-01-11T07:18:13.443992+0000","last_peered":"2024-01-11T07:18:13.443992+0000","last_clean":"2024-01-11T07:18:13.443992+0000","last_became_active":"2024-01-11T07:18:13.443723+0000","last_became_peered":"2024-01-11T07:18:13.443723+0000","last_unstale":"2024-01-11T07:18:13.443992+0000","last_undegraded":"2024-01-11T07:18:13.443992+0000","last_fullsized":"2024-01-11T07:18:13.443992+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:51.546656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439724+0000","last_change":"2024-01-11T07:18:13.439724+0000","last_active":"2024-01-11T07:18:13.439724+0000","last_peered":"2024-01-11T07:18:13.439724+0000","last_clean":"2024-01-11T07:18:13.439724+0000","last_became_active":"2024-01-11T07:18:13.439455+0000","last_became_peered":"2024-01-11T07:18:13.439455+0000","last_unstale":"2024-01-11T07:18:13.439724+0000","last_undegraded":"2024-01-11T07:18:13.439724+0000","last_fullsized":"2024-01-11T07:18:13.439724+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:16:24.570897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439455+0000","last_change":"2024-01-11T07:18:13.439455+0000","last_active":"2024-01-11T07:18:13.439455+0000","last_peered":"2024-01-11T07:18:13.439455+0000","last_clean":"2024-01-11T07:18:13.439455+0000","last_became_active":"2024-01-11T07:18:13.439215+0000","last_became_peered":"2024-01-11T07:18:13.439215+0000","last_unstale":"2024-01-11T07:18:13.439455+0000","last_undegraded":"2024-01-11T07:18:13.439455+0000","last_fullsized":"2024-01-11T07:18:13.439455+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:38:51.503114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.432761+0000","last_change":"2024-01-11T07:18:13.432761+0000","last_active":"2024-01-11T07:18:13.432761+0000","last_peered":"2024-01-11T07:18:13.432761+0000","last_clean":"2024-01-11T07:18:13.432761+0000","last_became_active":"2024-01-11T07:18:13.432373+0000","last_became_peered":"2024-01-11T07:18:13.432373+0000","last_unstale":"2024-01-11T07:18:13.432761+0000","last_undegraded":"2024-01-11T07:18:13.432761+0000","last_fullsized":"2024-01-11T07:18:13.432761+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:04:10.626736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.443002+0000","last_change":"2024-01-11T07:18:13.443002+0000","last_active":"2024-01-11T07:18:13.443002+0000","last_peered":"2024-01-11T07:18:13.443002+0000","last_clean":"2024-01-11T07:18:13.443002+0000","last_became_active":"2024-01-11T07:18:13.437719+0000","last_became_peered":"2024-01-11T07:18:13.437719+0000","last_unstale":"2024-01-11T07:18:13.443002+0000","last_undegraded":"2024-01-11T07:18:13.443002+0000","last_fullsized":"2024-01-11T07:18:13.443002+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:00:08.705803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435741+0000","last_change":"2024-01-11T07:18:13.435741+0000","last_active":"2024-01-11T07:18:13.435741+0000","last_peered":"2024-01-11T07:18:13.435741+0000","last_clean":"2024-01-11T07:18:13.435741+0000","last_became_active":"2024-01-11T07:18:13.435450+0000","last_became_peered":"2024-01-11T07:18:13.435450+0000","last_unstale":"2024-01-11T07:18:13.435741+0000","last_undegraded":"2024-01-11T07:18:13.435741+0000","last_fullsized":"2024-01-11T07:18:13.435741+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:18.385489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438557+0000","last_change":"2024-01-11T07:18:13.438557+0000","last_active":"2024-01-11T07:18:13.438557+0000","last_peered":"2024-01-11T07:18:13.438557+0000","last_clean":"2024-01-11T07:18:13.438557+0000","last_became_active":"2024-01-11T07:18:13.438310+0000","last_became_peered":"2024-01-11T07:18:13.438310+0000","last_unstale":"2024-01-11T07:18:13.438557+0000","last_undegraded":"2024-01-11T07:18:13.438557+0000","last_fullsized":"2024-01-11T07:18:13.438557+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T19:15:23.934950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435796+0000","last_change":"2024-01-11T07:18:13.435796+0000","last_active":"2024-01-11T07:18:13.435796+0000","last_peered":"2024-01-11T07:18:13.435796+0000","last_clean":"2024-01-11T07:18:13.435796+0000","last_became_active":"2024-01-11T07:18:13.435522+0000","last_became_peered":"2024-01-11T07:18:13.435522+0000","last_unstale":"2024-01-11T07:18:13.435796+0000","last_undegraded":"2024-01-11T07:18:13.435796+0000","last_fullsized":"2024-01-11T07:18:13.435796+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:51.546656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.430873+0000","last_change":"2024-01-11T07:18:13.430873+0000","last_active":"2024-01-11T07:18:13.430873+0000","last_peered":"2024-01-11T07:18:13.430873+0000","last_clean":"2024-01-11T07:18:13.430873+0000","last_became_active":"2024-01-11T07:18:13.430304+0000","last_became_peered":"2024-01-11T07:18:13.430304+0000","last_unstale":"2024-01-11T07:18:13.430873+0000","last_undegraded":"2024-01-11T07:18:13.430873+0000","last_fullsized":"2024-01-11T07:18:13.430873+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:27:35.430314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435559+0000","last_change":"2024-01-11T07:18:13.435559+0000","last_active":"2024-01-11T07:18:13.435559+0000","last_peered":"2024-01-11T07:18:13.435559+0000","last_clean":"2024-01-11T07:18:13.435559+0000","last_became_active":"2024-01-11T07:18:13.434507+0000","last_became_peered":"2024-01-11T07:18:13.434507+0000","last_unstale":"2024-01-11T07:18:13.435559+0000","last_undegraded":"2024-01-11T07:18:13.435559+0000","last_fullsized":"2024-01-11T07:18:13.435559+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:31:18.390754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437086+0000","last_change":"2024-01-11T07:18:13.437086+0000","last_active":"2024-01-11T07:18:13.437086+0000","last_peered":"2024-01-11T07:18:13.437086+0000","last_clean":"2024-01-11T07:18:13.437086+0000","last_became_active":"2024-01-11T07:18:13.436593+0000","last_became_peered":"2024-01-11T07:18:13.436593+0000","last_unstale":"2024-01-11T07:18:13.437086+0000","last_undegraded":"2024-01-11T07:18:13.437086+0000","last_fullsized":"2024-01-11T07:18:13.437086+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:26:57.208882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446691+0000","last_change":"2024-01-11T07:18:13.446691+0000","last_active":"2024-01-11T07:18:13.446691+0000","last_peered":"2024-01-11T07:18:13.446691+0000","last_clean":"2024-01-11T07:18:13.446691+0000","last_became_active":"2024-01-11T07:18:13.446257+0000","last_became_peered":"2024-01-11T07:18:13.446257+0000","last_unstale":"2024-01-11T07:18:13.446691+0000","last_undegraded":"2024-01-11T07:18:13.446691+0000","last_fullsized":"2024-01-11T07:18:13.446691+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:22:22.989834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"183'4","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.445178+0000","last_change":"2024-01-11T07:18:12.434944+0000","last_active":"2024-01-11T07:18:13.445178+0000","last_peered":"2024-01-11T07:18:13.445178+0000","last_clean":"2024-01-11T07:18:13.445178+0000","last_became_active":"2024-01-11T07:18:12.434704+0000","last_became_peered":"2024-01-11T07:18:12.434704+0000","last_unstale":"2024-01-11T07:18:13.445178+0000","last_undegraded":"2024-01-11T07:18:13.445178+0000","last_fullsized":"2024-01-11T07:18:13.445178+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:36:04.821856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435574+0000","last_change":"2024-01-11T07:18:13.435574+0000","last_active":"2024-01-11T07:18:13.435574+0000","last_peered":"2024-01-11T07:18:13.435574+0000","last_clean":"2024-01-11T07:18:13.435574+0000","last_became_active":"2024-01-11T07:18:13.434286+0000","last_became_peered":"2024-01-11T07:18:13.434286+0000","last_unstale":"2024-01-11T07:18:13.435574+0000","last_undegraded":"2024-01-11T07:18:13.435574+0000","last_fullsized":"2024-01-11T07:18:13.435574+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:38:33.528944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.427765+0000","last_change":"2024-01-11T07:18:12.434673+0000","last_active":"2024-01-11T07:18:13.427765+0000","last_peered":"2024-01-11T07:18:13.427765+0000","last_clean":"2024-01-11T07:18:13.427765+0000","last_became_active":"2024-01-11T07:18:12.434409+0000","last_became_peered":"2024-01-11T07:18:12.434409+0000","last_unstale":"2024-01-11T07:18:13.427765+0000","last_undegraded":"2024-01-11T07:18:13.427765+0000","last_fullsized":"2024-01-11T07:18:13.427765+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:39:00.978523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435758+0000","last_change":"2024-01-11T07:18:13.435758+0000","last_active":"2024-01-11T07:18:13.435758+0000","last_peered":"2024-01-11T07:18:13.435758+0000","last_clean":"2024-01-11T07:18:13.435758+0000","last_became_active":"2024-01-11T07:18:13.435414+0000","last_became_peered":"2024-01-11T07:18:13.435414+0000","last_unstale":"2024-01-11T07:18:13.435758+0000","last_undegraded":"2024-01-11T07:18:13.435758+0000","last_fullsized":"2024-01-11T07:18:13.435758+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:42:38.595631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.444209+0000","last_change":"2024-01-11T07:18:12.434576+0000","last_active":"2024-01-11T07:18:13.444209+0000","last_peered":"2024-01-11T07:18:13.444209+0000","last_clean":"2024-01-11T07:18:13.444209+0000","last_became_active":"2024-01-11T07:18:12.434186+0000","last_became_peered":"2024-01-11T07:18:12.434186+0000","last_unstale":"2024-01-11T07:18:13.444209+0000","last_undegraded":"2024-01-11T07:18:13.444209+0000","last_fullsized":"2024-01-11T07:18:13.444209+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:03:37.188892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434536+0000","last_change":"2024-01-11T07:18:13.434536+0000","last_active":"2024-01-11T07:18:13.434536+0000","last_peered":"2024-01-11T07:18:13.434536+0000","last_clean":"2024-01-11T07:18:13.434536+0000","last_became_active":"2024-01-11T07:18:13.434310+0000","last_became_peered":"2024-01-11T07:18:13.434310+0000","last_unstale":"2024-01-11T07:18:13.434536+0000","last_undegraded":"2024-01-11T07:18:13.434536+0000","last_fullsized":"2024-01-11T07:18:13.434536+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:54.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.444201+0000","last_change":"2024-01-11T07:18:12.434054+0000","last_active":"2024-01-11T07:18:13.444201+0000","last_peered":"2024-01-11T07:18:13.444201+0000","last_clean":"2024-01-11T07:18:13.444201+0000","last_became_active":"2024-01-11T07:18:12.433739+0000","last_became_peered":"2024-01-11T07:18:12.433739+0000","last_unstale":"2024-01-11T07:18:13.444201+0000","last_undegraded":"2024-01-11T07:18:13.444201+0000","last_fullsized":"2024-01-11T07:18:13.444201+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:27:34.424904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.433155+0000","last_change":"2024-01-11T07:18:13.433155+0000","last_active":"2024-01-11T07:18:13.433155+0000","last_peered":"2024-01-11T07:18:13.433155+0000","last_clean":"2024-01-11T07:18:13.433155+0000","last_became_active":"2024-01-11T07:18:13.432709+0000","last_became_peered":"2024-01-11T07:18:13.432709+0000","last_unstale":"2024-01-11T07:18:13.433155+0000","last_undegraded":"2024-01-11T07:18:13.433155+0000","last_fullsized":"2024-01-11T07:18:13.433155+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:53:35.336716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.431425+0000","last_change":"2024-01-11T07:18:12.432647+0000","last_active":"2024-01-11T07:18:13.431425+0000","last_peered":"2024-01-11T07:18:13.431425+0000","last_clean":"2024-01-11T07:18:13.431425+0000","last_became_active":"2024-01-11T07:18:12.429452+0000","last_became_peered":"2024-01-11T07:18:12.429452+0000","last_unstale":"2024-01-11T07:18:13.431425+0000","last_undegraded":"2024-01-11T07:18:13.431425+0000","last_fullsized":"2024-01-11T07:18:13.431425+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:35:58.458880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.447261+0000","last_change":"2024-01-11T07:18:13.447261+0000","last_active":"2024-01-11T07:18:13.447261+0000","last_peered":"2024-01-11T07:18:13.447261+0000","last_clean":"2024-01-11T07:18:13.447261+0000","last_became_active":"2024-01-11T07:18:13.446825+0000","last_became_peered":"2024-01-11T07:18:13.446825+0000","last_unstale":"2024-01-11T07:18:13.447261+0000","last_undegraded":"2024-01-11T07:18:13.447261+0000","last_fullsized":"2024-01-11T07:18:13.447261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:45:00.490536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.430195+0000","last_change":"2024-01-11T07:18:12.436217+0000","last_active":"2024-01-11T07:18:13.430195+0000","last_peered":"2024-01-11T07:18:13.430195+0000","last_clean":"2024-01-11T07:18:13.430195+0000","last_became_active":"2024-01-11T07:18:12.435827+0000","last_became_peered":"2024-01-11T07:18:12.435827+0000","last_unstale":"2024-01-11T07:18:13.430195+0000","last_undegraded":"2024-01-11T07:18:13.430195+0000","last_fullsized":"2024-01-11T07:18:13.430195+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:09:47.377371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434648+0000","last_change":"2024-01-11T07:18:13.434648+0000","last_active":"2024-01-11T07:18:13.434648+0000","last_peered":"2024-01-11T07:18:13.434648+0000","last_clean":"2024-01-11T07:18:13.434648+0000","last_became_active":"2024-01-11T07:18:13.434108+0000","last_became_peered":"2024-01-11T07:18:13.434108+0000","last_unstale":"2024-01-11T07:18:13.434648+0000","last_undegraded":"2024-01-11T07:18:13.434648+0000","last_fullsized":"2024-01-11T07:18:13.434648+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:00:45.671429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.434290+0000","last_change":"2024-01-11T07:18:12.434191+0000","last_active":"2024-01-11T07:18:12.434290+0000","last_peered":"2024-01-11T07:18:12.434290+0000","last_clean":"2024-01-11T07:18:12.434290+0000","last_became_active":"2024-01-11T07:18:12.433522+0000","last_became_peered":"2024-01-11T07:18:12.433522+0000","last_unstale":"2024-01-11T07:18:12.434290+0000","last_undegraded":"2024-01-11T07:18:12.434290+0000","last_fullsized":"2024-01-11T07:18:12.434290+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:05:48.780643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446975+0000","last_change":"2024-01-11T07:18:13.446975+0000","last_active":"2024-01-11T07:18:13.446975+0000","last_peered":"2024-01-11T07:18:13.446975+0000","last_clean":"2024-01-11T07:18:13.446975+0000","last_became_active":"2024-01-11T07:18:13.446777+0000","last_became_peered":"2024-01-11T07:18:13.446777+0000","last_unstale":"2024-01-11T07:18:13.446975+0000","last_undegraded":"2024-01-11T07:18:13.446975+0000","last_fullsized":"2024-01-11T07:18:13.446975+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:10:57.278546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.430244+0000","last_change":"2024-01-11T07:18:12.430160+0000","last_active":"2024-01-11T07:18:12.430244+0000","last_peered":"2024-01-11T07:18:12.430244+0000","last_clean":"2024-01-11T07:18:12.430244+0000","last_became_active":"2024-01-11T07:18:12.429880+0000","last_became_peered":"2024-01-11T07:18:12.429880+0000","last_unstale":"2024-01-11T07:18:12.430244+0000","last_undegraded":"2024-01-11T07:18:12.430244+0000","last_fullsized":"2024-01-11T07:18:12.430244+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:45:56.778317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438048+0000","last_change":"2024-01-11T07:18:13.438048+0000","last_active":"2024-01-11T07:18:13.438048+0000","last_peered":"2024-01-11T07:18:13.438048+0000","last_clean":"2024-01-11T07:18:13.438048+0000","last_became_active":"2024-01-11T07:18:13.437771+0000","last_became_peered":"2024-01-11T07:18:13.437771+0000","last_unstale":"2024-01-11T07:18:13.438048+0000","last_undegraded":"2024-01-11T07:18:13.438048+0000","last_fullsized":"2024-01-11T07:18:13.438048+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:11:50.494109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.429748+0000","last_change":"2024-01-11T07:18:12.429686+0000","last_active":"2024-01-11T07:18:12.429748+0000","last_peered":"2024-01-11T07:18:12.429748+0000","last_clean":"2024-01-11T07:18:12.429748+0000","last_became_active":"2024-01-11T07:18:12.429317+0000","last_became_peered":"2024-01-11T07:18:12.429317+0000","last_unstale":"2024-01-11T07:18:12.429748+0000","last_undegraded":"2024-01-11T07:18:12.429748+0000","last_fullsized":"2024-01-11T07:18:12.429748+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:50:45.944040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.440563+0000","last_change":"2024-01-11T07:18:13.440563+0000","last_active":"2024-01-11T07:18:13.440563+0000","last_peered":"2024-01-11T07:18:13.440563+0000","last_clean":"2024-01-11T07:18:13.440563+0000","last_became_active":"2024-01-11T07:18:13.440073+0000","last_became_peered":"2024-01-11T07:18:13.440073+0000","last_unstale":"2024-01-11T07:18:13.440563+0000","last_undegraded":"2024-01-11T07:18:13.440563+0000","last_fullsized":"2024-01-11T07:18:13.440563+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:47:25.278556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"183'3","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.445394+0000","last_change":"2024-01-11T07:18:12.433268+0000","last_active":"2024-01-11T07:18:13.445394+0000","last_peered":"2024-01-11T07:18:13.445394+0000","last_clean":"2024-01-11T07:18:13.445394+0000","last_became_active":"2024-01-11T07:18:12.433027+0000","last_became_peered":"2024-01-11T07:18:12.433027+0000","last_unstale":"2024-01-11T07:18:13.445394+0000","last_undegraded":"2024-01-11T07:18:13.445394+0000","last_fullsized":"2024-01-11T07:18:13.445394+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:03:20.180539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435423+0000","last_change":"2024-01-11T07:18:13.435423+0000","last_active":"2024-01-11T07:18:13.435423+0000","last_peered":"2024-01-11T07:18:13.435423+0000","last_clean":"2024-01-11T07:18:13.435423+0000","last_became_active":"2024-01-11T07:18:13.433970+0000","last_became_peered":"2024-01-11T07:18:13.433970+0000","last_unstale":"2024-01-11T07:18:13.435423+0000","last_undegraded":"2024-01-11T07:18:13.435423+0000","last_fullsized":"2024-01-11T07:18:13.435423+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:26:16.527337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"183'6","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446795+0000","last_change":"2024-01-11T07:18:12.431882+0000","last_active":"2024-01-11T07:18:13.446795+0000","last_peered":"2024-01-11T07:18:13.446795+0000","last_clean":"2024-01-11T07:18:13.446795+0000","last_became_active":"2024-01-11T07:18:12.431583+0000","last_became_peered":"2024-01-11T07:18:12.431583+0000","last_unstale":"2024-01-11T07:18:13.446795+0000","last_undegraded":"2024-01-11T07:18:13.446795+0000","last_fullsized":"2024-01-11T07:18:13.446795+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:06:38.394350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.441281+0000","last_change":"2024-01-11T07:18:13.441281+0000","last_active":"2024-01-11T07:18:13.441281+0000","last_peered":"2024-01-11T07:18:13.441281+0000","last_clean":"2024-01-11T07:18:13.441281+0000","last_became_active":"2024-01-11T07:18:13.439782+0000","last_became_peered":"2024-01-11T07:18:13.439782+0000","last_unstale":"2024-01-11T07:18:13.441281+0000","last_undegraded":"2024-01-11T07:18:13.441281+0000","last_fullsized":"2024-01-11T07:18:13.441281+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:42:12.592049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446153+0000","last_change":"2024-01-11T07:18:12.438486+0000","last_active":"2024-01-11T07:18:13.446153+0000","last_peered":"2024-01-11T07:18:13.446153+0000","last_clean":"2024-01-11T07:18:13.446153+0000","last_became_active":"2024-01-11T07:18:12.438244+0000","last_became_peered":"2024-01-11T07:18:12.438244+0000","last_unstale":"2024-01-11T07:18:13.446153+0000","last_undegraded":"2024-01-11T07:18:13.446153+0000","last_fullsized":"2024-01-11T07:18:13.446153+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:54:17.380079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437065+0000","last_change":"2024-01-11T07:18:13.437065+0000","last_active":"2024-01-11T07:18:13.437065+0000","last_peered":"2024-01-11T07:18:13.437065+0000","last_clean":"2024-01-11T07:18:13.437065+0000","last_became_active":"2024-01-11T07:18:13.436829+0000","last_became_peered":"2024-01-11T07:18:13.436829+0000","last_unstale":"2024-01-11T07:18:13.437065+0000","last_undegraded":"2024-01-11T07:18:13.437065+0000","last_fullsized":"2024-01-11T07:18:13.437065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:33.525439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.435958+0000","last_change":"2024-01-11T07:18:12.435851+0000","last_active":"2024-01-11T07:18:12.435958+0000","last_peered":"2024-01-11T07:18:12.435958+0000","last_clean":"2024-01-11T07:18:12.435958+0000","last_became_active":"2024-01-11T07:18:12.435484+0000","last_became_peered":"2024-01-11T07:18:12.435484+0000","last_unstale":"2024-01-11T07:18:12.435958+0000","last_undegraded":"2024-01-11T07:18:12.435958+0000","last_fullsized":"2024-01-11T07:18:12.435958+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:05:49.052324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.433803+0000","last_change":"2024-01-11T07:18:13.433803+0000","last_active":"2024-01-11T07:18:13.433803+0000","last_peered":"2024-01-11T07:18:13.433803+0000","last_clean":"2024-01-11T07:18:13.433803+0000","last_became_active":"2024-01-11T07:18:13.433527+0000","last_became_peered":"2024-01-11T07:18:13.433527+0000","last_unstale":"2024-01-11T07:18:13.433803+0000","last_undegraded":"2024-01-11T07:18:13.433803+0000","last_fullsized":"2024-01-11T07:18:13.433803+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:23:32.552145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.437869+0000","last_change":"2024-01-11T07:18:12.437767+0000","last_active":"2024-01-11T07:18:12.437869+0000","last_peered":"2024-01-11T07:18:12.437869+0000","last_clean":"2024-01-11T07:18:12.437869+0000","last_became_active":"2024-01-11T07:18:12.437024+0000","last_became_peered":"2024-01-11T07:18:12.437024+0000","last_unstale":"2024-01-11T07:18:12.437869+0000","last_undegraded":"2024-01-11T07:18:12.437869+0000","last_fullsized":"2024-01-11T07:18:12.437869+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:42:57.744348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.443960+0000","last_change":"2024-01-11T07:18:13.443960+0000","last_active":"2024-01-11T07:18:13.443960+0000","last_peered":"2024-01-11T07:18:13.443960+0000","last_clean":"2024-01-11T07:18:13.443960+0000","last_became_active":"2024-01-11T07:18:13.443685+0000","last_became_peered":"2024-01-11T07:18:13.443685+0000","last_unstale":"2024-01-11T07:18:13.443960+0000","last_undegraded":"2024-01-11T07:18:13.443960+0000","last_fullsized":"2024-01-11T07:18:13.443960+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:40:59.297452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"183'8","reported_seq":17,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446472+0000","last_change":"2024-01-11T07:18:12.433398+0000","last_active":"2024-01-11T07:18:13.446472+0000","last_peered":"2024-01-11T07:18:13.446472+0000","last_clean":"2024-01-11T07:18:13.446472+0000","last_became_active":"2024-01-11T07:18:12.433080+0000","last_became_peered":"2024-01-11T07:18:12.433080+0000","last_unstale":"2024-01-11T07:18:13.446472+0000","last_undegraded":"2024-01-11T07:18:13.446472+0000","last_fullsized":"2024-01-11T07:18:13.446472+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:38:50.497704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434792+0000","last_change":"2024-01-11T07:18:13.434792+0000","last_active":"2024-01-11T07:18:13.434792+0000","last_peered":"2024-01-11T07:18:13.434792+0000","last_clean":"2024-01-11T07:18:13.434792+0000","last_became_active":"2024-01-11T07:18:13.434420+0000","last_became_peered":"2024-01-11T07:18:13.434420+0000","last_unstale":"2024-01-11T07:18:13.434792+0000","last_undegraded":"2024-01-11T07:18:13.434792+0000","last_fullsized":"2024-01-11T07:18:13.434792+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:18:00.125897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.430847+0000","last_change":"2024-01-11T07:18:12.432911+0000","last_active":"2024-01-11T07:18:13.430847+0000","last_peered":"2024-01-11T07:18:13.430847+0000","last_clean":"2024-01-11T07:18:13.430847+0000","last_became_active":"2024-01-11T07:18:12.432730+0000","last_became_peered":"2024-01-11T07:18:12.432730+0000","last_unstale":"2024-01-11T07:18:13.430847+0000","last_undegraded":"2024-01-11T07:18:13.430847+0000","last_fullsized":"2024-01-11T07:18:13.430847+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:18:59.343833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.436900+0000","last_change":"2024-01-11T07:18:13.436900+0000","last_active":"2024-01-11T07:18:13.436900+0000","last_peered":"2024-01-11T07:18:13.436900+0000","last_clean":"2024-01-11T07:18:13.436900+0000","last_became_active":"2024-01-11T07:18:13.435690+0000","last_became_peered":"2024-01-11T07:18:13.435690+0000","last_unstale":"2024-01-11T07:18:13.436900+0000","last_undegraded":"2024-01-11T07:18:13.436900+0000","last_fullsized":"2024-01-11T07:18:13.436900+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:16:28.860520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.444183+0000","last_change":"2024-01-11T07:18:12.437228+0000","last_active":"2024-01-11T07:18:13.444183+0000","last_peered":"2024-01-11T07:18:13.444183+0000","last_clean":"2024-01-11T07:18:13.444183+0000","last_became_active":"2024-01-11T07:18:12.436991+0000","last_became_peered":"2024-01-11T07:18:12.436991+0000","last_unstale":"2024-01-11T07:18:13.444183+0000","last_undegraded":"2024-01-11T07:18:13.444183+0000","last_fullsized":"2024-01-11T07:18:13.444183+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T09:13:46.190492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.447100+0000","last_change":"2024-01-11T07:18:13.447100+0000","last_active":"2024-01-11T07:18:13.447100+0000","last_peered":"2024-01-11T07:18:13.447100+0000","last_clean":"2024-01-11T07:18:13.447100+0000","last_became_active":"2024-01-11T07:18:13.446480+0000","last_became_peered":"2024-01-11T07:18:13.446480+0000","last_unstale":"2024-01-11T07:18:13.447100+0000","last_undegraded":"2024-01-11T07:18:13.447100+0000","last_fullsized":"2024-01-11T07:18:13.447100+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T07:18:53.965555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"183'1","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.428874+0000","last_change":"2024-01-11T07:18:12.432131+0000","last_active":"2024-01-11T07:18:13.428874+0000","last_peered":"2024-01-11T07:18:13.428874+0000","last_clean":"2024-01-11T07:18:13.428874+0000","last_became_active":"2024-01-11T07:18:12.431903+0000","last_became_peered":"2024-01-11T07:18:12.431903+0000","last_unstale":"2024-01-11T07:18:13.428874+0000","last_undegraded":"2024-01-11T07:18:13.428874+0000","last_fullsized":"2024-01-11T07:18:13.428874+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:05:19.542803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.436011+0000","last_change":"2024-01-11T07:18:13.436011+0000","last_active":"2024-01-11T07:18:13.436011+0000","last_peered":"2024-01-11T07:18:13.436011+0000","last_clean":"2024-01-11T07:18:13.436011+0000","last_became_active":"2024-01-11T07:18:13.435608+0000","last_became_peered":"2024-01-11T07:18:13.435608+0000","last_unstale":"2024-01-11T07:18:13.436011+0000","last_undegraded":"2024-01-11T07:18:13.436011+0000","last_fullsized":"2024-01-11T07:18:13.436011+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:32:17.344291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.436379+0000","last_change":"2024-01-11T07:18:12.436309+0000","last_active":"2024-01-11T07:18:12.436379+0000","last_peered":"2024-01-11T07:18:12.436379+0000","last_clean":"2024-01-11T07:18:12.436379+0000","last_became_active":"2024-01-11T07:18:12.436046+0000","last_became_peered":"2024-01-11T07:18:12.436046+0000","last_unstale":"2024-01-11T07:18:12.436379+0000","last_undegraded":"2024-01-11T07:18:12.436379+0000","last_fullsized":"2024-01-11T07:18:12.436379+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T16:50:39.044311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.438191+0000","last_change":"2024-01-11T07:18:13.438191+0000","last_active":"2024-01-11T07:18:13.438191+0000","last_peered":"2024-01-11T07:18:13.438191+0000","last_clean":"2024-01-11T07:18:13.438191+0000","last_became_active":"2024-01-11T07:18:13.437914+0000","last_became_peered":"2024-01-11T07:18:13.437914+0000","last_unstale":"2024-01-11T07:18:13.438191+0000","last_undegraded":"2024-01-11T07:18:13.438191+0000","last_fullsized":"2024-01-11T07:18:13.438191+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:38:51.503114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.445458+0000","last_change":"2024-01-11T07:18:12.436073+0000","last_active":"2024-01-11T07:18:13.445458+0000","last_peered":"2024-01-11T07:18:13.445458+0000","last_clean":"2024-01-11T07:18:13.445458+0000","last_became_active":"2024-01-11T07:18:12.435530+0000","last_became_peered":"2024-01-11T07:18:12.435530+0000","last_unstale":"2024-01-11T07:18:13.445458+0000","last_undegraded":"2024-01-11T07:18:13.445458+0000","last_fullsized":"2024-01-11T07:18:13.445458+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:50.541247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.432992+0000","last_change":"2024-01-11T07:18:13.432992+0000","last_active":"2024-01-11T07:18:13.432992+0000","last_peered":"2024-01-11T07:18:13.432992+0000","last_clean":"2024-01-11T07:18:13.432992+0000","last_became_active":"2024-01-11T07:18:13.432356+0000","last_became_peered":"2024-01-11T07:18:13.432356+0000","last_unstale":"2024-01-11T07:18:13.432992+0000","last_undegraded":"2024-01-11T07:18:13.432992+0000","last_fullsized":"2024-01-11T07:18:13.432992+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:26:44.488764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.430076+0000","last_change":"2024-01-11T07:18:12.430024+0000","last_active":"2024-01-11T07:18:12.430076+0000","last_peered":"2024-01-11T07:18:12.430076+0000","last_clean":"2024-01-11T07:18:12.430076+0000","last_became_active":"2024-01-11T07:18:12.429807+0000","last_became_peered":"2024-01-11T07:18:12.429807+0000","last_unstale":"2024-01-11T07:18:12.430076+0000","last_undegraded":"2024-01-11T07:18:12.430076+0000","last_fullsized":"2024-01-11T07:18:12.430076+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T13:18:14.435862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439579+0000","last_change":"2024-01-11T07:18:13.439579+0000","last_active":"2024-01-11T07:18:13.439579+0000","last_peered":"2024-01-11T07:18:13.439579+0000","last_clean":"2024-01-11T07:18:13.439579+0000","last_became_active":"2024-01-11T07:18:13.439335+0000","last_became_peered":"2024-01-11T07:18:13.439335+0000","last_unstale":"2024-01-11T07:18:13.439579+0000","last_undegraded":"2024-01-11T07:18:13.439579+0000","last_fullsized":"2024-01-11T07:18:13.439579+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:05:57.809342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.434524+0000","last_change":"2024-01-11T07:18:12.434422+0000","last_active":"2024-01-11T07:18:12.434524+0000","last_peered":"2024-01-11T07:18:12.434524+0000","last_clean":"2024-01-11T07:18:12.434524+0000","last_became_active":"2024-01-11T07:18:12.433849+0000","last_became_peered":"2024-01-11T07:18:12.433849+0000","last_unstale":"2024-01-11T07:18:12.434524+0000","last_undegraded":"2024-01-11T07:18:12.434524+0000","last_fullsized":"2024-01-11T07:18:12.434524+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:28:03.367767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.443306+0000","last_change":"2024-01-11T07:18:13.443306+0000","last_active":"2024-01-11T07:18:13.443306+0000","last_peered":"2024-01-11T07:18:13.443306+0000","last_clean":"2024-01-11T07:18:13.443306+0000","last_became_active":"2024-01-11T07:18:13.443066+0000","last_became_peered":"2024-01-11T07:18:13.443066+0000","last_unstale":"2024-01-11T07:18:13.443306+0000","last_undegraded":"2024-01-11T07:18:13.443306+0000","last_fullsized":"2024-01-11T07:18:13.443306+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:33:42.414027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.432669+0000","last_change":"2024-01-11T07:18:12.432596+0000","last_active":"2024-01-11T07:18:12.432669+0000","last_peered":"2024-01-11T07:18:12.432669+0000","last_clean":"2024-01-11T07:18:12.432669+0000","last_became_active":"2024-01-11T07:18:12.432369+0000","last_became_peered":"2024-01-11T07:18:12.432369+0000","last_unstale":"2024-01-11T07:18:12.432669+0000","last_undegraded":"2024-01-11T07:18:12.432669+0000","last_fullsized":"2024-01-11T07:18:12.432669+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:45:56.778317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.444102+0000","last_change":"2024-01-11T07:18:13.444102+0000","last_active":"2024-01-11T07:18:13.444102+0000","last_peered":"2024-01-11T07:18:13.444102+0000","last_clean":"2024-01-11T07:18:13.444102+0000","last_became_active":"2024-01-11T07:18:13.443624+0000","last_became_peered":"2024-01-11T07:18:13.443624+0000","last_unstale":"2024-01-11T07:18:13.444102+0000","last_undegraded":"2024-01-11T07:18:13.444102+0000","last_fullsized":"2024-01-11T07:18:13.444102+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T15:00:08.705803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"183'2","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.445077+0000","last_change":"2024-01-11T07:18:12.436901+0000","last_active":"2024-01-11T07:18:13.445077+0000","last_peered":"2024-01-11T07:18:13.445077+0000","last_clean":"2024-01-11T07:18:13.445077+0000","last_became_active":"2024-01-11T07:18:12.436392+0000","last_became_peered":"2024-01-11T07:18:12.436392+0000","last_unstale":"2024-01-11T07:18:13.445077+0000","last_undegraded":"2024-01-11T07:18:13.445077+0000","last_fullsized":"2024-01-11T07:18:13.445077+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T12:39:59.914390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.442311+0000","last_change":"2024-01-11T07:18:13.442311+0000","last_active":"2024-01-11T07:18:13.442311+0000","last_peered":"2024-01-11T07:18:13.442311+0000","last_clean":"2024-01-11T07:18:13.442311+0000","last_became_active":"2024-01-11T07:18:13.441964+0000","last_became_peered":"2024-01-11T07:18:13.441964+0000","last_unstale":"2024-01-11T07:18:13.442311+0000","last_undegraded":"2024-01-11T07:18:13.442311+0000","last_fullsized":"2024-01-11T07:18:13.442311+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:32:58.893111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.434303+0000","last_change":"2024-01-11T07:18:12.434238+0000","last_active":"2024-01-11T07:18:12.434303+0000","last_peered":"2024-01-11T07:18:12.434303+0000","last_clean":"2024-01-11T07:18:12.434303+0000","last_became_active":"2024-01-11T07:18:12.434042+0000","last_became_peered":"2024-01-11T07:18:12.434042+0000","last_unstale":"2024-01-11T07:18:12.434303+0000","last_undegraded":"2024-01-11T07:18:12.434303+0000","last_fullsized":"2024-01-11T07:18:12.434303+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:42:37.590222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437355+0000","last_change":"2024-01-11T07:18:13.437355+0000","last_active":"2024-01-11T07:18:13.437355+0000","last_peered":"2024-01-11T07:18:13.437355+0000","last_clean":"2024-01-11T07:18:13.437355+0000","last_became_active":"2024-01-11T07:18:13.437093+0000","last_became_peered":"2024-01-11T07:18:13.437093+0000","last_unstale":"2024-01-11T07:18:13.437355+0000","last_undegraded":"2024-01-11T07:18:13.437355+0000","last_fullsized":"2024-01-11T07:18:13.437355+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:03:31.690384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.437614+0000","last_change":"2024-01-11T07:18:12.437526+0000","last_active":"2024-01-11T07:18:12.437614+0000","last_peered":"2024-01-11T07:18:12.437614+0000","last_clean":"2024-01-11T07:18:12.437614+0000","last_became_active":"2024-01-11T07:18:12.437266+0000","last_became_peered":"2024-01-11T07:18:12.437266+0000","last_unstale":"2024-01-11T07:18:12.437614+0000","last_undegraded":"2024-01-11T07:18:12.437614+0000","last_fullsized":"2024-01-11T07:18:12.437614+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T10:10:56.273136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.437055+0000","last_change":"2024-01-11T07:18:13.437055+0000","last_active":"2024-01-11T07:18:13.437055+0000","last_peered":"2024-01-11T07:18:13.437055+0000","last_clean":"2024-01-11T07:18:13.437055+0000","last_became_active":"2024-01-11T07:18:13.436546+0000","last_became_peered":"2024-01-11T07:18:13.436546+0000","last_unstale":"2024-01-11T07:18:13.437055+0000","last_undegraded":"2024-01-11T07:18:13.437055+0000","last_fullsized":"2024-01-11T07:18:13.437055+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:05:20.548212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.439388+0000","last_change":"2024-01-11T07:18:12.437511+0000","last_active":"2024-01-11T07:18:12.439388+0000","last_peered":"2024-01-11T07:18:12.439388+0000","last_clean":"2024-01-11T07:18:12.439388+0000","last_became_active":"2024-01-11T07:18:12.436266+0000","last_became_peered":"2024-01-11T07:18:12.436266+0000","last_unstale":"2024-01-11T07:18:12.439388+0000","last_undegraded":"2024-01-11T07:18:12.439388+0000","last_fullsized":"2024-01-11T07:18:12.439388+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:28:14.594448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.436926+0000","last_change":"2024-01-11T07:18:13.436926+0000","last_active":"2024-01-11T07:18:13.436926+0000","last_peered":"2024-01-11T07:18:13.436926+0000","last_clean":"2024-01-11T07:18:13.436926+0000","last_became_active":"2024-01-11T07:18:13.436203+0000","last_became_peered":"2024-01-11T07:18:13.436203+0000","last_unstale":"2024-01-11T07:18:13.436926+0000","last_undegraded":"2024-01-11T07:18:13.436926+0000","last_fullsized":"2024-01-11T07:18:13.436926+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:13:29.642574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.436807+0000","last_change":"2024-01-11T07:18:12.436738+0000","last_active":"2024-01-11T07:18:12.436807+0000","last_peered":"2024-01-11T07:18:12.436807+0000","last_clean":"2024-01-11T07:18:12.436807+0000","last_became_active":"2024-01-11T07:18:12.436496+0000","last_became_peered":"2024-01-11T07:18:12.436496+0000","last_unstale":"2024-01-11T07:18:12.436807+0000","last_undegraded":"2024-01-11T07:18:12.436807+0000","last_fullsized":"2024-01-11T07:18:12.436807+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:28:50.541247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.435710+0000","last_change":"2024-01-11T07:18:13.435710+0000","last_active":"2024-01-11T07:18:13.435710+0000","last_peered":"2024-01-11T07:18:13.435710+0000","last_clean":"2024-01-11T07:18:13.435710+0000","last_became_active":"2024-01-11T07:18:13.434837+0000","last_became_peered":"2024-01-11T07:18:13.434837+0000","last_unstale":"2024-01-11T07:18:13.435710+0000","last_undegraded":"2024-01-11T07:18:13.435710+0000","last_fullsized":"2024-01-11T07:18:13.435710+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:24:42.362653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.437850+0000","last_change":"2024-01-11T07:18:12.437585+0000","last_active":"2024-01-11T07:18:12.437850+0000","last_peered":"2024-01-11T07:18:12.437850+0000","last_clean":"2024-01-11T07:18:12.437850+0000","last_became_active":"2024-01-11T07:18:12.436473+0000","last_became_peered":"2024-01-11T07:18:12.436473+0000","last_unstale":"2024-01-11T07:18:12.437850+0000","last_undegraded":"2024-01-11T07:18:12.437850+0000","last_fullsized":"2024-01-11T07:18:12.437850+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:42:09.811842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.446543+0000","last_change":"2024-01-11T07:18:13.446543+0000","last_active":"2024-01-11T07:18:13.446543+0000","last_peered":"2024-01-11T07:18:13.446543+0000","last_clean":"2024-01-11T07:18:13.446543+0000","last_became_active":"2024-01-11T07:18:13.445913+0000","last_became_peered":"2024-01-11T07:18:13.445913+0000","last_unstale":"2024-01-11T07:18:13.446543+0000","last_undegraded":"2024-01-11T07:18:13.446543+0000","last_fullsized":"2024-01-11T07:18:13.446543+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:14:22.129741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.437221+0000","last_change":"2024-01-11T07:18:12.437168+0000","last_active":"2024-01-11T07:18:12.437221+0000","last_peered":"2024-01-11T07:18:12.437221+0000","last_clean":"2024-01-11T07:18:12.437221+0000","last_became_active":"2024-01-11T07:18:12.436979+0000","last_became_peered":"2024-01-11T07:18:12.436979+0000","last_unstale":"2024-01-11T07:18:12.437221+0000","last_undegraded":"2024-01-11T07:18:12.437221+0000","last_fullsized":"2024-01-11T07:18:12.437221+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T18:03:30.684974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.434298+0000","last_change":"2024-01-11T07:18:13.434298+0000","last_active":"2024-01-11T07:18:13.434298+0000","last_peered":"2024-01-11T07:18:13.434298+0000","last_clean":"2024-01-11T07:18:13.434298+0000","last_became_active":"2024-01-11T07:18:13.433918+0000","last_became_peered":"2024-01-11T07:18:13.433918+0000","last_unstale":"2024-01-11T07:18:13.434298+0000","last_undegraded":"2024-01-11T07:18:13.434298+0000","last_fullsized":"2024-01-11T07:18:13.434298+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T14:56:51.191605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.428221+0000","last_change":"2024-01-11T07:18:12.428149+0000","last_active":"2024-01-11T07:18:12.428221+0000","last_peered":"2024-01-11T07:18:12.428221+0000","last_clean":"2024-01-11T07:18:12.428221+0000","last_became_active":"2024-01-11T07:18:12.427881+0000","last_became_peered":"2024-01-11T07:18:12.427881+0000","last_unstale":"2024-01-11T07:18:12.428221+0000","last_undegraded":"2024-01-11T07:18:12.428221+0000","last_fullsized":"2024-01-11T07:18:12.428221+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:06:53.959251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-01-11T07:18:13.439231+0000","last_change":"2024-01-11T07:18:13.439231+0000","last_active":"2024-01-11T07:18:13.439231+0000","last_peered":"2024-01-11T07:18:13.439231+0000","last_clean":"2024-01-11T07:18:13.439231+0000","last_became_active":"2024-01-11T07:18:13.438912+0000","last_became_peered":"2024-01-11T07:18:13.438912+0000","last_unstale":"2024-01-11T07:18:13.439231+0000","last_undegraded":"2024-01-11T07:18:13.439231+0000","last_fullsized":"2024-01-11T07:18:13.439231+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:12.416736+0000","last_clean_scrub_stamp":"2024-01-11T07:18:12.416736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T11:20:33.525439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.438152+0000","last_change":"2024-01-11T07:18:12.438083+0000","last_active":"2024-01-11T07:18:12.438152+0000","last_peered":"2024-01-11T07:18:12.438152+0000","last_clean":"2024-01-11T07:18:12.438152+0000","last_became_active":"2024-01-11T07:18:12.437846+0000","last_became_peered":"2024-01-11T07:18:12.437846+0000","last_unstale":"2024-01-11T07:18:12.438152+0000","last_undegraded":"2024-01-11T07:18:12.438152+0000","last_fullsized":"2024-01-11T07:18:12.438152+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:18:11.411326+0000","last_clean_scrub_stamp":"2024-01-11T07:18:11.411326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:04:09.621327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":308,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-01-11T07:18:12.427740+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:18:12.427740+0000","last_peered":"2024-01-11T07:18:12.427740+0000","last_clean":"2024-01-11T07:18:12.427740+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:18:12.427740+0000","last_undegraded":"2024-01-11T07:18:12.427740+0000","last_fullsized":"2024-01-11T07:18:12.427740+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1569,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193742,"statfs":{"total":200000000,"available":198392404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.504}]},{"osd":1,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.433,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.56599999999999995},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.80500000000000005},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.81399999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75800000000000001,"5min":0.78700000000000003,"15min":0.82399999999999995},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.82999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42799999999999999,"15min":0.40600000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.83799999999999997},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45600000000000002,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.77700000000000002,"15min":0.85999999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.33800000000000002,"15min":0.308},"max":{"1min":0.59599999999999997,"5min":0.755,"15min":0.85899999999999999},"last":0.379}]},{"osd":5,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.66300000000000003,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.66600000000000004,"5min":0.84599999999999997,"15min":0.91600000000000004},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.82199999999999995,"15min":0.92500000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.90300000000000002},"last":0.437}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1000,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194312,"statfs":{"total":200000000,"available":198975987,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.81399999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.44600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77100000000000002,"5min":0.85199999999999998,"15min":1.157},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.45800000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65000000000000002,"5min":0.79500000000000004,"15min":2.5720000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":0.67300000000000004,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.66700000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.84599999999999997,"15min":2.621},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88500000000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48099999999999998,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.25800000000000001},"max":{"1min":0.66300000000000003,"5min":0.76500000000000001,"15min":0.82699999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.73999999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.253},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":2.0209999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.313},"max":{"1min":0.69299999999999995,"5min":0.77900000000000003,"15min":0.89100000000000001},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28699999999999998},"max":{"1min":0.64100000000000001,"5min":0.68600000000000005,"15min":0.76700000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53200000000000003,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":0.70899999999999996,"15min":2.0750000000000002},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":993,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194319,"statfs":{"total":200000000,"available":198982772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.247,"15min":0.216},"max":{"1min":0.53100000000000003,"5min":0.753,"15min":20.41},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.46100000000000002},"min":{"1min":0.371,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":0.60699999999999998,"5min":0.72099999999999997,"15min":3.7029999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44,"15min":0.497},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60199999999999998,"5min":3.597,"15min":15.611000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":3.0710000000000002,"15min":20.199999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":0.67900000000000005,"5min":3.8460000000000001,"15min":20.369},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.28299999999999997},"max":{"1min":0.499,"5min":4.7569999999999997,"15min":8.5790000000000006},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":1.3300000000000001,"15min":9.9570000000000007},"last":0.443},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":1.2030000000000001,"15min":12.449999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":1.3540000000000001,"15min":2.6030000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.51300000000000001,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.90900000000000003,"5min":1.177,"15min":9.891},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":1.389,"15min":1.5569999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69199999999999995,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.94699999999999995,"5min":2.9049999999999998,"15min":9.9900000000000002},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.1599999999999999,"15min":20.324000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.46800000000000003},"max":{"1min":0.96999999999999997,"5min":1.268,"15min":2.548},"last":0.42399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1576,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193735,"statfs":{"total":200000000,"available":198385641,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39600000000000002,"15min":0.42999999999999999},"min":{"1min":0.27100000000000002,"5min":0.23300000000000001,"15min":0.224},"max":{"1min":0.57899999999999996,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.29299999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57399999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.48799999999999999},"min":{"1min":0.36499999999999999,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.61699999999999999,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.57799999999999996},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58899999999999997,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64000000000000001,"5min":0.83499999999999996,"15min":2.855},"last":0.37}]},{"osd":4,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.495,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90900000000000003,"15min":1.3959999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.40999999999999998},"max":{"1min":0.92500000000000004,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":0.76200000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":1.032,"15min":2.819},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.96599999999999997,"15min":1.2490000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95899999999999996,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.501,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.68000000000000005},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.276},"last":0.746},{"interface":"front","average":{"1min":0.748,"5min":0.747,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.47499999999999998,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":993,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194319,"statfs":{"total":200000000,"available":198982772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.504,"5min":0.72999999999999998,"15min":1.472},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.311,"5min":0.27900000000000003,"15min":0.23599999999999999},"max":{"1min":0.57799999999999996,"5min":0.65500000000000003,"15min":0.86099999999999999},"last":0.29699999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.58899999999999997,"5min":0.628,"15min":1.2430000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48799999999999999},"min":{"1min":0.41399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63300000000000001,"5min":0.752,"15min":0.76800000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.63100000000000001,"5min":0.76500000000000001,"15min":1.139},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.0660000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.089},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48199999999999998,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":1.054,"15min":1.099},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":1.0389999999999999,"15min":1.3100000000000001},"last":0.623}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0109999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.052},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0580000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.55400000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.54600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":999,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194312,"statfs":{"total":200000000,"available":198976009,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.57399999999999995},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":0.58199999999999996,"5min":0.59699999999999998,"15min":11.481},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.47299999999999998},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":1.6200000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40699999999999997,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.23899999999999999},"max":{"1min":0.59899999999999998,"5min":0.70499999999999996,"15min":17.516999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67400000000000004},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":0.56000000000000005,"5min":0.746,"15min":0.76500000000000001},"last":0.5},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46400000000000002,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":17.553000000000001},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.94599999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.95099999999999996},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.36199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":17.474},"last":0.81799999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64600000000000002,"15min":0.89600000000000002},"min":{"1min":0.49199999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":17.393000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.81899999999999995},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.94499999999999995,"15min":11.394},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.82899999999999996},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.91900000000000004,"15min":11.436999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.878,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.96999999999999997,"15min":1.0449999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.93799999999999994},"min":{"1min":0.53700000000000003,"5min":0.45800000000000002,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":0.95499999999999996,"15min":17.434000000000001},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":993,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194319,"statfs":{"total":200000000,"available":198982772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.81299999999999994},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.73599999999999999,"5min":0.94799999999999995,"15min":8.484},"last":0.246},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.84599999999999997},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":0.84699999999999998,"15min":9.2599999999999998},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":1.1599999999999999,"15min":8.5340000000000007},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.78100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.81000000000000005,"15min":9.2279999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59999999999999998,"15min":0.78200000000000003},"min":{"1min":0.376,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.71999999999999997,"5min":1.1120000000000001,"15min":9.5640000000000001},"last":0.376},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.88600000000000001,"15min":6.8639999999999999},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.78500000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":8.9779999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.84699999999999998},"min":{"1min":0.50800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":1.218,"15min":8.7309999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.67700000000000005},"min":{"1min":0.318,"5min":0.313,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":11.388999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46899999999999997,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":8.6509999999999998},"last":0.501}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.69399999999999995},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":9.1449999999999996},"last":0.44},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.81899999999999995,"15min":1.119},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52200000000000002,"15min":0.69499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33500000000000002},"max":{"1min":0.70599999999999996,"5min":0.85399999999999998,"15min":8.6050000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":9.4640000000000004},"last":0.56399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":993,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194319,"statfs":{"total":200000000,"available":198982772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.81899999999999995},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.57699999999999996},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.24399999999999999},"max":{"1min":0.73999999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.93999999999999995},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34999999999999998},"max":{"1min":0.70399999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.74199999999999999,"15min":0.76000000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.496,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.73799999999999999,"15min":0.78600000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.56100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.0109999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.88200000000000001},"last":0.70799999999999996}]}]}],"pool_statfs":[]}} 2024-01-11T07:18:18.044 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-11T07:18:18.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-11T07:18:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.220+0000 7ff8478c6700 1 Processor -- start 2024-01-11T07:18:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.220+0000 7ff8478c6700 1 -- start start 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff8478c6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8400cafb0 0x7ff8400ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff8478c6700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8400b0b40 0x7ff840161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff8478c6700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff840159300 con 0x7ff840158280 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff8478c6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8400cf260 con 0x7ff8400cafb0 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff8478c6700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8400cf540 con 0x7ff8400b0b40 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff845662700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8400cafb0 0x7ff8400ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff845662700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8400cafb0 0x7ff8400ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57214/0 (socket says 172.21.15.7:57214) 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff845662700 1 -- 172.21.15.7:0/3478264091 learned_addr learned my addr 172.21.15.7:0/3478264091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.221+0000 7ff845e63700 1 --2- 172.21.15.7:0/3478264091 >> v2:172.21.15.113:3300/0 conn(0x7ff8400b0b40 0x7ff840161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.222+0000 7ff845662700 1 -- 172.21.15.7:0/3478264091 >> v2:172.21.15.113:3300/0 conn(0x7ff8400b0b40 msgr2=0x7ff840161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.222+0000 7ff845662700 1 --2- 172.21.15.7:0/3478264091 >> v2:172.21.15.113:3300/0 conn(0x7ff8400b0b40 0x7ff840161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.222+0000 7ff845662700 1 -- 172.21.15.7:0/3478264091 >> v1:172.21.15.113:6789/0 conn(0x7ff840158280 legacy=0x7ff8400b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.222+0000 7ff845662700 1 -- 172.21.15.7:0/3478264091 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400cf820 con 0x7ff8400cafb0 2024-01-11T07:18:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.222+0000 7ff845662700 1 --2- 172.21.15.7:0/3478264091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400cafb0 0x7ff8400ce8f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff84006cd70 tx=0x7ff83c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d478de3344d2d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.222+0000 7ff837fff700 1 -- 172.21.15.7:0/3478264091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c009e70 con 0x7ff8400cafb0 2024-01-11T07:18:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.222+0000 7ff837fff700 1 -- 172.21.15.7:0/3478264091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c00b860 con 0x7ff8400cafb0 2024-01-11T07:18:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.222+0000 7ff837fff700 1 -- 172.21.15.7:0/3478264091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c00bc00 con 0x7ff8400cafb0 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.223+0000 7ff8478c6700 1 -- 172.21.15.7:0/3478264091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400cafb0 msgr2=0x7ff8400ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.223+0000 7ff8478c6700 1 --2- 172.21.15.7:0/3478264091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400cafb0 0x7ff8400ce8f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff84006cd70 tx=0x7ff83c009250 comp rx=0 tx=0).stop 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.223+0000 7ff8478c6700 1 -- 172.21.15.7:0/3478264091 shutdown_connections 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.223+0000 7ff8478c6700 1 --2- 172.21.15.7:0/3478264091 >> v2:172.21.15.113:3300/0 conn(0x7ff8400b0b40 0x7ff840161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.223+0000 7ff8478c6700 1 --2- 172.21.15.7:0/3478264091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400cafb0 0x7ff8400ce8f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.223+0000 7ff8478c6700 1 -- 172.21.15.7:0/3478264091 >> 172.21.15.7:0/3478264091 conn(0x7ff840071100 msgr2=0x7ff84006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.223+0000 7ff8478c6700 1 -- 172.21.15.7:0/3478264091 shutdown_connections 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.223+0000 7ff8478c6700 1 -- 172.21.15.7:0/3478264091 wait complete. 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.224+0000 7ff8478c6700 1 Processor -- start 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.224+0000 7ff8478c6700 1 -- start start 2024-01-11T07:18:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff8478c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400b0b40 0x7ff8400b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff8478c6700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8400cafb0 0x7ff8400c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff8478c6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff840158280 0x7ff8400c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff8478c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff840167dd0 con 0x7ff8400b0b40 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff8478c6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff840168bd0 con 0x7ff840158280 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff8478c6700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff840167fc0 con 0x7ff8400cafb0 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845662700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400b0b40 0x7ff8400b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845662700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400b0b40 0x7ff8400b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57216/0 (socket says 172.21.15.7:57216) 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845662700 1 -- 172.21.15.7:0/1820659896 learned_addr learned my addr 172.21.15.7:0/1820659896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845e63700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff840158280 0x7ff8400c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff844e61700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8400cafb0 0x7ff8400c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845662700 1 -- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8400cafb0 msgr2=0x7ff8400c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845662700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8400cafb0 0x7ff8400c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845662700 1 -- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff840158280 msgr2=0x7ff8400c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845662700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff840158280 0x7ff8400c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845662700 1 -- 172.21.15.7:0/1820659896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c0094f0 con 0x7ff8400b0b40 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.225+0000 7ff845e63700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff840158280 0x7ff8400c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.226+0000 7ff845662700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400b0b40 0x7ff8400b39e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff84006a1d0 tx=0x7ff83c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.226+0000 7ff8357fa700 1 -- 172.21.15.7:0/1820659896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c011420 con 0x7ff8400b0b40 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.226+0000 7ff8357fa700 1 -- 172.21.15.7:0/1820659896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c0115a0 con 0x7ff8400b0b40 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.226+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8401682a0 con 0x7ff8400b0b40 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.226+0000 7ff8357fa700 1 -- 172.21.15.7:0/1820659896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c011960 con 0x7ff8400b0b40 2024-01-11T07:18:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.226+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff840168860 con 0x7ff8400b0b40 2024-01-11T07:18:18.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.227+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff840158ea0 con 0x7ff8400b0b40 2024-01-11T07:18:18.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.227+0000 7ff8357fa700 1 -- 172.21.15.7:0/1820659896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff83c0253f0 con 0x7ff8400b0b40 2024-01-11T07:18:18.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.228+0000 7ff8357fa700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff824066f30 0x7ff8240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.228+0000 7ff8357fa700 1 -- 172.21.15.7:0/1820659896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff83c085ef0 con 0x7ff8400b0b40 2024-01-11T07:18:18.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.228+0000 7ff844e61700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff824066f30 0x7ff8240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.228+0000 7ff844e61700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff824066f30 0x7ff8240693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff8400ca960 tx=0x7ff830009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:18.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.232+0000 7ff8357fa700 1 -- 172.21.15.7:0/1820659896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff83c053160 con 0x7ff8400b0b40 2024-01-11T07:18:18.409 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-01-11T07:18:18.409+0000 7f33f5fbf700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:18:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.526+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff84004c3c0 con 0x7ff8400b0b40 2024-01-11T07:18:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.530+0000 7ff8357fa700 1 -- 172.21.15.7:0/1820659896 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff83c0150e0 con 0x7ff8400b0b40 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff824066f30 msgr2=0x7ff8240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff824066f30 0x7ff8240693e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff8400ca960 tx=0x7ff830009040 comp rx=0 tx=0).stop 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400b0b40 msgr2=0x7ff8400b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400b0b40 0x7ff8400b39e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff84006a1d0 tx=0x7ff83c017000 comp rx=0 tx=0).stop 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 shutdown_connections 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff824066f30 0x7ff8240693e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff840158280 0x7ff8400c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8400b0b40 0x7ff8400b39e0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.532+0000 7ff8478c6700 1 --2- 172.21.15.7:0/1820659896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8400cafb0 0x7ff8400c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.533+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 >> 172.21.15.7:0/1820659896 conn(0x7ff840071100 msgr2=0x7ff8400ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.533+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 shutdown_connections 2024-01-11T07:18:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.533+0000 7ff8478c6700 1 -- 172.21.15.7:0/1820659896 wait complete. 2024-01-11T07:18:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-01-11T07:18:18.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:18.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.679+0000 7f49a3563700 1 Processor -- start 2024-01-11T07:18:18.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.680+0000 7f49a3563700 1 -- start start 2024-01-11T07:18:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.680+0000 7f49a3563700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f499c0aff20 0x7f499c0aab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.680+0000 7f49a3563700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c063680 0x7f499c06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.680+0000 7f49a3563700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c0ab040 con 0x7f499c063230 2024-01-11T07:18:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.680+0000 7f49a3563700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c0b02f0 con 0x7f499c0aff20 2024-01-11T07:18:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.680+0000 7f49a3563700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c154860 con 0x7f499c063680 2024-01-11T07:18:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a12ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f499c0aff20 0x7f499c0aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a12ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f499c0aff20 0x7f499c0aab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57226/0 (socket says 172.21.15.7:57226) 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a12ff700 1 -- 172.21.15.7:0/3090058178 learned_addr learned my addr 172.21.15.7:0/3090058178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a1b00700 1 --2- 172.21.15.7:0/3090058178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c063680 0x7f499c06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a12ff700 1 -- 172.21.15.7:0/3090058178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c063680 msgr2=0x7f499c06e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a12ff700 1 --2- 172.21.15.7:0/3090058178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c063680 0x7f499c06e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a12ff700 1 -- 172.21.15.7:0/3090058178 >> v1:172.21.15.113:6789/0 conn(0x7f499c063230 legacy=0x7f499c06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a12ff700 1 -- 172.21.15.7:0/3090058178 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c15d910 con 0x7f499c0aff20 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f49a12ff700 1 --2- 172.21.15.7:0/3090058178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c0aff20 0x7f499c0aab00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f49980099b0 tx=0x7f4998009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d531d73b218ed33a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f4993fff700 1 -- 172.21.15.7:0/3090058178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998009ea0 con 0x7f499c0aff20 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f4993fff700 1 -- 172.21.15.7:0/3090058178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499800b860 con 0x7f499c0aff20 2024-01-11T07:18:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.681+0000 7f4993fff700 1 -- 172.21.15.7:0/3090058178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499800bc00 con 0x7f499c0aff20 2024-01-11T07:18:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.682+0000 7f49a3563700 1 -- 172.21.15.7:0/3090058178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c0aff20 msgr2=0x7f499c0aab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.682+0000 7f49a3563700 1 --2- 172.21.15.7:0/3090058178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c0aff20 0x7f499c0aab00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f49980099b0 tx=0x7f4998009250 comp rx=0 tx=0).stop 2024-01-11T07:18:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.682+0000 7f49a3563700 1 -- 172.21.15.7:0/3090058178 shutdown_connections 2024-01-11T07:18:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.682+0000 7f49a3563700 1 --2- 172.21.15.7:0/3090058178 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c063680 0x7f499c06e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.682+0000 7f49a3563700 1 --2- 172.21.15.7:0/3090058178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c0aff20 0x7f499c0aab00 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.682+0000 7f49a3563700 1 -- 172.21.15.7:0/3090058178 >> 172.21.15.7:0/3090058178 conn(0x7f499c068090 msgr2=0x7f499c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.682+0000 7f49a3563700 1 -- 172.21.15.7:0/3090058178 shutdown_connections 2024-01-11T07:18:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.682+0000 7f49a3563700 1 -- 172.21.15.7:0/3090058178 wait complete. 2024-01-11T07:18:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.684+0000 7f49a3563700 1 Processor -- start 2024-01-11T07:18:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.684+0000 7f49a3563700 1 -- start start 2024-01-11T07:18:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a3563700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c063230 0x7f499c152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a3563700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499c063680 0x7f499c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a3563700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c0aff20 0x7f499c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a3563700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0b2bd0 con 0x7f499c063230 2024-01-11T07:18:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a3563700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0b1fc0 con 0x7f499c063680 2024-01-11T07:18:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a3563700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0b22a0 con 0x7f499c0aff20 2024-01-11T07:18:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a12ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c063230 0x7f499c152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a12ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c063230 0x7f499c152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57234/0 (socket says 172.21.15.7:57234) 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a12ff700 1 -- 172.21.15.7:0/3458764716 learned_addr learned my addr 172.21.15.7:0/3458764716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a1b00700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c0aff20 0x7f499c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.685+0000 7f49a0afe700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499c063680 0x7f499c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49a12ff700 1 -- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c0aff20 msgr2=0x7f499c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49a12ff700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c0aff20 0x7f499c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49a12ff700 1 -- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499c063680 msgr2=0x7f499c0b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49a12ff700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499c063680 0x7f499c0b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49a12ff700 1 -- 172.21.15.7:0/3458764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980094f0 con 0x7f499c063230 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49a12ff700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c063230 0x7f499c152e50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f499c0cd400 tx=0x7f4998017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49917fa700 1 -- 172.21.15.7:0/3458764716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998000d10 con 0x7f499c063230 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49917fa700 1 -- 172.21.15.7:0/3458764716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499800bbe0 con 0x7f499c063230 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c0b2580 con 0x7f499c063230 2024-01-11T07:18:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49917fa700 1 -- 172.21.15.7:0/3458764716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998011670 con 0x7f499c063230 2024-01-11T07:18:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.686+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c153390 con 0x7f499c063230 2024-01-11T07:18:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.687+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c069bc0 con 0x7f499c063230 2024-01-11T07:18:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.687+0000 7f49917fa700 1 -- 172.21.15.7:0/3458764716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4998011aa0 con 0x7f499c063230 2024-01-11T07:18:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.688+0000 7f49917fa700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4980066f30 0x7f49800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.688+0000 7f49917fa700 1 -- 172.21.15.7:0/3458764716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4998085d40 con 0x7f499c063230 2024-01-11T07:18:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.688+0000 7f49a0afe700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4980066f30 0x7f49800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.689+0000 7f49a0afe700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4980066f30 0x7f49800693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f499c15cd60 tx=0x7f498c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:18.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.693+0000 7f49917fa700 1 -- 172.21.15.7:0/3458764716 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4998052fb0 con 0x7f499c063230 2024-01-11T07:18:18.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.942+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f499c04c3c0 con 0x7f499c063230 2024-01-11T07:18:18.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.942+0000 7f49917fa700 1 -- 172.21.15.7:0/3458764716 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f49980565d0 con 0x7f499c063230 2024-01-11T07:18:18.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:18.944 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4980066f30 msgr2=0x7f49800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4980066f30 0x7f49800693e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f499c15cd60 tx=0x7f498c007ec0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c063230 msgr2=0x7f499c152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c063230 0x7f499c152e50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f499c0cd400 tx=0x7f4998017000 comp rx=0 tx=0).stop 2024-01-11T07:18:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 shutdown_connections 2024-01-11T07:18:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4980066f30 0x7f49800693e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f499c0aff20 0x7f499c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c063230 0x7f499c152e50 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 --2- 172.21.15.7:0/3458764716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f499c063680 0x7f499c0b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 >> 172.21.15.7:0/3458764716 conn(0x7f499c068090 msgr2=0x7f499c0c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 shutdown_connections 2024-01-11T07:18:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:18.945+0000 7f49a3563700 1 -- 172.21.15.7:0/3458764716 wait complete. 2024-01-11T07:18:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:19.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-11T07:18:12.853740+0000', 'modified': '2024-01-11T07:18:16.451460+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8064}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8064': {'gid': 8064, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/3920146306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3920146306}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3920146306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-01-11T07:18:19.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-11T07:18:19.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-11T07:18:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.505+0000 7f9a4b084700 1 Processor -- start 2024-01-11T07:18:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.505+0000 7f9a4b084700 1 -- start start 2024-01-11T07:18:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a4b084700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a441579b0 0x7f9a4415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a4b084700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a44063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a4b084700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a44063770 0x7f9a4406dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a4b084700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a44063bb0 con 0x7f9a44063770 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a4b084700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a440aeab0 con 0x7f9a441579b0 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a4b084700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440aed90 con 0x7f9a44157d80 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a44063770 0x7f9a4406dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a44063770 0x7f9a4406dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57246/0 (socket says 172.21.15.7:57246) 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 -- 172.21.15.7:0/2106692583 learned_addr learned my addr 172.21.15.7:0/2106692583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a43fff700 1 --2- 172.21.15.7:0/2106692583 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a44063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a48e20700 1 --2- 172.21.15.7:0/2106692583 >> v2:172.21.15.113:3300/0 conn(0x7f9a441579b0 0x7f9a4415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 -- 172.21.15.7:0/2106692583 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 msgr2=0x7f9a44063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 --2- 172.21.15.7:0/2106692583 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a44063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 -- 172.21.15.7:0/2106692583 >> v2:172.21.15.113:3300/0 conn(0x7f9a441579b0 msgr2=0x7f9a4415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 --2- 172.21.15.7:0/2106692583 >> v2:172.21.15.113:3300/0 conn(0x7f9a441579b0 0x7f9a4415bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 -- 172.21.15.7:0/2106692583 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a440af070 con 0x7f9a44063770 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a43fff700 1 --2- 172.21.15.7:0/2106692583 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a44063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.506+0000 7f9a49621700 1 --2- 172.21.15.7:0/2106692583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a44063770 0x7f9a4406dab0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9a4406d6b0 tx=0x7f9a3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70658a8bf3416c01 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a437fe700 1 -- 172.21.15.7:0/2106692583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00e070 con 0x7f9a44063770 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a437fe700 1 -- 172.21.15.7:0/2106692583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c00ba20 con 0x7f9a44063770 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a437fe700 1 -- 172.21.15.7:0/2106692583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0092e0 con 0x7f9a44063770 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 -- 172.21.15.7:0/2106692583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a44063770 msgr2=0x7f9a4406dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 --2- 172.21.15.7:0/2106692583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a44063770 0x7f9a4406dab0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9a4406d6b0 tx=0x7f9a3c009840 comp rx=0 tx=0).stop 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 -- 172.21.15.7:0/2106692583 shutdown_connections 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 --2- 172.21.15.7:0/2106692583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a44063770 0x7f9a4406dab0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 --2- 172.21.15.7:0/2106692583 >> v2:172.21.15.113:3300/0 conn(0x7f9a441579b0 0x7f9a4415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 --2- 172.21.15.7:0/2106692583 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a44063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 -- 172.21.15.7:0/2106692583 >> 172.21.15.7:0/2106692583 conn(0x7f9a440680e0 msgr2=0x7f9a440cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:19.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 -- 172.21.15.7:0/2106692583 shutdown_connections 2024-01-11T07:18:19.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.507+0000 7f9a4b084700 1 -- 172.21.15.7:0/2106692583 wait complete. 2024-01-11T07:18:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.508+0000 7f9a4b084700 1 Processor -- start 2024-01-11T07:18:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.509+0000 7f9a4b084700 1 -- start start 2024-01-11T07:18:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.509+0000 7f9a4b084700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a44063770 0x7f9a440b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.509+0000 7f9a4b084700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a441579b0 0x7f9a440b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.509+0000 7f9a4b084700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a4416b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.509+0000 7f9a4b084700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440b1ab0 con 0x7f9a441579b0 2024-01-11T07:18:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.509+0000 7f9a4b084700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440b0c00 con 0x7f9a44063770 2024-01-11T07:18:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.509+0000 7f9a4b084700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440b0ea0 con 0x7f9a44157d80 2024-01-11T07:18:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a441579b0 0x7f9a440b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a441579b0 0x7f9a440b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57262/0 (socket says 172.21.15.7:57262) 2024-01-11T07:18:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 -- 172.21.15.7:0/3600870123 learned_addr learned my addr 172.21.15.7:0/3600870123 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a49621700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a4416b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a48e20700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a44063770 0x7f9a440b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 -- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 msgr2=0x7f9a4416b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a4416b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 -- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a44063770 msgr2=0x7f9a440b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a44063770 0x7f9a440b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 -- 172.21.15.7:0/3600870123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c009ae0 con 0x7f9a441579b0 2024-01-11T07:18:19.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a43fff700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a441579b0 0x7f9a440b2df0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800d860 tx=0x7f9a38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a40ff9700 1 -- 172.21.15.7:0/3600870123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a380125f0 con 0x7f9a441579b0 2024-01-11T07:18:19.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a40ff9700 1 -- 172.21.15.7:0/3600870123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a38012750 con 0x7f9a441579b0 2024-01-11T07:18:19.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a40ff9700 1 -- 172.21.15.7:0/3600870123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38012af0 con 0x7f9a441579b0 2024-01-11T07:18:19.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a4b084700 1 -- 172.21.15.7:0/3600870123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a440b1180 con 0x7f9a441579b0 2024-01-11T07:18:19.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.510+0000 7f9a4b084700 1 -- 172.21.15.7:0/3600870123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a440b1740 con 0x7f9a441579b0 2024-01-11T07:18:19.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.512+0000 7f9a40ff9700 1 -- 172.21.15.7:0/3600870123 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9a38013480 con 0x7f9a441579b0 2024-01-11T07:18:19.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.512+0000 7f9a40ff9700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a30066f30 0x7f9a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.512+0000 7f9a40ff9700 1 -- 172.21.15.7:0/3600870123 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a38084710 con 0x7f9a441579b0 2024-01-11T07:18:19.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.513+0000 7f9a48e20700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a30066f30 0x7f9a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.513+0000 7f9a48e20700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a30066f30 0x7f9a300693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a440c37f0 tx=0x7f9a340091a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.514+0000 7f9a54384b80 1 Processor -- start 2024-01-11T07:18:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.514+0000 7f9a54384b80 1 -- start start 2024-01-11T07:18:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e32859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e32a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a54384b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e31b33e0 con 0x5613e32a1d10 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a54384b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e31b0be0 con 0x5613e32855f0 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a54384b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613e31ac9d0 con 0x5613e32a1360 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a49621700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e32a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a49621700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e32a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57266/0 (socket says 172.21.15.7:57266) 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a49621700 1 -- 172.21.15.7:0/1816795114 learned_addr learned my addr 172.21.15.7:0/1816795114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a48e20700 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e32859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.515+0000 7f9a43fff700 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a49621700 1 -- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 msgr2=0x5613e32a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a49621700 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a49621700 1 -- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 msgr2=0x5613e32859c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a49621700 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e32859c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a49621700 1 -- 172.21.15.7:0/1816795114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613e3040360 con 0x5613e32a1d10 2024-01-11T07:18:19.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a43fff700 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a49621700 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e32a2140 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f9a4406d820 tx=0x7f9a3c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60689715e38d57d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a29ffb700 1 -- 172.21.15.7:0/1816795114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0092e0 con 0x5613e32a1d10 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a29ffb700 1 -- 172.21.15.7:0/1816795114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c027430 con 0x5613e32a1d10 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a29ffb700 1 -- 172.21.15.7:0/1816795114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0277d0 con 0x5613e32a1d10 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a54384b80 1 -- 172.21.15.7:0/1816795114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 msgr2=0x5613e32a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.516+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e32a2140 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f9a4406d820 tx=0x7f9a3c018000 comp rx=0 tx=0).stop 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.517+0000 7f9a54384b80 1 -- 172.21.15.7:0/1816795114 shutdown_connections 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.517+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e32a2140 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.517+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e32859c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.517+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1816795114 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.517+0000 7f9a54384b80 1 -- 172.21.15.7:0/1816795114 >> 172.21.15.7:0/1816795114 conn(0x5613e325b440 msgr2=0x5613e32a6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.517+0000 7f9a54384b80 1 -- 172.21.15.7:0/1816795114 shutdown_connections 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.517+0000 7f9a54384b80 1 -- 172.21.15.7:0/1816795114 wait complete. 2024-01-11T07:18:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.517+0000 7f9a54384b80 1 -- start start 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a54384b80 1 Processor -- start 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32855f0 0x5613e32841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1360 0x5613e329bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 0x5613e3280cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a54384b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e31591b0 con 0x5613e32a1360 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a54384b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e3158e50 con 0x5613e32a1d10 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a54384b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613e30a5e40 con 0x5613e32855f0 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a54384b80 10 client.0 _ll_register_callbacks cb 0x5613e30a87b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1360 0x5613e329bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.518+0000 7f9a48e20700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32855f0 0x5613e32841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1360 0x5613e329bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57268/0 (socket says 172.21.15.7:57268) 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a48e20700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32855f0 0x5613e32841c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:33096/0 (socket says 172.21.15.7:33096) 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a49621700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 0x5613e3280cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a43fff700 1 -- 172.21.15.7:0/2501117474 learned_addr learned my addr 172.21.15.7:0/2501117474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a43fff700 1 -- 172.21.15.7:0/2501117474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32855f0 msgr2=0x5613e32841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a43fff700 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32855f0 0x5613e32841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a43fff700 1 -- 172.21.15.7:0/2501117474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 msgr2=0x5613e3280cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a43fff700 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 0x5613e3280cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a43fff700 1 -- 172.21.15.7:0/2501117474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a440af070 con 0x5613e32a1360 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a43fff700 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1360 0x5613e329bec0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9a380075a0 tx=0x7f9a38058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fee20df35bd178 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a22ffd700 1 -- 172.21.15.7:0/2501117474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38058870 con 0x5613e32a1360 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a22ffd700 1 -- 172.21.15.7:0/2501117474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a3805c540 con 0x5613e32a1360 2024-01-11T07:18:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a54384b80 1 -- 172.21.15.7:0/2501117474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613e322deb0 con 0x5613e32a1360 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a22ffd700 1 -- 172.21.15.7:0/2501117474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a380102e0 con 0x5613e32a1360 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a54384b80 10 client.8187 fetch_fsmap learned FSMap version 90 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a54384b80 1 -- 172.21.15.7:0/2501117474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5613e2fecb10 con 0x5613e32a1360 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.519+0000 7f9a22ffd700 1 -- 172.21.15.7:0/2501117474 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5613e322deb0 con 0x5613e32a1360 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a22ffd700 1 -- 172.21.15.7:0/2501117474 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9a38055800 con 0x5613e32a1360 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a54384b80 10 client.8187 fetch_fsmap finished waiting for FSMap version 90 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a54384b80 10 client.8187 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a54384b80 20 client.8187 populate_metadata read hostname 'smithi007' 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a54384b80 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] conn(0x5613e329ad90 0x5613e329b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a54384b80 4 client.8187 mds_command: new command op to 8064 tid=0[{"prefix": "get_command_descriptions"}] 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a54384b80 1 -- 172.21.15.7:0/2501117474 --> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613e3183880 con 0x5613e329ad90 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a48e20700 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] conn(0x5613e329ad90 0x5613e329b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a22ffd700 10 client.8187 ms_handle_connect on v2:172.21.15.7:6834/3920146306 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.520+0000 7f9a48e20700 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] conn(0x5613e329ad90 0x5613e329b160 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3c03870b27d90e server_cookie=f86c6bc7ca8250c5 in_seq=0 out_seq=0 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a22ffd700 1 -- 172.21.15.7:0/2501117474 <== mds.0 v2:172.21.15.7:6834/3920146306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5613e3183880 con 0x5613e329ad90 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a22ffd700 10 client.8187 handle_command_reply: tid=0 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 client.8187 shutdown 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 -- 172.21.15.7:0/2501117474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1360 msgr2=0x5613e329bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1360 0x5613e329bec0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9a380075a0 tx=0x7f9a38058040 comp rx=0 tx=0).stop 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 -- 172.21.15.7:0/2501117474 shutdown_connections 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] conn(0x5613e329ad90 0x5613e329b160 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 0x5613e3280cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32855f0 0x5613e32841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 --2- 172.21.15.7:0/2501117474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1360 0x5613e329bec0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a22ffd700 0 client.8187 ms_handle_reset on v2:172.21.15.7:6834/3920146306 2024-01-11T07:18:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 -- 172.21.15.7:0/2501117474 >> 172.21.15.7:0/2501117474 conn(0x5613e325b440 msgr2=0x5613e325baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 -- 172.21.15.7:0/2501117474 shutdown_connections 2024-01-11T07:18:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 1 -- 172.21.15.7:0/2501117474 wait complete. 2024-01-11T07:18:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.521+0000 7f9a54384b80 20 client.8187 trim_cache size 0 max 16384 2024-01-11T07:18:19.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.539+0000 7f9a54384b80 1 Processor -- start 2024-01-11T07:18:19.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.539+0000 7f9a54384b80 1 -- start start 2024-01-11T07:18:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e3259870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e329ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e325ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a54384b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e32588f0 con 0x5613e32a1d10 2024-01-11T07:18:19.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a54384b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e325b230 con 0x5613e32855f0 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a54384b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613e325d110 con 0x5613e32a1360 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a49621700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e325ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a49621700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e325ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57270/0 (socket says 172.21.15.7:57270) 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.540+0000 7f9a49621700 1 -- 172.21.15.7:0/1025936899 learned_addr learned my addr 172.21.15.7:0/1025936899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a48e20700 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e3259870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a43fff700 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e329ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a49621700 1 -- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 msgr2=0x5613e329ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a49621700 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e329ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a49621700 1 -- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 msgr2=0x5613e3259870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a49621700 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e3259870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a49621700 1 -- 172.21.15.7:0/1025936899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613e317b290 con 0x5613e32a1d10 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a43fff700 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e329ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a48e20700 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e3259870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a49621700 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e325ed30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c01f000 tx=0x7f9a3c020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d904f96c4a8356 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a2a7fc700 1 -- 172.21.15.7:0/1025936899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c001010 con 0x5613e32a1d10 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a2a7fc700 1 -- 172.21.15.7:0/1025936899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c02e040 con 0x5613e32a1d10 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a2a7fc700 1 -- 172.21.15.7:0/1025936899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c027e50 con 0x5613e32a1d10 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a54384b80 1 -- 172.21.15.7:0/1025936899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 msgr2=0x5613e325ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.541+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e325ed30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c01f000 tx=0x7f9a3c020040 comp rx=0 tx=0).stop 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 -- 172.21.15.7:0/1025936899 shutdown_connections 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32a1d10 0x5613e325ed30 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32855f0 0x5613e3259870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1025936899 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e329ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 -- 172.21.15.7:0/1025936899 >> 172.21.15.7:0/1025936899 conn(0x5613e325b440 msgr2=0x5613e32a2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 -- 172.21.15.7:0/1025936899 shutdown_connections 2024-01-11T07:18:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 -- 172.21.15.7:0/1025936899 wait complete. 2024-01-11T07:18:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 -- start start 2024-01-11T07:18:19.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.542+0000 7f9a54384b80 1 Processor -- start 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32855f0 0x5613e32a3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a54384b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 0x5613e32c0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a54384b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e32538f0 con 0x5613e32855f0 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a54384b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613e3284bd0 con 0x5613e32a1d10 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a54384b80 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613e330d600 con 0x5613e32a1360 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a54384b80 10 client.0 _ll_register_callbacks cb 0x7f9a38055ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a48e20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32855f0 0x5613e32a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a48e20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32855f0 0x5613e32a3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57276/0 (socket says 172.21.15.7:57276) 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.543+0000 7f9a48e20700 1 -- 172.21.15.7:0/1308800275 learned_addr learned my addr 172.21.15.7:0/1308800275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:19.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a49621700 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 0x5613e32c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a43fff700 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a48e20700 1 -- 172.21.15.7:0/1308800275 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 msgr2=0x5613e32a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a48e20700 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a48e20700 1 -- 172.21.15.7:0/1308800275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 msgr2=0x5613e32c0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a48e20700 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 0x5613e32c0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a48e20700 1 -- 172.21.15.7:0/1308800275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613e3040360 con 0x5613e32855f0 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a48e20700 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32855f0 0x5613e32a3190 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9a34011930 tx=0x7f9a34019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd4d15a72a83f0c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-01-11T07:18:19.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a22ffd700 1 -- 172.21.15.7:0/1308800275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34029070 con 0x5613e32855f0 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a22ffd700 1 -- 172.21.15.7:0/1308800275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a3401a040 con 0x5613e32855f0 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a54384b80 1 -- 172.21.15.7:0/1308800275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613e3251490 con 0x5613e32855f0 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a22ffd700 1 -- 172.21.15.7:0/1308800275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a34022820 con 0x5613e32855f0 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a54384b80 10 client.8193 fetch_fsmap learned FSMap version 90 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a54384b80 1 -- 172.21.15.7:0/1308800275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5613e3044c80 con 0x5613e32855f0 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.544+0000 7f9a22ffd700 1 -- 172.21.15.7:0/1308800275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5613e3251490 con 0x5613e32855f0 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a22ffd700 1 -- 172.21.15.7:0/1308800275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9a34022980 con 0x5613e32855f0 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 10 client.8193 fetch_fsmap finished waiting for FSMap version 90 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 10 client.8193 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-11T07:18:19.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 20 client.8193 populate_metadata read hostname 'smithi007' 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] conn(0x5613e32c11d0 0x5613e330bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 4 client.8193 mds_command: new command op to 8064 tid=0[{"prefix": "status"}] 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 1 -- 172.21.15.7:0/1308800275 --> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5613e314f910 con 0x5613e32c11d0 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a43fff700 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] conn(0x5613e32c11d0 0x5613e330bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a22ffd700 10 client.8193 ms_handle_connect on v2:172.21.15.7:6834/3920146306 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a43fff700 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] conn(0x5613e32c11d0 0x5613e330bfd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8dcb382dfe1da2 server_cookie=7f1be85805face4 in_seq=0 out_seq=0 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a22ffd700 1 -- 172.21.15.7:0/1308800275 <== mds.0 v2:172.21.15.7:6834/3920146306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5613e314f910 con 0x5613e32c11d0 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a22ffd700 10 client.8193 handle_command_reply: tid=0 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 1 client.8193 shutdown 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 1 -- 172.21.15.7:0/1308800275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32855f0 msgr2=0x5613e32a3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.545+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32855f0 0x5613e32a3190 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9a34011930 tx=0x7f9a34019000 comp rx=0 tx=0).stop 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 1 -- 172.21.15.7:0/1308800275 shutdown_connections 2024-01-11T07:18:19.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.7:6834/3920146306,v1:172.21.15.7:6835/3920146306] conn(0x5613e32c11d0 0x5613e330bfd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5613e32a1d10 0x5613e32c0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613e32855f0 0x5613e32a3190 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a22ffd700 0 client.8193 ms_handle_reset on v2:172.21.15.7:6834/3920146306 2024-01-11T07:18:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 1 --2- 172.21.15.7:0/1308800275 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5613e32a1360 0x5613e32a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 1 -- 172.21.15.7:0/1308800275 >> 172.21.15.7:0/1308800275 conn(0x5613e325b440 msgr2=0x5613e325a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 1 -- 172.21.15.7:0/1308800275 shutdown_connections 2024-01-11T07:18:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 1 -- 172.21.15.7:0/1308800275 wait complete. 2024-01-11T07:18:19.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a54384b80 20 client.8193 trim_cache size 0 max 16384 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 8064, 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1243291329999998, 2024-01-11T07:18:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 90, 2024-01-11T07:18:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 184, 2024-01-11T07:18:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 184, 2024-01-11T07:18:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 12.13888189 2024-01-11T07:18:19.556 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 -- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a30066f30 msgr2=0x7f9a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a30066f30 0x7f9a300693e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a440c37f0 tx=0x7f9a340091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 -- 172.21.15.7:0/3600870123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a441579b0 msgr2=0x7f9a440b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a441579b0 0x7f9a440b2df0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800d860 tx=0x7f9a38008040 comp rx=0 tx=0).stop 2024-01-11T07:18:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 -- 172.21.15.7:0/3600870123 shutdown_connections 2024-01-11T07:18:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9a30066f30 0x7f9a300693e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a44157d80 0x7f9a4416b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a44063770 0x7f9a440b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 --2- 172.21.15.7:0/3600870123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a441579b0 0x7f9a440b2df0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.546+0000 7f9a4b084700 1 -- 172.21.15.7:0/3600870123 >> 172.21.15.7:0/3600870123 conn(0x7f9a440680e0 msgr2=0x7f9a440afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.547+0000 7f9a4b084700 1 -- 172.21.15.7:0/3600870123 shutdown_connections 2024-01-11T07:18:19.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.547+0000 7f9a4b084700 1 -- 172.21.15.7:0/3600870123 wait complete. 2024-01-11T07:18:19.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:18:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.691+0000 7efc00cb9700 1 Processor -- start 2024-01-11T07:18:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.691+0000 7efc00cb9700 1 -- start start 2024-01-11T07:18:19.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efc00cb9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efbfc0c7f00 0x7efbfc0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efc00cb9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc157880 con 0x7efbfc153ec0 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efc00cb9700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc069a20 con 0x7efbfc0c7b30 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efc00cb9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc0cbd50 con 0x7efbfc0c7f00 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efbfad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efbfc0c7f00 0x7efbfc0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efbf9d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efbfc153ec0 0x7efbfc0cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43558/0 (socket says 172.21.15.7:43558) 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efbf9d9b700 1 -- 172.21.15.7:0/4001202519 learned_addr learned my addr 172.21.15.7:0/4001202519 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efbfad9d700 1 -- 172.21.15.7:0/4001202519 >> v1:172.21.15.113:6789/0 conn(0x7efbfc0c7b30 legacy=0x7efbfc1537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efbfad9d700 1 -- 172.21.15.7:0/4001202519 >> v1:172.21.15.7:6789/0 conn(0x7efbfc153ec0 legacy=0x7efbfc0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.694+0000 7efbfad9d700 1 -- 172.21.15.7:0/4001202519 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc0cc030 con 0x7efbfc0c7f00 2024-01-11T07:18:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.693+0000 7efbf959a700 1 -- 172.21.15.7:0/4001202519 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351553102 0 0) 0x7efbfc157880 con 0x7efbfc153ec0 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.694+0000 7efbfad9d700 1 --2- 172.21.15.7:0/4001202519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7f00 0x7efbfc0cb6c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7efbf400af70 tx=0x7efbf400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962e4aa719976edd server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.694+0000 7efbf959a700 1 -- 172.21.15.7:0/4001202519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf400f070 con 0x7efbfc0c7f00 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.694+0000 7efbf959a700 1 -- 172.21.15.7:0/4001202519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf400a2e0 con 0x7efbfc0c7f00 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.694+0000 7efbf959a700 1 -- 172.21.15.7:0/4001202519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf400cd20 con 0x7efbfc0c7f00 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.694+0000 7efc00cb9700 1 -- 172.21.15.7:0/4001202519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7f00 msgr2=0x7efbfc0cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.694+0000 7efc00cb9700 1 --2- 172.21.15.7:0/4001202519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7f00 0x7efbfc0cb6c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7efbf400af70 tx=0x7efbf400a890 comp rx=0 tx=0).stop 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.695+0000 7efc00cb9700 1 -- 172.21.15.7:0/4001202519 shutdown_connections 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.695+0000 7efc00cb9700 1 --2- 172.21.15.7:0/4001202519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7f00 0x7efbfc0cb6c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.695+0000 7efc00cb9700 1 -- 172.21.15.7:0/4001202519 >> 172.21.15.7:0/4001202519 conn(0x7efbfc0cd090 msgr2=0x7efbfc0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.695+0000 7efc00cb9700 1 -- 172.21.15.7:0/4001202519 shutdown_connections 2024-01-11T07:18:19.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.695+0000 7efc00cb9700 1 -- 172.21.15.7:0/4001202519 wait complete. 2024-01-11T07:18:19.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.696+0000 7efc00cb9700 1 Processor -- start 2024-01-11T07:18:19.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.696+0000 7efc00cb9700 1 -- start start 2024-01-11T07:18:19.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efc00cb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7b30 0x7efbfc16ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efc00cb9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbfc0c7f00 0x7efbfc0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efc00cb9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efbfc153ec0 0x7efbfc155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efc00cb9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc16b070 con 0x7efbfc0c7b30 2024-01-11T07:18:19.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efc00cb9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc16b5c0 con 0x7efbfc0c7f00 2024-01-11T07:18:19.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efc00cb9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc16bf90 con 0x7efbfc153ec0 2024-01-11T07:18:19.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efbfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7b30 0x7efbfc16ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efbfa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7b30 0x7efbfc16ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57282/0 (socket says 172.21.15.7:57282) 2024-01-11T07:18:19.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.697+0000 7efbfa59c700 1 -- 172.21.15.7:0/531150670 learned_addr learned my addr 172.21.15.7:0/531150670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:19.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbfad9d700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efbfc153ec0 0x7efbfc155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbfc0c7f00 0x7efbfc0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbfa59c700 1 -- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efbfc153ec0 msgr2=0x7efbfc155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbfa59c700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efbfc153ec0 0x7efbfc155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbfa59c700 1 -- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbfc0c7f00 msgr2=0x7efbfc0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbfa59c700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbfc0c7f00 0x7efbfc0b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbfa59c700 1 -- 172.21.15.7:0/531150670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf400ab30 con 0x7efbfc0c7b30 2024-01-11T07:18:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbfad9d700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efbfc153ec0 0x7efbfc155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbfa59c700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7b30 0x7efbfc16ba50 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7efbec00c590 tx=0x7efbec00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbea7fc700 1 -- 172.21.15.7:0/531150670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec011590 con 0x7efbfc0c7b30 2024-01-11T07:18:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbea7fc700 1 -- 172.21.15.7:0/531150670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbec011710 con 0x7efbfc0c7b30 2024-01-11T07:18:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc0affc0 con 0x7efbfc0c7b30 2024-01-11T07:18:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efbea7fc700 1 -- 172.21.15.7:0/531150670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec011ad0 con 0x7efbfc0c7b30 2024-01-11T07:18:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.698+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc16b250 con 0x7efbfc0c7b30 2024-01-11T07:18:19.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.699+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbfc04c3c0 con 0x7efbfc0c7b30 2024-01-11T07:18:19.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.700+0000 7efbea7fc700 1 -- 172.21.15.7:0/531150670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7efbec01b3b0 con 0x7efbfc0c7b30 2024-01-11T07:18:19.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.700+0000 7efbea7fc700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efbdc066f30 0x7efbdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:19.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.700+0000 7efbea7fc700 1 -- 172.21.15.7:0/531150670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efbec083a80 con 0x7efbfc0c7b30 2024-01-11T07:18:19.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.701+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efbdc066f30 0x7efbdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:19.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.701+0000 7efbf9d9b700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efbdc066f30 0x7efbdc0693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0c69f0 tx=0x7efbf0007f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:19.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.705+0000 7efbea7fc700 1 -- 172.21.15.7:0/531150670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efbec050c50 con 0x7efbfc0c7b30 2024-01-11T07:18:19.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.957+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efbfc0cc310 con 0x7efbfc0c7b30 2024-01-11T07:18:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.957+0000 7efbea7fc700 1 -- 172.21.15.7:0/531150670 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7efbec017020 con 0x7efbfc0c7b30 2024-01-11T07:18:19.958 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efbdc066f30 msgr2=0x7efbdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efbdc066f30 0x7efbdc0693e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0c69f0 tx=0x7efbf0007f30 comp rx=0 tx=0).stop 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7b30 msgr2=0x7efbfc16ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7b30 0x7efbfc16ba50 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7efbec00c590 tx=0x7efbec00c8a0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 shutdown_connections 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7efbdc066f30 0x7efbdc0693e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efbfc153ec0 0x7efbfc155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efbfc0c7b30 0x7efbfc16ba50 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 --2- 172.21.15.7:0/531150670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbfc0c7f00 0x7efbfc0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.959+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 >> 172.21.15.7:0/531150670 conn(0x7efbfc0cd090 msgr2=0x7efbfc153610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.960+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 shutdown_connections 2024-01-11T07:18:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:19.960+0000 7efc00cb9700 1 -- 172.21.15.7:0/531150670 wait complete. 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:18:19.974 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:18:19.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:19.975 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-01-11T07:18:19.994 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:18:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:18:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:18:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-01-11T07:18:19.994 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:19.994 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41718sec preferred_lft 41718sec 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:19.995 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::c42d:a6ff:fe5b:4c78/64 scope link 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 8a:e1:77:13:fe:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::88e1:77ff:fe13:fea5/64 scope link 2024-01-11T07:18:19.996 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:19.997 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:18:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 12:47:ad:a4:27:7c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:18:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::1047:adff:fea4:277c/64 scope link 2024-01-11T07:18:19.997 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:19.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:19.997 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:18:20.065 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-01-11T07:18:20.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:20.067 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-01-11T07:18:20.130 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-01-11T07:18:20.130 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:18:20.131 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-11T07:18:20.131 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-11T07:18:20.131 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:18:20.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:20.131 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-01-11T07:18:20.131 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-01-11T07:18:20.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-11T07:18:20.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-11T07:18:20.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-11T07:18:20.131 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-01-11T07:18:20.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:20.255 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:18:20.275 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:18:20.275 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:18:20.275 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:18:20.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:20.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-01-11T07:18:20.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:18:20.351 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-11T07:18:20.413 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-11T07:18:20.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:18:20.414 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:18:20.475 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:18:20.476 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:18:20.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:18:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:18:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:18:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:18:20.580 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:18:20.580 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:18:20.580 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:18:20.580 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:18:20.581 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:18:20.598 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-11T07:18:20.599 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-11T07:18:20.599 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-01-11T07:18:20.599 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:18:20.599 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:18:20.599 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:18:20.599 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:18:20.599 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:18:20.601 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:18:20.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:20.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-11T07:18:20.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-11T07:18:20.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.813+0000 7f59be84b700 1 Processor -- start 2024-01-11T07:18:20.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.813+0000 7f59be84b700 1 -- start start 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.814+0000 7f59be84b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b8159f70 0x7f59b80c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.814+0000 7f59be84b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b8067c60 con 0x7f59b80c9570 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.814+0000 7f59be84b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b8069230 con 0x7f59b8159ba0 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.814+0000 7f59be84b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b806d730 con 0x7f59b8159f70 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.814+0000 7f59b77fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b8159f70 0x7f59b80c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.814+0000 7f59bcde8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f59b80c9570 0x7f59b80680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43566/0 (socket says 172.21.15.7:43566) 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.814+0000 7f59bcde8700 1 -- 172.21.15.7:0/2551141104 learned_addr learned my addr 172.21.15.7:0/2551141104 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b6ffd700 1 -- 172.21.15.7:0/2551141104 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900818463 0 0) 0x7f59b8069230 con 0x7f59b8159ba0 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b6ffd700 1 -- 172.21.15.7:0/2551141104 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59a0003570 con 0x7f59b8159ba0 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b6ffd700 1 -- 172.21.15.7:0/2551141104 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267187810 0 0) 0x7f59b8067c60 con 0x7f59b80c9570 2024-01-11T07:18:20.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b6ffd700 1 -- 172.21.15.7:0/2551141104 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b8069230 con 0x7f59b80c9570 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b77fe700 1 -- 172.21.15.7:0/2551141104 >> v1:172.21.15.113:6789/0 conn(0x7f59b8159ba0 legacy=0x7f59b806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b77fe700 1 -- 172.21.15.7:0/2551141104 >> v1:172.21.15.7:6789/0 conn(0x7f59b80c9570 legacy=0x7f59b80680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b77fe700 1 -- 172.21.15.7:0/2551141104 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b8069510 con 0x7f59b8159f70 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b77fe700 1 --2- 172.21.15.7:0/2551141104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b8159f70 0x7f59b80c9030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009910 tx=0x7f59ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f26aa74c26740ac3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b6ffd700 1 -- 172.21.15.7:0/2551141104 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac011630 con 0x7f59b8159f70 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b6ffd700 1 -- 172.21.15.7:0/2551141104 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac011790 con 0x7f59b8159f70 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.815+0000 7f59b6ffd700 1 -- 172.21.15.7:0/2551141104 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac011b30 con 0x7f59b8159f70 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.816+0000 7f59be84b700 1 -- 172.21.15.7:0/2551141104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b8159f70 msgr2=0x7f59b80c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.816+0000 7f59be84b700 1 --2- 172.21.15.7:0/2551141104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b8159f70 0x7f59b80c9030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009910 tx=0x7f59ac0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.816+0000 7f59be84b700 1 -- 172.21.15.7:0/2551141104 shutdown_connections 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.816+0000 7f59be84b700 1 --2- 172.21.15.7:0/2551141104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b8159f70 0x7f59b80c9030 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.816+0000 7f59be84b700 1 -- 172.21.15.7:0/2551141104 >> 172.21.15.7:0/2551141104 conn(0x7f59b806ae00 msgr2=0x7f59b806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.816+0000 7f59be84b700 1 -- 172.21.15.7:0/2551141104 shutdown_connections 2024-01-11T07:18:20.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.816+0000 7f59be84b700 1 -- 172.21.15.7:0/2551141104 wait complete. 2024-01-11T07:18:20.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.817+0000 7f59be84b700 1 Processor -- start 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.818+0000 7f59be84b700 1 -- start start 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.818+0000 7f59be84b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b80c9570 0x7f59b81553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.818+0000 7f59be84b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8159ba0 0x7f59b8158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.818+0000 7f59be84b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59b8159f70 0x7f59b8158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.818+0000 7f59be84b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80b2f80 con 0x7f59b8159ba0 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.818+0000 7f59be84b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b815b820 con 0x7f59b8159f70 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.818+0000 7f59be84b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b815bb00 con 0x7f59b80c9570 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.818+0000 7f59b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8159ba0 0x7f59b8158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8159ba0 0x7f59b8158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57298/0 (socket says 172.21.15.7:57298) 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b77fe700 1 -- 172.21.15.7:0/112938760 learned_addr learned my addr 172.21.15.7:0/112938760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59bcde8700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59b8159f70 0x7f59b8158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b7fff700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b80c9570 0x7f59b81553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b77fe700 1 -- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b80c9570 msgr2=0x7f59b81553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:20.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b77fe700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b80c9570 0x7f59b81553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b77fe700 1 -- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59b8159f70 msgr2=0x7f59b8158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b77fe700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59b8159f70 0x7f59b8158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b77fe700 1 -- 172.21.15.7:0/112938760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac009420 con 0x7f59b8159ba0 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59b77fe700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8159ba0 0x7f59b8158600 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f59b806e9f0 tx=0x7f59ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f599ffff700 1 -- 172.21.15.7:0/112938760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac01f070 con 0x7f59b8159ba0 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f599ffff700 1 -- 172.21.15.7:0/112938760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac000c90 con 0x7f59b8159ba0 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b815bde0 con 0x7f59b8159ba0 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f599ffff700 1 -- 172.21.15.7:0/112938760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac025420 con 0x7f59b8159ba0 2024-01-11T07:18:20.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.819+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b8152650 con 0x7f59b8159ba0 2024-01-11T07:18:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.820+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b815c430 con 0x7f59b8159ba0 2024-01-11T07:18:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.821+0000 7f599ffff700 1 -- 172.21.15.7:0/112938760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f59ac023070 con 0x7f59b8159ba0 2024-01-11T07:18:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.821+0000 7f599ffff700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f59a4066f30 0x7f59a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.821+0000 7f599ffff700 1 -- 172.21.15.7:0/112938760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f59ac0858c0 con 0x7f59b8159ba0 2024-01-11T07:18:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.822+0000 7f59b7fff700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f59a4066f30 0x7f59a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:20.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.822+0000 7f59b7fff700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f59a4066f30 0x7f59a40693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f59a8003870 tx=0x7f59a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:20.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:20.826+0000 7f599ffff700 1 -- 172.21.15.7:0/112938760 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59ac052be0 con 0x7f59b8159ba0 2024-01-11T07:18:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.083+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59b804c3c0 con 0x7f59b8159ba0 2024-01-11T07:18:21.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.083+0000 7f599ffff700 1 -- 172.21.15.7:0/112938760 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59ac02a070 con 0x7f59b8159ba0 2024-01-11T07:18:21.090 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-11T07:18:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f59a4066f30 msgr2=0x7f59a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f59a4066f30 0x7f59a40693e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f59a8003870 tx=0x7f59a8009040 comp rx=0 tx=0).stop 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8159ba0 msgr2=0x7f59b8158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8159ba0 0x7f59b8158600 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f59b806e9f0 tx=0x7f59ac017000 comp rx=0 tx=0).stop 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 shutdown_connections 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f59a4066f30 0x7f59a40693e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59b8159f70 0x7f59b8158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59b80c9570 0x7f59b81553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 --2- 172.21.15.7:0/112938760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b8159ba0 0x7f59b8158600 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 >> 172.21.15.7:0/112938760 conn(0x7f59b806ae00 msgr2=0x7f59b80c4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 shutdown_connections 2024-01-11T07:18:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.086+0000 7f59be84b700 1 -- 172.21.15.7:0/112938760 wait complete. 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-11T07:18:21.100 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:18:21.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:21.101 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-01-11T07:18:21.122 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-11T07:18:21.122 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-11T07:18:21.123 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 41760sec preferred_lft 41760sec 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::8b5:231c:67c0:8514/64 scope link noprefixroute 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::f880:dff:fede:e0d4/64 scope link 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:21.124 INFO:teuthology.orchestra.run.smithi113.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:18:21.125 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether ae:bd:3d:cd:f3:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:18:21.125 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::acbd:3dff:fecd:f397/64 scope link 2024-01-11T07:18:21.125 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:21.125 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-11T07:18:21.125 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether aa:c4:4b:15:bd:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:18:21.125 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a8c4:4bff:fe15:bdcf/64 scope link 2024-01-11T07:18:21.125 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-01-11T07:18:21.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:21.125 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-11T07:18:21.192 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-01-11T07:18:21.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:21.194 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-01-11T07:18:21.256 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-01-11T07:18:21.256 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-11T07:18:21.256 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-11T07:18:21.256 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-11T07:18:21.256 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-11T07:18:21.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:21.256 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-01-11T07:18:21.257 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-01-11T07:18:21.257 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-11T07:18:21.257 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-11T07:18:21.257 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-11T07:18:21.257 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-01-11T07:18:21.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:21.366 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:18:21.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:18:21.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:18:21.383 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:18:21.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:21.383 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-01-11T07:18:21.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:18:21.452 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-11T07:18:21.510 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-11T07:18:21.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-11T07:18:21.510 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:18:21.567 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-11T07:18:21.567 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-01-11T07:18:21.567 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:18:21.649 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:18:21.649 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:18:21.649 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:18:21.658 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:18:21.658 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:18:21.658 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:18:21.658 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:18:21.658 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:18:21.676 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-11T07:18:21.676 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-11T07:18:21.676 INFO:teuthology.orchestra.run.smithi113.stdout:invalid new device string format 2024-01-11T07:18:21.676 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.113:3300,172.21.15.113:3301" 2024-01-11T07:18:21.676 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-11T07:18:21.676 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790" 2024-01-11T07:18:21.676 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:6789,172.21.15.113:6789,172.21.15.113:6790:/ 2024-01-11T07:18:21.676 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9c0c5e15-bc38-4016-89e4-891c3586efbb" will pass to kernel 2024-01-11T07:18:21.677 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-11T07:18:21.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:21.678 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-11T07:18:21.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:21.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.887+0000 7fc7395da700 1 Processor -- start 2024-01-11T07:18:21.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.887+0000 7fc7395da700 1 -- start start 2024-01-11T07:18:21.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.887+0000 7fc7395da700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc734159d10 0x7fc734068c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:21.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc7395da700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7340691d0 0x7fc73406ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:21.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc7395da700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73406f610 con 0x7fc73406f1d0 2024-01-11T07:18:21.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc7395da700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73415d9a0 con 0x7fc734159d10 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc7395da700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73415dbb0 con 0x7fc7340691d0 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc732ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc734159d10 0x7fc734068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc732ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc734159d10 0x7fc734068c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57312/0 (socket says 172.21.15.7:57312) 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc732ffd700 1 -- 172.21.15.7:0/205221705 learned_addr learned my addr 172.21.15.7:0/205221705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc7327fc700 1 --2- 172.21.15.7:0/205221705 >> v2:172.21.15.113:3300/0 conn(0x7fc7340691d0 0x7fc73406ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc731ffb700 1 -- 172.21.15.7:0/205221705 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365662174 0 0) 0x7fc73406f610 con 0x7fc73406f1d0 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc732ffd700 1 -- 172.21.15.7:0/205221705 >> v2:172.21.15.113:3300/0 conn(0x7fc7340691d0 msgr2=0x7fc73406ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc732ffd700 1 --2- 172.21.15.7:0/205221705 >> v2:172.21.15.113:3300/0 conn(0x7fc7340691d0 0x7fc73406ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc732ffd700 1 -- 172.21.15.7:0/205221705 >> v1:172.21.15.113:6789/0 conn(0x7fc73406f1d0 legacy=0x7fc7340b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc732ffd700 1 -- 172.21.15.7:0/205221705 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73415de90 con 0x7fc734159d10 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.888+0000 7fc732ffd700 1 --2- 172.21.15.7:0/205221705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc734159d10 0x7fc734068c90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc7240099b0 tx=0x7fc724009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d473493567ccf460 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc731ffb700 1 -- 172.21.15.7:0/205221705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724009ea0 con 0x7fc734159d10 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc731ffb700 1 -- 172.21.15.7:0/205221705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72400b820 con 0x7fc734159d10 2024-01-11T07:18:21.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc731ffb700 1 -- 172.21.15.7:0/205221705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72400bbc0 con 0x7fc734159d10 2024-01-11T07:18:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc7395da700 1 -- 172.21.15.7:0/205221705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc734159d10 msgr2=0x7fc734068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc7395da700 1 --2- 172.21.15.7:0/205221705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc734159d10 0x7fc734068c90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc7240099b0 tx=0x7fc724009250 comp rx=0 tx=0).stop 2024-01-11T07:18:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc7395da700 1 -- 172.21.15.7:0/205221705 shutdown_connections 2024-01-11T07:18:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc7395da700 1 --2- 172.21.15.7:0/205221705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc734159d10 0x7fc734068c90 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc7395da700 1 --2- 172.21.15.7:0/205221705 >> v2:172.21.15.113:3300/0 conn(0x7fc7340691d0 0x7fc73406ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc7395da700 1 -- 172.21.15.7:0/205221705 >> 172.21.15.7:0/205221705 conn(0x7fc7340ce9f0 msgr2=0x7fc7340cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc7395da700 1 -- 172.21.15.7:0/205221705 shutdown_connections 2024-01-11T07:18:21.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.889+0000 7fc7395da700 1 -- 172.21.15.7:0/205221705 wait complete. 2024-01-11T07:18:21.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7395da700 1 Processor -- start 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7395da700 1 -- start start 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7395da700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7340691d0 0x7fc734157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7395da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc73406f1d0 0x7fc73415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7395da700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc734159d10 0x7fc7340b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7395da700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340c5e70 con 0x7fc73406f1d0 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7395da700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340c6150 con 0x7fc734159d10 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7395da700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340c6430 con 0x7fc7340691d0 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.891+0000 7fc7327fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc73406f1d0 0x7fc73415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7327fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc73406f1d0 0x7fc73415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57328/0 (socket says 172.21.15.7:57328) 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7327fc700 1 -- 172.21.15.7:0/1562275379 learned_addr learned my addr 172.21.15.7:0/1562275379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc732ffd700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7340691d0 0x7fc734157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7337fe700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc734159d10 0x7fc7340b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7327fc700 1 -- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7340691d0 msgr2=0x7fc734157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7327fc700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7340691d0 0x7fc734157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7327fc700 1 -- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc734159d10 msgr2=0x7fc7340b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7327fc700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc734159d10 0x7fc7340b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7327fc700 1 -- 172.21.15.7:0/1562275379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240094f0 con 0x7fc73406f1d0 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7327fc700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc73406f1d0 0x7fc73415baa0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00ec80 tx=0x7fc71c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc72affd700 1 -- 172.21.15.7:0/1562275379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00c9d0 con 0x7fc73406f1d0 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc72affd700 1 -- 172.21.15.7:0/1562275379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c00cb30 con 0x7fc73406f1d0 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc72affd700 1 -- 172.21.15.7:0/1562275379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c013470 con 0x7fc73406f1d0 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7340c6710 con 0x7fc73406f1d0 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.892+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc734158330 con 0x7fc73406f1d0 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.893+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73416a3c0 con 0x7fc73406f1d0 2024-01-11T07:18:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.894+0000 7fc72affd700 1 -- 172.21.15.7:0/1562275379 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc71c0138a0 con 0x7fc73406f1d0 2024-01-11T07:18:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.894+0000 7fc72affd700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc714066f30 0x7fc7140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.895+0000 7fc72affd700 1 -- 172.21.15.7:0/1562275379 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc71c084870 con 0x7fc73406f1d0 2024-01-11T07:18:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.895+0000 7fc732ffd700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc714066f30 0x7fc7140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.895+0000 7fc732ffd700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc714066f30 0x7fc7140693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc7240099b0 tx=0x7fc724017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:21.899+0000 7fc72affd700 1 -- 172.21.15.7:0/1562275379 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc71c051ae0 con 0x7fc73406f1d0 2024-01-11T07:18:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.155+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc734069770 con 0x7fc73406f1d0 2024-01-11T07:18:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.155+0000 7fc72affd700 1 -- 172.21.15.7:0/1562275379 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fc71c055100 con 0x7fc73406f1d0 2024-01-11T07:18:22.156 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:22.156 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:22.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc714066f30 msgr2=0x7fc7140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc714066f30 0x7fc7140693e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc7240099b0 tx=0x7fc724017000 comp rx=0 tx=0).stop 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc73406f1d0 msgr2=0x7fc73415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc73406f1d0 0x7fc73415baa0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00ec80 tx=0x7fc71c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 shutdown_connections 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc714066f30 0x7fc7140693e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc734159d10 0x7fc7340b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc7340691d0 0x7fc734157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 --2- 172.21.15.7:0/1562275379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc73406f1d0 0x7fc73415baa0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 >> 172.21.15.7:0/1562275379 conn(0x7fc7340ce9f0 msgr2=0x7fc7340cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.157+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 shutdown_connections 2024-01-11T07:18:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.158+0000 7fc7395da700 1 -- 172.21.15.7:0/1562275379 wait complete. 2024-01-11T07:18:22.160 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:22.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-11T07:18:22.329 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":8064,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.8954740920000006,"mdsmap_epoch":90,"osdmap_epoch":184,"osdmap_epoch_barrier":184,"uptime":14.910027935} 2024-01-11T07:18:22.330 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 8064, "mdsmap_epoch": 90, "osdmap_epoch": 184, "osdmap_epoch_barrier": 184, "rank_uptime": 8.895474092, "state": "up:active", "uptime": 14.910027935, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:18:22.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-01-11T07:18:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.464+0000 7fb23690e700 1 Processor -- start 2024-01-11T07:18:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.464+0000 7fb23690e700 1 -- start start 2024-01-11T07:18:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.465+0000 7fb23690e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb2300b39e0 0x7fb23015a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.465+0000 7fb23690e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300b5d60 con 0x7fb23015ab50 2024-01-11T07:18:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.465+0000 7fb23690e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300b5150 con 0x7fb230143770 2024-01-11T07:18:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.465+0000 7fb23690e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2300b5430 con 0x7fb2300b39e0 2024-01-11T07:18:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.465+0000 7fb234eab700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb23015ab50 0x7fb2300b61f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43576/0 (socket says 172.21.15.7:43576) 2024-01-11T07:18:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.465+0000 7fb234eab700 1 -- 172.21.15.7:0/942918967 learned_addr learned my addr 172.21.15.7:0/942918967 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.466+0000 7fb22f7fe700 1 --2- 172.21.15.7:0/942918967 >> v2:172.21.15.113:3300/0 conn(0x7fb2300b39e0 0x7fb23015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.466+0000 7fb22f7fe700 1 -- 172.21.15.7:0/942918967 >> v1:172.21.15.113:6789/0 conn(0x7fb230143770 legacy=0x7fb2300b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.466+0000 7fb22f7fe700 1 -- 172.21.15.7:0/942918967 >> v1:172.21.15.7:6789/0 conn(0x7fb23015ab50 legacy=0x7fb2300b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.466+0000 7fb22f7fe700 1 -- 172.21.15.7:0/942918967 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300b5710 con 0x7fb2300b39e0 2024-01-11T07:18:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.466+0000 7fb22f7fe700 1 --2- 172.21.15.7:0/942918967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb23015a610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb2240098e0 tx=0x7fb2240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f30f033af1248b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.467+0000 7fb22effd700 1 -- 172.21.15.7:0/942918967 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224009e80 con 0x7fb2300b39e0 2024-01-11T07:18:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.467+0000 7fb22effd700 1 -- 172.21.15.7:0/942918967 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22400b860 con 0x7fb2300b39e0 2024-01-11T07:18:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.467+0000 7fb22effd700 1 -- 172.21.15.7:0/942918967 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22400bbe0 con 0x7fb2300b39e0 2024-01-11T07:18:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.468+0000 7fb23690e700 1 -- 172.21.15.7:0/942918967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 msgr2=0x7fb23015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.468+0000 7fb23690e700 1 --2- 172.21.15.7:0/942918967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb23015a610 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb2240098e0 tx=0x7fb2240091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.468+0000 7fb23690e700 1 -- 172.21.15.7:0/942918967 shutdown_connections 2024-01-11T07:18:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.468+0000 7fb23690e700 1 --2- 172.21.15.7:0/942918967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb23015a610 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.468+0000 7fb23690e700 1 -- 172.21.15.7:0/942918967 >> 172.21.15.7:0/942918967 conn(0x7fb230063cc0 msgr2=0x7fb2300b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.468+0000 7fb23690e700 1 -- 172.21.15.7:0/942918967 shutdown_connections 2024-01-11T07:18:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.468+0000 7fb23690e700 1 -- 172.21.15.7:0/942918967 wait complete. 2024-01-11T07:18:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.469+0000 7fb23690e700 1 Processor -- start 2024-01-11T07:18:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.469+0000 7fb23690e700 1 -- start start 2024-01-11T07:18:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb23690e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb2300cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb23690e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb230143770 0x7fb23016ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb23690e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb23015ab50 0x7fb2300b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb23690e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c79e0 con 0x7fb23015ab50 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb23690e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c7dd0 con 0x7fb2300b39e0 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb23690e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23016b170 con 0x7fb230143770 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb234eab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb23015ab50 0x7fb2300b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb2300cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb22f7fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb230143770 0x7fb23016ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb22ffff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb2300cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:41674/0 (socket says 172.21.15.7:41674) 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.470+0000 7fb22ffff700 1 -- 172.21.15.7:0/2760307611 learned_addr learned my addr 172.21.15.7:0/2760307611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb22ffff700 1 -- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb230143770 msgr2=0x7fb23016ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb22ffff700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb230143770 0x7fb23016ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb22ffff700 1 -- 172.21.15.7:0/2760307611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb23015ab50 msgr2=0x7fb2300b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb22ffff700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb23015ab50 0x7fb2300b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb22ffff700 1 -- 172.21.15.7:0/2760307611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb224009420 con 0x7fb2300b39e0 2024-01-11T07:18:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb22f7fe700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb230143770 0x7fb23016ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb22ffff700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb2300cd4f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb22000c7c0 tx=0x7fb220007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb217fff700 1 -- 172.21.15.7:0/2760307611 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220011590 con 0x7fb2300b39e0 2024-01-11T07:18:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb217fff700 1 -- 172.21.15.7:0/2760307611 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2200116f0 con 0x7fb2300b39e0 2024-01-11T07:18:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300c66e0 con 0x7fb2300b39e0 2024-01-11T07:18:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb217fff700 1 -- 172.21.15.7:0/2760307611 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220011ab0 con 0x7fb2300b39e0 2024-01-11T07:18:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.471+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300c8300 con 0x7fb2300b39e0 2024-01-11T07:18:22.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.472+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2300890f0 con 0x7fb2300b39e0 2024-01-11T07:18:22.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.472+0000 7fb217fff700 1 -- 172.21.15.7:0/2760307611 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb22001b3b0 con 0x7fb2300b39e0 2024-01-11T07:18:22.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.473+0000 7fb217fff700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:22.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.473+0000 7fb217fff700 1 -- 172.21.15.7:0/2760307611 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb220083950 con 0x7fb2300b39e0 2024-01-11T07:18:22.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.473+0000 7fb22f7fe700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:22.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.473+0000 7fb22f7fe700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb21c066f30 0x7fb21c0693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb2240093f0 tx=0x7fb224017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.477+0000 7fb217fff700 1 -- 172.21.15.7:0/2760307611 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb220050bd0 con 0x7fb2300b39e0 2024-01-11T07:18:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:22.684+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fb23004c3c0 con 0x7fb2300b39e0 2024-01-11T07:18:23.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.472+0000 7fb217fff700 1 -- 172.21.15.7:0/2760307611 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v185) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb220017070 con 0x7fb2300b39e0 2024-01-11T07:18:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb21c066f30 msgr2=0x7fb21c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb21c066f30 0x7fb21c0693e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb2240093f0 tx=0x7fb224017000 comp rx=0 tx=0).stop 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 msgr2=0x7fb2300cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb2300cd4f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb22000c7c0 tx=0x7fb220007f70 comp rx=0 tx=0).stop 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 shutdown_connections 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb21c066f30 0x7fb21c0693e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb23015ab50 0x7fb2300b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2300b39e0 0x7fb2300cd4f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 --2- 172.21.15.7:0/2760307611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb230143770 0x7fb23016ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 >> 172.21.15.7:0/2760307611 conn(0x7fb230063cc0 msgr2=0x7fb230141590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 shutdown_connections 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:23.475+0000 7fb23690e700 1 -- 172.21.15.7:0/2760307611 wait complete. 2024-01-11T07:18:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-01-11T07:18:23.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-01-11T07:18:23.644 INFO:teuthology.orchestra.run.smithi007.stdout:{"osd_mon_report_interval":"5"} 2024-01-11T07:18:23.644 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-01-11T07:18:23.644 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-01-11T07:18:23.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:18:23.645 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-01-11T07:18:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:16+0 records in 2024-01-11T07:18:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:16+0 records out 2024-01-11T07:18:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.100175 s, 167 MB/s 2024-01-11T07:18:23.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:18:23.769 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-01-11T07:18:23.890 INFO:teuthology.orchestra.run.smithi007.stderr:17+0 records in 2024-01-11T07:18:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:17+0 records out 2024-01-11T07:18:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.100449 s, 177 MB/s 2024-01-11T07:18:23.891 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-01-11T07:18:23.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:18:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.022+0000 7f2e8a93f700 1 Processor -- start 2024-01-11T07:18:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.022+0000 7f2e8a93f700 1 -- start start 2024-01-11T07:18:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.023+0000 7f2e8a93f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e84158280 0x7f2e840b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.023+0000 7f2e8a93f700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e8415aa80 0x7f2e840b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.023+0000 7f2e8a93f700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e84159300 con 0x7f2e840cafb0 2024-01-11T07:18:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.023+0000 7f2e8a93f700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e840cf000 con 0x7f2e8415aa80 2024-01-11T07:18:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.023+0000 7f2e8a93f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840cf260 con 0x7f2e84158280 2024-01-11T07:18:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e837fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e84158280 0x7f2e840b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e88edc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e8415aa80 0x7f2e840b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e83fff700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2e840cafb0 0x7f2e840ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:46760/0 (socket says 172.21.15.7:46760) 2024-01-11T07:18:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e83fff700 1 -- 172.21.15.7:0/1655516432 learned_addr learned my addr 172.21.15.7:0/1655516432 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e837fe700 1 -- 172.21.15.7:0/1655516432 >> v2:172.21.15.113:3300/0 conn(0x7f2e8415aa80 msgr2=0x7f2e840b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e837fe700 1 --2- 172.21.15.7:0/1655516432 >> v2:172.21.15.113:3300/0 conn(0x7f2e8415aa80 0x7f2e840b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e837fe700 1 -- 172.21.15.7:0/1655516432 >> v1:172.21.15.113:6789/0 conn(0x7f2e840cafb0 legacy=0x7f2e840ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e837fe700 1 -- 172.21.15.7:0/1655516432 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840cf540 con 0x7f2e84158280 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e88edc700 1 --2- 172.21.15.7:0/1655516432 >> v2:172.21.15.113:3300/0 conn(0x7f2e8415aa80 0x7f2e840b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.024+0000 7f2e837fe700 1 --2- 172.21.15.7:0/1655516432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e84158280 0x7f2e840b0430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0098e0 tx=0x7f2e6c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d9547a530f2c0d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e82ffd700 1 -- 172.21.15.7:0/1655516432 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00b640 con 0x7f2e84158280 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e82ffd700 1 -- 172.21.15.7:0/1655516432 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c00b7a0 con 0x7f2e84158280 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e82ffd700 1 -- 172.21.15.7:0/1655516432 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00bb40 con 0x7f2e84158280 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1655516432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e84158280 msgr2=0x7f2e840b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1655516432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e84158280 0x7f2e840b0430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0098e0 tx=0x7f2e6c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1655516432 shutdown_connections 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1655516432 >> v2:172.21.15.113:3300/0 conn(0x7f2e8415aa80 0x7f2e840b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1655516432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e84158280 0x7f2e840b0430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1655516432 >> 172.21.15.7:0/1655516432 conn(0x7f2e84071100 msgr2=0x7f2e8406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1655516432 shutdown_connections 2024-01-11T07:18:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.025+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1655516432 wait complete. 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.026+0000 7f2e8a93f700 1 Processor -- start 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.027+0000 7f2e8a93f700 1 -- start start 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.027+0000 7f2e8a93f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840cafb0 0x7f2e8416e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.027+0000 7f2e8a93f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e84158280 0x7f2e840cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.027+0000 7f2e8a93f700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8415aa80 0x7f2e840c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.027+0000 7f2e8a93f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8416dce0 con 0x7f2e84158280 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.027+0000 7f2e8a93f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840b06f0 con 0x7f2e840cafb0 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.027+0000 7f2e8a93f700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8416df80 con 0x7f2e8415aa80 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e84158280 0x7f2e840cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e84158280 0x7f2e840cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57346/0 (socket says 172.21.15.7:57346) 2024-01-11T07:18:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 -- 172.21.15.7:0/1527493451 learned_addr learned my addr 172.21.15.7:0/1527493451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e88edc700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8415aa80 0x7f2e840c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e83fff700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840cafb0 0x7f2e8416e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 -- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8415aa80 msgr2=0x7f2e840c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8415aa80 0x7f2e840c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 -- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840cafb0 msgr2=0x7f2e8416e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840cafb0 0x7f2e8416e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 -- 172.21.15.7:0/1527493451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c009a20 con 0x7f2e84158280 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e837fe700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e84158280 0x7f2e840cd480 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c013000 tx=0x7f2e6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e7bfff700 1 -- 172.21.15.7:0/1527493451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c01bcf0 con 0x7f2e84158280 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e7bfff700 1 -- 172.21.15.7:0/1527493451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c01be70 con 0x7f2e84158280 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c009420 con 0x7f2e84158280 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e7bfff700 1 -- 172.21.15.7:0/1527493451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c011650 con 0x7f2e84158280 2024-01-11T07:18:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.028+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e840b09d0 con 0x7f2e84158280 2024-01-11T07:18:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.030+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8404c3c0 con 0x7f2e84158280 2024-01-11T07:18:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.030+0000 7f2e7bfff700 1 -- 172.21.15.7:0/1527493451 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f2e6c011a00 con 0x7f2e84158280 2024-01-11T07:18:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.030+0000 7f2e7bfff700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e64066f30 0x7f2e640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.031+0000 7f2e7bfff700 1 -- 172.21.15.7:0/1527493451 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2e6c085ae0 con 0x7f2e84158280 2024-01-11T07:18:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.031+0000 7f2e83fff700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e64066f30 0x7f2e640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.031+0000 7f2e83fff700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e64066f30 0x7f2e640693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2e74007130 tx=0x7f2e74007eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.035+0000 7f2e7bfff700 1 -- 172.21.15.7:0/1527493451 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e6c052d70 con 0x7f2e84158280 2024-01-11T07:18:24.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.240+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2e8416e4a0 con 0x7f2e84158280 2024-01-11T07:18:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.241+0000 7f2e7bfff700 1 -- 172.21.15.7:0/1527493451 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) v1 ==== 81+0+32234 (secure 0 0 0) 0x7f2e8416e4a0 con 0x7f2e84158280 2024-01-11T07:18:24.242 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:24.242 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:24.242 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 185, 2024-01-11T07:18:24.242 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:18:24.242 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:18:23.469563+0000", 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:18:24.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:24.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:18:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:24.246 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:24.247 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:24.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:18:24.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.360758+0000", 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:24.250 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:24.251 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:24.252 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:24.253 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:24.254 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.823991+0000", 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:24.256 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "185", 2024-01-11T07:18:24.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:24.258 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:18:24.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:24.266 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:18:24.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:24.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:24.274 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:18:24.276 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:24.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:24.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:24.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:24.284 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.286 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:18:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:18:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:18:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:18:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:18:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:18:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:18:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:18:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:24.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e64066f30 msgr2=0x7f2e640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:24.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e64066f30 0x7f2e640693e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2e74007130 tx=0x7f2e74007eb0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e84158280 msgr2=0x7f2e840cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e84158280 0x7f2e840cd480 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c013000 tx=0x7f2e6c017000 comp rx=0 tx=0).stop 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 shutdown_connections 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f2e64066f30 0x7f2e640693e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2e8415aa80 0x7f2e840c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e840cafb0 0x7f2e8416e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 --2- 172.21.15.7:0/1527493451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e84158280 0x7f2e840cd480 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 >> 172.21.15.7:0/1527493451 conn(0x7f2e84071100 msgr2=0x7f2e840ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 shutdown_connections 2024-01-11T07:18:24.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:24.244+0000 7f2e8a93f700 1 -- 172.21.15.7:0/1527493451 wait complete. 2024-01-11T07:18:24.325 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:18:29.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:18:29.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.471+0000 7f6af3280700 1 Processor -- start 2024-01-11T07:18:29.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.471+0000 7f6af3280700 1 -- start start 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.472+0000 7f6af3280700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec1617c0 0x7f6aec0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.472+0000 7f6af3280700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec161b90 con 0x7f6aec0b1820 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.472+0000 7f6af3280700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec0b36c0 con 0x7f6aec0b0430 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.472+0000 7f6af3280700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0b3930 con 0x7f6aec1617c0 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.472+0000 7f6af081b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6aec0b1820 0x7f6aec0b4410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33216/0 (socket says 172.21.15.7:33216) 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.472+0000 7f6af081b700 1 -- 172.21.15.7:0/2055769665 learned_addr learned my addr 172.21.15.7:0/2055769665 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6af101c700 1 --2- 172.21.15.7:0/2055769665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec1617c0 0x7f6aec0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6adffff700 1 -- 172.21.15.7:0/2055769665 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626854435 0 0) 0x7f6aec161b90 con 0x7f6aec0b1820 2024-01-11T07:18:29.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6adffff700 1 -- 172.21.15.7:0/2055769665 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ad8003570 con 0x7f6aec0b1820 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6adffff700 1 -- 172.21.15.7:0/2055769665 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264145421 0 0) 0x7f6aec0b36c0 con 0x7f6aec0b0430 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6adffff700 1 -- 172.21.15.7:0/2055769665 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aec161b90 con 0x7f6aec0b0430 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6adffff700 1 -- 172.21.15.7:0/2055769665 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2895592173 0 0) 0x7f6ad8003570 con 0x7f6aec0b1820 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6af101c700 1 -- 172.21.15.7:0/2055769665 >> v1:172.21.15.113:6789/0 conn(0x7f6aec0b0430 legacy=0x7f6aec067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6af101c700 1 -- 172.21.15.7:0/2055769665 >> v1:172.21.15.7:6789/0 conn(0x7f6aec0b1820 legacy=0x7f6aec0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6af101c700 1 -- 172.21.15.7:0/2055769665 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec0b3c10 con 0x7f6aec1617c0 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6af101c700 1 --2- 172.21.15.7:0/2055769665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec1617c0 0x7f6aec0cf530 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009940 tx=0x7f6ae80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae05260ef00c671 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6adffff700 1 -- 172.21.15.7:0/2055769665 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0001470 con 0x7f6aec1617c0 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6adffff700 1 -- 172.21.15.7:0/2055769665 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae8011420 con 0x7f6aec1617c0 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.473+0000 7f6adffff700 1 -- 172.21.15.7:0/2055769665 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80117c0 con 0x7f6aec1617c0 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.474+0000 7f6af3280700 1 -- 172.21.15.7:0/2055769665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec1617c0 msgr2=0x7f6aec0cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.474+0000 7f6af3280700 1 --2- 172.21.15.7:0/2055769665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec1617c0 0x7f6aec0cf530 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009940 tx=0x7f6ae80091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:29.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.474+0000 7f6af3280700 1 -- 172.21.15.7:0/2055769665 shutdown_connections 2024-01-11T07:18:29.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.474+0000 7f6af3280700 1 --2- 172.21.15.7:0/2055769665 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec1617c0 0x7f6aec0cf530 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:29.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.474+0000 7f6af3280700 1 -- 172.21.15.7:0/2055769665 >> 172.21.15.7:0/2055769665 conn(0x7f6aec071100 msgr2=0x7f6aec06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:29.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.474+0000 7f6af3280700 1 -- 172.21.15.7:0/2055769665 shutdown_connections 2024-01-11T07:18:29.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.474+0000 7f6af3280700 1 -- 172.21.15.7:0/2055769665 wait complete. 2024-01-11T07:18:29.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.475+0000 7f6af3280700 1 Processor -- start 2024-01-11T07:18:29.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.475+0000 7f6af3280700 1 -- start start 2024-01-11T07:18:29.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.476+0000 7f6af3280700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec0b0430 0x7f6aec0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.476+0000 7f6af3280700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0b1820 0x7f6aec16eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.476+0000 7f6af3280700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aec1617c0 0x7f6aec0ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.476+0000 7f6af3280700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0cd7f0 con 0x7f6aec0b1820 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.476+0000 7f6af3280700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec16f0c0 con 0x7f6aec1617c0 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.476+0000 7f6af3280700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0a9290 con 0x7f6aec0b0430 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.476+0000 7f6af081b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0b1820 0x7f6aec16eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af081b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0b1820 0x7f6aec16eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60036/0 (socket says 172.21.15.7:60036) 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af081b700 1 -- 172.21.15.7:0/1127530496 learned_addr learned my addr 172.21.15.7:0/1127530496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af101c700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec0b0430 0x7f6aec0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af181d700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aec1617c0 0x7f6aec0ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af081b700 1 -- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec0b0430 msgr2=0x7f6aec0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af081b700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec0b0430 0x7f6aec0cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af081b700 1 -- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aec1617c0 msgr2=0x7f6aec0ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af081b700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aec1617c0 0x7f6aec0ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:29.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af081b700 1 -- 172.21.15.7:0/1127530496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8009420 con 0x7f6aec0b1820 2024-01-11T07:18:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af081b700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0b1820 0x7f6aec16eb80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000cb60 tx=0x7f6ae0001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6add7fa700 1 -- 172.21.15.7:0/1127530496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000ce70 con 0x7f6aec0b1820 2024-01-11T07:18:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6add7fa700 1 -- 172.21.15.7:0/1127530496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0007360 con 0x7f6aec0b1820 2024-01-11T07:18:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec0a9db0 con 0x7f6aec0b1820 2024-01-11T07:18:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6add7fa700 1 -- 172.21.15.7:0/1127530496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004830 con 0x7f6aec0b1820 2024-01-11T07:18:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.477+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec0aa370 con 0x7f6aec0b1820 2024-01-11T07:18:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.478+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec04c3c0 con 0x7f6aec0b1820 2024-01-11T07:18:29.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.479+0000 7f6add7fa700 1 -- 172.21.15.7:0/1127530496 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6ae0004be0 con 0x7f6aec0b1820 2024-01-11T07:18:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.479+0000 7f6add7fa700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ad0066f30 0x7f6ad00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.479+0000 7f6add7fa700 1 -- 172.21.15.7:0/1127530496 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ae0083c30 con 0x7f6aec0b1820 2024-01-11T07:18:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.479+0000 7f6af101c700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ad0066f30 0x7f6ad00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.480+0000 7f6af101c700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ad0066f30 0x7f6ad00693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009910 tx=0x7f6ae8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.484+0000 7f6add7fa700 1 -- 172.21.15.7:0/1127530496 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ae0050f20 con 0x7f6aec0b1820 2024-01-11T07:18:29.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.687+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6aec0ae100 con 0x7f6aec0b1820 2024-01-11T07:18:29.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.688+0000 7f6add7fa700 1 -- 172.21.15.7:0/1127530496 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) v1 ==== 81+0+32234 (secure 0 0 0) 0x7f6ae0018090 con 0x7f6aec0b1820 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 185, 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:18:23.469563+0000", 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:18:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:18:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:18:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:18:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:29.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:29.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:29.777 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.360758+0000", 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:29.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:18:29.779 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:29.780 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:29.781 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:29.782 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:29.783 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.823991+0000", 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:29.785 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "185", 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:29.786 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:29.787 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:29.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:18:29.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:29.790 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:18:29.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.792 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.793 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.794 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:29.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:29.796 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.798 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:29.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:18:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.808 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.809 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:29.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:18:29.811 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:29.812 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:18:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:18:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:18:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:29.820 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:29.826 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.827 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:29.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:29.831 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:18:29.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:29.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:29.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:29.838 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.839 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:29.840 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:18:29.842 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:18:29.843 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:18:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:18:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:18:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:18:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:18:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:18:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:18:29.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:29.844 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ad0066f30 msgr2=0x7f6ad00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ad0066f30 0x7f6ad00693e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009910 tx=0x7f6ae8016000 comp rx=0 tx=0).stop 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0b1820 msgr2=0x7f6aec16eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0b1820 0x7f6aec16eb80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000cb60 tx=0x7f6ae0001b80 comp rx=0 tx=0).stop 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 shutdown_connections 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ad0066f30 0x7f6ad00693e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6aec1617c0 0x7f6aec0ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6aec0b0430 0x7f6aec0cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 --2- 172.21.15.7:0/1127530496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6aec0b1820 0x7f6aec16eb80 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 >> 172.21.15.7:0/1127530496 conn(0x7f6aec071100 msgr2=0x7f6aec15cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 shutdown_connections 2024-01-11T07:18:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:29.691+0000 7f6af3280700 1 -- 172.21.15.7:0/1127530496 wait complete. 2024-01-11T07:18:29.847 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:18:32.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:18:32.067 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:18:34.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:18:34.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.989+0000 7fab03c28700 1 Processor -- start 2024-01-11T07:18:34.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.989+0000 7fab03c28700 1 -- start start 2024-01-11T07:18:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab03c28700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0a70 0x7faafc161da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab03c28700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faafc1622e0 0x7faafc063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab03c28700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faafc0c83f0 con 0x7faafc0c0e40 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab03c28700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faafc15e740 con 0x7faafc1622e0 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab03c28700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc162780 con 0x7faafc0c0a70 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab011c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faafc1622e0 0x7faafc063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab011c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faafc1622e0 0x7faafc063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60040/0 (socket says 172.21.15.7:60040) 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab011c3700 1 -- 172.21.15.7:0/3756175566 learned_addr learned my addr 172.21.15.7:0/3756175566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab021c5700 1 --1- 172.21.15.7:0/3756175566 >> v1:172.21.15.7:6789/0 conn(0x7faafc0c0e40 0x7faafc06dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33228/0 (socket says 172.21.15.7:33228) 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.990+0000 7fab019c4700 1 --2- 172.21.15.7:0/3756175566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0a70 0x7faafc161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab011c3700 1 -- 172.21.15.7:0/3756175566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0a70 msgr2=0x7faafc161da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab011c3700 1 --2- 172.21.15.7:0/3756175566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0a70 0x7faafc161da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab011c3700 1 -- 172.21.15.7:0/3756175566 >> v1:172.21.15.7:6789/0 conn(0x7faafc0c0e40 legacy=0x7faafc06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab009c2700 1 -- 172.21.15.7:0/3756175566 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589187444 0 0) 0x7faafc0c83f0 con 0x7faafc0c0e40 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab011c3700 1 -- 172.21.15.7:0/3756175566 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc162a60 con 0x7faafc1622e0 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab011c3700 1 --2- 172.21.15.7:0/3756175566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc063230 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7faaf00099b0 tx=0x7faaf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f4a98f6b7fc6ee server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab009c2700 1 -- 172.21.15.7:0/3756175566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000b860 con 0x7faafc1622e0 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab009c2700 1 -- 172.21.15.7:0/3756175566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf000b9c0 con 0x7faafc1622e0 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab009c2700 1 -- 172.21.15.7:0/3756175566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000bd60 con 0x7faafc1622e0 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab03c28700 1 -- 172.21.15.7:0/3756175566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 msgr2=0x7faafc063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.991+0000 7fab03c28700 1 --2- 172.21.15.7:0/3756175566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc063230 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7faaf00099b0 tx=0x7faaf0009250 comp rx=0 tx=0).stop 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.992+0000 7fab03c28700 1 -- 172.21.15.7:0/3756175566 shutdown_connections 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.992+0000 7fab03c28700 1 --2- 172.21.15.7:0/3756175566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0a70 0x7faafc161da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.992+0000 7fab03c28700 1 --2- 172.21.15.7:0/3756175566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc063230 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.992+0000 7fab03c28700 1 -- 172.21.15.7:0/3756175566 >> 172.21.15.7:0/3756175566 conn(0x7faafc0680e0 msgr2=0x7faafc0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.992+0000 7fab03c28700 1 -- 172.21.15.7:0/3756175566 shutdown_connections 2024-01-11T07:18:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.992+0000 7fab03c28700 1 -- 172.21.15.7:0/3756175566 wait complete. 2024-01-11T07:18:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.993+0000 7fab03c28700 1 Processor -- start 2024-01-11T07:18:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.993+0000 7fab03c28700 1 -- start start 2024-01-11T07:18:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab03c28700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faafc0c0a70 0x7faafc0c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab03c28700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0e40 0x7faafc15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab03c28700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc0b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab03c28700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0c7da0 con 0x7faafc1622e0 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab03c28700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0b15b0 con 0x7faafc0c0a70 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab03c28700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0b1840 con 0x7faafc0c0e40 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab021c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc0b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab021c5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc0b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60048/0 (socket says 172.21.15.7:60048) 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab011c3700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0e40 0x7faafc15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab019c4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faafc0c0a70 0x7faafc0c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab021c5700 1 -- 172.21.15.7:0/3591605168 learned_addr learned my addr 172.21.15.7:0/3591605168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab021c5700 1 -- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0e40 msgr2=0x7faafc15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab021c5700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0e40 0x7faafc15b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab021c5700 1 -- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faafc0c0a70 msgr2=0x7faafc0c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab021c5700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faafc0c0a70 0x7faafc0c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.994+0000 7fab021c5700 1 -- 172.21.15.7:0/3591605168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00094f0 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.995+0000 7fab021c5700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc0b1070 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7faaf400cd00 tx=0x7faaf4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.995+0000 7faaedffb700 1 -- 172.21.15.7:0/3591605168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf400ce60 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.995+0000 7faaedffb700 1 -- 172.21.15.7:0/3591605168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf4005df0 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.995+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc0b1b20 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.995+0000 7faaedffb700 1 -- 172.21.15.7:0/3591605168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf400aae0 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.995+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc0b20e0 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.996+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc04c3c0 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.996+0000 7faaedffb700 1 -- 172.21.15.7:0/3591605168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7faaf40070f0 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.997+0000 7faaedffb700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faae0066f30 0x7faae00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.997+0000 7faaedffb700 1 -- 172.21.15.7:0/3591605168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faaf4084570 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.997+0000 7faaedffb700 1 -- 172.21.15.7:0/3591605168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7faae006ea50 con 0x7faafc1622e0 2024-01-11T07:18:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.997+0000 7fab019c4700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faae0066f30 0x7faae00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:34.998+0000 7fab019c4700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faae0066f30 0x7faae00693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faaf8006fd0 tx=0x7faaf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.001+0000 7faaedffb700 1 -- 172.21.15.7:0/3591605168 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faaf4051860 con 0x7faafc1622e0 2024-01-11T07:18:35.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.206+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faafc0c8080 con 0x7faafc1622e0 2024-01-11T07:18:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.207+0000 7faaedffb700 1 -- 172.21.15.7:0/3591605168 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v186) v1 ==== 81+0+32253 (secure 0 0 0) 0x7faaf4054e80 con 0x7faafc1622e0 2024-01-11T07:18:35.207 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:35.207 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:35.207 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 186, 2024-01-11T07:18:35.207 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:18:32.297374+0000", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:18:35.208 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:35.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:18:35.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:35.211 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:35.212 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:35.213 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.360758+0000", 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:35.215 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-11T07:18:35.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:35.218 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:35.219 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.823991+0000", 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:35.220 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:35.221 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "186", 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:35.222 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:35.223 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:35.224 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:18:35.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:18:35.230 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.231 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.235 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.236 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:18:35.246 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.247 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.252 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:18:35.257 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.268 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.270 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.271 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.272 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.276 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faae0066f30 msgr2=0x7faae00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faae0066f30 0x7faae00693e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faaf8006fd0 tx=0x7faaf8008040 comp rx=0 tx=0).stop 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 msgr2=0x7faafc0b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc0b1070 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7faaf400cd00 tx=0x7faaf4001ac0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 shutdown_connections 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7faae0066f30 0x7faae00693e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faafc1622e0 0x7faafc0b1070 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faafc0c0a70 0x7faafc0c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 --2- 172.21.15.7:0/3591605168 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faafc0c0e40 0x7faafc15b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 >> 172.21.15.7:0/3591605168 conn(0x7faafc0680e0 msgr2=0x7faafc0c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:35.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.209+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 shutdown_connections 2024-01-11T07:18:35.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.210+0000 7fab03c28700 1 -- 172.21.15.7:0/3591605168 wait complete. 2024-01-11T07:18:35.279 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-01-11T07:18:35.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:18:35.280 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-01-11T07:18:35.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:18:35.560 INFO:teuthology.orchestra.run.smithi007.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2024-01-11T07:18:35.560 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-11T07:18:35.560 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-11T07:18:35.560 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.255533 s, 205 MB/s 2024-01-11T07:18:35.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:18:35.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.691+0000 7f5606d9b700 1 Processor -- start 2024-01-11T07:18:35.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.691+0000 7f5606d9b700 1 -- start start 2024-01-11T07:18:35.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.692+0000 7f5606d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f560006dd00 0x7f56000c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:35.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.692+0000 7f5606d9b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560015b770 con 0x7f560006e0d0 2024-01-11T07:18:35.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.692+0000 7f5606d9b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560015a620 con 0x7f560015a1c0 2024-01-11T07:18:35.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.692+0000 7f5606d9b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560015abf0 con 0x7f560006dd00 2024-01-11T07:18:35.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f5604b37700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f560006dd00 0x7f56000c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:35.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f55fffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f560006e0d0 0x7f5600159ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33240/0 (socket says 172.21.15.7:33240) 2024-01-11T07:18:35.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f55fffff700 1 -- 172.21.15.7:0/3693327269 learned_addr learned my addr 172.21.15.7:0/3693327269 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f5604b37700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f560006dd00 0x7f56000c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60050/0 (socket says 172.21.15.7:60050) 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f5604b37700 1 -- 172.21.15.7:0/3693327269 >> v1:172.21.15.113:6789/0 conn(0x7f560015a1c0 legacy=0x7f5600069fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f5604b37700 1 -- 172.21.15.7:0/3693327269 >> v1:172.21.15.7:6789/0 conn(0x7f560006e0d0 legacy=0x7f5600159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f5604b37700 1 -- 172.21.15.7:0/3693327269 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560015aed0 con 0x7f560006dd00 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f55ff7fe700 1 -- 172.21.15.7:0/3693327269 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880296452 0 0) 0x7f560015b770 con 0x7f560006e0d0 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f5604b37700 1 --2- 172.21.15.7:0/3693327269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006dd00 0x7f56000c98d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f55f000bb90 tx=0x7f55f000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad19809a1658ba7 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f55ff7fe700 1 -- 172.21.15.7:0/3693327269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000db60 con 0x7f560006dd00 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f55ff7fe700 1 -- 172.21.15.7:0/3693327269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f000dcc0 con 0x7f560006dd00 2024-01-11T07:18:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.693+0000 7f55ff7fe700 1 -- 172.21.15.7:0/3693327269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00106a0 con 0x7f560006dd00 2024-01-11T07:18:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.694+0000 7f5606d9b700 1 -- 172.21.15.7:0/3693327269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006dd00 msgr2=0x7f56000c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.694+0000 7f5606d9b700 1 --2- 172.21.15.7:0/3693327269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006dd00 0x7f56000c98d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f55f000bb90 tx=0x7f55f000b430 comp rx=0 tx=0).stop 2024-01-11T07:18:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.694+0000 7f5606d9b700 1 -- 172.21.15.7:0/3693327269 shutdown_connections 2024-01-11T07:18:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.694+0000 7f5606d9b700 1 --2- 172.21.15.7:0/3693327269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006dd00 0x7f56000c98d0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.694+0000 7f5606d9b700 1 -- 172.21.15.7:0/3693327269 >> 172.21.15.7:0/3693327269 conn(0x7f56000680e0 msgr2=0x7f56000cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.694+0000 7f5606d9b700 1 -- 172.21.15.7:0/3693327269 shutdown_connections 2024-01-11T07:18:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.694+0000 7f5606d9b700 1 -- 172.21.15.7:0/3693327269 wait complete. 2024-01-11T07:18:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.695+0000 7f5606d9b700 1 Processor -- start 2024-01-11T07:18:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.695+0000 7f5606d9b700 1 -- start start 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f5606d9b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f560006dd00 0x7f56000c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f5606d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006e0d0 0x7f56000cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f5606d9b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560015a1c0 0x7f56000cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f5606d9b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56000c6fc0 con 0x7f560006e0d0 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f5606d9b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56000c63f0 con 0x7f560015a1c0 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f5606d9b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56000c6690 con 0x7f560006dd00 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f55fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006e0d0 0x7f56000cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f55fffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006e0d0 0x7f56000cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60062/0 (socket says 172.21.15.7:60062) 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f55fffff700 1 -- 172.21.15.7:0/1568451882 learned_addr learned my addr 172.21.15.7:0/1568451882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f5604b37700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f560006dd00 0x7f56000c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.696+0000 7f5605338700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560015a1c0 0x7f56000cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fffff700 1 -- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f560006dd00 msgr2=0x7f56000c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fffff700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f560006dd00 0x7f56000c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fffff700 1 -- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560015a1c0 msgr2=0x7f56000cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fffff700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560015a1c0 0x7f56000cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fffff700 1 -- 172.21.15.7:0/1568451882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f000b6d0 con 0x7f560006e0d0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f5605338700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560015a1c0 0x7f56000cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fffff700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006e0d0 0x7f56000cb400 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c7e0 tx=0x7f55e8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fcff9700 1 -- 172.21.15.7:0/1568451882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8005b30 con 0x7f560006e0d0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fcff9700 1 -- 172.21.15.7:0/1568451882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e8005c90 con 0x7f560006e0d0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f55fcff9700 1 -- 172.21.15.7:0/1568451882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8007300 con 0x7f560006e0d0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56000c6970 con 0x7f560006e0d0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.697+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56000c7990 con 0x7f560006e0d0 2024-01-11T07:18:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.698+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56000c7170 con 0x7f560006e0d0 2024-01-11T07:18:35.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.699+0000 7f55fcff9700 1 -- 172.21.15.7:0/1568451882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f55e80076b0 con 0x7f560006e0d0 2024-01-11T07:18:35.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.699+0000 7f55fcff9700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55e0066f30 0x7f55e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.699+0000 7f55fcff9700 1 -- 172.21.15.7:0/1568451882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55e8083a90 con 0x7f560006e0d0 2024-01-11T07:18:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.699+0000 7f55fcff9700 1 -- 172.21.15.7:0/1568451882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f55e006ea50 con 0x7f560006e0d0 2024-01-11T07:18:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.699+0000 7f5604b37700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55e0066f30 0x7f55e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.700+0000 7f5604b37700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55e0066f30 0x7f55e00693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b470 tx=0x7f55f001a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.703+0000 7f55fcff9700 1 -- 172.21.15.7:0/1568451882 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55e8017060 con 0x7f560006e0d0 2024-01-11T07:18:35.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.910+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f560004c3c0 con 0x7f560006e0d0 2024-01-11T07:18:35.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.911+0000 7f55fcff9700 1 -- 172.21.15.7:0/1568451882 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v186) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f56000c7170 con 0x7f560006e0d0 2024-01-11T07:18:35.911 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 186, 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:18:32.297374+0000", 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:18:35.912 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:18:35.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:35.914 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:35.915 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:35.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:35.917 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:35.918 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.360758+0000", 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:35.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:18:35.920 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:35.921 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:35.922 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:35.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:35.924 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.823991+0000", 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:35.926 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "186", 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:35.927 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:35.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:35.929 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.935 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.936 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.938 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:18:35.939 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:18:35.940 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.941 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.942 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.946 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.947 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.952 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:35.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:18:35.963 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.964 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:18:35.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:35.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:35.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:18:35.974 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.975 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:18:35.976 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.977 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.978 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:35.980 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:18:35.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:18:35.982 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:18:35.983 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:18:35.983 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:18:35.983 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:18:35.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:35.983 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.913+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55e0066f30 msgr2=0x7f55e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.913+0000 7f5606d9b700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55e0066f30 0x7f55e00693e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f55f000b470 tx=0x7f55f001a000 comp rx=0 tx=0).stop 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.913+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006e0d0 msgr2=0x7f56000cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.913+0000 7f5606d9b700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006e0d0 0x7f56000cb400 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c7e0 tx=0x7f55e8007df0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.914+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 shutdown_connections 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.914+0000 7f5606d9b700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f55e0066f30 0x7f55e00693e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.914+0000 7f5606d9b700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f560015a1c0 0x7f56000cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.914+0000 7f5606d9b700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f560006dd00 0x7f56000c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.914+0000 7f5606d9b700 1 --2- 172.21.15.7:0/1568451882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f560006e0d0 0x7f56000cb400 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.914+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 >> 172.21.15.7:0/1568451882 conn(0x7f56000680e0 msgr2=0x7f5600069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.914+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 shutdown_connections 2024-01-11T07:18:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:35.914+0000 7f5606d9b700 1 -- 172.21.15.7:0/1568451882 wait complete. 2024-01-11T07:18:35.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.122+0000 7fbb1aafb700 1 Processor -- start 2024-01-11T07:18:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.122+0000 7fbb1aafb700 1 -- start start 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb1aafb700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb14159f50 0x7fbb140cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb1aafb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb140cdd40 0x7fbb1415a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb1aafb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1415e530 con 0x7fbb1415e9c0 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb1aafb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb140cc970 con 0x7fbb140cdd40 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb1aafb700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb140cc690 con 0x7fbb14159f50 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb19098700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb140cdd40 0x7fbb1415a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb13fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb1415e9c0 0x7fbb140cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33248/0 (socket says 172.21.15.7:33248) 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb19098700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb140cdd40 0x7fbb1415a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60078/0 (socket says 172.21.15.7:60078) 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb13fff700 1 -- 172.21.15.7:0/2849764226 learned_addr learned my addr 172.21.15.7:0/2849764226 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.123+0000 7fbb18897700 1 --2- 172.21.15.7:0/2849764226 >> v2:172.21.15.113:3300/0 conn(0x7fbb14159f50 0x7fbb140cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb19098700 1 -- 172.21.15.7:0/2849764226 >> v2:172.21.15.113:3300/0 conn(0x7fbb14159f50 msgr2=0x7fbb140cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb19098700 1 --2- 172.21.15.7:0/2849764226 >> v2:172.21.15.113:3300/0 conn(0x7fbb14159f50 0x7fbb140cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb19098700 1 -- 172.21.15.7:0/2849764226 >> v1:172.21.15.7:6789/0 conn(0x7fbb1415e9c0 legacy=0x7fbb140cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb19098700 1 -- 172.21.15.7:0/2849764226 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1415dee0 con 0x7fbb140cdd40 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb19098700 1 --2- 172.21.15.7:0/2849764226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb140cdd40 0x7fbb1415a320 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fbb140ce620 tx=0x7fbb0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463ef67e86574ab8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb137fe700 1 -- 172.21.15.7:0/2849764226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00e070 con 0x7fbb140cdd40 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb137fe700 1 -- 172.21.15.7:0/2849764226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c00bb30 con 0x7fbb140cdd40 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb137fe700 1 -- 172.21.15.7:0/2849764226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00bed0 con 0x7fbb140cdd40 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2849764226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb140cdd40 msgr2=0x7fbb1415a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.124+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2849764226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb140cdd40 0x7fbb1415a320 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fbb140ce620 tx=0x7fbb0c009780 comp rx=0 tx=0).stop 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.125+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2849764226 shutdown_connections 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.125+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2849764226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb140cdd40 0x7fbb1415a320 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.125+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2849764226 >> v2:172.21.15.113:3300/0 conn(0x7fbb14159f50 0x7fbb140cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.125+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2849764226 >> 172.21.15.7:0/2849764226 conn(0x7fbb1406e6b0 msgr2=0x7fbb14070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:36.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.125+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2849764226 shutdown_connections 2024-01-11T07:18:36.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.125+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2849764226 wait complete. 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.126+0000 7fbb1aafb700 1 Processor -- start 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.126+0000 7fbb1aafb700 1 -- start start 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb1aafb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb140cdd40 0x7fbb140d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb1aafb700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb14159f50 0x7fbb14147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb1aafb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb1415e9c0 0x7fbb140b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb1aafb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb140cfda0 con 0x7fbb1415e9c0 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb1aafb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb140d13e0 con 0x7fbb140cdd40 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb1aafb700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb140cf190 con 0x7fbb14159f50 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb1415e9c0 0x7fbb140b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb18897700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb140cdd40 0x7fbb140d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb1415e9c0 0x7fbb140b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60088/0 (socket says 172.21.15.7:60088) 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 -- 172.21.15.7:0/2365687428 learned_addr learned my addr 172.21.15.7:0/2365687428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb18897700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb140cdd40 0x7fbb140d0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:48116/0 (socket says 172.21.15.7:48116) 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb13fff700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb14159f50 0x7fbb14147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 -- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb14159f50 msgr2=0x7fbb14147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb14159f50 0x7fbb14147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 -- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb140cdd40 msgr2=0x7fbb140d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb140cdd40 0x7fbb140d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 -- 172.21.15.7:0/2365687428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c009a20 con 0x7fbb1415e9c0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.127+0000 7fbb19098700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb1415e9c0 0x7fbb140b5b30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fbb14067f20 tx=0x7fbb0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.128+0000 7fbb10ff9700 1 -- 172.21.15.7:0/2365687428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c012d60 con 0x7fbb1415e9c0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.128+0000 7fbb10ff9700 1 -- 172.21.15.7:0/2365687428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c026460 con 0x7fbb1415e9c0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.128+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb140cf470 con 0x7fbb1415e9c0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.128+0000 7fbb10ff9700 1 -- 172.21.15.7:0/2365687428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c026820 con 0x7fbb1415e9c0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.128+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb140cfa30 con 0x7fbb1415e9c0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.129+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb14089350 con 0x7fbb1415e9c0 2024-01-11T07:18:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.129+0000 7fbb10ff9700 1 -- 172.21.15.7:0/2365687428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fbb0c01d420 con 0x7fbb1415e9c0 2024-01-11T07:18:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.130+0000 7fbb10ff9700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbaf8066f30 0x7fbaf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.130+0000 7fbb10ff9700 1 -- 172.21.15.7:0/2365687428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb0c086990 con 0x7fbb1415e9c0 2024-01-11T07:18:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.130+0000 7fbb18897700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbaf8066f30 0x7fbaf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.130+0000 7fbb10ff9700 1 -- 172.21.15.7:0/2365687428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fbaf806ea50 con 0x7fbb1415e9c0 2024-01-11T07:18:36.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.130+0000 7fbb18897700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbaf8066f30 0x7fbaf80693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006fd0 tx=0x7fbb08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:36.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.134+0000 7fbb10ff9700 1 -- 172.21.15.7:0/2365687428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb0c053c00 con 0x7fbb1415e9c0 2024-01-11T07:18:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.389+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb140890f0 con 0x7fbb1415e9c0 2024-01-11T07:18:36.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.390+0000 7fbb10ff9700 1 -- 172.21.15.7:0/2365687428 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fbb0c02b070 con 0x7fbb1415e9c0 2024-01-11T07:18:36.390 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:36.390 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbaf8066f30 msgr2=0x7fbaf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbaf8066f30 0x7fbaf80693e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006fd0 tx=0x7fbb08008040 comp rx=0 tx=0).stop 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb1415e9c0 msgr2=0x7fbb140b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb1415e9c0 0x7fbb140b5b30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fbb14067f20 tx=0x7fbb0c018000 comp rx=0 tx=0).stop 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 shutdown_connections 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fbaf8066f30 0x7fbaf80693e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb1415e9c0 0x7fbb140b5b30 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb140cdd40 0x7fbb140d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 --2- 172.21.15.7:0/2365687428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbb14159f50 0x7fbb14147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.392+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 >> 172.21.15.7:0/2365687428 conn(0x7fbb1406e6b0 msgr2=0x7fbb1415a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.393+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 shutdown_connections 2024-01-11T07:18:36.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.393+0000 7fbb1aafb700 1 -- 172.21.15.7:0/2365687428 wait complete. 2024-01-11T07:18:36.394 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:36.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-11T07:18:36.559 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":8064,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.125775619999999,"mdsmap_epoch":90,"osdmap_epoch":186,"osdmap_epoch_barrier":186,"uptime":29.140332281999999} 2024-01-11T07:18:36.559 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 8064, "mdsmap_epoch": 90, "osdmap_epoch": 186, "osdmap_epoch_barrier": 186, "rank_uptime": 23.12577562, "state": "up:active", "uptime": 29.140332282, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:18:36.559 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:18:36.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:18:36.560 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-01-11T07:18:36.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:18:36.580 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-01-11T07:18:36.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:18:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.830+0000 7f922bb97700 1 Processor -- start 2024-01-11T07:18:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.830+0000 7f922bb97700 1 -- start start 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f922bb97700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92241617c0 0x7f92240cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f922bb97700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92240b1820 0x7f92240b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f922bb97700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9224161b90 con 0x7f92240b0430 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f922bb97700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240b36c0 con 0x7f92240b1820 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f922bb97700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240b1ca0 con 0x7f92241617c0 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f9229132700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92240b1820 0x7f92240b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f922a134700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f92240b0430 0x7f9224067610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33254/0 (socket says 172.21.15.7:33254) 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f9229132700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92240b1820 0x7f92240b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60092/0 (socket says 172.21.15.7:60092) 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f922a134700 1 -- 172.21.15.7:0/1558849809 learned_addr learned my addr 172.21.15.7:0/1558849809 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.831+0000 7f9229933700 1 --2- 172.21.15.7:0/1558849809 >> v2:172.21.15.113:3300/0 conn(0x7f92241617c0 0x7f92240cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9229132700 1 -- 172.21.15.7:0/1558849809 >> v2:172.21.15.113:3300/0 conn(0x7f92241617c0 msgr2=0x7f92240cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9229132700 1 --2- 172.21.15.7:0/1558849809 >> v2:172.21.15.113:3300/0 conn(0x7f92241617c0 0x7f92240cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9229132700 1 -- 172.21.15.7:0/1558849809 >> v1:172.21.15.7:6789/0 conn(0x7f92240b0430 legacy=0x7f9224067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9228931700 1 -- 172.21.15.7:0/1558849809 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422340659 0 0) 0x7f9224161b90 con 0x7f92240b0430 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9229132700 1 -- 172.21.15.7:0/1558849809 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92240b4950 con 0x7f92240b1820 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9229132700 1 --2- 172.21.15.7:0/1558849809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92240b4410 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f922405f2e0 tx=0x7f92140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64777d032f845b2a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9228931700 1 -- 172.21.15.7:0/1558849809 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214011630 con 0x7f92240b1820 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9228931700 1 -- 172.21.15.7:0/1558849809 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9214011790 con 0x7f92240b1820 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.832+0000 7f9228931700 1 -- 172.21.15.7:0/1558849809 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214011b30 con 0x7f92240b1820 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.833+0000 7f922bb97700 1 -- 172.21.15.7:0/1558849809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 msgr2=0x7f92240b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.833+0000 7f922bb97700 1 --2- 172.21.15.7:0/1558849809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92240b4410 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f922405f2e0 tx=0x7f92140091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.833+0000 7f922bb97700 1 -- 172.21.15.7:0/1558849809 shutdown_connections 2024-01-11T07:18:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.833+0000 7f922bb97700 1 --2- 172.21.15.7:0/1558849809 >> v2:172.21.15.113:3300/0 conn(0x7f92241617c0 0x7f92240cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.833+0000 7f922bb97700 1 --2- 172.21.15.7:0/1558849809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92240b4410 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.833+0000 7f922bb97700 1 -- 172.21.15.7:0/1558849809 >> 172.21.15.7:0/1558849809 conn(0x7f9224071100 msgr2=0x7f922406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.833+0000 7f922bb97700 1 -- 172.21.15.7:0/1558849809 shutdown_connections 2024-01-11T07:18:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.833+0000 7f922bb97700 1 -- 172.21.15.7:0/1558849809 wait complete. 2024-01-11T07:18:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.834+0000 7f922bb97700 1 Processor -- start 2024-01-11T07:18:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.834+0000 7f922bb97700 1 -- start start 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f922bb97700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92240b0430 0x7f9224159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f922bb97700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92241584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f922bb97700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92241617c0 0x7f92240acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f922bb97700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9224158cd0 con 0x7f92240b1820 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f922bb97700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92241597e0 con 0x7f92240b0430 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f922bb97700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92240a71b0 con 0x7f92241617c0 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229132700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92241584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229132700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92241584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60102/0 (socket says 172.21.15.7:60102) 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229132700 1 -- 172.21.15.7:0/1823280571 learned_addr learned my addr 172.21.15.7:0/1823280571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:36.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f922a134700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92241617c0 0x7f92240acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229933700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92240b0430 0x7f9224159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229132700 1 -- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92241617c0 msgr2=0x7f92240acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229132700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92241617c0 0x7f92240acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229132700 1 -- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92240b0430 msgr2=0x7f9224159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229132700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92240b0430 0x7f9224159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229132700 1 -- 172.21.15.7:0/1823280571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9214009420 con 0x7f92240b1820 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.835+0000 7f9229933700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92240b0430 0x7f9224159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.836+0000 7f9229132700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92241584d0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f922406a1d0 tx=0x7f9214017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.836+0000 7f9219ffb700 1 -- 172.21.15.7:0/1823280571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214011b10 con 0x7f92240b1820 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.836+0000 7f9219ffb700 1 -- 172.21.15.7:0/1823280571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92140253f0 con 0x7f92240b1820 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.836+0000 7f9219ffb700 1 -- 172.21.15.7:0/1823280571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214025790 con 0x7f92240b1820 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.836+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9224158eb0 con 0x7f92240b1820 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.836+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9224159470 con 0x7f92240b1820 2024-01-11T07:18:36.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.837+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922404c3c0 con 0x7f92240b1820 2024-01-11T07:18:36.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.838+0000 7f9219ffb700 1 -- 172.21.15.7:0/1823280571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f9214025b80 con 0x7f92240b1820 2024-01-11T07:18:36.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.838+0000 7f9219ffb700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9208066f30 0x7f92080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:36.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.838+0000 7f9219ffb700 1 -- 172.21.15.7:0/1823280571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9214086400 con 0x7f92240b1820 2024-01-11T07:18:36.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.839+0000 7f9219ffb700 1 -- 172.21.15.7:0/1823280571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f920806ea50 con 0x7f92240b1820 2024-01-11T07:18:36.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.839+0000 7f9229933700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9208066f30 0x7f92080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:36.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.839+0000 7f9229933700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9208066f30 0x7f92080693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9220009c50 tx=0x7f9220008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:36.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:36.842+0000 7f9219ffb700 1 -- 172.21.15.7:0/1823280571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9214053690 con 0x7f92240b1820 2024-01-11T07:18:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.048+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f92240b2070 con 0x7f92240b1820 2024-01-11T07:18:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.049+0000 7f9219ffb700 1 -- 172.21.15.7:0/1823280571 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v186) v1 ==== 81+0+32253 (secure 0 0 0) 0x7f9214015020 con 0x7f92240b1820 2024-01-11T07:18:37.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:37.049 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:37.049 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 186, 2024-01-11T07:18:37.049 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:18:32.297374+0000", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:18:37.050 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:18:37.051 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:37.052 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:37.054 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:37.055 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:37.056 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-11T07:18:37.057 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.360758+0000", 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:37.058 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:37.059 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.061 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:18:37.062 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.823991+0000", 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1027, 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,full,full_quota", 2024-01-11T07:18:37.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:18:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "186", 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:37.065 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:37.066 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:37.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:37.068 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:37.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:18:37.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:18:37.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:18:37.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.080 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.081 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:37.082 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:37.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:18:37.084 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:18:37.085 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.086 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:37.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:37.088 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:18:37.089 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.090 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:37.094 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.095 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.096 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:37.097 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:18:37.098 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:37.099 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:37.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:37.110 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.112 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.113 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:18:37.114 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:37.115 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:37.116 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:37.117 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:37.118 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:37.120 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:37.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:18:37.122 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:18:37.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:37.124 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9208066f30 msgr2=0x7f92080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9208066f30 0x7f92080693e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9220009c50 tx=0x7f9220008040 comp rx=0 tx=0).stop 2024-01-11T07:18:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 msgr2=0x7f92241584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92241584d0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f922406a1d0 tx=0x7f9214017000 comp rx=0 tx=0).stop 2024-01-11T07:18:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 shutdown_connections 2024-01-11T07:18:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f9208066f30 0x7f92080693e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f92241617c0 0x7f92240acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92240b0430 0x7f9224159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.051+0000 7f922bb97700 1 --2- 172.21.15.7:0/1823280571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240b1820 0x7f92241584d0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.052+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 >> 172.21.15.7:0/1823280571 conn(0x7f9224071100 msgr2=0x7f922415ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.052+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 shutdown_connections 2024-01-11T07:18:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:37.052+0000 7f922bb97700 1 -- 172.21.15.7:0/1823280571 wait complete. 2024-01-11T07:18:37.127 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-01-11T07:18:42.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:18:42.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.269+0000 7f04d8685700 1 Processor -- start 2024-01-11T07:18:42.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.269+0000 7f04d8685700 1 -- start start 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d8685700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04d0063680 0x7f04d00c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d8685700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d015d910 con 0x7f04d00aff20 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d8685700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d015dbf0 con 0x7f04d0063230 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d8685700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d00b02f0 con 0x7f04d0063680 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d6421700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f04d00aff20 0x7f04d00aab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48926/0 (socket says 172.21.15.7:48926) 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d6421700 1 -- 172.21.15.7:0/80556115 learned_addr learned my addr 172.21.15.7:0/80556115 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d6c22700 1 --2- 172.21.15.7:0/80556115 >> v2:172.21.15.113:3300/0 conn(0x7f04d0063680 0x7f04d00c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d541f700 1 -- 172.21.15.7:0/80556115 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399568515 0 0) 0x7f04d015d910 con 0x7f04d00aff20 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d541f700 1 -- 172.21.15.7:0/80556115 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04bc003570 con 0x7f04d00aff20 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d541f700 1 -- 172.21.15.7:0/80556115 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470915514 0 0) 0x7f04d015dbf0 con 0x7f04d0063230 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d541f700 1 -- 172.21.15.7:0/80556115 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04d015d910 con 0x7f04d0063230 2024-01-11T07:18:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d6c22700 1 -- 172.21.15.7:0/80556115 >> v1:172.21.15.113:6789/0 conn(0x7f04d0063230 legacy=0x7f04d00c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d6c22700 1 -- 172.21.15.7:0/80556115 >> v1:172.21.15.7:6789/0 conn(0x7f04d00aff20 legacy=0x7f04d00aab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.270+0000 7f04d6c22700 1 -- 172.21.15.7:0/80556115 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d015ded0 con 0x7f04d0063680 2024-01-11T07:18:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.271+0000 7f04d6c22700 1 --2- 172.21.15.7:0/80556115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063680 0x7f04d00c8730 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009e60 tx=0x7f04c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61cca8f67fa0a751 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.271+0000 7f04d541f700 1 -- 172.21.15.7:0/80556115 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800e070 con 0x7f04d0063680 2024-01-11T07:18:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.271+0000 7f04d541f700 1 -- 172.21.15.7:0/80556115 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c800bab0 con 0x7f04d0063680 2024-01-11T07:18:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.271+0000 7f04d541f700 1 -- 172.21.15.7:0/80556115 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800be70 con 0x7f04d0063680 2024-01-11T07:18:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.271+0000 7f04d8685700 1 -- 172.21.15.7:0/80556115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063680 msgr2=0x7f04d00c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.271+0000 7f04d8685700 1 --2- 172.21.15.7:0/80556115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063680 0x7f04d00c8730 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009e60 tx=0x7f04c8009780 comp rx=0 tx=0).stop 2024-01-11T07:18:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.272+0000 7f04d8685700 1 -- 172.21.15.7:0/80556115 shutdown_connections 2024-01-11T07:18:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.272+0000 7f04d8685700 1 --2- 172.21.15.7:0/80556115 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063680 0x7f04d00c8730 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.272+0000 7f04d8685700 1 -- 172.21.15.7:0/80556115 >> 172.21.15.7:0/80556115 conn(0x7f04d0068090 msgr2=0x7f04d00cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.272+0000 7f04d8685700 1 -- 172.21.15.7:0/80556115 shutdown_connections 2024-01-11T07:18:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.272+0000 7f04d8685700 1 -- 172.21.15.7:0/80556115 wait complete. 2024-01-11T07:18:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.273+0000 7f04d8685700 1 Processor -- start 2024-01-11T07:18:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.273+0000 7f04d8685700 1 -- start start 2024-01-11T07:18:42.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.274+0000 7f04d8685700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063230 0x7f04d00b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.274+0000 7f04d8685700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d0063680 0x7f04d00b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.274+0000 7f04d8685700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d00aff20 0x7f04d0156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.274+0000 7f04d8685700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0154ea0 con 0x7f04d0063680 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.274+0000 7f04d8685700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0155d00 con 0x7f04d0063230 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.274+0000 7f04d8685700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d01550f0 con 0x7f04d00aff20 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.274+0000 7f04d5c20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d0063680 0x7f04d00b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d5c20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d0063680 0x7f04d00b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49942/0 (socket says 172.21.15.7:49942) 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d5c20700 1 -- 172.21.15.7:0/1922589111 learned_addr learned my addr 172.21.15.7:0/1922589111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d6c22700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d00aff20 0x7f04d0156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d6421700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063230 0x7f04d00b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d5c20700 1 -- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d00aff20 msgr2=0x7f04d0156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d5c20700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d00aff20 0x7f04d0156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d5c20700 1 -- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063230 msgr2=0x7f04d00b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d5c20700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063230 0x7f04d00b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d5c20700 1 -- 172.21.15.7:0/1922589111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c8009a20 con 0x7f04d0063680 2024-01-11T07:18:42.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d5c20700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d0063680 0x7f04d00b3160 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f04c000c580 tx=0x7f04c0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:42.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0010510 con 0x7f04d0063680 2024-01-11T07:18:42.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c0010670 con 0x7f04d0063680 2024-01-11T07:18:42.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d01553d0 con 0x7f04d0063680 2024-01-11T07:18:42.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0010a30 con 0x7f04d0063680 2024-01-11T07:18:42.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.275+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d0155990 con 0x7f04d0063680 2024-01-11T07:18:42.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.276+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d0155e70 con 0x7f04d0063680 2024-01-11T07:18:42.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.277+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f04c001b400 con 0x7f04d0063680 2024-01-11T07:18:42.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.277+0000 7f04c67fc700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f04b4066f30 0x7f04b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.277+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f04c0084060 con 0x7f04d0063680 2024-01-11T07:18:42.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.277+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f04b406ea50 con 0x7f04d0063680 2024-01-11T07:18:42.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.277+0000 7f04d6421700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f04b4066f30 0x7f04b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.278+0000 7f04d6421700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f04b4066f30 0x7f04b40693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f04cc007fd0 tx=0x7f04cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:42.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.282+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04c00512d0 con 0x7f04d0063680 2024-01-11T07:18:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.300+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(187..187 src has 1..187) v4 ==== 671+0+0 (secure 0 0 0) 0x7f04c0055690 con 0x7f04d0063680 2024-01-11T07:18:42.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.300+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=188}) v3 -- 0x7f04b406f610 con 0x7f04d0063680 2024-01-11T07:18:42.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.492+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f04d004c3c0 con 0x7f04d0063680 2024-01-11T07:18:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.493+0000 7f04c67fc700 1 -- 172.21.15.7:0/1922589111 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v187) v1 ==== 81+0+32234 (secure 0 0 0) 0x7f04c00548f0 con 0x7f04d0063680 2024-01-11T07:18:42.493 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 187, 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:18:42.299913+0000", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:18:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:18:42.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:42.496 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:18:42.497 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:42.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:42.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:42.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:42.502 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.360758+0000", 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:42.503 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:18:42.504 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:42.505 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:42.506 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:42.507 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.823991+0000", 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:42.509 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "187", 2024-01-11T07:18:42.510 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:42.511 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:42.512 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:42.513 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.514 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.520 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.527 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.532 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:18:42.537 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.538 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:18:42.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.543 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:18:42.548 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.549 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.555 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.558 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:42.560 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.561 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.563 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.564 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.565 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:18:42.566 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:18:42.567 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:18:42.568 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:18:42.568 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:18:42.568 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:18:42.570 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:18:42.570 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:18:42.570 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:18:42.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.570 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f04b4066f30 msgr2=0x7f04b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f04b4066f30 0x7f04b40693e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f04cc007fd0 tx=0x7f04cc009040 comp rx=0 tx=0).stop 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d0063680 msgr2=0x7f04d00b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d0063680 0x7f04d00b3160 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f04c000c580 tx=0x7f04c0007e10 comp rx=0 tx=0).stop 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 shutdown_connections 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f04b4066f30 0x7f04b40693e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d00aff20 0x7f04d0156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d0063230 0x7f04d00b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 --2- 172.21.15.7:0/1922589111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04d0063680 0x7f04d00b3160 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 >> 172.21.15.7:0/1922589111 conn(0x7f04d0068090 msgr2=0x7f04d00c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 shutdown_connections 2024-01-11T07:18:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.496+0000 7f04d8685700 1 -- 172.21.15.7:0/1922589111 wait complete. 2024-01-11T07:18:42.573 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-01-11T07:18:42.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-01-11T07:18:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.708+0000 7fd1563e9700 1 Processor -- start 2024-01-11T07:18:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.709+0000 7fd1563e9700 1 -- start start 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.709+0000 7fd1563e9700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1500cafb0 0x7fd1500ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.709+0000 7fd1563e9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd150158280 0x7fd1500b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.709+0000 7fd1563e9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd150159300 con 0x7fd1500b0970 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.709+0000 7fd1563e9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1500b0dc0 con 0x7fd150158280 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.709+0000 7fd1563e9700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1500cf260 con 0x7fd1500cafb0 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd150158280 0x7fd1500b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd154986700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd1500b0970 0x7fd150159a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48940/0 (socket says 172.21.15.7:48940) 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd154986700 1 -- 172.21.15.7:0/378000009 learned_addr learned my addr 172.21.15.7:0/378000009 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14ffff700 1 --2- 172.21.15.7:0/378000009 >> v2:172.21.15.113:3300/0 conn(0x7fd1500cafb0 0x7fd1500ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14f7fe700 1 -- 172.21.15.7:0/378000009 >> v2:172.21.15.113:3300/0 conn(0x7fd1500cafb0 msgr2=0x7fd1500ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14f7fe700 1 --2- 172.21.15.7:0/378000009 >> v2:172.21.15.113:3300/0 conn(0x7fd1500cafb0 0x7fd1500ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14f7fe700 1 -- 172.21.15.7:0/378000009 >> v1:172.21.15.7:6789/0 conn(0x7fd1500b0970 legacy=0x7fd150159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14effd700 1 -- 172.21.15.7:0/378000009 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417907836 0 0) 0x7fd150159300 con 0x7fd1500b0970 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14f7fe700 1 -- 172.21.15.7:0/378000009 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1500cf540 con 0x7fd150158280 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14ffff700 1 --2- 172.21.15.7:0/378000009 >> v2:172.21.15.113:3300/0 conn(0x7fd1500cafb0 0x7fd1500ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14f7fe700 1 --2- 172.21.15.7:0/378000009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd150158280 0x7fd1500b0430 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd1500cfb70 tx=0x7fd1440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31603afe0401bd5e server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14effd700 1 -- 172.21.15.7:0/378000009 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400b6b0 con 0x7fd150158280 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14effd700 1 -- 172.21.15.7:0/378000009 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14400b810 con 0x7fd150158280 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.710+0000 7fd14effd700 1 -- 172.21.15.7:0/378000009 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400bbb0 con 0x7fd150158280 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.711+0000 7fd1563e9700 1 -- 172.21.15.7:0/378000009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd150158280 msgr2=0x7fd1500b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.711+0000 7fd1563e9700 1 --2- 172.21.15.7:0/378000009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd150158280 0x7fd1500b0430 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd1500cfb70 tx=0x7fd1440091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.711+0000 7fd1563e9700 1 -- 172.21.15.7:0/378000009 shutdown_connections 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.711+0000 7fd1563e9700 1 --2- 172.21.15.7:0/378000009 >> v2:172.21.15.113:3300/0 conn(0x7fd1500cafb0 0x7fd1500ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.711+0000 7fd1563e9700 1 --2- 172.21.15.7:0/378000009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd150158280 0x7fd1500b0430 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.711+0000 7fd1563e9700 1 -- 172.21.15.7:0/378000009 >> 172.21.15.7:0/378000009 conn(0x7fd150071100 msgr2=0x7fd15006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.711+0000 7fd1563e9700 1 -- 172.21.15.7:0/378000009 shutdown_connections 2024-01-11T07:18:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.711+0000 7fd1563e9700 1 -- 172.21.15.7:0/378000009 wait complete. 2024-01-11T07:18:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.713+0000 7fd1563e9700 1 Processor -- start 2024-01-11T07:18:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.713+0000 7fd1563e9700 1 -- start start 2024-01-11T07:18:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd1563e9700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1500b0970 0x7fd1500ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd1563e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1500cafb0 0x7fd1500c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd1563e9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd150158280 0x7fd150167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd1563e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1500cc900 con 0x7fd1500cafb0 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd1563e9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1500c5360 con 0x7fd150158280 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd1563e9700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1500ca660 con 0x7fd1500b0970 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd14f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1500cafb0 0x7fd1500c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd14f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1500cafb0 0x7fd1500c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49964/0 (socket says 172.21.15.7:49964) 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd14f7fe700 1 -- 172.21.15.7:0/1311534875 learned_addr learned my addr 172.21.15.7:0/1311534875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd14ffff700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1500b0970 0x7fd1500ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd154986700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd150158280 0x7fd150167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd14f7fe700 1 -- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1500b0970 msgr2=0x7fd1500ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd14f7fe700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1500b0970 0x7fd1500ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd14f7fe700 1 -- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd150158280 msgr2=0x7fd150167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.714+0000 7fd14f7fe700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd150158280 0x7fd150167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:42.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.715+0000 7fd14f7fe700 1 -- 172.21.15.7:0/1311534875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144009420 con 0x7fd1500cafb0 2024-01-11T07:18:42.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.715+0000 7fd14f7fe700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1500cafb0 0x7fd1500c4a80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd1440093f0 tx=0x7fd144017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:42.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.715+0000 7fd137fff700 1 -- 172.21.15.7:0/1311534875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400bb90 con 0x7fd1500cafb0 2024-01-11T07:18:42.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.715+0000 7fd137fff700 1 -- 172.21.15.7:0/1311534875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144011420 con 0x7fd1500cafb0 2024-01-11T07:18:42.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.715+0000 7fd137fff700 1 -- 172.21.15.7:0/1311534875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440117c0 con 0x7fd1500cafb0 2024-01-11T07:18:42.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.715+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1500ccae0 con 0x7fd1500cafb0 2024-01-11T07:18:42.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.715+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1500cd120 con 0x7fd1500cafb0 2024-01-11T07:18:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.716+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15004c3c0 con 0x7fd1500cafb0 2024-01-11T07:18:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.716+0000 7fd137fff700 1 -- 172.21.15.7:0/1311534875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd144011b70 con 0x7fd1500cafb0 2024-01-11T07:18:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.717+0000 7fd137fff700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd13c066f30 0x7fd13c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.717+0000 7fd137fff700 1 -- 172.21.15.7:0/1311534875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd1440857d0 con 0x7fd1500cafb0 2024-01-11T07:18:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.717+0000 7fd14ffff700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd13c066f30 0x7fd13c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.718+0000 7fd14ffff700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd13c066f30 0x7fd13c0693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd140006f00 tx=0x7fd140008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.722+0000 7fd137fff700 1 -- 172.21.15.7:0/1311534875 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1440529a0 con 0x7fd1500cafb0 2024-01-11T07:18:42.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.925+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd1500cf820 con 0x7fd1500cafb0 2024-01-11T07:18:42.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.926+0000 7fd137fff700 1 -- 172.21.15.7:0/1311534875 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v187) v1 ==== 81+0+32234 (secure 0 0 0) 0x7fd144055fc0 con 0x7fd1500cafb0 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 187, 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-01-11T07:06:42.317513+0000", 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-01-11T07:18:42.299913+0000", 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-01-11T07:06:47.885787+0000", 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-01-11T07:06:44.867018+0000", 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-01-11T07:18:42.927 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 4, 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.69999998807907104, 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.60000002384185791, 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 23, 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-01-11T07:18:42.928 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "quincy", 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:06:49.881877+0000", 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:42.929 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:42.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "108", 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:42.931 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:42.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:42.933 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 22, 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_metadata", 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.360758+0000", 2024-01-11T07:18:42.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-01-11T07:18:42.936 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "183", 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:42.937 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:42.938 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:42.939 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_bias": 4, 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 16, 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_priority": 5 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": "cephfs" 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 23, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "cephfs_data", 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-01-11T07:18:11.823991+0000", 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-01-11T07:18:42.941 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 64, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 64, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 64, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 64, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 64, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-01-11T07:18:42.942 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "187", 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-01-11T07:18:42.943 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 33554432, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-01-11T07:18:42.944 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-01-11T07:18:42.945 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 64, 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "target_size_ratio": 0.89999997615814209 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs": { 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "data": "cephfs" 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.946 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.949 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36945 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/36945", 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/36945", 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6824/36945", 2024-01-11T07:18:42.951 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/36945", 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.952 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.953 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-01-11T07:18:42.954 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36943 2024-01-11T07:18:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/36943", 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/36943", 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/36943", 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/36943", 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.957 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-01-11T07:18:42.958 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-01-11T07:18:42.959 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36946 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/36946", 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/36946", 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/36946", 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/36946", 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.963 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.964 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 36944 2024-01-11T07:18:42.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/36944", 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/36944", 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/36944", 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/36944", 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:42.968 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6800", 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6801", 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6802", 2024-01-11T07:18:42.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6803", 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6806", 2024-01-11T07:18:42.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6807", 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6804", 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6805", 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37188 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6801/37188", 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6803/37188", 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6807/37188", 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6805/37188", 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.973 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.974 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6816", 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6817", 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.975 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6818", 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6819", 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6822", 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6823", 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6820", 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6821", 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37190 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.978 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6817/37190", 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6819/37190", 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6823/37190", 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6821/37190", 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.979 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6808", 2024-01-11T07:18:42.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6809", 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6810", 2024-01-11T07:18:42.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6811", 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6814", 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6815", 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6812", 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6813", 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37189 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6809/37189", 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6811/37189", 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6815/37189", 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6813/37189", 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 182, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-01-11T07:18:42.985 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6824", 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6825", 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-01-11T07:18:42.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6826", 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6827", 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6830", 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6831", 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-01-11T07:18:42.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6828", 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.113:6829", 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 37240 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.113:6825/37240", 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.113:6827/37240", 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.113:6831/37240", 2024-01-11T07:18:42.989 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.113:6829/37240", 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.990 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.054179+0000", 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.051528+0000", 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-01-11T07:18:42.991 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.052224+0000", 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.038923+0000", 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.008477+0000", 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.032799+0000", 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.994 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:45.946032+0000", 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540138320759226367, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-01-11T07:06:46.000300+0000", 2024-01-11T07:18:42.995 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-01-11T07:18:42.996 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-01-11T07:18:42.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-01-11T07:18:42.998 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd13c066f30 msgr2=0x7fd13c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd13c066f30 0x7fd13c0693e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd140006f00 tx=0x7fd140008040 comp rx=0 tx=0).stop 2024-01-11T07:18:42.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1500cafb0 msgr2=0x7fd1500c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1500cafb0 0x7fd1500c4a80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd1440093f0 tx=0x7fd144017000 comp rx=0 tx=0).stop 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 shutdown_connections 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd13c066f30 0x7fd13c0693e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd150158280 0x7fd150167ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd1500b0970 0x7fd1500ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 --2- 172.21.15.7:0/1311534875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1500cafb0 0x7fd1500c4a80 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 >> 172.21.15.7:0/1311534875 conn(0x7fd150071100 msgr2=0x7fd1500ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 shutdown_connections 2024-01-11T07:18:43.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:42.929+0000 7fd1563e9700 1 -- 172.21.15.7:0/1311534875 wait complete. 2024-01-11T07:18:43.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:43.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.137+0000 7f1d5153a700 1 Processor -- start 2024-01-11T07:18:43.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.137+0000 7f1d5153a700 1 -- start start 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d5153a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c0cadf0 0x7f1d4c0cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d5153a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d4c0c6e10 0x7f1d4c0c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d5153a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c0699a0 con 0x7f1d4c1560d0 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d5153a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c0cc2f0 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d5153a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0c77d0 con 0x7f1d4c0cadf0 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d4b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d4c0c6e10 0x7f1d4c0c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d4affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1d4c1560d0 0x7f1d4c069e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48946/0 (socket says 172.21.15.7:48946) 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d4affd700 1 -- 172.21.15.7:0/4105426124 learned_addr learned my addr 172.21.15.7:0/4105426124 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d4a7fc700 1 --2- 172.21.15.7:0/4105426124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c0cadf0 0x7f1d4c0cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d49ffb700 1 -- 172.21.15.7:0/4105426124 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896829038 0 0) 0x7f1d4c0699a0 con 0x7f1d4c1560d0 2024-01-11T07:18:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d49ffb700 1 -- 172.21.15.7:0/4105426124 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d34003570 con 0x7f1d4c1560d0 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d4b7fe700 1 -- 172.21.15.7:0/4105426124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c0cadf0 msgr2=0x7f1d4c0cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d4b7fe700 1 --2- 172.21.15.7:0/4105426124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c0cadf0 0x7f1d4c0cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d4b7fe700 1 -- 172.21.15.7:0/4105426124 >> v1:172.21.15.7:6789/0 conn(0x7f1d4c1560d0 legacy=0x7f1d4c069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.138+0000 7f1d4b7fe700 1 -- 172.21.15.7:0/4105426124 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c06a540 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d4b7fe700 1 --2- 172.21.15.7:0/4105426124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c0c7290 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009e60 tx=0x7f1d44009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d11509ee369508 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d49ffb700 1 -- 172.21.15.7:0/4105426124 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400e070 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d49ffb700 1 -- 172.21.15.7:0/4105426124 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4400bab0 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d49ffb700 1 -- 172.21.15.7:0/4105426124 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400be70 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d5153a700 1 -- 172.21.15.7:0/4105426124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 msgr2=0x7f1d4c0c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d5153a700 1 --2- 172.21.15.7:0/4105426124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c0c7290 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009e60 tx=0x7f1d44009780 comp rx=0 tx=0).stop 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d5153a700 1 -- 172.21.15.7:0/4105426124 shutdown_connections 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d5153a700 1 --2- 172.21.15.7:0/4105426124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c0c7290 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d5153a700 1 --2- 172.21.15.7:0/4105426124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c0cadf0 0x7f1d4c0cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d5153a700 1 -- 172.21.15.7:0/4105426124 >> 172.21.15.7:0/4105426124 conn(0x7f1d4c063230 msgr2=0x7f1d4c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d5153a700 1 -- 172.21.15.7:0/4105426124 shutdown_connections 2024-01-11T07:18:43.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.139+0000 7f1d5153a700 1 -- 172.21.15.7:0/4105426124 wait complete. 2024-01-11T07:18:43.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.141+0000 7f1d5153a700 1 Processor -- start 2024-01-11T07:18:43.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.141+0000 7f1d5153a700 1 -- start start 2024-01-11T07:18:43.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d5153a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:43.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d5153a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d4c0cadf0 0x7f1d4c0c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:43.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d5153a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c1560d0 0x7f1d4c0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:43.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d5153a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c152e50 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d5153a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0acc90 con 0x7f1d4c0cadf0 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d5153a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0ac660 con 0x7f1d4c1560d0 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c15ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49986/0 (socket says 172.21.15.7:49986) 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 -- 172.21.15.7:0/1897708063 learned_addr learned my addr 172.21.15.7:0/1897708063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4b7fe700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c1560d0 0x7f1d4c0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4a7fc700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d4c0cadf0 0x7f1d4c0c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 -- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c1560d0 msgr2=0x7f1d4c0c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c1560d0 0x7f1d4c0c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 -- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d4c0cadf0 msgr2=0x7f1d4c0c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d4c0cadf0 0x7f1d4c0c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 -- 172.21.15.7:0/1897708063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009a20 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d4affd700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c15ebd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c590 tx=0x7f1d3c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d32ffd700 1 -- 172.21.15.7:0/1897708063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c005b30 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d32ffd700 1 -- 172.21.15.7:0/1897708063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c005cb0 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c0af940 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d32ffd700 1 -- 172.21.15.7:0/1897708063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c007320 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.142+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c0ac920 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.143+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c04c3c0 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.144+0000 7f1d32ffd700 1 -- 172.21.15.7:0/1897708063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1d3c01b3b0 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.144+0000 7f1d32ffd700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d38066f30 0x7f1d380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.144+0000 7f1d32ffd700 1 -- 172.21.15.7:0/1897708063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d3c083bf0 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.144+0000 7f1d4a7fc700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d38066f30 0x7f1d380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.145+0000 7f1d4a7fc700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d38066f30 0x7f1d380693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009d40 tx=0x7f1d40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.149+0000 7f1d32ffd700 1 -- 172.21.15.7:0/1897708063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d3c050dc0 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.399+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d4c068d90 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.400+0000 7f1d32ffd700 1 -- 172.21.15.7:0/1897708063 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f1d3c017020 con 0x7f1d4c0c6e10 2024-01-11T07:18:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:43.400 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d38066f30 msgr2=0x7f1d380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d38066f30 0x7f1d380693e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009d40 tx=0x7f1d40008040 comp rx=0 tx=0).stop 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 msgr2=0x7f1d4c15ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c15ebd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c590 tx=0x7f1d3c007e00 comp rx=0 tx=0).stop 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 shutdown_connections 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1d38066f30 0x7f1d380693e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d4c1560d0 0x7f1d4c0c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0c6e10 0x7f1d4c15ebd0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 --2- 172.21.15.7:0/1897708063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d4c0cadf0 0x7f1d4c0c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:43.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 >> 172.21.15.7:0/1897708063 conn(0x7f1d4c063230 msgr2=0x7f1d4c154be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:43.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 shutdown_connections 2024-01-11T07:18:43.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:43.402+0000 7f1d5153a700 1 -- 172.21.15.7:0/1897708063 wait complete. 2024-01-11T07:18:43.403 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:43.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-01-11T07:18:43.568 INFO:teuthology.orchestra.run.smithi007.stdout:{"cluster_fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","whoami":0,"id":8064,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":30.134499632000001,"mdsmap_epoch":90,"osdmap_epoch":187,"osdmap_epoch_barrier":186,"uptime":36.149055709999999} 2024-01-11T07:18:43.568 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "9c0c5e15-bc38-4016-89e4-891c3586efbb", "fs_name": "cephfs", "id": 8064, "mdsmap_epoch": 90, "osdmap_epoch": 187, "osdmap_epoch_barrier": 186, "rank_uptime": 30.134499632, "state": "up:active", "uptime": 36.14905571, "want_state": "up:active", "whoami": 0 } 2024-01-11T07:18:43.568 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-01-11T07:18:43.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:18:43.569 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-01-11T07:18:43.878 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records in 2024-01-11T07:18:43.878 INFO:teuthology.orchestra.run.smithi007.stderr:50+0 records out 2024-01-11T07:18:43.878 INFO:teuthology.orchestra.run.smithi007.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.288931 s, 181 MB/s 2024-01-11T07:18:43.879 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-11T07:18:43.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:18:43.934 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-11T07:18:43.977 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-11T07:18:43.977 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-11T07:18:43.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:18:43.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-11T07:18:44.050 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:18:44.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:44.051 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-11T07:18:44.083 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-11T07:18:44.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-01-11T07:18:44.083 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-01-11T07:18:44.125 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:18:44.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:44.125 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-11T07:18:44.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.323+0000 7f40cb65d700 1 Processor -- start 2024-01-11T07:18:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.323+0000 7f40cb65d700 1 -- start start 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40cb65d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40c4159ca0 0x7f40c406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40cb65d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c415a070 0x7f40c406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40cb65d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40c40ce7f0 0x7f40c40cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40cb65d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c40d18c0 con 0x7f40c4159ca0 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40cb65d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c40d0010 con 0x7f40c40ce7f0 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40cb65d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40cf170 con 0x7f40c415a070 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40c93f9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40c4159ca0 0x7f40c406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40c93f9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40c4159ca0 0x7f40c406dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49994/0 (socket says 172.21.15.7:49994) 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.324+0000 7f40c93f9700 1 -- 172.21.15.7:0/1012642101 learned_addr learned my addr 172.21.15.7:0/1012642101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40c9bfa700 1 --2- 172.21.15.7:0/1012642101 >> v2:172.21.15.113:3300/0 conn(0x7f40c40ce7f0 0x7f40c40cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40c8bf8700 1 --2- 172.21.15.7:0/1012642101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c415a070 0x7f40c406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40c93f9700 1 -- 172.21.15.7:0/1012642101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c415a070 msgr2=0x7f40c406e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40c93f9700 1 --2- 172.21.15.7:0/1012642101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c415a070 0x7f40c406e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40c93f9700 1 -- 172.21.15.7:0/1012642101 >> v2:172.21.15.113:3300/0 conn(0x7f40c40ce7f0 msgr2=0x7f40c40cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40c93f9700 1 --2- 172.21.15.7:0/1012642101 >> v2:172.21.15.113:3300/0 conn(0x7f40c40ce7f0 0x7f40c40cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40c93f9700 1 -- 172.21.15.7:0/1012642101 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40cf400 con 0x7f40c4159ca0 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40c93f9700 1 --2- 172.21.15.7:0/1012642101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c4159ca0 0x7f40c406dc30 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f40c4089fd0 tx=0x7f40c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdc99df7a83a16d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40b7fff700 1 -- 172.21.15.7:0/1012642101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c000b640 con 0x7f40c4159ca0 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40b7fff700 1 -- 172.21.15.7:0/1012642101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c000b7a0 con 0x7f40c4159ca0 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.325+0000 7f40b7fff700 1 -- 172.21.15.7:0/1012642101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c000bb60 con 0x7f40c4159ca0 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 -- 172.21.15.7:0/1012642101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c4159ca0 msgr2=0x7f40c406dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 --2- 172.21.15.7:0/1012642101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c4159ca0 0x7f40c406dc30 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f40c4089fd0 tx=0x7f40c0009250 comp rx=0 tx=0).stop 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 -- 172.21.15.7:0/1012642101 shutdown_connections 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 --2- 172.21.15.7:0/1012642101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c4159ca0 0x7f40c406dc30 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 --2- 172.21.15.7:0/1012642101 >> v2:172.21.15.113:3300/0 conn(0x7f40c40ce7f0 0x7f40c40cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 --2- 172.21.15.7:0/1012642101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c415a070 0x7f40c406e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 -- 172.21.15.7:0/1012642101 >> 172.21.15.7:0/1012642101 conn(0x7f40c4063cc0 msgr2=0x7f40c40d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 -- 172.21.15.7:0/1012642101 shutdown_connections 2024-01-11T07:18:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.326+0000 7f40cb65d700 1 -- 172.21.15.7:0/1012642101 wait complete. 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.327+0000 7f40cb65d700 1 Processor -- start 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.327+0000 7f40cb65d700 1 -- start start 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40cb65d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40ce7f0 0x7f40c414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40cb65d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c4159ca0 0x7f40c40c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40cb65d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c415a070 0x7f40c40c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40cb65d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c98b0 con 0x7f40c415a070 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40cb65d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c414fd70 con 0x7f40c4159ca0 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40cb65d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c8c30 con 0x7f40c40ce7f0 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40c9bfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c415a070 0x7f40c40c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40c9bfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c415a070 0x7f40c40c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49996/0 (socket says 172.21.15.7:49996) 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40c9bfa700 1 -- 172.21.15.7:0/3040591755 learned_addr learned my addr 172.21.15.7:0/3040591755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40c93f9700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40ce7f0 0x7f40c414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.328+0000 7f40c8bf8700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c4159ca0 0x7f40c40c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40c9bfa700 1 -- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40ce7f0 msgr2=0x7f40c414f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40c9bfa700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40ce7f0 0x7f40c414f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40c9bfa700 1 -- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c4159ca0 msgr2=0x7f40c40c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40c9bfa700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c4159ca0 0x7f40c40c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40c9bfa700 1 -- 172.21.15.7:0/3040591755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00094f0 con 0x7f40c415a070 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40c9bfa700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c415a070 0x7f40c40c6bb0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009e90 tx=0x7f40bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40b57fa700 1 -- 172.21.15.7:0/3040591755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00cb70 con 0x7f40c415a070 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40b57fa700 1 -- 172.21.15.7:0/3040591755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00f040 con 0x7f40c415a070 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c40c8f10 con 0x7f40c415a070 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40b57fa700 1 -- 172.21.15.7:0/3040591755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc014480 con 0x7f40c415a070 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.329+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c40c94d0 con 0x7f40c415a070 2024-01-11T07:18:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.330+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c4071ba0 con 0x7f40c415a070 2024-01-11T07:18:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.331+0000 7f40b57fa700 1 -- 172.21.15.7:0/3040591755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f40bc014830 con 0x7f40c415a070 2024-01-11T07:18:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.331+0000 7f40b57fa700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f40a8066f30 0x7f40a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.331+0000 7f40b57fa700 1 -- 172.21.15.7:0/3040591755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f40bc0851e0 con 0x7f40c415a070 2024-01-11T07:18:44.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.331+0000 7f40c93f9700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f40a8066f30 0x7f40a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.332+0000 7f40c93f9700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f40a8066f30 0x7f40a80693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f40c4089fd0 tx=0x7f40c0017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.336+0000 7f40b57fa700 1 -- 172.21.15.7:0/3040591755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40bc052450 con 0x7f40c415a070 2024-01-11T07:18:44.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.589+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40c40c99f0 con 0x7f40c415a070 2024-01-11T07:18:44.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.589+0000 7f40b57fa700 1 -- 172.21.15.7:0/3040591755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f40c40c99f0 con 0x7f40c415a070 2024-01-11T07:18:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:44.590 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.591+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f40a8066f30 msgr2=0x7f40a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.591+0000 7f40cb65d700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f40a8066f30 0x7f40a80693e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f40c4089fd0 tx=0x7f40c0017000 comp rx=0 tx=0).stop 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.591+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c415a070 msgr2=0x7f40c40c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.591+0000 7f40cb65d700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c415a070 0x7f40c40c6bb0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009e90 tx=0x7f40bc00b040 comp rx=0 tx=0).stop 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.592+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 shutdown_connections 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.592+0000 7f40cb65d700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f40a8066f30 0x7f40a80693e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.592+0000 7f40cb65d700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40c415a070 0x7f40c40c6bb0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.592+0000 7f40cb65d700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f40c40ce7f0 0x7f40c414f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.592+0000 7f40cb65d700 1 --2- 172.21.15.7:0/3040591755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40c4159ca0 0x7f40c40c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.592+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 >> 172.21.15.7:0/3040591755 conn(0x7f40c4063cc0 msgr2=0x7f40c406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.592+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 shutdown_connections 2024-01-11T07:18:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.592+0000 7f40cb65d700 1 -- 172.21.15.7:0/3040591755 wait complete. 2024-01-11T07:18:44.593 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:44.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.737+0000 7f833e36e700 1 Processor -- start 2024-01-11T07:18:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.737+0000 7f833e36e700 1 -- start start 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f833e36e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83380aff20 0x7f83380aab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f833e36e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8338063230 0x7f83380636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f833e36e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8338063be0 con 0x7f833806dae0 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f833e36e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83380b02f0 con 0x7f8338063230 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f833e36e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8338154860 con 0x7f83380aff20 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f8337fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83380aff20 0x7f83380aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f83377fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8338063230 0x7f83380636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f83377fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8338063230 0x7f83380636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:50444/0 (socket says 172.21.15.7:50444) 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f8337fff700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83380aff20 0x7f83380aab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:43774/0 (socket says 172.21.15.7:43774) 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f83377fe700 1 -- 172.21.15.7:0/900123497 learned_addr learned my addr 172.21.15.7:0/900123497 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.738+0000 7f833c90b700 1 --1- 172.21.15.7:0/900123497 >> v1:172.21.15.113:6789/0 conn(0x7f833806dae0 0x7f833806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:52924/0 (socket says 172.21.15.7:52924) 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8337fff700 1 -- 172.21.15.7:0/900123497 >> v2:172.21.15.113:3300/0 conn(0x7f8338063230 msgr2=0x7f83380636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8337fff700 1 --2- 172.21.15.7:0/900123497 >> v2:172.21.15.113:3300/0 conn(0x7f8338063230 0x7f83380636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8337fff700 1 -- 172.21.15.7:0/900123497 >> v1:172.21.15.113:6789/0 conn(0x7f833806dae0 legacy=0x7f833806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8337fff700 1 -- 172.21.15.7:0/900123497 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83380ab040 con 0x7f83380aff20 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8336ffd700 1 -- 172.21.15.7:0/900123497 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970269237 0 0) 0x7f8338063be0 con 0x7f833806dae0 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f83377fe700 1 --2- 172.21.15.7:0/900123497 >> v2:172.21.15.113:3300/0 conn(0x7f8338063230 0x7f83380636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8337fff700 1 --2- 172.21.15.7:0/900123497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83380aff20 0x7f83380aab00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83280098e0 tx=0x7f83280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7e7e23e11fa18f server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8336ffd700 1 -- 172.21.15.7:0/900123497 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800b640 con 0x7f83380aff20 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8336ffd700 1 -- 172.21.15.7:0/900123497 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832800b7a0 con 0x7f83380aff20 2024-01-11T07:18:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.739+0000 7f8336ffd700 1 -- 172.21.15.7:0/900123497 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832800bb40 con 0x7f83380aff20 2024-01-11T07:18:44.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.740+0000 7f833e36e700 1 -- 172.21.15.7:0/900123497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83380aff20 msgr2=0x7f83380aab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.740+0000 7f833e36e700 1 --2- 172.21.15.7:0/900123497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83380aff20 0x7f83380aab00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83280098e0 tx=0x7f83280091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.740+0000 7f833e36e700 1 -- 172.21.15.7:0/900123497 shutdown_connections 2024-01-11T07:18:44.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.740+0000 7f833e36e700 1 --2- 172.21.15.7:0/900123497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83380aff20 0x7f83380aab00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.740+0000 7f833e36e700 1 --2- 172.21.15.7:0/900123497 >> v2:172.21.15.113:3300/0 conn(0x7f8338063230 0x7f83380636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.740+0000 7f833e36e700 1 -- 172.21.15.7:0/900123497 >> 172.21.15.7:0/900123497 conn(0x7f8338068090 msgr2=0x7f83380cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.740+0000 7f833e36e700 1 -- 172.21.15.7:0/900123497 shutdown_connections 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.740+0000 7f833e36e700 1 -- 172.21.15.7:0/900123497 wait complete. 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.741+0000 7f833e36e700 1 Processor -- start 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.741+0000 7f833e36e700 1 -- start start 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f833e36e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8338063230 0x7f83380ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f833e36e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f833806dae0 0x7f8338069bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f833e36e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83380aff20 0x7f83380ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f833e36e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833806afb0 con 0x7f833806dae0 2024-01-11T07:18:44.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f833e36e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833806a100 con 0x7f83380aff20 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f833e36e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833806a3a0 con 0x7f8338063230 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f83377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f833806dae0 0x7f8338069bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f83377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f833806dae0 0x7f8338069bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50008/0 (socket says 172.21.15.7:50008) 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f83377fe700 1 -- 172.21.15.7:0/3504532664 learned_addr learned my addr 172.21.15.7:0/3504532664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f8337fff700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8338063230 0x7f83380ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.742+0000 7f833c90b700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83380aff20 0x7f83380ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f83377fe700 1 -- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8338063230 msgr2=0x7f83380ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f83377fe700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8338063230 0x7f83380ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f83377fe700 1 -- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83380aff20 msgr2=0x7f83380ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f83377fe700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83380aff20 0x7f83380ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f83377fe700 1 -- 172.21.15.7:0/3504532664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0094f0 con 0x7f833806dae0 2024-01-11T07:18:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f8337fff700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8338063230 0x7f83380ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f833c90b700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83380aff20 0x7f83380ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f83377fe700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f833806dae0 0x7f8338069bc0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f832c00ec50 tx=0x7f832c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f831ffff700 1 -- 172.21.15.7:0/3504532664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c00c9d0 con 0x7f833806dae0 2024-01-11T07:18:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f831ffff700 1 -- 172.21.15.7:0/3504532664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c00cb30 con 0x7f833806dae0 2024-01-11T07:18:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f831ffff700 1 -- 172.21.15.7:0/3504532664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c013470 con 0x7f833806dae0 2024-01-11T07:18:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8328009420 con 0x7f833806dae0 2024-01-11T07:18:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.743+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833806a960 con 0x7f833806dae0 2024-01-11T07:18:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.744+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833804c3c0 con 0x7f833806dae0 2024-01-11T07:18:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.744+0000 7f831ffff700 1 -- 172.21.15.7:0/3504532664 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f832c013860 con 0x7f833806dae0 2024-01-11T07:18:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.745+0000 7f831ffff700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8324067000 0x7f83240694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.745+0000 7f831ffff700 1 -- 172.21.15.7:0/3504532664 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f832c084be0 con 0x7f833806dae0 2024-01-11T07:18:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.745+0000 7f8337fff700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8324067000 0x7f83240694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:44.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.746+0000 7f8337fff700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8324067000 0x7f83240694b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f83280091e0 tx=0x7f8328017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.749+0000 7f831ffff700 1 -- 172.21.15.7:0/3504532664 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f832c051ed0 con 0x7f833806dae0 2024-01-11T07:18:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.954+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f833806b0f0 con 0x7f833806dae0 2024-01-11T07:18:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.954+0000 7f831ffff700 1 -- 172.21.15.7:0/3504532664 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+15324 (secure 0 0 0) 0x7f832c017090 con 0x7f833806dae0 2024-01-11T07:18:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:44.955 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":187,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:42.299913+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:18:11.360758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-11T07:18:11.823991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.956+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8324067000 msgr2=0x7f83240694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.956+0000 7f833e36e700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8324067000 0x7f83240694b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f83280091e0 tx=0x7f8328017000 comp rx=0 tx=0).stop 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.956+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f833806dae0 msgr2=0x7f8338069bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.956+0000 7f833e36e700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f833806dae0 0x7f8338069bc0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f832c00ec50 tx=0x7f832c009250 comp rx=0 tx=0).stop 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.957+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 shutdown_connections 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.957+0000 7f833e36e700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8324067000 0x7f83240694b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.957+0000 7f833e36e700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83380aff20 0x7f83380ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.957+0000 7f833e36e700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8338063230 0x7f83380ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.957+0000 7f833e36e700 1 --2- 172.21.15.7:0/3504532664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f833806dae0 0x7f8338069bc0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.957+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 >> 172.21.15.7:0/3504532664 conn(0x7f8338068090 msgr2=0x7f83380cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.957+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 shutdown_connections 2024-01-11T07:18:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:44.957+0000 7f833e36e700 1 -- 172.21.15.7:0/3504532664 wait complete. 2024-01-11T07:18:44.970 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-11T07:06:49.881877+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-11T07:18:11.360758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-01-11T07:18:11.823991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-11T07:18:44.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-11T07:18:45.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.105+0000 7f6bc8cd1700 1 Processor -- start 2024-01-11T07:18:45.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.105+0000 7f6bc8cd1700 1 -- start start 2024-01-11T07:18:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.106+0000 7f6bc8cd1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6bc4157940 0x7f6bc40ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.106+0000 7f6bc8cd1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc40ce580 0x7f6bc40caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.106+0000 7f6bc8cd1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc415c060 con 0x7f6bc4157d10 2024-01-11T07:18:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.106+0000 7f6bc8cd1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc40cd5d0 con 0x7f6bc4157940 2024-01-11T07:18:45.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.106+0000 7f6bc8cd1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc415b750 con 0x7f6bc40ce580 2024-01-11T07:18:45.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc259c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6bc4157940 0x7f6bc40ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc2d9d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc40ce580 0x7f6bc40caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc1d9b700 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6bc4157d10 0x7f6bc40cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:52932/0 (socket says 172.21.15.7:52932) 2024-01-11T07:18:45.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc1d9b700 1 -- 172.21.15.7:0/1086897193 learned_addr learned my addr 172.21.15.7:0/1086897193 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:45.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc2d9d700 1 -- 172.21.15.7:0/1086897193 >> v2:172.21.15.113:3300/0 conn(0x7f6bc4157940 msgr2=0x7f6bc40ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:45.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc2d9d700 1 --2- 172.21.15.7:0/1086897193 >> v2:172.21.15.113:3300/0 conn(0x7f6bc4157940 0x7f6bc40ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc2d9d700 1 -- 172.21.15.7:0/1086897193 >> v1:172.21.15.113:6789/0 conn(0x7f6bc4157d10 legacy=0x7f6bc40cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc2d9d700 1 -- 172.21.15.7:0/1086897193 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40cb0c0 con 0x7f6bc40ce580 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc259c700 1 --2- 172.21.15.7:0/1086897193 >> v2:172.21.15.113:3300/0 conn(0x7f6bc4157940 0x7f6bc40ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc2d9d700 1 --2- 172.21.15.7:0/1086897193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc40ce580 0x7f6bc40caab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009e60 tx=0x7f6bbc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee513abe74c8b50 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc159a700 1 -- 172.21.15.7:0/1086897193 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00e070 con 0x7f6bc40ce580 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc159a700 1 -- 172.21.15.7:0/1086897193 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc00bb60 con 0x7f6bc40ce580 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.107+0000 7f6bc159a700 1 -- 172.21.15.7:0/1086897193 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc012430 con 0x7f6bc40ce580 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.108+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/1086897193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc40ce580 msgr2=0x7f6bc40caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.108+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/1086897193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc40ce580 0x7f6bc40caab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009e60 tx=0x7f6bbc009780 comp rx=0 tx=0).stop 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.108+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/1086897193 shutdown_connections 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.108+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/1086897193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc40ce580 0x7f6bc40caab0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.108+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/1086897193 >> v2:172.21.15.113:3300/0 conn(0x7f6bc4157940 0x7f6bc40ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.108+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/1086897193 >> 172.21.15.7:0/1086897193 conn(0x7f6bc406ae00 msgr2=0x7f6bc406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:45.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.108+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/1086897193 shutdown_connections 2024-01-11T07:18:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.108+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/1086897193 wait complete. 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.109+0000 7f6bc8cd1700 1 Processor -- start 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.109+0000 7f6bc8cd1700 1 -- start start 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc8cd1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc40ce580 0x7f6bc40c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc8cd1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc4157940 0x7f6bc40c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc8cd1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc4157d10 0x7f6bc4146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc8cd1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40c8dd0 con 0x7f6bc40ce580 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc8cd1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40c7750 con 0x7f6bc4157d10 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc8cd1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40c7be0 con 0x7f6bc4157940 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc40ce580 0x7f6bc40c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc40ce580 0x7f6bc40c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50016/0 (socket says 172.21.15.7:50016) 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc259c700 1 -- 172.21.15.7:0/426128563 learned_addr learned my addr 172.21.15.7:0/426128563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc2d9d700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc4157d10 0x7f6bc4146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc1d9b700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc4157940 0x7f6bc40c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc259c700 1 -- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc4157940 msgr2=0x7f6bc40c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc259c700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc4157940 0x7f6bc40c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc259c700 1 -- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc4157d10 msgr2=0x7f6bc4146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc259c700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc4157d10 0x7f6bc4146360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.110+0000 7f6bc259c700 1 -- 172.21.15.7:0/426128563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4009420 con 0x7f6bc40ce580 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.111+0000 7f6bc259c700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc40ce580 0x7f6bc40c6460 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400ec80 tx=0x7f6bb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.111+0000 7f6bba7fc700 1 -- 172.21.15.7:0/426128563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400c920 con 0x7f6bc40ce580 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.111+0000 7f6bba7fc700 1 -- 172.21.15.7:0/426128563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb400ca80 con 0x7f6bc40ce580 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.111+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc009a20 con 0x7f6bc40ce580 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.111+0000 7f6bba7fc700 1 -- 172.21.15.7:0/426128563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400ce00 con 0x7f6bc40ce580 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.111+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc40c6e20 con 0x7f6bc40ce580 2024-01-11T07:18:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.112+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc404c3c0 con 0x7f6bc40ce580 2024-01-11T07:18:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.113+0000 7f6bba7fc700 1 -- 172.21.15.7:0/426128563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6bb4013680 con 0x7f6bc40ce580 2024-01-11T07:18:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.113+0000 7f6bba7fc700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ba4066f30 0x7f6ba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.113+0000 7f6bba7fc700 1 -- 172.21.15.7:0/426128563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6bb4084ca0 con 0x7f6bc40ce580 2024-01-11T07:18:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.113+0000 7f6bc1d9b700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ba4066f30 0x7f6ba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.114+0000 7f6bc1d9b700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ba4066f30 0x7f6ba40693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40c53d0 tx=0x7f6bac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.117+0000 7f6bba7fc700 1 -- 172.21.15.7:0/426128563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bb4051f10 con 0x7f6bc40ce580 2024-01-11T07:18:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.319+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6bc415ba10 con 0x7f6bc40ce580 2024-01-11T07:18:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.320+0000 7f6bba7fc700 1 -- 172.21.15.7:0/426128563 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v187) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6bb4055530 con 0x7f6bc40ce580 2024-01-11T07:18:45.320 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-01-11T07:18:45.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ba4066f30 msgr2=0x7f6ba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ba4066f30 0x7f6ba40693e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40c53d0 tx=0x7f6bac009040 comp rx=0 tx=0).stop 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc40ce580 msgr2=0x7f6bc40c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc40ce580 0x7f6bc40c6460 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400ec80 tx=0x7f6bb40091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 shutdown_connections 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ba4066f30 0x7f6ba40693e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bc4157d10 0x7f6bc4146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc40ce580 0x7f6bc40c6460 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 --2- 172.21.15.7:0/426128563 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6bc4157940 0x7f6bc40c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 >> 172.21.15.7:0/426128563 conn(0x7f6bc406ae00 msgr2=0x7f6bc4152760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 shutdown_connections 2024-01-11T07:18:45.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.322+0000 7f6bc8cd1700 1 -- 172.21.15.7:0/426128563 wait complete. 2024-01-11T07:18:45.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-11T07:18:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.472+0000 7f174b31c700 1 Processor -- start 2024-01-11T07:18:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.472+0000 7f174b31c700 1 -- start start 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f174b31c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17440afdc0 0x7f17440b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f174b31c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f174415c420 0x7f174406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f174b31c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174406b840 con 0x7f174406ae90 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f174b31c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174406eb20 con 0x7f17440afdc0 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f174b31c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174415c840 con 0x7f174415c420 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f17490b8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17440afdc0 0x7f17440b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f17488b7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f174406ae90 0x7f174415bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48960/0 (socket says 172.21.15.7:48960) 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f17488b7700 1 -- 172.21.15.7:0/2569190816 learned_addr learned my addr 172.21.15.7:0/2569190816 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f17498b9700 1 --2- 172.21.15.7:0/2569190816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f174415c420 0x7f174406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f17490b8700 1 -- 172.21.15.7:0/2569190816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f174415c420 msgr2=0x7f174406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f17490b8700 1 --2- 172.21.15.7:0/2569190816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f174415c420 0x7f174406b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f17490b8700 1 -- 172.21.15.7:0/2569190816 >> v1:172.21.15.7:6789/0 conn(0x7f174406ae90 legacy=0x7f174415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f17490b8700 1 -- 172.21.15.7:0/2569190816 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174406df10 con 0x7f17440afdc0 2024-01-11T07:18:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.473+0000 7f173bfff700 1 -- 172.21.15.7:0/2569190816 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303602192 0 0) 0x7f174406b840 con 0x7f174406ae90 2024-01-11T07:18:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f17490b8700 1 --2- 172.21.15.7:0/2569190816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17440afdc0 0x7f17440b0760 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f17400098e0 tx=0x7f17400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d3a3756fd31d07 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f173bfff700 1 -- 172.21.15.7:0/2569190816 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174000b640 con 0x7f17440afdc0 2024-01-11T07:18:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f173bfff700 1 -- 172.21.15.7:0/2569190816 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174000b7a0 con 0x7f17440afdc0 2024-01-11T07:18:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f173bfff700 1 -- 172.21.15.7:0/2569190816 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174000bb40 con 0x7f17440afdc0 2024-01-11T07:18:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f174b31c700 1 -- 172.21.15.7:0/2569190816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17440afdc0 msgr2=0x7f17440b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f174b31c700 1 --2- 172.21.15.7:0/2569190816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17440afdc0 0x7f17440b0760 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f17400098e0 tx=0x7f17400091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f174b31c700 1 -- 172.21.15.7:0/2569190816 shutdown_connections 2024-01-11T07:18:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f174b31c700 1 --2- 172.21.15.7:0/2569190816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f174415c420 0x7f174406b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.474+0000 7f174b31c700 1 --2- 172.21.15.7:0/2569190816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17440afdc0 0x7f17440b0760 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.475+0000 7f174b31c700 1 -- 172.21.15.7:0/2569190816 >> 172.21.15.7:0/2569190816 conn(0x7f17440680e0 msgr2=0x7f17440cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.475+0000 7f174b31c700 1 -- 172.21.15.7:0/2569190816 shutdown_connections 2024-01-11T07:18:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.475+0000 7f174b31c700 1 -- 172.21.15.7:0/2569190816 wait complete. 2024-01-11T07:18:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.476+0000 7f174b31c700 1 Processor -- start 2024-01-11T07:18:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.476+0000 7f174b31c700 1 -- start start 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f174b31c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f174406ae90 0x7f1744156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f174b31c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17440afdc0 0x7f17440c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f174b31c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f174415c420 0x7f1744158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f174b31c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441574e0 con 0x7f174415c420 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f174b31c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174416eb70 con 0x7f174406ae90 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f174b31c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174416dfa0 con 0x7f17440afdc0 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17498b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f174415c420 0x7f1744158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17498b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f174415c420 0x7f1744158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50034/0 (socket says 172.21.15.7:50034) 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17498b9700 1 -- 172.21.15.7:0/2131467261 learned_addr learned my addr 172.21.15.7:0/2131467261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17488b7700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17440afdc0 0x7f17440c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17490b8700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f174406ae90 0x7f1744156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17498b9700 1 -- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17440afdc0 msgr2=0x7f17440c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17498b9700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17440afdc0 0x7f17440c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17498b9700 1 -- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f174406ae90 msgr2=0x7f1744156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17498b9700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f174406ae90 0x7f1744156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.477+0000 7f17498b9700 1 -- 172.21.15.7:0/2131467261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740009420 con 0x7f174415c420 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.478+0000 7f17498b9700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f174415c420 0x7f1744158910 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f173c009df0 tx=0x7f173c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.478+0000 7f17397fa700 1 -- 172.21.15.7:0/2131467261 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c00caa0 con 0x7f174415c420 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.478+0000 7f17397fa700 1 -- 172.21.15.7:0/2131467261 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c00f040 con 0x7f174415c420 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.478+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174416e240 con 0x7f174415c420 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.478+0000 7f17397fa700 1 -- 172.21.15.7:0/2131467261 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c00ce70 con 0x7f174415c420 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.478+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174416e800 con 0x7f174415c420 2024-01-11T07:18:45.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.479+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174406a920 con 0x7f174415c420 2024-01-11T07:18:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.480+0000 7f17397fa700 1 -- 172.21.15.7:0/2131467261 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f173c014690 con 0x7f174415c420 2024-01-11T07:18:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.480+0000 7f17397fa700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1728066f30 0x7f17280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.480+0000 7f17397fa700 1 -- 172.21.15.7:0/2131467261 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f173c084d40 con 0x7f174415c420 2024-01-11T07:18:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.480+0000 7f17490b8700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1728066f30 0x7f17280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.481+0000 7f17490b8700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1728066f30 0x7f17280693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f17400093f0 tx=0x7f1740017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.484+0000 7f17397fa700 1 -- 172.21.15.7:0/2131467261 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f173c055a40 con 0x7f174415c420 2024-01-11T07:18:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.687+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1744064a40 con 0x7f174415c420 2024-01-11T07:18:45.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.687+0000 7f17397fa700 1 -- 172.21.15.7:0/2131467261 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v187) v1 ==== 104+0+11 (secure 0 0 0) 0x7f173c0555d0 con 0x7f174415c420 2024-01-11T07:18:45.687 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.689+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1728066f30 msgr2=0x7f17280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.689+0000 7f174b31c700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1728066f30 0x7f17280693e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f17400093f0 tx=0x7f1740017000 comp rx=0 tx=0).stop 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.689+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f174415c420 msgr2=0x7f1744158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f174415c420 0x7f1744158910 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f173c009df0 tx=0x7f173c00b040 comp rx=0 tx=0).stop 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 shutdown_connections 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1728066f30 0x7f17280693e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f174415c420 0x7f1744158910 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f174406ae90 0x7f1744156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 --2- 172.21.15.7:0/2131467261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f17440afdc0 0x7f17440c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 >> 172.21.15.7:0/2131467261 conn(0x7f17440680e0 msgr2=0x7f1744157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 shutdown_connections 2024-01-11T07:18:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.690+0000 7f174b31c700 1 -- 172.21.15.7:0/2131467261 wait complete. 2024-01-11T07:18:45.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-11T07:18:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.838+0000 7f5d0e937700 1 Processor -- start 2024-01-11T07:18:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.838+0000 7f5d0e937700 1 -- start start 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d0e937700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d0806e350 0x7f5d080c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d0e937700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0806a940 con 0x7f5d08156a80 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d0e937700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d080c8dc0 con 0x7f5d080c85f0 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d0e937700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d080c90a0 con 0x7f5d0806e350 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d0806e350 0x7f5d080c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d07fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5d08156a80 0x7f5d0806dc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48968/0 (socket says 172.21.15.7:48968) 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d07fff700 1 -- 172.21.15.7:0/1322390598 learned_addr learned my addr 172.21.15.7:0/1322390598 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d077fe700 1 -- 172.21.15.7:0/1322390598 >> v1:172.21.15.113:6789/0 conn(0x7f5d080c85f0 legacy=0x7f5d0806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d077fe700 1 -- 172.21.15.7:0/1322390598 >> v1:172.21.15.7:6789/0 conn(0x7f5d08156a80 legacy=0x7f5d0806dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d077fe700 1 -- 172.21.15.7:0/1322390598 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d080c9380 con 0x7f5d0806e350 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.839+0000 7f5d06ffd700 1 -- 172.21.15.7:0/1322390598 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477536701 0 0) 0x7f5d0806a940 con 0x7f5d08156a80 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d077fe700 1 --2- 172.21.15.7:0/1322390598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d0806e350 0x7f5d080c80b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00098e0 tx=0x7f5cf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efd2acdc22fb0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d06ffd700 1 -- 172.21.15.7:0/1322390598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0009e80 con 0x7f5d0806e350 2024-01-11T07:18:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d06ffd700 1 -- 172.21.15.7:0/1322390598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf000b860 con 0x7f5d0806e350 2024-01-11T07:18:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d06ffd700 1 -- 172.21.15.7:0/1322390598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000bbe0 con 0x7f5d0806e350 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d0e937700 1 -- 172.21.15.7:0/1322390598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d0806e350 msgr2=0x7f5d080c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d0e937700 1 --2- 172.21.15.7:0/1322390598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d0806e350 0x7f5d080c80b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00098e0 tx=0x7f5cf00091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d0e937700 1 -- 172.21.15.7:0/1322390598 shutdown_connections 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d0e937700 1 --2- 172.21.15.7:0/1322390598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d0806e350 0x7f5d080c80b0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d0e937700 1 -- 172.21.15.7:0/1322390598 >> 172.21.15.7:0/1322390598 conn(0x7f5d080cd860 msgr2=0x7f5d080cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.840+0000 7f5d0e937700 1 -- 172.21.15.7:0/1322390598 shutdown_connections 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.841+0000 7f5d0e937700 1 -- 172.21.15.7:0/1322390598 wait complete. 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.842+0000 7f5d0e937700 1 Processor -- start 2024-01-11T07:18:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.842+0000 7f5d0e937700 1 -- start start 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0e937700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d0806e350 0x7f5d08149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0e937700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d080c85f0 0x7f5d080aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0e937700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d08156a80 0x7f5d080cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0e937700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080abf20 con 0x7f5d08156a80 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0e937700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080ab310 con 0x7f5d0806e350 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0e937700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080ab5f0 con 0x7f5d080c85f0 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d08156a80 0x7f5d080cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d08156a80 0x7f5d080cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50050/0 (socket says 172.21.15.7:50050) 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 -- 172.21.15.7:0/673063136 learned_addr learned my addr 172.21.15.7:0/673063136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d07fff700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d0806e350 0x7f5d08149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d077fe700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d080c85f0 0x7f5d080aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 -- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d080c85f0 msgr2=0x7f5d080aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d080c85f0 0x7f5d080aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 -- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d0806e350 msgr2=0x7f5d08149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d0806e350 0x7f5d08149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 -- 172.21.15.7:0/673063136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0009420 con 0x7f5d08156a80 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.843+0000 7f5d0ced4700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d08156a80 0x7f5d080cc4b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c8e0 tx=0x7f5d00001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.844+0000 7f5cfffff700 1 -- 172.21.15.7:0/673063136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00012630 con 0x7f5d08156a80 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.844+0000 7f5cfffff700 1 -- 172.21.15.7:0/673063136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00012790 con 0x7f5d08156a80 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.844+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d080ab8d0 con 0x7f5d08156a80 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.844+0000 7f5cfffff700 1 -- 172.21.15.7:0/673063136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00012b50 con 0x7f5d08156a80 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.844+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d080ae380 con 0x7f5d08156a80 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.845+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0804c3c0 con 0x7f5d08156a80 2024-01-11T07:18:45.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.845+0000 7f5cfffff700 1 -- 172.21.15.7:0/673063136 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5d0001a3a0 con 0x7f5d08156a80 2024-01-11T07:18:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.846+0000 7f5cfffff700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ce8066f30 0x7f5ce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.846+0000 7f5cfffff700 1 -- 172.21.15.7:0/673063136 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d00084790 con 0x7f5d08156a80 2024-01-11T07:18:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.846+0000 7f5d07fff700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ce8066f30 0x7f5ce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:45.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.846+0000 7f5d07fff700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ce8066f30 0x7f5ce80693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8007fd0 tx=0x7f5cf8007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:45.850+0000 7f5cfffff700 1 -- 172.21.15.7:0/673063136 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d000553f0 con 0x7f5d08156a80 2024-01-11T07:18:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.052+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d081543b0 con 0x7f5d08156a80 2024-01-11T07:18:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.052+0000 7f5cfffff700 1 -- 172.21.15.7:0/673063136 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v187) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d00054f80 con 0x7f5d08156a80 2024-01-11T07:18:46.052 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.054+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ce8066f30 msgr2=0x7f5ce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.054+0000 7f5d0e937700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ce8066f30 0x7f5ce80693e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8007fd0 tx=0x7f5cf8007d10 comp rx=0 tx=0).stop 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.054+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d08156a80 msgr2=0x7f5d080cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.054+0000 7f5d0e937700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d08156a80 0x7f5d080cc4b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000c8e0 tx=0x7f5d00001710 comp rx=0 tx=0).stop 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.055+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 shutdown_connections 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.055+0000 7f5d0e937700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5ce8066f30 0x7f5ce80693e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.055+0000 7f5d0e937700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d08156a80 0x7f5d080cc4b0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.055+0000 7f5d0e937700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d0806e350 0x7f5d08149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.055+0000 7f5d0e937700 1 --2- 172.21.15.7:0/673063136 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d080c85f0 0x7f5d080aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.055+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 >> 172.21.15.7:0/673063136 conn(0x7f5d080cd860 msgr2=0x7f5d081545d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.055+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 shutdown_connections 2024-01-11T07:18:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.055+0000 7f5d0e937700 1 -- 172.21.15.7:0/673063136 wait complete. 2024-01-11T07:18:46.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.200+0000 7f78aca53700 1 Processor -- start 2024-01-11T07:18:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.200+0000 7f78aca53700 1 -- start start 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.201+0000 7f78aca53700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f78a8156760 0x7f78a806e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.201+0000 7f78aca53700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a806e3f0 con 0x7f78a815b260 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.201+0000 7f78aca53700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a80b1f10 con 0x7f78a8069c80 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.201+0000 7f78aca53700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80b1d20 con 0x7f78a8156760 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.201+0000 7f789ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f78a815b260 0x7f78a80b1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48970/0 (socket says 172.21.15.7:48970) 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.201+0000 7f789ffff700 1 -- 172.21.15.7:0/131185980 learned_addr learned my addr 172.21.15.7:0/131185980 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.201+0000 7f78a659c700 1 --2- 172.21.15.7:0/131185980 >> v2:172.21.15.113:3300/0 conn(0x7f78a8156760 0x7f78a806e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.201+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969707050 0 0) 0x7f78a806e3f0 con 0x7f78a815b260 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7894003570 con 0x7f78a815b260 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 219246101 0 0) 0x7f7894003570 con 0x7f78a815b260 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 >> v2:172.21.15.113:3300/0 conn(0x7f78a8156760 msgr2=0x7f78a806e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 --2- 172.21.15.7:0/131185980 >> v2:172.21.15.113:3300/0 conn(0x7f78a8156760 0x7f78a806e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 >> v1:172.21.15.113:6789/0 conn(0x7f78a8069c80 legacy=0x7f78a8156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a806dda0 con 0x7f78a815b260 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f7890001470 con 0x7f78a815b260 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 >> v1:172.21.15.7:6789/0 conn(0x7f78a815b260 legacy=0x7f78a80b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7894003a60 0x7f7894003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7894004810 0x7f7894004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78940051c0 0x7f789400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789400c050 con 0x7f7894003a60 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789400c190 con 0x7f7894004810 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789400c350 con 0x7f78940051c0 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a659c700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7894003a60 0x7f7894003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f789ffff700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7894004810 0x7f7894004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a6d9d700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78940051c0 0x7f789400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a659c700 1 -- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78940051c0 msgr2=0x7f789400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a659c700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78940051c0 0x7f789400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a659c700 1 -- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7894004810 msgr2=0x7f7894004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a659c700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7894004810 0x7f7894004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a659c700 1 -- 172.21.15.7:0/131185980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7894003570 con 0x7f7894003a60 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a6d9d700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f78940051c0 0x7f789400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.202+0000 7f78a6d9d700 1 -- 172.21.15.7:0/131185980 reap_dead start 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78a659c700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7894003a60 0x7f7894003e30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7898009c00 tx=0x7f78980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5148dd4dc87c581 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800e710 con 0x7f7894003a60 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789800e870 con 0x7f7894003a60 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78a5d9b700 1 -- 172.21.15.7:0/131185980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800ec30 con 0x7f7894003a60 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78aca53700 1 -- 172.21.15.7:0/131185980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7894003a60 msgr2=0x7f7894003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78aca53700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7894003a60 0x7f7894003e30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7898009c00 tx=0x7f78980091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78aca53700 1 -- 172.21.15.7:0/131185980 shutdown_connections 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78aca53700 1 --2- 172.21.15.7:0/131185980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7894003a60 0x7f7894003e30 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78aca53700 1 -- 172.21.15.7:0/131185980 >> 172.21.15.7:0/131185980 conn(0x7f78a80cd910 msgr2=0x7f78a80cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78aca53700 1 -- 172.21.15.7:0/131185980 shutdown_connections 2024-01-11T07:18:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.203+0000 7f78aca53700 1 -- 172.21.15.7:0/131185980 wait complete. 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.205+0000 7f78aca53700 1 Processor -- start 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.205+0000 7f78aca53700 1 -- start start 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.205+0000 7f78aca53700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7894003a60 0x7f78a815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.205+0000 7f78aca53700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a814d850 0x7f78a814e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.205+0000 7f78aca53700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a81668a0 0x7f78a8166d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.205+0000 7f78aca53700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80a8dd0 con 0x7f78a814d850 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.205+0000 7f78aca53700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8167580 con 0x7f78a81668a0 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.205+0000 7f78aca53700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8167820 con 0x7f7894003a60 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a814d850 0x7f78a814e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a814d850 0x7f78a814e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50066/0 (socket says 172.21.15.7:50066) 2024-01-11T07:18:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 -- 172.21.15.7:0/4095991290 learned_addr learned my addr 172.21.15.7:0/4095991290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f78a6d9d700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a81668a0 0x7f78a8166d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f78a659c700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7894003a60 0x7f78a815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 -- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7894003a60 msgr2=0x7f78a815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7894003a60 0x7f78a815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 -- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a81668a0 msgr2=0x7f78a8166d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a81668a0 0x7f78a8166d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 -- 172.21.15.7:0/4095991290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a814e820 con 0x7f78a814d850 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789ffff700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a814d850 0x7f78a814e2e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f78900073c0 tx=0x7f7890001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789e7fc700 1 -- 172.21.15.7:0/4095991290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000f870 con 0x7f78a814d850 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789e7fc700 1 -- 172.21.15.7:0/4095991290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789000f9f0 con 0x7f78a814d850 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a80a8a60 con 0x7f78a814d850 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f789e7fc700 1 -- 172.21.15.7:0/4095991290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000fdb0 con 0x7f78a814d850 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.206+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a814df90 con 0x7f78a814d850 2024-01-11T07:18:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.207+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a804c3c0 con 0x7f78a814d850 2024-01-11T07:18:46.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.208+0000 7f789e7fc700 1 -- 172.21.15.7:0/4095991290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7890013600 con 0x7f78a814d850 2024-01-11T07:18:46.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.208+0000 7f789e7fc700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7888066f30 0x7f78880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.208+0000 7f789e7fc700 1 -- 172.21.15.7:0/4095991290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7890084850 con 0x7f78a814d850 2024-01-11T07:18:46.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.208+0000 7f78a659c700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7888066f30 0x7f78880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.209+0000 7f78a659c700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7888066f30 0x7f78880693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f78a806dd00 tx=0x7f789801a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.213+0000 7f789e7fc700 1 -- 172.21.15.7:0/4095991290 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7890051a20 con 0x7f78a814d850 2024-01-11T07:18:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.469+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78a806e080 con 0x7f78a814d850 2024-01-11T07:18:46.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.469+0000 7f789e7fc700 1 -- 172.21.15.7:0/4095991290 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f7890055040 con 0x7f78a814d850 2024-01-11T07:18:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:46.469 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7888066f30 msgr2=0x7f78880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7888066f30 0x7f78880693e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f78a806dd00 tx=0x7f789801a000 comp rx=0 tx=0).stop 2024-01-11T07:18:46.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a814d850 msgr2=0x7f78a814e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a814d850 0x7f78a814e2e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f78900073c0 tx=0x7f7890001e70 comp rx=0 tx=0).stop 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 shutdown_connections 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7888066f30 0x7f78880693e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a81668a0 0x7f78a8166d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7894003a60 0x7f78a815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 --2- 172.21.15.7:0/4095991290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a814d850 0x7f78a814e2e0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 >> 172.21.15.7:0/4095991290 conn(0x7f78a80cd910 msgr2=0x7f78a8155330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 shutdown_connections 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.471+0000 7f78aca53700 1 -- 172.21.15.7:0/4095991290 wait complete. 2024-01-11T07:18:46.472 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:46.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.616+0000 7fdcb190b700 1 Processor -- start 2024-01-11T07:18:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.616+0000 7fdcb190b700 1 -- start start 2024-01-11T07:18:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcb190b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac0693a0 0x7fdcac06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcb190b700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdcac06f1d0 0x7fdcac0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcb190b700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcac15dbb0 con 0x7fdcac159d10 2024-01-11T07:18:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcb190b700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcac15d9a0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcb190b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac06f650 con 0x7fdcac0693a0 2024-01-11T07:18:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcaa7fc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac0693a0 0x7fdcac06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcab7fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdcac06f1d0 0x7fdcac0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcab7fe700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdcac06f1d0 0x7fdcac0b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:50528/0 (socket says 172.21.15.7:50528) 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcab7fe700 1 -- 172.21.15.7:0/1132830533 learned_addr learned my addr 172.21.15.7:0/1132830533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcaa7fc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac0693a0 0x7fdcac06ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:43866/0 (socket says 172.21.15.7:43866) 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.617+0000 7fdcaaffd700 1 --1- 172.21.15.7:0/1132830533 >> v1:172.21.15.113:6789/0 conn(0x7fdcac159d10 0x7fdcac068c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:52966/0 (socket says 172.21.15.7:52966) 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdcab7fe700 1 -- 172.21.15.7:0/1132830533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac0693a0 msgr2=0x7fdcac06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdcab7fe700 1 --2- 172.21.15.7:0/1132830533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac0693a0 0x7fdcac06ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdcab7fe700 1 -- 172.21.15.7:0/1132830533 >> v1:172.21.15.113:6789/0 conn(0x7fdcac159d10 legacy=0x7fdcac068c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdcab7fe700 1 -- 172.21.15.7:0/1132830533 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac15de90 con 0x7fdcac06f1d0 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdca9ffb700 1 -- 172.21.15.7:0/1132830533 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715733712 0 0) 0x7fdcac15dbb0 con 0x7fdcac159d10 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdcab7fe700 1 --2- 172.21.15.7:0/1132830533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 0x7fdcac0b3b80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009f20 tx=0x7fdca4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c25a9ae20b75c32d server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdca9ffb700 1 -- 172.21.15.7:0/1132830533 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca400e070 con 0x7fdcac06f1d0 2024-01-11T07:18:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdca9ffb700 1 -- 172.21.15.7:0/1132830533 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca400b9d0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.618+0000 7fdca9ffb700 1 -- 172.21.15.7:0/1132830533 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca40092e0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.619+0000 7fdcb190b700 1 -- 172.21.15.7:0/1132830533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 msgr2=0x7fdcac0b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.619+0000 7fdcb190b700 1 --2- 172.21.15.7:0/1132830533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 0x7fdcac0b3b80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009f20 tx=0x7fdca4009840 comp rx=0 tx=0).stop 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.619+0000 7fdcb190b700 1 -- 172.21.15.7:0/1132830533 shutdown_connections 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.619+0000 7fdcb190b700 1 --2- 172.21.15.7:0/1132830533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 0x7fdcac0b3b80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.619+0000 7fdcb190b700 1 --2- 172.21.15.7:0/1132830533 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac0693a0 0x7fdcac06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.619+0000 7fdcb190b700 1 -- 172.21.15.7:0/1132830533 >> 172.21.15.7:0/1132830533 conn(0x7fdcac0ce9f0 msgr2=0x7fdcac0cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.619+0000 7fdcb190b700 1 -- 172.21.15.7:0/1132830533 shutdown_connections 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.619+0000 7fdcb190b700 1 -- 172.21.15.7:0/1132830533 wait complete. 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.621+0000 7fdcb190b700 1 Processor -- start 2024-01-11T07:18:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.621+0000 7fdcb190b700 1 -- start start 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcb190b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcac0693a0 0x7fdcac158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcb190b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 0x7fdcac0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcb190b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac159d10 0x7fdcac15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcb190b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac0c2450 con 0x7fdcac0693a0 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcb190b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac0c2a10 con 0x7fdcac06f1d0 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcb190b700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac15bf20 con 0x7fdcac159d10 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcac0693a0 0x7fdcac158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcac0693a0 0x7fdcac158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50074/0 (socket says 172.21.15.7:50074) 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaa7fc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 0x7fdcac0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaaffd700 1 -- 172.21.15.7:0/2125574947 learned_addr learned my addr 172.21.15.7:0/2125574947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcab7fe700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac159d10 0x7fdcac15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaa7fc700 1 -- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac159d10 msgr2=0x7fdcac15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaa7fc700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac159d10 0x7fdcac15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaa7fc700 1 -- 172.21.15.7:0/2125574947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcac0693a0 msgr2=0x7fdcac158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaa7fc700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcac0693a0 0x7fdcac158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaa7fc700 1 -- 172.21.15.7:0/2125574947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca4009ae0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.622+0000 7fdcaa7fc700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 0x7fdcac0c7f40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400ea50 tx=0x7fdc940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.623+0000 7fdca2ffd700 1 -- 172.21.15.7:0/2125574947 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc940095d0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.623+0000 7fdca2ffd700 1 -- 172.21.15.7:0/2125574947 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9400c7f0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.623+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac15c200 con 0x7fdcac06f1d0 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.623+0000 7fdca2ffd700 1 -- 172.21.15.7:0/2125574947 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400cbb0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.623+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac158dc0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.624+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcac069770 con 0x7fdcac06f1d0 2024-01-11T07:18:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.624+0000 7fdca2ffd700 1 -- 172.21.15.7:0/2125574947 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fdc94013470 con 0x7fdcac06f1d0 2024-01-11T07:18:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.625+0000 7fdca2ffd700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdc8c066f30 0x7fdc8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.625+0000 7fdca2ffd700 1 -- 172.21.15.7:0/2125574947 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdc94084650 con 0x7fdcac06f1d0 2024-01-11T07:18:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.625+0000 7fdcaaffd700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdc8c066f30 0x7fdc8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.625+0000 7fdcaaffd700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdc8c066f30 0x7fdc8c0693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0b14f0 tx=0x7fdc9c001040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.630+0000 7fdca2ffd700 1 -- 172.21.15.7:0/2125574947 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc940518c0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.834+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcac04c3c0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.835+0000 7fdca2ffd700 1 -- 172.21.15.7:0/2125574947 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+15324 (secure 0 0 0) 0x7fdc94054ee0 con 0x7fdcac06f1d0 2024-01-11T07:18:46.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:46.835 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":187,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:42.299913+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:18:11.360758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-11T07:18:11.823991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdc8c066f30 msgr2=0x7fdc8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdc8c066f30 0x7fdc8c0693e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0b14f0 tx=0x7fdc9c001040 comp rx=0 tx=0).stop 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 msgr2=0x7fdcac0c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 0x7fdcac0c7f40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400ea50 tx=0x7fdc940091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 shutdown_connections 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdc8c066f30 0x7fdc8c0693e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdcac159d10 0x7fdcac15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcac0693a0 0x7fdcac158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 --2- 172.21.15.7:0/2125574947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdcac06f1d0 0x7fdcac0c7f40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 >> 172.21.15.7:0/2125574947 conn(0x7fdcac0ce9f0 msgr2=0x7fdcac068c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 shutdown_connections 2024-01-11T07:18:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.838+0000 7fdcb190b700 1 -- 172.21.15.7:0/2125574947 wait complete. 2024-01-11T07:18:46.851 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:18:46.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.983+0000 7fe93359e700 1 Processor -- start 2024-01-11T07:18:46.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.983+0000 7fe93359e700 1 -- start start 2024-01-11T07:18:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe93359e700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe934159f60 0x7fe9340cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe93359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9340c57f0 0x7fe9340cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe93359e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340cd5e0 0x7fe934068190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe93359e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9340cda60 con 0x7fe9340c57f0 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe93359e700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe934067d00 con 0x7fe934159f60 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe93359e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340ca980 con 0x7fe9340cd5e0 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe931d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9340c57f0 0x7fe9340cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe931d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9340c57f0 0x7fe9340cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50078/0 (socket says 172.21.15.7:50078) 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe931d9b700 1 -- 172.21.15.7:0/1639709401 learned_addr learned my addr 172.21.15.7:0/1639709401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe93259c700 1 --2- 172.21.15.7:0/1639709401 >> v2:172.21.15.113:3300/0 conn(0x7fe934159f60 0x7fe9340cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe932d9d700 1 --2- 172.21.15.7:0/1639709401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340cd5e0 0x7fe934068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe931d9b700 1 -- 172.21.15.7:0/1639709401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340cd5e0 msgr2=0x7fe934068190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe931d9b700 1 --2- 172.21.15.7:0/1639709401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340cd5e0 0x7fe934068190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe931d9b700 1 -- 172.21.15.7:0/1639709401 >> v2:172.21.15.113:3300/0 conn(0x7fe934159f60 msgr2=0x7fe9340cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.984+0000 7fe931d9b700 1 --2- 172.21.15.7:0/1639709401 >> v2:172.21.15.113:3300/0 conn(0x7fe934159f60 0x7fe9340cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.985+0000 7fe931d9b700 1 -- 172.21.15.7:0/1639709401 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340cac60 con 0x7fe9340c57f0 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.985+0000 7fe931d9b700 1 --2- 172.21.15.7:0/1639709401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9340c57f0 0x7fe9340cd0a0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe93406fce0 tx=0x7fe928009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad7d9457694e34 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.985+0000 7fe93159a700 1 -- 172.21.15.7:0/1639709401 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92800b640 con 0x7fe9340c57f0 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.985+0000 7fe93159a700 1 -- 172.21.15.7:0/1639709401 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92800b7a0 con 0x7fe9340c57f0 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.985+0000 7fe93159a700 1 -- 172.21.15.7:0/1639709401 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92800bb60 con 0x7fe9340c57f0 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.985+0000 7fe93359e700 1 -- 172.21.15.7:0/1639709401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9340c57f0 msgr2=0x7fe9340cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.985+0000 7fe93359e700 1 --2- 172.21.15.7:0/1639709401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9340c57f0 0x7fe9340cd0a0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe93406fce0 tx=0x7fe928009250 comp rx=0 tx=0).stop 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.986+0000 7fe93359e700 1 -- 172.21.15.7:0/1639709401 shutdown_connections 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.986+0000 7fe93359e700 1 --2- 172.21.15.7:0/1639709401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340cd5e0 0x7fe934068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.986+0000 7fe93359e700 1 --2- 172.21.15.7:0/1639709401 >> v2:172.21.15.113:3300/0 conn(0x7fe934159f60 0x7fe9340cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.986+0000 7fe93359e700 1 --2- 172.21.15.7:0/1639709401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9340c57f0 0x7fe9340cd0a0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.986+0000 7fe93359e700 1 -- 172.21.15.7:0/1639709401 >> 172.21.15.7:0/1639709401 conn(0x7fe934097a50 msgr2=0x7fe9340ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.986+0000 7fe93359e700 1 -- 172.21.15.7:0/1639709401 shutdown_connections 2024-01-11T07:18:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.986+0000 7fe93359e700 1 -- 172.21.15.7:0/1639709401 wait complete. 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.987+0000 7fe93359e700 1 Processor -- start 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.987+0000 7fe93359e700 1 -- start start 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93359e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340c57f0 0x7fe9340c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93359e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9340cd5e0 0x7fe9340c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe934159f60 0x7fe9341523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341539d0 con 0x7fe934159f60 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93359e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c6b40 con 0x7fe9340cd5e0 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93359e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe934153c70 con 0x7fe9340c57f0 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe934159f60 0x7fe9341523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe934159f60 0x7fe9341523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50082/0 (socket says 172.21.15.7:50082) 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 -- 172.21.15.7:0/2729003888 learned_addr learned my addr 172.21.15.7:0/2729003888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe931d9b700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9340cd5e0 0x7fe9340c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93259c700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340c57f0 0x7fe9340c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 -- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340c57f0 msgr2=0x7fe9340c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340c57f0 0x7fe9340c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 -- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9340cd5e0 msgr2=0x7fe9340c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9340cd5e0 0x7fe9340c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 -- 172.21.15.7:0/2729003888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280094f0 con 0x7fe934159f60 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe932d9d700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe934159f60 0x7fe9341523e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009e20 tx=0x7fe92c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe9227fc700 1 -- 172.21.15.7:0/2729003888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00caa0 con 0x7fe934159f60 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe9227fc700 1 -- 172.21.15.7:0/2729003888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c00f040 con 0x7fe934159f60 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9340c7d70 con 0x7fe934159f60 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe9227fc700 1 -- 172.21.15.7:0/2729003888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00ce70 con 0x7fe934159f60 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.988+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9340c6580 con 0x7fe934159f60 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.989+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe934153e00 con 0x7fe934159f60 2024-01-11T07:18:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.990+0000 7fe9227fc700 1 -- 172.21.15.7:0/2729003888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe92c014690 con 0x7fe934159f60 2024-01-11T07:18:46.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.990+0000 7fe9227fc700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe914066f30 0x7fe9140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:46.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.990+0000 7fe9227fc700 1 -- 172.21.15.7:0/2729003888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe92c084d20 con 0x7fe934159f60 2024-01-11T07:18:46.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.991+0000 7fe93259c700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe914066f30 0x7fe9140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:46.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.991+0000 7fe93259c700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe914066f30 0x7fe9140693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe924009d10 tx=0x7fe924009120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:46.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:46.995+0000 7fe9227fc700 1 -- 172.21.15.7:0/2729003888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe92c055a20 con 0x7fe934159f60 2024-01-11T07:18:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.245+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9340676b0 con 0x7fe934159f60 2024-01-11T07:18:47.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.246+0000 7fe9227fc700 1 -- 172.21.15.7:0/2729003888 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fe92c0555b0 con 0x7fe934159f60 2024-01-11T07:18:47.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:47.246 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe914066f30 msgr2=0x7fe9140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe914066f30 0x7fe9140693e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe924009d10 tx=0x7fe924009120 comp rx=0 tx=0).stop 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe934159f60 msgr2=0x7fe9341523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe934159f60 0x7fe9341523e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009e20 tx=0x7fe92c00b040 comp rx=0 tx=0).stop 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 shutdown_connections 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe914066f30 0x7fe9140693e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe934159f60 0x7fe9341523e0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9340c57f0 0x7fe9340c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 --2- 172.21.15.7:0/2729003888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9340cd5e0 0x7fe9340c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 >> 172.21.15.7:0/2729003888 conn(0x7fe934097a50 msgr2=0x7fe9340c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 shutdown_connections 2024-01-11T07:18:47.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.248+0000 7fe93359e700 1 -- 172.21.15.7:0/2729003888 wait complete. 2024-01-11T07:18:47.249 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:47.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:18:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.395+0000 7fb87e10c700 1 Processor -- start 2024-01-11T07:18:47.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.395+0000 7fb87e10c700 1 -- start start 2024-01-11T07:18:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb87e10c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87806e340 0x7fb87815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb87e10c700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb87815b3f0 0x7fb87815b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb87e10c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8780d0220 con 0x7fb878159d80 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb87e10c700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8780d06b0 con 0x7fb87815b3f0 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb87e10c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780cf610 con 0x7fb87806e340 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb8777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb878159d80 0x7fb87806dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33896/0 (socket says 172.21.15.7:33896) 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb8777fe700 1 -- 172.21.15.7:0/1055487207 learned_addr learned my addr 172.21.15.7:0/1055487207 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb877fff700 1 --2- 172.21.15.7:0/1055487207 >> v2:172.21.15.113:3300/0 conn(0x7fb87815b3f0 0x7fb87815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb876ffd700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87806e340 0x7fb87815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336855244 0 0) 0x7fb8780d0220 con 0x7fb878159d80 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.396+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb860003570 con 0x7fb878159d80 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656855510 0 0) 0x7fb860003570 con 0x7fb878159d80 2024-01-11T07:18:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87806e340 msgr2=0x7fb87815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87806e340 0x7fb87815aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 >> v2:172.21.15.113:3300/0 conn(0x7fb87815b3f0 msgr2=0x7fb87815b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 --2- 172.21.15.7:0/1055487207 >> v2:172.21.15.113:3300/0 conn(0x7fb87815b3f0 0x7fb87815b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8780cf8f0 con 0x7fb878159d80 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7fb868001470 con 0x7fb878159d80 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 >> v1:172.21.15.7:6789/0 conn(0x7fb878159d80 legacy=0x7fb87806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87806e340 0x7fb860003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb860003fe0 0x7fb860004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860004d50 0x7fb8600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780d06b0 con 0x7fb860003fe0 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780cf610 con 0x7fb860004d50 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86000ba90 con 0x7fb87806e340 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb876ffd700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb860003fe0 0x7fb860004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb8777fe700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87806e340 0x7fb860003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb877fff700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860004d50 0x7fb8600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb876ffd700 1 -- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87806e340 msgr2=0x7fb860003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb876ffd700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87806e340 0x7fb860003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb876ffd700 1 -- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860004d50 msgr2=0x7fb8600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb876ffd700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860004d50 0x7fb8600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.397+0000 7fb876ffd700 1 -- 172.21.15.7:0/1055487207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860003570 con 0x7fb860003fe0 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb876ffd700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb860003fe0 0x7fb860004810 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009c00 tx=0x7fb86c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a20a5c342771e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:47.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00e6b0 con 0x7fb860003fe0 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c00e810 con 0x7fb860003fe0 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb8767fc700 1 -- 172.21.15.7:0/1055487207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00ebd0 con 0x7fb860003fe0 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb87e10c700 1 -- 172.21.15.7:0/1055487207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb860003fe0 msgr2=0x7fb860004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb87e10c700 1 --2- 172.21.15.7:0/1055487207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb860003fe0 0x7fb860004810 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009c00 tx=0x7fb86c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb877fff700 1 -- 172.21.15.7:0/1055487207 reap_dead start 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb87e10c700 1 -- 172.21.15.7:0/1055487207 shutdown_connections 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb87e10c700 1 -- 172.21.15.7:0/1055487207 >> 172.21.15.7:0/1055487207 conn(0x7fb878071130 msgr2=0x7fb8780b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb87e10c700 1 -- 172.21.15.7:0/1055487207 shutdown_connections 2024-01-11T07:18:47.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.398+0000 7fb87e10c700 1 -- 172.21.15.7:0/1055487207 wait complete. 2024-01-11T07:18:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.400+0000 7fb87e10c700 1 Processor -- start 2024-01-11T07:18:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.400+0000 7fb87e10c700 1 -- start start 2024-01-11T07:18:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb87e10c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860003fe0 0x7fb878153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb87e10c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87815c200 0x7fb8780cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb87e10c700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87815d330 0x7fb87815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb87e10c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87815dcf0 con 0x7fb87815c200 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb87e10c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87815f7f0 con 0x7fb860003fe0 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb87e10c700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87815c700 con 0x7fb87815d330 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb876ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87815c200 0x7fb8780cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb876ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87815c200 0x7fb8780cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36578/0 (socket says 172.21.15.7:36578) 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb876ffd700 1 -- 172.21.15.7:0/3306042968 learned_addr learned my addr 172.21.15.7:0/3306042968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.401+0000 7fb8777fe700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860003fe0 0x7fb878153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb877fff700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87815d330 0x7fb87815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb876ffd700 1 -- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87815d330 msgr2=0x7fb87815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb876ffd700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87815d330 0x7fb87815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb876ffd700 1 -- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860003fe0 msgr2=0x7fb878153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb876ffd700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860003fe0 0x7fb878153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb876ffd700 1 -- 172.21.15.7:0/3306042968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87815c9e0 con 0x7fb87815c200 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb876ffd700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87815c200 0x7fb8780cd160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb860004730 tx=0x7fb86c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:47.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb85f7fe700 1 -- 172.21.15.7:0/3306042968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c023070 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb85f7fe700 1 -- 172.21.15.7:0/3306042968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c00ebb0 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8781516b0 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb85f7fe700 1 -- 172.21.15.7:0/3306042968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00e6b0 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.402+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8780cd6a0 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.403+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb878151b50 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.404+0000 7fb85f7fe700 1 -- 172.21.15.7:0/3306042968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fb86c013680 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.404+0000 7fb85f7fe700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb864066f30 0x7fb8640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.404+0000 7fb85f7fe700 1 -- 172.21.15.7:0/3306042968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb86c085ad0 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.404+0000 7fb85f7fe700 1 -- 172.21.15.7:0/3306042968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7fb86406ea50 con 0x7fb87815c200 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.404+0000 7fb8777fe700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb864066f30 0x7fb8640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.405+0000 7fb8777fe700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb864066f30 0x7fb8640693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb8680011e0 tx=0x7fb8780b3180 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.409+0000 7fb85f7fe700 1 -- 172.21.15.7:0/3306042968 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb86c017020 con 0x7fb87815c200 2024-01-11T07:18:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.658+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb878089080 con 0x7fb87815c200 2024-01-11T07:18:47.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.658+0000 7fb85f7fe700 1 -- 172.21.15.7:0/3306042968 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fb86c052d40 con 0x7fb87815c200 2024-01-11T07:18:47.658 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:47.659 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-11T07:18:12.853740+0000","modified":"2024-01-11T07:18:16.451460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3920146306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3920146306},{"type":"v1","addr":"172.21.15.7:6835","nonce":3920146306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb864066f30 msgr2=0x7fb8640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb864066f30 0x7fb8640693e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb8680011e0 tx=0x7fb8780b3180 comp rx=0 tx=0).stop 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87815c200 msgr2=0x7fb8780cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87815c200 0x7fb8780cd160 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb860004730 tx=0x7fb86c01a000 comp rx=0 tx=0).stop 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 shutdown_connections 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fb864066f30 0x7fb8640693e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb87815d330 0x7fb87815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb860003fe0 0x7fb878153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 --2- 172.21.15.7:0/3306042968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87815c200 0x7fb8780cd160 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 >> 172.21.15.7:0/3306042968 conn(0x7fb878071130 msgr2=0x7fb8780b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:47.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 shutdown_connections 2024-01-11T07:18:47.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.661+0000 7fb87e10c700 1 -- 172.21.15.7:0/3306042968 wait complete. 2024-01-11T07:18:47.662 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-01-11T07:18:47.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.805+0000 7f67c5df0700 1 Processor -- start 2024-01-11T07:18:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.805+0000 7f67c5df0700 1 -- start start 2024-01-11T07:18:47.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67c5df0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67c0158280 0x7f67c00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67c5df0700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67c015aa80 0x7f67c00b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67c5df0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c0159300 con 0x7f67c00cafb0 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67c5df0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c00cf000 con 0x7f67c0158280 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67c5df0700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c00cf260 con 0x7f67c015aa80 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67beffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67c0158280 0x7f67c00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67beffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67c0158280 0x7f67c00b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36588/0 (socket says 172.21.15.7:36588) 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67beffd700 1 -- 172.21.15.7:0/3122547135 learned_addr learned my addr 172.21.15.7:0/3122547135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67bffff700 1 --2- 172.21.15.7:0/3122547135 >> v2:172.21.15.113:3300/0 conn(0x7f67c015aa80 0x7f67c00b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67beffd700 1 -- 172.21.15.7:0/3122547135 >> v2:172.21.15.113:3300/0 conn(0x7f67c015aa80 msgr2=0x7f67c00b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67beffd700 1 --2- 172.21.15.7:0/3122547135 >> v2:172.21.15.113:3300/0 conn(0x7f67c015aa80 0x7f67c00b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67beffd700 1 -- 172.21.15.7:0/3122547135 >> v1:172.21.15.113:6789/0 conn(0x7f67c00cafb0 legacy=0x7f67c00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67beffd700 1 -- 172.21.15.7:0/3122547135 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00cf540 con 0x7f67c0158280 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.806+0000 7f67be7fc700 1 -- 172.21.15.7:0/3122547135 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433514632 0 0) 0x7f67c0159300 con 0x7f67c00cafb0 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67beffd700 1 --2- 172.21.15.7:0/3122547135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c00b0430 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f67c006cd70 tx=0x7f67b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38ea8416758ecce server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67be7fc700 1 -- 172.21.15.7:0/3122547135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4009e70 con 0x7f67c0158280 2024-01-11T07:18:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67be7fc700 1 -- 172.21.15.7:0/3122547135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b400b860 con 0x7f67c0158280 2024-01-11T07:18:47.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67be7fc700 1 -- 172.21.15.7:0/3122547135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400bc00 con 0x7f67c0158280 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67c5df0700 1 -- 172.21.15.7:0/3122547135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 msgr2=0x7f67c00b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67c5df0700 1 --2- 172.21.15.7:0/3122547135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c00b0430 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f67c006cd70 tx=0x7f67b4009250 comp rx=0 tx=0).stop 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67c5df0700 1 -- 172.21.15.7:0/3122547135 shutdown_connections 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67c5df0700 1 --2- 172.21.15.7:0/3122547135 >> v2:172.21.15.113:3300/0 conn(0x7f67c015aa80 0x7f67c00b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67c5df0700 1 --2- 172.21.15.7:0/3122547135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c00b0430 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67c5df0700 1 -- 172.21.15.7:0/3122547135 >> 172.21.15.7:0/3122547135 conn(0x7f67c0071100 msgr2=0x7f67c006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67c5df0700 1 -- 172.21.15.7:0/3122547135 shutdown_connections 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.807+0000 7f67c5df0700 1 -- 172.21.15.7:0/3122547135 wait complete. 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.809+0000 7f67c5df0700 1 Processor -- start 2024-01-11T07:18:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.809+0000 7f67c5df0700 1 -- start start 2024-01-11T07:18:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67c5df0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c00cafb0 0x7f67c015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67c5df0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c01598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67c5df0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c015aa80 0x7f67c0159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67c5df0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0155940 con 0x7f67c0158280 2024-01-11T07:18:47.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67c5df0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c0167be0 con 0x7f67c015aa80 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67c5df0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c015dc40 con 0x7f67c00cafb0 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c01598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c01598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36596/0 (socket says 172.21.15.7:36596) 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 -- 172.21.15.7:0/2074491412 learned_addr learned my addr 172.21.15.7:0/2074491412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67bf7fe700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c00cafb0 0x7f67c015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67bffff700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c015aa80 0x7f67c0159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 -- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c00cafb0 msgr2=0x7f67c015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c00cafb0 0x7f67c015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 -- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c015aa80 msgr2=0x7f67c0159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c015aa80 0x7f67c0159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 -- 172.21.15.7:0/2074491412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40094f0 con 0x7f67c0158280 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67bf7fe700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c00cafb0 0x7f67c015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67bffff700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c015aa80 0x7f67c0159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:47.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67beffd700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c01598c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f67c006a1d0 tx=0x7f67b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:47.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67a77fe700 1 -- 172.21.15.7:0/2074491412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400b9d0 con 0x7f67c0158280 2024-01-11T07:18:47.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67a77fe700 1 -- 172.21.15.7:0/2074491412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b400bb30 con 0x7f67c0158280 2024-01-11T07:18:47.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c0168b70 con 0x7f67c0158280 2024-01-11T07:18:47.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.810+0000 7f67a77fe700 1 -- 172.21.15.7:0/2074491412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4025670 con 0x7f67c0158280 2024-01-11T07:18:47.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.811+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c0168090 con 0x7f67c0158280 2024-01-11T07:18:47.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.812+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c004c3c0 con 0x7f67c0158280 2024-01-11T07:18:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.812+0000 7f67a77fe700 1 -- 172.21.15.7:0/2074491412 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f67b4023070 con 0x7f67c0158280 2024-01-11T07:18:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.812+0000 7f67a77fe700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67ac066f30 0x7f67ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.813+0000 7f67a77fe700 1 -- 172.21.15.7:0/2074491412 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f67b4085b30 con 0x7f67c0158280 2024-01-11T07:18:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.813+0000 7f67a77fe700 1 -- 172.21.15.7:0/2074491412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7f67ac06ea50 con 0x7f67c0158280 2024-01-11T07:18:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.813+0000 7f67bf7fe700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67ac066f30 0x7f67ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.813+0000 7f67bf7fe700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67ac066f30 0x7f67ac0693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f67b00011e0 tx=0x7f67b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:47.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:47.817+0000 7f67a77fe700 1 -- 172.21.15.7:0/2074491412 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67b4015060 con 0x7f67c0158280 2024-01-11T07:18:48.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.023+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67c00cf820 con 0x7f67c0158280 2024-01-11T07:18:48.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.024+0000 7f67a77fe700 1 -- 172.21.15.7:0/2074491412 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+15343 (secure 0 0 0) 0x7f67b402a070 con 0x7f67c0158280 2024-01-11T07:18:48.024 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:48.024 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":188,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:47.301202+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-01-11T07:18:11.360758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-01-11T07:18:11.823991+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67ac066f30 msgr2=0x7f67ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67ac066f30 0x7f67ac0693e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f67b00011e0 tx=0x7f67b0009040 comp rx=0 tx=0).stop 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 msgr2=0x7f67c01598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c01598c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f67c006a1d0 tx=0x7f67b4017000 comp rx=0 tx=0).stop 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 shutdown_connections 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f67ac066f30 0x7f67ac0693e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c015aa80 0x7f67c0159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67c00cafb0 0x7f67c015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 --2- 172.21.15.7:0/2074491412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0158280 0x7f67c01598c0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.026+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 >> 172.21.15.7:0/2074491412 conn(0x7f67c0071100 msgr2=0x7f67c00653b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.027+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 shutdown_connections 2024-01-11T07:18:48.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.027+0000 7f67c5df0700 1 -- 172.21.15.7:0/2074491412 wait complete. 2024-01-11T07:18:48.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-11T07:18:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.173+0000 7f8e1590d700 1 Processor -- start 2024-01-11T07:18:48.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.173+0000 7f8e1590d700 1 -- start start 2024-01-11T07:18:48.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e1590d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e101560d0 0x7f8e10069e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e1590d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8e100c7520 0x7f8e100cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e1590d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e100699a0 con 0x7f8e100cadf0 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e1590d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e100cc2f0 con 0x7f8e101560d0 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e1590d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e10068d90 con 0x7f8e100c7520 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e101560d0 0x7f8e10069e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e101560d0 0x7f8e10069e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36606/0 (socket says 172.21.15.7:36606) 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0effd700 1 -- 172.21.15.7:0/3477420732 learned_addr learned my addr 172.21.15.7:0/3477420732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0f7fe700 1 --2- 172.21.15.7:0/3477420732 >> v2:172.21.15.113:3300/0 conn(0x7f8e100c7520 0x7f8e100cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0effd700 1 -- 172.21.15.7:0/3477420732 >> v2:172.21.15.113:3300/0 conn(0x7f8e100c7520 msgr2=0x7f8e100cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0effd700 1 --2- 172.21.15.7:0/3477420732 >> v2:172.21.15.113:3300/0 conn(0x7f8e100c7520 0x7f8e100cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0effd700 1 -- 172.21.15.7:0/3477420732 >> v1:172.21.15.113:6789/0 conn(0x7f8e100cadf0 legacy=0x7f8e100c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0effd700 1 -- 172.21.15.7:0/3477420732 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10069070 con 0x7f8e101560d0 2024-01-11T07:18:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.174+0000 7f8e0effd700 1 --2- 172.21.15.7:0/3477420732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e10069e30 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8e100ac080 tx=0x7f8e0000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacac31f40785266 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:48.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e0dffb700 1 -- 172.21.15.7:0/3477420732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0000be60 con 0x7f8e101560d0 2024-01-11T07:18:48.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e0dffb700 1 -- 172.21.15.7:0/3477420732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0000ddc0 con 0x7f8e101560d0 2024-01-11T07:18:48.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e0dffb700 1 -- 172.21.15.7:0/3477420732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e000196b0 con 0x7f8e101560d0 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e1590d700 1 -- 172.21.15.7:0/3477420732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 msgr2=0x7f8e10069e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e1590d700 1 --2- 172.21.15.7:0/3477420732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e10069e30 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8e100ac080 tx=0x7f8e0000b4f0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e1590d700 1 -- 172.21.15.7:0/3477420732 shutdown_connections 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e1590d700 1 --2- 172.21.15.7:0/3477420732 >> v2:172.21.15.113:3300/0 conn(0x7f8e100c7520 0x7f8e100cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e1590d700 1 --2- 172.21.15.7:0/3477420732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e10069e30 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e1590d700 1 -- 172.21.15.7:0/3477420732 >> 172.21.15.7:0/3477420732 conn(0x7f8e10063230 msgr2=0x7f8e10063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e1590d700 1 -- 172.21.15.7:0/3477420732 shutdown_connections 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.175+0000 7f8e1590d700 1 -- 172.21.15.7:0/3477420732 wait complete. 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.177+0000 7f8e1590d700 1 Processor -- start 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.177+0000 7f8e1590d700 1 -- start start 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.177+0000 7f8e1590d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e100c7520 0x7f8e100b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.177+0000 7f8e1590d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e100cadf0 0x7f8e1016a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e1590d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e100c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e1590d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100c3880 con 0x7f8e101560d0 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e1590d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100c3af0 con 0x7f8e100cadf0 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e1590d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1016abe0 con 0x7f8e100c7520 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e100c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e100c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36610/0 (socket says 172.21.15.7:36610) 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 -- 172.21.15.7:0/2789344506 learned_addr learned my addr 172.21.15.7:0/2789344506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0effd700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e100c7520 0x7f8e100b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0e7fc700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e100cadf0 0x7f8e1016a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 -- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e100c7520 msgr2=0x7f8e100b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e100c7520 0x7f8e100b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 -- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e100cadf0 msgr2=0x7f8e1016a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e100cadf0 0x7f8e1016a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 -- 172.21.15.7:0/2789344506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0000b790 con 0x7f8e101560d0 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8e0f7fe700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e100c5470 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8e08007f20 tx=0x7f8e080098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0800eb90 con 0x7f8e101560d0 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.178+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0800ed10 con 0x7f8e101560d0 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.179+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08014670 con 0x7f8e101560d0 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.179+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1016ae60 con 0x7f8e101560d0 2024-01-11T07:18:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.179+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1016b420 con 0x7f8e101560d0 2024-01-11T07:18:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.180+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1016b640 con 0x7f8e101560d0 2024-01-11T07:18:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.180+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8e08014a60 con 0x7f8e101560d0 2024-01-11T07:18:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.181+0000 7f8df6ffd700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8dfc066f30 0x7f8dfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.181+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8e08084b50 con 0x7f8e101560d0 2024-01-11T07:18:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.181+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=189}) v3 -- 0x7f8dfc06ea50 con 0x7f8e101560d0 2024-01-11T07:18:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.181+0000 7f8e0effd700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8dfc066f30 0x7f8dfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.181+0000 7f8e0effd700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8dfc066f30 0x7f8dfc0693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8e100ac080 tx=0x7f8e0001b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:48.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.185+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e08051dc0 con 0x7f8e101560d0 2024-01-11T07:18:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.441+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8e1004c3c0 con 0x7f8e101560d0 2024-01-11T07:18:48.441 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-01-11T07:18:48.442+0000 7f12fccd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-11T07:18:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.443+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(189..189 src has 1..189) v4 ==== 366+0+0 (secure 0 0 0) 0x7f8e08055a80 con 0x7f8e101560d0 2024-01-11T07:18:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.443+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f8dfc06f3a0 con 0x7f8e101560d0 2024-01-11T07:18:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.446+0000 7f8df6ffd700 1 -- 172.21.15.7:0/2789344506 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8e080553e0 con 0x7f8e101560d0 2024-01-11T07:18:48.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8dfc066f30 msgr2=0x7f8dfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8dfc066f30 0x7f8dfc0693e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8e100ac080 tx=0x7f8e0001b000 comp rx=0 tx=0).stop 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 msgr2=0x7f8e100c5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e100c5470 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8e08007f20 tx=0x7f8e080098a0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 shutdown_connections 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8dfc066f30 0x7f8dfc0693e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e101560d0 0x7f8e100c5470 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8e100c7520 0x7f8e100b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 --2- 172.21.15.7:0/2789344506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e100cadf0 0x7f8e1016a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 >> 172.21.15.7:0/2789344506 conn(0x7f8e10063230 msgr2=0x7f8e1006ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 shutdown_connections 2024-01-11T07:18:48.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.449+0000 7f8e1590d700 1 -- 172.21.15.7:0/2789344506 wait complete. 2024-01-11T07:18:48.451 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-11T07:18:48.451 INFO:tasks.ceph.mds.a.smithi007.stderr: -7574> 2024-01-11T07:18:18.409+0000 7f33f5fbf700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-11T07:18:48.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-11T07:18:48.483 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-01-11T07:18:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.599+0000 7fabd6287700 1 Processor -- start 2024-01-11T07:18:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.599+0000 7fabd6287700 1 -- start start 2024-01-11T07:18:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabd6287700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabd00c3690 0x7fabd006e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabd6287700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 0x7fabd00c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabd6287700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd006d3b0 con 0x7fabd015db90 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabd6287700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd006d690 con 0x7fabd00c3690 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabd6287700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd006d970 con 0x7fabd006ec70 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabcf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabd00c3690 0x7fabd006e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabcffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fabd015db90 0x7fabd015df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33908/0 (socket says 172.21.15.7:33908) 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabcffff700 1 -- 172.21.15.7:0/1288371781 learned_addr learned my addr 172.21.15.7:0/1288371781 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabd4824700 1 --2- 172.21.15.7:0/1288371781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 0x7fabd00c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabcf7fe700 1 -- 172.21.15.7:0/1288371781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 msgr2=0x7fabd00c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabcf7fe700 1 --2- 172.21.15.7:0/1288371781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 0x7fabd00c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabcf7fe700 1 -- 172.21.15.7:0/1288371781 >> v1:172.21.15.7:6789/0 conn(0x7fabd015db90 legacy=0x7fabd015df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabcf7fe700 1 -- 172.21.15.7:0/1288371781 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd006dc50 con 0x7fabd00c3690 2024-01-11T07:18:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.600+0000 7fabceffd700 1 -- 172.21.15.7:0/1288371781 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331054019 0 0) 0x7fabd006d3b0 con 0x7fabd015db90 2024-01-11T07:18:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabcf7fe700 1 --2- 172.21.15.7:0/1288371781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd006e730 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098e0 tx=0x7fabc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26be9acc3c4fce02 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabceffd700 1 -- 172.21.15.7:0/1288371781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4009e80 con 0x7fabd00c3690 2024-01-11T07:18:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabceffd700 1 -- 172.21.15.7:0/1288371781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc400b860 con 0x7fabd00c3690 2024-01-11T07:18:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabceffd700 1 -- 172.21.15.7:0/1288371781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400bbe0 con 0x7fabd00c3690 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabd6287700 1 -- 172.21.15.7:0/1288371781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 msgr2=0x7fabd006e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabd6287700 1 --2- 172.21.15.7:0/1288371781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd006e730 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fabc40098e0 tx=0x7fabc40091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabd6287700 1 -- 172.21.15.7:0/1288371781 shutdown_connections 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabd6287700 1 --2- 172.21.15.7:0/1288371781 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 0x7fabd00c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabd6287700 1 --2- 172.21.15.7:0/1288371781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd006e730 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.601+0000 7fabd6287700 1 -- 172.21.15.7:0/1288371781 >> 172.21.15.7:0/1288371781 conn(0x7fabd00cd070 msgr2=0x7fabd00cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.602+0000 7fabd6287700 1 -- 172.21.15.7:0/1288371781 shutdown_connections 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.602+0000 7fabd6287700 1 -- 172.21.15.7:0/1288371781 wait complete. 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.603+0000 7fabd6287700 1 Processor -- start 2024-01-11T07:18:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.603+0000 7fabd6287700 1 -- start start 2024-01-11T07:18:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabd6287700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 0x7fabd016a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabd6287700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd0099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabd6287700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd015db90 0x7fabd015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabd6287700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0097a50 con 0x7fabd00c3690 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabd6287700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd00982f0 con 0x7fabd015db90 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabd6287700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0098780 con 0x7fabd006ec70 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd0099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd0099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36652/0 (socket says 172.21.15.7:36652) 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 -- 172.21.15.7:0/3660311025 learned_addr learned my addr 172.21.15.7:0/3660311025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcffff700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 0x7fabd016a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabd4824700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd015db90 0x7fabd015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 -- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 msgr2=0x7fabd016a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 0x7fabd016a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 -- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd015db90 msgr2=0x7fabd015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd015db90 0x7fabd015bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 -- 172.21.15.7:0/3660311025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4009420 con 0x7fabd00c3690 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabd4824700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd015db90 0x7fabd015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabcf7fe700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd0099d50 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b3c0 tx=0x7fabc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabb7fff700 1 -- 172.21.15.7:0/3660311025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400bbe0 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.604+0000 7fabb7fff700 1 -- 172.21.15.7:0/3660311025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc4011420 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.605+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd009a6a0 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.605+0000 7fabb7fff700 1 -- 172.21.15.7:0/3660311025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc40117e0 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.605+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd0097f80 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.606+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd0098480 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.606+0000 7fabb7fff700 1 -- 172.21.15.7:0/3660311025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fabc4011bd0 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.606+0000 7fabb7fff700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabbc066f30 0x7fabbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.607+0000 7fabb7fff700 1 -- 172.21.15.7:0/3660311025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fabc4085e90 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.607+0000 7fabb7fff700 1 -- 172.21.15.7:0/3660311025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7fabbc06eaf0 con 0x7fabd00c3690 2024-01-11T07:18:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.607+0000 7fabcffff700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabbc066f30 0x7fabbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.607+0000 7fabcffff700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabbc066f30 0x7fabbc0693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fabc0007fd0 tx=0x7fabc0007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.611+0000 7fabb7fff700 1 -- 172.21.15.7:0/3660311025 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabc40530b0 con 0x7fabd00c3690 2024-01-11T07:18:48.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:48.860+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fabd004c3c0 con 0x7fabd00c3690 2024-01-11T07:18:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.446+0000 7fabb7fff700 1 -- 172.21.15.7:0/3660311025 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7fabc40150a0 con 0x7fabd00c3690 2024-01-11T07:18:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabbc066f30 msgr2=0x7fabbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabbc066f30 0x7fabbc0693e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fabc0007fd0 tx=0x7fabc0007d30 comp rx=0 tx=0).stop 2024-01-11T07:18:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 msgr2=0x7fabd0099d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd0099d50 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b3c0 tx=0x7fabc4017000 comp rx=0 tx=0).stop 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 shutdown_connections 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fabbc066f30 0x7fabbc0693e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabd015db90 0x7fabd015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fabd006ec70 0x7fabd016a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 --2- 172.21.15.7:0/3660311025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabd00c3690 0x7fabd0099d50 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.449+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 >> 172.21.15.7:0/3660311025 conn(0x7fabd00cd070 msgr2=0x7fabd006ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.450+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 shutdown_connections 2024-01-11T07:18:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.450+0000 7fabd6287700 1 -- 172.21.15.7:0/3660311025 wait complete. 2024-01-11T07:18:49.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-11T07:18:49.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.649+0000 7f0edbe45700 1 Processor -- start 2024-01-11T07:18:49.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.650+0000 7f0edbe45700 1 -- start start 2024-01-11T07:18:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.650+0000 7f0edbe45700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ed4063480 0x7f0ed40acc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.650+0000 7f0edbe45700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ed4063850 0x7f0ed415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.650+0000 7f0edbe45700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed40c7e70 con 0x7f0ed4069c50 2024-01-11T07:18:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.650+0000 7f0edbe45700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed40c8150 con 0x7f0ed4063850 2024-01-11T07:18:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.650+0000 7f0edbe45700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed415dee0 con 0x7f0ed4063480 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed93e0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ed4063850 0x7f0ed415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed93e0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ed4063850 0x7f0ed415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36666/0 (socket says 172.21.15.7:36666) 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed93e0700 1 -- 172.21.15.7:0/1806110856 learned_addr learned my addr 172.21.15.7:0/1806110856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed9be1700 1 --2- 172.21.15.7:0/1806110856 >> v2:172.21.15.113:3300/0 conn(0x7f0ed4063480 0x7f0ed40acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed93e0700 1 -- 172.21.15.7:0/1806110856 >> v2:172.21.15.113:3300/0 conn(0x7f0ed4063480 msgr2=0x7f0ed40acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed93e0700 1 --2- 172.21.15.7:0/1806110856 >> v2:172.21.15.113:3300/0 conn(0x7f0ed4063480 0x7f0ed40acc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed93e0700 1 -- 172.21.15.7:0/1806110856 >> v1:172.21.15.113:6789/0 conn(0x7f0ed4069c50 legacy=0x7f0ed4159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed93e0700 1 -- 172.21.15.7:0/1806110856 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed415e180 con 0x7f0ed4063850 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed93e0700 1 --2- 172.21.15.7:0/1806110856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed415d9a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0ed406d600 tx=0x7f0ec4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf56a474a53bde9 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed8bdf700 1 -- 172.21.15.7:0/1806110856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400b820 con 0x7f0ed4063850 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed8bdf700 1 -- 172.21.15.7:0/1806110856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec400b980 con 0x7f0ed4063850 2024-01-11T07:18:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.651+0000 7f0ed8bdf700 1 -- 172.21.15.7:0/1806110856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400bd20 con 0x7f0ed4063850 2024-01-11T07:18:49.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.652+0000 7f0edbe45700 1 -- 172.21.15.7:0/1806110856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 msgr2=0x7f0ed415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:49.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.652+0000 7f0edbe45700 1 --2- 172.21.15.7:0/1806110856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed415d9a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0ed406d600 tx=0x7f0ec4009250 comp rx=0 tx=0).stop 2024-01-11T07:18:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.652+0000 7f0edbe45700 1 -- 172.21.15.7:0/1806110856 shutdown_connections 2024-01-11T07:18:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.652+0000 7f0edbe45700 1 --2- 172.21.15.7:0/1806110856 >> v2:172.21.15.113:3300/0 conn(0x7f0ed4063480 0x7f0ed40acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.652+0000 7f0edbe45700 1 --2- 172.21.15.7:0/1806110856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed415d9a0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.652+0000 7f0edbe45700 1 -- 172.21.15.7:0/1806110856 >> 172.21.15.7:0/1806110856 conn(0x7f0ed40680a0 msgr2=0x7f0ed40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.652+0000 7f0edbe45700 1 -- 172.21.15.7:0/1806110856 shutdown_connections 2024-01-11T07:18:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.652+0000 7f0edbe45700 1 -- 172.21.15.7:0/1806110856 wait complete. 2024-01-11T07:18:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.653+0000 7f0edbe45700 1 Processor -- start 2024-01-11T07:18:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.653+0000 7f0edbe45700 1 -- start start 2024-01-11T07:18:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0edbe45700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed4063480 0x7f0ed40c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0edbe45700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed40af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0edbe45700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed4069c50 0x7f0ed40c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0edbe45700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40c3640 con 0x7f0ed4063850 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0edbe45700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40c2790 con 0x7f0ed4069c50 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0edbe45700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40c2a30 con 0x7f0ed4063480 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0ed93e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed40af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0ed93e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed40af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36682/0 (socket says 172.21.15.7:36682) 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.654+0000 7f0ed93e0700 1 -- 172.21.15.7:0/4044948247 learned_addr learned my addr 172.21.15.7:0/4044948247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ed9be1700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed4063480 0x7f0ed40c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0eda3e2700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed4069c50 0x7f0ed40c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ed93e0700 1 -- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed4063480 msgr2=0x7f0ed40c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ed93e0700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed4063480 0x7f0ed40c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ed93e0700 1 -- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed4069c50 msgr2=0x7f0ed40c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ed93e0700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed4069c50 0x7f0ed40c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ed93e0700 1 -- 172.21.15.7:0/4044948247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec40094f0 con 0x7f0ed4063850 2024-01-11T07:18:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ed93e0700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed40af2a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40030c0 tx=0x7f0ec4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4011e60 con 0x7f0ed4063850 2024-01-11T07:18:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec401b680 con 0x7f0ed4063850 2024-01-11T07:18:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec401ba20 con 0x7f0ed4063850 2024-01-11T07:18:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed40c2d10 con 0x7f0ed4063850 2024-01-11T07:18:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.655+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed40c32d0 con 0x7f0ed4063850 2024-01-11T07:18:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.656+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed40c37b0 con 0x7f0ed4063850 2024-01-11T07:18:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.657+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0ec401c450 con 0x7f0ed4063850 2024-01-11T07:18:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.657+0000 7f0ec9ffb700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0eb8066f30 0x7f0eb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.657+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f0ec40863a0 con 0x7f0ed4063850 2024-01-11T07:18:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.658+0000 7f0ed9be1700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0eb8066f30 0x7f0eb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:49.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.658+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f0eb806eaf0 con 0x7f0ed4063850 2024-01-11T07:18:49.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.658+0000 7f0ed9be1700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0eb8066f30 0x7f0eb80693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0006fd0 tx=0x7f0ed0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.662+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ec402a060 con 0x7f0ed4063850 2024-01-11T07:18:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:49.866+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ed404c3c0 con 0x7f0ed4063850 2024-01-11T07:18:50.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.451+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(190..190 src has 1..190) v4 ==== 288+0+0 (secure 0 0 0) 0x7f0ec40578e0 con 0x7f0ed4063850 2024-01-11T07:18:50.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.451+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=191}) v3 -- 0x7f0eb806cf20 con 0x7f0ed4063850 2024-01-11T07:18:50.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.458+0000 7f0ec9ffb700 1 -- 172.21.15.7:0/4044948247 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v190) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0ec405c030 con 0x7f0ed4063850 2024-01-11T07:18:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0eb8066f30 msgr2=0x7f0eb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0eb8066f30 0x7f0eb80693e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0006fd0 tx=0x7f0ed0008040 comp rx=0 tx=0).stop 2024-01-11T07:18:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 msgr2=0x7f0ed40af2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed40af2a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40030c0 tx=0x7f0ec4016000 comp rx=0 tx=0).stop 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 shutdown_connections 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0eb8066f30 0x7f0eb80693e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed4069c50 0x7f0ed40c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ed4063480 0x7f0ed40c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 --2- 172.21.15.7:0/4044948247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4063850 0x7f0ed40af2a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 >> 172.21.15.7:0/4044948247 conn(0x7f0ed40680a0 msgr2=0x7f0ed4158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 shutdown_connections 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.463+0000 7f0edbe45700 1 -- 172.21.15.7:0/4044948247 wait complete. 2024-01-11T07:18:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-01-11T07:18:50.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-11T07:18:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.618+0000 7f16ac609700 1 Processor -- start 2024-01-11T07:18:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.618+0000 7f16ac609700 1 -- start start 2024-01-11T07:18:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.619+0000 7f16ac609700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16a40cdf70 0x7f16a4067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.619+0000 7f16ac609700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a4067b20 con 0x7f16a40ce340 2024-01-11T07:18:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.619+0000 7f16ac609700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a415e3c0 con 0x7f16a40684f0 2024-01-11T07:18:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.619+0000 7f16ac609700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a415e6a0 con 0x7f16a40cdf70 2024-01-11T07:18:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16aaba6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16a40ce340 0x7f16a40aa8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33924/0 (socket says 172.21.15.7:33924) 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16aaba6700 1 -- 172.21.15.7:0/2191073878 learned_addr learned my addr 172.21.15.7:0/2191073878 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16aa3a5700 1 --2- 172.21.15.7:0/2191073878 >> v2:172.21.15.113:3300/0 conn(0x7f16a40cdf70 0x7f16a4067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775722114 0 0) 0x7f16a4067b20 con 0x7f16a40ce340 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1690003570 con 0x7f16a40ce340 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1103286423 0 0) 0x7f1690003570 con 0x7f16a40ce340 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 >> v2:172.21.15.113:3300/0 conn(0x7f16a40cdf70 msgr2=0x7f16a4067fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 --2- 172.21.15.7:0/2191073878 >> v2:172.21.15.113:3300/0 conn(0x7f16a40cdf70 0x7f16a4067fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 >> v1:172.21.15.113:6789/0 conn(0x7f16a40684f0 legacy=0x7f16a415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40677b0 con 0x7f16a40ce340 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f169c0017f0 con 0x7f16a40ce340 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 >> v1:172.21.15.7:6789/0 conn(0x7f16a40ce340 legacy=0x7f16a40aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690003ab0 0x7f1690003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690004810 0x7f1690004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690005210 0x7f169000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415e6a0 con 0x7f1690003ab0 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169000c130 con 0x7f1690005210 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169000c2f0 con 0x7f1690004810 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.620+0000 7f16aa3a5700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690003ab0 0x7f1690003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16a9ba4700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690004810 0x7f1690004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16aaba6700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690005210 0x7f169000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16aa3a5700 1 -- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690004810 msgr2=0x7f1690004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16aa3a5700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690004810 0x7f1690004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16aa3a5700 1 -- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690005210 msgr2=0x7f169000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16aa3a5700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1690005210 0x7f169000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16aaba6700 1 -- 172.21.15.7:0/2191073878 reap_dead start 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16aa3a5700 1 -- 172.21.15.7:0/2191073878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690003570 con 0x7f1690003ab0 2024-01-11T07:18:50.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16a9ba4700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690004810 0x7f1690004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16aa3a5700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690003ab0 0x7f1690003ec0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1698009c00 tx=0x7f16980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b60f88129470c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800e710 con 0x7f1690003ab0 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800e870 con 0x7f1690003ab0 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.621+0000 7f16a93a3700 1 -- 172.21.15.7:0/2191073878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800ec30 con 0x7f1690003ab0 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.622+0000 7f16ac609700 1 -- 172.21.15.7:0/2191073878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690003ab0 msgr2=0x7f1690003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.622+0000 7f16ac609700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690003ab0 0x7f1690003ec0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f1698009c00 tx=0x7f16980091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.622+0000 7f16ac609700 1 -- 172.21.15.7:0/2191073878 shutdown_connections 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.622+0000 7f16ac609700 1 --2- 172.21.15.7:0/2191073878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1690003ab0 0x7f1690003ec0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.622+0000 7f16ac609700 1 -- 172.21.15.7:0/2191073878 >> 172.21.15.7:0/2191073878 conn(0x7f16a406edc0 msgr2=0x7f16a406f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.622+0000 7f16ac609700 1 -- 172.21.15.7:0/2191073878 shutdown_connections 2024-01-11T07:18:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.622+0000 7f16ac609700 1 -- 172.21.15.7:0/2191073878 wait complete. 2024-01-11T07:18:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.623+0000 7f16ac609700 1 Processor -- start 2024-01-11T07:18:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.623+0000 7f16ac609700 1 -- start start 2024-01-11T07:18:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16ac609700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690003ab0 0x7f16a40ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16ac609700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a4169d90 0x7f16a40b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16ac609700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a40b3550 0x7f16a4150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16ac609700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4155e20 con 0x7f16a40b3550 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16ac609700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40b3ad0 con 0x7f16a4169d90 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16ac609700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a41560c0 con 0x7f1690003ab0 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a40b3550 0x7f16a4150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a40b3550 0x7f16a4150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36704/0 (socket says 172.21.15.7:36704) 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 -- 172.21.15.7:0/440730430 learned_addr learned my addr 172.21.15.7:0/440730430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16a9ba4700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a4169d90 0x7f16a40b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aa3a5700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690003ab0 0x7f16a40ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 -- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690003ab0 msgr2=0x7f16a40ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690003ab0 0x7f16a40ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 -- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a4169d90 msgr2=0x7f16a40b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a4169d90 0x7f16a40b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 -- 172.21.15.7:0/440730430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a41563a0 con 0x7f16a40b3550 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.624+0000 7f16aaba6700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a40b3550 0x7f16a4150040 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f16a40cfa10 tx=0x7f169c0018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.625+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00eb10 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.625+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c00ec70 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.625+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c013420 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.625+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a4156680 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.625+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a40ab630 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.626+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a40ab820 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.626+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f169c0137f0 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.627+0000 7f16967fc700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1688066f30 0x7f16880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.627+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f169c084950 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.627+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=191}) v3 -- 0x7f168806dcf0 con 0x7f16a40b3550 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.627+0000 7f16aa3a5700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1688066f30 0x7f16880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.628+0000 7f16aa3a5700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1688066f30 0x7f16880693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f16980091e0 tx=0x7f169801a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.631+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f169c051d20 con 0x7f16a40b3550 2024-01-11T07:18:50.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:50.832+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f16a404c3c0 con 0x7f16a40b3550 2024-01-11T07:18:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.456+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 <== mon.0 v2:172.21.15.7:3300/0 7 ==== osd_map(191..191 src has 1..191) v4 ==== 288+0+0 (secure 0 0 0) 0x7f169c0560e0 con 0x7f16a40b3550 2024-01-11T07:18:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.456+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=192}) v3 -- 0x7f168806ca80 con 0x7f16a40b3550 2024-01-11T07:18:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.459+0000 7f16967fc700 1 -- 172.21.15.7:0/440730430 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v191) v1 ==== 169+0+0 (secure 0 0 0) 0x7f169c055340 con 0x7f16a40b3550 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1688066f30 msgr2=0x7f16880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1688066f30 0x7f16880693e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f16980091e0 tx=0x7f169801a000 comp rx=0 tx=0).stop 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a40b3550 msgr2=0x7f16a4150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a40b3550 0x7f16a4150040 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f16a40cfa10 tx=0x7f169c0018a0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 shutdown_connections 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1688066f30 0x7f16880693e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a40b3550 0x7f16a4150040 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1690003ab0 0x7f16a40ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 --2- 172.21.15.7:0/440730430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a4169d90 0x7f16a40b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.464+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 >> 172.21.15.7:0/440730430 conn(0x7f16a406edc0 msgr2=0x7f16a4154890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.465+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 shutdown_connections 2024-01-11T07:18:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.465+0000 7f16ac609700 1 -- 172.21.15.7:0/440730430 wait complete. 2024-01-11T07:18:51.469 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-01-11T07:18:51.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-11T07:18:51.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.623+0000 7f7e0dcac700 1 Processor -- start 2024-01-11T07:18:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.623+0000 7f7e0dcac700 1 -- start start 2024-01-11T07:18:51.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.623+0000 7f7e0dcac700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e0815e5a0 0x7f7e0815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e0dcac700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0815e110 con 0x7f7e080cb790 2024-01-11T07:18:51.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e0dcac700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0815b050 con 0x7f7e080cbb60 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e0dcac700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0815b330 con 0x7f7e0815e5a0 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e077fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7e080cb790 0x7f7e080cdc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33934/0 (socket says 172.21.15.7:33934) 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e077fe700 1 -- 172.21.15.7:0/485163722 learned_addr learned my addr 172.21.15.7:0/485163722 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e07fff700 1 --2- 172.21.15.7:0/485163722 >> v2:172.21.15.7:3300/0 conn(0x7f7e0815e5a0 0x7f7e0815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e067fc700 1 -- 172.21.15.7:0/485163722 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258581750 0 0) 0x7f7e0815e110 con 0x7f7e080cb790 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e067fc700 1 -- 172.21.15.7:0/485163722 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df4003570 con 0x7f7e080cb790 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e07fff700 1 -- 172.21.15.7:0/485163722 >> v1:172.21.15.113:6789/0 conn(0x7f7e080cbb60 legacy=0x7f7e0815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e07fff700 1 -- 172.21.15.7:0/485163722 >> v1:172.21.15.7:6789/0 conn(0x7f7e080cb790 legacy=0x7f7e080cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e07fff700 1 -- 172.21.15.7:0/485163722 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0815dac0 con 0x7f7e0815e5a0 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e07fff700 1 --2- 172.21.15.7:0/485163722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0815e5a0 0x7f7e0815aa20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009e60 tx=0x7f7e00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8760cf1bad3ce8 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e067fc700 1 -- 172.21.15.7:0/485163722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000e070 con 0x7f7e0815e5a0 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.624+0000 7f7e067fc700 1 -- 172.21.15.7:0/485163722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0000bb60 con 0x7f7e0815e5a0 2024-01-11T07:18:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.625+0000 7f7e067fc700 1 -- 172.21.15.7:0/485163722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00012430 con 0x7f7e0815e5a0 2024-01-11T07:18:51.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.625+0000 7f7e0dcac700 1 -- 172.21.15.7:0/485163722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0815e5a0 msgr2=0x7f7e0815aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.625+0000 7f7e0dcac700 1 --2- 172.21.15.7:0/485163722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0815e5a0 0x7f7e0815aa20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009e60 tx=0x7f7e00009780 comp rx=0 tx=0).stop 2024-01-11T07:18:51.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.625+0000 7f7e0dcac700 1 -- 172.21.15.7:0/485163722 shutdown_connections 2024-01-11T07:18:51.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.625+0000 7f7e0dcac700 1 --2- 172.21.15.7:0/485163722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e0815e5a0 0x7f7e0815aa20 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.625+0000 7f7e0dcac700 1 -- 172.21.15.7:0/485163722 >> 172.21.15.7:0/485163722 conn(0x7f7e0806ae00 msgr2=0x7f7e0806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:51.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.625+0000 7f7e0dcac700 1 -- 172.21.15.7:0/485163722 shutdown_connections 2024-01-11T07:18:51.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.625+0000 7f7e0dcac700 1 -- 172.21.15.7:0/485163722 wait complete. 2024-01-11T07:18:51.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.627+0000 7f7e0dcac700 1 Processor -- start 2024-01-11T07:18:51.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.627+0000 7f7e0dcac700 1 -- start start 2024-01-11T07:18:51.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.627+0000 7f7e0dcac700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e080cb790 0x7f7e0816a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.627+0000 7f7e0dcac700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e080cbb60 0x7f7e08157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.627+0000 7f7e0dcac700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0815e5a0 0x7f7e080b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.627+0000 7f7e0dcac700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e081598b0 con 0x7f7e080cb790 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.627+0000 7f7e0dcac700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08158810 con 0x7f7e0815e5a0 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.627+0000 7f7e0dcac700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08158af0 con 0x7f7e080cbb60 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e080cb790 0x7f7e0816a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e080cb790 0x7f7e0816a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36732/0 (socket says 172.21.15.7:36732) 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 -- 172.21.15.7:0/4226120878 learned_addr learned my addr 172.21.15.7:0/4226120878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e07fff700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0815e5a0 0x7f7e080b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e06ffd700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e080cbb60 0x7f7e08157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 -- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e080cbb60 msgr2=0x7f7e08157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e080cbb60 0x7f7e08157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 -- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0815e5a0 msgr2=0x7f7e080b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0815e5a0 0x7f7e080b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 -- 172.21.15.7:0/4226120878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00009a20 con 0x7f7e080cb790 2024-01-11T07:18:51.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e077fe700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e080cb790 0x7f7e0816a340 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c7c0 tx=0x7f7df8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:51.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7dff7fe700 1 -- 172.21.15.7:0/4226120878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8005cd0 con 0x7f7e080cb790 2024-01-11T07:18:51.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7dff7fe700 1 -- 172.21.15.7:0/4226120878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df8005e30 con 0x7f7e080cb790 2024-01-11T07:18:51.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e08158dd0 con 0x7f7e080cb790 2024-01-11T07:18:51.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7dff7fe700 1 -- 172.21.15.7:0/4226120878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8007320 con 0x7f7e080cb790 2024-01-11T07:18:51.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.628+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e08159390 con 0x7f7e080cb790 2024-01-11T07:18:51.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.629+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0804c3c0 con 0x7f7e080cb790 2024-01-11T07:18:51.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.630+0000 7f7dff7fe700 1 -- 172.21.15.7:0/4226120878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7df8007690 con 0x7f7e080cb790 2024-01-11T07:18:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.630+0000 7f7dff7fe700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7de8066f30 0x7f7de80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.630+0000 7f7dff7fe700 1 -- 172.21.15.7:0/4226120878 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7df8083870 con 0x7f7e080cb790 2024-01-11T07:18:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.630+0000 7f7e06ffd700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7de8066f30 0x7f7de80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.631+0000 7f7e06ffd700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7de8066f30 0x7f7de80693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7e080cb490 tx=0x7f7df0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.635+0000 7f7dff7fe700 1 -- 172.21.15.7:0/4226120878 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7df8050de0 con 0x7f7e080cb790 2024-01-11T07:18:51.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.839+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7e081595b0 con 0x7f7e080cb790 2024-01-11T07:18:51.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.843+0000 7f7dff7fe700 1 -- 172.21.15.7:0/4226120878 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7df8017020 con 0x7f7e080cb790 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7de8066f30 msgr2=0x7f7de80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7de8066f30 0x7f7de80693e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7e080cb490 tx=0x7f7df0007ec0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e080cb790 msgr2=0x7f7e0816a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e080cb790 0x7f7e0816a340 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c7c0 tx=0x7f7df8007e00 comp rx=0 tx=0).stop 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 shutdown_connections 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7de8066f30 0x7f7de80693e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e0815e5a0 0x7f7e080b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e080cb790 0x7f7e0816a340 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.845+0000 7f7e0dcac700 1 --2- 172.21.15.7:0/4226120878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e080cbb60 0x7f7e08157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.846+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 >> 172.21.15.7:0/4226120878 conn(0x7f7e0806ae00 msgr2=0x7f7e080c4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.846+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 shutdown_connections 2024-01-11T07:18:51.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.846+0000 7f7e0dcac700 1 -- 172.21.15.7:0/4226120878 wait complete. 2024-01-11T07:18:51.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-01-11T07:18:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.991+0000 7fde07c17700 1 Processor -- start 2024-01-11T07:18:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.991+0000 7fde07c17700 1 -- start start 2024-01-11T07:18:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde07c17700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde001579d0 0x7fde0015c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde07c17700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde000ce3f0 0x7fde000ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde07c17700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0006d270 con 0x7fde00157da0 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde07c17700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde000cda40 con 0x7fde000ce3f0 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde07c17700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde000cd5b0 con 0x7fde001579d0 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde061b4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde000ce3f0 0x7fde000ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde061b4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde000ce3f0 0x7fde000ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36738/0 (socket says 172.21.15.7:36738) 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde061b4700 1 -- 172.21.15.7:0/3382293577 learned_addr learned my addr 172.21.15.7:0/3382293577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde059b3700 1 --2- 172.21.15.7:0/3382293577 >> v2:172.21.15.113:3300/0 conn(0x7fde001579d0 0x7fde0015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde061b4700 1 -- 172.21.15.7:0/3382293577 >> v2:172.21.15.113:3300/0 conn(0x7fde001579d0 msgr2=0x7fde0015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde061b4700 1 --2- 172.21.15.7:0/3382293577 >> v2:172.21.15.113:3300/0 conn(0x7fde001579d0 0x7fde0015c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde061b4700 1 -- 172.21.15.7:0/3382293577 >> v1:172.21.15.113:6789/0 conn(0x7fde00157da0 legacy=0x7fde000cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.992+0000 7fde061b4700 1 -- 172.21.15.7:0/3382293577 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000cc9a0 con 0x7fde000ce3f0 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde061b4700 1 --2- 172.21.15.7:0/3382293577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde000ca500 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fde0006c560 tx=0x7fddf8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb2c19cc4bfad79 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde049b1700 1 -- 172.21.15.7:0/3382293577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800e070 con 0x7fde000ce3f0 2024-01-11T07:18:51.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde049b1700 1 -- 172.21.15.7:0/3382293577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf800ba20 con 0x7fde000ce3f0 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde049b1700 1 -- 172.21.15.7:0/3382293577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf80092e0 con 0x7fde000ce3f0 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde07c17700 1 -- 172.21.15.7:0/3382293577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 msgr2=0x7fde000ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde07c17700 1 --2- 172.21.15.7:0/3382293577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde000ca500 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fde0006c560 tx=0x7fddf8009840 comp rx=0 tx=0).stop 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde07c17700 1 -- 172.21.15.7:0/3382293577 shutdown_connections 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde07c17700 1 --2- 172.21.15.7:0/3382293577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde000ca500 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde07c17700 1 --2- 172.21.15.7:0/3382293577 >> v2:172.21.15.113:3300/0 conn(0x7fde001579d0 0x7fde0015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.993+0000 7fde07c17700 1 -- 172.21.15.7:0/3382293577 >> 172.21.15.7:0/3382293577 conn(0x7fde0006ae00 msgr2=0x7fde0006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.994+0000 7fde07c17700 1 -- 172.21.15.7:0/3382293577 shutdown_connections 2024-01-11T07:18:51.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.994+0000 7fde07c17700 1 -- 172.21.15.7:0/3382293577 wait complete. 2024-01-11T07:18:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.995+0000 7fde07c17700 1 Processor -- start 2024-01-11T07:18:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.995+0000 7fde07c17700 1 -- start start 2024-01-11T07:18:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.995+0000 7fde07c17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde00150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.995+0000 7fde07c17700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde001579d0 0x7fde000b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.995+0000 7fde07c17700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00157da0 0x7fde000cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.995+0000 7fde07c17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde00150f90 con 0x7fde000ce3f0 2024-01-11T07:18:51.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.995+0000 7fde07c17700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde00151db0 con 0x7fde001579d0 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.995+0000 7fde07c17700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde00150d70 con 0x7fde00157da0 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde00150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde00150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36750/0 (socket says 172.21.15.7:36750) 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 -- 172.21.15.7:0/2456088859 learned_addr learned my addr 172.21.15.7:0/2456088859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde051b2700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde001579d0 0x7fde000b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde061b4700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00157da0 0x7fde000cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 -- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00157da0 msgr2=0x7fde000cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00157da0 0x7fde000cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 -- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde001579d0 msgr2=0x7fde000b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde001579d0 0x7fde000b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 -- 172.21.15.7:0/2456088859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf8009ae0 con 0x7fde000ce3f0 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde059b3700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde00150830 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00ec80 tx=0x7fddfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fddf5ffb700 1 -- 172.21.15.7:0/2456088859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00c850 con 0x7fde000ce3f0 2024-01-11T07:18:51.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fddf5ffb700 1 -- 172.21.15.7:0/2456088859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc00c9b0 con 0x7fde000ce3f0 2024-01-11T07:18:51.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde001511a0 con 0x7fde000ce3f0 2024-01-11T07:18:51.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.996+0000 7fddf5ffb700 1 -- 172.21.15.7:0/2456088859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00cd50 con 0x7fde000ce3f0 2024-01-11T07:18:51.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.997+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde00151760 con 0x7fde000ce3f0 2024-01-11T07:18:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.998+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0004c3c0 con 0x7fde000ce3f0 2024-01-11T07:18:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.998+0000 7fddf5ffb700 1 -- 172.21.15.7:0/2456088859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fddfc013680 con 0x7fde000ce3f0 2024-01-11T07:18:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.999+0000 7fddf5ffb700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde4066f30 0x7fdde40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.999+0000 7fddf5ffb700 1 -- 172.21.15.7:0/2456088859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fddfc084530 con 0x7fde000ce3f0 2024-01-11T07:18:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:51.999+0000 7fde051b2700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde4066f30 0x7fdde40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.000+0000 7fde051b2700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde4066f30 0x7fdde40693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fde001507c0 tx=0x7fddf0007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.003+0000 7fddf5ffb700 1 -- 172.21.15.7:0/2456088859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddfc051a00 con 0x7fde000ce3f0 2024-01-11T07:18:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.292+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7fde000ccc80 con 0x7fde000ce3f0 2024-01-11T07:18:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.456+0000 7fddf5ffb700 1 -- 172.21.15.7:0/2456088859 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v556) v1 ==== 135+0+0 (secure 0 0 0) 0x7fddfc055020 con 0x7fde000ce3f0 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde4066f30 msgr2=0x7fdde40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde4066f30 0x7fdde40693e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fde001507c0 tx=0x7fddf0007ec0 comp rx=0 tx=0).stop 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 msgr2=0x7fde00150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde00150830 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00ec80 tx=0x7fddfc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 shutdown_connections 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fdde4066f30 0x7fdde40693e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde00157da0 0x7fde000cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde000ce3f0 0x7fde00150830 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 --2- 172.21.15.7:0/2456088859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde001579d0 0x7fde000b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 >> 172.21.15.7:0/2456088859 conn(0x7fde0006ae00 msgr2=0x7fde0006b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 shutdown_connections 2024-01-11T07:18:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:52.459+0000 7fde07c17700 1 -- 172.21.15.7:0/2456088859 wait complete. 2024-01-11T07:18:52.474 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-01-11T07:18:52.474 INFO:tasks.cephfs_test_runner: 2024-01-11T07:18:52.474 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-11T07:18:52.474 INFO:tasks.cephfs_test_runner:Ran 10 tests in 680.254s 2024-01-11T07:18:52.474 INFO:tasks.cephfs_test_runner: 2024-01-11T07:18:52.474 INFO:tasks.cephfs_test_runner:OK 2024-01-11T07:18:52.475 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-01-11T07:18:52.486 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-01-11T07:18:52.509 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-11T07:18:52.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.509 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:18:52.532 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-11T07:18:52.532 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-11T07:18:52.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-11T07:18:52.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-11T07:18:52.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-11T07:18:52.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.771 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-01-11T07:18:52.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-01-11T07:18:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-01-11T07:18:52.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:18:52.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.875 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:18:52.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-11T07:18:52.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-11T07:18:52.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.896 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-11T07:18:52.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:52.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-11T07:18:53.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:53.057 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-01-11T07:18:53.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:53.157 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-01-11T07:18:53.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-11T07:18:53.175 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-01-11T07:18:53.247 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-01-11T07:18:53.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:18:53.249 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-11T07:18:53.259 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-01-11T07:18:53.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:53.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.401+0000 7f0e9d14d700 1 Processor -- start 2024-01-11T07:18:53.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.401+0000 7f0e9d14d700 1 -- start start 2024-01-11T07:18:53.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.402+0000 7f0e9d14d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 0x7f0e98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.402+0000 7f0e9d14d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e9806e1b0 0x7f0e9806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.402+0000 7f0e9d14d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e980c87a0 con 0x7f0e9815b850 2024-01-11T07:18:53.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.402+0000 7f0e9d14d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e980c9cb0 con 0x7f0e9806e1b0 2024-01-11T07:18:53.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.402+0000 7f0e9d14d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98069280 con 0x7f0e9815c330 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.402+0000 7f0e96d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0e9815b850 0x7f0e9815bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33948/0 (socket says 172.21.15.7:33948) 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e96d9d700 1 -- 172.21.15.7:0/3754197688 learned_addr learned my addr 172.21.15.7:0/3754197688 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e9759e700 1 --2- 172.21.15.7:0/3754197688 >> v2:172.21.15.113:3300/0 conn(0x7f0e9806e1b0 0x7f0e9806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e9659c700 1 --2- 172.21.15.7:0/3754197688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 0x7f0e98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e95d9b700 1 -- 172.21.15.7:0/3754197688 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759165606 0 0) 0x7f0e980c87a0 con 0x7f0e9815b850 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e95d9b700 1 -- 172.21.15.7:0/3754197688 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e84003570 con 0x7f0e9815b850 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e95d9b700 1 -- 172.21.15.7:0/3754197688 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2080221394 0 0) 0x7f0e84003570 con 0x7f0e9815b850 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e9759e700 1 -- 172.21.15.7:0/3754197688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 msgr2=0x7f0e98068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e9759e700 1 --2- 172.21.15.7:0/3754197688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 0x7f0e98068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e9759e700 1 -- 172.21.15.7:0/3754197688 >> v1:172.21.15.7:6789/0 conn(0x7f0e9815b850 legacy=0x7f0e9815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e9759e700 1 -- 172.21.15.7:0/3754197688 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e98069560 con 0x7f0e9806e1b0 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e9759e700 1 --2- 172.21.15.7:0/3754197688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 0x7f0e9806e610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009e60 tx=0x7f0e90009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b77f895d4e50860 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.403+0000 7f0e95d9b700 1 -- 172.21.15.7:0/3754197688 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9000e070 con 0x7f0e9806e1b0 2024-01-11T07:18:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e95d9b700 1 -- 172.21.15.7:0/3754197688 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9000b7a0 con 0x7f0e9806e1b0 2024-01-11T07:18:53.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e95d9b700 1 -- 172.21.15.7:0/3754197688 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88001470 con 0x7f0e9806e1b0 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e9d14d700 1 -- 172.21.15.7:0/3754197688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 msgr2=0x7f0e9806e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/3754197688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 0x7f0e9806e610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009e60 tx=0x7f0e90009780 comp rx=0 tx=0).stop 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e9d14d700 1 -- 172.21.15.7:0/3754197688 shutdown_connections 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/3754197688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 0x7f0e9806e610 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/3754197688 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 0x7f0e98068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e9d14d700 1 -- 172.21.15.7:0/3754197688 >> 172.21.15.7:0/3754197688 conn(0x7f0e98063230 msgr2=0x7f0e98063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e9d14d700 1 -- 172.21.15.7:0/3754197688 shutdown_connections 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.404+0000 7f0e9d14d700 1 -- 172.21.15.7:0/3754197688 wait complete. 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.405+0000 7f0e9d14d700 1 Processor -- start 2024-01-11T07:18:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9d14d700 1 -- start start 2024-01-11T07:18:53.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9d14d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 0x7f0e981462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9d14d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9815b850 0x7f0e9816b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9d14d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 0x7f0e980afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9d14d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980c3c50 con 0x7f0e9815b850 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9d14d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981467e0 con 0x7f0e9806e1b0 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9d14d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980c3920 con 0x7f0e9815c330 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9815b850 0x7f0e9816b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9815b850 0x7f0e9816b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36762/0 (socket says 172.21.15.7:36762) 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.406+0000 7f0e9659c700 1 -- 172.21.15.7:0/673086409 learned_addr learned my addr 172.21.15.7:0/673086409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9759e700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 0x7f0e980afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e96d9d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 0x7f0e981462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9659c700 1 -- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 msgr2=0x7f0e980afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9659c700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 0x7f0e980afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9659c700 1 -- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 msgr2=0x7f0e981462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9659c700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 0x7f0e981462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9659c700 1 -- 172.21.15.7:0/673086409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e90009a20 con 0x7f0e9815b850 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9659c700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9815b850 0x7f0e9816b230 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000e960 tx=0x7f0e800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:53.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e8effd700 1 -- 172.21.15.7:0/673086409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000c850 con 0x7f0e9815b850 2024-01-11T07:18:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e8effd700 1 -- 172.21.15.7:0/673086409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000c9b0 con 0x7f0e9815b850 2024-01-11T07:18:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e8effd700 1 -- 172.21.15.7:0/673086409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000cd50 con 0x7f0e9815b850 2024-01-11T07:18:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9816bb10 con 0x7f0e9815b850 2024-01-11T07:18:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.407+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9816c0d0 con 0x7f0e9815b850 2024-01-11T07:18:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.408+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9804c3c0 con 0x7f0e9815b850 2024-01-11T07:18:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.409+0000 7f0e8effd700 1 -- 172.21.15.7:0/673086409 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f0e80013680 con 0x7f0e9815b850 2024-01-11T07:18:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.409+0000 7f0e8effd700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.409+0000 7f0e8effd700 1 -- 172.21.15.7:0/673086409 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0e80084460 con 0x7f0e9815b850 2024-01-11T07:18:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.409+0000 7f0e96d9d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.410+0000 7f0e96d9d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e78066f30 0x7f0e780693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e880011e0 tx=0x7f0e88001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:53.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.414+0000 7f0e8effd700 1 -- 172.21.15.7:0/673086409 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e800519d0 con 0x7f0e9815b850 2024-01-11T07:18:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.622+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e980c3d90 con 0x7f0e9815b850 2024-01-11T07:18:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.623+0000 7f0e8effd700 1 -- 172.21.15.7:0/673086409 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+12224 (secure 0 0 0) 0x7f0e80054ff0 con 0x7f0e9815b850 2024-01-11T07:18:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:53.623 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":191,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:51.452494+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3920146306":"2024-01-12T07:18:48.442926+0000","172.21.15.7:6834/3920146306":"2024-01-12T07:18:48.442926+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.625+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e78066f30 msgr2=0x7f0e780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.625+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e78066f30 0x7f0e780693e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e880011e0 tx=0x7f0e88001520 comp rx=0 tx=0).stop 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.625+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9815b850 msgr2=0x7f0e9816b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.625+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9815b850 0x7f0e9816b230 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000e960 tx=0x7f0e800091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.626+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 shutdown_connections 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.626+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f0e78066f30 0x7f0e780693e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.626+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e9815c330 0x7f0e980afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.626+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e9806e1b0 0x7f0e981462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.626+0000 7f0e9d14d700 1 --2- 172.21.15.7:0/673086409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e9815b850 0x7f0e9816b230 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.626+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 >> 172.21.15.7:0/673086409 conn(0x7f0e98063230 msgr2=0x7f0e9806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.626+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 shutdown_connections 2024-01-11T07:18:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.626+0000 7f0e9d14d700 1 -- 172.21.15.7:0/673086409 wait complete. 2024-01-11T07:18:53.639 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-11T07:18:53.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:18:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.771+0000 7f4b0865d700 1 Processor -- start 2024-01-11T07:18:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.771+0000 7f4b0865d700 1 -- start start 2024-01-11T07:18:53.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b0865d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b00159ca0 0x7f4b0006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b0865d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b0015a070 0x7f4b0006e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b0865d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b000d18c0 con 0x7f4b000ce7f0 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b0865d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b000d0010 con 0x7f4b0015a070 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b0865d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b000cf400 con 0x7f4b00159ca0 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b05bf8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b0015a070 0x7f4b0006e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b05bf8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b0015a070 0x7f4b0006e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36774/0 (socket says 172.21.15.7:36774) 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b05bf8700 1 -- 172.21.15.7:0/6705268 learned_addr learned my addr 172.21.15.7:0/6705268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b063f9700 1 --2- 172.21.15.7:0/6705268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b00159ca0 0x7f4b0006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b05bf8700 1 -- 172.21.15.7:0/6705268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b00159ca0 msgr2=0x7f4b0006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b05bf8700 1 --2- 172.21.15.7:0/6705268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b00159ca0 0x7f4b0006dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b05bf8700 1 -- 172.21.15.7:0/6705268 >> v1:172.21.15.113:6789/0 conn(0x7f4b000ce7f0 legacy=0x7f4b000cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.772+0000 7f4b05bf8700 1 -- 172.21.15.7:0/6705268 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000cf6e0 con 0x7f4b0015a070 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.773+0000 7f4b05bf8700 1 --2- 172.21.15.7:0/6705268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b0006e170 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4af00099b0 tx=0x7f4af0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac0ea2ab3997797 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:53.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.773+0000 7f4b053f7700 1 -- 172.21.15.7:0/6705268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000b860 con 0x7f4b0015a070 2024-01-11T07:18:53.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.773+0000 7f4b053f7700 1 -- 172.21.15.7:0/6705268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af000b9c0 con 0x7f4b0015a070 2024-01-11T07:18:53.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.773+0000 7f4b053f7700 1 -- 172.21.15.7:0/6705268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000bd60 con 0x7f4b0015a070 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.773+0000 7f4b0865d700 1 -- 172.21.15.7:0/6705268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 msgr2=0x7f4b0006e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.773+0000 7f4b0865d700 1 --2- 172.21.15.7:0/6705268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b0006e170 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4af00099b0 tx=0x7f4af0009250 comp rx=0 tx=0).stop 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.774+0000 7f4b0865d700 1 -- 172.21.15.7:0/6705268 shutdown_connections 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.774+0000 7f4b0865d700 1 --2- 172.21.15.7:0/6705268 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b00159ca0 0x7f4b0006dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.774+0000 7f4b0865d700 1 --2- 172.21.15.7:0/6705268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b0006e170 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.774+0000 7f4b0865d700 1 -- 172.21.15.7:0/6705268 >> 172.21.15.7:0/6705268 conn(0x7f4b00063cc0 msgr2=0x7f4b000d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.774+0000 7f4b0865d700 1 -- 172.21.15.7:0/6705268 shutdown_connections 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.774+0000 7f4b0865d700 1 -- 172.21.15.7:0/6705268 wait complete. 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.775+0000 7f4b0865d700 1 Processor -- start 2024-01-11T07:18:53.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.775+0000 7f4b0865d700 1 -- start start 2024-01-11T07:18:53.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.776+0000 7f4b0865d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b000ce7f0 0x7f4b00154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.776+0000 7f4b0865d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b00159ca0 0x7f4b0016aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.776+0000 7f4b0865d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b000b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.776+0000 7f4b0865d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00154ed0 con 0x7f4b0015a070 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.776+0000 7f4b0865d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0016a5c0 con 0x7f4b00159ca0 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.776+0000 7f4b0865d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0016ab60 con 0x7f4b000ce7f0 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.776+0000 7f4b06bfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b000b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.776+0000 7f4b06bfa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b000b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36780/0 (socket says 172.21.15.7:36780) 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b06bfa700 1 -- 172.21.15.7:0/927581347 learned_addr learned my addr 172.21.15.7:0/927581347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b063f9700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b000ce7f0 0x7f4b00154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b05bf8700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b00159ca0 0x7f4b0016aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b06bfa700 1 -- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b000ce7f0 msgr2=0x7f4b00154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b06bfa700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b000ce7f0 0x7f4b00154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b06bfa700 1 -- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b00159ca0 msgr2=0x7f4b0016aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b06bfa700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b00159ca0 0x7f4b0016aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b06bfa700 1 -- 172.21.15.7:0/927581347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00094f0 con 0x7f4b0015a070 2024-01-11T07:18:53.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b06bfa700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b000b39c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4af800c910 tx=0x7f4af8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:53.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4af67fc700 1 -- 172.21.15.7:0/927581347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8012630 con 0x7f4b0015a070 2024-01-11T07:18:53.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4af67fc700 1 -- 172.21.15.7:0/927581347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af80127b0 con 0x7f4b0015a070 2024-01-11T07:18:53.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b000b4380 con 0x7f4b0015a070 2024-01-11T07:18:53.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4af67fc700 1 -- 172.21.15.7:0/927581347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8012b70 con 0x7f4b0015a070 2024-01-11T07:18:53.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.777+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0016a7f0 con 0x7f4b0015a070 2024-01-11T07:18:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.778+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b000b48a0 con 0x7f4b0015a070 2024-01-11T07:18:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.779+0000 7f4af67fc700 1 -- 172.21.15.7:0/927581347 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f4af8003030 con 0x7f4b0015a070 2024-01-11T07:18:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.779+0000 7f4af67fc700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ae4066f30 0x7f4ae40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.779+0000 7f4af67fc700 1 -- 172.21.15.7:0/927581347 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4af80841f0 con 0x7f4b0015a070 2024-01-11T07:18:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.780+0000 7f4b063f9700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ae4066f30 0x7f4ae40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:53.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.780+0000 7f4b063f9700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ae4066f30 0x7f4ae40693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006fd0 tx=0x7f4afc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:53.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.784+0000 7f4af67fc700 1 -- 172.21.15.7:0/927581347 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4af80516c0 con 0x7f4b0015a070 2024-01-11T07:18:53.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.983+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b0016acd0 con 0x7f4ae4066f30 2024-01-11T07:18:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.986+0000 7f4af67fc700 1 -- 172.21.15.7:0/927581347 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50389 (secure 0 0 0) 0x7f4b0016acd0 con 0x7f4ae4066f30 2024-01-11T07:18:53.986 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ae4066f30 msgr2=0x7f4ae40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ae4066f30 0x7f4ae40693e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006fd0 tx=0x7f4afc008040 comp rx=0 tx=0).stop 2024-01-11T07:18:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 msgr2=0x7f4b000b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b000b39c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4af800c910 tx=0x7f4af8001710 comp rx=0 tx=0).stop 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 shutdown_connections 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f4ae4066f30 0x7f4ae40693e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b0015a070 0x7f4b000b39c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b000ce7f0 0x7f4b00154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 --2- 172.21.15.7:0/927581347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b00159ca0 0x7f4b0016aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 >> 172.21.15.7:0/927581347 conn(0x7f4b00063cc0 msgr2=0x7f4b001553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 shutdown_connections 2024-01-11T07:18:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:53.988+0000 7f4b0865d700 1 -- 172.21.15.7:0/927581347 wait complete. 2024-01-11T07:18:53.990 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:18:54.001 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-01-11T07:18:52.012036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":111934,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450565,"statfs":{"total":1600000000,"available":1485379123,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52484901,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-117,"num_write_kb":-85075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.707796"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":314,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-01-11T07:18:47.307533+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:18:47.307533+0000","last_peered":"2024-01-11T07:18:47.307533+0000","last_clean":"2024-01-11T07:18:47.307533+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:18:47.307533+0000","last_undegraded":"2024-01-11T07:18:47.307533+0000","last_fullsized":"2024-01-11T07:18:47.307533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9858,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185454,"statfs":{"total":200000000,"available":189905408,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.433,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.56599999999999995},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.80500000000000005},"last":0.498}]},{"osd":2,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.81399999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75800000000000001,"5min":0.78700000000000003,"15min":0.82399999999999995},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.82999999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42799999999999999,"15min":0.40600000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.83799999999999997},"last":0.67500000000000004}]},{"osd":4,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45600000000000002,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.77700000000000002,"15min":0.85999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.33800000000000002,"15min":0.308},"max":{"1min":0.59599999999999997,"5min":0.755,"15min":0.85899999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.66300000000000003,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.66600000000000004,"5min":0.84599999999999997,"15min":0.91600000000000004},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.82199999999999995,"15min":0.92500000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.90300000000000002},"last":0.57899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19468,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175843,"statfs":{"total":200000000,"available":180063908,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.81399999999999995},"last":0.628},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.44600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77100000000000002,"5min":0.85199999999999998,"15min":1.157},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.45800000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65000000000000002,"5min":0.79500000000000004,"15min":2.5720000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":0.67300000000000004,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.84599999999999997,"15min":2.621},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88500000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48099999999999998,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.25800000000000001},"max":{"1min":0.66300000000000003,"5min":0.76500000000000001,"15min":0.82699999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.253},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":2.0209999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.313},"max":{"1min":0.69299999999999995,"5min":0.77900000000000003,"15min":0.89100000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28699999999999998},"max":{"1min":0.64100000000000001,"5min":0.68600000000000005,"15min":0.76700000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53200000000000003,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":0.70899999999999996,"15min":2.0750000000000002},"last":0.67400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13318,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181993,"statfs":{"total":200000000,"available":186361623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.247,"15min":0.216},"max":{"1min":0.53100000000000003,"5min":0.753,"15min":20.41},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.46100000000000002},"min":{"1min":0.371,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":0.60699999999999998,"5min":0.72099999999999997,"15min":3.7029999999999998},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44,"15min":0.497},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60199999999999998,"5min":3.597,"15min":15.611000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":3.0710000000000002,"15min":20.199999999999999},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":0.67900000000000005,"5min":3.8460000000000001,"15min":20.369},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.28299999999999997},"max":{"1min":0.499,"5min":4.7569999999999997,"15min":8.5790000000000006},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":1.3300000000000001,"15min":9.9570000000000007},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":1.2030000000000001,"15min":12.449999999999999},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":1.3540000000000001,"15min":2.6030000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.51300000000000001,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.90900000000000003,"5min":1.177,"15min":9.891},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":1.389,"15min":1.5569999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69199999999999995,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.94699999999999995,"5min":2.9049999999999998,"15min":9.9900000000000002},"last":0.48699999999999999}]},{"osd":7,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.1599999999999999,"15min":20.324000000000002},"last":0.621},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.46800000000000003},"max":{"1min":0.96999999999999997,"5min":1.268,"15min":2.548},"last":0.71899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9805,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185507,"statfs":{"total":200000000,"available":189959524,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39600000000000002,"15min":0.42999999999999999},"min":{"1min":0.27100000000000002,"5min":0.23300000000000001,"15min":0.224},"max":{"1min":0.57899999999999996,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.29299999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57399999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.48799999999999999},"min":{"1min":0.36499999999999999,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.61699999999999999,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.57799999999999996},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58899999999999997,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64000000000000001,"5min":0.83499999999999996,"15min":2.855},"last":0.38}]},{"osd":4,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.495,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90900000000000003,"15min":1.3959999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.40999999999999998},"max":{"1min":0.92500000000000004,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":1.032,"15min":2.819},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.96599999999999997,"15min":1.2490000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95899999999999996,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.501,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.68000000000000005},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.276},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.747,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.47499999999999998,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":0.54800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173801,"statfs":{"total":200000000,"available":177972959,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.504,"5min":0.72999999999999998,"15min":1.472},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.311,"5min":0.27900000000000003,"15min":0.23599999999999999},"max":{"1min":0.57799999999999996,"5min":0.65500000000000003,"15min":0.86099999999999999},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.58899999999999997,"5min":0.628,"15min":1.2430000000000001},"last":0.375},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48799999999999999},"min":{"1min":0.41399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63300000000000001,"5min":0.752,"15min":0.76800000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.63100000000000001,"5min":0.76500000000000001,"15min":1.139},"last":0.45300000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.0660000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.089},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48199999999999998,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":1.054,"15min":1.099},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":1.0389999999999999,"15min":1.3100000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0109999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.052},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0580000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.55400000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.59599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9222,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186089,"statfs":{"total":200000000,"available":190555725,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.57399999999999995},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":0.58199999999999996,"5min":0.59699999999999998,"15min":11.481},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.47299999999999998},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":1.6200000000000001},"last":0.378}]},{"osd":1,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40699999999999997,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.23899999999999999},"max":{"1min":0.59899999999999998,"5min":0.70499999999999996,"15min":17.516999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67400000000000004},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":0.56000000000000005,"5min":0.746,"15min":0.76500000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46400000000000002,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":17.553000000000001},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.94599999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.95099999999999996},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.36199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":17.474},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64600000000000002,"15min":0.89600000000000002},"min":{"1min":0.49199999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":17.393000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.81899999999999995},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.94499999999999995,"15min":11.394},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.82899999999999996},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.91900000000000004,"15min":11.436999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.878,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.96999999999999997,"15min":1.0449999999999999},"last":0.747},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.93799999999999994},"min":{"1min":0.53700000000000003,"5min":0.45800000000000002,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":0.95499999999999996,"15min":17.434000000000001},"last":0.58799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7232,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188079,"statfs":{"total":200000000,"available":192593572,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.81299999999999994},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.73599999999999999,"5min":0.94799999999999995,"15min":8.484},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.84599999999999997},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":0.84699999999999998,"15min":9.2599999999999998},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":1.1599999999999999,"15min":8.5340000000000007},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.78100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.81000000000000005,"15min":9.2279999999999998},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59999999999999998,"15min":0.78200000000000003},"min":{"1min":0.376,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.71999999999999997,"5min":1.1120000000000001,"15min":9.5640000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.88600000000000001,"15min":6.8639999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.78500000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":8.9779999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.84699999999999998},"min":{"1min":0.50800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":1.218,"15min":8.7309999999999999},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.67700000000000005},"min":{"1min":0.318,"5min":0.313,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":11.388999999999999},"last":0.624},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46899999999999997,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":8.6509999999999998},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.69399999999999995},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":9.1449999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.81899999999999995,"15min":1.119},"last":0.66400000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52200000000000002,"15min":0.69499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33500000000000002},"max":{"1min":0.70599999999999996,"5min":0.85399999999999998,"15min":8.6050000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":9.4640000000000004},"last":0.71799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21517,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173795,"statfs":{"total":200000000,"available":177966404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.81899999999999995},"last":0.432}]},{"osd":1,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.57699999999999996},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.24399999999999999},"max":{"1min":0.73999999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.376},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.93999999999999995},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34999999999999998},"max":{"1min":0.70399999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.74199999999999999,"15min":0.76000000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.496,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.73799999999999999,"15min":0.78600000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.56100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.0109999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.88200000000000001},"last":0.52700000000000002}]}]}],"pool_statfs":[]}} 2024-01-11T07:18:54.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:18:54.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.136+0000 7fc822e58700 1 Processor -- start 2024-01-11T07:18:54.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.136+0000 7fc822e58700 1 -- start start 2024-01-11T07:18:54.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.136+0000 7fc822e58700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc81c06e340 0x7fc81c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.136+0000 7fc822e58700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c15a070 0x7fc81c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.136+0000 7fc822e58700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81c0d18c0 con 0x7fc81c159ca0 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.136+0000 7fc822e58700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c0d0010 con 0x7fc81c06e340 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.136+0000 7fc822e58700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0cf400 con 0x7fc81c15a070 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc81c06e340 0x7fc81c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc81c06e340 0x7fc81c0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36790/0 (socket says 172.21.15.7:36790) 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81bfff700 1 -- 172.21.15.7:0/2689181696 learned_addr learned my addr 172.21.15.7:0/2689181696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc8213f5700 1 --2- 172.21.15.7:0/2689181696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c15a070 0x7fc81c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81bfff700 1 -- 172.21.15.7:0/2689181696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c15a070 msgr2=0x7fc81c0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81bfff700 1 --2- 172.21.15.7:0/2689181696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c15a070 0x7fc81c0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81bfff700 1 -- 172.21.15.7:0/2689181696 >> v1:172.21.15.113:6789/0 conn(0x7fc81c159ca0 legacy=0x7fc81c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81bfff700 1 -- 172.21.15.7:0/2689181696 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0cf6e0 con 0x7fc81c06e340 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81bfff700 1 --2- 172.21.15.7:0/2689181696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c06e340 0x7fc81c0ce7f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc8080099b0 tx=0x7fc808009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190880b71653353a server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.137+0000 7fc81b7fe700 1 -- 172.21.15.7:0/2689181696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800b820 con 0x7fc81c06e340 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc81b7fe700 1 -- 172.21.15.7:0/2689181696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80800b980 con 0x7fc81c06e340 2024-01-11T07:18:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc81b7fe700 1 -- 172.21.15.7:0/2689181696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800bd20 con 0x7fc81c06e340 2024-01-11T07:18:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc822e58700 1 -- 172.21.15.7:0/2689181696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c06e340 msgr2=0x7fc81c0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc822e58700 1 --2- 172.21.15.7:0/2689181696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c06e340 0x7fc81c0ce7f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc8080099b0 tx=0x7fc808009250 comp rx=0 tx=0).stop 2024-01-11T07:18:54.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc822e58700 1 -- 172.21.15.7:0/2689181696 shutdown_connections 2024-01-11T07:18:54.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc822e58700 1 --2- 172.21.15.7:0/2689181696 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c15a070 0x7fc81c0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc822e58700 1 --2- 172.21.15.7:0/2689181696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c06e340 0x7fc81c0ce7f0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc822e58700 1 -- 172.21.15.7:0/2689181696 >> 172.21.15.7:0/2689181696 conn(0x7fc81c063cc0 msgr2=0x7fc81c0d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:54.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc822e58700 1 -- 172.21.15.7:0/2689181696 shutdown_connections 2024-01-11T07:18:54.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.138+0000 7fc822e58700 1 -- 172.21.15.7:0/2689181696 wait complete. 2024-01-11T07:18:54.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.140+0000 7fc822e58700 1 Processor -- start 2024-01-11T07:18:54.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.140+0000 7fc822e58700 1 -- start start 2024-01-11T07:18:54.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc822e58700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c06e340 0x7fc81c150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc822e58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c159ca0 0x7fc81c0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc822e58700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81c15a070 0x7fc81c152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc822e58700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15d5a0 con 0x7fc81c159ca0 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc822e58700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c1508b0 con 0x7fc81c15a070 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc822e58700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0b5880 con 0x7fc81c06e340 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c159ca0 0x7fc81c0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c159ca0 0x7fc81c0b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36798/0 (socket says 172.21.15.7:36798) 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc81bfff700 1 -- 172.21.15.7:0/3297262187 learned_addr learned my addr 172.21.15.7:0/3297262187 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc820bf4700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c06e340 0x7fc81c150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.141+0000 7fc8213f5700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81c15a070 0x7fc81c152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc81bfff700 1 -- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c06e340 msgr2=0x7fc81c150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc81bfff700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c06e340 0x7fc81c150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc81bfff700 1 -- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81c15a070 msgr2=0x7fc81c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc81bfff700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81c15a070 0x7fc81c152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc81bfff700 1 -- 172.21.15.7:0/3297262187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080094f0 con 0x7fc81c159ca0 2024-01-11T07:18:54.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc81bfff700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c159ca0 0x7fc81c0b5340 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc808009290 tx=0x7fc808016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc818ff9700 1 -- 172.21.15.7:0/3297262187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808011d90 con 0x7fc81c159ca0 2024-01-11T07:18:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc818ff9700 1 -- 172.21.15.7:0/3297262187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808020070 con 0x7fc81c159ca0 2024-01-11T07:18:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc818ff9700 1 -- 172.21.15.7:0/3297262187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80801b6a0 con 0x7fc81c159ca0 2024-01-11T07:18:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c15c6b0 con 0x7fc81c159ca0 2024-01-11T07:18:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.142+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c15cc70 con 0x7fc81c159ca0 2024-01-11T07:18:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.143+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c15d710 con 0x7fc81c159ca0 2024-01-11T07:18:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.143+0000 7fc818ff9700 1 -- 172.21.15.7:0/3297262187 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fc808022030 con 0x7fc81c159ca0 2024-01-11T07:18:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.144+0000 7fc818ff9700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc800066f30 0x7fc8000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.144+0000 7fc818ff9700 1 -- 172.21.15.7:0/3297262187 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc8080861b0 con 0x7fc81c159ca0 2024-01-11T07:18:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.144+0000 7fc820bf4700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc800066f30 0x7fc8000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.145+0000 7fc820bf4700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc800066f30 0x7fc8000693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc810003870 tx=0x7fc810009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.148+0000 7fc818ff9700 1 -- 172.21.15.7:0/3297262187 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc808053720 con 0x7fc81c159ca0 2024-01-11T07:18:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.344+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc81c15d6e0 con 0x7fc800066f30 2024-01-11T07:18:54.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.347+0000 7fc818ff9700 1 -- 172.21.15.7:0/3297262187 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50284 (secure 0 0 0) 0x7fc81c15d6e0 con 0x7fc800066f30 2024-01-11T07:18:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc800066f30 msgr2=0x7fc8000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc800066f30 0x7fc8000693e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc810003870 tx=0x7fc810009040 comp rx=0 tx=0).stop 2024-01-11T07:18:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c159ca0 msgr2=0x7fc81c0b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c159ca0 0x7fc81c0b5340 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc808009290 tx=0x7fc808016000 comp rx=0 tx=0).stop 2024-01-11T07:18:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 shutdown_connections 2024-01-11T07:18:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fc800066f30 0x7fc8000693e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc81c15a070 0x7fc81c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc81c06e340 0x7fc81c150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 --2- 172.21.15.7:0/3297262187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc81c159ca0 0x7fc81c0b5340 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 >> 172.21.15.7:0/3297262187 conn(0x7fc81c063cc0 msgr2=0x7fc81c14fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 shutdown_connections 2024-01-11T07:18:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.349+0000 7fc822e58700 1 -- 172.21.15.7:0/3297262187 wait complete. 2024-01-11T07:18:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:18:54.362 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2024-01-11T07:18:54.012376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9536,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552963,"statfs":{"total":1600000000,"available":1590234912,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52484901,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-117,"num_write_kb":-85075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001417"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":316,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-01-11T07:18:50.457881+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:18:50.457881+0000","last_peered":"2024-01-11T07:18:50.457881+0000","last_clean":"2024-01-11T07:18:50.457881+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:18:50.457881+0000","last_undegraded":"2024-01-11T07:18:50.457881+0000","last_fullsized":"2024-01-11T07:18:50.457881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193688,"statfs":{"total":200000000,"available":198336588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.433,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":4.9210000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.56599999999999995},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.80500000000000005},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.81399999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75800000000000001,"5min":0.78700000000000003,"15min":0.82399999999999995},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.82999999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42799999999999999,"15min":0.40600000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.83799999999999997},"last":4.9589999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45600000000000002,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.77700000000000002,"15min":0.85999999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.33800000000000002,"15min":0.308},"max":{"1min":0.59599999999999997,"5min":0.755,"15min":0.85899999999999999},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.66300000000000003,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.66600000000000004,"5min":0.84599999999999997,"15min":0.91600000000000004},"last":0.628}]},{"osd":6,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.82199999999999995,"15min":0.92500000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.90300000000000002},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.81399999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.44600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77100000000000002,"5min":0.85199999999999998,"15min":1.157},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.45800000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65000000000000002,"5min":0.79500000000000004,"15min":2.5720000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":0.67300000000000004,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.84599999999999997,"15min":2.621},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88500000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48099999999999998,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.25800000000000001},"max":{"1min":0.66300000000000003,"5min":0.76500000000000001,"15min":0.82699999999999996},"last":0.5},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.253},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":2.0209999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.313},"max":{"1min":0.69299999999999995,"5min":0.77900000000000003,"15min":0.89100000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28699999999999998},"max":{"1min":0.64100000000000001,"5min":0.68600000000000005,"15min":0.76700000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53200000000000003,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":0.70899999999999996,"15min":2.0750000000000002},"last":0.54500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.247,"15min":0.216},"max":{"1min":0.53100000000000003,"5min":0.753,"15min":20.41},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.46100000000000002},"min":{"1min":0.371,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":0.60699999999999998,"5min":0.72099999999999997,"15min":3.7029999999999998},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44,"15min":0.497},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60199999999999998,"5min":3.597,"15min":15.611000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":3.0710000000000002,"15min":20.199999999999999},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":0.67900000000000005,"5min":3.8460000000000001,"15min":20.369},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.28299999999999997},"max":{"1min":0.499,"5min":4.7569999999999997,"15min":8.5790000000000006},"last":0.317}]},{"osd":4,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":1.3300000000000001,"15min":9.9570000000000007},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":1.2030000000000001,"15min":12.449999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":1.3540000000000001,"15min":2.6030000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.51300000000000001,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.90900000000000003,"5min":1.177,"15min":9.891},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":1.389,"15min":1.5569999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69199999999999995,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.94699999999999995,"5min":2.9049999999999998,"15min":9.9900000000000002},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.1599999999999999,"15min":20.324000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.46800000000000003},"max":{"1min":0.96999999999999997,"5min":1.268,"15min":2.548},"last":0.63}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193688,"statfs":{"total":200000000,"available":198336588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39600000000000002,"15min":0.42999999999999999},"min":{"1min":0.27100000000000002,"5min":0.23300000000000001,"15min":0.224},"max":{"1min":0.57899999999999996,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.29299999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57399999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.48799999999999999},"min":{"1min":0.36499999999999999,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.61699999999999999,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.57799999999999996},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58899999999999997,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64000000000000001,"5min":0.83499999999999996,"15min":2.855},"last":0.752}]},{"osd":4,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.495,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90900000000000003,"15min":1.3959999999999999},"last":1.004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.40999999999999998},"max":{"1min":0.92500000000000004,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":1.4710000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":1.032,"15min":2.819},"last":0.872},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.96599999999999997,"15min":1.2490000000000001},"last":1.1140000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95899999999999996,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":1.649},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.501,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":2.3620000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.68000000000000005},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.276},"last":1.206},{"interface":"front","average":{"1min":0.748,"5min":0.747,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.47499999999999998,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":1.595}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.504,"5min":0.72999999999999998,"15min":1.472},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.311,"5min":0.27900000000000003,"15min":0.23599999999999999},"max":{"1min":0.57799999999999996,"5min":0.65500000000000003,"15min":0.86099999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.58899999999999997,"5min":0.628,"15min":1.2430000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48799999999999999},"min":{"1min":0.41399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63300000000000001,"5min":0.752,"15min":0.76800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.63100000000000001,"5min":0.76500000000000001,"15min":1.139},"last":0.499}]},{"osd":4,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.0660000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.089},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48199999999999998,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":1.054,"15min":1.099},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":1.0389999999999999,"15min":1.3100000000000001},"last":0.625}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0109999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.052},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0580000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.55400000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.57399999999999995},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":0.58199999999999996,"5min":0.59699999999999998,"15min":11.481},"last":0.376},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.47299999999999998},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":1.6200000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40699999999999997,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.23899999999999999},"max":{"1min":0.59899999999999998,"5min":0.70499999999999996,"15min":17.516999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67400000000000004},"last":0.435}]},{"osd":2,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":0.56000000000000005,"5min":0.746,"15min":0.76500000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46400000000000002,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":17.553000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.94599999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.95099999999999996},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.36199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":17.474},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64600000000000002,"15min":0.89600000000000002},"min":{"1min":0.49199999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":17.393000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.81899999999999995},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.94499999999999995,"15min":11.394},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.82899999999999996},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.91900000000000004,"15min":11.436999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.878,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.96999999999999997,"15min":1.0449999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.93799999999999994},"min":{"1min":0.53700000000000003,"5min":0.45800000000000002,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":0.95499999999999996,"15min":17.434000000000001},"last":0.64900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.81299999999999994},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.73599999999999999,"5min":0.94799999999999995,"15min":8.484},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.84599999999999997},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":0.84699999999999998,"15min":9.2599999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":1.1599999999999999,"15min":8.5340000000000007},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.78100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.81000000000000005,"15min":9.2279999999999998},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59999999999999998,"15min":0.78200000000000003},"min":{"1min":0.376,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.71999999999999997,"5min":1.1120000000000001,"15min":9.5640000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.88600000000000001,"15min":6.8639999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.78500000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":8.9779999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.84699999999999998},"min":{"1min":0.50800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":1.218,"15min":8.7309999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.67700000000000005},"min":{"1min":0.318,"5min":0.313,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":11.388999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46899999999999997,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":8.6509999999999998},"last":0.41499999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.69399999999999995},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":9.1449999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.81899999999999995,"15min":1.119},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52200000000000002,"15min":0.69499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33500000000000002},"max":{"1min":0.70599999999999996,"5min":0.85399999999999998,"15min":8.6050000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":9.4640000000000004},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.81899999999999995},"last":0.495}]},{"osd":1,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.57699999999999996},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.24399999999999999},"max":{"1min":0.73999999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.93999999999999995},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34999999999999998},"max":{"1min":0.70399999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.74199999999999999,"15min":0.76000000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.496,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.73799999999999999,"15min":0.78600000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.56100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.39300000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.0109999999999999},"last":0.436},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.88200000000000001},"last":0.55400000000000005}]}]}],"pool_statfs":[]}} 2024-01-11T07:18:54.364 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-11T07:18:54.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:18:54.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.497+0000 7f69b8370700 1 Processor -- start 2024-01-11T07:18:54.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.497+0000 7f69b8370700 1 -- start start 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.498+0000 7f69b8370700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b0063230 0x7f69b0063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.498+0000 7f69b8370700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b00ab100 con 0x7f69b00ac5b0 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.498+0000 7f69b8370700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b015d880 con 0x7f69b015a280 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.498+0000 7f69b8370700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b006c5b0 con 0x7f69b0063230 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.498+0000 7f69b690d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f69b00ac5b0 0x7f69b00aca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33962/0 (socket says 172.21.15.7:33962) 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.498+0000 7f69b690d700 1 -- 172.21.15.7:0/3190145419 learned_addr learned my addr 172.21.15.7:0/3190145419 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.498+0000 7f69b610c700 1 --2- 172.21.15.7:0/3190145419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b0063230 0x7f69b0063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b510a700 1 -- 172.21.15.7:0/3190145419 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862486977 0 0) 0x7f69b00ab100 con 0x7f69b00ac5b0 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b510a700 1 -- 172.21.15.7:0/3190145419 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f699c003570 con 0x7f69b00ac5b0 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b510a700 1 -- 172.21.15.7:0/3190145419 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409045262 0 0) 0x7f69b015d880 con 0x7f69b015a280 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b510a700 1 -- 172.21.15.7:0/3190145419 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69b00ab100 con 0x7f69b015a280 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b610c700 1 -- 172.21.15.7:0/3190145419 >> v1:172.21.15.113:6789/0 conn(0x7f69b015a280 legacy=0x7f69b00aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b610c700 1 -- 172.21.15.7:0/3190145419 >> v1:172.21.15.7:6789/0 conn(0x7f69b00ac5b0 legacy=0x7f69b00aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b610c700 1 -- 172.21.15.7:0/3190145419 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b0063b40 con 0x7f69b0063230 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b610c700 1 --2- 172.21.15.7:0/3190145419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b0063230 0x7f69b0063600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009910 tx=0x7f69a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee385f3c350b01d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b510a700 1 -- 172.21.15.7:0/3190145419 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4011630 con 0x7f69b0063230 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b510a700 1 -- 172.21.15.7:0/3190145419 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a4011790 con 0x7f69b0063230 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.499+0000 7f69b510a700 1 -- 172.21.15.7:0/3190145419 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4011b10 con 0x7f69b0063230 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.500+0000 7f69b8370700 1 -- 172.21.15.7:0/3190145419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b0063230 msgr2=0x7f69b0063600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.500+0000 7f69b8370700 1 --2- 172.21.15.7:0/3190145419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b0063230 0x7f69b0063600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009910 tx=0x7f69a40091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.500+0000 7f69b8370700 1 -- 172.21.15.7:0/3190145419 shutdown_connections 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.500+0000 7f69b8370700 1 --2- 172.21.15.7:0/3190145419 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b0063230 0x7f69b0063600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.500+0000 7f69b8370700 1 -- 172.21.15.7:0/3190145419 >> 172.21.15.7:0/3190145419 conn(0x7f69b00680e0 msgr2=0x7f69b00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.500+0000 7f69b8370700 1 -- 172.21.15.7:0/3190145419 shutdown_connections 2024-01-11T07:18:54.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.500+0000 7f69b8370700 1 -- 172.21.15.7:0/3190145419 wait complete. 2024-01-11T07:18:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.501+0000 7f69b8370700 1 Processor -- start 2024-01-11T07:18:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.501+0000 7f69b8370700 1 -- start start 2024-01-11T07:18:54.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b8370700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b0063230 0x7f69b00c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b8370700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b00ac5b0 0x7f69b00b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b8370700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b015a280 0x7f69b00b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b8370700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0069f00 con 0x7f69b0063230 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b8370700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b006a4c0 con 0x7f69b015a280 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b8370700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00c36d0 con 0x7f69b00ac5b0 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b610c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b0063230 0x7f69b00c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b610c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b0063230 0x7f69b00c30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36814/0 (socket says 172.21.15.7:36814) 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b610c700 1 -- 172.21.15.7:0/4092434492 learned_addr learned my addr 172.21.15.7:0/4092434492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b590b700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b00ac5b0 0x7f69b00b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b690d700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b015a280 0x7f69b00b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b610c700 1 -- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b00ac5b0 msgr2=0x7f69b00b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b610c700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b00ac5b0 0x7f69b00b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b610c700 1 -- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b015a280 msgr2=0x7f69b00b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b610c700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b015a280 0x7f69b00b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b610c700 1 -- 172.21.15.7:0/4092434492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a4009420 con 0x7f69b0063230 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b590b700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b00ac5b0 0x7f69b00b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.502+0000 7f69b690d700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b015a280 0x7f69b00b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.503+0000 7f69b610c700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b0063230 0x7f69b00c30e0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f69b006d820 tx=0x7f69a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.503+0000 7f69a27fc700 1 -- 172.21.15.7:0/4092434492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4011b10 con 0x7f69b0063230 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.503+0000 7f69a27fc700 1 -- 172.21.15.7:0/4092434492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a40253f0 con 0x7f69b0063230 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.503+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b00c39b0 con 0x7f69b0063230 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.503+0000 7f69a27fc700 1 -- 172.21.15.7:0/4092434492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a40257d0 con 0x7f69b0063230 2024-01-11T07:18:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.503+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b00b3410 con 0x7f69b0063230 2024-01-11T07:18:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.504+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b004c3c0 con 0x7f69b0063230 2024-01-11T07:18:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.504+0000 7f69a27fc700 1 -- 172.21.15.7:0/4092434492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f69a40193b0 con 0x7f69b0063230 2024-01-11T07:18:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.505+0000 7f69a27fc700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.505+0000 7f69a27fc700 1 -- 172.21.15.7:0/4092434492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f69a4085ef0 con 0x7f69b0063230 2024-01-11T07:18:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.505+0000 7f69b590b700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.506+0000 7f69b590b700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6994066f30 0x7f69940693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f69b00c10a0 tx=0x7f69ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.510+0000 7f69a27fc700 1 -- 172.21.15.7:0/4092434492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69a4053460 con 0x7f69b0063230 2024-01-11T07:18:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.710+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69b015a7f0 con 0x7f6994066f30 2024-01-11T07:18:54.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.713+0000 7f69a27fc700 1 -- 172.21.15.7:0/4092434492 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50284 (secure 0 0 0) 0x7f69b015a7f0 con 0x7f6994066f30 2024-01-11T07:18:54.713 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6994066f30 msgr2=0x7f69940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6994066f30 0x7f69940693e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f69b00c10a0 tx=0x7f69ac009040 comp rx=0 tx=0).stop 2024-01-11T07:18:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b0063230 msgr2=0x7f69b00c30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b0063230 0x7f69b00c30e0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f69b006d820 tx=0x7f69a4017000 comp rx=0 tx=0).stop 2024-01-11T07:18:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 shutdown_connections 2024-01-11T07:18:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6994066f30 0x7f69940693e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b015a280 0x7f69b00b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b0063230 0x7f69b00c30e0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 --2- 172.21.15.7:0/4092434492 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f69b00ac5b0 0x7f69b00b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 >> 172.21.15.7:0/4092434492 conn(0x7f69b00680e0 msgr2=0x7f69b015c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 shutdown_connections 2024-01-11T07:18:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.715+0000 7f69b8370700 1 -- 172.21.15.7:0/4092434492 wait complete. 2024-01-11T07:18:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:18:54.728 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2024-01-11T07:18:54.012376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9536,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552963,"statfs":{"total":1600000000,"available":1590234912,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52484901,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-117,"num_write_kb":-85075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001417"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":316,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-01-11T07:18:50.457881+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:18:50.457881+0000","last_peered":"2024-01-11T07:18:50.457881+0000","last_clean":"2024-01-11T07:18:50.457881+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:18:50.457881+0000","last_undegraded":"2024-01-11T07:18:50.457881+0000","last_fullsized":"2024-01-11T07:18:50.457881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193688,"statfs":{"total":200000000,"available":198336588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.433,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":4.9210000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.56599999999999995},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.80500000000000005},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.81399999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75800000000000001,"5min":0.78700000000000003,"15min":0.82399999999999995},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.82999999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42799999999999999,"15min":0.40600000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.83799999999999997},"last":4.9589999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45600000000000002,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.77700000000000002,"15min":0.85999999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.33800000000000002,"15min":0.308},"max":{"1min":0.59599999999999997,"5min":0.755,"15min":0.85899999999999999},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.66300000000000003,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.66600000000000004,"5min":0.84599999999999997,"15min":0.91600000000000004},"last":0.628}]},{"osd":6,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.82199999999999995,"15min":0.92500000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.90300000000000002},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.81399999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.44600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77100000000000002,"5min":0.85199999999999998,"15min":1.157},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.45800000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65000000000000002,"5min":0.79500000000000004,"15min":2.5720000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":0.67300000000000004,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.84599999999999997,"15min":2.621},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88500000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48099999999999998,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.25800000000000001},"max":{"1min":0.66300000000000003,"5min":0.76500000000000001,"15min":0.82699999999999996},"last":0.5},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.253},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":2.0209999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.313},"max":{"1min":0.69299999999999995,"5min":0.77900000000000003,"15min":0.89100000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28699999999999998},"max":{"1min":0.64100000000000001,"5min":0.68600000000000005,"15min":0.76700000000000002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53200000000000003,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":0.70899999999999996,"15min":2.0750000000000002},"last":0.54500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.247,"15min":0.216},"max":{"1min":0.53100000000000003,"5min":0.753,"15min":20.41},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.46100000000000002},"min":{"1min":0.371,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":0.60699999999999998,"5min":0.72099999999999997,"15min":3.7029999999999998},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44,"15min":0.497},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60199999999999998,"5min":3.597,"15min":15.611000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":3.0710000000000002,"15min":20.199999999999999},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":0.67900000000000005,"5min":3.8460000000000001,"15min":20.369},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.28299999999999997},"max":{"1min":0.499,"5min":4.7569999999999997,"15min":8.5790000000000006},"last":0.317}]},{"osd":4,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":1.3300000000000001,"15min":9.9570000000000007},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":1.2030000000000001,"15min":12.449999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":1.3540000000000001,"15min":2.6030000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.51300000000000001,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.90900000000000003,"5min":1.177,"15min":9.891},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":1.389,"15min":1.5569999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69199999999999995,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.94699999999999995,"5min":2.9049999999999998,"15min":9.9900000000000002},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.1599999999999999,"15min":20.324000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.46800000000000003},"max":{"1min":0.96999999999999997,"5min":1.268,"15min":2.548},"last":0.63}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193688,"statfs":{"total":200000000,"available":198336588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39600000000000002,"15min":0.42999999999999999},"min":{"1min":0.27100000000000002,"5min":0.23300000000000001,"15min":0.224},"max":{"1min":0.57899999999999996,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.29299999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57399999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.48799999999999999},"min":{"1min":0.36499999999999999,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.61699999999999999,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.57799999999999996},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58899999999999997,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64000000000000001,"5min":0.83499999999999996,"15min":2.855},"last":0.752}]},{"osd":4,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.495,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90900000000000003,"15min":1.3959999999999999},"last":1.004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.40999999999999998},"max":{"1min":0.92500000000000004,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":1.4710000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":1.032,"15min":2.819},"last":0.872},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.96599999999999997,"15min":1.2490000000000001},"last":1.1140000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95899999999999996,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":1.649},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.501,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":2.3620000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.68000000000000005},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.276},"last":1.206},{"interface":"front","average":{"1min":0.748,"5min":0.747,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.47499999999999998,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":1.595}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.504,"5min":0.72999999999999998,"15min":1.472},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.311,"5min":0.27900000000000003,"15min":0.23599999999999999},"max":{"1min":0.57799999999999996,"5min":0.65500000000000003,"15min":0.86099999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.58899999999999997,"5min":0.628,"15min":1.2430000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48799999999999999},"min":{"1min":0.41399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63300000000000001,"5min":0.752,"15min":0.76800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.63100000000000001,"5min":0.76500000000000001,"15min":1.139},"last":0.499}]},{"osd":4,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.0660000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.089},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48199999999999998,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":1.054,"15min":1.099},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":1.0389999999999999,"15min":1.3100000000000001},"last":0.625}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0109999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.052},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0580000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.55400000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.57399999999999995},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":0.58199999999999996,"5min":0.59699999999999998,"15min":11.481},"last":0.376},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.47299999999999998},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":1.6200000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40699999999999997,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.23899999999999999},"max":{"1min":0.59899999999999998,"5min":0.70499999999999996,"15min":17.516999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67400000000000004},"last":0.435}]},{"osd":2,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":0.56000000000000005,"5min":0.746,"15min":0.76500000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46400000000000002,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":17.553000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.94599999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.95099999999999996},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.36199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":17.474},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64600000000000002,"15min":0.89600000000000002},"min":{"1min":0.49199999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":17.393000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.81899999999999995},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.94499999999999995,"15min":11.394},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.82899999999999996},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.91900000000000004,"15min":11.436999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.878,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.96999999999999997,"15min":1.0449999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.93799999999999994},"min":{"1min":0.53700000000000003,"5min":0.45800000000000002,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":0.95499999999999996,"15min":17.434000000000001},"last":0.64900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.81299999999999994},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.73599999999999999,"5min":0.94799999999999995,"15min":8.484},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.84599999999999997},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":0.84699999999999998,"15min":9.2599999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":1.1599999999999999,"15min":8.5340000000000007},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.78100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.81000000000000005,"15min":9.2279999999999998},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59999999999999998,"15min":0.78200000000000003},"min":{"1min":0.376,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.71999999999999997,"5min":1.1120000000000001,"15min":9.5640000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.88600000000000001,"15min":6.8639999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.78500000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":8.9779999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.84699999999999998},"min":{"1min":0.50800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":1.218,"15min":8.7309999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.67700000000000005},"min":{"1min":0.318,"5min":0.313,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":11.388999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46899999999999997,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":8.6509999999999998},"last":0.41499999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.69399999999999995},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":9.1449999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.81899999999999995,"15min":1.119},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52200000000000002,"15min":0.69499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33500000000000002},"max":{"1min":0.70599999999999996,"5min":0.85399999999999998,"15min":8.6050000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":9.4640000000000004},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.81899999999999995},"last":0.495}]},{"osd":1,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.57699999999999996},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.24399999999999999},"max":{"1min":0.73999999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.93999999999999995},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34999999999999998},"max":{"1min":0.70399999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.74199999999999999,"15min":0.76000000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.496,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.73799999999999999,"15min":0.78600000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.56100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.39300000000000002}]},{"osd":7,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.0109999999999999},"last":0.436},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.88200000000000001},"last":0.55400000000000005}]}]}],"pool_statfs":[]}} 2024-01-11T07:18:54.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-11T07:18:54.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.861+0000 7ff7b9e62700 1 Processor -- start 2024-01-11T07:18:54.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.861+0000 7ff7b9e62700 1 -- start start 2024-01-11T07:18:54.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b9e62700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff7b4159ba0 0x7ff7b406d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b9e62700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159f70 0x7ff7b40c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b9e62700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b4067c60 con 0x7ff7b40c9570 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b9e62700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b4069230 con 0x7ff7b4159ba0 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b9e62700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4069510 con 0x7ff7b4159f70 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff7b4159ba0 0x7ff7b406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff7b40c9570 0x7ff7b40680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33964/0 (socket says 172.21.15.7:33964) 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b3fff700 1 -- 172.21.15.7:0/1448819803 learned_addr learned my addr 172.21.15.7:0/1448819803 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b2ffd700 1 --2- 172.21.15.7:0/1448819803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159f70 0x7ff7b40c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b27fc700 1 -- 172.21.15.7:0/1448819803 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144944475 0 0) 0x7ff7b4067c60 con 0x7ff7b40c9570 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b27fc700 1 -- 172.21.15.7:0/1448819803 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff79c003570 con 0x7ff7b40c9570 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b37fe700 1 -- 172.21.15.7:0/1448819803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159f70 msgr2=0x7ff7b40c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b37fe700 1 --2- 172.21.15.7:0/1448819803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159f70 0x7ff7b40c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b37fe700 1 -- 172.21.15.7:0/1448819803 >> v1:172.21.15.7:6789/0 conn(0x7ff7b40c9570 legacy=0x7ff7b40680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.862+0000 7ff7b37fe700 1 -- 172.21.15.7:0/1448819803 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4067610 con 0x7ff7b4159ba0 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b37fe700 1 --2- 172.21.15.7:0/1448819803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159ba0 0x7ff7b406d020 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098e0 tx=0x7ff7a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198c591534281c93 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b27fc700 1 -- 172.21.15.7:0/1448819803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4011630 con 0x7ff7b4159ba0 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b27fc700 1 -- 172.21.15.7:0/1448819803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a4011790 con 0x7ff7b4159ba0 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b27fc700 1 -- 172.21.15.7:0/1448819803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4011b30 con 0x7ff7b4159ba0 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b9e62700 1 -- 172.21.15.7:0/1448819803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159ba0 msgr2=0x7ff7b406d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/1448819803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159ba0 0x7ff7b406d020 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098e0 tx=0x7ff7a40091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b9e62700 1 -- 172.21.15.7:0/1448819803 shutdown_connections 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/1448819803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159ba0 0x7ff7b406d020 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/1448819803 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159f70 0x7ff7b40c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b9e62700 1 -- 172.21.15.7:0/1448819803 >> 172.21.15.7:0/1448819803 conn(0x7ff7b406ae00 msgr2=0x7ff7b406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b9e62700 1 -- 172.21.15.7:0/1448819803 shutdown_connections 2024-01-11T07:18:54.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.863+0000 7ff7b9e62700 1 -- 172.21.15.7:0/1448819803 wait complete. 2024-01-11T07:18:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.865+0000 7ff7b9e62700 1 Processor -- start 2024-01-11T07:18:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.865+0000 7ff7b9e62700 1 -- start start 2024-01-11T07:18:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.866+0000 7ff7b9e62700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b40c9570 0x7ff7b40c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.866+0000 7ff7b9e62700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159ba0 0x7ff7b415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.866+0000 7ff7b9e62700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159f70 0x7ff7b415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.866+0000 7ff7b9e62700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40c5fe0 con 0x7ff7b4159f70 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.866+0000 7ff7b9e62700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4152e20 con 0x7ff7b40c9570 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.866+0000 7ff7b9e62700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40c6ab0 con 0x7ff7b4159ba0 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.866+0000 7ff7b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159f70 0x7ff7b415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.866+0000 7ff7b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159f70 0x7ff7b415da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36838/0 (socket says 172.21.15.7:36838) 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b3fff700 1 -- 172.21.15.7:0/3488975594 learned_addr learned my addr 172.21.15.7:0/3488975594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b37fe700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b40c9570 0x7ff7b40c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b2ffd700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159ba0 0x7ff7b415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b3fff700 1 -- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159ba0 msgr2=0x7ff7b415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b3fff700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159ba0 0x7ff7b415cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b3fff700 1 -- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b40c9570 msgr2=0x7ff7b40c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:54.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b3fff700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b40c9570 0x7ff7b40c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b3fff700 1 -- 172.21.15.7:0/3488975594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4009420 con 0x7ff7b4159f70 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b37fe700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b40c9570 0x7ff7b40c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b2ffd700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159ba0 0x7ff7b415cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b3fff700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159f70 0x7ff7b415da00 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac001460 tx=0x7ff7ac001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff79b7fe700 1 -- 172.21.15.7:0/3488975594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00a950 con 0x7ff7b4159f70 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff79b7fe700 1 -- 172.21.15.7:0/3488975594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac00aab0 con 0x7ff7b4159f70 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b415ac00 con 0x7ff7b4159f70 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff79b7fe700 1 -- 172.21.15.7:0/3488975594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00ae30 con 0x7ff7b4159f70 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.867+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b40c62c0 con 0x7ff7b4159f70 2024-01-11T07:18:54.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.868+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b415ad90 con 0x7ff7b4159f70 2024-01-11T07:18:54.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.869+0000 7ff79b7fe700 1 -- 172.21.15.7:0/3488975594 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff7ac007300 con 0x7ff7b4159f70 2024-01-11T07:18:54.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.869+0000 7ff79b7fe700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7a0066f30 0x7ff7a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:54.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.869+0000 7ff79b7fe700 1 -- 172.21.15.7:0/3488975594 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff7ac0846d0 con 0x7ff7b4159f70 2024-01-11T07:18:54.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.869+0000 7ff7b37fe700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7a0066f30 0x7ff7a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:54.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.870+0000 7ff7b37fe700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7a0066f30 0x7ff7a00693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40091e0 tx=0x7ff7a4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:54.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:54.874+0000 7ff79b7fe700 1 -- 172.21.15.7:0/3488975594 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7ac051c30 con 0x7ff7b4159f70 2024-01-11T07:18:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.077+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7b404c3c0 con 0x7ff7b4159f70 2024-01-11T07:18:55.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.078+0000 7ff79b7fe700 1 -- 172.21.15.7:0/3488975594 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+12224 (secure 0 0 0) 0x7ff7ac055250 con 0x7ff7b4159f70 2024-01-11T07:18:55.078 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:18:55.078 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":191,"fsid":"9c0c5e15-bc38-4016-89e4-891c3586efbb","created":"2024-01-11T07:06:42.317513+0000","modified":"2024-01-11T07:18:51.452494+0000","last_up_change":"2024-01-11T07:06:47.885787+0000","last_in_change":"2024-01-11T07:06:44.867018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-11T07:06:49.881877+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":36945},{"type":"v1","addr":"172.21.15.7:6817","nonce":36945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":36945},{"type":"v1","addr":"172.21.15.7:6819","nonce":36945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":36945},{"type":"v1","addr":"172.21.15.7:6824","nonce":36945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":36945},{"type":"v1","addr":"172.21.15.7:6821","nonce":36945}]},"public_addr":"172.21.15.7:6817/36945","cluster_addr":"172.21.15.7:6819/36945","heartbeat_back_addr":"172.21.15.7:6824/36945","heartbeat_front_addr":"172.21.15.7:6821/36945","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":36943},{"type":"v1","addr":"172.21.15.7:6809","nonce":36943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":36943},{"type":"v1","addr":"172.21.15.7:6811","nonce":36943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":36943},{"type":"v1","addr":"172.21.15.7:6815","nonce":36943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":36943},{"type":"v1","addr":"172.21.15.7:6813","nonce":36943}]},"public_addr":"172.21.15.7:6809/36943","cluster_addr":"172.21.15.7:6811/36943","heartbeat_back_addr":"172.21.15.7:6815/36943","heartbeat_front_addr":"172.21.15.7:6813/36943","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":36946},{"type":"v1","addr":"172.21.15.7:6825","nonce":36946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":36946},{"type":"v1","addr":"172.21.15.7:6827","nonce":36946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":36946},{"type":"v1","addr":"172.21.15.7:6831","nonce":36946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":36946},{"type":"v1","addr":"172.21.15.7:6829","nonce":36946}]},"public_addr":"172.21.15.7:6825/36946","cluster_addr":"172.21.15.7:6827/36946","heartbeat_back_addr":"172.21.15.7:6831/36946","heartbeat_front_addr":"172.21.15.7:6829/36946","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":36944},{"type":"v1","addr":"172.21.15.7:6801","nonce":36944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":36944},{"type":"v1","addr":"172.21.15.7:6803","nonce":36944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":36944},{"type":"v1","addr":"172.21.15.7:6807","nonce":36944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":36944},{"type":"v1","addr":"172.21.15.7:6805","nonce":36944}]},"public_addr":"172.21.15.7:6801/36944","cluster_addr":"172.21.15.7:6803/36944","heartbeat_back_addr":"172.21.15.7:6807/36944","heartbeat_front_addr":"172.21.15.7:6805/36944","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":37188},{"type":"v1","addr":"172.21.15.113:6801","nonce":37188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":37188},{"type":"v1","addr":"172.21.15.113:6803","nonce":37188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":37188},{"type":"v1","addr":"172.21.15.113:6807","nonce":37188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":37188},{"type":"v1","addr":"172.21.15.113:6805","nonce":37188}]},"public_addr":"172.21.15.113:6801/37188","cluster_addr":"172.21.15.113:6803/37188","heartbeat_back_addr":"172.21.15.113:6807/37188","heartbeat_front_addr":"172.21.15.113:6805/37188","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":37190},{"type":"v1","addr":"172.21.15.113:6817","nonce":37190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":37190},{"type":"v1","addr":"172.21.15.113:6819","nonce":37190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":37190},{"type":"v1","addr":"172.21.15.113:6823","nonce":37190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":37190},{"type":"v1","addr":"172.21.15.113:6821","nonce":37190}]},"public_addr":"172.21.15.113:6817/37190","cluster_addr":"172.21.15.113:6819/37190","heartbeat_back_addr":"172.21.15.113:6823/37190","heartbeat_front_addr":"172.21.15.113:6821/37190","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":37189},{"type":"v1","addr":"172.21.15.113:6809","nonce":37189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":37189},{"type":"v1","addr":"172.21.15.113:6811","nonce":37189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":37189},{"type":"v1","addr":"172.21.15.113:6815","nonce":37189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":37189},{"type":"v1","addr":"172.21.15.113:6813","nonce":37189}]},"public_addr":"172.21.15.113:6809/37189","cluster_addr":"172.21.15.113:6811/37189","heartbeat_back_addr":"172.21.15.113:6815/37189","heartbeat_front_addr":"172.21.15.113:6813/37189","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":37240},{"type":"v1","addr":"172.21.15.113:6825","nonce":37240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":37240},{"type":"v1","addr":"172.21.15.113:6827","nonce":37240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":37240},{"type":"v1","addr":"172.21.15.113:6831","nonce":37240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":37240},{"type":"v1","addr":"172.21.15.113:6829","nonce":37240}]},"public_addr":"172.21.15.113:6825/37240","cluster_addr":"172.21.15.113:6827/37240","heartbeat_back_addr":"172.21.15.113:6831/37240","heartbeat_front_addr":"172.21.15.113:6829/37240","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.054179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.051528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.052224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.038923+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.008477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.032799+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:45.946032+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-01-11T07:06:46.000300+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3920146306":"2024-01-12T07:18:48.442926+0000","172.21.15.7:6834/3920146306":"2024-01-12T07:18:48.442926+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7a0066f30 msgr2=0x7ff7a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7a0066f30 0x7ff7a00693e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40091e0 tx=0x7ff7a4017000 comp rx=0 tx=0).stop 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159f70 msgr2=0x7ff7b415da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159f70 0x7ff7b415da00 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac001460 tx=0x7ff7ac001a40 comp rx=0 tx=0).stop 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 shutdown_connections 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff7a0066f30 0x7ff7a00693e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b4159f70 0x7ff7b415da00 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7b40c9570 0x7ff7b40c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 --2- 172.21.15.7:0/3488975594 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff7b4159ba0 0x7ff7b415cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 >> 172.21.15.7:0/3488975594 conn(0x7ff7b406ae00 msgr2=0x7ff7b40680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:55.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 shutdown_connections 2024-01-11T07:18:55.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:55.081+0000 7ff7b9e62700 1 -- 172.21.15.7:0/3488975594 wait complete. 2024-01-11T07:18:56.095 INFO:tasks.ceph:Scrubbing osd.0 2024-01-11T07:18:56.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-11T07:18:56.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.238+0000 7ff3c4ede700 1 Processor -- start 2024-01-11T07:18:56.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.238+0000 7ff3c4ede700 1 -- start start 2024-01-11T07:18:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.239+0000 7ff3c4ede700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3c00b33b0 0x7ff3c00adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.239+0000 7ff3c4ede700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c00b1d40 con 0x7ff3c0159a80 2024-01-11T07:18:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.239+0000 7ff3c4ede700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c015e370 con 0x7ff3c00cede0 2024-01-11T07:18:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.239+0000 7ff3c4ede700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c00696c0 con 0x7ff3c00b33b0 2024-01-11T07:18:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.239+0000 7ff3bdd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff3c00b33b0 0x7ff3c00adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3be59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff3c0159a80 0x7ff3c00ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33980/0 (socket says 172.21.15.7:33980) 2024-01-11T07:18:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3be59c700 1 -- 172.21.15.7:0/845841143 learned_addr learned my addr 172.21.15.7:0/845841143 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/845841143 >> v1:172.21.15.113:6789/0 conn(0x7ff3c00cede0 legacy=0x7ff3c0067710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/845841143 >> v1:172.21.15.7:6789/0 conn(0x7ff3c0159a80 legacy=0x7ff3c00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/845841143 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c015e650 con 0x7ff3c00b33b0 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3bd59a700 1 -- 172.21.15.7:0/845841143 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692247604 0 0) 0x7ff3c00b1d40 con 0x7ff3c0159a80 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3bdd9b700 1 --2- 172.21.15.7:0/845841143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00b33b0 0x7ff3c00adbe0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80098e0 tx=0x7ff3a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17da30808be420c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3bd59a700 1 -- 172.21.15.7:0/845841143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8009e80 con 0x7ff3c00b33b0 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3bd59a700 1 -- 172.21.15.7:0/845841143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a800b820 con 0x7ff3c00b33b0 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.240+0000 7ff3bd59a700 1 -- 172.21.15.7:0/845841143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800bba0 con 0x7ff3c00b33b0 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.241+0000 7ff3c4ede700 1 -- 172.21.15.7:0/845841143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00b33b0 msgr2=0x7ff3c00adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.241+0000 7ff3c4ede700 1 --2- 172.21.15.7:0/845841143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00b33b0 0x7ff3c00adbe0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80098e0 tx=0x7ff3a80091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.241+0000 7ff3c4ede700 1 -- 172.21.15.7:0/845841143 shutdown_connections 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.241+0000 7ff3c4ede700 1 --2- 172.21.15.7:0/845841143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00b33b0 0x7ff3c00adbe0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.241+0000 7ff3c4ede700 1 -- 172.21.15.7:0/845841143 >> 172.21.15.7:0/845841143 conn(0x7ff3c006b910 msgr2=0x7ff3c006ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:56.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.241+0000 7ff3c4ede700 1 -- 172.21.15.7:0/845841143 shutdown_connections 2024-01-11T07:18:56.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.241+0000 7ff3c4ede700 1 -- 172.21.15.7:0/845841143 wait complete. 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.242+0000 7ff3c4ede700 1 Processor -- start 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.242+0000 7ff3c4ede700 1 -- start start 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3c4ede700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c00b33b0 0x7ff3c016b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3c4ede700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00cede0 0x7ff3c015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3c4ede700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c0159a80 0x7ff3c015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3c4ede700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c014fdd0 con 0x7ff3c00cede0 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3c4ede700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c0159e50 con 0x7ff3c00b33b0 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3c4ede700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c015a410 con 0x7ff3c0159a80 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00cede0 0x7ff3c015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00cede0 0x7ff3c015a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36860/0 (socket says 172.21.15.7:36860) 2024-01-11T07:18:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/3445215670 learned_addr learned my addr 172.21.15.7:0/3445215670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3be59c700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c00b33b0 0x7ff3c016b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bed9d700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c0159a80 0x7ff3c015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c0159a80 msgr2=0x7ff3c015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bdd9b700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff3c0159a80 0x7ff3c015ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c00b33b0 msgr2=0x7ff3c016b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bdd9b700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3c00b33b0 0x7ff3c016b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.243+0000 7ff3bdd9b700 1 -- 172.21.15.7:0/3445215670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a8009420 con 0x7ff3c00cede0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.244+0000 7ff3bdd9b700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00cede0 0x7ff3c015a8a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8002000 tx=0x7ff3a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.244+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a801b470 con 0x7ff3c00cede0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.244+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a801b5d0 con 0x7ff3c00cede0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.244+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c015f5a0 con 0x7ff3c00cede0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.244+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a801b990 con 0x7ff3c00cede0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.244+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c015a0a0 con 0x7ff3c00cede0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.245+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3c004c3c0 con 0x7ff3c00cede0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.245+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7ff3a801bd00 con 0x7ff3c00cede0 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.246+0000 7ff3b67fc700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3a0066f30 0x7ff3a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.246+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff3a8086560 con 0x7ff3c00cede0 2024-01-11T07:18:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.246+0000 7ff3b67fc700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7ff3a006cd80 0x7ff3a006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.246+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 --> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3a006f860 con 0x7ff3a006cd80 2024-01-11T07:18:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.246+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3a8086930 con 0x7ff3c00cede0 2024-01-11T07:18:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.246+0000 7ff3bed9d700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7ff3a006cd80 0x7ff3a006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.246+0000 7ff3be59c700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3a0066f30 0x7ff3a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.246+0000 7ff3bed9d700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7ff3a006cd80 0x7ff3a006f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.247+0000 7ff3be59c700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3a0066f30 0x7ff3a00693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff3c01551d0 tx=0x7ff3b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.247+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 <== osd.0 v2:172.21.15.7:6816/36945 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7ff3a006f860 con 0x7ff3a006cd80 2024-01-11T07:18:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 --> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff3c00b0c60 con 0x7ff3a006cd80 2024-01-11T07:18:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3b67fc700 1 -- 172.21.15.7:0/3445215670 <== osd.0 v2:172.21.15.7:6816/36945 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7ff3c00b0c60 con 0x7ff3a006cd80 2024-01-11T07:18:56.261 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:56.261 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-11T07:18:56.261 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7ff3a006cd80 msgr2=0x7ff3a006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.7:6816/36945,v1:172.21.15.7:6817/36945] conn(0x7ff3a006cd80 0x7ff3a006f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3a0066f30 msgr2=0x7ff3a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7ff3a0066f30 0x7ff3a00693e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff3c01551d0 tx=0x7ff3b0009040 comp rx=0 tx=0).stop 2024-01-11T07:18:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00cede0 msgr2=0x7ff3c015a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 --2- 172.21.15.7:0/3445215670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3c00cede0 0x7ff3c015a8a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8002000 tx=0x7ff3a8016000 comp rx=0 tx=0).stop 2024-01-11T07:18:56.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3bed9d700 1 -- 172.21.15.7:0/3445215670 reap_dead start 2024-01-11T07:18:56.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 shutdown_connections 2024-01-11T07:18:56.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 >> 172.21.15.7:0/3445215670 conn(0x7ff3c006b910 msgr2=0x7ff3c0067710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:56.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 shutdown_connections 2024-01-11T07:18:56.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.261+0000 7ff3c4ede700 1 -- 172.21.15.7:0/3445215670 wait complete. 2024-01-11T07:18:56.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-11T07:18:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.407+0000 7fd681532700 1 Processor -- start 2024-01-11T07:18:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.407+0000 7fd681532700 1 -- start start 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd681532700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd67c0cadf0 0x7fd67c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd681532700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67c0699a0 con 0x7fd67c0c6e10 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd681532700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67c06a540 con 0x7fd67c1560d0 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd681532700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67c0cc2f0 con 0x7fd67c0cadf0 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd67a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd67c0cadf0 0x7fd67c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd67b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd67c0c6e10 0x7fd67c06cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33990/0 (socket says 172.21.15.7:33990) 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd67b7fe700 1 -- 172.21.15.7:0/102475857 learned_addr learned my addr 172.21.15.7:0/102475857 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd67a7fc700 1 -- 172.21.15.7:0/102475857 >> v1:172.21.15.113:6789/0 conn(0x7fd67c1560d0 legacy=0x7fd67c069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd67a7fc700 1 -- 172.21.15.7:0/102475857 >> v1:172.21.15.7:6789/0 conn(0x7fd67c0c6e10 legacy=0x7fd67c06cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd679ffb700 1 -- 172.21.15.7:0/102475857 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956433629 0 0) 0x7fd67c0699a0 con 0x7fd67c0c6e10 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd67a7fc700 1 -- 172.21.15.7:0/102475857 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c068d90 con 0x7fd67c0cadf0 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd67a7fc700 1 --2- 172.21.15.7:0/102475857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c0cadf0 0x7fd67c0cb290 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd6700098e0 tx=0x7fd6700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce5062f8c6084b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.408+0000 7fd679ffb700 1 -- 172.21.15.7:0/102475857 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670009e80 con 0x7fd67c0cadf0 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd679ffb700 1 -- 172.21.15.7:0/102475857 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67000b860 con 0x7fd67c0cadf0 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd679ffb700 1 -- 172.21.15.7:0/102475857 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000bbe0 con 0x7fd67c0cadf0 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd681532700 1 -- 172.21.15.7:0/102475857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c0cadf0 msgr2=0x7fd67c0cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd681532700 1 --2- 172.21.15.7:0/102475857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c0cadf0 0x7fd67c0cb290 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd6700098e0 tx=0x7fd6700091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd681532700 1 -- 172.21.15.7:0/102475857 shutdown_connections 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd681532700 1 --2- 172.21.15.7:0/102475857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c0cadf0 0x7fd67c0cb290 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd681532700 1 -- 172.21.15.7:0/102475857 >> 172.21.15.7:0/102475857 conn(0x7fd67c063230 msgr2=0x7fd67c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd681532700 1 -- 172.21.15.7:0/102475857 shutdown_connections 2024-01-11T07:18:56.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.409+0000 7fd681532700 1 -- 172.21.15.7:0/102475857 wait complete. 2024-01-11T07:18:56.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.410+0000 7fd681532700 1 Processor -- start 2024-01-11T07:18:56.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.410+0000 7fd681532700 1 -- start start 2024-01-11T07:18:56.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd681532700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd67c0c6e10 0x7fd67c152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd681532700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd67c0cadf0 0x7fd67c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd681532700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c1560d0 0x7fd67c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd681532700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c16af10 con 0x7fd67c1560d0 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd681532700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c169f20 con 0x7fd67c0cadf0 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd681532700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c16a190 con 0x7fd67c0c6e10 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd67b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c1560d0 0x7fd67c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd67b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c1560d0 0x7fd67c06d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36866/0 (socket says 172.21.15.7:36866) 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.411+0000 7fd67b7fe700 1 -- 172.21.15.7:0/836753980 learned_addr learned my addr 172.21.15.7:0/836753980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd67a7fc700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd67c0cadf0 0x7fd67c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd67affd700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd67c0c6e10 0x7fd67c152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd67b7fe700 1 -- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd67c0c6e10 msgr2=0x7fd67c152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd67b7fe700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd67c0c6e10 0x7fd67c152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd67b7fe700 1 -- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd67c0cadf0 msgr2=0x7fd67c06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd67b7fe700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd67c0cadf0 0x7fd67c06d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd67b7fe700 1 -- 172.21.15.7:0/836753980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670009420 con 0x7fd67c1560d0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd67b7fe700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c1560d0 0x7fd67c06d9f0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd67400cc90 tx=0x7fd674001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd662ffd700 1 -- 172.21.15.7:0/836753980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67400cea0 con 0x7fd67c1560d0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd662ffd700 1 -- 172.21.15.7:0/836753980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67400ab30 con 0x7fd67c1560d0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67c0c4d00 con 0x7fd67c1560d0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd662ffd700 1 -- 172.21.15.7:0/836753980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6740070f0 con 0x7fd67c1560d0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.412+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c16a710 con 0x7fd67c1560d0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.413+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67c04c3c0 con 0x7fd67c1560d0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.414+0000 7fd662ffd700 1 -- 172.21.15.7:0/836753980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd6740074a0 con 0x7fd67c1560d0 2024-01-11T07:18:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.414+0000 7fd662ffd700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd668066f30 0x7fd6680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.414+0000 7fd662ffd700 1 -- 172.21.15.7:0/836753980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd674084650 con 0x7fd67c1560d0 2024-01-11T07:18:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.414+0000 7fd67affd700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd668066f30 0x7fd6680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.415+0000 7fd67affd700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd668066f30 0x7fd6680693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0037c0 tx=0x7fd66c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.418+0000 7fd662ffd700 1 -- 172.21.15.7:0/836753980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd674051c40 con 0x7fd67c1560d0 2024-01-11T07:18:56.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.626+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd67c069070 con 0x7fd668066f30 2024-01-11T07:18:56.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.627+0000 7fd662ffd700 1 -- 172.21.15.7:0/836753980 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd67c069070 con 0x7fd668066f30 2024-01-11T07:18:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd668066f30 msgr2=0x7fd6680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd668066f30 0x7fd6680693e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0037c0 tx=0x7fd66c009040 comp rx=0 tx=0).stop 2024-01-11T07:18:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c1560d0 msgr2=0x7fd67c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c1560d0 0x7fd67c06d9f0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd67400cc90 tx=0x7fd674001a20 comp rx=0 tx=0).stop 2024-01-11T07:18:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 shutdown_connections 2024-01-11T07:18:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd668066f30 0x7fd6680693e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd67c1560d0 0x7fd67c06d9f0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd67c0c6e10 0x7fd67c152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 --2- 172.21.15.7:0/836753980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd67c0cadf0 0x7fd67c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 >> 172.21.15.7:0/836753980 conn(0x7fd67c063230 msgr2=0x7fd67c155030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 shutdown_connections 2024-01-11T07:18:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.629+0000 7fd681532700 1 -- 172.21.15.7:0/836753980 wait complete. 2024-01-11T07:18:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-01-11T07:18:56.643 INFO:tasks.ceph:Scrubbing osd.1 2024-01-11T07:18:56.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-11T07:18:56.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.780+0000 7fe803dc4700 1 Processor -- start 2024-01-11T07:18:56.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.780+0000 7fe803dc4700 1 -- start start 2024-01-11T07:18:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe803dc4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc157990 0x7fe7fc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe803dc4700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7fc157d60 0x7fe7fc0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe803dc4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7fc15c3b0 0x7fe7fc0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe803dc4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc0cd260 con 0x7fe7fc15c3b0 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe803dc4700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc0caf40 con 0x7fe7fc157d60 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe803dc4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0cc690 con 0x7fe7fc157990 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe802361700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7fc15c3b0 0x7fe7fc0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe802361700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7fc15c3b0 0x7fe7fc0caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36874/0 (socket says 172.21.15.7:36874) 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe802361700 1 -- 172.21.15.7:0/3106919930 learned_addr learned my addr 172.21.15.7:0/3106919930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe801b60700 1 --2- 172.21.15.7:0/3106919930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc157990 0x7fe7fc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe80135f700 1 --2- 172.21.15.7:0/3106919930 >> v2:172.21.15.113:3300/0 conn(0x7fe7fc157d60 0x7fe7fc0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe802361700 1 -- 172.21.15.7:0/3106919930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc157990 msgr2=0x7fe7fc06d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe802361700 1 --2- 172.21.15.7:0/3106919930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc157990 0x7fe7fc06d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe802361700 1 -- 172.21.15.7:0/3106919930 >> v2:172.21.15.113:3300/0 conn(0x7fe7fc157d60 msgr2=0x7fe7fc0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe802361700 1 --2- 172.21.15.7:0/3106919930 >> v2:172.21.15.113:3300/0 conn(0x7fe7fc157d60 0x7fe7fc0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe802361700 1 -- 172.21.15.7:0/3106919930 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc0cc930 con 0x7fe7fc15c3b0 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe801b60700 1 --2- 172.21.15.7:0/3106919930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc157990 0x7fe7fc06d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:56.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.781+0000 7fe80135f700 1 --2- 172.21.15.7:0/3106919930 >> v2:172.21.15.113:3300/0 conn(0x7fe7fc157d60 0x7fe7fc0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe802361700 1 --2- 172.21.15.7:0/3106919930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc15c3b0 0x7fe7fc0caa00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0cc500 tx=0x7fe7f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f91160f7e705a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe800b5e700 1 -- 172.21.15.7:0/3106919930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400e070 con 0x7fe7fc15c3b0 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe800b5e700 1 -- 172.21.15.7:0/3106919930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f400ba50 con 0x7fe7fc15c3b0 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe800b5e700 1 -- 172.21.15.7:0/3106919930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40092e0 con 0x7fe7fc15c3b0 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe803dc4700 1 -- 172.21.15.7:0/3106919930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc15c3b0 msgr2=0x7fe7fc0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe803dc4700 1 --2- 172.21.15.7:0/3106919930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc15c3b0 0x7fe7fc0caa00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0cc500 tx=0x7fe7f4009840 comp rx=0 tx=0).stop 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe803dc4700 1 -- 172.21.15.7:0/3106919930 shutdown_connections 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe803dc4700 1 --2- 172.21.15.7:0/3106919930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc15c3b0 0x7fe7fc0caa00 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe803dc4700 1 --2- 172.21.15.7:0/3106919930 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc157990 0x7fe7fc06d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe803dc4700 1 --2- 172.21.15.7:0/3106919930 >> v2:172.21.15.113:3300/0 conn(0x7fe7fc157d60 0x7fe7fc0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.782+0000 7fe803dc4700 1 -- 172.21.15.7:0/3106919930 >> 172.21.15.7:0/3106919930 conn(0x7fe7fc063230 msgr2=0x7fe7fc063630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.783+0000 7fe803dc4700 1 -- 172.21.15.7:0/3106919930 shutdown_connections 2024-01-11T07:18:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.783+0000 7fe803dc4700 1 -- 172.21.15.7:0/3106919930 wait complete. 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.784+0000 7fe803dc4700 1 Processor -- start 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.784+0000 7fe803dc4700 1 -- start start 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe803dc4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7fc157990 0x7fe7fc0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe803dc4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc157d60 0x7fe7fc0c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe803dc4700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc15c3b0 0x7fe7fc1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe803dc4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b2b40 con 0x7fe7fc157d60 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe803dc4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0c8310 con 0x7fe7fc157990 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe803dc4700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b1f30 con 0x7fe7fc15c3b0 2024-01-11T07:18:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe80135f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc157d60 0x7fe7fc0c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe80135f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc157d60 0x7fe7fc0c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36878/0 (socket says 172.21.15.7:36878) 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.785+0000 7fe80135f700 1 -- 172.21.15.7:0/923764879 learned_addr learned my addr 172.21.15.7:0/923764879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe801b60700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7fc157990 0x7fe7fc0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe802361700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc15c3b0 0x7fe7fc1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe80135f700 1 -- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc15c3b0 msgr2=0x7fe7fc1568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe80135f700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7fc15c3b0 0x7fe7fc1568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe80135f700 1 -- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7fc157990 msgr2=0x7fe7fc0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe80135f700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7fc157990 0x7fe7fc0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe80135f700 1 -- 172.21.15.7:0/923764879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4009ae0 con 0x7fe7fc157d60 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe80135f700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc157d60 0x7fe7fc0c7dd0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c880 tx=0x7fe7ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec011600 con 0x7fe7fc157d60 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec011760 con 0x7fe7fc157d60 2024-01-11T07:18:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc0b2210 con 0x7fe7fc157d60 2024-01-11T07:18:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.786+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec011ae0 con 0x7fe7fc157d60 2024-01-11T07:18:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.787+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc0b27d0 con 0x7fe7fc157d60 2024-01-11T07:18:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.787+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7fc04c3c0 con 0x7fe7fc157d60 2024-01-11T07:18:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.788+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fe7ec0193b0 con 0x7fe7fc157d60 2024-01-11T07:18:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.788+0000 7fe7f1ffb700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7e0066f30 0x7fe7e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.788+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe7ec0841d0 con 0x7fe7fc157d60 2024-01-11T07:18:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.788+0000 7fe7f1ffb700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fe7e006cd80 0x7fe7e006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.788+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 --> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7e006f860 con 0x7fe7e006cd80 2024-01-11T07:18:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.789+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7ec086810 con 0x7fe7fc157d60 2024-01-11T07:18:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.789+0000 7fe801b60700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7e0066f30 0x7fe7e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.789+0000 7fe802361700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fe7e006cd80 0x7fe7e006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.789+0000 7fe802361700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fe7e006cd80 0x7fe7e006f190 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.789+0000 7fe801b60700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7e0066f30 0x7fe7e00693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800acc0 tx=0x7fe7f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.790+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 <== osd.1 v2:172.21.15.7:6808/36943 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fe7e006f860 con 0x7fe7e006cd80 2024-01-11T07:18:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.803+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 --> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe7fc0ccc10 con 0x7fe7e006cd80 2024-01-11T07:18:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe7f1ffb700 1 -- 172.21.15.7:0/923764879 <== osd.1 v2:172.21.15.7:6808/36943 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fe7fc0ccc10 con 0x7fe7e006cd80 2024-01-11T07:18:56.803 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:56.804 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-11T07:18:56.804 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fe7e006cd80 msgr2=0x7fe7e006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.7:6808/36943,v1:172.21.15.7:6809/36943] conn(0x7fe7e006cd80 0x7fe7e006f190 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7e0066f30 msgr2=0x7fe7e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fe7e0066f30 0x7fe7e00693e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800acc0 tx=0x7fe7f8008040 comp rx=0 tx=0).stop 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc157d60 msgr2=0x7fe7fc0c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 --2- 172.21.15.7:0/923764879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7fc157d60 0x7fe7fc0c7dd0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c880 tx=0x7fe7ec008040 comp rx=0 tx=0).stop 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe802361700 1 -- 172.21.15.7:0/923764879 reap_dead start 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 shutdown_connections 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 >> 172.21.15.7:0/923764879 conn(0x7fe7fc063230 msgr2=0x7fe7fc063600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 shutdown_connections 2024-01-11T07:18:56.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.804+0000 7fe803dc4700 1 -- 172.21.15.7:0/923764879 wait complete. 2024-01-11T07:18:56.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-11T07:18:56.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.950+0000 7f7ec8001700 1 Processor -- start 2024-01-11T07:18:56.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.950+0000 7f7ec8001700 1 -- start start 2024-01-11T07:18:56.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.951+0000 7f7ec8001700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec00cafb0 0x7f7ec00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.951+0000 7f7ec8001700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec0159300 con 0x7f7ec0158280 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.951+0000 7f7ec8001700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00cf260 con 0x7f7ec00b0b40 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.951+0000 7f7ec8001700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00cf540 con 0x7f7ec00cafb0 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.951+0000 7f7ec559c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ec0158280 0x7f7ec00b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33994/0 (socket says 172.21.15.7:33994) 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.951+0000 7f7ec559c700 1 -- 172.21.15.7:0/294678705 learned_addr learned my addr 172.21.15.7:0/294678705 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.951+0000 7f7ec5d9d700 1 --2- 172.21.15.7:0/294678705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec00cafb0 0x7f7ec00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec4d9b700 1 -- 172.21.15.7:0/294678705 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040477747 0 0) 0x7f7ec0159300 con 0x7f7ec0158280 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec4d9b700 1 -- 172.21.15.7:0/294678705 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eac003570 con 0x7f7ec0158280 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec4d9b700 1 -- 172.21.15.7:0/294678705 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28879119 0 0) 0x7f7ec00cf260 con 0x7f7ec00b0b40 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec4d9b700 1 -- 172.21.15.7:0/294678705 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec0159300 con 0x7f7ec00b0b40 2024-01-11T07:18:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec5d9d700 1 -- 172.21.15.7:0/294678705 >> v1:172.21.15.113:6789/0 conn(0x7f7ec00b0b40 legacy=0x7f7ec0161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec5d9d700 1 -- 172.21.15.7:0/294678705 >> v1:172.21.15.7:6789/0 conn(0x7f7ec0158280 legacy=0x7f7ec00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec5d9d700 1 -- 172.21.15.7:0/294678705 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00cf820 con 0x7f7ec00cafb0 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec5d9d700 1 --2- 172.21.15.7:0/294678705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec00cafb0 0x7f7ec00ce8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009910 tx=0x7f7ebc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a00661cb896e0c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec4d9b700 1 -- 172.21.15.7:0/294678705 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc011630 con 0x7f7ec00cafb0 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec4d9b700 1 -- 172.21.15.7:0/294678705 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc011790 con 0x7f7ec00cafb0 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.952+0000 7f7ec4d9b700 1 -- 172.21.15.7:0/294678705 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc011b10 con 0x7f7ec00cafb0 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.953+0000 7f7ec8001700 1 -- 172.21.15.7:0/294678705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec00cafb0 msgr2=0x7f7ec00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.953+0000 7f7ec8001700 1 --2- 172.21.15.7:0/294678705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec00cafb0 0x7f7ec00ce8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009910 tx=0x7f7ebc0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.953+0000 7f7ec8001700 1 -- 172.21.15.7:0/294678705 shutdown_connections 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.953+0000 7f7ec8001700 1 --2- 172.21.15.7:0/294678705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec00cafb0 0x7f7ec00ce8f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.953+0000 7f7ec8001700 1 -- 172.21.15.7:0/294678705 >> 172.21.15.7:0/294678705 conn(0x7f7ec0071100 msgr2=0x7f7ec006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.953+0000 7f7ec8001700 1 -- 172.21.15.7:0/294678705 shutdown_connections 2024-01-11T07:18:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.953+0000 7f7ec8001700 1 -- 172.21.15.7:0/294678705 wait complete. 2024-01-11T07:18:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.954+0000 7f7ec8001700 1 Processor -- start 2024-01-11T07:18:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.954+0000 7f7ec8001700 1 -- start start 2024-01-11T07:18:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec8001700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ec00b0b40 0x7f7ec00c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec8001700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ec00cafb0 0x7f7ec0168170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec8001700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec0158280 0x7f7ec00cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec8001700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec016ba50 con 0x7f7ec00cafb0 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec8001700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec016b770 con 0x7f7ec00b0b40 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec8001700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec01689c0 con 0x7f7ec0158280 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ec00cafb0 0x7f7ec0168170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ec00cafb0 0x7f7ec0168170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36880/0 (socket says 172.21.15.7:36880) 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec559c700 1 -- 172.21.15.7:0/2699527903 learned_addr learned my addr 172.21.15.7:0/2699527903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec659e700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec0158280 0x7f7ec00cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec5d9d700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ec00b0b40 0x7f7ec00c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec559c700 1 -- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec0158280 msgr2=0x7f7ec00cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec559c700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec0158280 0x7f7ec00cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec559c700 1 -- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ec00b0b40 msgr2=0x7f7ec00c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec559c700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ec00b0b40 0x7f7ec00c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.955+0000 7f7ec559c700 1 -- 172.21.15.7:0/2699527903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc009420 con 0x7f7ec00cafb0 2024-01-11T07:18:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.956+0000 7f7ec559c700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ec00cafb0 0x7f7ec0168170 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000c580 tx=0x7f7eb0007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.956+0000 7f7eb5ffb700 1 -- 172.21.15.7:0/2699527903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0005b30 con 0x7f7ec00cafb0 2024-01-11T07:18:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.956+0000 7f7eb5ffb700 1 -- 172.21.15.7:0/2699527903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb0005c90 con 0x7f7ec00cafb0 2024-01-11T07:18:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.956+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec0168ca0 con 0x7f7ec00cafb0 2024-01-11T07:18:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.956+0000 7f7eb5ffb700 1 -- 172.21.15.7:0/2699527903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0007320 con 0x7f7ec00cafb0 2024-01-11T07:18:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.956+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec01686b0 con 0x7f7ec00cafb0 2024-01-11T07:18:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.957+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec004c3c0 con 0x7f7ec00cafb0 2024-01-11T07:18:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.957+0000 7f7eb5ffb700 1 -- 172.21.15.7:0/2699527903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f7eb001b3b0 con 0x7f7ec00cafb0 2024-01-11T07:18:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.958+0000 7f7eb5ffb700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.958+0000 7f7eb5ffb700 1 -- 172.21.15.7:0/2699527903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7eb0083690 con 0x7f7ec00cafb0 2024-01-11T07:18:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.958+0000 7f7ec5d9d700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.959+0000 7f7ec5d9d700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ea4066f30 0x7f7ea40693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009860 tx=0x7f7ebc017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:56.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:56.962+0000 7f7eb5ffb700 1 -- 172.21.15.7:0/2699527903 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7eb0050c00 con 0x7f7ec00cafb0 2024-01-11T07:18:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.169+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec0159550 con 0x7f7ea4066f30 2024-01-11T07:18:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.169+0000 7f7eb5ffb700 1 -- 172.21.15.7:0/2699527903 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7ec0159550 con 0x7f7ea4066f30 2024-01-11T07:18:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ea4066f30 msgr2=0x7f7ea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ea4066f30 0x7f7ea40693e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009860 tx=0x7f7ebc017000 comp rx=0 tx=0).stop 2024-01-11T07:18:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ec00cafb0 msgr2=0x7f7ec0168170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ec00cafb0 0x7f7ec0168170 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000c580 tx=0x7f7eb0007e40 comp rx=0 tx=0).stop 2024-01-11T07:18:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 shutdown_connections 2024-01-11T07:18:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f7ea4066f30 0x7f7ea40693e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ec0158280 0x7f7ec00cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ec00b0b40 0x7f7ec00c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 --2- 172.21.15.7:0/2699527903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ec00cafb0 0x7f7ec0168170 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 >> 172.21.15.7:0/2699527903 conn(0x7f7ec0071100 msgr2=0x7f7ec01552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.171+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 shutdown_connections 2024-01-11T07:18:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.172+0000 7f7ec8001700 1 -- 172.21.15.7:0/2699527903 wait complete. 2024-01-11T07:18:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-01-11T07:18:57.185 INFO:tasks.ceph:Scrubbing osd.2 2024-01-11T07:18:57.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-11T07:18:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.323+0000 7feea1d83700 1 Processor -- start 2024-01-11T07:18:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.323+0000 7feea1d83700 1 -- start start 2024-01-11T07:18:57.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7feea1d83700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 0x7fee9c069680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7feea1d83700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee9c063940 0x7fee9c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7feea1d83700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c0c8570 con 0x7fee9c159bf0 2024-01-11T07:18:57.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7feea1d83700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c06e920 con 0x7fee9c063940 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7feea1d83700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c06ec00 con 0x7fee9c159fc0 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee9c063940 0x7fee9c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fee9c159bf0 0x7fee9c063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51886/0 (socket says 172.21.15.7:51886) 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9b7fe700 1 -- 172.21.15.7:0/2887445068 learned_addr learned my addr 172.21.15.7:0/2887445068 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9affd700 1 --2- 172.21.15.7:0/2887445068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 0x7fee9c069680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9bfff700 1 -- 172.21.15.7:0/2887445068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 msgr2=0x7fee9c069680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9bfff700 1 --2- 172.21.15.7:0/2887445068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 0x7fee9c069680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9bfff700 1 -- 172.21.15.7:0/2887445068 >> v1:172.21.15.7:6789/0 conn(0x7fee9c159bf0 legacy=0x7fee9c063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9bfff700 1 -- 172.21.15.7:0/2887445068 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0c7f20 con 0x7fee9c063940 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9a7fc700 1 -- 172.21.15.7:0/2887445068 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7731320 0 0) 0x7fee9c0c8570 con 0x7fee9c159bf0 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.324+0000 7fee9affd700 1 --2- 172.21.15.7:0/2887445068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 0x7fee9c069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7fee9bfff700 1 --2- 172.21.15.7:0/2887445068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 0x7fee9c06dc60 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fee94009e60 tx=0x7fee94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac942777b3084d41 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7fee9a7fc700 1 -- 172.21.15.7:0/2887445068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9400e070 con 0x7fee9c063940 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7fee9a7fc700 1 -- 172.21.15.7:0/2887445068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9400bc30 con 0x7fee9c063940 2024-01-11T07:18:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7fee9a7fc700 1 -- 172.21.15.7:0/2887445068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94012430 con 0x7fee9c063940 2024-01-11T07:18:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7feea1d83700 1 -- 172.21.15.7:0/2887445068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 msgr2=0x7fee9c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7feea1d83700 1 --2- 172.21.15.7:0/2887445068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 0x7fee9c06dc60 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fee94009e60 tx=0x7fee94009780 comp rx=0 tx=0).stop 2024-01-11T07:18:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7feea1d83700 1 -- 172.21.15.7:0/2887445068 shutdown_connections 2024-01-11T07:18:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7feea1d83700 1 --2- 172.21.15.7:0/2887445068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 0x7fee9c06dc60 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7feea1d83700 1 --2- 172.21.15.7:0/2887445068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 0x7fee9c069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7feea1d83700 1 -- 172.21.15.7:0/2887445068 >> 172.21.15.7:0/2887445068 conn(0x7fee9c0680a0 msgr2=0x7fee9c0cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7feea1d83700 1 -- 172.21.15.7:0/2887445068 shutdown_connections 2024-01-11T07:18:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.325+0000 7feea1d83700 1 -- 172.21.15.7:0/2887445068 wait complete. 2024-01-11T07:18:57.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.327+0000 7feea1d83700 1 Processor -- start 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.327+0000 7feea1d83700 1 -- start start 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7feea1d83700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 0x7fee9c0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7feea1d83700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee9c159bf0 0x7fee9c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7feea1d83700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 0x7fee9c15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7feea1d83700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0b2ff0 con 0x7fee9c063940 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7feea1d83700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c15b620 con 0x7fee9c159bf0 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7feea1d83700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0b2060 con 0x7fee9c159fc0 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7fee9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 0x7fee9c0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7fee9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 0x7fee9c0b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52700/0 (socket says 172.21.15.7:52700) 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.328+0000 7fee9b7fe700 1 -- 172.21.15.7:0/3322114295 learned_addr learned my addr 172.21.15.7:0/3322114295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee9affd700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee9c159bf0 0x7fee9c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee9bfff700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 0x7fee9c15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee9b7fe700 1 -- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 msgr2=0x7fee9c15ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee9b7fe700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fee9c159fc0 0x7fee9c15ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee9b7fe700 1 -- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee9c159bf0 msgr2=0x7fee9c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee9b7fe700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee9c159bf0 0x7fee9c15a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee9b7fe700 1 -- 172.21.15.7:0/3322114295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94009a20 con 0x7fee9c063940 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee9b7fe700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 0x7fee9c0b2ab0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00c810 tx=0x7fee8c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c005b30 con 0x7fee9c063940 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c005c90 con 0x7fee9c063940 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c0b2620 con 0x7fee9c063940 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c007300 con 0x7fee9c063940 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.329+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c15b340 con 0x7fee9c063940 2024-01-11T07:18:57.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.330+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee9c04c3c0 con 0x7fee9c063940 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.331+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fee8c0076b0 con 0x7fee9c063940 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.331+0000 7fee837fe700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee88066f30 0x7fee880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.331+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fee8c084850 con 0x7fee9c063940 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.331+0000 7fee837fe700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fee8806cd80 0x7fee8806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.331+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 --> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee8806f860 con 0x7fee8806cd80 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.331+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8c084b40 con 0x7fee9c063940 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.332+0000 7fee9bfff700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fee8806cd80 0x7fee8806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.332+0000 7fee9affd700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee88066f30 0x7fee880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.332+0000 7fee9bfff700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fee8806cd80 0x7fee8806f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.332+0000 7fee9affd700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee88066f30 0x7fee880693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee90006fd0 tx=0x7fee90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.333+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 <== osd.2 v2:172.21.15.7:6823/36946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fee8806f860 con 0x7fee8806cd80 2024-01-11T07:18:57.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.346+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 --> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fee9c0c7d00 con 0x7fee8806cd80 2024-01-11T07:18:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.346+0000 7fee837fe700 1 -- 172.21.15.7:0/3322114295 <== osd.2 v2:172.21.15.7:6823/36946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fee9c0c7d00 con 0x7fee8806cd80 2024-01-11T07:18:57.346 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-11T07:18:57.346 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fee8806cd80 msgr2=0x7fee8806f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.7:6823/36946,v1:172.21.15.7:6825/36946] conn(0x7fee8806cd80 0x7fee8806f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee88066f30 msgr2=0x7fee880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fee88066f30 0x7fee880693e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee90006fd0 tx=0x7fee90008040 comp rx=0 tx=0).stop 2024-01-11T07:18:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 msgr2=0x7fee9c0b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 --2- 172.21.15.7:0/3322114295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee9c063940 0x7fee9c0b2ab0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00c810 tx=0x7fee8c007df0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7fee9bfff700 1 -- 172.21.15.7:0/3322114295 reap_dead start 2024-01-11T07:18:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 shutdown_connections 2024-01-11T07:18:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 >> 172.21.15.7:0/3322114295 conn(0x7fee9c0680a0 msgr2=0x7fee9c069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 shutdown_connections 2024-01-11T07:18:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.347+0000 7feea1d83700 1 -- 172.21.15.7:0/3322114295 wait complete. 2024-01-11T07:18:57.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-11T07:18:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.494+0000 7f70ee172700 1 Processor -- start 2024-01-11T07:18:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.495+0000 7f70ee172700 1 -- start start 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.495+0000 7f70ee172700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70e8063230 0x7f70e8063600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.495+0000 7f70ee172700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f70e815a280 0x7f70e80aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.495+0000 7f70ee172700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80aca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.495+0000 7f70ee172700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e815d880 con 0x7f70e815a280 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.495+0000 7f70ee172700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e806c5b0 con 0x7f70e8063230 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.495+0000 7f70ee172700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e8063b40 con 0x7f70e80ac5b0 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.495+0000 7f70e6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f70e815a280 0x7f70e80aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f70e815a280 0x7f70e80aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52704/0 (socket says 172.21.15.7:52704) 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e6ffd700 1 -- 172.21.15.7:0/483941738 learned_addr learned my addr 172.21.15.7:0/483941738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e7fff700 1 --2- 172.21.15.7:0/483941738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e6ffd700 1 -- 172.21.15.7:0/483941738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 msgr2=0x7f70e80aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e77fe700 1 --2- 172.21.15.7:0/483941738 >> v2:172.21.15.113:3300/0 conn(0x7f70e8063230 0x7f70e8063600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e6ffd700 1 --2- 172.21.15.7:0/483941738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e6ffd700 1 -- 172.21.15.7:0/483941738 >> v2:172.21.15.113:3300/0 conn(0x7f70e8063230 msgr2=0x7f70e8063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e6ffd700 1 --2- 172.21.15.7:0/483941738 >> v2:172.21.15.113:3300/0 conn(0x7f70e8063230 0x7f70e8063600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e6ffd700 1 -- 172.21.15.7:0/483941738 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e815a7f0 con 0x7f70e815a280 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e77fe700 1 --2- 172.21.15.7:0/483941738 >> v2:172.21.15.113:3300/0 conn(0x7f70e8063230 0x7f70e8063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e7fff700 1 --2- 172.21.15.7:0/483941738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e6ffd700 1 --2- 172.21.15.7:0/483941738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e80aa9f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f70e80ad1f0 tx=0x7f70dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83db390ecb933c7c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e67fc700 1 -- 172.21.15.7:0/483941738 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc00b860 con 0x7f70e815a280 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e67fc700 1 -- 172.21.15.7:0/483941738 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc00b9c0 con 0x7f70e815a280 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.496+0000 7f70e67fc700 1 -- 172.21.15.7:0/483941738 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc00bd60 con 0x7f70e815a280 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 -- 172.21.15.7:0/483941738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 msgr2=0x7f70e80aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 --2- 172.21.15.7:0/483941738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e80aa9f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f70e80ad1f0 tx=0x7f70dc009250 comp rx=0 tx=0).stop 2024-01-11T07:18:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 -- 172.21.15.7:0/483941738 shutdown_connections 2024-01-11T07:18:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 --2- 172.21.15.7:0/483941738 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 --2- 172.21.15.7:0/483941738 >> v2:172.21.15.113:3300/0 conn(0x7f70e8063230 0x7f70e8063600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 --2- 172.21.15.7:0/483941738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e80aa9f0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 -- 172.21.15.7:0/483941738 >> 172.21.15.7:0/483941738 conn(0x7f70e80680e0 msgr2=0x7f70e80cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 -- 172.21.15.7:0/483941738 shutdown_connections 2024-01-11T07:18:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.497+0000 7f70ee172700 1 -- 172.21.15.7:0/483941738 wait complete. 2024-01-11T07:18:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.498+0000 7f70ee172700 1 Processor -- start 2024-01-11T07:18:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.498+0000 7f70ee172700 1 -- start start 2024-01-11T07:18:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70ee172700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e8063230 0x7f70e814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70ee172700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70ee172700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e806deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70ee172700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80b0aa0 con 0x7f70e815a280 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70ee172700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80b0d80 con 0x7f70e8063230 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70ee172700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e80b1060 con 0x7f70e80ac5b0 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e806deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e806deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52712/0 (socket says 172.21.15.7:52712) 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70e7fff700 1 -- 172.21.15.7:0/2763038040 learned_addr learned my addr 172.21.15.7:0/2763038040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70e6ffd700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.499+0000 7f70e77fe700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e8063230 0x7f70e814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70e7fff700 1 -- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 msgr2=0x7f70e80b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70e7fff700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70e7fff700 1 -- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e8063230 msgr2=0x7f70e814e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70e7fff700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e8063230 0x7f70e814e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70e7fff700 1 -- 172.21.15.7:0/2763038040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0094f0 con 0x7f70e815a280 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70e6ffd700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70e7fff700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e806deb0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f70e000a380 tx=0x7f70e0007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70cf7fe700 1 -- 172.21.15.7:0/2763038040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e000ae10 con 0x7f70e815a280 2024-01-11T07:18:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70cf7fe700 1 -- 172.21.15.7:0/2763038040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e000d650 con 0x7f70e815a280 2024-01-11T07:18:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70cf7fe700 1 -- 172.21.15.7:0/2763038040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e000da10 con 0x7f70e815a280 2024-01-11T07:18:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e80b1340 con 0x7f70e815a280 2024-01-11T07:18:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.500+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e814e650 con 0x7f70e815a280 2024-01-11T07:18:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.501+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e804c3c0 con 0x7f70e815a280 2024-01-11T07:18:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.502+0000 7f70cf7fe700 1 -- 172.21.15.7:0/2763038040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f70e0015440 con 0x7f70e815a280 2024-01-11T07:18:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.502+0000 7f70cf7fe700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d4066f30 0x7f70d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.502+0000 7f70cf7fe700 1 -- 172.21.15.7:0/2763038040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f70e0084c10 con 0x7f70e815a280 2024-01-11T07:18:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.502+0000 7f70e77fe700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d4066f30 0x7f70d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.503+0000 7f70e77fe700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d4066f30 0x7f70d40693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f70d8006fd0 tx=0x7f70d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.507+0000 7f70cf7fe700 1 -- 172.21.15.7:0/2763038040 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70e0052180 con 0x7f70e815a280 2024-01-11T07:18:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.712+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f70e80b0ef0 con 0x7f70d4066f30 2024-01-11T07:18:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.713+0000 7f70cf7fe700 1 -- 172.21.15.7:0/2763038040 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f70e80b0ef0 con 0x7f70d4066f30 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d4066f30 msgr2=0x7f70d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d4066f30 0x7f70d40693e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f70d8006fd0 tx=0x7f70d8008040 comp rx=0 tx=0).stop 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 msgr2=0x7f70e806deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e806deb0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f70e000a380 tx=0x7f70e0007900 comp rx=0 tx=0).stop 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 shutdown_connections 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f70d4066f30 0x7f70d40693e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70e815a280 0x7f70e806deb0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70e8063230 0x7f70e814e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 --2- 172.21.15.7:0/2763038040 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70e80ac5b0 0x7f70e80b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 >> 172.21.15.7:0/2763038040 conn(0x7f70e80680e0 msgr2=0x7f70e80cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 shutdown_connections 2024-01-11T07:18:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.715+0000 7f70ee172700 1 -- 172.21.15.7:0/2763038040 wait complete. 2024-01-11T07:18:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-01-11T07:18:57.729 INFO:tasks.ceph:Scrubbing osd.3 2024-01-11T07:18:57.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-11T07:18:57.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.864+0000 7f02f427a700 1 Processor -- start 2024-01-11T07:18:57.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.864+0000 7f02f427a700 1 -- start start 2024-01-11T07:18:57.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f427a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec150600 0x7f02ec068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f427a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ec0b00d0 0x7f02ec069150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f427a700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec157380 con 0x7f02ec1509d0 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f427a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec156790 con 0x7f02ec0b00d0 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f427a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec156a50 con 0x7f02ec150600 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f2817700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ec0b00d0 0x7f02ec069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f2817700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ec0b00d0 0x7f02ec069150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52726/0 (socket says 172.21.15.7:52726) 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f2817700 1 -- 172.21.15.7:0/1345638251 learned_addr learned my addr 172.21.15.7:0/1345638251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f2016700 1 --2- 172.21.15.7:0/1345638251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec150600 0x7f02ec068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f2817700 1 -- 172.21.15.7:0/1345638251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec150600 msgr2=0x7f02ec068c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f2817700 1 --2- 172.21.15.7:0/1345638251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec150600 0x7f02ec068c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f2817700 1 -- 172.21.15.7:0/1345638251 >> v1:172.21.15.113:6789/0 conn(0x7f02ec1509d0 legacy=0x7f02ec0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.865+0000 7f02f2817700 1 -- 172.21.15.7:0/1345638251 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec156d30 con 0x7f02ec0b00d0 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.866+0000 7f02f2817700 1 --2- 172.21.15.7:0/1345638251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec0b00d0 0x7f02ec069150 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009f20 tx=0x7f02e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25de4eefb4e346b server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.866+0000 7f02f1014700 1 -- 172.21.15.7:0/1345638251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400e070 con 0x7f02ec0b00d0 2024-01-11T07:18:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.866+0000 7f02f1014700 1 -- 172.21.15.7:0/1345638251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e400ba40 con 0x7f02ec0b00d0 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.866+0000 7f02f1014700 1 -- 172.21.15.7:0/1345638251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e40092e0 con 0x7f02ec0b00d0 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.866+0000 7f02f427a700 1 -- 172.21.15.7:0/1345638251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec0b00d0 msgr2=0x7f02ec069150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.866+0000 7f02f427a700 1 --2- 172.21.15.7:0/1345638251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec0b00d0 0x7f02ec069150 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009f20 tx=0x7f02e4009840 comp rx=0 tx=0).stop 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.867+0000 7f02f427a700 1 -- 172.21.15.7:0/1345638251 shutdown_connections 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.867+0000 7f02f427a700 1 --2- 172.21.15.7:0/1345638251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec0b00d0 0x7f02ec069150 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.867+0000 7f02f427a700 1 --2- 172.21.15.7:0/1345638251 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec150600 0x7f02ec068c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.867+0000 7f02f427a700 1 -- 172.21.15.7:0/1345638251 >> 172.21.15.7:0/1345638251 conn(0x7f02ec070690 msgr2=0x7f02ec070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.867+0000 7f02f427a700 1 -- 172.21.15.7:0/1345638251 shutdown_connections 2024-01-11T07:18:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.867+0000 7f02f427a700 1 -- 172.21.15.7:0/1345638251 wait complete. 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.868+0000 7f02f427a700 1 Processor -- start 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.868+0000 7f02f427a700 1 -- start start 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f427a700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec0b00d0 0x7f02ec167750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f427a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec150600 0x7f02ec14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f427a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ec1509d0 0x7f02ec0ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f427a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec166ab0 con 0x7f02ec150600 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f427a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec1672c0 con 0x7f02ec1509d0 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f427a700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec14b8e0 con 0x7f02ec0b00d0 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f1815700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec150600 0x7f02ec14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f1815700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec150600 0x7f02ec14b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52742/0 (socket says 172.21.15.7:52742) 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.869+0000 7f02f1815700 1 -- 172.21.15.7:0/3015857526 learned_addr learned my addr 172.21.15.7:0/3015857526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f2016700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec0b00d0 0x7f02ec167750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f2817700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ec1509d0 0x7f02ec0ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f1815700 1 -- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec0b00d0 msgr2=0x7f02ec167750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f1815700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec0b00d0 0x7f02ec167750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f1815700 1 -- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ec1509d0 msgr2=0x7f02ec0ab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f1815700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ec1509d0 0x7f02ec0ab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f1815700 1 -- 172.21.15.7:0/3015857526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4009ae0 con 0x7f02ec150600 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f2016700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02ec0b00d0 0x7f02ec167750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f2817700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02ec1509d0 0x7f02ec0ab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f1815700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec150600 0x7f02ec14b3a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f02e800b610 tx=0x7f02e8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800f510 con 0x7f02ec150600 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e800f670 con 0x7f02ec150600 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec153c40 con 0x7f02ec150600 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800fa30 con 0x7f02ec150600 2024-01-11T07:18:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.870+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec153960 con 0x7f02ec150600 2024-01-11T07:18:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.871+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02ec04c3c0 con 0x7f02ec150600 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.872+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f02e8024430 con 0x7f02ec150600 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.872+0000 7f02de7fc700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02d0066f30 0x7f02d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.872+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f02e8084840 con 0x7f02ec150600 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.872+0000 7f02de7fc700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f02d006cd80 0x7f02d006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.872+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 --> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02d006f860 con 0x7f02d006cd80 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.872+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02e8084b30 con 0x7f02ec150600 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.872+0000 7f02f2817700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f02d006cd80 0x7f02d006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.873+0000 7f02f2016700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02d0066f30 0x7f02d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.873+0000 7f02f2817700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f02d006cd80 0x7f02d006f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.873+0000 7f02f2016700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02d0066f30 0x7f02d00693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f02ec164700 tx=0x7f02e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.874+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 <== osd.3 v2:172.21.15.7:6800/36944 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f02d006f860 con 0x7f02d006cd80 2024-01-11T07:18:57.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.887+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 --> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f02ec0cd5e0 con 0x7f02d006cd80 2024-01-11T07:18:57.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.887+0000 7f02de7fc700 1 -- 172.21.15.7:0/3015857526 <== osd.3 v2:172.21.15.7:6800/36944 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f02ec0cd5e0 con 0x7f02d006cd80 2024-01-11T07:18:57.887 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:57.887 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-11T07:18:57.888 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f02d006cd80 msgr2=0x7f02d006f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.7:6800/36944,v1:172.21.15.7:6801/36944] conn(0x7f02d006cd80 0x7f02d006f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02d0066f30 msgr2=0x7f02d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f02d0066f30 0x7f02d00693e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f02ec164700 tx=0x7f02e0008040 comp rx=0 tx=0).stop 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec150600 msgr2=0x7f02ec14b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 --2- 172.21.15.7:0/3015857526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ec150600 0x7f02ec14b3a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f02e800b610 tx=0x7f02e8009120 comp rx=0 tx=0).stop 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f2817700 1 -- 172.21.15.7:0/3015857526 reap_dead start 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 shutdown_connections 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 >> 172.21.15.7:0/3015857526 conn(0x7f02ec070690 msgr2=0x7f02ec0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 shutdown_connections 2024-01-11T07:18:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:57.888+0000 7f02f427a700 1 -- 172.21.15.7:0/3015857526 wait complete. 2024-01-11T07:18:57.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-11T07:18:58.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.033+0000 7f1f79fdc700 1 Processor -- start 2024-01-11T07:18:58.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.033+0000 7f1f79fdc700 1 -- start start 2024-01-11T07:18:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f79fdc700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f74159c20 0x7f1f740b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f79fdc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f7406f4d0 0x7f1f740b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f79fdc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f74067de0 con 0x7f1f74159ff0 2024-01-11T07:18:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f79fdc700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f74069570 con 0x7f1f74159c20 2024-01-11T07:18:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f79fdc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b20e0 con 0x7f1f7406f4d0 2024-01-11T07:18:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f72ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f74159ff0 0x7f1f74068270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51896/0 (socket says 172.21.15.7:51896) 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f72ffd700 1 -- 172.21.15.7:0/2682126085 learned_addr learned my addr 172.21.15.7:0/2682126085 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f737fe700 1 --2- 172.21.15.7:0/2682126085 >> v2:172.21.15.113:3300/0 conn(0x7f1f74159c20 0x7f1f740b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.034+0000 7f1f73fff700 1 --2- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f7406f4d0 0x7f1f740b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f727fc700 1 -- 172.21.15.7:0/2682126085 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423358698 0 0) 0x7f1f74067de0 con 0x7f1f74159ff0 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f727fc700 1 -- 172.21.15.7:0/2682126085 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f60003570 con 0x7f1f74159ff0 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f737fe700 1 -- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f7406f4d0 msgr2=0x7f1f740b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f737fe700 1 --2- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f7406f4d0 0x7f1f740b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f737fe700 1 -- 172.21.15.7:0/2682126085 >> v1:172.21.15.7:6789/0 conn(0x7f1f74159ff0 legacy=0x7f1f74068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f737fe700 1 -- 172.21.15.7:0/2682126085 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7406f950 con 0x7f1f74159c20 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f73fff700 1 --2- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f7406f4d0 0x7f1f740b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f737fe700 1 --2- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f74159c20 0x7f1f740b1820 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098e0 tx=0x7f1f5c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb299919038346c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f727fc700 1 -- 172.21.15.7:0/2682126085 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011630 con 0x7f1f74159c20 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f727fc700 1 -- 172.21.15.7:0/2682126085 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c011790 con 0x7f1f74159c20 2024-01-11T07:18:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.035+0000 7f1f727fc700 1 -- 172.21.15.7:0/2682126085 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011b30 con 0x7f1f74159c20 2024-01-11T07:18:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.036+0000 7f1f79fdc700 1 -- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f74159c20 msgr2=0x7f1f740b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.036+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f74159c20 0x7f1f740b1820 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098e0 tx=0x7f1f5c0091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.036+0000 7f1f79fdc700 1 -- 172.21.15.7:0/2682126085 shutdown_connections 2024-01-11T07:18:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.036+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f7406f4d0 0x7f1f740b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.036+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/2682126085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f74159c20 0x7f1f740b1820 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.036+0000 7f1f79fdc700 1 -- 172.21.15.7:0/2682126085 >> 172.21.15.7:0/2682126085 conn(0x7f1f74071160 msgr2=0x7f1f7406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.036+0000 7f1f79fdc700 1 -- 172.21.15.7:0/2682126085 shutdown_connections 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.036+0000 7f1f79fdc700 1 -- 172.21.15.7:0/2682126085 wait complete. 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.037+0000 7f1f79fdc700 1 Processor -- start 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.037+0000 7f1f79fdc700 1 -- start start 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f79fdc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f7406f4d0 0x7f1f7416b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f79fdc700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f74159c20 0x7f1f740a7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f79fdc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74159ff0 0x7f1f74157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f79fdc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740a7a90 con 0x7f1f74159ff0 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f79fdc700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f74157d80 con 0x7f1f7406f4d0 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f79fdc700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740a64b0 con 0x7f1f74159c20 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f73fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74159ff0 0x7f1f74157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f73fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74159ff0 0x7f1f74157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52752/0 (socket says 172.21.15.7:52752) 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f73fff700 1 -- 172.21.15.7:0/1147707446 learned_addr learned my addr 172.21.15.7:0/1147707446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f737fe700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f7406f4d0 0x7f1f7416b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.038+0000 7f1f72ffd700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f74159c20 0x7f1f740a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f73fff700 1 -- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f74159c20 msgr2=0x7f1f740a7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f73fff700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f74159c20 0x7f1f740a7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f73fff700 1 -- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f7406f4d0 msgr2=0x7f1f7416b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f73fff700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f7406f4d0 0x7f1f7416b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f73fff700 1 -- 172.21.15.7:0/1147707446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c009a20 con 0x7f1f74159ff0 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f73fff700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74159ff0 0x7f1f74157840 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009fc0 tx=0x7f1f6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f6b7fe700 1 -- 172.21.15.7:0/1147707446 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00cbd0 con 0x7f1f74159ff0 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f6b7fe700 1 -- 172.21.15.7:0/1147707446 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c00f040 con 0x7f1f74159ff0 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c009420 con 0x7f1f74159ff0 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f6b7fe700 1 -- 172.21.15.7:0/1147707446 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c014480 con 0x7f1f74159ff0 2024-01-11T07:18:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.039+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740a6a70 con 0x7f1f74159ff0 2024-01-11T07:18:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.040+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7406b730 con 0x7f1f74159ff0 2024-01-11T07:18:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.041+0000 7f1f6b7fe700 1 -- 172.21.15.7:0/1147707446 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1f6c003020 con 0x7f1f74159ff0 2024-01-11T07:18:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.041+0000 7f1f6b7fe700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f54066f30 0x7f1f540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.041+0000 7f1f6b7fe700 1 -- 172.21.15.7:0/1147707446 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1f6c084b50 con 0x7f1f74159ff0 2024-01-11T07:18:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.041+0000 7f1f737fe700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f54066f30 0x7f1f540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.042+0000 7f1f737fe700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f54066f30 0x7f1f540693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0091e0 tx=0x7f1f5c017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.045+0000 7f1f6b7fe700 1 -- 172.21.15.7:0/1147707446 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f6c0520c0 con 0x7f1f74159ff0 2024-01-11T07:18:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.253+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f740c7e40 con 0x7f1f54066f30 2024-01-11T07:18:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.254+0000 7f1f6b7fe700 1 -- 172.21.15.7:0/1147707446 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1f740c7e40 con 0x7f1f54066f30 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f54066f30 msgr2=0x7f1f540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f54066f30 0x7f1f540693e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0091e0 tx=0x7f1f5c017000 comp rx=0 tx=0).stop 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74159ff0 msgr2=0x7f1f74157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74159ff0 0x7f1f74157840 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009fc0 tx=0x7f1f6c00b040 comp rx=0 tx=0).stop 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 shutdown_connections 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1f54066f30 0x7f1f540693e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f74159ff0 0x7f1f74157840 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f7406f4d0 0x7f1f7416b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 --2- 172.21.15.7:0/1147707446 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1f74159c20 0x7f1f740a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 >> 172.21.15.7:0/1147707446 conn(0x7f1f74071160 msgr2=0x7f1f740b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 shutdown_connections 2024-01-11T07:18:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.256+0000 7f1f79fdc700 1 -- 172.21.15.7:0/1147707446 wait complete. 2024-01-11T07:18:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-01-11T07:18:58.269 INFO:tasks.ceph:Scrubbing osd.4 2024-01-11T07:18:58.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-11T07:18:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.405+0000 7f8ddaeb0700 1 Processor -- start 2024-01-11T07:18:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.405+0000 7f8ddaeb0700 1 -- start start 2024-01-11T07:18:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.406+0000 7f8ddaeb0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 0x7f8dd40cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.406+0000 7f8ddaeb0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd406a820 con 0x7f8dd40c93a0 2024-01-11T07:18:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.406+0000 7f8ddaeb0700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40ccd50 con 0x7f8dd40aafd0 2024-01-11T07:18:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.406+0000 7f8ddaeb0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40c86e0 con 0x7f8dd40c8fd0 2024-01-11T07:18:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.406+0000 7f8dd3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8dd40c93a0 0x7f8dd40aa8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51904/0 (socket says 172.21.15.7:51904) 2024-01-11T07:18:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.406+0000 7f8dd3fff700 1 -- 172.21.15.7:0/3473321460 learned_addr learned my addr 172.21.15.7:0/3473321460 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd8c4c700 1 --2- 172.21.15.7:0/3473321460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 0x7f8dd40cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd37fe700 1 -- 172.21.15.7:0/3473321460 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084000633 0 0) 0x7f8dd406a820 con 0x7f8dd40c93a0 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd37fe700 1 -- 172.21.15.7:0/3473321460 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc4003570 con 0x7f8dd40c93a0 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd8c4c700 1 -- 172.21.15.7:0/3473321460 >> v1:172.21.15.113:6789/0 conn(0x7f8dd40aafd0 legacy=0x7f8dd41593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd37fe700 1 -- 172.21.15.7:0/3473321460 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392068961 0 0) 0x7f8dd40ccd50 con 0x7f8dd40aafd0 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd8c4c700 1 -- 172.21.15.7:0/3473321460 >> v1:172.21.15.7:6789/0 conn(0x7f8dd40c93a0 legacy=0x7f8dd40aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd8c4c700 1 -- 172.21.15.7:0/3473321460 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd406a1d0 con 0x7f8dd40c8fd0 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd8c4c700 1 --2- 172.21.15.7:0/3473321460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 0x7f8dd40cc750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009910 tx=0x7f8dc80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d41f70896b60c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd37fe700 1 -- 172.21.15.7:0/3473321460 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800b680 con 0x7f8dd40c8fd0 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd37fe700 1 -- 172.21.15.7:0/3473321460 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc800b7e0 con 0x7f8dd40c8fd0 2024-01-11T07:18:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.407+0000 7f8dd37fe700 1 -- 172.21.15.7:0/3473321460 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800bba0 con 0x7f8dd40c8fd0 2024-01-11T07:18:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.408+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/3473321460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 msgr2=0x7f8dd40cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.408+0000 7f8ddaeb0700 1 --2- 172.21.15.7:0/3473321460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 0x7f8dd40cc750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009910 tx=0x7f8dc80091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.408+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/3473321460 shutdown_connections 2024-01-11T07:18:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.408+0000 7f8ddaeb0700 1 --2- 172.21.15.7:0/3473321460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 0x7f8dd40cc750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.408+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/3473321460 >> 172.21.15.7:0/3473321460 conn(0x7f8dd40680e0 msgr2=0x7f8dd40cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.408+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/3473321460 shutdown_connections 2024-01-11T07:18:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.408+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/3473321460 wait complete. 2024-01-11T07:18:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.410+0000 7f8ddaeb0700 1 Processor -- start 2024-01-11T07:18:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.410+0000 7f8ddaeb0700 1 -- start start 2024-01-11T07:18:58.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8ddaeb0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40aafd0 0x7f8dd4155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8ddaeb0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 0x7f8dd40aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8ddaeb0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c93a0 0x7f8dd415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8ddaeb0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4155970 con 0x7f8dd40aafd0 2024-01-11T07:18:58.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8ddaeb0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd416f730 con 0x7f8dd40c93a0 2024-01-11T07:18:58.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8ddaeb0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd416f350 con 0x7f8dd40c8fd0 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd8c4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40aafd0 0x7f8dd4155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd8c4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40aafd0 0x7f8dd4155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52756/0 (socket says 172.21.15.7:52756) 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd8c4c700 1 -- 172.21.15.7:0/1295807206 learned_addr learned my addr 172.21.15.7:0/1295807206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd3fff700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 0x7f8dd40aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd944d700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c93a0 0x7f8dd415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd8c4c700 1 -- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 msgr2=0x7f8dd40aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd8c4c700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8dd40c8fd0 0x7f8dd40aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd8c4c700 1 -- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c93a0 msgr2=0x7f8dd415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd8c4c700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c93a0 0x7f8dd415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd8c4c700 1 -- 172.21.15.7:0/1295807206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc8009420 con 0x7f8dd40aafd0 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.411+0000 7f8dd944d700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c93a0 0x7f8dd415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.412+0000 7f8dd8c4c700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40aafd0 0x7f8dd4155430 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4068ef0 tx=0x7f8dc8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.412+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800bb60 con 0x7f8dd40aafd0 2024-01-11T07:18:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.412+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc8027450 con 0x7f8dd40aafd0 2024-01-11T07:18:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.412+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd416e740 con 0x7f8dd40aafd0 2024-01-11T07:18:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.412+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80277f0 con 0x7f8dd40aafd0 2024-01-11T07:18:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.412+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd416ed00 con 0x7f8dd40aafd0 2024-01-11T07:18:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.413+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8dd404c3c0 con 0x7f8dd40aafd0 2024-01-11T07:18:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.413+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8dc801a430 con 0x7f8dd40aafd0 2024-01-11T07:18:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.414+0000 7f8dd0ff9700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8db8066f30 0x7f8db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.414+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8dc8086410 con 0x7f8dd40aafd0 2024-01-11T07:18:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.414+0000 7f8dd0ff9700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f8db806cd80 0x7f8db806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.414+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 --> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8db806f860 con 0x7f8db806cd80 2024-01-11T07:18:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.414+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dc80867e0 con 0x7f8dd40aafd0 2024-01-11T07:18:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.414+0000 7f8dd3fff700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8db8066f30 0x7f8db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.414+0000 7f8dd944d700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f8db806cd80 0x7f8db806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.415+0000 7f8dd3fff700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8db8066f30 0x7f8db80693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00058d0 tx=0x7f8dc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.415+0000 7f8dd944d700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f8db806cd80 0x7f8db806f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.415+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 <== osd.4 v2:172.21.15.113:6800/37188 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f8db806f860 con 0x7f8db806cd80 2024-01-11T07:18:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.429+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 --> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8dd406a4b0 con 0x7f8db806cd80 2024-01-11T07:18:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8dd0ff9700 1 -- 172.21.15.7:0/1295807206 <== osd.4 v2:172.21.15.113:6800/37188 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f8dd406a4b0 con 0x7f8db806cd80 2024-01-11T07:18:58.430 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:58.430 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-11T07:18:58.430 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f8db806cd80 msgr2=0x7f8db806f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6800/37188,v1:172.21.15.113:6801/37188] conn(0x7f8db806cd80 0x7f8db806f190 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8db8066f30 msgr2=0x7f8db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8db8066f30 0x7f8db80693e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00058d0 tx=0x7f8dc0009040 comp rx=0 tx=0).stop 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40aafd0 msgr2=0x7f8dd4155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 --2- 172.21.15.7:0/1295807206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dd40aafd0 0x7f8dd4155430 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4068ef0 tx=0x7f8dc8017000 comp rx=0 tx=0).stop 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8dd944d700 1 -- 172.21.15.7:0/1295807206 reap_dead start 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 shutdown_connections 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 >> 172.21.15.7:0/1295807206 conn(0x7f8dd40680e0 msgr2=0x7f8dd40c1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 shutdown_connections 2024-01-11T07:18:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.430+0000 7f8ddaeb0700 1 -- 172.21.15.7:0/1295807206 wait complete. 2024-01-11T07:18:58.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.578+0000 7f1b50ce7700 1 Processor -- start 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.578+0000 7f1b50ce7700 1 -- start start 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.579+0000 7f1b50ce7700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b4c159ba0 0x7f1b4c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.579+0000 7f1b50ce7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b4c159f70 0x7f1b4c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.579+0000 7f1b50ce7700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4c067c60 con 0x7f1b4c0c9570 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.579+0000 7f1b50ce7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c069230 con 0x7f1b4c159f70 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.579+0000 7f1b50ce7700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c069510 con 0x7f1b4c159ba0 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.579+0000 7f1b49d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b4c159f70 0x7f1b4c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b49d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b4c159f70 0x7f1b4c0c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52766/0 (socket says 172.21.15.7:52766) 2024-01-11T07:18:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b49d9b700 1 -- 172.21.15.7:0/1933254961 learned_addr learned my addr 172.21.15.7:0/1933254961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b4a59c700 1 --2- 172.21.15.7:0/1933254961 >> v2:172.21.15.113:3300/0 conn(0x7f1b4c159ba0 0x7f1b4c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b49d9b700 1 -- 172.21.15.7:0/1933254961 >> v2:172.21.15.113:3300/0 conn(0x7f1b4c159ba0 msgr2=0x7f1b4c06d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b49d9b700 1 --2- 172.21.15.7:0/1933254961 >> v2:172.21.15.113:3300/0 conn(0x7f1b4c159ba0 0x7f1b4c06d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b49d9b700 1 -- 172.21.15.7:0/1933254961 >> v1:172.21.15.113:6789/0 conn(0x7f1b4c0c9570 legacy=0x7f1b4c0680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b49d9b700 1 -- 172.21.15.7:0/1933254961 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c067610 con 0x7f1b4c159f70 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b49d9b700 1 --2- 172.21.15.7:0/1933254961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c0c9030 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c06e920 tx=0x7f1b3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427508331907c282 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b4959a700 1 -- 172.21.15.7:0/1933254961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00b820 con 0x7f1b4c159f70 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b4959a700 1 -- 172.21.15.7:0/1933254961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c00b980 con 0x7f1b4c159f70 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.580+0000 7f1b4959a700 1 -- 172.21.15.7:0/1933254961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00bd20 con 0x7f1b4c159f70 2024-01-11T07:18:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.581+0000 7f1b50ce7700 1 -- 172.21.15.7:0/1933254961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 msgr2=0x7f1b4c0c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.581+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/1933254961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c0c9030 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c06e920 tx=0x7f1b3c009250 comp rx=0 tx=0).stop 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.581+0000 7f1b50ce7700 1 -- 172.21.15.7:0/1933254961 shutdown_connections 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.581+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/1933254961 >> v2:172.21.15.113:3300/0 conn(0x7f1b4c159ba0 0x7f1b4c06d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.581+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/1933254961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c0c9030 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.581+0000 7f1b50ce7700 1 -- 172.21.15.7:0/1933254961 >> 172.21.15.7:0/1933254961 conn(0x7f1b4c06ae00 msgr2=0x7f1b4c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.581+0000 7f1b50ce7700 1 -- 172.21.15.7:0/1933254961 shutdown_connections 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.581+0000 7f1b50ce7700 1 -- 172.21.15.7:0/1933254961 wait complete. 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.583+0000 7f1b50ce7700 1 Processor -- start 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.583+0000 7f1b50ce7700 1 -- start start 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b50ce7700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b4c0c9570 0x7f1b4c1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b50ce7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b4c159ba0 0x7f1b4c16a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b50ce7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c16a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b50ce7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c15b380 con 0x7f1b4c159f70 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b50ce7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c15b940 con 0x7f1b4c159ba0 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b50ce7700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c16af20 con 0x7f1b4c0c9570 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c16a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c16a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52776/0 (socket says 172.21.15.7:52776) 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4ad9d700 1 -- 172.21.15.7:0/2486613117 learned_addr learned my addr 172.21.15.7:0/2486613117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4a59c700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b4c0c9570 0x7f1b4c1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b49d9b700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b4c159ba0 0x7f1b4c16a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4ad9d700 1 -- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b4c0c9570 msgr2=0x7f1b4c1525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4ad9d700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b4c0c9570 0x7f1b4c1525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4ad9d700 1 -- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b4c159ba0 msgr2=0x7f1b4c16a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4ad9d700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b4c159ba0 0x7f1b4c16a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.584+0000 7f1b4ad9d700 1 -- 172.21.15.7:0/2486613117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c0094f0 con 0x7f1b4c159f70 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.585+0000 7f1b4ad9d700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c16a8e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c930 tx=0x7f1b440026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.585+0000 7f1b427fc700 1 -- 172.21.15.7:0/2486613117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44012630 con 0x7f1b4c159f70 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.585+0000 7f1b427fc700 1 -- 172.21.15.7:0/2486613117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b440127d0 con 0x7f1b4c159f70 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.585+0000 7f1b427fc700 1 -- 172.21.15.7:0/2486613117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44012b90 con 0x7f1b4c159f70 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.585+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c16b200 con 0x7f1b4c159f70 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.585+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c0b23e0 con 0x7f1b4c159f70 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.586+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c16b390 con 0x7f1b4c159f70 2024-01-11T07:18:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.586+0000 7f1b427fc700 1 -- 172.21.15.7:0/2486613117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1b4400a3b0 con 0x7f1b4c159f70 2024-01-11T07:18:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.587+0000 7f1b427fc700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b2c066f30 0x7f1b2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.587+0000 7f1b427fc700 1 -- 172.21.15.7:0/2486613117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1b44083f90 con 0x7f1b4c159f70 2024-01-11T07:18:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.587+0000 7f1b4a59c700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b2c066f30 0x7f1b2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.587+0000 7f1b4a59c700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b2c066f30 0x7f1b2c0693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1b34006fd0 tx=0x7f1b34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.591+0000 7f1b427fc700 1 -- 172.21.15.7:0/2486613117 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b44051500 con 0x7f1b4c159f70 2024-01-11T07:18:58.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.795+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b4c15bab0 con 0x7f1b2c066f30 2024-01-11T07:18:58.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.796+0000 7f1b427fc700 1 -- 172.21.15.7:0/2486613117 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1b4c15bab0 con 0x7f1b2c066f30 2024-01-11T07:18:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b2c066f30 msgr2=0x7f1b2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b2c066f30 0x7f1b2c0693e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1b34006fd0 tx=0x7f1b34008040 comp rx=0 tx=0).stop 2024-01-11T07:18:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 msgr2=0x7f1b4c16a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c16a8e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c930 tx=0x7f1b440026e0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 shutdown_connections 2024-01-11T07:18:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1b2c066f30 0x7f1b2c0693e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b4c159f70 0x7f1b4c16a8e0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1b4c0c9570 0x7f1b4c1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 --2- 172.21.15.7:0/2486613117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b4c159ba0 0x7f1b4c16a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 >> 172.21.15.7:0/2486613117 conn(0x7f1b4c06ae00 msgr2=0x7f1b4c153510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 shutdown_connections 2024-01-11T07:18:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.798+0000 7f1b50ce7700 1 -- 172.21.15.7:0/2486613117 wait complete. 2024-01-11T07:18:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-01-11T07:18:58.811 INFO:tasks.ceph:Scrubbing osd.5 2024-01-11T07:18:58.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-11T07:18:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.947+0000 7f72f8940700 1 Processor -- start 2024-01-11T07:18:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.947+0000 7f72f8940700 1 -- start start 2024-01-11T07:18:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f8940700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72f4159a80 0x7f72f40adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f8940700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 0x7f72f406f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f8940700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f40b1d40 con 0x7f72f40b33b0 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f8940700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f40696c0 con 0x7f72f4159a80 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f8940700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40b17c0 con 0x7f72f40cede0 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f1d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72f40b33b0 0x7f72f40ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51910/0 (socket says 172.21.15.7:51910) 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f1d9b700 1 -- 172.21.15.7:0/2155629163 learned_addr learned my addr 172.21.15.7:0/2155629163 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f259c700 1 --2- 172.21.15.7:0/2155629163 >> v2:172.21.15.113:3300/0 conn(0x7f72f4159a80 0x7f72f40adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f2d9d700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 0x7f72f406f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362223809 0 0) 0x7f72f40b1d40 con 0x7f72f40b33b0 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.948+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e0003570 con 0x7f72f40b33b0 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123921840 0 0) 0x7f72e0003570 con 0x7f72f40b33b0 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 msgr2=0x7f72f406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 0x7f72f406f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 >> v2:172.21.15.113:3300/0 conn(0x7f72f4159a80 msgr2=0x7f72f40adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 --2- 172.21.15.7:0/2155629163 >> v2:172.21.15.113:3300/0 conn(0x7f72f4159a80 0x7f72f40adbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4068720 con 0x7f72f40b33b0 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f72dc001470 con 0x7f72f40b33b0 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f2d9d700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 0x7f72f406f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 >> v1:172.21.15.7:6789/0 conn(0x7f72f40b33b0 legacy=0x7f72f40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e0003a60 0x7f72e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 0x7f72e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e0004d50 0x7f72e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40696c0 con 0x7f72e0003a60 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40b17c0 con 0x7f72e0004d50 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e000ba60 con 0x7f72f40cede0 2024-01-11T07:18:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f259c700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e0003a60 0x7f72e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f1d9b700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 0x7f72e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f2d9d700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e0004d50 0x7f72e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f259c700 1 -- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 msgr2=0x7f72e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f259c700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f40cede0 0x7f72e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f259c700 1 -- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e0004d50 msgr2=0x7f72e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f259c700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e0004d50 0x7f72e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.949+0000 7f72f259c700 1 -- 172.21.15.7:0/2155629163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec009a20 con 0x7f72e0003a60 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.950+0000 7f72f259c700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e0003a60 0x7f72e0003e70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009c00 tx=0x7f72e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeea47067227de4 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.950+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400e6b0 con 0x7f72e0003a60 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.950+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e400e810 con 0x7f72e0003a60 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.950+0000 7f72f159a700 1 -- 172.21.15.7:0/2155629163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400ebd0 con 0x7f72e0003a60 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.950+0000 7f72f8940700 1 -- 172.21.15.7:0/2155629163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e0003a60 msgr2=0x7f72e0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.950+0000 7f72f8940700 1 --2- 172.21.15.7:0/2155629163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e0003a60 0x7f72e0003e70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009c00 tx=0x7f72e40091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.950+0000 7f72f2d9d700 1 -- 172.21.15.7:0/2155629163 reap_dead start 2024-01-11T07:18:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.951+0000 7f72f8940700 1 -- 172.21.15.7:0/2155629163 shutdown_connections 2024-01-11T07:18:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.951+0000 7f72f8940700 1 -- 172.21.15.7:0/2155629163 >> 172.21.15.7:0/2155629163 conn(0x7f72f406b910 msgr2=0x7f72f406ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.951+0000 7f72f8940700 1 -- 172.21.15.7:0/2155629163 shutdown_connections 2024-01-11T07:18:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.951+0000 7f72f8940700 1 -- 172.21.15.7:0/2155629163 wait complete. 2024-01-11T07:18:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.952+0000 7f72f8940700 1 Processor -- start 2024-01-11T07:18:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.952+0000 7f72f8940700 1 -- start start 2024-01-11T07:18:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f8940700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e0003a60 0x7f72f4159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f8940700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f4141470 0x7f72f415c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f8940700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f4154840 0x7f72f4154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f8940700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f41551f0 con 0x7f72f4154840 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f8940700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f415cd50 con 0x7f72e0003a60 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f8940700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f415d010 con 0x7f72f4141470 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f4154840 0x7f72f4154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f4154840 0x7f72f4154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52796/0 (socket says 172.21.15.7:52796) 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 -- 172.21.15.7:0/3716036101 learned_addr learned my addr 172.21.15.7:0/3716036101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f259c700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e0003a60 0x7f72f4159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f1d9b700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f4141470 0x7f72f415c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 -- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f4141470 msgr2=0x7f72f415c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f72f4141470 0x7f72f415c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 -- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e0003a60 msgr2=0x7f72f4159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72e0003a60 0x7f72f4159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 -- 172.21.15.7:0/3716036101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f415d2f0 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f2d9d700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f4154840 0x7f72f4154cb0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f72f4068ac0 tx=0x7f72ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00e070 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec011c50 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f415d5d0 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.953+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec016470 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.954+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f415db90 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.954+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72f404c3c0 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.955+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f72ec016840 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.955+0000 7f72ea7fc700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.955+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f72ec084e80 con 0x7f72f4154840 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.956+0000 7f72f259c700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.956+0000 7f72ea7fc700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f72d406cd80 0x7f72d406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.956+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 --> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72d406f860 con 0x7f72d406cd80 2024-01-11T07:18:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.956+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72ec0874c0 con 0x7f72f4154840 2024-01-11T07:18:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.956+0000 7f72f1d9b700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f72d406cd80 0x7f72d406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.956+0000 7f72f259c700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f72e40091e0 tx=0x7f72e401a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.956+0000 7f72f1d9b700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f72d406cd80 0x7f72d406f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.957+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 <== osd.5 v2:172.21.15.113:6816/37190 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f72d406f860 con 0x7f72d406cd80 2024-01-11T07:18:58.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.970+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 --> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f72f415e370 con 0x7f72d406cd80 2024-01-11T07:18:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72ea7fc700 1 -- 172.21.15.7:0/3716036101 <== osd.5 v2:172.21.15.113:6816/37190 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f72f415e370 con 0x7f72d406cd80 2024-01-11T07:18:58.971 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-11T07:18:58.971 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f72d406cd80 msgr2=0x7f72d406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f8940700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6816/37190,v1:172.21.15.113:6817/37190] conn(0x7f72d406cd80 0x7f72d406f190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 msgr2=0x7f72d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f8940700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f72d4066f30 0x7f72d40693e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f72e40091e0 tx=0x7f72e401a000 comp rx=0 tx=0).stop 2024-01-11T07:18:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f4154840 msgr2=0x7f72f4154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f8940700 1 --2- 172.21.15.7:0/3716036101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72f4154840 0x7f72f4154cb0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f72f4068ac0 tx=0x7f72ec00b040 comp rx=0 tx=0).stop 2024-01-11T07:18:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f2d9d700 1 -- 172.21.15.7:0/3716036101 reap_dead start 2024-01-11T07:18:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 shutdown_connections 2024-01-11T07:18:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.971+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 >> 172.21.15.7:0/3716036101 conn(0x7f72f406b910 msgr2=0x7f72f4155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.972+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 shutdown_connections 2024-01-11T07:18:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:58.972+0000 7f72f8940700 1 -- 172.21.15.7:0/3716036101 wait complete. 2024-01-11T07:18:58.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-11T07:18:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.117+0000 7f886842e700 1 Processor -- start 2024-01-11T07:18:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.117+0000 7f886842e700 1 -- start start 2024-01-11T07:18:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.118+0000 7f886842e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8860063230 0x7f886015e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f886842e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f886015aa40 0x7f886006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f886842e700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886015df80 con 0x7f8860063600 2024-01-11T07:18:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f886842e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886015e950 con 0x7f886015aa40 2024-01-11T07:18:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f886842e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88600aab70 con 0x7f8860063230 2024-01-11T07:18:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88669cb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f886015aa40 0x7f886006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88669cb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f886015aa40 0x7f886006b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52800/0 (socket says 172.21.15.7:52800) 2024-01-11T07:18:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88669cb700 1 -- 172.21.15.7:0/2560760713 learned_addr learned my addr 172.21.15.7:0/2560760713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88661ca700 1 --2- 172.21.15.7:0/2560760713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8860063230 0x7f886015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88669cb700 1 -- 172.21.15.7:0/2560760713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8860063230 msgr2=0x7f886015e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88669cb700 1 --2- 172.21.15.7:0/2560760713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8860063230 0x7f886015e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88669cb700 1 -- 172.21.15.7:0/2560760713 >> v1:172.21.15.113:6789/0 conn(0x7f8860063600 legacy=0x7f886015a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88669cb700 1 -- 172.21.15.7:0/2560760713 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886015af10 con 0x7f886015aa40 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88669cb700 1 --2- 172.21.15.7:0/2560760713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f886015aa40 0x7f886006b530 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8858009f20 tx=0x7f8858009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ab1f8d1c9cf3cc server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.119+0000 7f88651c8700 1 -- 172.21.15.7:0/2560760713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885800e070 con 0x7f886015aa40 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f88651c8700 1 -- 172.21.15.7:0/2560760713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885800ba80 con 0x7f886015aa40 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f88651c8700 1 -- 172.21.15.7:0/2560760713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88580092e0 con 0x7f886015aa40 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f886842e700 1 -- 172.21.15.7:0/2560760713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f886015aa40 msgr2=0x7f886006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f886842e700 1 --2- 172.21.15.7:0/2560760713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f886015aa40 0x7f886006b530 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8858009f20 tx=0x7f8858009840 comp rx=0 tx=0).stop 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f886842e700 1 -- 172.21.15.7:0/2560760713 shutdown_connections 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f886842e700 1 --2- 172.21.15.7:0/2560760713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f886015aa40 0x7f886006b530 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f886842e700 1 --2- 172.21.15.7:0/2560760713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8860063230 0x7f886015e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f886842e700 1 -- 172.21.15.7:0/2560760713 >> 172.21.15.7:0/2560760713 conn(0x7f88600cd910 msgr2=0x7f88600cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f886842e700 1 -- 172.21.15.7:0/2560760713 shutdown_connections 2024-01-11T07:18:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.120+0000 7f886842e700 1 -- 172.21.15.7:0/2560760713 wait complete. 2024-01-11T07:18:59.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.121+0000 7f886842e700 1 Processor -- start 2024-01-11T07:18:59.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.121+0000 7f886842e700 1 -- start start 2024-01-11T07:18:59.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f886842e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8860063230 0x7f88601484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f886842e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8860063600 0x7f88601506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f886842e700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f886015aa40 0x7f8860150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f886842e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8860148a20 con 0x7f8860063230 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f886842e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88600cd2c0 con 0x7f8860063600 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f886842e700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88600cc710 con 0x7f886015aa40 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f88661ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8860063230 0x7f88601484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f88661ca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8860063230 0x7f88601484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52816/0 (socket says 172.21.15.7:52816) 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.122+0000 7f88661ca700 1 -- 172.21.15.7:0/2313089180 learned_addr learned my addr 172.21.15.7:0/2313089180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88669cb700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f886015aa40 0x7f8860150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88659c9700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8860063600 0x7f88601506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88661ca700 1 -- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f886015aa40 msgr2=0x7f8860150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88661ca700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f886015aa40 0x7f8860150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88661ca700 1 -- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8860063600 msgr2=0x7f88601506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88661ca700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8860063600 0x7f88601506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88661ca700 1 -- 172.21.15.7:0/2313089180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8858009ae0 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88659c9700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8860063600 0x7f88601506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88669cb700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f886015aa40 0x7f8860150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88661ca700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8860063230 0x7f88601484e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f885c00d890 tx=0x7f885c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88567fc700 1 -- 172.21.15.7:0/2313089180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c00b600 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88567fc700 1 -- 172.21.15.7:0/2313089180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c00b760 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88600cc990 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f88567fc700 1 -- 172.21.15.7:0/2313089180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c00bae0 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.123+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88600ccf50 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.124+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886004c3c0 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.125+0000 7f88567fc700 1 -- 172.21.15.7:0/2313089180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f885c01e460 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.125+0000 7f88567fc700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8844066f30 0x7f88440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.125+0000 7f88567fc700 1 -- 172.21.15.7:0/2313089180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f885c0848e0 con 0x7f8860063230 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.125+0000 7f88659c9700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8844066f30 0x7f88440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.126+0000 7f88659c9700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8844066f30 0x7f88440693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f88600cb790 tx=0x7f8850009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.130+0000 7f88567fc700 1 -- 172.21.15.7:0/2313089180 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f885c051e50 con 0x7f8860063230 2024-01-11T07:18:59.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.331+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f88600cd400 con 0x7f8844066f30 2024-01-11T07:18:59.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.331+0000 7f88567fc700 1 -- 172.21.15.7:0/2313089180 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8850001490 con 0x7f8844066f30 2024-01-11T07:18:59.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8844066f30 msgr2=0x7f88440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8844066f30 0x7f88440693e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f88600cb790 tx=0x7f8850009040 comp rx=0 tx=0).stop 2024-01-11T07:18:59.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8860063230 msgr2=0x7f88601484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8860063230 0x7f88601484e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f885c00d890 tx=0x7f885c008040 comp rx=0 tx=0).stop 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 shutdown_connections 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f8844066f30 0x7f88440693e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f886015aa40 0x7f8860150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8860063230 0x7f88601484e0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 --2- 172.21.15.7:0/2313089180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8860063600 0x7f88601506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 >> 172.21.15.7:0/2313089180 conn(0x7f88600cd910 msgr2=0x7f886015c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 shutdown_connections 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.334+0000 7f886842e700 1 -- 172.21.15.7:0/2313089180 wait complete. 2024-01-11T07:18:59.335 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-01-11T07:18:59.347 INFO:tasks.ceph:Scrubbing osd.6 2024-01-11T07:18:59.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-11T07:18:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.484+0000 7fd8c8922700 1 Processor -- start 2024-01-11T07:18:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.484+0000 7fd8c8922700 1 -- start start 2024-01-11T07:18:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.485+0000 7fd8c8922700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8c4063480 0x7fd8c406e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.485+0000 7fd8c8922700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd8c4063850 0x7fd8c40aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.485+0000 7fd8c8922700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c41590a0 con 0x7fd8c40acfb0 2024-01-11T07:18:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.485+0000 7fd8c8922700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c4158ae0 con 0x7fd8c4063480 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.485+0000 7fd8c8922700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c4158dc0 con 0x7fd8c4063850 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.485+0000 7fd8c259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8c4063480 0x7fd8c406e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.485+0000 7fd8c259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8c4063480 0x7fd8c406e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52818/0 (socket says 172.21.15.7:52818) 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.485+0000 7fd8c259c700 1 -- 172.21.15.7:0/2699950637 learned_addr learned my addr 172.21.15.7:0/2699950637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c2d9d700 1 --2- 172.21.15.7:0/2699950637 >> v2:172.21.15.113:3300/0 conn(0x7fd8c4063850 0x7fd8c40aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c259c700 1 -- 172.21.15.7:0/2699950637 >> v2:172.21.15.113:3300/0 conn(0x7fd8c4063850 msgr2=0x7fd8c40aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c259c700 1 --2- 172.21.15.7:0/2699950637 >> v2:172.21.15.113:3300/0 conn(0x7fd8c4063850 0x7fd8c40aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c259c700 1 -- 172.21.15.7:0/2699950637 >> v1:172.21.15.113:6789/0 conn(0x7fd8c40acfb0 legacy=0x7fd8c40ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c259c700 1 -- 172.21.15.7:0/2699950637 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c406dc80 con 0x7fd8c4063480 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c259c700 1 --2- 172.21.15.7:0/2699950637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c4063480 0x7fd8c406e480 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40c7de0 tx=0x7fd8b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982fb0df654de6c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c159a700 1 -- 172.21.15.7:0/2699950637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400b820 con 0x7fd8c4063480 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c159a700 1 -- 172.21.15.7:0/2699950637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400b980 con 0x7fd8c4063480 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.486+0000 7fd8c159a700 1 -- 172.21.15.7:0/2699950637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400bd20 con 0x7fd8c4063480 2024-01-11T07:18:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.487+0000 7fd8c8922700 1 -- 172.21.15.7:0/2699950637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c4063480 msgr2=0x7fd8c406e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.487+0000 7fd8c8922700 1 --2- 172.21.15.7:0/2699950637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c4063480 0x7fd8c406e480 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40c7de0 tx=0x7fd8b4009250 comp rx=0 tx=0).stop 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.487+0000 7fd8c8922700 1 -- 172.21.15.7:0/2699950637 shutdown_connections 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.487+0000 7fd8c8922700 1 --2- 172.21.15.7:0/2699950637 >> v2:172.21.15.113:3300/0 conn(0x7fd8c4063850 0x7fd8c40aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.487+0000 7fd8c8922700 1 --2- 172.21.15.7:0/2699950637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c4063480 0x7fd8c406e480 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.487+0000 7fd8c8922700 1 -- 172.21.15.7:0/2699950637 >> 172.21.15.7:0/2699950637 conn(0x7fd8c40680a0 msgr2=0x7fd8c40cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.487+0000 7fd8c8922700 1 -- 172.21.15.7:0/2699950637 shutdown_connections 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.487+0000 7fd8c8922700 1 -- 172.21.15.7:0/2699950637 wait complete. 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.488+0000 7fd8c8922700 1 Processor -- start 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.488+0000 7fd8c8922700 1 -- start start 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c8922700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8c4063480 0x7fd8c40b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c8922700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c4063850 0x7fd8c40c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c8922700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c40acfb0 0x7fd8c40c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c8922700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c41555e0 con 0x7fd8c40acfb0 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c8922700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b35d0 con 0x7fd8c4063480 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c8922700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c415bf70 con 0x7fd8c4063850 2024-01-11T07:18:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c40acfb0 0x7fd8c40c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c40acfb0 0x7fd8c40c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52828/0 (socket says 172.21.15.7:52828) 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c2d9d700 1 -- 172.21.15.7:0/1914234810 learned_addr learned my addr 172.21.15.7:0/1914234810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c259c700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8c4063480 0x7fd8c40b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c1d9b700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c4063850 0x7fd8c40c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c2d9d700 1 -- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c4063850 msgr2=0x7fd8c40c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c2d9d700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c4063850 0x7fd8c40c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c2d9d700 1 -- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8c4063480 msgr2=0x7fd8c40b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c2d9d700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8c4063480 0x7fd8c40b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c2d9d700 1 -- 172.21.15.7:0/1914234810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40094f0 con 0x7fd8c40acfb0 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.489+0000 7fd8c1d9b700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd8c4063850 0x7fd8c40c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.490+0000 7fd8c2d9d700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c40acfb0 0x7fd8c40c5030 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009fa0 tx=0x7fd8bc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.490+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00ec10 con 0x7fd8c40acfb0 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.490+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc00ed70 con 0x7fd8c40acfb0 2024-01-11T07:18:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.490+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc014670 con 0x7fd8c40acfb0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.490+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c40b3010 con 0x7fd8c40acfb0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.490+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c415bc00 con 0x7fd8c40acfb0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.491+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd8c415ba60 con 0x7fd8c40acfb0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.491+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fd8bc0149e0 con 0x7fd8c40acfb0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.492+0000 7fd8ba7fc700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd8a4066f30 0x7fd8a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.492+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd8bc0857f0 con 0x7fd8c40acfb0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.492+0000 7fd8ba7fc700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7fd8a406cd80 0x7fd8a406f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.492+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 --> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8a406f860 con 0x7fd8a406cd80 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.492+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8bc085ae0 con 0x7fd8c40acfb0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.492+0000 7fd8c259c700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd8a4066f30 0x7fd8a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.492+0000 7fd8c1d9b700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7fd8a406cd80 0x7fd8a406f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.492+0000 7fd8c259c700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd8a4066f30 0x7fd8a40693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40c7de0 tx=0x7fd8b4016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.493+0000 7fd8c1d9b700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7fd8a406cd80 0x7fd8a406f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.493+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 <== osd.6 v2:172.21.15.113:6808/37189 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7fd8a406f860 con 0x7fd8a406cd80 2024-01-11T07:18:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.507+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 --> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd8c404c3c0 con 0x7fd8a406cd80 2024-01-11T07:18:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.507+0000 7fd8ba7fc700 1 -- 172.21.15.7:0/1914234810 <== osd.6 v2:172.21.15.113:6808/37189 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7fd8c404c3c0 con 0x7fd8a406cd80 2024-01-11T07:18:59.507 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:18:59.507 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-11T07:18:59.507 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:18:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7fd8a406cd80 msgr2=0x7fd8a406f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6808/37189,v1:172.21.15.113:6809/37189] conn(0x7fd8a406cd80 0x7fd8a406f190 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd8a4066f30 msgr2=0x7fd8a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fd8a4066f30 0x7fd8a40693e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40c7de0 tx=0x7fd8b4016000 comp rx=0 tx=0).stop 2024-01-11T07:18:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c40acfb0 msgr2=0x7fd8c40c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 --2- 172.21.15.7:0/1914234810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8c40acfb0 0x7fd8c40c5030 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009fa0 tx=0x7fd8bc009900 comp rx=0 tx=0).stop 2024-01-11T07:18:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c2d9d700 1 -- 172.21.15.7:0/1914234810 reap_dead start 2024-01-11T07:18:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 shutdown_connections 2024-01-11T07:18:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 >> 172.21.15.7:0/1914234810 conn(0x7fd8c40680a0 msgr2=0x7fd8c4156e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 shutdown_connections 2024-01-11T07:18:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.508+0000 7fd8c8922700 1 -- 172.21.15.7:0/1914234810 wait complete. 2024-01-11T07:18:59.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-11T07:18:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.652+0000 7f8308be0700 1 Processor -- start 2024-01-11T07:18:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.652+0000 7f8308be0700 1 -- start start 2024-01-11T07:18:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f8308be0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83040cafb0 0x7f83040ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f8308be0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 0x7f8304161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f8308be0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8304159300 con 0x7f8304158280 2024-01-11T07:18:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f8308be0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83040cf260 con 0x7f83040cafb0 2024-01-11T07:18:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f8308be0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83040cf540 con 0x7f83040b0b40 2024-01-11T07:18:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f830259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83040cafb0 0x7f83040ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f8301d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8304158280 0x7f83040b0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51918/0 (socket says 172.21.15.7:51918) 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f8301d9b700 1 -- 172.21.15.7:0/3396424686 learned_addr learned my addr 172.21.15.7:0/3396424686 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f8302d9d700 1 --2- 172.21.15.7:0/3396424686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 0x7f8304161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f830259c700 1 -- 172.21.15.7:0/3396424686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 msgr2=0x7f8304161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f830259c700 1 --2- 172.21.15.7:0/3396424686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 0x7f8304161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f830259c700 1 -- 172.21.15.7:0/3396424686 >> v1:172.21.15.7:6789/0 conn(0x7f8304158280 legacy=0x7f83040b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f830259c700 1 -- 172.21.15.7:0/3396424686 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83040cf820 con 0x7f83040cafb0 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f830159a700 1 -- 172.21.15.7:0/3396424686 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123260535 0 0) 0x7f8304159300 con 0x7f8304158280 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f830259c700 1 --2- 172.21.15.7:0/3396424686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040ce8f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f82f40098e0 tx=0x7f82f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5665c2710beb66e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.654+0000 7f830159a700 1 -- 172.21.15.7:0/3396424686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4009e80 con 0x7f83040cafb0 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f830159a700 1 -- 172.21.15.7:0/3396424686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f400b860 con 0x7f83040cafb0 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f830159a700 1 -- 172.21.15.7:0/3396424686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f400bbe0 con 0x7f83040cafb0 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f8308be0700 1 -- 172.21.15.7:0/3396424686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 msgr2=0x7f83040ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f8308be0700 1 --2- 172.21.15.7:0/3396424686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040ce8f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f82f40098e0 tx=0x7f82f40091a0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f8308be0700 1 -- 172.21.15.7:0/3396424686 shutdown_connections 2024-01-11T07:18:59.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f8308be0700 1 --2- 172.21.15.7:0/3396424686 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 0x7f8304161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f8308be0700 1 --2- 172.21.15.7:0/3396424686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040ce8f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f8308be0700 1 -- 172.21.15.7:0/3396424686 >> 172.21.15.7:0/3396424686 conn(0x7f8304071100 msgr2=0x7f830406edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:59.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f8308be0700 1 -- 172.21.15.7:0/3396424686 shutdown_connections 2024-01-11T07:18:59.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.655+0000 7f8308be0700 1 -- 172.21.15.7:0/3396424686 wait complete. 2024-01-11T07:18:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.656+0000 7f8308be0700 1 Processor -- start 2024-01-11T07:18:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8308be0700 1 -- start start 2024-01-11T07:18:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8308be0700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 0x7f830414a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8308be0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8308be0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304158280 0x7f83040cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8308be0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83040ca7b0 con 0x7f83040cafb0 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8308be0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830414aa80 con 0x7f8304158280 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8308be0700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83040c9ba0 con 0x7f83040b0b40 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8301d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8301d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52858/0 (socket says 172.21.15.7:52858) 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.657+0000 7f8301d9b700 1 -- 172.21.15.7:0/4052209956 learned_addr learned my addr 172.21.15.7:0/4052209956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f830259c700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 0x7f830414a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8302d9d700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304158280 0x7f83040cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8301d9b700 1 -- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 msgr2=0x7f830414a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8301d9b700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 0x7f830414a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8301d9b700 1 -- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304158280 msgr2=0x7f83040cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8301d9b700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304158280 0x7f83040cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8301d9b700 1 -- 172.21.15.7:0/4052209956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4009420 con 0x7f83040cafb0 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8301d9b700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040cce80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c5e0 tx=0x7f82f8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f82f27fc700 1 -- 172.21.15.7:0/4052209956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8005b30 con 0x7f83040cafb0 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f82f27fc700 1 -- 172.21.15.7:0/4052209956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f8005c90 con 0x7f83040cafb0 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83040c9e80 con 0x7f83040cafb0 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f82f27fc700 1 -- 172.21.15.7:0/4052209956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8007300 con 0x7f83040cafb0 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.658+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83040ca440 con 0x7f83040cafb0 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.659+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8304067610 con 0x7f83040cafb0 2024-01-11T07:18:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.660+0000 7f82f27fc700 1 -- 172.21.15.7:0/4052209956 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f82f80076b0 con 0x7f83040cafb0 2024-01-11T07:18:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.660+0000 7f82f27fc700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82e4066f30 0x7f82e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:18:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.660+0000 7f82f27fc700 1 -- 172.21.15.7:0/4052209956 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f82f8083770 con 0x7f83040cafb0 2024-01-11T07:18:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.660+0000 7f830259c700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82e4066f30 0x7f82e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:18:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.661+0000 7f830259c700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82e4066f30 0x7f82e40693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f82f4003080 tx=0x7f82f4017000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:18:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.665+0000 7f82f27fc700 1 -- 172.21.15.7:0/4052209956 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8304067610 con 0x7f83040cafb0 2024-01-11T07:18:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.869+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f8304067610 con 0x7f82e4066f30 2024-01-11T07:18:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.869+0000 7f82f27fc700 1 -- 172.21.15.7:0/4052209956 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8304067610 con 0x7f82e4066f30 2024-01-11T07:18:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82e4066f30 msgr2=0x7f82e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82e4066f30 0x7f82e40693e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f82f4003080 tx=0x7f82f4017000 comp rx=0 tx=0).stop 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 msgr2=0x7f83040cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040cce80 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c5e0 tx=0x7f82f8000fe0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 shutdown_connections 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f82e4066f30 0x7f82e40693e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8304158280 0x7f83040cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83040b0b40 0x7f830414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 --2- 172.21.15.7:0/4052209956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83040cafb0 0x7f83040cce80 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 >> 172.21.15.7:0/4052209956 conn(0x7f8304071100 msgr2=0x7f83040b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 shutdown_connections 2024-01-11T07:18:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:18:59.871+0000 7f8308be0700 1 -- 172.21.15.7:0/4052209956 wait complete. 2024-01-11T07:18:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-01-11T07:18:59.885 INFO:tasks.ceph:Scrubbing osd.7 2024-01-11T07:18:59.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-11T07:19:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.022+0000 7f85a5aab700 1 Processor -- start 2024-01-11T07:19:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.022+0000 7f85a5aab700 1 -- start start 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f85a5aab700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a0143770 0x7f85a00b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f85a5aab700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85a00b61f0 0x7f85a00b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f85a5aab700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a00b5d60 con 0x7f85a00b39e0 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f85a5aab700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a00b5150 con 0x7f85a00b61f0 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f85a5aab700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00b5430 con 0x7f85a0143770 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f859ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85a00b61f0 0x7f85a00b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f859ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85a00b61f0 0x7f85a00b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52874/0 (socket says 172.21.15.7:52874) 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f859ffff700 1 -- 172.21.15.7:0/1537622641 learned_addr learned my addr 172.21.15.7:0/1537622641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.023+0000 7f859f7fe700 1 --2- 172.21.15.7:0/1537622641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a0143770 0x7f85a00b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859ffff700 1 -- 172.21.15.7:0/1537622641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a0143770 msgr2=0x7f85a00b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859ffff700 1 --2- 172.21.15.7:0/1537622641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a0143770 0x7f85a00b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859ffff700 1 -- 172.21.15.7:0/1537622641 >> v1:172.21.15.113:6789/0 conn(0x7f85a00b39e0 legacy=0x7f85a015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859ffff700 1 -- 172.21.15.7:0/1537622641 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00b5710 con 0x7f85a00b61f0 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859f7fe700 1 --2- 172.21.15.7:0/1537622641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a0143770 0x7f85a00b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859ffff700 1 --2- 172.21.15.7:0/1537622641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00b61f0 0x7f85a00b6670 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8598009f20 tx=0x7f8598009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3f131d250516b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859e7fc700 1 -- 172.21.15.7:0/1537622641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859800e070 con 0x7f85a00b61f0 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859e7fc700 1 -- 172.21.15.7:0/1537622641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859800ba80 con 0x7f85a00b61f0 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.024+0000 7f859e7fc700 1 -- 172.21.15.7:0/1537622641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85980092e0 con 0x7f85a00b61f0 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.025+0000 7f85a5aab700 1 -- 172.21.15.7:0/1537622641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00b61f0 msgr2=0x7f85a00b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.025+0000 7f85a5aab700 1 --2- 172.21.15.7:0/1537622641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00b61f0 0x7f85a00b6670 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8598009f20 tx=0x7f8598009840 comp rx=0 tx=0).stop 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.025+0000 7f85a5aab700 1 -- 172.21.15.7:0/1537622641 shutdown_connections 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.025+0000 7f85a5aab700 1 --2- 172.21.15.7:0/1537622641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a00b61f0 0x7f85a00b6670 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.025+0000 7f85a5aab700 1 --2- 172.21.15.7:0/1537622641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a0143770 0x7f85a00b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.025+0000 7f85a5aab700 1 -- 172.21.15.7:0/1537622641 >> 172.21.15.7:0/1537622641 conn(0x7f85a0063cc0 msgr2=0x7f85a00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.025+0000 7f85a5aab700 1 -- 172.21.15.7:0/1537622641 shutdown_connections 2024-01-11T07:19:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.025+0000 7f85a5aab700 1 -- 172.21.15.7:0/1537622641 wait complete. 2024-01-11T07:19:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.026+0000 7f85a5aab700 1 Processor -- start 2024-01-11T07:19:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.026+0000 7f85a5aab700 1 -- start start 2024-01-11T07:19:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f85a5aab700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a00b39e0 0x7f85a0159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f85a5aab700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a00b61f0 0x7f85a0169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f85a5aab700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a0143770 0x7f85a015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f85a5aab700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a016a780 con 0x7f85a0143770 2024-01-11T07:19:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f85a5aab700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a016aa60 con 0x7f85a00b61f0 2024-01-11T07:19:00.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f85a5aab700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a016ad40 con 0x7f85a00b39e0 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a0143770 0x7f85a015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a0143770 0x7f85a015c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52878/0 (socket says 172.21.15.7:52878) 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859ffff700 1 -- 172.21.15.7:0/1718774919 learned_addr learned my addr 172.21.15.7:0/1718774919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859f7fe700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a00b39e0 0x7f85a0159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859effd700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a00b61f0 0x7f85a0169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859ffff700 1 -- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a00b39e0 msgr2=0x7f85a0159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859ffff700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a00b39e0 0x7f85a0159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859ffff700 1 -- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a00b61f0 msgr2=0x7f85a0169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859ffff700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a00b61f0 0x7f85a0169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859ffff700 1 -- 172.21.15.7:0/1718774919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8598009ae0 con 0x7f85a0143770 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859f7fe700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f85a00b39e0 0x7f85a0159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.027+0000 7f859effd700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85a00b61f0 0x7f85a0169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.028+0000 7f859ffff700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a0143770 0x7f85a015c340 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f859800b6c0 tx=0x7f8598018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.028+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85980092e0 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.028+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598026460 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.028+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a016b020 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.028+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598026820 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.028+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a013f040 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.029+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85a00890f0 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.029+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f8598003040 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.030+0000 7f85877fe700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f858c066f30 0x7f858c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.030+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8598086370 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.030+0000 7f85877fe700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f858c06cd80 0x7f858c06f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.030+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 --> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f858c06f860 con 0x7f858c06cd80 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.030+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=191) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8598088870 con 0x7f85a0143770 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.030+0000 7f859f7fe700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f858c066f30 0x7f858c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.030+0000 7f859effd700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f858c06cd80 0x7f858c06f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.031+0000 7f859f7fe700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f858c066f30 0x7f858c0693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8590006ea0 tx=0x7f8590008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.031+0000 7f859effd700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f858c06cd80 0x7f858c06f190 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.031+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 <== osd.7 v2:172.21.15.113:6824/37240 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22488 (crc 0 0 0) 0x7f858c06f860 con 0x7f858c06cd80 2024-01-11T07:19:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.045+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 --> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f85a013f1d0 con 0x7f858c06cd80 2024-01-11T07:19:00.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85877fe700 1 -- 172.21.15.7:0/1718774919 <== osd.7 v2:172.21.15.113:6824/37240 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+682 (crc 0 0 0) 0x7f85a013f1d0 con 0x7f858c06cd80 2024-01-11T07:19:00.045 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-01-11T07:19:00.046 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2024-01-11T07:19:00.046 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-01-11T07:19:00.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f858c06cd80 msgr2=0x7f858c06f190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6824/37240,v1:172.21.15.113:6825/37240] conn(0x7f858c06cd80 0x7f858c06f190 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f858c066f30 msgr2=0x7f858c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f858c066f30 0x7f858c0693e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8590006ea0 tx=0x7f8590008040 comp rx=0 tx=0).stop 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a0143770 msgr2=0x7f85a015c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 --2- 172.21.15.7:0/1718774919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a0143770 0x7f85a015c340 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f859800b6c0 tx=0x7f8598018000 comp rx=0 tx=0).stop 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f859ffff700 1 -- 172.21.15.7:0/1718774919 reap_dead start 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 shutdown_connections 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 >> 172.21.15.7:0/1718774919 conn(0x7f85a0063cc0 msgr2=0x7f85a00d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 shutdown_connections 2024-01-11T07:19:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.046+0000 7f85a5aab700 1 -- 172.21.15.7:0/1718774919 wait complete. 2024-01-11T07:19:00.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-11T07:19:00.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.191+0000 7f96ce84d700 1 Processor -- start 2024-01-11T07:19:00.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.191+0000 7f96ce84d700 1 -- start start 2024-01-11T07:19:00.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.192+0000 7f96ce84d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96c8159c90 0x7f96c80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.192+0000 7f96ce84d700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f96c815a060 0x7f96c80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.192+0000 7f96ce84d700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c80ce9b0 con 0x7f96c8069320 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.192+0000 7f96ce84d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c80cf530 con 0x7f96c8159c90 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.192+0000 7f96ce84d700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c80cf9c0 con 0x7f96c815a060 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.192+0000 7f96c7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96c8159c90 0x7f96c80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.192+0000 7f96c7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96c8159c90 0x7f96c80b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52882/0 (socket says 172.21.15.7:52882) 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.192+0000 7f96c7fff700 1 -- 172.21.15.7:0/370424409 learned_addr learned my addr 172.21.15.7:0/370424409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c77fe700 1 --2- 172.21.15.7:0/370424409 >> v2:172.21.15.113:3300/0 conn(0x7f96c815a060 0x7f96c80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c7fff700 1 -- 172.21.15.7:0/370424409 >> v2:172.21.15.113:3300/0 conn(0x7f96c815a060 msgr2=0x7f96c80b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c7fff700 1 --2- 172.21.15.7:0/370424409 >> v2:172.21.15.113:3300/0 conn(0x7f96c815a060 0x7f96c80b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c7fff700 1 -- 172.21.15.7:0/370424409 >> v1:172.21.15.113:6789/0 conn(0x7f96c8069320 legacy=0x7f96c8067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c7fff700 1 -- 172.21.15.7:0/370424409 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c80cec00 con 0x7f96c8159c90 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c7fff700 1 --2- 172.21.15.7:0/370424409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8159c90 0x7f96c80b1820 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f96c80699e0 tx=0x7f96b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cce7bf13d41d5c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c6ffd700 1 -- 172.21.15.7:0/370424409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800b820 con 0x7f96c8159c90 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c6ffd700 1 -- 172.21.15.7:0/370424409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b800b980 con 0x7f96c8159c90 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.193+0000 7f96c6ffd700 1 -- 172.21.15.7:0/370424409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800bd20 con 0x7f96c8159c90 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.194+0000 7f96ce84d700 1 -- 172.21.15.7:0/370424409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8159c90 msgr2=0x7f96c80b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.194+0000 7f96ce84d700 1 --2- 172.21.15.7:0/370424409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8159c90 0x7f96c80b1820 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f96c80699e0 tx=0x7f96b8009250 comp rx=0 tx=0).stop 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.194+0000 7f96ce84d700 1 -- 172.21.15.7:0/370424409 shutdown_connections 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.194+0000 7f96ce84d700 1 --2- 172.21.15.7:0/370424409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8159c90 0x7f96c80b1820 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.194+0000 7f96ce84d700 1 --2- 172.21.15.7:0/370424409 >> v2:172.21.15.113:3300/0 conn(0x7f96c815a060 0x7f96c80b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.194+0000 7f96ce84d700 1 -- 172.21.15.7:0/370424409 >> 172.21.15.7:0/370424409 conn(0x7f96c8071100 msgr2=0x7f96c806edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.194+0000 7f96ce84d700 1 -- 172.21.15.7:0/370424409 shutdown_connections 2024-01-11T07:19:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.194+0000 7f96ce84d700 1 -- 172.21.15.7:0/370424409 wait complete. 2024-01-11T07:19:00.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.195+0000 7f96ce84d700 1 Processor -- start 2024-01-11T07:19:00.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.195+0000 7f96ce84d700 1 -- start start 2024-01-11T07:19:00.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ce84d700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c8069320 0x7f96c8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ce84d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c8159c90 0x7f96c816aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ce84d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c815a060 0x7f96c81584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ce84d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80c9550 con 0x7f96c815a060 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ce84d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80c90c0 con 0x7f96c8159c90 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ce84d700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81550f0 con 0x7f96c8069320 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ccdea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c815a060 0x7f96c81584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ccdea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c815a060 0x7f96c81584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52894/0 (socket says 172.21.15.7:52894) 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ccdea700 1 -- 172.21.15.7:0/2481942296 learned_addr learned my addr 172.21.15.7:0/2481942296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96c7fff700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c8069320 0x7f96c8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96c77fe700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c8159c90 0x7f96c816aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ccdea700 1 -- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c8069320 msgr2=0x7f96c8154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ccdea700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c8069320 0x7f96c8154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ccdea700 1 -- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c8159c90 msgr2=0x7f96c816aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ccdea700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c8159c90 0x7f96c816aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96ccdea700 1 -- 172.21.15.7:0/2481942296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80094f0 con 0x7f96c815a060 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.196+0000 7f96c77fe700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c8159c90 0x7f96c816aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.197+0000 7f96ccdea700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c815a060 0x7f96c81584d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c960 tx=0x7f96c00026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.197+0000 7f96affff700 1 -- 172.21.15.7:0/2481942296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000cea0 con 0x7f96c815a060 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.197+0000 7f96affff700 1 -- 172.21.15.7:0/2481942296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c000a7a0 con 0x7f96c815a060 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.197+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c80c84b0 con 0x7f96c815a060 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.197+0000 7f96affff700 1 -- 172.21.15.7:0/2481942296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000ab60 con 0x7f96c815a060 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.197+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c80c8a70 con 0x7f96c815a060 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.198+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c80c9250 con 0x7f96c815a060 2024-01-11T07:19:00.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.198+0000 7f96affff700 1 -- 172.21.15.7:0/2481942296 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f96c0012420 con 0x7f96c815a060 2024-01-11T07:19:00.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.199+0000 7f96affff700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f96b4066f30 0x7f96b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.199+0000 7f96affff700 1 -- 172.21.15.7:0/2481942296 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f96c0084650 con 0x7f96c815a060 2024-01-11T07:19:00.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.199+0000 7f96c7fff700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f96b4066f30 0x7f96b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.200+0000 7f96c7fff700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f96b4066f30 0x7f96b40693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f96c80699e0 tx=0x7f96b8016000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.203+0000 7f96affff700 1 -- 172.21.15.7:0/2481942296 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96c0051bc0 con 0x7f96c815a060 2024-01-11T07:19:00.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.405+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f96c804c3c0 con 0x7f96b4066f30 2024-01-11T07:19:00.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.405+0000 7f96affff700 1 -- 172.21.15.7:0/2481942296 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f96c804c3c0 con 0x7f96b4066f30 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.407+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f96b4066f30 msgr2=0x7f96b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.407+0000 7f96ce84d700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f96b4066f30 0x7f96b40693e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f96c80699e0 tx=0x7f96b8016000 comp rx=0 tx=0).stop 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.407+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c815a060 msgr2=0x7f96c81584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.407+0000 7f96ce84d700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c815a060 0x7f96c81584d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c960 tx=0x7f96c00026e0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.408+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 shutdown_connections 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.408+0000 7f96ce84d700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f96b4066f30 0x7f96b40693e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.408+0000 7f96ce84d700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c815a060 0x7f96c81584d0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.408+0000 7f96ce84d700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96c8069320 0x7f96c8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.408+0000 7f96ce84d700 1 --2- 172.21.15.7:0/2481942296 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96c8159c90 0x7f96c816aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.408+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 >> 172.21.15.7:0/2481942296 conn(0x7f96c8071100 msgr2=0x7f96c80cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:00.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.408+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 shutdown_connections 2024-01-11T07:19:00.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.408+0000 7f96ce84d700 1 -- 172.21.15.7:0/2481942296 wait complete. 2024-01-11T07:19:00.409 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-01-11T07:19:00.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:19:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.555+0000 7f6ae76ee700 1 Processor -- start 2024-01-11T07:19:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.555+0000 7f6ae76ee700 1 -- start start 2024-01-11T07:19:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.556+0000 7f6ae76ee700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ae015a060 0x7f6ae00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.556+0000 7f6ae76ee700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae00b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.556+0000 7f6ae76ee700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae00ce9b0 con 0x7f6ae0159c90 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.556+0000 7f6ae76ee700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae00b1f30 con 0x7f6ae015a060 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.556+0000 7f6ae76ee700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00cf530 con 0x7f6ae0069320 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae4c89700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ae015a060 0x7f6ae00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae5c8b700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae00b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae4c89700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ae015a060 0x7f6ae00b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:56972/0 (socket says 172.21.15.7:56972) 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae4c89700 1 -- 172.21.15.7:0/863203734 learned_addr learned my addr 172.21.15.7:0/863203734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:00.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae548a700 1 --1- 172.21.15.7:0/863203734 >> v1:172.21.15.113:6789/0 conn(0x7f6ae0159c90 0x7f6ae00b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:58162/0 (socket says 172.21.15.7:58162) 2024-01-11T07:19:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae4c89700 1 -- 172.21.15.7:0/863203734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 msgr2=0x7f6ae00b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae4c89700 1 --2- 172.21.15.7:0/863203734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae00b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae4c89700 1 -- 172.21.15.7:0/863203734 >> v1:172.21.15.113:6789/0 conn(0x7f6ae0159c90 legacy=0x7f6ae00b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae4c89700 1 -- 172.21.15.7:0/863203734 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae00cf9c0 con 0x7f6ae015a060 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ad7fff700 1 -- 172.21.15.7:0/863203734 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397091783 0 0) 0x7f6ae00ce9b0 con 0x7f6ae0159c90 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae5c8b700 1 --2- 172.21.15.7:0/863203734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae00b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.557+0000 7f6ae4c89700 1 --2- 172.21.15.7:0/863203734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae015a060 0x7f6ae00b0430 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0099b0 tx=0x7f6adc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571c52762c63d9b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ad7fff700 1 -- 172.21.15.7:0/863203734 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc009ea0 con 0x7f6ae015a060 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ad7fff700 1 -- 172.21.15.7:0/863203734 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00b860 con 0x7f6ae015a060 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ad7fff700 1 -- 172.21.15.7:0/863203734 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00bc00 con 0x7f6ae015a060 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ae76ee700 1 -- 172.21.15.7:0/863203734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae015a060 msgr2=0x7f6ae00b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/863203734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae015a060 0x7f6ae00b0430 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0099b0 tx=0x7f6adc009250 comp rx=0 tx=0).stop 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ae76ee700 1 -- 172.21.15.7:0/863203734 shutdown_connections 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/863203734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae00b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/863203734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae015a060 0x7f6ae00b0430 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ae76ee700 1 -- 172.21.15.7:0/863203734 >> 172.21.15.7:0/863203734 conn(0x7f6ae0071100 msgr2=0x7f6ae006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ae76ee700 1 -- 172.21.15.7:0/863203734 shutdown_connections 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.558+0000 7f6ae76ee700 1 -- 172.21.15.7:0/863203734 wait complete. 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.559+0000 7f6ae76ee700 1 Processor -- start 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.560+0000 7f6ae76ee700 1 -- start start 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.560+0000 7f6ae76ee700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae0148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.560+0000 7f6ae76ee700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae0159c90 0x7f6ae00af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae76ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae015a060 0x7f6ae015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae76ee700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00b08a0 con 0x7f6ae015a060 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae76ee700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae0148a60 con 0x7f6ae0159c90 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae76ee700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00afc90 con 0x7f6ae0069320 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae015a060 0x7f6ae015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae015a060 0x7f6ae015b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52896/0 (socket says 172.21.15.7:52896) 2024-01-11T07:19:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 -- 172.21.15.7:0/3706893008 learned_addr learned my addr 172.21.15.7:0/3706893008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae4c89700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae0159c90 0x7f6ae00af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae548a700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae0148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 -- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 msgr2=0x7f6ae0148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae0148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 -- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae0159c90 msgr2=0x7f6ae00af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae0159c90 0x7f6ae00af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 -- 172.21.15.7:0/3706893008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad8009a20 con 0x7f6ae015a060 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae5c8b700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae015a060 0x7f6ae015b7e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009ec0 tx=0x7f6ad800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ad57fa700 1 -- 172.21.15.7:0/3706893008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad800cb70 con 0x7f6ae015a060 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ad57fa700 1 -- 172.21.15.7:0/3706893008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad800f040 con 0x7f6ae015a060 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.561+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc0094f0 con 0x7f6ae015a060 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.562+0000 7f6ad57fa700 1 -- 172.21.15.7:0/3706893008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8014480 con 0x7f6ae015a060 2024-01-11T07:19:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.562+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae00b0250 con 0x7f6ae015a060 2024-01-11T07:19:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.563+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae0159520 con 0x7f6ae015a060 2024-01-11T07:19:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.563+0000 7f6ad57fa700 1 -- 172.21.15.7:0/3706893008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f6ad8003020 con 0x7f6ae015a060 2024-01-11T07:19:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.564+0000 7f6ad57fa700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac4066f30 0x7f6ac40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.564+0000 7f6ad57fa700 1 -- 172.21.15.7:0/3706893008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6ad8085390 con 0x7f6ae015a060 2024-01-11T07:19:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.564+0000 7f6ae548a700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac4066f30 0x7f6ac40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.565+0000 7f6ae548a700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac4066f30 0x7f6ac40693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007fd0 tx=0x7f6ad0007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:00.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.568+0000 7f6ad57fa700 1 -- 172.21.15.7:0/3706893008 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ad80528e0 con 0x7f6ae015a060 2024-01-11T07:19:00.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.765+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ae004c3c0 con 0x7f6ac4066f30 2024-01-11T07:19:00.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.768+0000 7f6ad57fa700 1 -- 172.21.15.7:0/3706893008 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50325 (secure 0 0 0) 0x7f6ae004c3c0 con 0x7f6ac4066f30 2024-01-11T07:19:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:19:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac4066f30 msgr2=0x7f6ac40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac4066f30 0x7f6ac40693e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007fd0 tx=0x7f6ad0007d50 comp rx=0 tx=0).stop 2024-01-11T07:19:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae015a060 msgr2=0x7f6ae015b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae015a060 0x7f6ae015b7e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009ec0 tx=0x7f6ad800b040 comp rx=0 tx=0).stop 2024-01-11T07:19:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 shutdown_connections 2024-01-11T07:19:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f6ac4066f30 0x7f6ac40693e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ae015a060 0x7f6ae015b7e0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ae0069320 0x7f6ae0148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 --2- 172.21.15.7:0/3706893008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ae0159c90 0x7f6ae00af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:00.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 >> 172.21.15.7:0/3706893008 conn(0x7f6ae0071100 msgr2=0x7f6ae0067610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:00.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 shutdown_connections 2024-01-11T07:19:00.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:00.770+0000 7f6ae76ee700 1 -- 172.21.15.7:0/3706893008 wait complete. 2024-01-11T07:19:00.772 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:19:00.783 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":548,"stamp":"2024-01-11T07:19:00.013489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9536,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552963,"statfs":{"total":1600000000,"available":1590234912,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52428800,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-32,"num_write_kb":-84992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.548294"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":316,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-01-11T07:18:50.457881+0000","last_change":"2024-01-11T07:06:51.932891+0000","last_active":"2024-01-11T07:18:50.457881+0000","last_peered":"2024-01-11T07:18:50.457881+0000","last_clean":"2024-01-11T07:18:50.457881+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:18:50.457881+0000","last_undegraded":"2024-01-11T07:18:50.457881+0000","last_fullsized":"2024-01-11T07:18:50.457881+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-11T07:06:49.907594+0000","last_clean_scrub_stamp":"2024-01-11T07:06:49.907594+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T17:59:27.181622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193688,"statfs":{"total":200000000,"available":198336588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55500000000000005},"min":{"1min":0.35799999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.40500000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.433,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.58899999999999997,"15min":0.56599999999999995},"min":{"1min":0.46700000000000003,"5min":0.41599999999999998,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.80500000000000005},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.77500000000000002,"15min":0.81399999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59799999999999998,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75800000000000001,"5min":0.78700000000000003,"15min":0.82399999999999995},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.42399999999999999,"15min":0.35099999999999998},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.82999999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.434,"5min":0.42799999999999999,"15min":0.40600000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.83799999999999997},"last":0.495}]},{"osd":4,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45600000000000002,"15min":0.47799999999999998},"min":{"1min":0.33500000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58899999999999997,"5min":0.77700000000000002,"15min":0.85999999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.34999999999999998,"5min":0.33800000000000002,"15min":0.308},"max":{"1min":0.59599999999999997,"5min":0.755,"15min":0.85899999999999999},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.314},"max":{"1min":0.66300000000000003,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.66600000000000004,"5min":0.84599999999999997,"15min":0.91600000000000004},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.82199999999999995,"15min":0.92500000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.81000000000000005,"15min":0.90300000000000002},"last":0.56100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.438,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.81399999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.56999999999999995},"min":{"1min":0.44600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77100000000000002,"5min":0.85199999999999998,"15min":1.157},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.45800000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65000000000000002,"5min":0.79500000000000004,"15min":2.5720000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.82999999999999996,"15min":0.85099999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.432,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":0.67300000000000004,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.34699999999999998},"max":{"1min":0.69899999999999995,"5min":0.81699999999999995,"15min":0.83699999999999997},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.29599999999999999},"max":{"1min":0.73699999999999999,"5min":0.84599999999999997,"15min":2.621},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.71799999999999997,"5min":0.85799999999999998,"15min":0.88500000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.48099999999999998,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.30299999999999999,"15min":0.25800000000000001},"max":{"1min":0.66300000000000003,"5min":0.76500000000000001,"15min":0.82699999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.253},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":2.0209999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.313},"max":{"1min":0.69299999999999995,"5min":0.77900000000000003,"15min":0.89100000000000001},"last":0.75}]},{"osd":7,"last update":"Thu Jan 11 07:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28699999999999998},"max":{"1min":0.64100000000000001,"5min":0.68600000000000005,"15min":0.76700000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53200000000000003,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30099999999999999},"max":{"1min":0.64800000000000002,"5min":0.70899999999999996,"15min":2.0750000000000002},"last":0.69499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39600000000000002,"15min":0.497},"min":{"1min":0.34200000000000003,"5min":0.247,"15min":0.216},"max":{"1min":0.53100000000000003,"5min":0.753,"15min":20.41},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.45200000000000001,"15min":0.46100000000000002},"min":{"1min":0.371,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":0.60699999999999998,"5min":0.72099999999999997,"15min":3.7029999999999998},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.44,"15min":0.497},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60199999999999998,"5min":3.597,"15min":15.611000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.51000000000000001,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.64100000000000001,"5min":3.0710000000000002,"15min":20.199999999999999},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48299999999999998,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.311,"15min":0.29999999999999999},"max":{"1min":0.67900000000000005,"5min":3.8460000000000001,"15min":20.369},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47199999999999998,"15min":0.46999999999999997},"min":{"1min":0.33100000000000002,"5min":0.30399999999999999,"15min":0.28299999999999997},"max":{"1min":0.499,"5min":4.7569999999999997,"15min":8.5790000000000006},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":1.3300000000000001,"15min":9.9570000000000007},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":1.2030000000000001,"15min":12.449999999999999},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.63800000000000001},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.376},"max":{"1min":0.82799999999999996,"5min":1.3540000000000001,"15min":2.6030000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.51300000000000001,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.90900000000000003,"5min":1.177,"15min":9.891},"last":0.747}]},{"osd":6,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66200000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":1.389,"15min":1.5569999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69199999999999995,"15min":0.72899999999999998},"min":{"1min":0.53100000000000003,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.94699999999999995,"5min":2.9049999999999998,"15min":9.9900000000000002},"last":0.79000000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69299999999999995,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.47499999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.1599999999999999,"15min":20.324000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71799999999999997,"15min":0.71499999999999997},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.46800000000000003},"max":{"1min":0.96999999999999997,"5min":1.268,"15min":2.548},"last":0.70999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193688,"statfs":{"total":200000000,"available":198336588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39600000000000002,"15min":0.42999999999999999},"min":{"1min":0.27100000000000002,"5min":0.23300000000000001,"15min":0.224},"max":{"1min":0.57899999999999996,"5min":0.67700000000000005,"15min":2.7429999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.41199999999999998},"min":{"1min":0.29299999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57399999999999995,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.23999999999999999},"max":{"1min":0.623,"5min":1.2050000000000001,"15min":7.7960000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.48799999999999999},"min":{"1min":0.36499999999999999,"5min":0.29299999999999998,"15min":0.27300000000000002},"max":{"1min":0.61699999999999999,"5min":0.83699999999999997,"15min":2.5880000000000001},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47599999999999998,"15min":0.57799999999999996},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.58899999999999997,"5min":5.0330000000000004,"15min":8.0730000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.52900000000000003},"min":{"1min":0.374,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64000000000000001,"5min":0.83499999999999996,"15min":2.855},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.495,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90900000000000003,"15min":1.3959999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.69899999999999995,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.40999999999999998},"max":{"1min":0.92500000000000004,"5min":4.9279999999999999,"15min":8.0399999999999991},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67100000000000004,"15min":0.68700000000000006},"min":{"1min":0.54900000000000004,"5min":0.438,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":1.032,"15min":2.819},"last":0.629},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.54800000000000004,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":0.95399999999999996,"5min":0.96599999999999997,"15min":1.2490000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71999999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95899999999999996,"5min":4.9850000000000003,"15min":7.9539999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.502,"5min":0.501,"15min":0.41799999999999998},"max":{"1min":0.94999999999999996,"5min":1.0840000000000001,"15min":7.4690000000000003},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.70099999999999996,"15min":0.68000000000000005},"min":{"1min":0.52900000000000003,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.276},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.747,"15min":0.82299999999999995},"min":{"1min":0.57199999999999995,"5min":0.47499999999999998,"15min":0.44900000000000001},"max":{"1min":0.97299999999999998,"5min":5.0720000000000001,"15min":8.0030000000000001},"last":0.64900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38,"15min":0.40400000000000003},"min":{"1min":0.28399999999999997,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.504,"5min":0.72999999999999998,"15min":1.472},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.311,"5min":0.27900000000000003,"15min":0.23599999999999999},"max":{"1min":0.57799999999999996,"5min":0.65500000000000003,"15min":0.86099999999999999},"last":0.25}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.58899999999999997,"5min":0.628,"15min":1.2430000000000001},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":0.57399999999999995,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48799999999999999},"min":{"1min":0.41399999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63300000000000001,"5min":0.752,"15min":0.76800000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.503},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.63100000000000001,"5min":0.76500000000000001,"15min":1.139},"last":0.376}]},{"osd":4,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.41899999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.0660000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64800000000000002,"15min":0.63500000000000001},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.40100000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.089},"last":0.45100000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48199999999999998,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":1.054,"15min":1.099},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":1.0389999999999999,"15min":1.3100000000000001},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.0109999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.50900000000000001,"5min":0.47199999999999998,"15min":0.45800000000000002},"max":{"1min":1.002,"5min":1.0289999999999999,"15min":1.052},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0580000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.68300000000000005},"min":{"1min":0.55400000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.505}]}]},{"osd":3,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.57399999999999995},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":0.58199999999999996,"5min":0.59699999999999998,"15min":11.481},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.46800000000000003,"15min":0.47299999999999998},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.31},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":1.6200000000000001},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40699999999999997,"15min":0.66300000000000003},"min":{"1min":0.30399999999999999,"5min":0.253,"15min":0.23899999999999999},"max":{"1min":0.59899999999999998,"5min":0.70499999999999996,"15min":17.516999999999999},"last":0.315},{"interface":"front","average":{"1min":0.432,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.29699999999999999,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.67400000000000004},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":0.56000000000000005,"5min":0.746,"15min":0.76500000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46400000000000002,"15min":0.72299999999999998},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.64700000000000002,"5min":0.64900000000000002,"15min":17.553000000000001},"last":0.379}]},{"osd":4,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.40300000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.94599999999999995},"last":0.621},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69299999999999995,"15min":0.95099999999999996},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.36199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":17.474},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64600000000000002,"15min":0.89600000000000002},"min":{"1min":0.49199999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":17.393000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.81899999999999995},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.379},"max":{"1min":0.89100000000000001,"5min":0.94499999999999995,"15min":11.394},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66000000000000003,"15min":0.82899999999999996},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.91900000000000004,"15min":11.436999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"min":{"1min":0.502,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.878,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jan 11 07:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.40400000000000003},"max":{"1min":0.89900000000000002,"5min":0.96999999999999997,"15min":1.0449999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.93799999999999994},"min":{"1min":0.53700000000000003,"5min":0.45800000000000002,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":0.95499999999999996,"15min":17.434000000000001},"last":0.57499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.81299999999999994},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.73599999999999999,"5min":0.94799999999999995,"15min":8.484},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61399999999999999,"15min":0.84599999999999997},"min":{"1min":0.42199999999999999,"5min":0.40999999999999998,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":0.84699999999999998,"15min":9.2599999999999998},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56899999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36499999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":1.1599999999999999,"15min":8.5340000000000007},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.78100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.81000000000000005,"15min":9.2279999999999998},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59999999999999998,"15min":0.78200000000000003},"min":{"1min":0.376,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.71999999999999997,"5min":1.1120000000000001,"15min":9.5640000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.71299999999999997},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.88600000000000001,"15min":6.8639999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56100000000000005,"15min":0.78500000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.34699999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":8.9779999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60099999999999998,"15min":0.84699999999999998},"min":{"1min":0.50800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":1.218,"15min":8.7309999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45300000000000001,"15min":0.67700000000000005},"min":{"1min":0.318,"5min":0.313,"15min":0.23999999999999999},"max":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":11.388999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46899999999999997,"15min":0.65500000000000003},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.254},"max":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":8.6509999999999998},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.69399999999999995},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":9.1449999999999996},"last":0.625},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73699999999999999,"5min":0.81899999999999995,"15min":1.119},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52200000000000002,"15min":0.69499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.33500000000000002},"max":{"1min":0.70599999999999996,"5min":0.85399999999999998,"15min":8.6050000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":0.77800000000000002},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":9.4640000000000004},"last":0.68999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59499999999999997,"15min":0.57699999999999996},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.49099999999999999,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.81899999999999995},"last":0.623}]},{"osd":1,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.57699999999999996},"min":{"1min":0.36799999999999999,"5min":0.32700000000000001,"15min":0.24399999999999999},"max":{"1min":0.73999999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.47199999999999998,"5min":0.40300000000000002,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.504,"5min":0.36499999999999999,"15min":0.35399999999999998},"max":{"1min":0.72399999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60999999999999999,"15min":0.59699999999999998},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.93999999999999995},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.41599999999999998,"5min":0.39900000000000002,"15min":0.34999999999999998},"max":{"1min":0.70399999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.628,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.36499999999999999},"max":{"1min":0.79600000000000004,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47299999999999998,"15min":0.495},"min":{"1min":0.35599999999999998,"5min":0.314,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.74199999999999999,"15min":0.76000000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.496,"15min":0.50800000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.50700000000000001,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.313},"max":{"1min":0.67600000000000005,"5min":0.73799999999999999,"15min":0.78600000000000003},"last":0.753},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.56100000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.79300000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.0109999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.77700000000000002,"5min":0.78400000000000003,"15min":0.88200000000000001},"last":0.78500000000000003}]}]}],"pool_statfs":[]}} 2024-01-11T07:19:00.787 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-01-11T07:06:49.907594+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=11, tm_hour=7, tm_min=6, tm_sec=49, tm_wday=3, tm_yday=11, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=11, tm_hour=7, tm_min=18, tm_sec=55, tm_wday=3, tm_yday=11, tm_isdst=0) 2024-01-11T07:19:00.787 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-11T07:19:02.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:19:02.154 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:19:20.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-11T07:19:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.929+0000 7fcd45a18700 1 Processor -- start 2024-01-11T07:19:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.929+0000 7fcd45a18700 1 -- start start 2024-01-11T07:19:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd45a18700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd40159ca0 0x7fcd4006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd45a18700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcd400cef00 0x7fcd4006e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd45a18700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd400d18c0 con 0x7fcd4015a070 2024-01-11T07:19:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd45a18700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400d0010 con 0x7fcd40159ca0 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd45a18700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400cf400 con 0x7fcd400cef00 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd3effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd40159ca0 0x7fcd4006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd3effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd40159ca0 0x7fcd4006dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50790/0 (socket says 172.21.15.7:50790) 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd3effd700 1 -- 172.21.15.7:0/787341268 learned_addr learned my addr 172.21.15.7:0/787341268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd3f7fe700 1 --2- 172.21.15.7:0/787341268 >> v2:172.21.15.113:3300/0 conn(0x7fcd400cef00 0x7fcd4006e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd3effd700 1 -- 172.21.15.7:0/787341268 >> v2:172.21.15.113:3300/0 conn(0x7fcd400cef00 msgr2=0x7fcd4006e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd3effd700 1 --2- 172.21.15.7:0/787341268 >> v2:172.21.15.113:3300/0 conn(0x7fcd400cef00 0x7fcd4006e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.930+0000 7fcd3effd700 1 -- 172.21.15.7:0/787341268 >> v1:172.21.15.113:6789/0 conn(0x7fcd4015a070 legacy=0x7fcd400ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd3effd700 1 -- 172.21.15.7:0/787341268 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400cf6e0 con 0x7fcd40159ca0 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd3effd700 1 --2- 172.21.15.7:0/787341268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd40159ca0 0x7fcd4006dc30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fcd40089e10 tx=0x7fcd30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeadc08f67c6def1 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd3e7fc700 1 -- 172.21.15.7:0/787341268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30009e70 con 0x7fcd40159ca0 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd3e7fc700 1 -- 172.21.15.7:0/787341268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3000b860 con 0x7fcd40159ca0 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd3e7fc700 1 -- 172.21.15.7:0/787341268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000bc00 con 0x7fcd40159ca0 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd45a18700 1 -- 172.21.15.7:0/787341268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd40159ca0 msgr2=0x7fcd4006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd45a18700 1 --2- 172.21.15.7:0/787341268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd40159ca0 0x7fcd4006dc30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fcd40089e10 tx=0x7fcd30009250 comp rx=0 tx=0).stop 2024-01-11T07:19:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd45a18700 1 -- 172.21.15.7:0/787341268 shutdown_connections 2024-01-11T07:19:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd45a18700 1 --2- 172.21.15.7:0/787341268 >> v2:172.21.15.113:3300/0 conn(0x7fcd400cef00 0x7fcd4006e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.931+0000 7fcd45a18700 1 --2- 172.21.15.7:0/787341268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd40159ca0 0x7fcd4006dc30 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.932+0000 7fcd45a18700 1 -- 172.21.15.7:0/787341268 >> 172.21.15.7:0/787341268 conn(0x7fcd40063cc0 msgr2=0x7fcd400d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.932+0000 7fcd45a18700 1 -- 172.21.15.7:0/787341268 shutdown_connections 2024-01-11T07:19:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.932+0000 7fcd45a18700 1 -- 172.21.15.7:0/787341268 wait complete. 2024-01-11T07:19:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.933+0000 7fcd45a18700 1 Processor -- start 2024-01-11T07:19:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.933+0000 7fcd45a18700 1 -- start start 2024-01-11T07:19:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd45a18700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd400cef00 0x7fcd4014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd45a18700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd40159ca0 0x7fcd4015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd45a18700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd4015a070 0x7fcd4015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd45a18700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40150330 con 0x7fcd400cef00 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd45a18700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd401508b0 con 0x7fcd40159ca0 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd45a18700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400ca630 con 0x7fcd4015a070 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd400cef00 0x7fcd4014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd400cef00 0x7fcd4014f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50798/0 (socket says 172.21.15.7:50798) 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 -- 172.21.15.7:0/2042158332 learned_addr learned my addr 172.21.15.7:0/2042158332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3f7fe700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd4015a070 0x7fcd4015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd367fc700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd40159ca0 0x7fcd4015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 -- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd4015a070 msgr2=0x7fcd4015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd4015a070 0x7fcd4015bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 -- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd40159ca0 msgr2=0x7fcd4015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd40159ca0 0x7fcd4015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 -- 172.21.15.7:0/2042158332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300094f0 con 0x7fcd400cef00 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd367fc700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd40159ca0 0x7fcd4015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3f7fe700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd4015a070 0x7fcd4015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.934+0000 7fcd3effd700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd400cef00 0x7fcd4014f580 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fcd40068ad0 tx=0x7fcd30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.935+0000 7fcd377fe700 1 -- 172.21.15.7:0/2042158332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30011420 con 0x7fcd400cef00 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.935+0000 7fcd377fe700 1 -- 172.21.15.7:0/2042158332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd300115a0 con 0x7fcd400cef00 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.935+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd400ca910 con 0x7fcd400cef00 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.935+0000 7fcd377fe700 1 -- 172.21.15.7:0/2042158332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30011960 con 0x7fcd400cef00 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.935+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4014fac0 con 0x7fcd400cef00 2024-01-11T07:19:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.936+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd400caad0 con 0x7fcd400cef00 2024-01-11T07:19:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.936+0000 7fcd377fe700 1 -- 172.21.15.7:0/2042158332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7fcd300253f0 con 0x7fcd400cef00 2024-01-11T07:19:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.936+0000 7fcd377fe700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd20066f30 0x7fcd200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.937+0000 7fcd377fe700 1 -- 172.21.15.7:0/2042158332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fcd30085d10 con 0x7fcd400cef00 2024-01-11T07:19:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.937+0000 7fcd367fc700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd20066f30 0x7fcd200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.937+0000 7fcd367fc700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd20066f30 0x7fcd200693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcd400ccea0 tx=0x7fcd28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:20.941+0000 7fcd377fe700 1 -- 172.21.15.7:0/2042158332 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd3008b020 con 0x7fcd400cef00 2024-01-11T07:19:21.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.140+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 --> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd401509f0 con 0x7fcd20066f30 2024-01-11T07:19:21.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.143+0000 7fcd377fe700 1 -- 172.21.15.7:0/2042158332 <== mgr.4105 v2:172.21.15.113:6832/36527 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50331 (secure 0 0 0) 0x7fcd28014420 con 0x7fcd20066f30 2024-01-11T07:19:21.144 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:19:21.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd20066f30 msgr2=0x7fcd200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd20066f30 0x7fcd200693e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcd400ccea0 tx=0x7fcd28009040 comp rx=0 tx=0).stop 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd400cef00 msgr2=0x7fcd4014f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd400cef00 0x7fcd4014f580 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fcd40068ad0 tx=0x7fcd30017000 comp rx=0 tx=0).stop 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 shutdown_connections 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7fcd20066f30 0x7fcd200693e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcd4015a070 0x7fcd4015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd400cef00 0x7fcd4014f580 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 --2- 172.21.15.7:0/2042158332 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcd40159ca0 0x7fcd4015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 >> 172.21.15.7:0/2042158332 conn(0x7fcd40063cc0 msgr2=0x7fcd400cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 shutdown_connections 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.145+0000 7fcd45a18700 1 -- 172.21.15.7:0/2042158332 wait complete. 2024-01-11T07:19:21.147 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-01-11T07:19:21.160 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":558,"stamp":"2024-01-11T07:19:20.018278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9536,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552963,"statfs":{"total":1600000000,"available":1590234912,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002775"},"pg_stats":[{"pgid":"1.0","version":"191'119","reported_seq":329,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-01-11T07:19:01.111700+0000","last_change":"2024-01-11T07:19:01.111674+0000","last_active":"2024-01-11T07:19:01.111700+0000","last_peered":"2024-01-11T07:19:01.111700+0000","last_clean":"2024-01-11T07:19:01.111700+0000","last_became_active":"2024-01-11T07:06:51.932040+0000","last_became_peered":"2024-01-11T07:06:51.932040+0000","last_unstale":"2024-01-11T07:19:01.111700+0000","last_undegraded":"2024-01-11T07:19:01.111700+0000","last_fullsized":"2024-01-11T07:19:01.111700+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'119","last_scrub_stamp":"2024-01-11T07:19:01.111618+0000","last_deep_scrub":"191'119","last_deep_scrub_stamp":"2024-01-11T07:19:01.111618+0000","last_clean_scrub_stamp":"2024-01-11T07:19:01.111618+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-12T08:49:59.868977+0000","scrub_duration":0.0025573359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193688,"statfs":{"total":200000000,"available":198336588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.32800000000000001,"5min":0.32500000000000001,"15min":0.32400000000000001},"max":{"1min":0.67500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.57099999999999995,"15min":0.55400000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74299999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.437}]},{"osd":1,"last update":"Thu Jan 11 07:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.60899999999999999,"15min":0.56599999999999995},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58499999999999996,"15min":0.56799999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66600000000000004,"5min":0.78200000000000003,"15min":0.80500000000000005},"last":0.51500000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":0.81399999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":0.78700000000000003,"15min":0.82399999999999995},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 11 07:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.35099999999999998},"max":{"1min":0.69299999999999995,"5min":0.79800000000000004,"15min":0.82999999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.65100000000000002,"15min":0.60699999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.40600000000000003},"max":{"1min":4.9589999999999996,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 11 07:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46000000000000002,"15min":0.47899999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29499999999999998},"max":{"1min":0.72199999999999998,"5min":0.77700000000000002,"15min":0.85999999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47599999999999998,"15min":0.48999999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.308},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.85899999999999999},"last":0.371}]},{"osd":5,"last update":"Thu Jan 11 07:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48899999999999999,"15min":0.51700000000000002},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.314},"max":{"1min":0.78400000000000003,"5min":0.81799999999999995,"15min":0.83999999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54200000000000004,"15min":0.54900000000000004},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.79700000000000004,"5min":0.84599999999999997,"15min":0.91600000000000004},"last":0.39000000000000001}]},{"osd":6,"last update":"Thu Jan 11 07:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.33700000000000002},"max":{"1min":0.80100000000000005,"5min":0.82199999999999995,"15min":0.92500000000000004},"last":0.623},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57399999999999995,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.90300000000000002},"last":0.496}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.27100000000000002},"max":{"1min":0.70799999999999996,"5min":0.85199999999999998,"15min":1.157},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.28699999999999998},"max":{"1min":0.77300000000000002,"5min":0.77400000000000002,"15min":2.5720000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.41599999999999998,"5min":0.38100000000000001,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":0.78800000000000003,"15min":0.85099999999999998},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Jan 11 07:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.29999999999999999},"max":{"1min":0.75700000000000001,"5min":0.78600000000000003,"15min":0.80500000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59199999999999997,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34699999999999998},"max":{"1min":0.746,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 11 07:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40200000000000002,"5min":0.38400000000000001,"15min":0.29599999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":2.621},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60999999999999999,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.88500000000000001},"last":0.621}]},{"osd":4,"last update":"Thu Jan 11 07:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.33100000000000002,"5min":0.30299999999999999,"15min":0.25800000000000001},"max":{"1min":0.68500000000000005,"5min":0.76500000000000001,"15min":0.82699999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.78800000000000003},"last":0.40300000000000002}]},{"osd":5,"last update":"Thu Jan 11 07:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.503,"15min":0.50900000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.253},"max":{"1min":0.71199999999999997,"5min":0.72299999999999998,"15min":2.0209999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.313},"max":{"1min":0.75,"5min":0.77900000000000003,"15min":0.89100000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Thu Jan 11 07:19:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.504,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.28699999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.76700000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53100000000000003,"15min":0.55600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.30099999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":2.0750000000000002},"last":0.47399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.496},"min":{"1min":0.28899999999999998,"5min":0.247,"15min":0.216},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":20.41},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.46100000000000002},"min":{"1min":0.34899999999999998,"5min":0.29799999999999999,"15min":0.28100000000000003},"max":{"1min":0.63200000000000001,"5min":0.64600000000000002,"15min":3.7029999999999998},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 11 07:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40899999999999997,"15min":0.495},"min":{"1min":0.28299999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57699999999999996,"5min":0.60299999999999998,"15min":15.611000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48699999999999999,"15min":0.57599999999999996},"min":{"1min":0.35499999999999998,"5min":0.32800000000000001,"15min":0.26200000000000001},"max":{"1min":0.63700000000000001,"5min":0.67900000000000005,"15min":20.199999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 11 07:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45200000000000001,"15min":0.55300000000000005},"min":{"1min":0.34599999999999997,"5min":0.33300000000000002,"15min":0.29999999999999999},"max":{"1min":0.56599999999999995,"5min":0.67900000000000005,"15min":20.369},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.433,"15min":0.46899999999999997},"min":{"1min":0.317,"5min":0.30399999999999999,"15min":0.28299999999999997},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":8.5790000000000006},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.66700000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.317},"max":{"1min":0.73199999999999998,"5min":0.86499999999999999,"15min":9.9570000000000007},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65400000000000003,"15min":0.69599999999999995},"min":{"1min":0.49399999999999999,"5min":0.47199999999999998,"15min":0.41899999999999998},"max":{"1min":0.83499999999999996,"5min":0.92000000000000004,"15min":12.449999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.47299999999999998,"5min":0.45700000000000002,"15min":0.376},"max":{"1min":0.79300000000000004,"5min":0.995,"15min":2.6030000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.69999999999999996},"min":{"1min":0.48999999999999999,"5min":0.46300000000000002,"15min":0.38100000000000001},"max":{"1min":0.81499999999999995,"5min":0.90900000000000003,"15min":9.891},"last":0.84799999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.66000000000000003,"15min":0.65900000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.41899999999999998},"max":{"1min":0.745,"5min":0.94299999999999995,"15min":1.5569999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.66000000000000003,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.45900000000000002,"15min":0.41699999999999998},"max":{"1min":0.79000000000000004,"5min":0.94699999999999995,"15min":9.9900000000000002},"last":0.79900000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:19:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68300000000000005,"15min":0.77200000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.442},"max":{"1min":0.88,"5min":1.0149999999999999,"15min":20.324000000000002},"last":0.746},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82499999999999996,"5min":0.96999999999999997,"15min":2.548},"last":0.81100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1624,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193688,"statfs":{"total":200000000,"available":198336588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 11 07:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39600000000000002,"15min":0.42399999999999999},"min":{"1min":0.28699999999999998,"5min":0.23300000000000001,"15min":0.224},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":2.7429999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.441,"5min":0.435,"15min":0.41499999999999998},"min":{"1min":0.32700000000000001,"5min":0.26800000000000002,"15min":0.23499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":1.4219999999999999},"last":0.37}]},{"osd":2,"last update":"Thu Jan 11 07:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45500000000000002,"15min":0.54400000000000004},"min":{"1min":0.29999999999999999,"5min":0.29099999999999998,"15min":0.23999999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":7.7960000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45800000000000002,"15min":0.48099999999999998},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.27300000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":2.5880000000000001},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 11 07:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.438,"15min":0.55100000000000005},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":8.0730000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.52000000000000002},"min":{"1min":0.37,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.752,"5min":0.752,"15min":2.855},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 11 07:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65700000000000003,"15min":0.629},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.39800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.3959999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.755},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.40999999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":8.0399999999999991},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 11 07:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67400000000000004,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.39400000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":2.819},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69999999999999996,"15min":0.66800000000000004},"min":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.41599999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.2490000000000001},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 11 07:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.79100000000000004},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.39300000000000002},"max":{"1min":1.649,"5min":1.649,"15min":7.9539999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.41799999999999998},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":7.4690000000000003},"last":0.752}]},{"osd":7,"last update":"Thu Jan 11 07:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70999999999999996,"15min":0.68200000000000005},"min":{"1min":0.51100000000000001,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.276},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71199999999999997,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.47499999999999998,"15min":0.44900000000000001},"max":{"1min":1.595,"5min":1.595,"15min":8.0030000000000001},"last":0.68100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.40100000000000002},"min":{"1min":0.27700000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57599999999999996,"5min":0.72999999999999998,"15min":1.472},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40500000000000003,"15min":0.39600000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.23599999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.86099999999999999},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jan 11 07:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.41899999999999998,"15min":0.42899999999999999},"min":{"1min":0.29899999999999999,"5min":0.27800000000000002,"15min":0.24199999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":1.2430000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41699999999999998,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":0.57099999999999995,"5min":0.60799999999999998,"15min":0.60899999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.30299999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70099999999999996,"5min":0.752,"15min":0.76800000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.503},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.139},"last":0.502}]},{"osd":4,"last update":"Thu Jan 11 07:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.627,"15min":0.64200000000000002},"min":{"1min":0.45700000000000002,"5min":0.41899999999999998,"15min":0.39200000000000002},"max":{"1min":0.72799999999999998,"5min":1.02,"15min":1.0660000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.40100000000000002},"max":{"1min":0.83599999999999997,"5min":1.0429999999999999,"15min":1.089},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jan 11 07:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.51900000000000002,"5min":0.434,"15min":0.41999999999999998},"max":{"1min":0.92000000000000004,"5min":1.054,"15min":1.099},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.52100000000000002,"5min":0.51300000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.0389999999999999,"15min":1.3100000000000001},"last":0.81499999999999995}]},{"osd":6,"last update":"Thu Jan 11 07:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67200000000000004,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93899999999999995,"5min":0.98099999999999998,"15min":1.0109999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.45800000000000002},"max":{"1min":0.86699999999999999,"5min":1.0289999999999999,"15min":1.052},"last":0.76300000000000001}]},{"osd":7,"last update":"Thu Jan 11 07:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.53200000000000003,"5min":0.47199999999999998,"15min":0.43099999999999999},"max":{"1min":0.88400000000000001,"5min":1.0109999999999999,"15min":1.0580000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.42999999999999999},"max":{"1min":0.86199999999999999,"5min":1.044,"15min":1.044},"last":0.82899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.53900000000000003},"min":{"1min":0.28999999999999998,"5min":0.249,"15min":0.22500000000000001},"max":{"1min":0.51400000000000001,"5min":0.59699999999999998,"15min":11.481},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.378,"5min":0.33400000000000002,"15min":0.31},"max":{"1min":0.59099999999999997,"5min":0.70499999999999996,"15min":1.6200000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 11 07:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.61099999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23899999999999999},"max":{"1min":0.60899999999999999,"5min":0.70499999999999996,"15min":17.516999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.437,"15min":0.42999999999999999},"min":{"1min":0.32300000000000001,"5min":0.27300000000000002,"15min":0.20200000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.67400000000000004},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 11 07:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.32100000000000001,"5min":0.308,"15min":0.28999999999999998},"max":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.76500000000000001},"last":0.375},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.47099999999999997,"15min":0.67300000000000004},"min":{"1min":0.379,"5min":0.34899999999999998,"15min":0.30599999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":17.553000000000001},"last":0.443}]},{"osd":4,"last update":"Thu Jan 11 07:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.55400000000000005,"5min":0.50600000000000001,"15min":0.40300000000000002},"max":{"1min":0.84899999999999998,"5min":0.93799999999999994,"15min":0.94599999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.89900000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.36199999999999999},"max":{"1min":0.88100000000000001,"5min":0.95499999999999996,"15min":17.474},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 11 07:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.84499999999999997},"min":{"1min":0.50800000000000001,"5min":0.40100000000000002,"15min":0.38900000000000001},"max":{"1min":0.79900000000000004,"5min":0.98399999999999999,"15min":17.393000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.78800000000000003},"min":{"1min":0.53600000000000003,"5min":0.47799999999999998,"15min":0.379},"max":{"1min":0.871,"5min":0.94499999999999995,"15min":11.394},"last":0.628}]},{"osd":6,"last update":"Thu Jan 11 07:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67000000000000004,"15min":0.79700000000000004},"min":{"1min":0.48999999999999999,"5min":0.44700000000000001,"15min":0.34699999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":11.436999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68300000000000005,"15min":0.67500000000000004},"min":{"1min":0.48799999999999999,"5min":0.46300000000000002,"15min":0.44400000000000001},"max":{"1min":0.89700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 11 07:19:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.51900000000000002,"5min":0.497,"15min":0.40400000000000003},"max":{"1min":0.89200000000000002,"5min":0.96999999999999997,"15min":1.0449999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70399999999999996,"15min":0.89100000000000001},"min":{"1min":0.57099999999999995,"5min":0.45800000000000002,"15min":0.42999999999999999},"max":{"1min":0.93899999999999995,"5min":0.95499999999999996,"15min":17.434000000000001},"last":0.65600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.77200000000000002},"min":{"1min":0.34599999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.751,"5min":0.94799999999999995,"15min":8.484},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61499999999999999,"15min":0.80000000000000004},"min":{"1min":0.42099999999999999,"5min":0.40999999999999998,"15min":0.34999999999999998},"max":{"1min":0.755,"5min":0.84699999999999998,"15min":9.2599999999999998},"last":0.68899999999999995}]},{"osd":1,"last update":"Thu Jan 11 07:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56299999999999994,"15min":0.68100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.30199999999999999},"max":{"1min":0.73799999999999999,"5min":1.1599999999999999,"15min":8.5340000000000007},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57799999999999996,"15min":0.74099999999999999},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72399999999999998,"5min":0.81000000000000005,"15min":9.2279999999999998},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 11 07:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58799999999999997,"15min":0.747},"min":{"1min":0.42199999999999999,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.76400000000000001,"5min":1.1120000000000001,"15min":9.5640000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57499999999999996,"15min":0.68400000000000005},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.33400000000000002},"max":{"1min":0.73999999999999999,"5min":0.88600000000000001,"15min":6.8639999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Thu Jan 11 07:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56100000000000005,"15min":0.74099999999999999},"min":{"1min":0.36199999999999999,"5min":0.35499999999999998,"15min":0.34699999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":8.9779999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59799999999999998,"15min":0.79900000000000004},"min":{"1min":0.42499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.755,"5min":1.218,"15min":8.7309999999999999},"last":0.621}]},{"osd":5,"last update":"Thu Jan 11 07:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44900000000000001,"15min":0.63200000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.23999999999999999},"max":{"1min":0.71799999999999997,"5min":0.77100000000000002,"15min":11.388999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46899999999999997,"15min":0.62},"min":{"1min":0.32800000000000001,"5min":0.30199999999999999,"15min":0.254},"max":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":8.6509999999999998},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 11 07:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48499999999999999,"15min":0.65500000000000003},"min":{"1min":0.32900000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68999999999999995,"5min":0.70499999999999996,"15min":9.1449999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.376,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":0.81899999999999995,"15min":1.119},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 11 07:19:13 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.66100000000000003},"min":{"1min":0.34200000000000003,"5min":0.34100000000000003,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.85399999999999998,"15min":8.6050000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58299999999999996,"15min":0.74099999999999999},"min":{"1min":0.40799999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72099999999999997,"5min":0.97799999999999998,"15min":9.4640000000000004},"last":0.77500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1047,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194264,"statfs":{"total":200000000,"available":198926956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 11 07:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.44400000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":1.1739999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.30599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.501}]},{"osd":1,"last update":"Thu Jan 11 07:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59799999999999998,"15min":0.57899999999999996},"min":{"1min":0.376,"5min":0.32700000000000001,"15min":0.24399999999999999},"max":{"1min":0.872,"5min":0.872,"15min":1.3129999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.59999999999999998},"min":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.32100000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 11 07:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.73399999999999999,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.61899999999999999,"15min":0.59999999999999998},"min":{"1min":0.45300000000000001,"5min":0.40500000000000003,"15min":0.29399999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.93999999999999995},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 11 07:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.43099999999999999,"5min":0.39900000000000002,"15min":0.34999999999999998},"max":{"1min":0.71699999999999997,"5min":0.78500000000000003,"15min":0.99099999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.36499999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":1.1919999999999999},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 11 07:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.28299999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32500000000000001},"max":{"1min":0.71299999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 11 07:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.51100000000000001,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.313},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54300000000000004,"15min":0.56100000000000005},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.32500000000000001},"max":{"1min":0.79300000000000004,"5min":0.79700000000000004,"15min":0.86399999999999999},"last":0.34799999999999998}]},{"osd":7,"last update":"Thu Jan 11 07:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80800000000000005,"5min":0.81499999999999995,"15min":1.0109999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.42299999999999999,"5min":0.38100000000000001,"15min":0.34000000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41399999999999998}]}]}],"pool_statfs":[]}} 2024-01-11T07:19:21.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-11T07:19:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.292+0000 7f1c14e77700 1 Processor -- start 2024-01-11T07:19:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.292+0000 7f1c14e77700 1 -- start start 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.293+0000 7f1c14e77700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c1015a280 0x7f1c100aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.293+0000 7f1c14e77700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1015d880 con 0x7f1c10063230 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.293+0000 7f1c14e77700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1006c5b0 con 0x7f1c100ac5b0 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.293+0000 7f1c14e77700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1015a7f0 con 0x7f1c1015a280 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.293+0000 7f1c0e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1c10063230 0x7f1c10063600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59394/0 (socket says 172.21.15.7:59394) 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.293+0000 7f1c0e59c700 1 -- 172.21.15.7:0/4110993422 learned_addr learned my addr 172.21.15.7:0/4110993422 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0dd9b700 1 --2- 172.21.15.7:0/4110993422 >> v2:172.21.15.113:3300/0 conn(0x7f1c1015a280 0x7f1c100aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521540764 0 0) 0x7f1c1015d880 con 0x7f1c10063230 2024-01-11T07:19:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bfc003570 con 0x7f1c10063230 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355706553 0 0) 0x7f1bfc003570 con 0x7f1c10063230 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 >> v2:172.21.15.113:3300/0 conn(0x7f1c1015a280 msgr2=0x7f1c100aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 --2- 172.21.15.7:0/4110993422 >> v2:172.21.15.113:3300/0 conn(0x7f1c1015a280 0x7f1c100aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 >> v1:172.21.15.113:6789/0 conn(0x7f1c100ac5b0 legacy=0x7f1c100ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1015aad0 con 0x7f1c10063230 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2260953232 0 0) 0x7f1c00001470 con 0x7f1c10063230 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 >> v1:172.21.15.7:6789/0 conn(0x7f1c10063230 legacy=0x7f1c10063600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bfc003a60 0x7f1bfc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc004810 0x7f1bfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0051c0 0x7f1bfc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1015a7f0 con 0x7f1bfc003a60 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c0d0 con 0x7f1bfc004810 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c290 con 0x7f1bfc0051c0 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0e59c700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bfc003a60 0x7f1bfc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0dd9b700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc004810 0x7f1bfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0ed9d700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0051c0 0x7f1bfc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.294+0000 7f1c0e59c700 1 -- 172.21.15.7:0/4110993422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0051c0 msgr2=0x7f1bfc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0e59c700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1bfc0051c0 0x7f1bfc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0e59c700 1 -- 172.21.15.7:0/4110993422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc004810 msgr2=0x7f1bfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0e59c700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc004810 0x7f1bfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0e59c700 1 -- 172.21.15.7:0/4110993422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc003570 con 0x7f1bfc003a60 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0ed9d700 1 -- 172.21.15.7:0/4110993422 reap_dead start 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0e59c700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bfc003a60 0x7f1bfc003e70 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c3c0 tx=0x7f1c00001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd28bd0b2686a91c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000cc90 con 0x7f1bfc003a60 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0000cdf0 con 0x7f1bfc003a60 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c0d59a700 1 -- 172.21.15.7:0/4110993422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00005b50 con 0x7f1bfc003a60 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c14e77700 1 -- 172.21.15.7:0/4110993422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bfc003a60 msgr2=0x7f1bfc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c14e77700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bfc003a60 0x7f1bfc003e70 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c3c0 tx=0x7f1c00001e50 comp rx=0 tx=0).stop 2024-01-11T07:19:21.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.295+0000 7f1c14e77700 1 -- 172.21.15.7:0/4110993422 shutdown_connections 2024-01-11T07:19:21.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.296+0000 7f1c14e77700 1 --2- 172.21.15.7:0/4110993422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bfc003a60 0x7f1bfc003e70 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.296+0000 7f1c14e77700 1 -- 172.21.15.7:0/4110993422 >> 172.21.15.7:0/4110993422 conn(0x7f1c100680e0 msgr2=0x7f1c100cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:21.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.296+0000 7f1c14e77700 1 -- 172.21.15.7:0/4110993422 shutdown_connections 2024-01-11T07:19:21.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.296+0000 7f1c14e77700 1 -- 172.21.15.7:0/4110993422 wait complete. 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.297+0000 7f1c14e77700 1 Processor -- start 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.297+0000 7f1c14e77700 1 -- start start 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.297+0000 7f1c14e77700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc003a60 0x7f1c100cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.297+0000 7f1c14e77700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c100cdae0 0x7f1c1015f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c14e77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015fc00 0x7f1c10153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c14e77700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1016ac60 con 0x7f1c1015fc00 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c14e77700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1016ae40 con 0x7f1bfc003a60 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c14e77700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1016a0e0 con 0x7f1c100cdae0 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015fc00 0x7f1c10153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015fc00 0x7f1c10153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50830/0 (socket says 172.21.15.7:50830) 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 -- 172.21.15.7:0/1943808216 learned_addr learned my addr 172.21.15.7:0/1943808216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0e59c700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc003a60 0x7f1c100cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0dd9b700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c100cdae0 0x7f1c1015f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 -- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c100cdae0 msgr2=0x7f1c1015f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c100cdae0 0x7f1c1015f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 -- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc003a60 msgr2=0x7f1c100cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc003a60 0x7f1c100cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 -- 172.21.15.7:0/1943808216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1016a3c0 con 0x7f1c1015fc00 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c0ed9d700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015fc00 0x7f1c10153950 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006d820 tx=0x7f1c08009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c067fc700 1 -- 172.21.15.7:0/1943808216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c080032e0 con 0x7f1c1015fc00 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c067fc700 1 -- 172.21.15.7:0/1943808216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c08003460 con 0x7f1c1015fc00 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1016a6a0 con 0x7f1c1015fc00 2024-01-11T07:19:21.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.298+0000 7f1c067fc700 1 -- 172.21.15.7:0/1943808216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c080017d0 con 0x7f1c1015fc00 2024-01-11T07:19:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.299+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c10169c90 con 0x7f1c1015fc00 2024-01-11T07:19:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.300+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1004c3c0 con 0x7f1c1015fc00 2024-01-11T07:19:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.300+0000 7f1c067fc700 1 -- 172.21.15.7:0/1943808216 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f1c08001bc0 con 0x7f1c1015fc00 2024-01-11T07:19:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.301+0000 7f1c067fc700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1bf0066f30 0x7f1bf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.301+0000 7f1c067fc700 1 -- 172.21.15.7:0/1943808216 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1c08083f30 con 0x7f1c1015fc00 2024-01-11T07:19:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.301+0000 7f1c0e59c700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1bf0066f30 0x7f1bf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.302+0000 7f1c0e59c700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1bf0066f30 0x7f1bf00693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1c00007cd0 tx=0x7f1c00015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.305+0000 7f1c067fc700 1 -- 172.21.15.7:0/1943808216 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c08051520 con 0x7f1c1015fc00 2024-01-11T07:19:21.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.511+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f1c1015adb0 con 0x7f1c1015fc00 2024-01-11T07:19:21.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.515+0000 7f1c067fc700 1 -- 172.21.15.7:0/1943808216 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1c08001e70 con 0x7f1c1015fc00 2024-01-11T07:19:21.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.515+0000 7f1c067fc700 1 -- 172.21.15.7:0/1943808216 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f1c080190c0 con 0x7f1c1015fc00 2024-01-11T07:19:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1bf0066f30 msgr2=0x7f1bf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1bf0066f30 0x7f1bf00693e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1c00007cd0 tx=0x7f1c00015000 comp rx=0 tx=0).stop 2024-01-11T07:19:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015fc00 msgr2=0x7f1c10153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015fc00 0x7f1c10153950 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006d820 tx=0x7f1c08009040 comp rx=0 tx=0).stop 2024-01-11T07:19:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 shutdown_connections 2024-01-11T07:19:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f1bf0066f30 0x7f1bf00693e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c1015fc00 0x7f1c10153950 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1bfc003a60 0x7f1c100cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 --2- 172.21.15.7:0/1943808216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c100cdae0 0x7f1c1015f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 >> 172.21.15.7:0/1943808216 conn(0x7f1c100680e0 msgr2=0x7f1c100cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:21.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 shutdown_connections 2024-01-11T07:19:21.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.518+0000 7f1c14e77700 1 -- 172.21.15.7:0/1943808216 wait complete. 2024-01-11T07:19:21.532 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-11T07:19:21.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-11T07:19:21.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.663+0000 7f599e4a1700 1 Processor -- start 2024-01-11T07:19:21.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.663+0000 7f599e4a1700 1 -- start start 2024-01-11T07:19:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.664+0000 7f599e4a1700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f599806dd00 0x7f59980c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.664+0000 7f599e4a1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806e0d0 0x7f5998159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.664+0000 7f599e4a1700 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599815b770 con 0x7f5998159ff0 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.664+0000 7f599e4a1700 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599815abf0 con 0x7f599806dd00 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.664+0000 7f599e4a1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599815aed0 con 0x7f599806e0d0 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5997fff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f599806dd00 0x7f59980c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f59977fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806e0d0 0x7f5998159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f59977fe700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806e0d0 0x7f5998159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.7:55306/0 (socket says 172.21.15.7:55306) 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5997fff700 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f599806dd00 0x7f59980c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.7:58424/0 (socket says 172.21.15.7:58424) 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f59977fe700 1 -- 172.21.15.7:0/2732145016 learned_addr learned my addr 172.21.15.7:0/2732145016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f599ca3e700 1 --1- 172.21.15.7:0/2732145016 >> v1:172.21.15.113:6789/0 conn(0x7f5998159ff0 0x7f59980c7e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.7:42786/0 (socket says 172.21.15.7:42786) 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5997fff700 1 -- 172.21.15.7:0/2732145016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806e0d0 msgr2=0x7f5998159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5997fff700 1 --2- 172.21.15.7:0/2732145016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806e0d0 0x7f5998159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5997fff700 1 -- 172.21.15.7:0/2732145016 >> v1:172.21.15.113:6789/0 conn(0x7f5998159ff0 legacy=0x7f59980c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5997fff700 1 -- 172.21.15.7:0/2732145016 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599815b1b0 con 0x7f599806dd00 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5996ffd700 1 -- 172.21.15.7:0/2732145016 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195209211 0 0) 0x7f599815b770 con 0x7f5998159ff0 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f59977fe700 1 --2- 172.21.15.7:0/2732145016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806e0d0 0x7f5998159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5997fff700 1 --2- 172.21.15.7:0/2732145016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806dd00 0x7f59980c98d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f598800dd10 tx=0x7f598800d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db8816c91903246c server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.665+0000 7f5996ffd700 1 -- 172.21.15.7:0/2732145016 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988012070 con 0x7f599806dd00 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f5996ffd700 1 -- 172.21.15.7:0/2732145016 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f598800de70 con 0x7f599806dd00 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f5996ffd700 1 -- 172.21.15.7:0/2732145016 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800fb80 con 0x7f599806dd00 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f599e4a1700 1 -- 172.21.15.7:0/2732145016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806dd00 msgr2=0x7f59980c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f599e4a1700 1 --2- 172.21.15.7:0/2732145016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806dd00 0x7f59980c98d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f598800dd10 tx=0x7f598800d630 comp rx=0 tx=0).stop 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f599e4a1700 1 -- 172.21.15.7:0/2732145016 shutdown_connections 2024-01-11T07:19:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f599e4a1700 1 --2- 172.21.15.7:0/2732145016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806dd00 0x7f59980c98d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f599e4a1700 1 --2- 172.21.15.7:0/2732145016 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f599806e0d0 0x7f5998159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f599e4a1700 1 -- 172.21.15.7:0/2732145016 >> 172.21.15.7:0/2732145016 conn(0x7f59980680e0 msgr2=0x7f59980cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f599e4a1700 1 -- 172.21.15.7:0/2732145016 shutdown_connections 2024-01-11T07:19:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.666+0000 7f599e4a1700 1 -- 172.21.15.7:0/2732145016 wait complete. 2024-01-11T07:19:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.667+0000 7f599e4a1700 1 Processor -- start 2024-01-11T07:19:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f599e4a1700 1 -- start start 2024-01-11T07:19:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f599e4a1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599806dd00 0x7f59980a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f599e4a1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806e0d0 0x7f599814bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f599e4a1700 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159ff0 0x7f59980c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f599e4a1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980c7cb0 con 0x7f599806dd00 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f599e4a1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980c50d0 con 0x7f599806e0d0 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f599e4a1700 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59980c79d0 con 0x7f5998159ff0 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f5997fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599806dd00 0x7f59980a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f5997fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599806dd00 0x7f59980a64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50832/0 (socket says 172.21.15.7:50832) 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.668+0000 7f5997fff700 1 -- 172.21.15.7:0/1737307749 learned_addr learned my addr 172.21.15.7:0/1737307749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f59977fe700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806e0d0 0x7f599814bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f599ca3e700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159ff0 0x7f59980c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f5997fff700 1 -- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159ff0 msgr2=0x7f59980c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f5997fff700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159ff0 0x7f59980c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f5997fff700 1 -- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806e0d0 msgr2=0x7f599814bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f5997fff700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806e0d0 0x7f599814bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f5997fff700 1 -- 172.21.15.7:0/1737307749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c00b5a0 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f5997fff700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599806dd00 0x7f59980a64c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f598800f6c0 tx=0x7f598800d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f597ffff700 1 -- 172.21.15.7:0/1737307749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598800ca90 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f597ffff700 1 -- 172.21.15.7:0/1737307749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f598800cc10 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598800d8d0 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f597ffff700 1 -- 172.21.15.7:0/1737307749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988020440 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.669+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59980c4d60 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.670+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59980c53b0 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.671+0000 7f597ffff700 1 -- 172.21.15.7:0/1737307749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 84742+0+0 (secure 0 0 0) 0x7f5988020810 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.671+0000 7f597ffff700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5984066f30 0x7f59840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.671+0000 7f597ffff700 1 -- 172.21.15.7:0/1737307749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5988089790 con 0x7f599806dd00 2024-01-11T07:19:21.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.672+0000 7f59977fe700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5984066f30 0x7f59840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-11T07:19:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.672+0000 7f59977fe700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5984066f30 0x7f59840693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f598c006d40 tx=0x7f598c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-11T07:19:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.676+0000 7f597ffff700 1 -- 172.21.15.7:0/1737307749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5988056d80 con 0x7f599806dd00 2024-01-11T07:19:21.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.924+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59980c48e0 con 0x7f599806dd00 2024-01-11T07:19:21.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.925+0000 7f597ffff700 1 -- 172.21.15.7:0/1737307749 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f5988024de0 con 0x7f599806dd00 2024-01-11T07:19:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-01-11T07:19:21.926 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2324299050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2324299050},{"type":"v1","addr":"172.21.15.113:6837","nonce":2324299050}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":6791,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1217900748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1217900748},{"type":"v1","addr":"172.21.15.113:6836","nonce":1217900748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1573270369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1573270369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1573270369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":8358,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3114463876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3114463876},{"type":"v1","addr":"172.21.15.7:6835","nonce":3114463876}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":92}],"filesystems":[]} 2024-01-11T07:19:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5984066f30 msgr2=0x7f59840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5984066f30 0x7f59840693e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f598c006d40 tx=0x7f598c008040 comp rx=0 tx=0).stop 2024-01-11T07:19:21.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599806dd00 msgr2=0x7f59980a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599806dd00 0x7f59980a64c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f598800f6c0 tx=0x7f598800d630 comp rx=0 tx=0).stop 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 shutdown_connections 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:6832/36527,v1:172.21.15.113:6833/36527] conn(0x7f5984066f30 0x7f59840693e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5998159ff0 0x7f59980c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599806dd00 0x7f59980a64c0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 --2- 172.21.15.7:0/1737307749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f599806e0d0 0x7f599814bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 >> 172.21.15.7:0/1737307749 conn(0x7f59980680e0 msgr2=0x7f59980ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 shutdown_connections 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-01-11T07:19:21.927+0000 7f599e4a1700 1 -- 172.21.15.7:0/1737307749 wait complete. 2024-01-11T07:19:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 92 2024-01-11T07:19:21.941 DEBUG:tasks.cephfs.filesystem:already dead... 2024-01-11T07:19:21.941 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-11T07:19:21.942 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-11T07:19:21.942 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:21.950 INFO:tasks.ceph.mds.a:Stopped 2024-01-11T07:19:21.951 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-11T07:19:21.951 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.001 INFO:tasks.ceph.mds.c:Stopped 2024-01-11T07:19:22.001 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-11T07:19:22.001 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.012 INFO:tasks.ceph.mds.b:Stopped 2024-01-11T07:19:22.012 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-11T07:19:22.012 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.062 INFO:tasks.ceph.mds.d:Stopped 2024-01-11T07:19:22.062 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-11T07:19:22.062 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-11T07:19:22.062 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.086 INFO:tasks.ceph.osd.0:Stopped 2024-01-11T07:19:22.086 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-11T07:19:22.086 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.154 INFO:tasks.ceph.osd.1:Stopped 2024-01-11T07:19:22.154 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-11T07:19:22.154 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.224 INFO:tasks.ceph.osd.2:Stopped 2024-01-11T07:19:22.224 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-11T07:19:22.224 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.288 INFO:tasks.ceph.osd.3:Stopped 2024-01-11T07:19:22.288 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-11T07:19:22.288 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.310 INFO:tasks.ceph.osd.4:Stopped 2024-01-11T07:19:22.310 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-11T07:19:22.310 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.369 INFO:tasks.ceph.osd.5:Stopped 2024-01-11T07:19:22.369 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-11T07:19:22.369 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.431 INFO:tasks.ceph.osd.6:Stopped 2024-01-11T07:19:22.432 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-11T07:19:22.432 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.492 INFO:tasks.ceph.osd.7:Stopped 2024-01-11T07:19:22.492 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-11T07:19:22.492 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-11T07:19:22.492 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.519 INFO:tasks.ceph.mgr.y:Stopped 2024-01-11T07:19:22.520 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-11T07:19:22.520 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.557 INFO:tasks.ceph.mgr.x:Stopped 2024-01-11T07:19:22.557 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-11T07:19:22.558 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-11T07:19:22.558 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.574 INFO:tasks.ceph.mon.a:Stopped 2024-01-11T07:19:22.574 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-11T07:19:22.574 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.608 INFO:tasks.ceph.mon.b:Stopped 2024-01-11T07:19:22.608 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-11T07:19:22.608 INFO:teuthology.orchestra.run:waiting for 300 2024-01-11T07:19:22.659 INFO:tasks.ceph.mon.c:Stopped 2024-01-11T07:19:22.659 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-11T07:19:22.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-01-11T07:19:22.705 INFO:teuthology.orchestra.run.smithi007.stdout:1704956806.8902118 osd.4 (osd.4) 3 : cluster [WRN] OSD bench result of 225823.776910 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.4. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2024-01-11T07:19:22.705 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-01-11T07:19:22.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-01-11T07:19:22.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-01-11T07:19:22.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-01-11T07:19:22.878 INFO:teuthology.orchestra.run.smithi007.stdout:1704956806.8902118 osd.4 (osd.4) 3 : cluster [WRN] OSD bench result of 225823.776910 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.4. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2024-01-11T07:19:22.878 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:19:22.878 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-11T07:19:23.085 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:19:23.085 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-11T07:19:23.192 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:19:23.192 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-11T07:19:23.298 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:19:23.298 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-11T07:19:23.377 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:19:23.377 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-11T07:19:23.602 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:19:23.603 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-11T07:19:23.692 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:19:23.692 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-11T07:19:23.763 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:19:23.763 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-11T07:19:23.866 INFO:tasks.ceph:Archiving mon data... 2024-01-11T07:19:23.868 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/data/mon.a.tgz 2024-01-11T07:19:23.868 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-01-11T07:19:23.890 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.7hJuGyDUKt 2024-01-11T07:19:23.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.7hJuGyDUKt 2024-01-11T07:19:24.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.7hJuGyDUKt 2024-01-11T07:19:24.151 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.7hJuGyDUKt is 400KB 2024-01-11T07:19:24.226 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.7hJuGyDUKt 2024-01-11T07:19:24.247 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/data/mon.b.tgz 2024-01-11T07:19:24.247 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-01-11T07:19:24.268 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.fH8l2Y3VEp 2024-01-11T07:19:24.268 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.fH8l2Y3VEp 2024-01-11T07:19:24.449 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.fH8l2Y3VEp 2024-01-11T07:19:24.537 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.fH8l2Y3VEp is 407KB 2024-01-11T07:19:24.632 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.fH8l2Y3VEp 2024-01-11T07:19:24.649 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/data/mon.c.tgz 2024-01-11T07:19:24.649 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-01-11T07:19:24.705 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.VsfgmV3HN1 2024-01-11T07:19:24.705 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.VsfgmV3HN1 2024-01-11T07:19:24.848 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.VsfgmV3HN1 2024-01-11T07:19:24.934 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.VsfgmV3HN1 is 407KB 2024-01-11T07:19:25.003 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.VsfgmV3HN1 2024-01-11T07:19:25.020 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-11T07:19:25.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-11T07:19:25.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-11T07:19:25.086 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-01-11T07:19:25.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-01-11T07:19:25.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-01-11T07:19:25.130 INFO:teuthology.orchestra.run.smithi007.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-01-11T07:19:25.154 INFO:teuthology.orchestra.run.smithi113.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-01-11T07:19:25.159 INFO:tasks.ceph:Archiving crash dumps... 2024-01-11T07:19:25.161 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/remote/smithi007/crash 2024-01-11T07:19:25.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-11T07:19:25.213 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/remote/smithi113/crash 2024-01-11T07:19:25.214 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-11T07:19:25.240 INFO:tasks.ceph:Shutting down logrotate 2024-01-11T07:19:25.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:19:25.258 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-11T07:19:25.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-11T07:19:25.380 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-11T07:19:25.406 INFO:tasks.ceph:Compressing logs... 2024-01-11T07:19:25.407 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-11T07:19:25.423 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-11T07:19:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57978.log 2024-01-11T07:19:25.455 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60834.log 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48457.log 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57978.log: /var/log/ceph/ceph-client.admin.60834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70066.log 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57978.log.gz 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60834.log.gz 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67151.log 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48457.log.gz 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69534.log 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70066.log: /var/log/ceph/ceph-client.admin.67151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58544.log 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70066.log.gz 2024-01-11T07:19:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69534.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67151.log.gz 2024-01-11T07:19:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66911.log 2024-01-11T07:19:25.459 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69534.log.gz 2024-01-11T07:19:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69626.log 2024-01-11T07:19:25.466 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58544.log.gz 2024-01-11T07:19:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50317.log 2024-01-11T07:19:25.466 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0%/var/log/ceph/ceph-client.admin.69626.log: -- replaced with /var/log/ceph/ceph-client.admin.66911.log.gz 2024-01-11T07:19:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45136.log 2024-01-11T07:19:25.466 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69626.log.gz 2024-01-11T07:19:25.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42335.log 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50317.log.gz 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70303.log 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45136.log.gz 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38133.log 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42335.log.gz 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.70303.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.70303.log.gz 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.64188.log 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55552.log 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38133.log.gz 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44221.log 2024-01-11T07:19:25.467 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64188.log.gz 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43989.log 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55552.log.gz 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50896.log 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44221.log.gz 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61624.log 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43989.log.gz 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.50896.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.0.50896.log.gz--verbose 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:-- /var/log/ceph/ceph-client.admin.54871.log 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44758.log 2024-01-11T07:19:25.468 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61624.log.gz 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43793.log 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.44758.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54871.log.gz 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 0.0%--verbose -- replaced with /var/log/ceph/ceph-client.0.44758.log.gz-- 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56293.log 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37648.log 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43793.log.gz 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65555.log 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56293.log.gz 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62119.log 2024-01-11T07:19:25.469 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37648.log.gz 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64074.log 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65555.log.gz 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56607.log 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62119.log.gz 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65273.log 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64074.log.gz 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56339.log 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56607.log.gz 2024-01-11T07:19:25.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56515.log 2024-01-11T07:19:25.500 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-11T07:19:25.500 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36738.log 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39571.log 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.36738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42429.log 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.39571.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36738.log.gz 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39571.log.gz 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36834.log 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40563.log /var/log/ceph/ceph-client.1.42429.log: 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42429.log.gz 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39129.log 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.40563.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36834.log.gz 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38215.log 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40563.log.gz 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42962.log 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.39129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39129.log.gz 2024-01-11T07:19:25.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.38215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39125.log 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.38215.log.gz 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.42962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42962.log.gz 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.39125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42966.log 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39125.log.gz 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40043.log 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.42966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42966.log.gz 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42425.log 2024-01-11T07:19:25.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.40043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41959.log 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40043.log.gz 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.42425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42425.log.gz 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.41959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41963.log 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41959.log.gz 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41068.log 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.41963.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41963.log.gz 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36788.log 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.41068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41068.log.gz 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36788.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.d.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36788.log.gz 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.audit.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-11T07:19:25.503 INFO:teuthology.orchestra.run.smithi113.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43456.log 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40559.log 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.43456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36933.log 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43456.log.gz 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.40559.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40559.log.gz 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36933.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-11T07:19:25.504 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36933.log.gz 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39575.log 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.39575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39575.log.gz 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37026.log 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38207.log 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37026.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37026.log.gz 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.38207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.38207.log.gz 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.38650.log 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41517.log 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.3.38650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.38650.log.gz 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.41517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41517.log.gz 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-01-11T07:19:25.505 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.admin.36978.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40039.log 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.40039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40039.log.gz 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41513.log 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.41513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41513.log.gz 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43460.log 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.43460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43460.log.gz 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-11T07:19:25.506 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-11T07:19:25.509 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41072.log 2024-01-11T07:19:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.41072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41072.log.gz 2024-01-11T07:19:25.527 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36887.log 2024-01-11T07:19:25.538 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36887.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36887.log.gz 2024-01-11T07:19:25.544 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37072.log 2024-01-11T07:19:25.553 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65273.log.gz 2024-01-11T07:19:25.553 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47281.log 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56339.log.gz 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56515.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56515.log.gz 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53069.log 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47281.log.gz 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42151.log 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39437.log 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53069.log.gz 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69812.log 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42151.log.gz 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49935.log 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39437.log: /var/log/ceph/ceph-client.admin.69812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43609.log 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39437.log.gz 2024-01-11T07:19:25.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49935.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69812.log.gz 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65157.log 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49935.log.gz 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71741.log 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43609.log.gz 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48411.log 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65157.log.gz 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57115.log 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71741.log.gz 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42697.log 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48411.log.gz 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60396.log 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42697.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57115.log.gz 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58220.log 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42697.log.gz 2024-01-11T07:19:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58070.log 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58220.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60396.log.gz 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72663.log 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58220.log.gz 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44129.log 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58070.log.gz 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44129.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72663.log.gz 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55414.log 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44129.log.gz 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38363.log 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69480.log 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55414.log.gz 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38087.log 2024-01-11T07:19:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69480.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38363.log.gzgzip 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.45327.log 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69480.log.gz 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50223.log 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38087.log.gz 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49119.log 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50223.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.45327.log.gz-5 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:--verbose -- /var/log/ceph/ceph-client.admin.55460.log 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50223.log.gz 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67289.log 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49119.log.gz 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41937.log 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67289.log: gzip -5 87.9%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55460.log.gz/var/log/ceph/ceph-client.admin.48705.log 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67289.log.gz 2024-01-11T07:19:25.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56001.log 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41937.log.gz 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42743.log 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48705.log.gz 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46257.log 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56001.log.gz 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59039.log 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42743.log.gz 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69338.log 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46257.log.gz 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68748.log 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59039.log.gz 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39283.log 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0%/var/log/ceph/ceph-client.admin.68748.log: -- replaced with /var/log/ceph/ceph-client.admin.69338.log.gz 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48189.log 2024-01-11T07:19:25.558 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5%/var/log/ceph/ceph-client.admin.39283.log: -- replaced with /var/log/ceph/ceph-client.admin.68748.log.gz 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41523.log 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39283.log.gz 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38708.log 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3%/var/log/ceph/ceph-client.admin.41523.log: -- replaced with /var/log/ceph/ceph-client.admin.48189.log.gz 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70633.log 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41523.log.gz 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63691.log 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38708.log.gz 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42197.log 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70633.log.gz 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58024.log 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42197.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63691.log.gz 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63166.log 2024-01-11T07:19:25.559 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42197.log.gz 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44451.log 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58024.log.gz 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63166.log.gz 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71493.log 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43203.log 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44451.log.gz 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71493.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63737.log 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71493.log.gz 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71060.log 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43203.log: /var/log/ceph/ceph-client.admin.63737.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43203.log.gz 2024-01-11T07:19:25.560 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57594.log 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63737.log.gz 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72801.log 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57594.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71060.log.gz 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49887.log 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57594.log.gz 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71834.log 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49887.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72801.log.gz 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38915.log 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3%/var/log/ceph/ceph-client.admin.71834.log: -- replaced with /var/log/ceph/ceph-client.admin.49887.log.gz 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68223.log 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71834.log.gz 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68223.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38915.log.gz 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60056.log 2024-01-11T07:19:25.561 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68223.log.gz 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54741.log 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44405.log.gz 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66079.log 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54741.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60056.log.gz 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43425.log 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66079.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54741.log.gz 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48751.log 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66079.log.gz 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47511.log 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48751.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43425.log.gz 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47581.log 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48751.log.gz 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41845.log 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38697.log 2024-01-11T07:19:25.562 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47581.log.gz 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41845.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47511.log.gz 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42651.log 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41845.log.gz 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41661.log 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42651.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38697.log.gz 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64662.log 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42651.log.gz 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40249.log 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41661.log.gz 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55598.log 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64662.log.gz 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72387.log 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40249.log.gz 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39743.log 2024-01-11T07:19:25.563 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55598.log.gz 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72295.log 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72387.log.gz 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63897.log 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72295.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39743.log.gz 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69246.log 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72295.log.gz 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62596.log 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63897.log.gz 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72157.log 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62596.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69246.log.gz 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53518.log 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62596.log.gz 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60489.log 2024-01-11T07:19:25.564 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72157.log.gz 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49229.log 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60489.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53518.log.gz 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63120.log 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60489.log.gz 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47235.log 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49229.log.gz 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45090.log 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47235.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63120.log.gz 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58452.log 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47235.log.gz 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38225.log 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45090.log.gz 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61716.log 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38225.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58452.log.gz 2024-01-11T07:19:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61786.log 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38225.log.gz 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61716.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64430.log 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.c.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61786.log.gz 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61716.log.gzgzip 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.68361.log 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61254.log 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64430.log.gz 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59964.log 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68361.log.gz 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49322.log 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61254.log: /var/log/ceph/ceph-client.admin.59964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66769.log 2024-01-11T07:19:25.566 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61254.log.gz 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59964.log.gz 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64524.log 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40341.log 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49322.log.gz 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66769.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66769.log.gz 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59085.log 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40341.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64524.log.gz 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58752.log 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59085.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40341.log.gz 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54581.log 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59085.log.gz 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58636.log 2024-01-11T07:19:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54965.log 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58752.log.gz 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54581.log.gz 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61200.log 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47814.log 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58636.log.gz 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45640.log 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54965.log.gz 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40979.log 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.47814.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61200.log.gz 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47814.log.gz 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57299.log 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45640.log.gz 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.40979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40979.log.gz 2024-01-11T07:19:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68407.log 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39441.log 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57299.log.gz 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67734.log 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43747.log 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6%/var/log/ceph/ceph-client.admin.39441.log: -- replaced with /var/log/ceph/ceph-client.admin.68407.log.gz 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39441.log.gz 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.67734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57932.log 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67734.log.gz 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66218.log 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43747.log.gz 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43471.log 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57932.log.gz 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51946.log 2024-01-11T07:19:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64142.log 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66218.log.gz 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43471.log.gz 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38179.log 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51375.log 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51946.log.gz 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64142.log: /var/log/ceph/ceph-client.admin.38179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62936.log 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64142.log.gz 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38179.log.gz 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42243.log 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62936.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51375.log.gz 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36629.log 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62936.log.gz 2024-01-11T07:19:25.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49368.log 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52818.log 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36629.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42243.log.gz 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49368.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36629.log.gz 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60148.log 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49368.log.gz 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:gzip /var/log/ceph/ceph-client.admin.52818.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50029.log 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72617.log 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52818.log.gz 2024-01-11T07:19:25.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50029.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60148.log.gz 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73190.log 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72617.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50029.log.gz 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72617.log.gz 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73190.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73190.log.gz 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61012.log 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63982.log 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46899.log 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63982.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61012.log.gz 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64870.log 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63982.log.gz 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47868.log 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40065.log 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64870.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46899.log.gz 2024-01-11T07:19:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52726.log 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64870.log.gz 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47868.log.gz 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53231.log 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52726.log: gzip -5 88.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.40065.log.gz/var/log/ceph/ceph-client.admin.37694.log 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67381.log 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52726.log.gz 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53231.log.gz 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51329.log 2024-01-11T07:19:25.573 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37694.log.gz 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54511.log 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67381.log: /var/log/ceph/ceph-client.admin.51329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49276.log 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67381.log.gz 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51329.log.gz 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52178.log 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip 88.5%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.54511.log.gz-- 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67830.log 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49276.log: /var/log/ceph/ceph-client.admin.52178.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49276.log.gz 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62299.log 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52772.log 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67830.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52178.log.gz 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62299.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67830.log.gz 2024-01-11T07:19:25.574 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6%/var/log/ceph/ceph-client.admin.52772.log: -- replaced with /var/log/ceph/ceph-client.admin.62299.log.gz 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57023.log 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52772.log.gz 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45778.log 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73236.log 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57023.log.gz 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73146.log 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45778.log.gz 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57800.log 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73236.log: /var/log/ceph/ceph-client.admin.73146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73236.log.gz 2024-01-11T07:19:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71677.log 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73146.log.gz 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59537.log 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72709.log 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57800.log.gz 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71677.log.gz 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51807.log 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65693.log 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59537.log.gz 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51807.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72709.log.gz 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60010.log 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65693.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51807.log.gz 2024-01-11T07:19:25.576 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65693.log.gz 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70469.log 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60010.log.gz 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61440.log 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.70469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43019.log 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70469.log.gz 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61440.log: /var/log/ceph/ceph-client.admin.43019.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61440.log.gz 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43019.log.gz 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51513.log 2024-01-11T07:19:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37602.log 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39442.log 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37602.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51513.log.gz 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37602.log.gz 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70922.log 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39442.log.gz 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62189.log 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70922.log: /var/log/ceph/ceph-client.admin.46349.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70922.log.gz 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62189.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62189.log.gz 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62642.log 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38698.log 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71539.log 2024-01-11T07:19:25.578 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62642.log.gz 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57456.log 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3%/var/log/ceph/ceph-client.admin.71539.log: -- replaced with /var/log/ceph/ceph-client.admin.38698.log.gz 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57456.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71539.log.gz 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57456.log.gz 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50996.log 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41037.log 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60332.log 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50996.log.gz 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68702.log 2024-01-11T07:19:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59346.log 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60332.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.41037.log.gz 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68702.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60332.log.gz 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68702.log.gz 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59346.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59346.log.gz 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69996.log 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57502.log 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73077.log 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64570.log 2024-01-11T07:19:25.580 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57502.log.gz 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73077.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.69996.log.gz 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73077.log.gz 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64570.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64570.log.gz 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40987.log 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.40987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54143.log 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40987.log.gz 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54189.log 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54143.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53564.log 2024-01-11T07:19:25.581 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54143.log.gz 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54005.log 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50900.log 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1%/var/log/ceph/ceph-client.admin.53564.log: -- replaced with /var/log/ceph/ceph-client.admin.54189.log.gz 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54005.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53564.log.gz 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50593.log 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54005.log.gz 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40019.log 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.50900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50900.log.gz 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39453.log 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42881.log 2024-01-11T07:19:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40019.log: /var/log/ceph/ceph-client.admin.39453.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40019.log.gz 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50593.log.gz 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39453.log.gz 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42881.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42881.log.gz 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67197.log 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51697.log 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51467.log 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51697.log: 87.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.67197.log.gz--verbose 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:-- /var/log/ceph/ceph-client.admin.63074.log 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6%/var/log/ceph/ceph-client.admin.51467.log: -- replaced with /var/log/ceph/ceph-client.admin.51697.log.gz 2024-01-11T07:19:25.583 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51467.log.gz 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63074.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63074.log.gz 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55301.log 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37790.log 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55301.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37790.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55301.log.gz 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58360.log 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.a.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37790.log.gz 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72525.log 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48123.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58360.log.gz 2024-01-11T07:19:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51421.log 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72525.log: /var/log/ceph/ceph-client.admin.51421.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72525.log.gz 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39697.log 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51421.log.gz 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56227.log 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39697.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39697.log.gz 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56227.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56227.log.gz 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60240.log 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38501.log 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60240.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60240.log.gz 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38501.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38501.log.gz 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45686.log 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64616.log 2024-01-11T07:19:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53867.log 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64616.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45686.log.gz 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53867.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64616.log.gz 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53867.log.gz 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49481.log 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65923.log 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49481.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49481.log.gz 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65923.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65923.log.gz 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58498.log 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69003.log 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58498.log: /var/log/ceph/ceph-client.admin.69003.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58498.log.gz 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69003.log.gz 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70812.log 2024-01-11T07:19:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54235.log 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70812.log: /var/log/ceph/ceph-client.admin.54235.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70812.log.gz 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54235.log.gz 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46760.log 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.41468.log 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44267.log 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.2.41468.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46760.log.gz 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.41468.log.gz 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44267.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44267.log.gz 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66965.log 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61058.log 2024-01-11T07:19:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66863.log 2024-01-11T07:19:25.588 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.38654.log 2024-01-11T07:19:25.589 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37072.log.gz 2024-01-11T07:19:25.589 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.3.38654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.38654.log.gz 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2%/var/log/ceph/ceph-client.admin.61058.log: -- replaced with /var/log/ceph/ceph-client.admin.66965.log.gz 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61058.log.gz 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66863.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66863.log.gz 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51743.log 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57410.log 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41799.log 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57410.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51743.log.gz 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57410.log.gz 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41799.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41799.log.gz 2024-01-11T07:19:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67730.log 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57207.log 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.67730.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67730.log.gz 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57207.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57207.log.gz 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41615.log 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69950.log 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45824.log 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69950.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41615.log.gz 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45962.log 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69950.log.gz 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37509.log 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45962.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45824.log.gz 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37509.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45962.log.gz 2024-01-11T07:19:25.602 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37509.log.gz 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56818.log 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43517.log 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71585.log 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56818.log.gz 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39306.log 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71585.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43517.log.gz 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60194.log 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71585.log.gz 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64754.log 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39306.log.gz 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70399.log 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64754.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60194.log.gz 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64754.log.gz 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70399.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70399.log.gz 2024-01-11T07:19:25.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67427.log 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41753.log 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47049.log 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41753.log: /var/log/ceph/ceph-client.admin.47049.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67427.log.gz 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41753.log.gz 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47049.log.gz 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68499.log 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48549.log 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68499.log.gz 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49981.log 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48549.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48549.log.gz 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.41472.log 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49981.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49981.log.gz 2024-01-11T07:19:25.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70299.log 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.2.41472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.41472.log.gz 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48843.log 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.70299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70299.log.gz 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48843.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48843.log.gz 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50131.log 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37556.log 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50131.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50131.log.gz 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37556.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37556.log.gz 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43065.log 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43563.log 2024-01-11T07:19:25.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43065.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43065.log.gz 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43563.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43563.log.gz 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68932.log 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.68932.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68932.log.gz 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60443.log 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60443.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60443.log.gz 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50455.log 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50455.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50455.log.gz 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71788.log 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71788.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71788.log.gz 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45870.log 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45870.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gz 2024-01-11T07:19:25.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41569.log 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41569.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41569.log.gz 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57253.log 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57253.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57253.log.gz 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46303.log 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46303.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46303.log.gz 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50177.log 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50177.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50177.log.gz 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68794.log 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68794.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68794.log.gz 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52586.log 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44175.log 2024-01-11T07:19:25.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39927.log 2024-01-11T07:19:25.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44175.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44175.log.gz 2024-01-11T07:19:25.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39927.log: /var/log/ceph/ceph-client.admin.52586.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39927.log.gz 2024-01-11T07:19:25.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57363.log 2024-01-11T07:19:25.608 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52586.log.gz 2024-01-11T07:19:25.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57363.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57363.log.gz 2024-01-11T07:19:25.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54917.log 2024-01-11T07:19:25.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54917.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54917.log.gz 2024-01-11T07:19:25.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46395.log 2024-01-11T07:19:25.609 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46395.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46395.log.gz 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51900.log 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51900.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51900.log.gz 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58166.log 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58166.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58166.log.gz 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46008.log 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46008.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46008.log.gz 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64384.log 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64384.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64384.log.gz 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58406.log 2024-01-11T07:19:25.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58406.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58406.log.gz 2024-01-11T07:19:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53468.log 2024-01-11T07:19:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.53468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53468.log.gz 2024-01-11T07:19:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66126.log 2024-01-11T07:19:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66126.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66126.log.gz 2024-01-11T07:19:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72249.log 2024-01-11T07:19:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72249.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72249.log.gz 2024-01-11T07:19:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54097.log 2024-01-11T07:19:25.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54097.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54097.log.gz 2024-01-11T07:19:25.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55951.log 2024-01-11T07:19:25.613 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63212.log 2024-01-11T07:19:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.55951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55951.log.gz 2024-01-11T07:19:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63212.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63212.log.gz 2024-01-11T07:19:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59300.log 2024-01-11T07:19:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59300.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59300.log.gz 2024-01-11T07:19:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62780.log 2024-01-11T07:19:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62780.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62780.log.gz 2024-01-11T07:19:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60628.log 2024-01-11T07:19:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60628.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60628.log.gz 2024-01-11T07:19:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49841.log 2024-01-11T07:19:25.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49841.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49841.log.gz 2024-01-11T07:19:25.620 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58989.log 2024-01-11T07:19:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55253.log 2024-01-11T07:19:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.58989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58989.log.gz 2024-01-11T07:19:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55253.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55253.log.gz 2024-01-11T07:19:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48797.log 2024-01-11T07:19:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47373.log 2024-01-11T07:19:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48797.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48797.log.gz 2024-01-11T07:19:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47373.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47373.log.gz 2024-01-11T07:19:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65831.log 2024-01-11T07:19:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65831.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65831.log.gz 2024-01-11T07:19:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55107.log 2024-01-11T07:19:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53464.log 2024-01-11T07:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55107.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55107.log.gz 2024-01-11T07:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.53464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53464.log.gz 2024-01-11T07:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38700.log 2024-01-11T07:19:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38700.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38700.log.gz 2024-01-11T07:19:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72847.log 2024-01-11T07:19:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72847.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72847.log.gz 2024-01-11T07:19:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52910.log 2024-01-11T07:19:25.632 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52910.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52910.log.gz 2024-01-11T07:19:25.632 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65647.log 2024-01-11T07:19:25.633 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65647.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65647.log.gz 2024-01-11T07:19:25.633 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62688.log 2024-01-11T07:19:25.634 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62688.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62688.log.gz 2024-01-11T07:19:25.634 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63396.log 2024-01-11T07:19:25.636 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63396.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63396.log.gz 2024-01-11T07:19:25.636 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46054.log 2024-01-11T07:19:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46054.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46054.log.gz 2024-01-11T07:19:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-11T07:19:25.645 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50363.log 2024-01-11T07:19:25.652 INFO:teuthology.orchestra.run.smithi113.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-11T07:19:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50363.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50363.log.gz 2024-01-11T07:19:25.658 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61486.log 2024-01-11T07:19:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61486.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61486.log.gz 2024-01-11T07:19:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52038.log 2024-01-11T07:19:25.660 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52038.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52038.log.gz 2024-01-11T07:19:25.660 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42789.log 2024-01-11T07:19:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42789.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42789.log.gz 2024-01-11T07:19:25.662 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2024-01-11T07:19:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42037.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2024-01-11T07:19:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49165.log 2024-01-11T07:19:25.665 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49165.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49165.log.gz 2024-01-11T07:19:25.665 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2024-01-11T07:19:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39072.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2024-01-11T07:19:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2024-01-11T07:19:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50547.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2024-01-11T07:19:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70766.log 2024-01-11T07:19:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70766.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70766.log.gz 2024-01-11T07:19:25.669 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52538.log 2024-01-11T07:19:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52538.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52538.log.gz 2024-01-11T07:19:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39465.log 2024-01-11T07:19:25.684 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39465.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39465.log.gz 2024-01-11T07:19:25.684 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64800.log 2024-01-11T07:19:25.686 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64800.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.64800.log.gz 2024-01-11T07:19:25.686 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51992.log 2024-01-11T07:19:25.687 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51992.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51992.log.gz 2024-01-11T07:19:25.687 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65103.log 2024-01-11T07:19:25.688 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.65103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65103.log.gz 2024-01-11T07:19:25.688 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51605.log 2024-01-11T07:19:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51605.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51605.log.gz 2024-01-11T07:19:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43111.log 2024-01-11T07:19:25.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43111.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43111.log.gz 2024-01-11T07:19:25.691 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65877.log 2024-01-11T07:19:25.692 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65877.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65877.log.gz 2024-01-11T07:19:25.692 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49073.log 2024-01-11T07:19:25.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49073.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49073.log.gz 2024-01-11T07:19:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72893.log 2024-01-11T07:19:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72893.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72893.log.gz 2024-01-11T07:19:25.695 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53821.log 2024-01-11T07:19:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53821.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53821.log.gz 2024-01-11T07:19:25.696 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52268.log 2024-01-11T07:19:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52268.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52268.log.gz 2024-01-11T07:19:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60102.log 2024-01-11T07:19:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60102.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60102.log.gz 2024-01-11T07:19:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50077.log 2024-01-11T07:19:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50077.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50077.log.gz 2024-01-11T07:19:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43887.log 2024-01-11T07:19:25.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43887.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43887.log.gz 2024-01-11T07:19:25.701 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50950.log 2024-01-11T07:19:25.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50950.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50950.log.gz 2024-01-11T07:19:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53913.log 2024-01-11T07:19:25.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53913.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53913.log.gz 2024-01-11T07:19:25.704 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45182.log 2024-01-11T07:19:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45182.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45182.log.gz 2024-01-11T07:19:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-01-11T07:19:25.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47419.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2024-01-11T07:19:25.706 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46714.log 2024-01-11T07:19:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46714.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46714.log.gz 2024-01-11T07:19:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72571.log 2024-01-11T07:19:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72571.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72571.log.gz 2024-01-11T07:19:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63258.log 2024-01-11T07:19:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63258.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63258.log.gz 2024-01-11T07:19:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47465.log 2024-01-11T07:19:25.714 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47465.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47465.log.gz 2024-01-11T07:19:25.714 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55059.log 2024-01-11T07:19:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55059.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55059.log.gz 2024-01-11T07:19:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49621.log 2024-01-11T07:19:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49621.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49621.log.gz 2024-01-11T07:19:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70700.log 2024-01-11T07:19:25.718 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70700.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70700.log.gz 2024-01-11T07:19:25.718 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52492.log 2024-01-11T07:19:25.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52492.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52492.log.gz 2024-01-11T07:19:25.720 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44521.log 2024-01-11T07:19:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44521.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44521.log.gz 2024-01-11T07:19:25.721 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57069.log 2024-01-11T07:19:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57069.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57069.log.gz 2024-01-11T07:19:25.722 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68315.log 2024-01-11T07:19:25.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68315.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68315.log.gz 2024-01-11T07:19:25.724 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64338.log 2024-01-11T07:19:25.725 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64338.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64338.log.gz 2024-01-11T07:19:25.725 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69580.log 2024-01-11T07:19:25.726 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69580.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69580.log.gz 2024-01-11T07:19:25.726 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51559.log 2024-01-11T07:19:25.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51559.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51559.log.gz 2024-01-11T07:19:25.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59392.log 2024-01-11T07:19:25.728 INFO:teuthology.orchestra.run.smithi113.stderr: 95.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-11T07:19:25.729 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59392.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59392.log.gz 2024-01-11T07:19:25.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-11T07:19:25.731 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69432.log 2024-01-11T07:19:25.733 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-11T07:19:25.734 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69432.log: 92.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-11T07:19:25.734 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69432.log.gz 2024-01-11T07:19:25.734 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58118.log 2024-01-11T07:19:25.735 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48935.log 2024-01-11T07:19:25.735 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58118.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58118.log.gz 2024-01-11T07:19:25.736 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45916.log 2024-01-11T07:19:25.736 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48935.log.gz 2024-01-11T07:19:25.736 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40203.log 2024-01-11T07:19:25.737 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45916.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45916.log.gz 2024-01-11T07:19:25.737 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69384.log 2024-01-11T07:19:25.738 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40203.log.gz 2024-01-11T07:19:25.738 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40295.log 2024-01-11T07:19:25.738 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69384.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69384.log.gz 2024-01-11T07:19:25.739 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61532.log 2024-01-11T07:19:25.739 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40295.log.gz 2024-01-11T07:19:25.740 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66172.log 2024-01-11T07:19:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61532.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61532.log.gz 2024-01-11T07:19:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63807.log 2024-01-11T07:19:25.741 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66172.log.gz 2024-01-11T07:19:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52108.log 2024-01-11T07:19:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63807.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63807.log.gz 2024-01-11T07:19:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59872.log 2024-01-11T07:19:25.742 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52108.log.gz 2024-01-11T07:19:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47914.log 2024-01-11T07:19:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59872.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59872.log.gz 2024-01-11T07:19:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49711.log 2024-01-11T07:19:25.744 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47914.log.gz 2024-01-11T07:19:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72985.log 2024-01-11T07:19:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49711.log: /var/log/ceph/ceph-client.admin.72985.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49711.log.gz 2024-01-11T07:19:25.745 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72985.log.gz 2024-01-11T07:19:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46164.log 2024-01-11T07:19:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62982.log 2024-01-11T07:19:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46164.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46164.log.gz 2024-01-11T07:19:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39063.log 2024-01-11T07:19:25.748 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62982.log.gz 2024-01-11T07:19:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57640.log 2024-01-11T07:19:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39063.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39063.log.gz 2024-01-11T07:19:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67105.log 2024-01-11T07:19:25.749 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57640.log.gz 2024-01-11T07:19:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-01-11T07:19:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67105.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67105.log.gz 2024-01-11T07:19:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55714.log 2024-01-11T07:19:25.750 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gz 2024-01-11T07:19:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38455.log 2024-01-11T07:19:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55714.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55714.log.gz 2024-01-11T07:19:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71631.log 2024-01-11T07:19:25.752 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38455.log.gz 2024-01-11T07:19:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52634.log 2024-01-11T07:19:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71631.log: /var/log/ceph/ceph-client.admin.52634.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71631.log.gz 2024-01-11T07:19:25.754 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52634.log.gz 2024-01-11T07:19:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55347.log 2024-01-11T07:19:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60674.log 2024-01-11T07:19:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55347.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55347.log.gz 2024-01-11T07:19:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61300.log 2024-01-11T07:19:25.755 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60674.log.gz 2024-01-11T07:19:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56181.log 2024-01-11T07:19:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61300.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61300.log.gz 2024-01-11T07:19:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39074.log 2024-01-11T07:19:25.757 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56181.log.gz 2024-01-11T07:19:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52400.log 2024-01-11T07:19:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39074.log: /var/log/ceph/ceph-client.admin.52400.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39074.log.gz 2024-01-11T07:19:25.758 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52400.log.gz 2024-01-11T07:19:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.34887.log 2024-01-11T07:19:25.759 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.34887.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.34887.log.gz 2024-01-11T07:19:25.759 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60535.log 2024-01-11T07:19:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60535.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60535.log.gz 2024-01-11T07:19:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69858.log 2024-01-11T07:19:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69858.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69858.log.gz 2024-01-11T07:19:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44035.log 2024-01-11T07:19:25.762 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44035.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44035.log.gz 2024-01-11T07:19:25.762 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39881.log 2024-01-11T07:19:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39881.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39881.log.gz 2024-01-11T07:19:25.763 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41707.log 2024-01-11T07:19:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41707.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41707.log.gz 2024-01-11T07:19:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72341.log 2024-01-11T07:19:25.765 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72341.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72341.log.gz 2024-01-11T07:19:25.765 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63350.log 2024-01-11T07:19:25.766 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63350.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63350.log.gz 2024-01-11T07:19:25.766 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39075.log 2024-01-11T07:19:25.767 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39075.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39075.log.gz 2024-01-11T07:19:25.767 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64478.log 2024-01-11T07:19:25.768 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64478.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64478.log.gz 2024-01-11T07:19:25.768 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51283.log 2024-01-11T07:19:25.769 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51283.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51283.log.gz 2024-01-11T07:19:25.769 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40111.log 2024-01-11T07:19:25.770 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40111.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40111.log.gz 2024-01-11T07:19:25.770 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52680.log 2024-01-11T07:19:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52680.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52680.log.gz 2024-01-11T07:19:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63645.log 2024-01-11T07:19:25.772 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63645.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63645.log.gz 2024-01-11T07:19:25.772 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68453.log 2024-01-11T07:19:25.773 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68453.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68453.log.gz 2024-01-11T07:19:25.773 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71880.log 2024-01-11T07:19:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71880.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71880.log.gz 2024-01-11T07:19:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42289.log 2024-01-11T07:19:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52354.log 2024-01-11T07:19:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42289.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42289.log.gz 2024-01-11T07:19:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68840.log 2024-01-11T07:19:25.783 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52354.log.gz 2024-01-11T07:19:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43701.log 2024-01-11T07:19:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68840.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68840.log.gz 2024-01-11T07:19:25.785 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51651.log 2024-01-11T07:19:25.785 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43701.log.gz 2024-01-11T07:19:25.785 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53775.log 2024-01-11T07:19:25.786 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51651.log.gz 2024-01-11T07:19:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46995.log 2024-01-11T07:19:25.786 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53775.log.gz 2024-01-11T07:19:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71014.log 2024-01-11T07:19:25.787 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46995.log: /var/log/ceph/ceph-client.admin.71014.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2024-01-11T07:19:25.787 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71014.log.gz 2024-01-11T07:19:25.788 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37995.log 2024-01-11T07:19:25.788 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37194.log 2024-01-11T07:19:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37995.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37995.log.gz 2024-01-11T07:19:25.789 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67059.log 2024-01-11T07:19:25.789 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37194.log.gz 2024-01-11T07:19:25.790 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69292.log 2024-01-11T07:19:25.790 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67059.log.gz 2024-01-11T07:19:25.790 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71217.log 2024-01-11T07:19:25.791 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69292.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69292.log.gz 2024-01-11T07:19:25.791 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61152.log 2024-01-11T07:19:25.792 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47141.log 2024-01-11T07:19:25.792 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71217.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71217.log.gz 2024-01-11T07:19:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47141.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47141.log.gz 2024-01-11T07:19:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40157.log 2024-01-11T07:19:25.794 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61152.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61152.log.gz 2024-01-11T07:19:25.794 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64284.log 2024-01-11T07:19:25.794 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40157.log.gz 2024-01-11T07:19:25.794 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46806.log 2024-01-11T07:19:25.795 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64284.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64284.log.gz 2024-01-11T07:19:25.795 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66434.log 2024-01-11T07:19:25.795 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46806.log.gz 2024-01-11T07:19:25.796 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43935.log 2024-01-11T07:19:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66434.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66434.log.gz 2024-01-11T07:19:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56864.log 2024-01-11T07:19:25.797 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43935.log.gz 2024-01-11T07:19:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71355.log 2024-01-11T07:19:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56864.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56864.log.gz 2024-01-11T07:19:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41991.log 2024-01-11T07:19:25.799 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71355.log.gz 2024-01-11T07:19:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44754.log 2024-01-11T07:19:25.799 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41991.log.gz 2024-01-11T07:19:25.800 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.44754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44754.log.gz 2024-01-11T07:19:25.800 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56469.log 2024-01-11T07:19:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39651.log 2024-01-11T07:19:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56469.log: 88.0%/var/log/ceph/ceph-client.admin.39651.log: -- replaced with /var/log/ceph/ceph-client.admin.56469.log.gz 2024-01-11T07:19:25.802 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39651.log.gz 2024-01-11T07:19:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71263.log 2024-01-11T07:19:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71263.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71263.log.gz 2024-01-11T07:19:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70353.log 2024-01-11T07:19:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63553.log 2024-01-11T07:19:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63553.log: /var/log/ceph/ceph-client.admin.70353.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63553.log.gz 2024-01-11T07:19:25.804 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70353.log.gz 2024-01-11T07:19:25.805 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38707.log 2024-01-11T07:19:25.805 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49435.log 2024-01-11T07:19:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38707.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38707.log.gz 2024-01-11T07:19:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49435.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49435.log.gz 2024-01-11T07:19:25.806 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43839.log 2024-01-11T07:19:25.807 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43839.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43839.log.gz 2024-01-11T07:19:25.807 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62374.log 2024-01-11T07:19:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62374.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62374.log.gz 2024-01-11T07:19:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65601.log 2024-01-11T07:19:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65601.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65601.log.gz 2024-01-11T07:19:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65107.log 2024-01-11T07:19:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.65107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65107.log.gz 2024-01-11T07:19:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64028.log 2024-01-11T07:19:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64028.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64028.log.gz 2024-01-11T07:19:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60920.log 2024-01-11T07:19:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60920.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60920.log.gz 2024-01-11T07:19:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65203.log 2024-01-11T07:19:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65203.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65203.log.gz 2024-01-11T07:19:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44808.log 2024-01-11T07:19:25.814 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44808.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44808.log.gz 2024-01-11T07:19:25.814 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68269.log 2024-01-11T07:19:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68269.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68269.log.gz 2024-01-11T07:19:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70968.log 2024-01-11T07:19:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70968.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70968.log.gz 2024-01-11T07:19:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56047.log 2024-01-11T07:19:25.817 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56047.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56047.log.gz 2024-01-11T07:19:25.817 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55644.log 2024-01-11T07:19:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55644.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55644.log.gz 2024-01-11T07:19:25.818 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55207.log 2024-01-11T07:19:25.819 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55207.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55207.log.gz 2024-01-11T07:19:25.819 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67243.log 2024-01-11T07:19:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67243.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67243.log.gz 2024-01-11T07:19:25.820 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66815.log 2024-01-11T07:19:25.820 INFO:teuthology.orchestra.run.smithi007.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-11T07:19:25.821 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66815.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66815.log.gz 2024-01-11T07:19:25.821 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61104.log 2024-01-11T07:19:25.822 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54465.log 2024-01-11T07:19:25.822 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61104.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61104.log.gz 2024-01-11T07:19:25.822 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54465.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54465.log.gz 2024-01-11T07:19:25.823 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50409.log 2024-01-11T07:19:25.823 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2024-01-11T07:19:25.823 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50409.log: /var/log/ceph/ceph-client.admin.48889.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50409.log.gz 2024-01-11T07:19:25.824 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2024-01-11T07:19:25.824 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62440.log 2024-01-11T07:19:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58985.log 2024-01-11T07:19:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62440.log: /var/log/ceph/ceph-client.0.58985.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62440.log.gz 2024-01-11T07:19:25.825 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58985.log.gz 2024-01-11T07:19:25.825 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67011.log 2024-01-11T07:19:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60744.log 2024-01-11T07:19:25.826 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67011.log: /var/log/ceph/ceph-client.admin.60744.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67011.log.gz 2024-01-11T07:19:25.827 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60744.log.gz 2024-01-11T07:19:25.827 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46211.log 2024-01-11T07:19:25.828 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36942.log 2024-01-11T07:19:25.828 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46211.log: /var/log/ceph/ceph-client.admin.36942.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46211.log.gz 2024-01-11T07:19:25.828 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36942.log.gz 2024-01-11T07:19:25.828 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39069.log 2024-01-11T07:19:25.829 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72755.log 2024-01-11T07:19:25.829 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39069.log: /var/log/ceph/ceph-client.admin.72755.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39069.log.gz 2024-01-11T07:19:25.829 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72755.log.gz 2024-01-11T07:19:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56910.log 2024-01-11T07:19:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66631.log 2024-01-11T07:19:25.830 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56910.log: /var/log/ceph/ceph-client.admin.66631.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56910.log.gz 2024-01-11T07:19:25.831 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66631.log.gz 2024-01-11T07:19:25.831 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37269.log 2024-01-11T07:19:25.832 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43655.log 2024-01-11T07:19:25.832 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37269.log: /var/log/ceph/ceph-client.admin.43655.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37269.log.gz 2024-01-11T07:19:25.832 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43655.log.gz 2024-01-11T07:19:25.832 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54651.log 2024-01-11T07:19:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42381.log 2024-01-11T07:19:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54651.log: /var/log/ceph/ceph-client.admin.42381.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54651.log.gz 2024-01-11T07:19:25.834 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42381.log.gz 2024-01-11T07:19:25.834 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38317.log 2024-01-11T07:19:25.834 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62734.log 2024-01-11T07:19:25.835 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38317.log: /var/log/ceph/ceph-client.admin.62734.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38317.log.gz 2024-01-11T07:19:25.835 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62734.log.gz 2024-01-11T07:19:25.835 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46535.log 2024-01-11T07:19:25.836 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41891.log 2024-01-11T07:19:25.836 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46535.log: /var/log/ceph/ceph-client.admin.41891.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46535.log.gz 2024-01-11T07:19:25.837 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41891.log.gz 2024-01-11T07:19:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68545.log 2024-01-11T07:19:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37742.log 2024-01-11T07:19:25.838 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68545.log: /var/log/ceph/ceph-client.admin.37742.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68545.log.gz 2024-01-11T07:19:25.838 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37742.log.gz 2024-01-11T07:19:25.838 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66310.log 2024-01-11T07:19:25.839 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45732.log 2024-01-11T07:19:25.839 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66310.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66310.log.gz 2024-01-11T07:19:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45732.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45732.log.gz 2024-01-11T07:19:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39789.log 2024-01-11T07:19:25.841 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47818.log 2024-01-11T07:19:25.841 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39789.log: /var/log/ceph/ceph-client.0.47818.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39789.log.gz 2024-01-11T07:19:25.841 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47818.log.gz 2024-01-11T07:19:25.841 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72203.log 2024-01-11T07:19:25.842 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67335.log 2024-01-11T07:19:25.842 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72203.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72203.log.gz 2024-01-11T07:19:25.844 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67335.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67335.log.gz 2024-01-11T07:19:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39076.log 2024-01-11T07:19:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63507.log 2024-01-11T07:19:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-01-11T07:19:25.845 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39076.log.gz 2024-01-11T07:19:25.845 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63507.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63507.log.gz 2024-01-11T07:19:25.846 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47327.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-01-11T07:19:25.846 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65739.log 2024-01-11T07:19:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52956.log 2024-01-11T07:19:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65739.log: /var/log/ceph/ceph-client.admin.52956.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65739.log.gz 2024-01-11T07:19:25.847 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52956.log.gz 2024-01-11T07:19:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38699.log 2024-01-11T07:19:25.849 INFO:teuthology.orchestra.run.smithi113.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-11T07:19:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38702.log 2024-01-11T07:19:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38699.log: /var/log/ceph/ceph-client.admin.38702.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38699.log.gz 2024-01-11T07:19:25.849 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38702.log.gz 2024-01-11T07:19:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61578.log 2024-01-11T07:19:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53161.log 2024-01-11T07:19:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61578.log: /var/log/ceph/ceph-client.admin.53161.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61578.log.gz 2024-01-11T07:19:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69134.log 2024-01-11T07:19:25.850 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53161.log.gz 2024-01-11T07:19:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68177.log 2024-01-11T07:19:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69134.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69134.log.gz 2024-01-11T07:19:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69674.log 2024-01-11T07:19:25.852 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68177.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68177.log.gz 2024-01-11T07:19:25.853 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69674.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69674.log.gz 2024-01-11T07:19:25.853 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60966.log 2024-01-11T07:19:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63304.log 2024-01-11T07:19:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60966.log: /var/log/ceph/ceph-client.admin.63304.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60966.log.gz 2024-01-11T07:19:25.854 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63304.log.gz 2024-01-11T07:19:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71401.log 2024-01-11T07:19:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67784.log 2024-01-11T07:19:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71401.log: /var/log/ceph/ceph-client.admin.67784.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71401.log.gz 2024-01-11T07:19:25.856 INFO:teuthology.orchestra.run.smithi007.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.67784.log.gz 2024-01-11T07:19:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46626.log 2024-01-11T07:19:25.857 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56977.log 2024-01-11T07:19:25.857 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46626.log: /var/log/ceph/ceph-client.admin.56977.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46626.log.gz 2024-01-11T07:19:25.857 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56977.log.gz 2024-01-11T07:19:25.857 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61346.log 2024-01-11T07:19:25.858 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72939.log 2024-01-11T07:19:25.858 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61346.log: /var/log/ceph/ceph-client.admin.72939.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61346.log.gz 2024-01-11T07:19:25.858 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72939.log.gz 2024-01-11T07:19:25.858 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60581.log 2024-01-11T07:19:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38409.log 2024-01-11T07:19:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60581.log: /var/log/ceph/ceph-client.admin.38409.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60581.log.gz 2024-01-11T07:19:25.860 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38409.log.gz 2024-01-11T07:19:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2024-01-11T07:19:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58590.log 2024-01-11T07:19:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48365.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2024-01-11T07:19:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58590.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58590.log.gz 2024-01-11T07:19:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70579.log 2024-01-11T07:19:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47095.log 2024-01-11T07:19:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70579.log: /var/log/ceph/ceph-client.admin.47095.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70579.log.gz 2024-01-11T07:19:25.863 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47095.log.gz 2024-01-11T07:19:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54419.log 2024-01-11T07:19:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64236.log 2024-01-11T07:19:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54419.log: /var/log/ceph/ceph-client.admin.64236.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54419.log.gz 2024-01-11T07:19:25.864 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64236.log.gz 2024-01-11T07:19:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53115.log 2024-01-11T07:19:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42451.log 2024-01-11T07:19:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53115.log: /var/log/ceph/ceph-client.admin.42451.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53115.log.gz 2024-01-11T07:19:25.866 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42451.log.gz 2024-01-11T07:19:25.866 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44081.log 2024-01-11T07:19:25.866 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68886.log 2024-01-11T07:19:25.867 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44081.log: /var/log/ceph/ceph-client.admin.68886.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44081.log.gz 2024-01-11T07:19:25.867 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68886.log.gz 2024-01-11T07:19:25.867 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58314.log 2024-01-11T07:19:25.868 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62073.log 2024-01-11T07:19:25.868 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58314.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58314.log.gz 2024-01-11T07:19:25.868 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62073.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62073.log.gz 2024-01-11T07:19:25.869 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73031.log 2024-01-11T07:19:25.869 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59155.log 2024-01-11T07:19:25.869 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73031.log: /var/log/ceph/ceph-client.admin.59155.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73031.log.gz 2024-01-11T07:19:25.870 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59155.log.gz 2024-01-11T07:19:25.870 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37463.log 2024-01-11T07:19:25.871 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72111.log 2024-01-11T07:19:25.871 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37463.log: /var/log/ceph/ceph-client.admin.72111.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37463.log.gz 2024-01-11T07:19:25.871 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72111.log.gz 2024-01-11T07:19:25.871 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50271.log 2024-01-11T07:19:25.873 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42083.log 2024-01-11T07:19:25.873 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50271.log: /var/log/ceph/ceph-client.admin.42083.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50271.log.gz 2024-01-11T07:19:25.873 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42083.log.gz 2024-01-11T07:19:25.873 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52446.log 2024-01-11T07:19:25.874 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42605.log 2024-01-11T07:19:25.874 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52446.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52446.log.gz 2024-01-11T07:19:25.874 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42605.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42605.log.gz 2024-01-11T07:19:25.874 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58682.log 2024-01-11T07:19:25.875 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52864.log 2024-01-11T07:19:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58682.log: /var/log/ceph/ceph-client.admin.52864.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.58682.log.gz 2024-01-11T07:19:25.876 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52864.log.gz 2024-01-11T07:19:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62023.log 2024-01-11T07:19:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46465.log 2024-01-11T07:19:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.62023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62023.log.gz 2024-01-11T07:19:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46465.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46465.log.gz 2024-01-11T07:19:25.877 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37882.log 2024-01-11T07:19:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55947.log 2024-01-11T07:19:25.878 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37882.log: /var/log/ceph/ceph-client.0.55947.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37882.log.gz 2024-01-11T07:19:25.878 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55947.log.gz 2024-01-11T07:19:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54373.log 2024-01-11T07:19:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66264.log 2024-01-11T07:19:25.879 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54373.log: /var/log/ceph/ceph-client.admin.66264.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54373.log.gz 2024-01-11T07:19:25.880 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66264.log.gz 2024-01-11T07:19:25.880 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48503.log 2024-01-11T07:19:25.881 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54825.log 2024-01-11T07:19:25.881 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48503.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48503.log.gz 2024-01-11T07:19:25.881 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54825.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54825.log.gz 2024-01-11T07:19:25.881 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-11T07:19:25.885 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72479.log 2024-01-11T07:19:25.885 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.72479.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72479.log.gz 2024-01-11T07:19:25.885 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60286.log 2024-01-11T07:19:25.886 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60286.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60286.log.gz 2024-01-11T07:19:25.886 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50501.log 2024-01-11T07:19:25.886 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50501.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50501.log.gz 2024-01-11T07:19:25.886 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68131.log 2024-01-11T07:19:25.887 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68131.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68131.log.gz 2024-01-11T07:19:25.887 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55011.log 2024-01-11T07:19:25.888 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55011.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55011.log.gz 2024-01-11T07:19:25.888 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40495.log 2024-01-11T07:19:25.890 INFO:teuthology.orchestra.run.smithi007.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-11T07:19:25.890 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40495.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40495.log.gz 2024-01-11T07:19:25.890 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68591.log 2024-01-11T07:19:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63028.log 2024-01-11T07:19:25.892 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39973.log 2024-01-11T07:19:25.892 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68591.log.gz 2024-01-11T07:19:25.892 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63028.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63028.log.gz 2024-01-11T07:19:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39973.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39973.log.gz 2024-01-11T07:19:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37378.log 2024-01-11T07:19:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39448.log 2024-01-11T07:19:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37378.log: /var/log/ceph/ceph-client.admin.39448.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37378.log.gz 2024-01-11T07:19:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37928.log 2024-01-11T07:19:25.894 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39448.log.gz 2024-01-11T07:19:25.895 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71926.log 2024-01-11T07:19:25.896 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37928.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37928.log.gz 2024-01-11T07:19:25.896 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67497.log 2024-01-11T07:19:25.896 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71926.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71926.log.gz 2024-01-11T07:19:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65969.log 2024-01-11T07:19:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67497.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67497.log.gz 2024-01-11T07:19:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43293.log 2024-01-11T07:19:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65969.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65969.log.gz 2024-01-11T07:19:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61670.log 2024-01-11T07:19:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43293.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43293.log.gz 2024-01-11T07:19:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49551.log 2024-01-11T07:19:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61670.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61670.log.gz 2024-01-11T07:19:25.901 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42835.log 2024-01-11T07:19:25.901 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49551.log: /var/log/ceph/ceph-client.admin.42835.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49551.log.gz 2024-01-11T07:19:25.901 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42835.log.gz 2024-01-11T07:19:25.901 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71447.log 2024-01-11T07:19:25.902 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48981.log 2024-01-11T07:19:25.902 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71447.log: /var/log/ceph/ceph-client.admin.48981.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71447.log.gz 2024-01-11T07:19:25.902 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48981.log.gz 2024-01-11T07:19:25.902 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38041.log 2024-01-11T07:19:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44924.log 2024-01-11T07:19:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38041.log: /var/log/ceph/ceph-client.admin.44924.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38041.log.gz 2024-01-11T07:19:25.904 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44924.log.gz 2024-01-11T07:19:25.904 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69200.log 2024-01-11T07:19:25.904 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57710.log 2024-01-11T07:19:25.904 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69200.log: /var/log/ceph/ceph-client.admin.57710.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69200.log.gz 2024-01-11T07:19:25.905 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57710.log.gz 2024-01-11T07:19:25.905 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58266.log 2024-01-11T07:19:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65785.log 2024-01-11T07:19:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58266.log: /var/log/ceph/ceph-client.admin.65785.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58266.log.gz 2024-01-11T07:19:25.907 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65785.log.gz 2024-01-11T07:19:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54051.log 2024-01-11T07:19:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46853.log 2024-01-11T07:19:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54051.log: /var/log/ceph/ceph-client.admin.46853.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54051.log.gz 2024-01-11T07:19:25.908 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46853.log.gz 2024-01-11T07:19:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46947.log 2024-01-11T07:19:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49027.log 2024-01-11T07:19:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46947.log: /var/log/ceph/ceph-client.admin.49027.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46947.log.gz 2024-01-11T07:19:25.909 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49027.log.gz 2024-01-11T07:19:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39071.log 2024-01-11T07:19:25.910 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61394.log 2024-01-11T07:19:25.910 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39071.log: /var/log/ceph/ceph-client.admin.61394.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39071.log.gz 2024-01-11T07:19:25.911 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57886.log 2024-01-11T07:19:25.911 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61394.log.gz 2024-01-11T07:19:25.912 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62486.log 2024-01-11T07:19:25.912 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57886.log: /var/log/ceph/ceph-client.admin.62486.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57886.log.gz 2024-01-11T07:19:25.912 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62486.log.gz 2024-01-11T07:19:25.912 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72433.log 2024-01-11T07:19:25.913 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67900.log 2024-01-11T07:19:25.913 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72433.log: /var/log/ceph/ceph-client.admin.67900.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72433.log.gz 2024-01-11T07:19:25.913 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67900.log.gz 2024-01-11T07:19:25.913 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64708.log 2024-01-11T07:19:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62019.log 2024-01-11T07:19:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64708.log: /var/log/ceph/ceph-client.0.62019.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64708.log.gz 2024-01-11T07:19:25.914 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62019.log.gz 2024-01-11T07:19:25.914 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38701.log 2024-01-11T07:19:25.915 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43379.log 2024-01-11T07:19:25.916 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38701.log: /var/log/ceph/ceph-client.admin.43379.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38701.log.gz 2024-01-11T07:19:25.916 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43379.log.gz 2024-01-11T07:19:25.916 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44359.log 2024-01-11T07:19:25.917 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54281.log 2024-01-11T07:19:25.917 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44359.log: /var/log/ceph/ceph-client.admin.54281.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44359.log.gz 2024-01-11T07:19:25.917 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54281.log.gz 2024-01-11T07:19:25.917 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66364.log 2024-01-11T07:19:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44854.log 2024-01-11T07:19:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66364.log: /var/log/ceph/ceph-client.admin.44854.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66364.log.gz 2024-01-11T07:19:25.918 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44854.log.gz 2024-01-11T07:19:25.918 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68655.log 2024-01-11T07:19:25.919 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53002.log 2024-01-11T07:19:25.920 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68655.log: /var/log/ceph/ceph-client.admin.53002.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68655.log.gz 2024-01-11T07:19:25.920 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53002.log.gz 2024-01-11T07:19:25.920 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59918.log 2024-01-11T07:19:25.921 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48235.log 2024-01-11T07:19:25.921 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59918.log: /var/log/ceph/ceph-client.admin.48235.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59918.log.gz 2024-01-11T07:19:25.921 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48235.log.gz 2024-01-11T07:19:25.921 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69720.log 2024-01-11T07:19:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44313.log 2024-01-11T07:19:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69720.log: /var/log/ceph/ceph-client.admin.44313.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69720.log.gz 2024-01-11T07:19:25.922 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44313.log.gz 2024-01-11T07:19:25.922 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66723.log 2024-01-11T07:19:25.924 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66015.log 2024-01-11T07:19:25.924 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66723.log: /var/log/ceph/ceph-client.admin.66015.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66723.log.gz 2024-01-11T07:19:25.924 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66015.log.gz 2024-01-11T07:19:25.924 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63599.log 2024-01-11T07:19:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69766.log 2024-01-11T07:19:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63599.log: /var/log/ceph/ceph-client.admin.69766.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63599.log.gz 2024-01-11T07:19:25.925 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69766.log.gz 2024-01-11T07:19:25.925 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54327.log 2024-01-11T07:19:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46100.log 2024-01-11T07:19:25.926 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54327.log: /var/log/ceph/ceph-client.admin.46100.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54327.log.gz 2024-01-11T07:19:25.926 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46100.log.gz 2024-01-11T07:19:25.927 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50663.log 2024-01-11T07:19:25.928 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66677.log 2024-01-11T07:19:25.928 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50663.log: /var/log/ceph/ceph-client.admin.66677.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50663.log.gz 2024-01-11T07:19:25.928 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66677.log.gz 2024-01-11T07:19:25.928 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57548.log 2024-01-11T07:19:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49795.log 2024-01-11T07:19:25.929 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48077.log 2024-01-11T07:19:25.929 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57548.log.gz 2024-01-11T07:19:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49795.log: /var/log/ceph/ceph-client.admin.48077.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49795.log.gz 2024-01-11T07:19:25.930 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48077.log.gz 2024-01-11T07:19:25.930 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57161.log 2024-01-11T07:19:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53959.log 2024-01-11T07:19:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57161.log: /var/log/ceph/ceph-client.admin.53959.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57161.log.gz 2024-01-11T07:19:25.931 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53959.log.gz 2024-01-11T07:19:25.931 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56653.log 2024-01-11T07:19:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63460.log 2024-01-11T07:19:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56653.log: /var/log/ceph/ceph-client.admin.63460.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56653.log.gz 2024-01-11T07:19:25.932 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63460.log.gz 2024-01-11T07:19:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55506.log 2024-01-11T07:19:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71309.log 2024-01-11T07:19:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55506.log: /var/log/ceph/ceph-client.admin.71309.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55506.log.gz 2024-01-11T07:19:25.933 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71309.log.gz 2024-01-11T07:19:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55161.log 2024-01-11T07:19:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66524.log 2024-01-11T07:19:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55161.log: /var/log/ceph/ceph-client.admin.66524.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55161.log.gz 2024-01-11T07:19:25.935 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66524.log.gz 2024-01-11T07:19:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-01-11T07:19:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42927.log 2024-01-11T07:19:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47189.log: /var/log/ceph/ceph-client.admin.42927.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-01-11T07:19:25.936 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42927.log.gz 2024-01-11T07:19:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38271.log 2024-01-11T07:19:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39835.log 2024-01-11T07:19:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38271.log: /var/log/ceph/ceph-client.admin.39835.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38271.log.gz 2024-01-11T07:19:25.938 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39835.log.gz 2024-01-11T07:19:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56561.log 2024-01-11T07:19:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69904.log 2024-01-11T07:19:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56561.log: /var/log/ceph/ceph-client.admin.69904.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56561.log.gz 2024-01-11T07:19:25.939 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69904.log.gz 2024-01-11T07:19:25.954 INFO:teuthology.orchestra.run.smithi113.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-11T07:19:26.138 INFO:teuthology.orchestra.run.smithi113.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-11T07:19:26.287 INFO:teuthology.orchestra.run.smithi113.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-11T07:19:26.290 INFO:teuthology.orchestra.run.smithi113.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-11T07:19:26.328 INFO:teuthology.orchestra.run.smithi113.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-11T07:19:26.329 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-01-11T07:19:26.330 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m0.867s 2024-01-11T07:19:26.330 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m4.222s 2024-01-11T07:19:26.330 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m0.154s 2024-01-11T07:19:26.439 INFO:teuthology.orchestra.run.smithi007.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-11T07:19:26.443 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-11T07:19:26.504 INFO:teuthology.orchestra.run.smithi007.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-11T07:19:26.534 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-11T07:19:26.641 INFO:teuthology.orchestra.run.smithi007.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-11T07:19:26.643 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-01-11T07:19:26.643 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m1.202s 2024-01-11T07:19:26.643 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m5.420s 2024-01-11T07:19:26.643 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.572s 2024-01-11T07:19:26.643 INFO:tasks.ceph:Archiving logs... 2024-01-11T07:19:26.643 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/remote/smithi007/log 2024-01-11T07:19:26.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-11T07:19:27.759 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/remote/smithi113/log 2024-01-11T07:19:27.760 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-11T07:19:28.339 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-11T07:19:28.348 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-11T07:19:28.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-11T07:19:28.351 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-11T07:19:28.454 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-11T07:19:28.466 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-11T07:19:28.466 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T07:19:28.469 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-11T07:19:28.487 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2024-01-11T07:19:28.487 INFO:teuthology.orchestra.run.smithi113.stderr:bash: ntpq: command not found 2024-01-11T07:19:28.492 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T07:19:28.492 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-01-11T07:19:28.492 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 +273us[ +274us] +/- 101ms 2024-01-11T07:19:28.492 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -1073us[-1072us] +/- 64ms 2024-01-11T07:19:28.492 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +1217us[+1218us] +/- 73ms 2024-01-11T07:19:28.493 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:19:28.494 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-11T07:19:28.494 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-01-11T07:19:28.494 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +185us[ +185us] +/- 101ms 2024-01-11T07:19:28.494 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -1157us[-1146us] +/- 64ms 2024-01-11T07:19:28.494 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +1138us[+1138us] +/- 73ms 2024-01-11T07:19:28.494 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-01-11T07:19:28.494 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-11T07:19:28.504 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-11T07:19:28.505 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-11T07:19:28.514 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-11T07:19:28.633 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-11T07:19:28.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T07:19:28.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:19:28.745 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-11T07:19:28.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:19:28.775 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-11T07:19:28.785 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-11T07:19:28.793 INFO:teuthology.task.internal:Duration was 1460.434582 seconds 2024-01-11T07:19:28.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-11T07:19:28.804 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-11T07:19:28.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-11T07:19:28.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-11T07:19:28.863 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T07:19:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-11T07:19:29.102 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-11T07:19:29.102 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-01-11T07:19:29.102 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-11T07:19:29.131 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-01-11T07:19:29.131 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-11T07:19:29.153 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-11T07:19:29.153 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-11T07:19:29.175 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-11T07:19:29.237 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-11T07:19:29.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-11T07:19:29.240 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-11T07:19:29.342 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-11T07:19:29.353 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-11T07:19:29.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-11T07:19:29.384 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-11T07:19:29.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-11T07:19:29.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-11T07:19:29.427 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-11T07:19:29.456 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-01-11T07:19:29.478 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-01-11T07:19:29.492 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-11T07:19:29.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:19:29.535 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-11T07:19:29.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-11T07:19:29.551 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-11T07:19:29.561 INFO:teuthology.task.internal:Transferring archived files... 2024-01-11T07:19:29.562 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/remote/smithi007 2024-01-11T07:19:29.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-11T07:19:29.626 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480/remote/smithi113 2024-01-11T07:19:29.627 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-11T07:19:29.665 INFO:teuthology.task.internal:Removing archive directory... 2024-01-11T07:19:29.665 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-11T07:19:29.668 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-11T07:19:29.715 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-11T07:19:29.727 INFO:teuthology.task.internal:Not uploading archives. 2024-01-11T07:19:29.727 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-11T07:19:29.735 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-11T07:19:29.735 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-11T07:19:29.738 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-11T07:19:29.757 INFO:teuthology.orchestra.run.smithi007.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 11 07:19 /home/ubuntu/cephtest 2024-01-11T07:19:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 11 07:19 /home/ubuntu/cephtest 2024-01-11T07:19:29.773 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-11T07:19:29.783 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-11T07:19:29.803 INFO:teuthology.nuke:Checking targets against current locks 2024-01-11T07:19:29.830 DEBUG:teuthology.nuke:shortname: smithi007 2024-01-11T07:19:29.830 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-11T07:19:29.857 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 06:49:09.807831', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGz211efhD5EafEB9aDd+x9Gp4uo/MSCiUXSAMaCF7LXcrpy+Q65Jc7jyCCWbGzGhr/661Jg0zmzjivmOuURb2A='} 2024-01-11T07:19:29.858 DEBUG:teuthology.nuke:shortname: smithi113 2024-01-11T07:19:29.858 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-11T07:19:29.879 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-10_19:20:36-fs-wip-vshankar-testing1-quincy-2024-01-10-2010-quincy-distro-default-smithi/7512480', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-11 06:49:09.806431', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInZZZqUlAa+c4h9rdzlNKl38WHPrfAS2CrZ06y59npUesOmvftW9ucInWPMoRKsDGYZxqPe4Gc529D0iXBIUx8='} 2024-01-11T07:19:29.911 INFO:teuthology.orchestra.console.smithi007:Power off 2024-01-11T07:19:29.911 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-11T07:19:29.934 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-01-11T07:19:29.934 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T07:19:29.950 INFO:teuthology.orchestra.console.smithi113:Power off 2024-01-11T07:19:29.951 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-11T07:19:29.959 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-11T07:19:29.973 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2024-01-11T07:19:29.973 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T07:19:29.997 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-01-11T07:19:33.961 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T07:19:33.998 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T07:19:34.072 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-11T07:19:34.108 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-01-11T07:19:38.075 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T07:19:38.109 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-11T07:19:38.191 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-01-11T07:19:38.221 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2024-01-11T07:19:38.292 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-01-11T07:19:38.322 INFO:teuthology.orchestra.console.smithi113:Power off completed 2024-01-11T07:19:38.428 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1460.4345815181732 failure_reason: '"1704956806.8902118 osd.4 (osd.4) 3 : cluster [WRN] OSD bench result of 225823.776910 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.4. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-01-11T07:19:38.428 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-11T07:19:38.510 INFO:teuthology.run:FAIL